2023-07-14T17:42:48.759 INFO:root:teuthology version: 0.0.1.dev145+gaa6a80e 2023-07-14T17:42:48.760 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi --archive /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7337992 --verbose --description fs:workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/postgres}} --owner scheduled_pdonnell@teuthology -- /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7337992/orig.config.yaml 2023-07-14T17:42:48.821 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T17:42:48.916 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7337992 branch: wip-pdonnell-testing-20230714.134335 description: fs:workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/postgres}} email: pdonnell@redhat.com first_in_suite: false job_id: '7337992' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-pdonnell-testing-20230714.134335 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 81c9387d2e7576da065747ec9c16d1c7d69016ec subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt install: ceph: flavor: default sha1: 81c9387d2e7576da065747ec9c16d1c7d69016ec kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20230714.134335 sha1: 81c9387d2e7576da065747ec9c16d1c7d69016ec owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5179 sha1: 81c9387d2e7576da065747ec9c16d1c7d69016ec sleep_before_teardown: 0 subset: 285/512 suite: fs:workload suite_branch: wip-pdonnell-testing-20230714.134335 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_81c9387d2e7576da065747ec9c16d1c7d69016ec/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 81c9387d2e7576da065747ec9c16d1c7d69016ec syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi133.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi161.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi171.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 500 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: aa6a80e6cc655fff48452a7eca02070e61d074fc timestamp: 2023-07-14_16:44:47 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-07-14T17:42:48.917 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_81c9387d2e7576da065747ec9c16d1c7d69016ec/qa; will attempt to use it 2023-07-14T17:42:48.918 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_81c9387d2e7576da065747ec9c16d1c7d69016ec/qa/tasks 2023-07-14T17:42:48.918 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-07-14T17:42:48.924 INFO:teuthology.task.internal:Checking packages... 2023-07-14T17:42:48.954 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '81c9387d2e7576da065747ec9c16d1c7d69016ec' 2023-07-14T17:42:48.955 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-07-14T17:42:48.955 INFO:teuthology.packaging:ref: None 2023-07-14T17:42:48.955 INFO:teuthology.packaging:tag: None 2023-07-14T17:42:48.955 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20230714.134335 2023-07-14T17:42:48.956 INFO:teuthology.packaging:sha1: 81c9387d2e7576da065747ec9c16d1c7d69016ec 2023-07-14T17:42:48.956 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-pdonnell-testing-20230714.134335 2023-07-14T17:42:49.198 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4940.g81c9387d 2023-07-14T17:42:49.199 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-07-14T17:42:49.205 INFO:teuthology.task.internal:no buildpackages task found 2023-07-14T17:42:49.205 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-07-14T17:42:49.211 INFO:teuthology.task.internal:Saving configuration 2023-07-14T17:42:49.234 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-07-14T17:42:49.241 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T17:42:49.270 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7337992', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 17:37:21.090790', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-14T17:42:49.294 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7337992', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 17:37:21.099246', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-14T17:42:49.319 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi171.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7337992', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 17:37:21.101371', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-14T17:42:49.319 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-07-14T17:42:49.326 INFO:teuthology.task.internal:roles: ubuntu@smithi133.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-07-14T17:42:49.326 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-07-14T17:42:49.326 INFO:teuthology.task.internal:roles: ubuntu@smithi171.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-07-14T17:42:49.326 INFO:teuthology.run_tasks:Running task console_log... 2023-07-14T17:42:49.438 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3844466430>, signals=[15]) 2023-07-14T17:42:49.439 INFO:teuthology.run_tasks:Running task internal.connect... 2023-07-14T17:42:49.445 INFO:teuthology.task.internal:Opening connections... 2023-07-14T17:42:49.445 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi133.front.sepia.ceph.com 2023-07-14T17:42:49.447 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T17:42:49.515 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi161.front.sepia.ceph.com 2023-07-14T17:42:49.516 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T17:43:04.595 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi171.front.sepia.ceph.com 2023-07-14T17:43:04.596 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T17:43:09.668 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-07-14T17:43:09.675 DEBUG:teuthology.orchestra.run.smithi133:> uname -m 2023-07-14T17:43:09.696 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2023-07-14T17:43:09.697 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release 2023-07-14T17:43:09.756 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="Red Hat Enterprise Linux" 2023-07-14T17:43:09.757 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="8.6 (Ootpa)" 2023-07-14T17:43:09.757 INFO:teuthology.orchestra.run.smithi133.stdout:ID="rhel" 2023-07-14T17:43:09.757 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE="fedora" 2023-07-14T17:43:09.757 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="8.6" 2023-07-14T17:43:09.757 INFO:teuthology.orchestra.run.smithi133.stdout:PLATFORM_ID="platform:el8" 2023-07-14T17:43:09.758 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-14T17:43:09.758 INFO:teuthology.orchestra.run.smithi133.stdout:ANSI_COLOR="0;31" 2023-07-14T17:43:09.758 INFO:teuthology.orchestra.run.smithi133.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-14T17:43:09.758 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://www.redhat.com/" 2023-07-14T17:43:09.758 INFO:teuthology.orchestra.run.smithi133.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-14T17:43:09.758 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T17:43:09.758 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T17:43:09.758 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-14T17:43:09.759 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-14T17:43:09.759 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-14T17:43:09.759 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-14T17:43:09.760 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2023-07-14T17:43:12.546 DEBUG:teuthology.orchestra.run.smithi161:> uname -m 2023-07-14T17:43:12.568 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2023-07-14T17:43:12.568 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2023-07-14T17:43:12.627 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="Red Hat Enterprise Linux" 2023-07-14T17:43:12.627 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="8.6 (Ootpa)" 2023-07-14T17:43:12.627 INFO:teuthology.orchestra.run.smithi161.stdout:ID="rhel" 2023-07-14T17:43:12.627 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="fedora" 2023-07-14T17:43:12.627 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="8.6" 2023-07-14T17:43:12.627 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el8" 2023-07-14T17:43:12.628 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-14T17:43:12.628 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31" 2023-07-14T17:43:12.628 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-14T17:43:12.628 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://www.redhat.com/" 2023-07-14T17:43:12.628 INFO:teuthology.orchestra.run.smithi161.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-14T17:43:12.628 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T17:43:12.629 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T17:43:12.629 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-14T17:43:12.629 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-14T17:43:12.629 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-14T17:43:12.629 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-14T17:43:12.629 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2023-07-14T17:43:15.520 DEBUG:teuthology.orchestra.run.smithi171:> uname -m 2023-07-14T17:43:15.542 INFO:teuthology.orchestra.run.smithi171.stdout:x86_64 2023-07-14T17:43:15.542 DEBUG:teuthology.orchestra.run.smithi171:> cat /etc/os-release 2023-07-14T17:43:15.600 INFO:teuthology.orchestra.run.smithi171.stdout:NAME="Red Hat Enterprise Linux" 2023-07-14T17:43:15.601 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION="8.6 (Ootpa)" 2023-07-14T17:43:15.601 INFO:teuthology.orchestra.run.smithi171.stdout:ID="rhel" 2023-07-14T17:43:15.601 INFO:teuthology.orchestra.run.smithi171.stdout:ID_LIKE="fedora" 2023-07-14T17:43:15.601 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION_ID="8.6" 2023-07-14T17:43:15.601 INFO:teuthology.orchestra.run.smithi171.stdout:PLATFORM_ID="platform:el8" 2023-07-14T17:43:15.601 INFO:teuthology.orchestra.run.smithi171.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-14T17:43:15.601 INFO:teuthology.orchestra.run.smithi171.stdout:ANSI_COLOR="0;31" 2023-07-14T17:43:15.602 INFO:teuthology.orchestra.run.smithi171.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-14T17:43:15.602 INFO:teuthology.orchestra.run.smithi171.stdout:HOME_URL="https://www.redhat.com/" 2023-07-14T17:43:15.602 INFO:teuthology.orchestra.run.smithi171.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-14T17:43:15.602 INFO:teuthology.orchestra.run.smithi171.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T17:43:15.602 INFO:teuthology.orchestra.run.smithi171.stdout: 2023-07-14T17:43:15.602 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-14T17:43:15.602 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-14T17:43:15.602 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-14T17:43:15.603 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-14T17:43:15.603 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server 2023-07-14T17:43:15.629 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-07-14T17:43:15.638 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-07-14T17:43:15.656 INFO:teuthology.task.internal:Checking for old test directory... 2023-07-14T17:43:15.657 DEBUG:teuthology.orchestra.run.smithi133:> test '!' -e /home/ubuntu/cephtest 2023-07-14T17:43:15.659 DEBUG:teuthology.orchestra.run.smithi161:> test '!' -e /home/ubuntu/cephtest 2023-07-14T17:43:15.662 DEBUG:teuthology.orchestra.run.smithi171:> test '!' -e /home/ubuntu/cephtest 2023-07-14T17:43:15.680 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-07-14T17:43:15.687 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-07-14T17:43:15.688 DEBUG:teuthology.orchestra.run.smithi133:> test -z $(ls -A /var/lib/ceph) 2023-07-14T17:43:15.718 DEBUG:teuthology.orchestra.run.smithi161:> test -z $(ls -A /var/lib/ceph) 2023-07-14T17:43:15.720 DEBUG:teuthology.orchestra.run.smithi171:> test -z $(ls -A /var/lib/ceph) 2023-07-14T17:43:15.752 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-07-14T17:43:15.828 INFO:teuthology.run_tasks:Running task kernel... 2023-07-14T17:43:15.843 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-07-14T17:43:15.844 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-07-14T17:43:15.844 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-07-14T17:43:15.844 DEBUG:teuthology.orchestra.run.smithi133:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-14T17:43:15.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T17:43:15.864 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-07-14T17:43:16.155 DEBUG:teuthology.task.kernel:sha1 for client.0 is a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T17:43:16.155 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "a440c79eee0258ae07e4988d8690fe6187b8b7a3"... 2023-07-14T17:43:16.156 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2023-07-14T17:43:16.175 INFO:teuthology.orchestra.run.smithi133.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-14T17:43:16.175 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T17:43:16.176 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-07-14T17:43:16.176 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum upgrade -y linux-firmware 2023-07-14T17:43:17.475 INFO:teuthology.orchestra.run.smithi133.stdout:Updating Subscription Management repositories. 2023-07-14T17:43:17.475 INFO:teuthology.orchestra.run.smithi133.stdout:Unable to read consumer identity 2023-07-14T17:43:18.998 INFO:teuthology.orchestra.run.smithi133.stdout:Copr repo for python3-asyncssh owned by ceph 12 kB/s | 3.5 kB 00:00 2023-07-14T17:43:23.389 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 3.6 MB/s | 16 MB 00:04 2023-07-14T17:43:26.806 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 488 kB/s | 24 kB 00:00 2023-07-14T17:43:27.582 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2023-07-14T17:43:27.583 INFO:teuthology.orchestra.run.smithi133.stdout:Nothing to do. 2023-07-14T17:43:27.583 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2023-07-14T17:43:27.706 INFO:teuthology.task.kernel:Downloading kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on client.0... 2023-07-14T17:43:27.707 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T17:43:28.138 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/ 2023-07-14T17:43:28.139 DEBUG:teuthology.orchestra.run.smithi133:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc1_ga440c79eee02-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/ --input-file=- 2023-07-14T17:43:28.141 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi133.front.sepia.ceph.com to complete... 2023-07-14T17:43:41.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14 17:43:42 URL:https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/kernel-6.5.0_rc1_ga440c79eee02-1.x86_64.rpm [380684764/380684764] -> "/tmp/kernel.x86_64.rpm" [1] 2023-07-14T17:43:41.985 INFO:teuthology.orchestra.run.smithi133.stderr:FINISHED --2023-07-14 17:43:42-- 2023-07-14T17:43:41.985 INFO:teuthology.orchestra.run.smithi133.stderr:Total wall clock time: 14s 2023-07-14T17:43:41.985 INFO:teuthology.orchestra.run.smithi133.stderr:Downloaded: 1 files, 363M in 13s (27.2 MB/s) 2023-07-14T17:43:41.986 INFO:teuthology.task.kernel:Installing kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on client.0... 2023-07-14T17:43:41.986 DEBUG:teuthology.orchestra.run.smithi133:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-07-14T17:43:43.578 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying... ######################################## 2023-07-14T17:43:43.643 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing... ######################################## 2023-07-14T17:43:43.729 INFO:teuthology.orchestra.run.smithi133.stdout:Updating / installing... 2023-07-14T17:44:21.928 INFO:teuthology.orchestra.run.smithi133.stdout:kernel-6.5.0_rc1_ga440c79eee02-1 ######################################## 2023-07-14T17:44:45.263 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi133.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-07-14T17:44:45.264 DEBUG:teuthology.orchestra.run.smithi133:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-07-14T17:44:45.312 INFO:teuthology.orchestra.run.smithi133.stdout:/boot/System.map-6.5.0-rc1-ga440c79eee02 2023-07-14T17:44:45.313 INFO:teuthology.orchestra.run.smithi133.stdout:/boot/config-6.5.0-rc1-ga440c79eee02 2023-07-14T17:44:45.313 INFO:teuthology.orchestra.run.smithi133.stdout:/boot/vmlinuz-6.5.0-rc1-ga440c79eee02 2023-07-14T17:44:45.313 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02 2023-07-14T17:44:45.313 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel 2023-07-14T17:44:45.313 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch 2023-07-14T17:44:45.314 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86 2023-07-14T17:44:45.314 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto 2023-07-14T17:44:45.314 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-07-14T17:44:45.314 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-07-14T17:44:45.314 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-07-14T17:44:45.314 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-07-14T17:44:45.314 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-07-14T17:44:45.314 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-07-14T17:44:45.315 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-07-14T17:44:45.315 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crc32c-intel.ko 2023-07-14T17:44:45.315 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-07-14T17:44:45.315 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-07-14T17:44:45.315 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-07-14T17:44:45.315 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-07-14T17:44:45.315 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx2.ko 2023-07-14T17:44:45.316 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-07-14T17:44:45.316 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-07-14T17:44:45.316 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-07-14T17:44:45.316 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-07-14T17:44:45.316 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-07-14T17:44:45.316 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-07-14T17:44:45.316 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-07-14T17:44:45.317 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel 2023-07-14T17:44:45.317 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu 2023-07-14T17:44:45.317 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce 2023-07-14T17:44:45.317 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-07-14T17:44:45.317 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm 2023-07-14T17:44:45.317 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-amd.ko 2023-07-14T17:44:45.317 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-intel.ko 2023-07-14T17:44:45.317 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm.ko 2023-07-14T17:44:45.318 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto 2023-07-14T17:44:45.318 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ansi_cprng.ko 2023-07-14T17:44:45.318 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/anubis.ko 2023-07-14T17:44:45.318 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/arc4.ko 2023-07-14T17:44:45.318 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx 2023-07-14T17:44:45.318 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_memcpy.ko 2023-07-14T17:44:45.318 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_pq.ko 2023-07-14T17:44:45.319 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_raid6_recov.ko 2023-07-14T17:44:45.319 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_tx.ko 2023-07-14T17:44:45.319 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_xor.ko 2023-07-14T17:44:45.319 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/raid6test.ko 2023-07-14T17:44:45.319 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authenc.ko 2023-07-14T17:44:45.319 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authencesn.ko 2023-07-14T17:44:45.319 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blake2b_generic.ko 2023-07-14T17:44:45.319 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_common.ko 2023-07-14T17:44:45.320 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_generic.ko 2023-07-14T17:44:45.320 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/camellia_generic.ko 2023-07-14T17:44:45.320 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast5_generic.ko 2023-07-14T17:44:45.320 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast6_generic.ko 2023-07-14T17:44:45.320 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast_common.ko 2023-07-14T17:44:45.320 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ccm.ko 2023-07-14T17:44:45.320 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cmac.ko 2023-07-14T17:44:45.321 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crc32_generic.ko 2023-07-14T17:44:45.321 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_engine.ko 2023-07-14T17:44:45.321 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_user.ko 2023-07-14T17:44:45.321 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cts.ko 2023-07-14T17:44:45.321 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/des_generic.ko 2023-07-14T17:44:45.322 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/dh_generic.ko 2023-07-14T17:44:45.322 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/echainiv.ko 2023-07-14T17:44:45.322 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/essiv.ko 2023-07-14T17:44:45.322 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/fcrypt.ko 2023-07-14T17:44:45.322 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/khazad.ko 2023-07-14T17:44:45.322 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4.ko 2023-07-14T17:44:45.323 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4hc.ko 2023-07-14T17:44:45.323 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/md4.ko 2023-07-14T17:44:45.323 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/michael_mic.ko 2023-07-14T17:44:45.323 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcbc.ko 2023-07-14T17:44:45.323 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcrypt.ko 2023-07-14T17:44:45.323 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/rmd160.ko 2023-07-14T17:44:45.323 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/seed.ko 2023-07-14T17:44:45.324 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/serpent_generic.ko 2023-07-14T17:44:45.324 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tcrypt.ko 2023-07-14T17:44:45.324 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tea.ko 2023-07-14T17:44:45.324 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_common.ko 2023-07-14T17:44:45.324 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_generic.ko 2023-07-14T17:44:45.325 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/vmac.ko 2023-07-14T17:44:45.325 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/wp512.ko 2023-07-14T17:44:45.325 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xcbc.ko 2023-07-14T17:44:45.325 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xor.ko 2023-07-14T17:44:45.325 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xxhash_generic.ko 2023-07-14T17:44:45.325 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers 2023-07-14T17:44:45.326 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi 2023-07-14T17:44:45.326 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_ipmi.ko 2023-07-14T17:44:45.326 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_pad.ko 2023-07-14T17:44:45.326 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/custom_method.ko 2023-07-14T17:44:45.326 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/ec_sys.ko 2023-07-14T17:44:45.326 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/platform_profile.ko 2023-07-14T17:44:45.326 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbs.ko 2023-07-14T17:44:45.326 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbshc.ko 2023-07-14T17:44:45.327 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/video.ko 2023-07-14T17:44:45.327 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata 2023-07-14T17:44:45.327 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/acard-ahci.ko 2023-07-14T17:44:45.327 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ahci_platform.ko 2023-07-14T17:44:45.327 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ata_generic.ko 2023-07-14T17:44:45.327 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/libahci_platform.ko 2023-07-14T17:44:45.327 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_acpi.ko 2023-07-14T17:44:45.328 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ali.ko 2023-07-14T17:44:45.328 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_amd.ko 2023-07-14T17:44:45.328 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_artop.ko 2023-07-14T17:44:45.328 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atiixp.ko 2023-07-14T17:44:45.328 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atp867x.ko 2023-07-14T17:44:45.328 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd640.ko 2023-07-14T17:44:45.328 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd64x.ko 2023-07-14T17:44:45.329 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cypress.ko 2023-07-14T17:44:45.329 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_efar.ko 2023-07-14T17:44:45.329 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt366.ko 2023-07-14T17:44:45.329 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt37x.ko 2023-07-14T17:44:45.329 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x2n.ko 2023-07-14T17:44:45.329 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x3.ko 2023-07-14T17:44:45.329 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it8213.ko 2023-07-14T17:44:45.329 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it821x.ko 2023-07-14T17:44:45.330 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_jmicron.ko 2023-07-14T17:44:45.330 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_marvell.ko 2023-07-14T17:44:45.330 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_mpiix.ko 2023-07-14T17:44:45.330 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_netcell.ko 2023-07-14T17:44:45.330 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ninja32.ko 2023-07-14T17:44:45.330 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87410.ko 2023-07-14T17:44:45.330 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87415.ko 2023-07-14T17:44:45.330 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_oldpiix.ko 2023-07-14T17:44:45.331 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_opti.ko 2023-07-14T17:44:45.331 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_optidma.ko 2023-07-14T17:44:45.331 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pcmcia.ko 2023-07-14T17:44:45.331 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc2027x.ko 2023-07-14T17:44:45.331 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-07-14T17:44:45.331 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_piccolo.ko 2023-07-14T17:44:45.331 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_rdc.ko 2023-07-14T17:44:45.332 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sch.ko 2023-07-14T17:44:45.332 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_serverworks.ko 2023-07-14T17:44:45.332 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sil680.ko 2023-07-14T17:44:45.332 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sis.ko 2023-07-14T17:44:45.332 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sl82c105.ko 2023-07-14T17:44:45.333 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_triflex.ko 2023-07-14T17:44:45.333 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_via.ko 2023-07-14T17:44:45.333 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pdc_adma.ko 2023-07-14T17:44:45.333 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_inic162x.ko 2023-07-14T17:44:45.333 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_mv.ko 2023-07-14T17:44:45.333 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_nv.ko 2023-07-14T17:44:45.334 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_promise.ko 2023-07-14T17:44:45.334 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_qstor.ko 2023-07-14T17:44:45.334 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil.ko 2023-07-14T17:44:45.334 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil24.ko 2023-07-14T17:44:45.334 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sis.ko 2023-07-14T17:44:45.334 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_svw.ko 2023-07-14T17:44:45.335 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sx4.ko 2023-07-14T17:44:45.335 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_uli.ko 2023-07-14T17:44:45.335 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_via.ko 2023-07-14T17:44:45.335 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_vsc.ko 2023-07-14T17:44:45.335 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay 2023-07-14T17:44:45.335 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay/ks0108.ko 2023-07-14T17:44:45.335 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base 2023-07-14T17:44:45.335 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap 2023-07-14T17:44:45.336 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap/regmap-i2c.ko 2023-07-14T17:44:45.336 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap/regmap-mmio.ko 2023-07-14T17:44:45.336 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma 2023-07-14T17:44:45.336 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma/bcma.ko 2023-07-14T17:44:45.336 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block 2023-07-14T17:44:45.336 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe 2023-07-14T17:44:45.336 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe/aoe.ko 2023-07-14T17:44:45.337 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/brd.ko 2023-07-14T17:44:45.337 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd 2023-07-14T17:44:45.337 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd/drbd.ko 2023-07-14T17:44:45.337 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/floppy.ko 2023-07-14T17:44:45.337 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/loop.ko 2023-07-14T17:44:45.337 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx 2023-07-14T17:44:45.337 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-07-14T17:44:45.337 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/nbd.ko 2023-07-14T17:44:45.338 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk 2023-07-14T17:44:45.338 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk/null_blk.ko 2023-07-14T17:44:45.338 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/pktcdvd.ko 2023-07-14T17:44:45.338 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/rbd.ko 2023-07-14T17:44:45.338 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/virtio_blk.ko 2023-07-14T17:44:45.338 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback 2023-07-14T17:44:45.338 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-07-14T17:44:45.339 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkfront.ko 2023-07-14T17:44:45.339 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram 2023-07-14T17:44:45.339 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram/zram.ko 2023-07-14T17:44:45.339 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char 2023-07-14T17:44:45.339 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hangcheck-timer.ko 2023-07-14T17:44:45.339 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random 2023-07-14T17:44:45.339 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/amd-rng.ko 2023-07-14T17:44:45.339 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/intel-rng.ko 2023-07-14T17:44:45.340 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-07-14T17:44:45.340 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/via-rng.ko 2023-07-14T17:44:45.340 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/virtio-rng.ko 2023-07-14T17:44:45.340 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi 2023-07-14T17:44:45.340 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-07-14T17:44:45.341 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-07-14T17:44:45.341 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-07-14T17:44:45.341 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_si.ko 2023-07-14T17:44:45.341 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-07-14T17:44:45.341 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-07-14T17:44:45.341 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/lp.ko 2023-07-14T17:44:45.341 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave 2023-07-14T17:44:45.342 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave/mwave.ko 2023-07-14T17:44:45.342 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ppdev.ko 2023-07-14T17:44:45.342 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tlclk.ko 2023-07-14T17:44:45.342 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm 2023-07-14T17:44:45.342 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm.ko 2023-07-14T17:44:45.342 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_atmel.ko 2023-07-14T17:44:45.342 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_infineon.ko 2023-07-14T17:44:45.343 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_nsc.ko 2023-07-14T17:44:45.343 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_tis.ko 2023-07-14T17:44:45.343 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-07-14T17:44:45.343 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/uv_mmtimer.ko 2023-07-14T17:44:45.344 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/virtio_console.ko 2023-07-14T17:44:45.344 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq 2023-07-14T17:44:45.344 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-07-14T17:44:45.344 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-07-14T17:44:45.344 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/p4-clockmod.ko 2023-07-14T17:44:45.345 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-07-14T17:44:45.345 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/powernow-k8.ko 2023-07-14T17:44:45.345 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/speedstep-lib.ko 2023-07-14T17:44:45.345 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto 2023-07-14T17:44:45.346 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp 2023-07-14T17:44:45.346 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-07-14T17:44:45.346 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp.ko 2023-07-14T17:44:45.346 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel 2023-07-14T17:44:45.346 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat 2023-07-14T17:44:45.346 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common 2023-07-14T17:44:45.346 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-07-14T17:44:45.347 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-07-14T17:44:45.347 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-07-14T17:44:45.347 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-aes.ko 2023-07-14T17:44:45.347 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-sha.ko 2023-07-14T17:44:45.347 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio 2023-07-14T17:44:45.347 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-07-14T17:44:45.347 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca 2023-07-14T17:44:45.347 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca/dca.ko 2023-07-14T17:44:45.348 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/devfreq 2023-07-14T17:44:45.348 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-07-14T17:44:45.348 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma 2023-07-14T17:44:45.348 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw 2023-07-14T17:44:45.349 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw/dw_dmac.ko 2023-07-14T17:44:45.349 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat 2023-07-14T17:44:45.349 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat/ioatdma.ko 2023-07-14T17:44:45.349 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac 2023-07-14T17:44:45.350 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/amd64_edac.ko 2023-07-14T17:44:45.350 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/e752x_edac.ko 2023-07-14T17:44:45.350 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/edac_mce_amd.ko 2023-07-14T17:44:45.351 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3000_edac.ko 2023-07-14T17:44:45.351 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3200_edac.ko 2023-07-14T17:44:45.351 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5100_edac.ko 2023-07-14T17:44:45.352 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5400_edac.ko 2023-07-14T17:44:45.352 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7300_edac.ko 2023-07-14T17:44:45.352 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7core_edac.ko 2023-07-14T17:44:45.352 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i82975x_edac.ko 2023-07-14T17:44:45.353 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/ie31200_edac.ko 2023-07-14T17:44:45.353 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/sb_edac.ko 2023-07-14T17:44:45.353 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/x38_edac.ko 2023-07-14T17:44:45.354 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware 2023-07-14T17:44:45.354 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/edd.ko 2023-07-14T17:44:45.354 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/iscsi_ibft.ko 2023-07-14T17:44:45.354 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio 2023-07-14T17:44:45.355 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-ich.ko 2023-07-14T17:44:45.355 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-viperboard.ko 2023-07-14T17:44:45.355 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid 2023-07-14T17:44:45.356 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-appleir.ko 2023-07-14T17:44:45.356 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-aureal.ko 2023-07-14T17:44:45.356 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-axff.ko 2023-07-14T17:44:45.356 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-dr.ko 2023-07-14T17:44:45.357 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elecom.ko 2023-07-14T17:44:45.357 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elo.ko 2023-07-14T17:44:45.357 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-emsff.ko 2023-07-14T17:44:45.357 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gaff.ko 2023-07-14T17:44:45.357 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gt683r.ko 2023-07-14T17:44:45.358 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gyration.ko 2023-07-14T17:44:45.358 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-kbd.ko 2023-07-14T17:44:45.358 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-mouse.ko 2023-07-14T17:44:45.359 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtekff.ko 2023-07-14T17:44:45.359 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-hyperv.ko 2023-07-14T17:44:45.359 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-icade.ko 2023-07-14T17:44:45.359 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-keytouch.ko 2023-07-14T17:44:45.359 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-kye.ko 2023-07-14T17:44:45.359 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lcpower.ko 2023-07-14T17:44:45.360 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-led.ko 2023-07-14T17:44:45.360 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lenovo.ko 2023-07-14T17:44:45.360 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-dj.ko 2023-07-14T17:44:45.360 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-07-14T17:44:45.360 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-multitouch.ko 2023-07-14T17:44:45.361 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-ortek.ko 2023-07-14T17:44:45.361 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-penmount.ko 2023-07-14T17:44:45.361 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-petalynx.ko 2023-07-14T17:44:45.361 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-picolcd.ko 2023-07-14T17:44:45.361 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-pl.ko 2023-07-14T17:44:45.361 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-plantronics.ko 2023-07-14T17:44:45.362 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-primax.ko 2023-07-14T17:44:45.362 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-rmi.ko 2023-07-14T17:44:45.362 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-arvo.ko 2023-07-14T17:44:45.362 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-common.ko 2023-07-14T17:44:45.363 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-isku.ko 2023-07-14T17:44:45.363 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kone.ko 2023-07-14T17:44:45.363 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-07-14T17:44:45.363 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-konepure.ko 2023-07-14T17:44:45.364 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-07-14T17:44:45.364 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-lua.ko 2023-07-14T17:44:45.364 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-pyra.ko 2023-07-14T17:44:45.364 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-ryos.ko 2023-07-14T17:44:45.365 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-savu.ko 2023-07-14T17:44:45.365 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat.ko 2023-07-14T17:44:45.365 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-saitek.ko 2023-07-14T17:44:45.365 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-samsung.ko 2023-07-14T17:44:45.366 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sensor-hub.ko 2023-07-14T17:44:45.366 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sjoy.ko 2023-07-14T17:44:45.366 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sony.ko 2023-07-14T17:44:45.366 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-speedlink.ko 2023-07-14T17:44:45.366 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-steelseries.ko 2023-07-14T17:44:45.367 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sunplus.ko 2023-07-14T17:44:45.367 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-thrustmaster.ko 2023-07-14T17:44:45.367 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tivo.ko 2023-07-14T17:44:45.367 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tmff.ko 2023-07-14T17:44:45.367 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-topseed.ko 2023-07-14T17:44:45.367 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-twinhan.ko 2023-07-14T17:44:45.367 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-uclogic.ko 2023-07-14T17:44:45.368 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-waltop.ko 2023-07-14T17:44:45.368 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-wiimote.ko 2023-07-14T17:44:45.368 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-xinmo.ko 2023-07-14T17:44:45.368 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zpff.ko 2023-07-14T17:44:45.368 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zydacron.ko 2023-07-14T17:44:45.368 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/uhid.ko 2023-07-14T17:44:45.368 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/wacom.ko 2023-07-14T17:44:45.369 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv 2023-07-14T17:44:45.369 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_balloon.ko 2023-07-14T17:44:45.369 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_utils.ko 2023-07-14T17:44:45.369 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_vmbus.ko 2023-07-14T17:44:45.369 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon 2023-07-14T17:44:45.369 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru.ko 2023-07-14T17:44:45.369 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru3.ko 2023-07-14T17:44:45.369 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/acpi_power_meter.ko 2023-07-14T17:44:45.370 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7414.ko 2023-07-14T17:44:45.370 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7418.ko 2023-07-14T17:44:45.370 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adc128d818.ko 2023-07-14T17:44:45.370 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1025.ko 2023-07-14T17:44:45.370 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1026.ko 2023-07-14T17:44:45.370 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1029.ko 2023-07-14T17:44:45.370 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1031.ko 2023-07-14T17:44:45.370 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm9240.ko 2023-07-14T17:44:45.371 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ads7828.ko 2023-07-14T17:44:45.371 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7410.ko 2023-07-14T17:44:45.371 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7411.ko 2023-07-14T17:44:45.371 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7462.ko 2023-07-14T17:44:45.371 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7470.ko 2023-07-14T17:44:45.371 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7475.ko 2023-07-14T17:44:45.371 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7x10.ko 2023-07-14T17:44:45.372 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/amc6821.ko 2023-07-14T17:44:45.372 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/applesmc.ko 2023-07-14T17:44:45.372 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asb100.ko 2023-07-14T17:44:45.372 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asc7621.ko 2023-07-14T17:44:45.372 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asus_atk0110.ko 2023-07-14T17:44:45.372 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/atxp1.ko 2023-07-14T17:44:45.372 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/coretemp.ko 2023-07-14T17:44:45.373 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-07-14T17:44:45.373 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dme1737.ko 2023-07-14T17:44:45.373 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds1621.ko 2023-07-14T17:44:45.373 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds620.ko 2023-07-14T17:44:45.373 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc1403.ko 2023-07-14T17:44:45.373 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc6w201.ko 2023-07-14T17:44:45.373 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71805f.ko 2023-07-14T17:44:45.373 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71882fg.ko 2023-07-14T17:44:45.374 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f75375s.ko 2023-07-14T17:44:45.374 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fam15h_power.ko 2023-07-14T17:44:45.374 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fschmd.ko 2023-07-14T17:44:45.374 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g760a.ko 2023-07-14T17:44:45.374 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g762.ko 2023-07-14T17:44:45.374 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl518sm.ko 2023-07-14T17:44:45.374 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl520sm.ko 2023-07-14T17:44:45.374 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/hwmon-vid.ko 2023-07-14T17:44:45.375 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5500_temp.ko 2023-07-14T17:44:45.375 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5k_amb.ko 2023-07-14T17:44:45.375 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmaem.ko 2023-07-14T17:44:45.375 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmpex.ko 2023-07-14T17:44:45.375 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina209.ko 2023-07-14T17:44:45.375 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina2xx.ko 2023-07-14T17:44:45.376 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/it87.ko 2023-07-14T17:44:45.376 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k10temp.ko 2023-07-14T17:44:45.376 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k8temp.ko 2023-07-14T17:44:45.376 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lineage-pem.ko 2023-07-14T17:44:45.376 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm63.ko 2023-07-14T17:44:45.377 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm73.ko 2023-07-14T17:44:45.377 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm75.ko 2023-07-14T17:44:45.377 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm77.ko 2023-07-14T17:44:45.377 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm78.ko 2023-07-14T17:44:45.378 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm80.ko 2023-07-14T17:44:45.378 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm83.ko 2023-07-14T17:44:45.378 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm85.ko 2023-07-14T17:44:45.378 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm87.ko 2023-07-14T17:44:45.379 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm90.ko 2023-07-14T17:44:45.379 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm92.ko 2023-07-14T17:44:45.379 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm93.ko 2023-07-14T17:44:45.380 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95234.ko 2023-07-14T17:44:45.380 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95241.ko 2023-07-14T17:44:45.380 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95245.ko 2023-07-14T17:44:45.380 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc2945.ko 2023-07-14T17:44:45.381 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4151.ko 2023-07-14T17:44:45.381 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4215.ko 2023-07-14T17:44:45.381 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4222.ko 2023-07-14T17:44:45.382 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4245.ko 2023-07-14T17:44:45.382 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4260.ko 2023-07-14T17:44:45.382 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4261.ko 2023-07-14T17:44:45.382 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max16065.ko 2023-07-14T17:44:45.382 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1619.ko 2023-07-14T17:44:45.382 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1668.ko 2023-07-14T17:44:45.383 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max197.ko 2023-07-14T17:44:45.383 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6639.ko 2023-07-14T17:44:45.383 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6650.ko 2023-07-14T17:44:45.383 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6697.ko 2023-07-14T17:44:45.383 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/mcp3021.ko 2023-07-14T17:44:45.383 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6683.ko 2023-07-14T17:44:45.383 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775-core.ko 2023-07-14T17:44:45.384 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775.ko 2023-07-14T17:44:45.384 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct7802.ko 2023-07-14T17:44:45.384 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87360.ko 2023-07-14T17:44:45.384 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87427.ko 2023-07-14T17:44:45.384 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pcf8591.ko 2023-07-14T17:44:45.384 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus 2023-07-14T17:44:45.384 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-07-14T17:44:45.385 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-07-14T17:44:45.385 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-07-14T17:44:45.385 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max16064.ko 2023-07-14T17:44:45.385 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max34440.ko 2023-07-14T17:44:45.385 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max8688.ko 2023-07-14T17:44:45.385 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-07-14T17:44:45.385 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-07-14T17:44:45.386 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-07-14T17:44:45.386 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-07-14T17:44:45.386 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-07-14T17:44:45.386 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-07-14T17:44:45.386 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/powr1220.ko 2023-07-14T17:44:45.386 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5627.ko 2023-07-14T17:44:45.386 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5636.ko 2023-07-14T17:44:45.386 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch56xx-common.ko 2023-07-14T17:44:45.387 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht15.ko 2023-07-14T17:44:45.387 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht21.ko 2023-07-14T17:44:45.387 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/shtc1.ko 2023-07-14T17:44:45.388 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sis5595.ko 2023-07-14T17:44:45.388 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47b397.ko 2023-07-14T17:44:45.388 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m1.ko 2023-07-14T17:44:45.389 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m192.ko 2023-07-14T17:44:45.389 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/thmc50.ko 2023-07-14T17:44:45.389 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp102.ko 2023-07-14T17:44:45.390 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp103.ko 2023-07-14T17:44:45.390 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp401.ko 2023-07-14T17:44:45.390 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp421.ko 2023-07-14T17:44:45.390 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via-cputemp.ko 2023-07-14T17:44:45.391 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via686a.ko 2023-07-14T17:44:45.391 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt1211.ko 2023-07-14T17:44:45.391 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt8231.ko 2023-07-14T17:44:45.391 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627ehf.ko 2023-07-14T17:44:45.391 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627hf.ko 2023-07-14T17:44:45.392 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83781d.ko 2023-07-14T17:44:45.392 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83791d.ko 2023-07-14T17:44:45.392 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83792d.ko 2023-07-14T17:44:45.392 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83793.ko 2023-07-14T17:44:45.393 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83795.ko 2023-07-14T17:44:45.393 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l785ts.ko 2023-07-14T17:44:45.393 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l786ng.ko 2023-07-14T17:44:45.394 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c 2023-07-14T17:44:45.394 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos 2023-07-14T17:44:45.394 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-07-14T17:44:45.394 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-07-14T17:44:45.394 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses 2023-07-14T17:44:45.394 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-07-14T17:44:45.394 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-07-14T17:44:45.395 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-07-14T17:44:45.395 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-07-14T17:44:45.395 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-07-14T17:44:45.395 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-07-14T17:44:45.395 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-07-14T17:44:45.395 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-07-14T17:44:45.395 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-i801.ko 2023-07-14T17:44:45.396 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-isch.ko 2023-07-14T17:44:45.396 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-07-14T17:44:45.396 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-07-14T17:44:45.396 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-07-14T17:44:45.396 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-parport.ko 2023-07-14T17:44:45.396 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-07-14T17:44:45.396 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-07-14T17:44:45.397 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-07-14T17:44:45.397 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-07-14T17:44:45.397 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-07-14T17:44:45.397 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-07-14T17:44:45.397 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-via.ko 2023-07-14T17:44:45.397 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-07-14T17:44:45.397 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-07-14T17:44:45.398 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-dev.ko 2023-07-14T17:44:45.398 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-mux.ko 2023-07-14T17:44:45.398 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-07-14T17:44:45.398 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-smbus.ko 2023-07-14T17:44:45.398 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-stub.ko 2023-07-14T17:44:45.398 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband 2023-07-14T17:44:45.398 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core 2023-07-14T17:44:45.398 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_cm.ko 2023-07-14T17:44:45.399 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_core.ko 2023-07-14T17:44:45.399 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_umad.ko 2023-07-14T17:44:45.399 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-07-14T17:44:45.399 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/iw_cm.ko 2023-07-14T17:44:45.399 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_cm.ko 2023-07-14T17:44:45.399 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-07-14T17:44:45.399 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw 2023-07-14T17:44:45.400 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4 2023-07-14T17:44:45.400 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-07-14T17:44:45.400 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4 2023-07-14T17:44:45.400 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-07-14T17:44:45.400 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5 2023-07-14T17:44:45.400 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-07-14T17:44:45.400 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca 2023-07-14T17:44:45.400 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-07-14T17:44:45.401 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp 2023-07-14T17:44:45.401 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib 2023-07-14T17:44:45.401 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-07-14T17:44:45.401 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser 2023-07-14T17:44:45.401 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-07-14T17:44:45.401 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert 2023-07-14T17:44:45.402 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-07-14T17:44:45.402 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp 2023-07-14T17:44:45.402 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-07-14T17:44:45.402 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt 2023-07-14T17:44:45.402 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-07-14T17:44:45.402 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input 2023-07-14T17:44:45.402 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport 2023-07-14T17:44:45.403 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-07-14T17:44:45.403 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/fm801-gp.ko 2023-07-14T17:44:45.403 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/gameport.ko 2023-07-14T17:44:45.403 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/lightning.ko 2023-07-14T17:44:45.403 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/ns558.ko 2023-07-14T17:44:45.404 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/joydev.ko 2023-07-14T17:44:45.404 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard 2023-07-14T17:44:45.404 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/gpio_keys.ko 2023-07-14T17:44:45.404 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse 2023-07-14T17:44:45.404 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/appletouch.ko 2023-07-14T17:44:45.404 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/bcm5974.ko 2023-07-14T17:44:45.404 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/cyapatp.ko 2023-07-14T17:44:45.405 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/elan_i2c.ko 2023-07-14T17:44:45.405 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/sermouse.ko 2023-07-14T17:44:45.405 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-07-14T17:44:45.405 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_usb.ko 2023-07-14T17:44:45.405 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/vsxxxaa.ko 2023-07-14T17:44:45.405 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4 2023-07-14T17:44:45.405 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4/rmi_core.ko 2023-07-14T17:44:45.406 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio 2023-07-14T17:44:45.406 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/altera_ps2.ko 2023-07-14T17:44:45.406 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/arc_ps2.ko 2023-07-14T17:44:45.406 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-07-14T17:44:45.406 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/serio_raw.ko 2023-07-14T17:44:45.406 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/sparse-keymap.ko 2023-07-14T17:44:45.406 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu 2023-07-14T17:44:45.407 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd 2023-07-14T17:44:45.407 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd/iommu_v2.ko 2023-07-14T17:44:45.407 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds 2023-07-14T17:44:45.407 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-blinkm.ko 2023-07-14T17:44:45.407 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lm3530.ko 2023-07-14T17:44:45.407 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lp3944.ko 2023-07-14T17:44:45.407 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-ss4200.ko 2023-07-14T17:44:45.408 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger 2023-07-14T17:44:45.408 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-07-14T17:44:45.408 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-07-14T17:44:45.408 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-07-14T17:44:45.408 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-07-14T17:44:45.408 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-07-14T17:44:45.408 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-07-14T17:44:45.408 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-07-14T17:44:45.409 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-07-14T17:44:45.409 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md 2023-07-14T17:44:45.409 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache 2023-07-14T17:44:45.409 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache/bcache.ko 2023-07-14T17:44:45.409 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-bio-prison.ko 2023-07-14T17:44:45.409 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache-smq.ko 2023-07-14T17:44:45.409 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache.ko 2023-07-14T17:44:45.410 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-crypt.ko 2023-07-14T17:44:45.410 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-delay.ko 2023-07-14T17:44:45.410 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-flakey.ko 2023-07-14T17:44:45.410 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-log-userspace.ko 2023-07-14T17:44:45.410 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-multipath.ko 2023-07-14T17:44:45.410 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-queue-length.ko 2023-07-14T17:44:45.410 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-raid.ko 2023-07-14T17:44:45.411 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-round-robin.ko 2023-07-14T17:44:45.411 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-service-time.ko 2023-07-14T17:44:45.411 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-switch.ko 2023-07-14T17:44:45.411 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-thin-pool.ko 2023-07-14T17:44:45.411 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-verity.ko 2023-07-14T17:44:45.411 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/faulty.ko 2023-07-14T17:44:45.411 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/linear.ko 2023-07-14T17:44:45.411 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/multipath.ko 2023-07-14T17:44:45.412 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data 2023-07-14T17:44:45.412 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-07-14T17:44:45.412 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid0.ko 2023-07-14T17:44:45.412 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid1.ko 2023-07-14T17:44:45.412 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid10.ko 2023-07-14T17:44:45.412 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid456.ko 2023-07-14T17:44:45.412 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message 2023-07-14T17:44:45.413 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion 2023-07-14T17:44:45.413 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptbase.ko 2023-07-14T17:44:45.413 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptctl.ko 2023-07-14T17:44:45.413 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptfc.ko 2023-07-14T17:44:45.413 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptlan.ko 2023-07-14T17:44:45.413 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptsas.ko 2023-07-14T17:44:45.413 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptscsih.ko 2023-07-14T17:44:45.413 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptspi.ko 2023-07-14T17:44:45.414 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd 2023-07-14T17:44:45.414 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_ich.ko 2023-07-14T17:44:45.414 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_sch.ko 2023-07-14T17:44:45.414 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/mfd-core.ko 2023-07-14T17:44:45.414 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/sm501.ko 2023-07-14T17:44:45.414 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/viperboard.ko 2023-07-14T17:44:45.414 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/vx855.ko 2023-07-14T17:44:45.415 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/wl1273-core.ko 2023-07-14T17:44:45.415 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc 2023-07-14T17:44:45.415 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl 2023-07-14T17:44:45.415 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-07-14T17:44:45.415 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds9802als.ko 2023-07-14T17:44:45.416 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds990x.ko 2023-07-14T17:44:45.416 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/bh1770glc.ko 2023-07-14T17:44:45.416 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710 2023-07-14T17:44:45.416 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710/cb710.ko 2023-07-14T17:44:45.416 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom 2023-07-14T17:44:45.416 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/at24.ko 2023-07-14T17:44:45.416 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom.ko 2023-07-14T17:44:45.417 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-07-14T17:44:45.417 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/max6875.ko 2023-07-14T17:44:45.417 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/enclosure.ko 2023-07-14T17:44:45.417 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/hpilo.ko 2023-07-14T17:44:45.417 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29003.ko 2023-07-14T17:44:45.417 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29020.ko 2023-07-14T17:44:45.417 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d 2023-07-14T17:44:45.417 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-07-14T17:44:45.418 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-07-14T17:44:45.418 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei 2023-07-14T17:44:45.418 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-me.ko 2023-07-14T17:44:45.418 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-txe.ko 2023-07-14T17:44:45.418 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei.ko 2023-07-14T17:44:45.418 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-gru 2023-07-14T17:44:45.418 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-gru/gru.ko 2023-07-14T17:44:45.418 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp 2023-07-14T17:44:45.419 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xp.ko 2023-07-14T17:44:45.419 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xpc.ko 2023-07-14T17:44:45.419 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-07-14T17:44:45.419 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_7xx1.ko 2023-07-14T17:44:45.419 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_core.ko 2023-07-14T17:44:45.419 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tsl2550.ko 2023-07-14T17:44:45.419 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_balloon.ko 2023-07-14T17:44:45.420 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci 2023-07-14T17:44:45.420 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-07-14T17:44:45.420 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net 2023-07-14T17:44:45.420 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding 2023-07-14T17:44:45.420 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding/bonding.ko 2023-07-14T17:44:45.420 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa 2023-07-14T17:44:45.420 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53 2023-07-14T17:44:45.421 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53/b53_common.ko 2023-07-14T17:44:45.421 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/bcm-sf2.ko 2023-07-14T17:44:45.421 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6060.ko 2023-07-14T17:44:45.421 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx 2023-07-14T17:44:45.421 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-07-14T17:44:45.421 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dummy.ko 2023-07-14T17:44:45.421 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/eql.ko 2023-07-14T17:44:45.422 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet 2023-07-14T17:44:45.422 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com 2023-07-14T17:44:45.422 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-07-14T17:44:45.422 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-07-14T17:44:45.422 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-07-14T17:44:45.422 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-07-14T17:44:45.423 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390 2023-07-14T17:44:45.423 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/8390.ko 2023-07-14T17:44:45.423 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-07-14T17:44:45.423 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-07-14T17:44:45.423 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-07-14T17:44:45.423 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec 2023-07-14T17:44:45.423 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-07-14T17:44:45.423 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere 2023-07-14T17:44:45.424 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere/et131x.ko 2023-07-14T17:44:45.424 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon 2023-07-14T17:44:45.424 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-07-14T17:44:45.424 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera 2023-07-14T17:44:45.424 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-07-14T17:44:45.424 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd 2023-07-14T17:44:45.424 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-07-14T17:44:45.424 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-07-14T17:44:45.425 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-07-14T17:44:45.425 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros 2023-07-14T17:44:45.425 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx 2023-07-14T17:44:45.425 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-07-14T17:44:45.425 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c 2023-07-14T17:44:45.425 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-07-14T17:44:45.425 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e 2023-07-14T17:44:45.426 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-07-14T17:44:45.426 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx 2023-07-14T17:44:45.426 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-07-14T17:44:45.426 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-07-14T17:44:45.426 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom 2023-07-14T17:44:45.426 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-07-14T17:44:45.426 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-07-14T17:44:45.426 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-07-14T17:44:45.427 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-07-14T17:44:45.427 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-07-14T17:44:45.427 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet 2023-07-14T17:44:45.427 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-07-14T17:44:45.427 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-07-14T17:44:45.427 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade 2023-07-14T17:44:45.427 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna 2023-07-14T17:44:45.427 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-07-14T17:44:45.428 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio 2023-07-14T17:44:45.428 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb 2023-07-14T17:44:45.428 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-07-14T17:44:45.428 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-07-14T17:44:45.428 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-07-14T17:44:45.428 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-07-14T17:44:45.428 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-07-14T17:44:45.429 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-07-14T17:44:45.429 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-07-14T17:44:45.429 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-07-14T17:44:45.429 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-07-14T17:44:45.429 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco 2023-07-14T17:44:45.429 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic 2023-07-14T17:44:45.429 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-07-14T17:44:45.429 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec 2023-07-14T17:44:45.430 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip 2023-07-14T17:44:45.430 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-07-14T17:44:45.430 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-07-14T17:44:45.430 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-07-14T17:44:45.430 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-07-14T17:44:45.430 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-07-14T17:44:45.430 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-07-14T17:44:45.430 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink 2023-07-14T17:44:45.431 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-07-14T17:44:45.431 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-07-14T17:44:45.431 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dnet.ko 2023-07-14T17:44:45.431 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex 2023-07-14T17:44:45.431 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet 2023-07-14T17:44:45.431 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-07-14T17:44:45.431 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ethoc.ko 2023-07-14T17:44:45.432 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/fealnx.ko 2023-07-14T17:44:45.432 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel 2023-07-14T17:44:45.432 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e100.ko 2023-07-14T17:44:45.432 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000 2023-07-14T17:44:45.432 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-07-14T17:44:45.432 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e 2023-07-14T17:44:45.432 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-07-14T17:44:45.432 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k 2023-07-14T17:44:45.433 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-07-14T17:44:45.433 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e 2023-07-14T17:44:45.433 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-07-14T17:44:45.433 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf 2023-07-14T17:44:45.433 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-07-14T17:44:45.433 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb 2023-07-14T17:44:45.433 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-07-14T17:44:45.433 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf 2023-07-14T17:44:45.434 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-07-14T17:44:45.434 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe 2023-07-14T17:44:45.434 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-07-14T17:44:45.434 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf 2023-07-14T17:44:45.434 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-07-14T17:44:45.434 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/jme.ko 2023-07-14T17:44:45.434 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell 2023-07-14T17:44:45.434 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-07-14T17:44:45.435 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/skge.ko 2023-07-14T17:44:45.435 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-07-14T17:44:45.435 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox 2023-07-14T17:44:45.435 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4 2023-07-14T17:44:45.435 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-07-14T17:44:45.435 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-07-14T17:44:45.435 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5 2023-07-14T17:44:45.435 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-07-14T17:44:45.436 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-07-14T17:44:45.436 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel 2023-07-14T17:44:45.436 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-07-14T17:44:45.436 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom 2023-07-14T17:44:45.436 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge 2023-07-14T17:44:45.436 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-07-14T17:44:45.436 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi 2023-07-14T17:44:45.437 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-07-14T17:44:45.437 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-07-14T17:44:45.437 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion 2023-07-14T17:44:45.437 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-07-14T17:44:45.437 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia 2023-07-14T17:44:45.437 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-07-14T17:44:45.437 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines 2023-07-14T17:44:45.437 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-07-14T17:44:45.438 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-07-14T17:44:45.438 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic 2023-07-14T17:44:45.438 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen 2023-07-14T17:44:45.438 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-07-14T17:44:45.438 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-07-14T17:44:45.438 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-07-14T17:44:45.438 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-07-14T17:44:45.438 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc 2023-07-14T17:44:45.439 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-07-14T17:44:45.439 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek 2023-07-14T17:44:45.439 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-07-14T17:44:45.439 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-07-14T17:44:45.439 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/atp.ko 2023-07-14T17:44:45.439 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-07-14T17:44:45.439 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rocker 2023-07-14T17:44:45.440 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-07-14T17:44:45.440 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc 2023-07-14T17:44:45.440 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-07-14T17:44:45.440 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan 2023-07-14T17:44:45.440 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-07-14T17:44:45.440 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis 2023-07-14T17:44:45.440 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis190.ko 2023-07-14T17:44:45.440 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis900.ko 2023-07-14T17:44:45.441 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc 2023-07-14T17:44:45.441 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-07-14T17:44:45.441 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-07-14T17:44:45.441 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-07-14T17:44:45.441 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-07-14T17:44:45.441 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro 2023-07-14T17:44:45.441 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac 2023-07-14T17:44:45.441 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-07-14T17:44:45.442 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-07-14T17:44:45.442 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun 2023-07-14T17:44:45.442 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/cassini.ko 2023-07-14T17:44:45.442 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/niu.ko 2023-07-14T17:44:45.442 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sungem.ko 2023-07-14T17:44:45.442 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-07-14T17:44:45.442 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti 2023-07-14T17:44:45.442 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-07-14T17:44:45.443 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti 2023-07-14T17:44:45.443 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti/tlan.ko 2023-07-14T17:44:45.443 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via 2023-07-14T17:44:45.443 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-07-14T17:44:45.443 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-07-14T17:44:45.443 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet 2023-07-14T17:44:45.443 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-07-14T17:44:45.444 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-07-14T17:44:45.444 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom 2023-07-14T17:44:45.444 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-07-14T17:44:45.444 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/geneve.ko 2023-07-14T17:44:45.444 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv 2023-07-14T17:44:45.444 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-07-14T17:44:45.444 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154 2023-07-14T17:44:45.444 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154/fakelb.ko 2023-07-14T17:44:45.445 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ifb.ko 2023-07-14T17:44:45.445 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan 2023-07-14T17:44:45.445 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan/ipvlan.ko 2023-07-14T17:44:45.445 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvlan.ko 2023-07-14T17:44:45.445 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvtap.ko 2023-07-14T17:44:45.445 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio 2023-07-14T17:44:45.445 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio.ko 2023-07-14T17:44:45.445 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-07-14T17:44:45.446 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-07-14T17:44:45.446 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-regmap.ko 2023-07-14T17:44:45.446 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mii.ko 2023-07-14T17:44:45.446 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/net_failover.ko 2023-07-14T17:44:45.446 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/netconsole.ko 2023-07-14T17:44:45.446 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/nlmon.ko 2023-07-14T17:44:45.446 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs 2023-07-14T17:44:45.446 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs-lynx.ko 2023-07-14T17:44:45.447 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-07-14T17:44:45.447 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy 2023-07-14T17:44:45.447 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/amd.ko 2023-07-14T17:44:45.447 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-07-14T17:44:45.447 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-07-14T17:44:45.447 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm7xxx.ko 2023-07-14T17:44:45.447 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm87xx.ko 2023-07-14T17:44:45.448 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/broadcom.ko 2023-07-14T17:44:45.448 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/cicada.ko 2023-07-14T17:44:45.448 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/davicom.ko 2023-07-14T17:44:45.448 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/dp83640.ko 2023-07-14T17:44:45.448 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/et1011c.ko 2023-07-14T17:44:45.448 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/icplus.ko 2023-07-14T17:44:45.448 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/lxt.ko 2023-07-14T17:44:45.448 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/marvell.ko 2023-07-14T17:44:45.449 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/micrel.ko 2023-07-14T17:44:45.449 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/national.ko 2023-07-14T17:44:45.449 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/phylink.ko 2023-07-14T17:44:45.449 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/qsemi.ko 2023-07-14T17:44:45.449 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/realtek.ko 2023-07-14T17:44:45.449 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/smsc.ko 2023-07-14T17:44:45.449 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/ste10Xp.ko 2023-07-14T17:44:45.449 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/vitesse.ko 2023-07-14T17:44:45.450 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp 2023-07-14T17:44:45.450 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/bsd_comp.ko 2023-07-14T17:44:45.450 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_async.ko 2023-07-14T17:44:45.450 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_deflate.ko 2023-07-14T17:44:45.450 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_generic.ko 2023-07-14T17:44:45.450 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_mppe.ko 2023-07-14T17:44:45.450 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_synctty.ko 2023-07-14T17:44:45.451 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppoe.ko 2023-07-14T17:44:45.451 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppox.ko 2023-07-14T17:44:45.451 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pptp.ko 2023-07-14T17:44:45.451 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip 2023-07-14T17:44:45.451 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/slhc.ko 2023-07-14T17:44:45.451 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/slip.ko 2023-07-14T17:44:45.451 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/sungem_phy.ko 2023-07-14T17:44:45.451 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/tap.ko 2023-07-14T17:44:45.452 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team 2023-07-14T17:44:45.452 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team.ko 2023-07-14T17:44:45.452 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_activebackup.ko 2023-07-14T17:44:45.452 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_broadcast.ko 2023-07-14T17:44:45.452 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-07-14T17:44:45.452 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_random.ko 2023-07-14T17:44:45.452 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-07-14T17:44:45.452 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/tun.ko 2023-07-14T17:44:45.453 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/veth.ko 2023-07-14T17:44:45.453 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/virtio_net.ko 2023-07-14T17:44:45.453 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3 2023-07-14T17:44:45.453 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-07-14T17:44:45.453 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan 2023-07-14T17:44:45.453 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan/vxlan.ko 2023-07-14T17:44:45.453 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback 2023-07-14T17:44:45.453 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback/xen-netback.ko 2023-07-14T17:44:45.454 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netfront.ko 2023-07-14T17:44:45.454 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb 2023-07-14T17:44:45.454 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb/ntb.ko 2023-07-14T17:44:45.454 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme 2023-07-14T17:44:45.454 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host 2023-07-14T17:44:45.454 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme-core.ko 2023-07-14T17:44:45.454 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme.ko 2023-07-14T17:44:45.455 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport 2023-07-14T17:44:45.455 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport.ko 2023-07-14T17:44:45.455 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_cs.ko 2023-07-14T17:44:45.455 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_pc.ko 2023-07-14T17:44:45.455 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_serial.ko 2023-07-14T17:44:45.455 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci 2023-07-14T17:44:45.455 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug 2023-07-14T17:44:45.455 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-07-14T17:44:45.456 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/pcie 2023-07-14T17:44:45.456 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/pcie/aer_inject.ko 2023-07-14T17:44:45.456 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/xen-pcifront.ko 2023-07-14T17:44:45.456 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia 2023-07-14T17:44:45.456 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/i82092.ko 2023-07-14T17:44:45.456 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/pd6729.ko 2023-07-14T17:44:45.456 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/yenta_socket.ko 2023-07-14T17:44:45.456 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl 2023-07-14T17:44:45.457 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel 2023-07-14T17:44:45.457 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-07-14T17:44:45.457 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform 2023-07-14T17:44:45.457 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86 2023-07-14T17:44:45.457 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acer-wmi.ko 2023-07-14T17:44:45.457 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acerhdf.ko 2023-07-14T17:44:45.457 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/apple-gmux.ko 2023-07-14T17:44:45.458 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-laptop.ko 2023-07-14T17:44:45.458 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-07-14T17:44:45.458 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-wmi.ko 2023-07-14T17:44:45.458 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/classmate-laptop.ko 2023-07-14T17:44:45.458 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-07-14T17:44:45.458 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-07-14T17:44:45.458 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-07-14T17:44:45.458 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-07-14T17:44:45.459 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/hdaps.ko 2023-07-14T17:44:45.459 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel 2023-07-14T17:44:45.459 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-07-14T17:44:45.459 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel_ips.ko 2023-07-14T17:44:45.459 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/msi-wmi.ko 2023-07-14T17:44:45.459 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/mxm-wmi.ko 2023-07-14T17:44:45.459 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-07-14T17:44:45.459 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-laptop.ko 2023-07-14T17:44:45.460 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-q10.ko 2023-07-14T17:44:45.460 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/topstar-laptop.ko 2023-07-14T17:44:45.460 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-07-14T17:44:45.460 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_haps.ko 2023-07-14T17:44:45.460 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi-bmof.ko 2023-07-14T17:44:45.460 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi.ko 2023-07-14T17:44:45.460 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap 2023-07-14T17:44:45.460 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_common.ko 2023-07-14T17:44:45.461 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_msr.ko 2023-07-14T17:44:45.461 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps 2023-07-14T17:44:45.461 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients 2023-07-14T17:44:45.461 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-gpio.ko 2023-07-14T17:44:45.461 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-ldisc.ko 2023-07-14T17:44:45.461 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps_parport.ko 2023-07-14T17:44:45.461 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/pps_core.ko 2023-07-14T17:44:45.462 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp 2023-07-14T17:44:45.462 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp.ko 2023-07-14T17:44:45.462 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp_kvm.ko 2023-07-14T17:44:45.462 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc 2023-07-14T17:44:45.462 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq32k.ko 2023-07-14T17:44:45.462 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq4802.ko 2023-07-14T17:44:45.462 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1286.ko 2023-07-14T17:44:45.462 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1307.ko 2023-07-14T17:44:45.463 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1374.ko 2023-07-14T17:44:45.463 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1511.ko 2023-07-14T17:44:45.463 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1553.ko 2023-07-14T17:44:45.463 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1672.ko 2023-07-14T17:44:45.463 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1742.ko 2023-07-14T17:44:45.463 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds2404.ko 2023-07-14T17:44:45.463 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds3232.ko 2023-07-14T17:44:45.463 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-em3027.ko 2023-07-14T17:44:45.464 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-fm3130.ko 2023-07-14T17:44:45.464 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl12022.ko 2023-07-14T17:44:45.464 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl1208.ko 2023-07-14T17:44:45.464 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m41t80.ko 2023-07-14T17:44:45.464 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t35.ko 2023-07-14T17:44:45.464 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t59.ko 2023-07-14T17:44:45.464 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-max6900.ko 2023-07-14T17:44:45.464 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-msm6242.ko 2023-07-14T17:44:45.465 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf2127.ko 2023-07-14T17:44:45.465 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf85063.ko 2023-07-14T17:44:45.465 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8523.ko 2023-07-14T17:44:45.465 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8563.ko 2023-07-14T17:44:45.465 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8583.ko 2023-07-14T17:44:45.465 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rp5c01.ko 2023-07-14T17:44:45.465 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rs5c372.ko 2023-07-14T17:44:45.465 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-07-14T17:44:45.466 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8025.ko 2023-07-14T17:44:45.466 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8581.ko 2023-07-14T17:44:45.466 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-07-14T17:44:45.466 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-x1205.ko 2023-07-14T17:44:45.466 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi 2023-07-14T17:44:45.466 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-9xxx.ko 2023-07-14T17:44:45.466 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-sas.ko 2023-07-14T17:44:45.466 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-xxxx.ko 2023-07-14T17:44:45.467 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/BusLogic.ko 2023-07-14T17:44:45.467 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/a100u2w.ko 2023-07-14T17:44:45.467 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid 2023-07-14T17:44:45.467 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid/aacraid.ko 2023-07-14T17:44:45.467 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/advansys.ko 2023-07-14T17:44:45.467 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx 2023-07-14T17:44:45.467 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-07-14T17:44:45.467 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-07-14T17:44:45.468 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/am53c974.ko 2023-07-14T17:44:45.468 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr 2023-07-14T17:44:45.468 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-07-14T17:44:45.468 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/atp870u.ko 2023-07-14T17:44:45.468 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi 2023-07-14T17:44:45.468 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-07-14T17:44:45.468 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa 2023-07-14T17:44:45.468 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa/bfa.ko 2023-07-14T17:44:45.469 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc 2023-07-14T17:44:45.469 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-07-14T17:44:45.469 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i 2023-07-14T17:44:45.469 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-07-14T17:44:45.469 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ch.ko 2023-07-14T17:44:45.469 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor 2023-07-14T17:44:45.469 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor/csiostor.ko 2023-07-14T17:44:45.470 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi 2023-07-14T17:44:45.470 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i 2023-07-14T17:44:45.470 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-07-14T17:44:45.470 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i 2023-07-14T17:44:45.470 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-07-14T17:44:45.470 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-07-14T17:44:45.470 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dc395x.ko 2023-07-14T17:44:45.470 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler 2023-07-14T17:44:45.471 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-07-14T17:44:45.471 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-07-14T17:44:45.471 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-07-14T17:44:45.471 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-07-14T17:44:45.471 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dmx3191d.ko 2023-07-14T17:44:45.471 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r 2023-07-14T17:44:45.471 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r/esas2r.ko 2023-07-14T17:44:45.471 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esp_scsi.ko 2023-07-14T17:44:45.472 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe 2023-07-14T17:44:45.472 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/fcoe.ko 2023-07-14T17:44:45.472 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-07-14T17:44:45.472 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic 2023-07-14T17:44:45.472 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic/fnic.ko 2023-07-14T17:44:45.472 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hpsa.ko 2023-07-14T17:44:45.472 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hptiop.ko 2023-07-14T17:44:45.472 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hv_storvsc.ko 2023-07-14T17:44:45.473 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/initio.ko 2023-07-14T17:44:45.473 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ipr.ko 2023-07-14T17:44:45.473 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ips.ko 2023-07-14T17:44:45.473 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci 2023-07-14T17:44:45.473 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci/isci.ko 2023-07-14T17:44:45.473 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-07-14T17:44:45.473 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_tcp.ko 2023-07-14T17:44:45.474 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc 2023-07-14T17:44:45.474 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc/libfc.ko 2023-07-14T17:44:45.474 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi.ko 2023-07-14T17:44:45.474 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi_tcp.ko 2023-07-14T17:44:45.474 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas 2023-07-14T17:44:45.474 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas/libsas.ko 2023-07-14T17:44:45.474 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc 2023-07-14T17:44:45.474 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc/lpfc.ko 2023-07-14T17:44:45.475 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid 2023-07-14T17:44:45.475 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid.ko 2023-07-14T17:44:45.475 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-07-14T17:44:45.475 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-07-14T17:44:45.475 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-07-14T17:44:45.475 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas 2023-07-14T17:44:45.475 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-07-14T17:44:45.475 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas 2023-07-14T17:44:45.476 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas/mvsas.ko 2023-07-14T17:44:45.476 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvumi.ko 2023-07-14T17:44:45.476 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001 2023-07-14T17:44:45.476 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-07-14T17:44:45.476 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pmcraid.ko 2023-07-14T17:44:45.476 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla1280.ko 2023-07-14T17:44:45.476 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx 2023-07-14T17:44:45.476 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-07-14T17:44:45.477 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-07-14T17:44:45.477 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx 2023-07-14T17:44:45.477 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-07-14T17:44:45.477 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/raid_class.ko 2023-07-14T17:44:45.477 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_debug.ko 2023-07-14T17:44:45.477 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_fc.ko 2023-07-14T17:44:45.477 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-07-14T17:44:45.477 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_sas.ko 2023-07-14T17:44:45.478 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_spi.ko 2023-07-14T17:44:45.478 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_srp.ko 2023-07-14T17:44:45.478 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ses.ko 2023-07-14T17:44:45.478 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/st.ko 2023-07-14T17:44:45.478 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/stex.ko 2023-07-14T17:44:45.478 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2 2023-07-14T17:44:45.478 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-07-14T17:44:45.478 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/virtio_scsi.ko 2023-07-14T17:44:45.479 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/vmw_pvscsi.ko 2023-07-14T17:44:45.479 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/wd719x.ko 2023-07-14T17:44:45.479 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb 2023-07-14T17:44:45.479 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb/ssb.ko 2023-07-14T17:44:45.479 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target 2023-07-14T17:44:45.479 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi 2023-07-14T17:44:45.479 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-07-14T17:44:45.479 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback 2023-07-14T17:44:45.480 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback/tcm_loop.ko 2023-07-14T17:44:45.480 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_file.ko 2023-07-14T17:44:45.480 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_iblock.ko 2023-07-14T17:44:45.480 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_mod.ko 2023-07-14T17:44:45.480 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_pscsi.ko 2023-07-14T17:44:45.480 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_user.ko 2023-07-14T17:44:45.480 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc 2023-07-14T17:44:45.480 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-07-14T17:44:45.481 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal 2023-07-14T17:44:45.481 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel 2023-07-14T17:44:45.481 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal 2023-07-14T17:44:45.481 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-07-14T17:44:45.481 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-07-14T17:44:45.481 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-07-14T17:44:45.481 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-07-14T17:44:45.481 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-07-14T17:44:45.482 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-07-14T17:44:45.482 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-07-14T17:44:45.482 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-07-14T17:44:45.482 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-07-14T17:44:45.482 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-07-14T17:44:45.482 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-07-14T17:44:45.482 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-07-14T17:44:45.483 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-07-14T17:44:45.483 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-07-14T17:44:45.483 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-07-14T17:44:45.483 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty 2023-07-14T17:44:45.483 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless 2023-07-14T17:44:45.483 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-07-14T17:44:45.483 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_gsm.ko 2023-07-14T17:44:45.483 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_hdlc.ko 2023-07-14T17:44:45.484 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/nozomi.ko 2023-07-14T17:44:45.484 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial 2023-07-14T17:44:45.484 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250 2023-07-14T17:44:45.484 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-07-14T17:44:45.484 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/arc_uart.ko 2023-07-14T17:44:45.484 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm 2023-07-14T17:44:45.484 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm/jsm.ko 2023-07-14T17:44:45.484 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/synclink_gt.ko 2023-07-14T17:44:45.485 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs 2023-07-14T17:44:45.485 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core 2023-07-14T17:44:45.485 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core/ufshcd-core.ko 2023-07-14T17:44:45.485 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host 2023-07-14T17:44:45.485 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-07-14T17:44:45.485 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio 2023-07-14T17:44:45.485 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio.ko 2023-07-14T17:44:45.485 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_aec.ko 2023-07-14T17:44:45.486 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_cif.ko 2023-07-14T17:44:45.486 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_pci_generic.ko 2023-07-14T17:44:45.486 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_sercos3.ko 2023-07-14T17:44:45.486 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb 2023-07-14T17:44:45.486 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class 2023-07-14T17:44:45.486 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-acm.ko 2023-07-14T17:44:45.486 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-wdm.ko 2023-07-14T17:44:45.486 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usblp.ko 2023-07-14T17:44:45.487 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usbtmc.ko 2023-07-14T17:44:45.487 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host 2023-07-14T17:44:45.487 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/sl811-hcd.ko 2023-07-14T17:44:45.487 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image 2023-07-14T17:44:45.487 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/mdc800.ko 2023-07-14T17:44:45.487 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/microtek.ko 2023-07-14T17:44:45.487 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc 2023-07-14T17:44:45.488 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/adutux.ko 2023-07-14T17:44:45.488 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/appledisplay.ko 2023-07-14T17:44:45.488 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi26.ko 2023-07-14T17:44:45.488 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi62.ko 2023-07-14T17:44:45.488 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ezusb.ko 2023-07-14T17:44:45.488 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/idmouse.ko 2023-07-14T17:44:45.488 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/iowarrior.ko 2023-07-14T17:44:45.488 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/isight_firmware.ko 2023-07-14T17:44:45.489 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ldusb.ko 2023-07-14T17:44:45.489 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/legousbtower.ko 2023-07-14T17:44:45.489 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga 2023-07-14T17:44:45.489 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-07-14T17:44:45.489 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/trancevibrator.ko 2023-07-14T17:44:45.489 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usb3503.ko 2023-07-14T17:44:45.489 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usblcd.ko 2023-07-14T17:44:45.489 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usbsevseg.ko 2023-07-14T17:44:45.490 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/uss720.ko 2023-07-14T17:44:45.490 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/yurex.ko 2023-07-14T17:44:45.490 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy 2023-07-14T17:44:45.490 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy/phy-generic.ko 2023-07-14T17:44:45.490 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial 2023-07-14T17:44:45.490 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/aircable.ko 2023-07-14T17:44:45.490 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ark3116.ko 2023-07-14T17:44:45.490 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/belkin_sa.ko 2023-07-14T17:44:45.491 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ch341.ko 2023-07-14T17:44:45.491 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cp210x.ko 2023-07-14T17:44:45.491 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cyberjack.ko 2023-07-14T17:44:45.491 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cypress_m8.ko 2023-07-14T17:44:45.491 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/digi_acceleport.ko 2023-07-14T17:44:45.491 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/empeg.ko 2023-07-14T17:44:45.491 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ftdi_sio.ko 2023-07-14T17:44:45.491 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/garmin_gps.ko 2023-07-14T17:44:45.492 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_edgeport.ko 2023-07-14T17:44:45.492 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_ti.ko 2023-07-14T17:44:45.492 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipaq.ko 2023-07-14T17:44:45.492 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipw.ko 2023-07-14T17:44:45.492 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ir-usb.ko 2023-07-14T17:44:45.493 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-07-14T17:44:45.493 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan.ko 2023-07-14T17:44:45.493 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan_pda.ko 2023-07-14T17:44:45.493 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kl5kusb105.ko 2023-07-14T17:44:45.493 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kobil_sct.ko 2023-07-14T17:44:45.493 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mct_u232.ko 2023-07-14T17:44:45.493 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7720.ko 2023-07-14T17:44:45.493 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7840.ko 2023-07-14T17:44:45.494 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/navman.ko 2023-07-14T17:44:45.494 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/omninet.ko 2023-07-14T17:44:45.494 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/opticon.ko 2023-07-14T17:44:45.494 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/option.ko 2023-07-14T17:44:45.494 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/oti6858.ko 2023-07-14T17:44:45.494 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/pl2303.ko 2023-07-14T17:44:45.494 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcaux.ko 2023-07-14T17:44:45.494 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcserial.ko 2023-07-14T17:44:45.495 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/quatech2.ko 2023-07-14T17:44:45.495 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/safe_serial.ko 2023-07-14T17:44:45.495 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/sierra.ko 2023-07-14T17:44:45.495 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/spcp8x5.ko 2023-07-14T17:44:45.495 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ssu100.ko 2023-07-14T17:44:45.495 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/symbolserial.ko 2023-07-14T17:44:45.495 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-07-14T17:44:45.495 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-07-14T17:44:45.496 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_debug.ko 2023-07-14T17:44:45.496 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_wwan.ko 2023-07-14T17:44:45.496 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/visor.ko 2023-07-14T17:44:45.496 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/whiteheat.ko 2023-07-14T17:44:45.496 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/xsens_mt.ko 2023-07-14T17:44:45.496 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage 2023-07-14T17:44:45.496 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/uas.ko 2023-07-14T17:44:45.496 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-alauda.ko 2023-07-14T17:44:45.497 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-cypress.ko 2023-07-14T17:44:45.497 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-datafab.ko 2023-07-14T17:44:45.497 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-07-14T17:44:45.497 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-freecom.ko 2023-07-14T17:44:45.497 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-isd200.ko 2023-07-14T17:44:45.497 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-07-14T17:44:45.497 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-karma.ko 2023-07-14T17:44:45.498 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-onetouch.ko 2023-07-14T17:44:45.498 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-realtek.ko 2023-07-14T17:44:45.498 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr09.ko 2023-07-14T17:44:45.498 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr55.ko 2023-07-14T17:44:45.498 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-usbat.ko 2023-07-14T17:44:45.498 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/usb-storage.ko 2023-07-14T17:44:45.498 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip 2023-07-14T17:44:45.498 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-core.ko 2023-07-14T17:44:45.499 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-host.ko 2023-07-14T17:44:45.499 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-07-14T17:44:45.499 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio 2023-07-14T17:44:45.499 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci 2023-07-14T17:44:45.499 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-07-14T17:44:45.499 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci.ko 2023-07-14T17:44:45.499 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio.ko 2023-07-14T17:44:45.499 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-07-14T17:44:45.500 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost 2023-07-14T17:44:45.500 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost.ko 2023-07-14T17:44:45.500 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_iotlb.ko 2023-07-14T17:44:45.500 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_net.ko 2023-07-14T17:44:45.500 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_scsi.ko 2023-07-14T17:44:45.500 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video 2023-07-14T17:44:45.500 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight 2023-07-14T17:44:45.500 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/apple_bl.ko 2023-07-14T17:44:45.501 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/lcd.ko 2023-07-14T17:44:45.501 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/platform_lcd.ko 2023-07-14T17:44:45.501 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio 2023-07-14T17:44:45.501 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio.ko 2023-07-14T17:44:45.501 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_balloon.ko 2023-07-14T17:44:45.501 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_mmio.ko 2023-07-14T17:44:45.501 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci.ko 2023-07-14T17:44:45.502 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-07-14T17:44:45.502 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-07-14T17:44:45.502 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_ring.ko 2023-07-14T17:44:45.502 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1 2023-07-14T17:44:45.502 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters 2023-07-14T17:44:45.502 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2482.ko 2023-07-14T17:44:45.502 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2490.ko 2023-07-14T17:44:45.502 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves 2023-07-14T17:44:45.503 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-07-14T17:44:45.503 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-07-14T17:44:45.503 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-07-14T17:44:45.503 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-07-14T17:44:45.503 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-07-14T17:44:45.503 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-07-14T17:44:45.503 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-07-14T17:44:45.503 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-07-14T17:44:45.504 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-07-14T17:44:45.504 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_smem.ko 2023-07-14T17:44:45.504 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_therm.ko 2023-07-14T17:44:45.504 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/wire.ko 2023-07-14T17:44:45.504 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog 2023-07-14T17:44:45.504 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim1535_wdt.ko 2023-07-14T17:44:45.504 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim7101_wdt.ko 2023-07-14T17:44:45.504 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/f71808e_wdt.ko 2023-07-14T17:44:45.505 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/hpwdt.ko 2023-07-14T17:44:45.505 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/i6300esb.ko 2023-07-14T17:44:45.505 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-07-14T17:44:45.505 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_wdt.ko 2023-07-14T17:44:45.505 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ib700wdt.ko 2023-07-14T17:44:45.505 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ibmasr.ko 2023-07-14T17:44:45.505 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-07-14T17:44:45.505 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it8712f_wdt.ko 2023-07-14T17:44:45.506 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it87_wdt.ko 2023-07-14T17:44:45.506 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/machzwd.ko 2023-07-14T17:44:45.506 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/nv_tco.ko 2023-07-14T17:44:45.506 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_pci.ko 2023-07-14T17:44:45.506 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_usb.ko 2023-07-14T17:44:45.506 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-07-14T17:44:45.506 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sch311x_wdt.ko 2023-07-14T17:44:45.506 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/softdog.ko 2023-07-14T17:44:45.507 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sp5100_tco.ko 2023-07-14T17:44:45.507 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/via_wdt.ko 2023-07-14T17:44:45.507 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-07-14T17:44:45.507 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83877f_wdt.ko 2023-07-14T17:44:45.507 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83977f_wdt.ko 2023-07-14T17:44:45.507 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/wdt_pci.ko 2023-07-14T17:44:45.507 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/xen_wdt.ko 2023-07-14T17:44:45.507 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen 2023-07-14T17:44:45.508 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-acpi-processor.ko 2023-07-14T17:44:45.508 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-evtchn.ko 2023-07-14T17:44:45.508 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntalloc.ko 2023-07-14T17:44:45.508 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntdev.ko 2023-07-14T17:44:45.508 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback 2023-07-14T17:44:45.508 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-07-14T17:44:45.508 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-privcmd.ko 2023-07-14T17:44:45.509 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs 2023-07-14T17:44:45.509 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs/xenfs.ko 2023-07-14T17:44:45.509 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs 2023-07-14T17:44:45.509 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p 2023-07-14T17:44:45.509 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p/9p.ko 2023-07-14T17:44:45.509 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs 2023-07-14T17:44:45.509 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs/affs.ko 2023-07-14T17:44:45.509 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs 2023-07-14T17:44:45.510 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs/befs.ko 2023-07-14T17:44:45.510 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/binfmt_misc.ko 2023-07-14T17:44:45.510 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs 2023-07-14T17:44:45.510 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs/btrfs.ko 2023-07-14T17:44:45.510 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles 2023-07-14T17:44:45.510 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles/cachefiles.ko 2023-07-14T17:44:45.510 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph 2023-07-14T17:44:45.510 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph/ceph.ko 2023-07-14T17:44:45.511 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda 2023-07-14T17:44:45.511 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda/coda.ko 2023-07-14T17:44:45.511 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs 2023-07-14T17:44:45.511 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs/cramfs.ko 2023-07-14T17:44:45.511 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm 2023-07-14T17:44:45.511 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm/dlm.ko 2023-07-14T17:44:45.511 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ecryptfs 2023-07-14T17:44:45.511 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ecryptfs/ecryptfs.ko 2023-07-14T17:44:45.512 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs 2023-07-14T17:44:45.512 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs/f2fs.ko 2023-07-14T17:44:45.512 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat 2023-07-14T17:44:45.512 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/fat.ko 2023-07-14T17:44:45.512 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/msdos.ko 2023-07-14T17:44:45.512 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/vfat.ko 2023-07-14T17:44:45.512 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache 2023-07-14T17:44:45.512 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache/fscache.ko 2023-07-14T17:44:45.513 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse 2023-07-14T17:44:45.513 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/cuse.ko 2023-07-14T17:44:45.513 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/fuse.ko 2023-07-14T17:44:45.513 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2 2023-07-14T17:44:45.513 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2/gfs2.ko 2023-07-14T17:44:45.513 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs 2023-07-14T17:44:45.513 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs/hfs.ko 2023-07-14T17:44:45.513 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus 2023-07-14T17:44:45.514 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus/hfsplus.ko 2023-07-14T17:44:45.514 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs 2023-07-14T17:44:45.514 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs/isofs.ko 2023-07-14T17:44:45.514 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs 2023-07-14T17:44:45.514 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs/jfs.ko 2023-07-14T17:44:45.514 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd 2023-07-14T17:44:45.514 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd/lockd.ko 2023-07-14T17:44:45.514 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix 2023-07-14T17:44:45.515 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix/minix.ko 2023-07-14T17:44:45.515 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs 2023-07-14T17:44:45.515 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs/netfs.ko 2023-07-14T17:44:45.515 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs 2023-07-14T17:44:45.515 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout 2023-07-14T17:44:45.515 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-07-14T17:44:45.515 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout 2023-07-14T17:44:45.515 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-07-14T17:44:45.516 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout 2023-07-14T17:44:45.516 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-07-14T17:44:45.516 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfs.ko 2023-07-14T17:44:45.516 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv3.ko 2023-07-14T17:44:45.516 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv4.ko 2023-07-14T17:44:45.516 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common 2023-07-14T17:44:45.516 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/grace.ko 2023-07-14T17:44:45.517 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/nfs_acl.ko 2023-07-14T17:44:45.517 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd 2023-07-14T17:44:45.517 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd/nfsd.ko 2023-07-14T17:44:45.517 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2 2023-07-14T17:44:45.517 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2/nilfs2.ko 2023-07-14T17:44:45.517 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls 2023-07-14T17:44:45.517 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-celtic.ko 2023-07-14T17:44:45.517 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-centeuro.ko 2023-07-14T17:44:45.518 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-croatian.ko 2023-07-14T17:44:45.518 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-cyrillic.ko 2023-07-14T17:44:45.518 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-gaelic.ko 2023-07-14T17:44:45.518 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-greek.ko 2023-07-14T17:44:45.518 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-iceland.ko 2023-07-14T17:44:45.518 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-inuit.ko 2023-07-14T17:44:45.518 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-roman.ko 2023-07-14T17:44:45.518 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-romanian.ko 2023-07-14T17:44:45.519 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-turkish.ko 2023-07-14T17:44:45.519 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1250.ko 2023-07-14T17:44:45.519 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1251.ko 2023-07-14T17:44:45.519 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1255.ko 2023-07-14T17:44:45.519 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp737.ko 2023-07-14T17:44:45.519 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp775.ko 2023-07-14T17:44:45.519 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp850.ko 2023-07-14T17:44:45.519 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp852.ko 2023-07-14T17:44:45.520 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp855.ko 2023-07-14T17:44:45.520 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp857.ko 2023-07-14T17:44:45.520 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp860.ko 2023-07-14T17:44:45.520 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp861.ko 2023-07-14T17:44:45.520 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp862.ko 2023-07-14T17:44:45.520 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp863.ko 2023-07-14T17:44:45.520 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp864.ko 2023-07-14T17:44:45.520 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp865.ko 2023-07-14T17:44:45.521 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp866.ko 2023-07-14T17:44:45.521 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp869.ko 2023-07-14T17:44:45.521 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp874.ko 2023-07-14T17:44:45.521 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp932.ko 2023-07-14T17:44:45.521 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp936.ko 2023-07-14T17:44:45.521 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp949.ko 2023-07-14T17:44:45.521 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp950.ko 2023-07-14T17:44:45.521 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_euc-jp.ko 2023-07-14T17:44:45.522 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-1.ko 2023-07-14T17:44:45.522 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-13.ko 2023-07-14T17:44:45.522 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-14.ko 2023-07-14T17:44:45.522 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-15.ko 2023-07-14T17:44:45.522 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-2.ko 2023-07-14T17:44:45.522 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-3.ko 2023-07-14T17:44:45.522 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-4.ko 2023-07-14T17:44:45.522 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-5.ko 2023-07-14T17:44:45.523 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-6.ko 2023-07-14T17:44:45.523 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-7.ko 2023-07-14T17:44:45.523 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-9.ko 2023-07-14T17:44:45.523 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-r.ko 2023-07-14T17:44:45.523 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-ru.ko 2023-07-14T17:44:45.523 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-u.ko 2023-07-14T17:44:45.523 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_utf8.ko 2023-07-14T17:44:45.523 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2 2023-07-14T17:44:45.524 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster 2023-07-14T17:44:45.524 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-07-14T17:44:45.524 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm 2023-07-14T17:44:45.524 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-07-14T17:44:45.524 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs 2023-07-14T17:44:45.524 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-07-14T17:44:45.524 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2.ko 2023-07-14T17:44:45.524 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-07-14T17:44:45.525 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-07-14T17:44:45.525 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-07-14T17:44:45.525 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs 2023-07-14T17:44:45.525 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs/overlay.ko 2023-07-14T17:44:45.525 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore 2023-07-14T17:44:45.525 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore/ramoops.ko 2023-07-14T17:44:45.525 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs 2023-07-14T17:44:45.526 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs/reiserfs.ko 2023-07-14T17:44:45.526 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs 2023-07-14T17:44:45.526 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs/romfs.ko 2023-07-14T17:44:45.526 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb 2023-07-14T17:44:45.526 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client 2023-07-14T17:44:45.526 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client/cifs.ko 2023-07-14T17:44:45.526 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common 2023-07-14T17:44:45.526 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_arc4.ko 2023-07-14T17:44:45.527 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_md4.ko 2023-07-14T17:44:45.527 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs 2023-07-14T17:44:45.527 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs/squashfs.ko 2023-07-14T17:44:45.527 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv 2023-07-14T17:44:45.527 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv/sysv.ko 2023-07-14T17:44:45.527 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf 2023-07-14T17:44:45.527 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf/udf.ko 2023-07-14T17:44:45.527 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs 2023-07-14T17:44:45.528 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs/ufs.ko 2023-07-14T17:44:45.528 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs 2023-07-14T17:44:45.528 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs/xfs.ko 2023-07-14T17:44:45.528 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel 2023-07-14T17:44:45.528 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/rcu 2023-07-14T17:44:45.528 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/rcu/rcutorture.ko 2023-07-14T17:44:45.528 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/torture.ko 2023-07-14T17:44:45.528 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/trace 2023-07-14T17:44:45.529 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-07-14T17:44:45.529 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib 2023-07-14T17:44:45.529 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/asn1_encoder.ko 2023-07-14T17:44:45.529 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc-itu-t.ko 2023-07-14T17:44:45.529 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc8.ko 2023-07-14T17:44:45.529 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto 2023-07-14T17:44:45.529 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libarc4.ko 2023-07-14T17:44:45.529 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libdes.ko 2023-07-14T17:44:45.530 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/libcrc32c.ko 2023-07-14T17:44:45.530 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lru_cache.ko 2023-07-14T17:44:45.530 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4 2023-07-14T17:44:45.530 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4_compress.ko 2023-07-14T17:44:45.530 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4hc_compress.ko 2023-07-14T17:44:45.530 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math 2023-07-14T17:44:45.530 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math/cordic.ko 2023-07-14T17:44:45.530 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6 2023-07-14T17:44:45.531 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6/raid6_pq.ko 2023-07-14T17:44:45.531 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon 2023-07-14T17:44:45.531 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon/reed_solomon.ko 2023-07-14T17:44:45.531 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_bm.ko 2023-07-14T17:44:45.531 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_fsm.ko 2023-07-14T17:44:45.531 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_kmp.ko 2023-07-14T17:44:45.531 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/zstd 2023-07-14T17:44:45.531 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/zstd/zstd_compress.ko 2023-07-14T17:44:45.532 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm 2023-07-14T17:44:45.532 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm/hwpoison-inject.ko 2023-07-14T17:44:45.532 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net 2023-07-14T17:44:45.532 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan 2023-07-14T17:44:45.532 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/6lowpan.ko 2023-07-14T17:44:45.532 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_dest.ko 2023-07-14T17:44:45.532 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_fragment.ko 2023-07-14T17:44:45.533 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_hop.ko 2023-07-14T17:44:45.533 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_ipv6.ko 2023-07-14T17:44:45.533 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_mobility.ko 2023-07-14T17:44:45.533 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_routing.ko 2023-07-14T17:44:45.533 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_udp.ko 2023-07-14T17:44:45.533 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802 2023-07-14T17:44:45.533 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/garp.ko 2023-07-14T17:44:45.533 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/mrp.ko 2023-07-14T17:44:45.534 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/p8022.ko 2023-07-14T17:44:45.534 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/psnap.ko 2023-07-14T17:44:45.534 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/stp.ko 2023-07-14T17:44:45.534 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q 2023-07-14T17:44:45.534 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q/8021q.ko 2023-07-14T17:44:45.534 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p 2023-07-14T17:44:45.534 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet.ko 2023-07-14T17:44:45.534 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_fd.ko 2023-07-14T17:44:45.535 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_rdma.ko 2023-07-14T17:44:45.535 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_virtio.ko 2023-07-14T17:44:45.535 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv 2023-07-14T17:44:45.535 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv/batman-adv.ko 2023-07-14T17:44:45.535 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge 2023-07-14T17:44:45.535 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/br_netfilter.ko 2023-07-14T17:44:45.535 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/bridge.ko 2023-07-14T17:44:45.535 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter 2023-07-14T17:44:45.536 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-07-14T17:44:45.536 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_among.ko 2023-07-14T17:44:45.536 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arp.ko 2023-07-14T17:44:45.536 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-07-14T17:44:45.536 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-07-14T17:44:45.536 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip.ko 2023-07-14T17:44:45.536 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-07-14T17:44:45.536 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_limit.ko 2023-07-14T17:44:45.537 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_log.ko 2023-07-14T17:44:45.537 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark.ko 2023-07-14T17:44:45.537 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-07-14T17:44:45.537 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-07-14T17:44:45.537 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-07-14T17:44:45.537 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-07-14T17:44:45.537 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_snat.ko 2023-07-14T17:44:45.537 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_stp.ko 2023-07-14T17:44:45.538 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-07-14T17:44:45.538 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-07-14T17:44:45.538 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-07-14T17:44:45.538 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-07-14T17:44:45.538 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtables.ko 2023-07-14T17:44:45.538 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-07-14T17:44:45.538 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-07-14T17:44:45.538 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-07-14T17:44:45.539 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph 2023-07-14T17:44:45.539 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph/libceph.ko 2023-07-14T17:44:45.539 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core 2023-07-14T17:44:45.539 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/failover.ko 2023-07-14T17:44:45.539 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/pktgen.ko 2023-07-14T17:44:45.539 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp 2023-07-14T17:44:45.539 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp.ko 2023-07-14T17:44:45.539 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_diag.ko 2023-07-14T17:44:45.540 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv4.ko 2023-07-14T17:44:45.540 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv6.ko 2023-07-14T17:44:45.540 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dns_resolver 2023-07-14T17:44:45.540 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dns_resolver/dns_resolver.ko 2023-07-14T17:44:45.540 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa 2023-07-14T17:44:45.540 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/dsa_core.ko 2023-07-14T17:44:45.540 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_brcm.ko 2023-07-14T17:44:45.541 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_dsa.ko 2023-07-14T17:44:45.541 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_none.ko 2023-07-14T17:44:45.541 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_trailer.ko 2023-07-14T17:44:45.541 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154 2023-07-14T17:44:45.541 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan 2023-07-14T17:44:45.541 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-07-14T17:44:45.541 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154.ko 2023-07-14T17:44:45.541 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154_socket.ko 2023-07-14T17:44:45.542 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4 2023-07-14T17:44:45.542 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ah4.ko 2023-07-14T17:44:45.542 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/esp4.ko 2023-07-14T17:44:45.542 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/fou.ko 2023-07-14T17:44:45.542 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/gre.ko 2023-07-14T17:44:45.542 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/inet_diag.ko 2023-07-14T17:44:45.542 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_gre.ko 2023-07-14T17:44:45.542 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_tunnel.ko 2023-07-14T17:44:45.543 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_vti.ko 2023-07-14T17:44:45.543 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipcomp.ko 2023-07-14T17:44:45.543 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipip.ko 2023-07-14T17:44:45.543 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter 2023-07-14T17:44:45.543 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arp_tables.ko 2023-07-14T17:44:45.543 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-07-14T17:44:45.543 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-07-14T17:44:45.543 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-07-14T17:44:45.544 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-07-14T17:44:45.544 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-07-14T17:44:45.544 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-07-14T17:44:45.544 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-07-14T17:44:45.544 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-07-14T17:44:45.544 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-07-14T17:44:45.544 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_security.ko 2023-07-14T17:44:45.544 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-07-14T17:44:45.545 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-07-14T17:44:45.545 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-07-14T17:44:45.545 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-07-14T17:44:45.545 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-07-14T17:44:45.545 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-07-14T17:44:45.545 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-07-14T17:44:45.545 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-07-14T17:44:45.545 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-07-14T17:44:45.546 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-07-14T17:44:45.546 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_bic.ko 2023-07-14T17:44:45.546 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_dctcp.ko 2023-07-14T17:44:45.546 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_diag.ko 2023-07-14T17:44:45.546 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_highspeed.ko 2023-07-14T17:44:45.546 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_htcp.ko 2023-07-14T17:44:45.546 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_hybla.ko 2023-07-14T17:44:45.547 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_illinois.ko 2023-07-14T17:44:45.547 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_lp.ko 2023-07-14T17:44:45.547 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_scalable.ko 2023-07-14T17:44:45.547 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_vegas.ko 2023-07-14T17:44:45.547 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_veno.ko 2023-07-14T17:44:45.547 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_westwood.ko 2023-07-14T17:44:45.547 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_yeah.ko 2023-07-14T17:44:45.547 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tunnel4.ko 2023-07-14T17:44:45.548 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_diag.ko 2023-07-14T17:44:45.548 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_tunnel.ko 2023-07-14T17:44:45.548 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/xfrm4_tunnel.ko 2023-07-14T17:44:45.548 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6 2023-07-14T17:44:45.548 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ah6.ko 2023-07-14T17:44:45.548 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/esp6.ko 2023-07-14T17:44:45.548 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/fou6.ko 2023-07-14T17:44:45.548 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_tunnel.ko 2023-07-14T17:44:45.549 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-07-14T17:44:45.549 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_vti.ko 2023-07-14T17:44:45.549 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ipcomp6.ko 2023-07-14T17:44:45.549 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter 2023-07-14T17:44:45.549 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-07-14T17:44:45.549 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-07-14T17:44:45.549 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-07-14T17:44:45.549 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-07-14T17:44:45.550 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-07-14T17:44:45.550 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-07-14T17:44:45.550 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-07-14T17:44:45.550 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-07-14T17:44:45.550 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-07-14T17:44:45.550 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-07-14T17:44:45.550 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-07-14T17:44:45.550 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-07-14T17:44:45.551 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-07-14T17:44:45.551 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-07-14T17:44:45.551 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-07-14T17:44:45.551 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-07-14T17:44:45.551 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-07-14T17:44:45.551 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-07-14T17:44:45.551 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-07-14T17:44:45.551 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-07-14T17:44:45.552 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-07-14T17:44:45.552 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-07-14T17:44:45.552 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-07-14T17:44:45.552 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-07-14T17:44:45.552 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/sit.ko 2023-07-14T17:44:45.552 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/tunnel6.ko 2023-07-14T17:44:45.552 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/xfrm6_tunnel.ko 2023-07-14T17:44:45.552 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key 2023-07-14T17:44:45.553 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key/af_key.ko 2023-07-14T17:44:45.553 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp 2023-07-14T17:44:45.553 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_core.ko 2023-07-14T17:44:45.553 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_debugfs.ko 2023-07-14T17:44:45.553 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_eth.ko 2023-07-14T17:44:45.553 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip.ko 2023-07-14T17:44:45.553 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip6.ko 2023-07-14T17:44:45.553 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_netlink.ko 2023-07-14T17:44:45.554 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ppp.ko 2023-07-14T17:44:45.554 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc 2023-07-14T17:44:45.554 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc/llc.ko 2023-07-14T17:44:45.554 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154 2023-07-14T17:44:45.554 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154/mac802154.ko 2023-07-14T17:44:45.554 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls 2023-07-14T17:44:45.554 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls/mpls_gso.ko 2023-07-14T17:44:45.555 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter 2023-07-14T17:44:45.555 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset 2023-07-14T17:44:45.555 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set.ko 2023-07-14T17:44:45.555 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-07-14T17:44:45.555 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-07-14T17:44:45.555 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-07-14T17:44:45.555 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-07-14T17:44:45.555 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-07-14T17:44:45.556 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-07-14T17:44:45.556 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-07-14T17:44:45.556 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-07-14T17:44:45.556 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-07-14T17:44:45.556 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-07-14T17:44:45.556 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-07-14T17:44:45.556 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-07-14T17:44:45.556 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-07-14T17:44:45.557 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-07-14T17:44:45.557 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-07-14T17:44:45.557 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs 2023-07-14T17:44:45.557 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs.ko 2023-07-14T17:44:45.557 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-07-14T17:44:45.557 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-07-14T17:44:45.557 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-07-14T17:44:45.557 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-07-14T17:44:45.558 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-07-14T17:44:45.558 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-07-14T17:44:45.558 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-07-14T17:44:45.558 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-07-14T17:44:45.558 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-07-14T17:44:45.558 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-07-14T17:44:45.558 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-07-14T17:44:45.558 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-07-14T17:44:45.559 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-07-14T17:44:45.559 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conncount.ko 2023-07-14T17:44:45.559 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack.ko 2023-07-14T17:44:45.559 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-07-14T17:44:45.559 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-07-14T17:44:45.559 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-07-14T17:44:45.559 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_h323.ko 2023-07-14T17:44:45.559 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_irc.ko 2023-07-14T17:44:45.560 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-07-14T17:44:45.560 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-07-14T17:44:45.560 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-07-14T17:44:45.560 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sane.ko 2023-07-14T17:44:45.560 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sip.ko 2023-07-14T17:44:45.560 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-07-14T17:44:45.560 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-07-14T17:44:45.560 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_dup_netdev.ko 2023-07-14T17:44:45.561 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table.ko 2023-07-14T17:44:45.561 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table_inet.ko 2023-07-14T17:44:45.561 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_log_syslog.ko 2023-07-14T17:44:45.561 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat.ko 2023-07-14T17:44:45.561 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_amanda.ko 2023-07-14T17:44:45.561 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_ftp.ko 2023-07-14T17:44:45.561 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_irc.ko 2023-07-14T17:44:45.561 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_sip.ko 2023-07-14T17:44:45.562 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_tftp.ko 2023-07-14T17:44:45.562 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_synproxy_core.ko 2023-07-14T17:44:45.562 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_tables.ko 2023-07-14T17:44:45.562 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink.ko 2023-07-14T17:44:45.562 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_acct.ko 2023-07-14T17:44:45.562 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_log.ko 2023-07-14T17:44:45.562 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_osf.ko 2023-07-14T17:44:45.562 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_queue.ko 2023-07-14T17:44:45.563 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_chain_nat.ko 2023-07-14T17:44:45.563 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_compat.ko 2023-07-14T17:44:45.563 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_ct.ko 2023-07-14T17:44:45.563 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_dup_netdev.ko 2023-07-14T17:44:45.563 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fib.ko 2023-07-14T17:44:45.563 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fwd_netdev.ko 2023-07-14T17:44:45.563 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_hash.ko 2023-07-14T17:44:45.563 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_limit.ko 2023-07-14T17:44:45.564 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_log.ko 2023-07-14T17:44:45.564 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_masq.ko 2023-07-14T17:44:45.564 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_nat.ko 2023-07-14T17:44:45.564 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_numgen.ko 2023-07-14T17:44:45.564 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_queue.ko 2023-07-14T17:44:45.564 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_quota.ko 2023-07-14T17:44:45.564 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_redir.ko 2023-07-14T17:44:45.564 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject.ko 2023-07-14T17:44:45.565 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject_inet.ko 2023-07-14T17:44:45.565 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_socket.ko 2023-07-14T17:44:45.565 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_tunnel.ko 2023-07-14T17:44:45.565 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_AUDIT.ko 2023-07-14T17:44:45.565 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CHECKSUM.ko 2023-07-14T17:44:45.565 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CLASSIFY.ko 2023-07-14T17:44:45.565 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-07-14T17:44:45.565 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CT.ko 2023-07-14T17:44:45.566 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_DSCP.ko 2023-07-14T17:44:45.566 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HL.ko 2023-07-14T17:44:45.566 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HMARK.ko 2023-07-14T17:44:45.566 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_IDLETIMER.ko 2023-07-14T17:44:45.566 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LED.ko 2023-07-14T17:44:45.566 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LOG.ko 2023-07-14T17:44:45.566 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_MASQUERADE.ko 2023-07-14T17:44:45.566 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NETMAP.ko 2023-07-14T17:44:45.567 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFLOG.ko 2023-07-14T17:44:45.567 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFQUEUE.ko 2023-07-14T17:44:45.567 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_RATEEST.ko 2023-07-14T17:44:45.567 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_REDIRECT.ko 2023-07-14T17:44:45.567 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_SECMARK.ko 2023-07-14T17:44:45.567 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPMSS.ko 2023-07-14T17:44:45.567 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-07-14T17:44:45.567 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TEE.ko 2023-07-14T17:44:45.568 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TPROXY.ko 2023-07-14T17:44:45.568 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TRACE.ko 2023-07-14T17:44:45.568 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_addrtype.ko 2023-07-14T17:44:45.568 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_bpf.ko 2023-07-14T17:44:45.568 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cgroup.ko 2023-07-14T17:44:45.568 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cluster.ko 2023-07-14T17:44:45.568 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_comment.ko 2023-07-14T17:44:45.568 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connbytes.ko 2023-07-14T17:44:45.569 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlabel.ko 2023-07-14T17:44:45.569 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlimit.ko 2023-07-14T17:44:45.569 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connmark.ko 2023-07-14T17:44:45.569 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_conntrack.ko 2023-07-14T17:44:45.569 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cpu.ko 2023-07-14T17:44:45.569 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dccp.ko 2023-07-14T17:44:45.569 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_devgroup.ko 2023-07-14T17:44:45.570 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dscp.ko 2023-07-14T17:44:45.570 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ecn.ko 2023-07-14T17:44:45.570 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_esp.ko 2023-07-14T17:44:45.570 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hashlimit.ko 2023-07-14T17:44:45.570 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_helper.ko 2023-07-14T17:44:45.570 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hl.ko 2023-07-14T17:44:45.570 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipcomp.ko 2023-07-14T17:44:45.570 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_iprange.ko 2023-07-14T17:44:45.571 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipvs.ko 2023-07-14T17:44:45.571 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_l2tp.ko 2023-07-14T17:44:45.571 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_length.ko 2023-07-14T17:44:45.571 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_limit.ko 2023-07-14T17:44:45.571 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mac.ko 2023-07-14T17:44:45.571 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mark.ko 2023-07-14T17:44:45.571 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_multiport.ko 2023-07-14T17:44:45.571 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nat.ko 2023-07-14T17:44:45.572 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nfacct.ko 2023-07-14T17:44:45.572 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_osf.ko 2023-07-14T17:44:45.572 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_owner.ko 2023-07-14T17:44:45.572 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_physdev.ko 2023-07-14T17:44:45.572 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_pkttype.ko 2023-07-14T17:44:45.572 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_policy.ko 2023-07-14T17:44:45.572 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_quota.ko 2023-07-14T17:44:45.572 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_rateest.ko 2023-07-14T17:44:45.573 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_realm.ko 2023-07-14T17:44:45.573 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_recent.ko 2023-07-14T17:44:45.573 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_sctp.ko 2023-07-14T17:44:45.573 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_set.ko 2023-07-14T17:44:45.573 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_socket.ko 2023-07-14T17:44:45.573 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_state.ko 2023-07-14T17:44:45.573 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_statistic.ko 2023-07-14T17:44:45.573 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_string.ko 2023-07-14T17:44:45.574 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_tcpmss.ko 2023-07-14T17:44:45.574 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_time.ko 2023-07-14T17:44:45.574 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_u32.ko 2023-07-14T17:44:45.574 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink 2023-07-14T17:44:45.574 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink/netlink_diag.ko 2023-07-14T17:44:45.574 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh 2023-07-14T17:44:45.574 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh/nsh.ko 2023-07-14T17:44:45.574 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch 2023-07-14T17:44:45.575 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/openvswitch.ko 2023-07-14T17:44:45.575 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-geneve.ko 2023-07-14T17:44:45.575 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-gre.ko 2023-07-14T17:44:45.575 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-vxlan.ko 2023-07-14T17:44:45.575 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet 2023-07-14T17:44:45.575 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet/af_packet_diag.ko 2023-07-14T17:44:45.575 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds 2023-07-14T17:44:45.575 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds.ko 2023-07-14T17:44:45.576 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_rdma.ko 2023-07-14T17:44:45.576 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_tcp.ko 2023-07-14T17:44:45.576 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched 2023-07-14T17:44:45.576 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_csum.ko 2023-07-14T17:44:45.576 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_gact.ko 2023-07-14T17:44:45.576 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_ipt.ko 2023-07-14T17:44:45.576 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_mirred.ko 2023-07-14T17:44:45.576 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_nat.ko 2023-07-14T17:44:45.577 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_pedit.ko 2023-07-14T17:44:45.577 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_police.ko 2023-07-14T17:44:45.577 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_simple.ko 2023-07-14T17:44:45.577 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_skbedit.ko 2023-07-14T17:44:45.577 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_vlan.ko 2023-07-14T17:44:45.577 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_basic.ko 2023-07-14T17:44:45.577 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_bpf.ko 2023-07-14T17:44:45.577 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_flow.ko 2023-07-14T17:44:45.578 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_fw.ko 2023-07-14T17:44:45.578 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_route.ko 2023-07-14T17:44:45.578 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_u32.ko 2023-07-14T17:44:45.578 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_cmp.ko 2023-07-14T17:44:45.578 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_ipset.ko 2023-07-14T17:44:45.578 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_meta.ko 2023-07-14T17:44:45.578 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_nbyte.ko 2023-07-14T17:44:45.579 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_text.ko 2023-07-14T17:44:45.579 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_u32.ko 2023-07-14T17:44:45.579 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_choke.ko 2023-07-14T17:44:45.579 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_codel.ko 2023-07-14T17:44:45.579 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_drr.ko 2023-07-14T17:44:45.579 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_fq.ko 2023-07-14T17:44:45.579 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_gred.ko 2023-07-14T17:44:45.579 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hfsc.ko 2023-07-14T17:44:45.580 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hhf.ko 2023-07-14T17:44:45.580 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_htb.ko 2023-07-14T17:44:45.580 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_ingress.ko 2023-07-14T17:44:45.580 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio.ko 2023-07-14T17:44:45.580 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio_lib.ko 2023-07-14T17:44:45.580 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_multiq.ko 2023-07-14T17:44:45.580 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_netem.ko 2023-07-14T17:44:45.580 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_pie.ko 2023-07-14T17:44:45.581 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_plug.ko 2023-07-14T17:44:45.581 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_prio.ko 2023-07-14T17:44:45.581 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_qfq.ko 2023-07-14T17:44:45.581 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_red.ko 2023-07-14T17:44:45.581 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfb.ko 2023-07-14T17:44:45.581 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfq.ko 2023-07-14T17:44:45.581 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_tbf.ko 2023-07-14T17:44:45.581 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_teql.ko 2023-07-14T17:44:45.582 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp 2023-07-14T17:44:45.582 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp.ko 2023-07-14T17:44:45.582 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp_diag.ko 2023-07-14T17:44:45.582 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc 2023-07-14T17:44:45.582 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss 2023-07-14T17:44:45.582 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-07-14T17:44:45.582 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-07-14T17:44:45.582 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/sunrpc.ko 2023-07-14T17:44:45.583 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma 2023-07-14T17:44:45.583 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-07-14T17:44:45.583 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc 2023-07-14T17:44:45.583 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/diag.ko 2023-07-14T17:44:45.583 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/tipc.ko 2023-07-14T17:44:45.583 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix 2023-07-14T17:44:45.583 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix/unix_diag.ko 2023-07-14T17:44:45.583 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock 2023-07-14T17:44:45.584 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-07-14T17:44:45.584 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-07-14T17:44:45.584 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock.ko 2023-07-14T17:44:45.584 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_diag.ko 2023-07-14T17:44:45.584 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_loopback.ko 2023-07-14T17:44:45.584 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm 2023-07-14T17:44:45.584 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm/xfrm_ipcomp.ko 2023-07-14T17:44:45.585 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security 2023-07-14T17:44:45.585 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys 2023-07-14T17:44:45.585 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/encrypted-keys 2023-07-14T17:44:45.585 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-07-14T17:44:45.585 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/trusted-keys 2023-07-14T17:44:45.585 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/trusted-keys/trusted.ko 2023-07-14T17:44:45.585 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt 2023-07-14T17:44:45.585 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib 2023-07-14T17:44:45.586 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib/irqbypass.ko 2023-07-14T17:44:45.586 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias 2023-07-14T17:44:45.586 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias.bin 2023-07-14T17:44:45.586 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin 2023-07-14T17:44:45.586 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.bin 2023-07-14T17:44:45.586 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.modinfo 2023-07-14T17:44:45.586 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep 2023-07-14T17:44:45.586 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep.bin 2023-07-14T17:44:45.587 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.devname 2023-07-14T17:44:45.587 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.order 2023-07-14T17:44:45.587 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.softdep 2023-07-14T17:44:45.587 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols 2023-07-14T17:44:45.587 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols.bin 2023-07-14T17:44:45.588 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc1-ga440c79eee02 2023-07-14T17:44:45.588 DEBUG:teuthology.orchestra.run.smithi133:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-07-14T17:44:45.614 INFO:teuthology.orchestra.run.smithi133.stdout:postinstall scriptlet (using /bin/sh): 2023-07-14T17:44:45.614 INFO:teuthology.orchestra.run.smithi133.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 -a -r /boot/System.map-6.5.0-rc1-ga440c79eee02 ]; then 2023-07-14T17:44:45.614 INFO:teuthology.orchestra.run.smithi133.stdout:cp /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm 2023-07-14T17:44:45.615 INFO:teuthology.orchestra.run.smithi133.stdout:cp /boot/System.map-6.5.0-rc1-ga440c79eee02 /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-14T17:44:45.615 INFO:teuthology.orchestra.run.smithi133.stdout:rm -f /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 /boot/System.map-6.5.0-rc1-ga440c79eee02 2023-07-14T17:44:45.615 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/installkernel 6.5.0-rc1-ga440c79eee02 /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-14T17:44:45.615 INFO:teuthology.orchestra.run.smithi133.stdout:rm -f /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-14T17:44:45.615 INFO:teuthology.orchestra.run.smithi133.stdout:fi 2023-07-14T17:44:45.615 INFO:teuthology.orchestra.run.smithi133.stdout:preuninstall scriptlet (using /bin/sh): 2023-07-14T17:44:45.615 INFO:teuthology.orchestra.run.smithi133.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-07-14T17:44:45.616 INFO:teuthology.orchestra.run.smithi133.stdout:new-kernel-pkg --remove 6.5.0-rc1-ga440c79eee02 --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc1-ga440c79eee02.img 2023-07-14T17:44:45.616 INFO:teuthology.orchestra.run.smithi133.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-07-14T17:44:45.616 INFO:teuthology.orchestra.run.smithi133.stdout:kernel-install remove 6.5.0-rc1-ga440c79eee02 2023-07-14T17:44:45.616 INFO:teuthology.orchestra.run.smithi133.stdout:fi 2023-07-14T17:44:45.616 INFO:teuthology.orchestra.run.smithi133.stdout:postuninstall scriptlet (using /bin/sh): 2023-07-14T17:44:45.616 INFO:teuthology.orchestra.run.smithi133.stdout:if [ -x /sbin/update-bootloader ]; then 2023-07-14T17:44:45.616 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/update-bootloader --remove 6.5.0-rc1-ga440c79eee02 2023-07-14T17:44:45.616 INFO:teuthology.orchestra.run.smithi133.stdout:fi 2023-07-14T17:44:45.617 DEBUG:teuthology.orchestra.run.smithi133:> sudo rpm -qi grub2-tools 2023-07-14T17:44:45.696 INFO:teuthology.orchestra.run.smithi133.stdout:Name : grub2-tools 2023-07-14T17:44:45.696 INFO:teuthology.orchestra.run.smithi133.stdout:Epoch : 1 2023-07-14T17:44:45.696 INFO:teuthology.orchestra.run.smithi133.stdout:Version : 2.02 2023-07-14T17:44:45.697 INFO:teuthology.orchestra.run.smithi133.stdout:Release : 123.el8 2023-07-14T17:44:45.697 INFO:teuthology.orchestra.run.smithi133.stdout:Architecture: x86_64 2023-07-14T17:44:45.697 INFO:teuthology.orchestra.run.smithi133.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-07-14T17:44:45.697 INFO:teuthology.orchestra.run.smithi133.stdout:Group : System Environment/Base 2023-07-14T17:44:45.698 INFO:teuthology.orchestra.run.smithi133.stdout:Size : 9265450 2023-07-14T17:44:45.698 INFO:teuthology.orchestra.run.smithi133.stdout:License : GPLv3+ 2023-07-14T17:44:45.698 INFO:teuthology.orchestra.run.smithi133.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-07-14T17:44:45.698 INFO:teuthology.orchestra.run.smithi133.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-07-14T17:44:45.698 INFO:teuthology.orchestra.run.smithi133.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-07-14T17:44:45.699 INFO:teuthology.orchestra.run.smithi133.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-07-14T17:44:45.699 INFO:teuthology.orchestra.run.smithi133.stdout:Relocations : (not relocatable) 2023-07-14T17:44:45.699 INFO:teuthology.orchestra.run.smithi133.stdout:Packager : Red Hat, Inc. 2023-07-14T17:44:45.699 INFO:teuthology.orchestra.run.smithi133.stdout:Vendor : Red Hat, Inc. 2023-07-14T17:44:45.699 INFO:teuthology.orchestra.run.smithi133.stdout:URL : http://www.gnu.org/software/grub/ 2023-07-14T17:44:45.699 INFO:teuthology.orchestra.run.smithi133.stdout:Summary : Support tools for GRUB. 2023-07-14T17:44:45.699 INFO:teuthology.orchestra.run.smithi133.stdout:Description : 2023-07-14T17:44:45.699 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T17:44:45.700 INFO:teuthology.orchestra.run.smithi133.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-07-14T17:44:45.700 INFO:teuthology.orchestra.run.smithi133.stdout:customizable bootloader with modular architecture. It supports a rich 2023-07-14T17:44:45.700 INFO:teuthology.orchestra.run.smithi133.stdout:variety of kernel formats, file systems, computer architectures and 2023-07-14T17:44:45.700 INFO:teuthology.orchestra.run.smithi133.stdout:hardware devices. 2023-07-14T17:44:45.700 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T17:44:45.701 INFO:teuthology.orchestra.run.smithi133.stdout:This subpackage provides tools for support of all platforms. 2023-07-14T17:44:45.702 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-07-14T17:44:45.703 INFO:teuthology.task.kernel:Updating grub on smithi133 to boot 6.5.0-rc1-ga440c79eee02 2023-07-14T17:44:45.703 DEBUG:teuthology.orchestra.run.smithi133:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-07-14T17:44:50.416 INFO:teuthology.orchestra.run.smithi133.stderr:Generating grub configuration file ... 2023-07-14T17:44:52.363 INFO:teuthology.orchestra.run.smithi133.stderr:done 2023-07-14T17:44:52.365 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2023-07-14T17:44:52.381 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.dYHKrTZ2qM 2023-07-14T17:44:52.382 DEBUG:teuthology.orchestra.run.smithi133:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.dYHKrTZ2qM 2023-07-14T17:44:52.471 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.dYHKrTZ2qM 2023-07-14T17:44:52.584 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.dYHKrTZ2qM is 6KB 2023-07-14T17:44:52.631 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.dYHKrTZ2qM 2023-07-14T17:44:52.648 DEBUG:teuthology.orchestra.run.smithi133:> sudo /bin/ls /boot/loader/entries || true 2023-07-14T17:44:52.722 INFO:teuthology.orchestra.run.smithi133.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-07-14T17:44:52.722 INFO:teuthology.orchestra.run.smithi133.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-07-14T17:44:52.722 INFO:teuthology.orchestra.run.smithi133.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-ga440c79eee02.conf 2023-07-14T17:44:52.723 DEBUG:teuthology.orchestra.run.smithi133:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-ga440c79eee02 2023-07-14T17:44:52.926 DEBUG:teuthology.orchestra.run.smithi133:> sudo shutdown -r now 2023-07-14T17:45:22.929 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-07-14T17:45:22.929 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2023-07-14T17:45:22.930 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T17:45:41.488 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2023-07-14T17:45:50.490 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2023-07-14T17:45:50.490 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T17:46:55.494 DEBUG:teuthology.orchestra.remote:timed out 2023-07-14T17:47:07.496 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2023-07-14T17:47:07.497 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T17:47:12.852 DEBUG:teuthology.orchestra.run.smithi133:> true 2023-07-14T17:47:13.270 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2023-07-14T17:47:13.271 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.5.0-rc1-ga440c79eee02"... 2023-07-14T17:47:13.271 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2023-07-14T17:47:13.314 INFO:teuthology.orchestra.run.smithi133.stdout:6.5.0-rc1-ga440c79eee02 2023-07-14T17:47:13.314 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc1-ga440c79eee02 vs 6.5.0-rc1-ga440c79eee02 2023-07-14T17:47:13.314 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-07-14T17:47:13.315 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-07-14T17:47:14.315 DEBUG:teuthology.parallel:result is None 2023-07-14T17:47:14.316 INFO:teuthology.run_tasks:Running task internal.base... 2023-07-14T17:47:14.324 INFO:teuthology.task.internal:Creating test directory... 2023-07-14T17:47:14.324 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-14T17:47:14.329 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-14T17:47:14.332 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-14T17:47:14.371 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-07-14T17:47:14.378 INFO:teuthology.run_tasks:Running task internal.archive... 2023-07-14T17:47:14.385 INFO:teuthology.task.internal:Creating archive directory... 2023-07-14T17:47:14.385 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-14T17:47:14.419 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-14T17:47:14.422 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-14T17:47:14.483 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-07-14T17:47:14.489 INFO:teuthology.task.internal:Enabling coredump saving... 2023-07-14T17:47:14.490 DEBUG:teuthology.orchestra.run.smithi161:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-14T17:47:14.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T17:47:14.514 DEBUG:teuthology.orchestra.run.smithi171:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-14T17:47:14.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T17:47:14.534 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-14T17:47:14.539 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-14T17:47:14.557 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-14T17:47:14.609 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T17:47:14.613 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T17:47:14.629 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T17:47:14.632 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T17:47:14.665 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T17:47:14.694 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T17:47:14.698 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-07-14T17:47:14.706 INFO:teuthology.task.internal:Configuring sudo... 2023-07-14T17:47:14.706 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-14T17:47:14.711 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-14T17:47:14.714 DEBUG:teuthology.orchestra.run.smithi171:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-14T17:47:14.775 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-07-14T17:47:14.786 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-07-14T17:47:14.787 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-14T17:47:14.792 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-14T17:47:14.794 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-14T17:47:14.830 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T17:47:19.947 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T17:47:20.023 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T17:47:25.102 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T17:47:25.177 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T17:47:25.178 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-14T17:47:25.248 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T17:47:29.218 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T17:47:29.266 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T17:47:29.318 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T17:47:29.358 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T17:47:29.358 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-14T17:47:29.428 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T17:47:29.480 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T17:47:29.529 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T17:47:29.620 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T17:47:29.662 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2023-07-14T17:47:29.662 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-14T17:47:29.733 DEBUG:teuthology.orchestra.run.smithi133:> sudo service rsyslog restart 2023-07-14T17:47:29.739 DEBUG:teuthology.orchestra.run.smithi161:> sudo service rsyslog restart 2023-07-14T17:47:29.742 DEBUG:teuthology.orchestra.run.smithi171:> sudo service rsyslog restart 2023-07-14T17:47:29.809 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T17:47:29.825 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T17:47:29.854 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T17:47:30.124 INFO:teuthology.run_tasks:Running task internal.timer... 2023-07-14T17:47:30.132 INFO:teuthology.task.internal:Starting timer... 2023-07-14T17:47:30.133 INFO:teuthology.run_tasks:Running task pcp... 2023-07-14T17:47:30.154 INFO:teuthology.run_tasks:Running task selinux... 2023-07-14T17:47:30.165 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-07-14T17:47:30.166 DEBUG:teuthology.orchestra.run.smithi133:> sudo service auditd rotate 2023-07-14T17:47:30.290 INFO:teuthology.orchestra.run.smithi133.stdout:Rotating logs: 2023-07-14T17:47:30.293 DEBUG:teuthology.orchestra.run.smithi161:> sudo service auditd rotate 2023-07-14T17:47:30.348 INFO:teuthology.orchestra.run.smithi161.stdout:Rotating logs: 2023-07-14T17:47:30.350 DEBUG:teuthology.orchestra.run.smithi171:> sudo service auditd rotate 2023-07-14T17:47:30.421 INFO:teuthology.orchestra.run.smithi171.stdout:Rotating logs: 2023-07-14T17:47:30.423 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-07-14T17:47:30.423 DEBUG:teuthology.orchestra.run.smithi133:> /usr/sbin/getenforce 2023-07-14T17:47:30.479 INFO:teuthology.orchestra.run.smithi133.stdout:Permissive 2023-07-14T17:47:30.480 DEBUG:teuthology.orchestra.run.smithi161:> /usr/sbin/getenforce 2023-07-14T17:47:30.507 INFO:teuthology.orchestra.run.smithi161.stdout:Permissive 2023-07-14T17:47:30.508 DEBUG:teuthology.orchestra.run.smithi171:> /usr/sbin/getenforce 2023-07-14T17:47:30.528 INFO:teuthology.orchestra.run.smithi171.stdout:Permissive 2023-07-14T17:47:30.529 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi133.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi161.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi171.front.sepia.ceph.com': 'permissive'} 2023-07-14T17:47:30.529 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T17:47:30.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T17:47:30.599 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T17:47:30.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T17:47:30.627 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T17:47:30.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T17:47:30.657 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-07-14T17:47:30.658 DEBUG:teuthology.orchestra.run.smithi133:> sudo /usr/sbin/setenforce permissive 2023-07-14T17:47:30.729 DEBUG:teuthology.orchestra.run.smithi161:> sudo /usr/sbin/setenforce permissive 2023-07-14T17:47:30.759 DEBUG:teuthology.orchestra.run.smithi171:> sudo /usr/sbin/setenforce permissive 2023-07-14T17:47:30.787 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-07-14T17:47:30.809 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-07-14T17:47:30.810 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-07-14T17:47:30.833 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-07-14T17:47:30.835 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi133.front.sepia.ceph.com,smithi161.front.sepia.ceph.com,smithi171.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-07-14T18:01:14.801 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi133.front.sepia.ceph.com'), Remote(name='ubuntu@smithi161.front.sepia.ceph.com'), Remote(name='ubuntu@smithi171.front.sepia.ceph.com')] 2023-07-14T18:01:14.803 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2023-07-14T18:01:14.804 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T18:01:19.897 DEBUG:teuthology.orchestra.run.smithi133:> true 2023-07-14T18:01:20.017 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2023-07-14T18:01:20.018 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-07-14T18:01:20.019 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T18:01:25.092 DEBUG:teuthology.orchestra.run.smithi161:> true 2023-07-14T18:01:25.164 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-07-14T18:01:25.164 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2023-07-14T18:01:25.165 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T18:01:25.232 DEBUG:teuthology.orchestra.run.smithi171:> true 2023-07-14T18:01:25.303 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi171.front.sepia.ceph.com' 2023-07-14T18:01:25.304 INFO:teuthology.run_tasks:Running task clock... 2023-07-14T18:01:25.314 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-07-14T18:01:25.314 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-14T18:01:25.315 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T18:01:25.319 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-14T18:01:25.320 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T18:01:25.322 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-14T18:01:25.322 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T18:01:25.357 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-14T18:01:25.373 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-14T18:01:25.375 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-14T18:01:25.390 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-14T18:01:25.414 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-14T18:01:25.440 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: ntpd: command not found 2023-07-14T18:01:25.450 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-14T18:01:25.453 INFO:teuthology.orchestra.run.smithi161.stdout:506 Cannot talk to daemon 2023-07-14T18:01:25.467 INFO:teuthology.orchestra.run.smithi171.stderr:sudo: ntpd: command not found 2023-07-14T18:01:25.468 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-14T18:01:25.479 INFO:teuthology.orchestra.run.smithi171.stdout:506 Cannot talk to daemon 2023-07-14T18:01:25.482 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-14T18:01:25.494 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-14T18:01:25.508 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-14T18:01:25.539 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: ntpd: command not found 2023-07-14T18:01:25.554 INFO:teuthology.orchestra.run.smithi161.stderr:bash: ntpq: command not found 2023-07-14T18:01:25.558 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T18:01:25.559 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2023-07-14T18:01:25.559 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T18:01:25.559 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T18:01:25.559 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T18:01:25.559 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T18:01:25.568 INFO:teuthology.orchestra.run.smithi133.stdout:506 Cannot talk to daemon 2023-07-14T18:01:25.584 INFO:teuthology.orchestra.run.smithi171.stderr:bash: ntpq: command not found 2023-07-14T18:01:25.587 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T18:01:25.588 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2023-07-14T18:01:25.588 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T18:01:25.588 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T18:01:25.588 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T18:01:25.588 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T18:01:25.602 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-14T18:01:25.636 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-14T18:01:25.729 INFO:teuthology.orchestra.run.smithi133.stderr:bash: ntpq: command not found 2023-07-14T18:01:25.737 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T18:01:25.737 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2023-07-14T18:01:25.737 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T18:01:25.737 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T18:01:25.737 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T18:01:25.737 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T18:01:25.738 INFO:teuthology.run_tasks:Running task pexec... 2023-07-14T18:01:25.747 INFO:teuthology.task.pexec:Executing custom commands... 2023-07-14T18:01:25.748 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi133.front.sepia.ceph.com 2023-07-14T18:01:25.748 DEBUG:teuthology.orchestra.run.smithi133:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-14T18:01:25.749 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi161.front.sepia.ceph.com 2023-07-14T18:01:25.749 DEBUG:teuthology.orchestra.run.smithi161:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-14T18:01:25.749 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi171.front.sepia.ceph.com 2023-07-14T18:01:25.750 DEBUG:teuthology.orchestra.run.smithi171:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-14T18:01:26.121 INFO:teuthology.orchestra.run.smithi171.stdout:Updating Subscription Management repositories. 2023-07-14T18:01:26.123 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2023-07-14T18:01:26.234 INFO:teuthology.orchestra.run.smithi133.stdout:Updating Subscription Management repositories. 2023-07-14T18:01:27.621 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:05:09 ago on Fri 14 Jul 2023 05:56:18 PM UTC. 2023-07-14T18:01:27.655 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:05:17 ago on Fri 14 Jul 2023 05:56:10 PM UTC. 2023-07-14T18:01:27.937 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:02:45 ago on Fri 14 Jul 2023 05:58:42 PM UTC. 2023-07-14T18:01:29.357 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2023-07-14T18:01:29.358 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2023-07-14T18:01:29.359 INFO:teuthology.orchestra.run.smithi171.stdout: Package Architecture Version Repository Size 2023-07-14T18:01:29.359 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2023-07-14T18:01:29.359 INFO:teuthology.orchestra.run.smithi171.stdout:Resetting modules: 2023-07-14T18:01:29.360 INFO:teuthology.orchestra.run.smithi171.stdout: container-tools 2023-07-14T18:01:29.360 INFO:teuthology.orchestra.run.smithi171.stdout: 2023-07-14T18:01:29.360 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2023-07-14T18:01:29.360 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2023-07-14T18:01:29.361 INFO:teuthology.orchestra.run.smithi171.stdout: 2023-07-14T18:01:29.371 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-07-14T18:01:29.372 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-07-14T18:01:29.373 INFO:teuthology.orchestra.run.smithi161.stdout: Package Architecture Version Repository Size 2023-07-14T18:01:29.373 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-07-14T18:01:29.373 INFO:teuthology.orchestra.run.smithi161.stdout:Resetting modules: 2023-07-14T18:01:29.373 INFO:teuthology.orchestra.run.smithi161.stdout: container-tools 2023-07-14T18:01:29.373 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T18:01:29.373 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2023-07-14T18:01:29.373 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-07-14T18:01:29.374 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T18:01:29.731 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2023-07-14T18:01:29.732 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2023-07-14T18:01:29.733 INFO:teuthology.orchestra.run.smithi133.stdout: Package Architecture Version Repository Size 2023-07-14T18:01:29.733 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2023-07-14T18:01:29.733 INFO:teuthology.orchestra.run.smithi133.stdout:Resetting modules: 2023-07-14T18:01:29.733 INFO:teuthology.orchestra.run.smithi133.stdout: container-tools 2023-07-14T18:01:29.733 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:01:29.733 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2023-07-14T18:01:29.734 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2023-07-14T18:01:29.734 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:01:30.305 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:05:11 ago on Fri 14 Jul 2023 05:56:18 PM UTC. 2023-07-14T18:01:30.329 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:05:19 ago on Fri 14 Jul 2023 05:56:10 PM UTC. 2023-07-14T18:01:30.966 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:02:48 ago on Fri 14 Jul 2023 05:58:42 PM UTC. 2023-07-14T18:01:39.164 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2023-07-14T18:01:39.759 INFO:teuthology.orchestra.run.smithi171.stdout:Updating Subscription Management repositories. 2023-07-14T18:01:41.281 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-07-14T18:01:41.426 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:05:22 ago on Fri 14 Jul 2023 05:56:18 PM UTC. 2023-07-14T18:01:41.739 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2023-07-14T18:01:41.901 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2023-07-14T18:01:42.494 INFO:teuthology.orchestra.run.smithi133.stdout:Updating Subscription Management repositories. 2023-07-14T18:01:43.288 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2023-07-14T18:01:43.338 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:05:32 ago on Fri 14 Jul 2023 05:56:10 PM UTC. 2023-07-14T18:01:43.380 INFO:teuthology.orchestra.run.smithi171.stdout:============================================================================================================ 2023-07-14T18:01:43.380 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2023-07-14T18:01:43.381 INFO:teuthology.orchestra.run.smithi171.stdout:============================================================================================================ 2023-07-14T18:01:43.381 INFO:teuthology.orchestra.run.smithi171.stdout:Installing group/module packages: 2023-07-14T18:01:43.381 INFO:teuthology.orchestra.run.smithi171.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-14T18:01:43.381 INFO:teuthology.orchestra.run.smithi171.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-14T18:01:43.381 INFO:teuthology.orchestra.run.smithi171.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-14T18:01:43.381 INFO:teuthology.orchestra.run.smithi171.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-14T18:01:43.381 INFO:teuthology.orchestra.run.smithi171.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-14T18:01:43.382 INFO:teuthology.orchestra.run.smithi171.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-14T18:01:43.382 INFO:teuthology.orchestra.run.smithi171.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-14T18:01:43.382 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2023-07-14T18:01:43.382 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-14T18:01:43.382 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-14T18:01:43.382 INFO:teuthology.orchestra.run.smithi171.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-14T18:01:43.382 INFO:teuthology.orchestra.run.smithi171.stdout:Installing module profiles: 2023-07-14T18:01:43.382 INFO:teuthology.orchestra.run.smithi171.stdout: container-tools/common 2023-07-14T18:01:43.383 INFO:teuthology.orchestra.run.smithi171.stdout: 2023-07-14T18:01:43.383 INFO:teuthology.orchestra.run.smithi171.stdout:Enabling module streams: 2023-07-14T18:01:43.383 INFO:teuthology.orchestra.run.smithi171.stdout: container-tools rhel8 2023-07-14T18:01:43.383 INFO:teuthology.orchestra.run.smithi171.stdout: 2023-07-14T18:01:43.383 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2023-07-14T18:01:43.383 INFO:teuthology.orchestra.run.smithi171.stdout:============================================================================================================ 2023-07-14T18:01:43.383 INFO:teuthology.orchestra.run.smithi171.stdout:Install 10 Packages 2023-07-14T18:01:43.383 INFO:teuthology.orchestra.run.smithi171.stdout: 2023-07-14T18:01:43.384 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 18 M 2023-07-14T18:01:43.384 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 65 M 2023-07-14T18:01:43.384 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2023-07-14T18:01:43.529 INFO:teuthology.orchestra.run.smithi171.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 280 kB/s | 41 kB 00:00 2023-07-14T18:01:43.545 INFO:teuthology.orchestra.run.smithi171.stdout:(2/10): python3-pyxdg-0.25-16.el8.noarch.rpm 578 kB/s | 94 kB 00:00 2023-07-14T18:01:43.579 INFO:teuthology.orchestra.run.smithi171.stdout:(3/10): python3-pytoml-0.1.14-5.git7dea353.el8. 127 kB/s | 25 kB 00:00 2023-07-14T18:01:43.687 INFO:teuthology.orchestra.run.smithi171.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 311 kB/s | 49 kB 00:00 2023-07-14T18:01:43.763 INFO:teuthology.orchestra.run.smithi171.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 2.4 MB/s | 524 kB 00:00 2023-07-14T18:01:43.897 INFO:teuthology.orchestra.run.smithi171.stdout:(6/10): buildah-1.26.2-1.module+el8.6.0+16771+2 25 MB/s | 8.0 MB 00:00 2023-07-14T18:01:43.922 INFO:teuthology.orchestra.run.smithi171.stdout:(7/10): python3-podman-4.0.1-1.module+el8.6.0+1 638 kB/s | 149 kB 00:00 2023-07-14T18:01:43.947 INFO:teuthology.orchestra.run.smithi171.stdout:(8/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.1 MB/s | 213 kB 00:00 2023-07-14T18:01:44.106 INFO:teuthology.orchestra.run.smithi171.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 12 MB/s | 2.2 MB 00:00 2023-07-14T18:01:44.179 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:03:01 ago on Fri 14 Jul 2023 05:58:42 PM UTC. 2023-07-14T18:01:44.314 INFO:teuthology.orchestra.run.smithi171.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 17 MB/s | 7.0 MB 00:00 2023-07-14T18:01:44.315 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2023-07-14T18:01:44.315 INFO:teuthology.orchestra.run.smithi171.stdout:Total 20 MB/s | 18 MB 00:00 2023-07-14T18:01:44.515 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2023-07-14T18:01:44.533 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2023-07-14T18:01:44.534 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2023-07-14T18:01:44.718 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2023-07-14T18:01:44.722 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2023-07-14T18:01:45.212 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-07-14T18:01:45.222 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2023-07-14T18:01:45.308 INFO:teuthology.orchestra.run.smithi161.stdout:============================================================================================================ 2023-07-14T18:01:45.308 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2023-07-14T18:01:45.308 INFO:teuthology.orchestra.run.smithi161.stdout:============================================================================================================ 2023-07-14T18:01:45.308 INFO:teuthology.orchestra.run.smithi161.stdout:Installing group/module packages: 2023-07-14T18:01:45.309 INFO:teuthology.orchestra.run.smithi161.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-14T18:01:45.309 INFO:teuthology.orchestra.run.smithi161.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-14T18:01:45.309 INFO:teuthology.orchestra.run.smithi161.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-14T18:01:45.309 INFO:teuthology.orchestra.run.smithi161.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-14T18:01:45.309 INFO:teuthology.orchestra.run.smithi161.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-14T18:01:45.310 INFO:teuthology.orchestra.run.smithi161.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-14T18:01:45.310 INFO:teuthology.orchestra.run.smithi161.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-14T18:01:45.310 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2023-07-14T18:01:45.310 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-14T18:01:45.311 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-14T18:01:45.311 INFO:teuthology.orchestra.run.smithi161.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-14T18:01:45.311 INFO:teuthology.orchestra.run.smithi161.stdout:Installing module profiles: 2023-07-14T18:01:45.311 INFO:teuthology.orchestra.run.smithi161.stdout: container-tools/common 2023-07-14T18:01:45.312 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T18:01:45.312 INFO:teuthology.orchestra.run.smithi161.stdout:Enabling module streams: 2023-07-14T18:01:45.312 INFO:teuthology.orchestra.run.smithi161.stdout: container-tools rhel8 2023-07-14T18:01:45.313 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T18:01:45.313 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2023-07-14T18:01:45.313 INFO:teuthology.orchestra.run.smithi161.stdout:============================================================================================================ 2023-07-14T18:01:45.313 INFO:teuthology.orchestra.run.smithi161.stdout:Install 10 Packages 2023-07-14T18:01:45.314 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T18:01:45.315 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 18 M 2023-07-14T18:01:45.315 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 65 M 2023-07-14T18:01:45.315 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2023-07-14T18:01:45.382 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-14T18:01:45.467 INFO:teuthology.orchestra.run.smithi161.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 602 kB/s | 94 kB 00:00 2023-07-14T18:01:45.492 INFO:teuthology.orchestra.run.smithi161.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 137 kB/s | 25 kB 00:00 2023-07-14T18:01:45.517 INFO:teuthology.orchestra.run.smithi161.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 198 kB/s | 41 kB 00:00 2023-07-14T18:01:45.587 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T18:01:45.609 INFO:teuthology.orchestra.run.smithi161.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 347 kB/s | 49 kB 00:00 2023-07-14T18:01:45.651 INFO:teuthology.orchestra.run.smithi161.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 3.2 MB/s | 524 kB 00:00 2023-07-14T18:01:45.781 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-14T18:01:45.826 INFO:teuthology.orchestra.run.smithi161.stdout:(6/10): buildah-1.26.2-1.module+el8.6.0+16771+2 26 MB/s | 8.0 MB 00:00 2023-07-14T18:01:45.852 INFO:teuthology.orchestra.run.smithi161.stdout:(7/10): python3-podman-4.0.1-1.module+el8.6.0+1 616 kB/s | 149 kB 00:00 2023-07-14T18:01:45.877 INFO:teuthology.orchestra.run.smithi161.stdout:(8/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 944 kB/s | 213 kB 00:00 2023-07-14T18:01:45.937 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2023-07-14T18:01:45.971 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-14T18:01:46.011 INFO:teuthology.orchestra.run.smithi161.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 38 MB/s | 7.0 MB 00:00 2023-07-14T18:01:46.037 INFO:teuthology.orchestra.run.smithi133.stdout:============================================================================================================ 2023-07-14T18:01:46.038 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2023-07-14T18:01:46.038 INFO:teuthology.orchestra.run.smithi133.stdout:============================================================================================================ 2023-07-14T18:01:46.038 INFO:teuthology.orchestra.run.smithi133.stdout:Installing group/module packages: 2023-07-14T18:01:46.038 INFO:teuthology.orchestra.run.smithi133.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-14T18:01:46.038 INFO:teuthology.orchestra.run.smithi133.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-14T18:01:46.038 INFO:teuthology.orchestra.run.smithi133.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-14T18:01:46.039 INFO:teuthology.orchestra.run.smithi133.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-14T18:01:46.039 INFO:teuthology.orchestra.run.smithi133.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-14T18:01:46.039 INFO:teuthology.orchestra.run.smithi133.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-14T18:01:46.039 INFO:teuthology.orchestra.run.smithi133.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-14T18:01:46.039 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2023-07-14T18:01:46.039 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-14T18:01:46.039 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-14T18:01:46.040 INFO:teuthology.orchestra.run.smithi133.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-14T18:01:46.040 INFO:teuthology.orchestra.run.smithi133.stdout:Installing module profiles: 2023-07-14T18:01:46.040 INFO:teuthology.orchestra.run.smithi133.stdout: container-tools/common 2023-07-14T18:01:46.041 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:01:46.041 INFO:teuthology.orchestra.run.smithi133.stdout:Enabling module streams: 2023-07-14T18:01:46.041 INFO:teuthology.orchestra.run.smithi133.stdout: container-tools rhel8 2023-07-14T18:01:46.041 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:01:46.041 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2023-07-14T18:01:46.041 INFO:teuthology.orchestra.run.smithi133.stdout:============================================================================================================ 2023-07-14T18:01:46.041 INFO:teuthology.orchestra.run.smithi133.stdout:Install 10 Packages 2023-07-14T18:01:46.042 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:01:46.042 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 18 M 2023-07-14T18:01:46.042 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 65 M 2023-07-14T18:01:46.043 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2023-07-14T18:01:46.060 INFO:teuthology.orchestra.run.smithi161.stdout:(10/10): toolbox-0.0.99.3-0.6.module+el8.6.0+16 11 MB/s | 2.2 MB 00:00 2023-07-14T18:01:46.061 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2023-07-14T18:01:46.061 INFO:teuthology.orchestra.run.smithi161.stdout:Total 24 MB/s | 18 MB 00:00 2023-07-14T18:01:46.162 INFO:teuthology.orchestra.run.smithi133.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 337 kB/s | 41 kB 00:00 2023-07-14T18:01:46.187 INFO:teuthology.orchestra.run.smithi133.stdout:(2/10): python3-pyxdg-0.25-16.el8.noarch.rpm 640 kB/s | 94 kB 00:00 2023-07-14T18:01:46.246 INFO:teuthology.orchestra.run.smithi133.stdout:(3/10): python3-pytoml-0.1.14-5.git7dea353.el8. 121 kB/s | 25 kB 00:00 2023-07-14T18:01:46.263 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2023-07-14T18:01:46.282 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2023-07-14T18:01:46.282 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2023-07-14T18:01:46.304 INFO:teuthology.orchestra.run.smithi133.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 348 kB/s | 49 kB 00:00 2023-07-14T18:01:46.305 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-14T18:01:46.380 INFO:teuthology.orchestra.run.smithi133.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 2.7 MB/s | 524 kB 00:00 2023-07-14T18:01:46.438 INFO:teuthology.orchestra.run.smithi133.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.1 MB/s | 149 kB 00:00 2023-07-14T18:01:46.465 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2023-07-14T18:01:46.469 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2023-07-14T18:01:46.514 INFO:teuthology.orchestra.run.smithi133.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.6 MB/s | 213 kB 00:00 2023-07-14T18:01:46.664 INFO:teuthology.orchestra.run.smithi133.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 19 MB/s | 8.0 MB 00:00 2023-07-14T18:01:46.772 INFO:teuthology.orchestra.run.smithi133.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 8.5 MB/s | 2.2 MB 00:00 2023-07-14T18:01:46.889 INFO:teuthology.orchestra.run.smithi133.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 16 MB/s | 7.0 MB 00:00 2023-07-14T18:01:46.890 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2023-07-14T18:01:46.890 INFO:teuthology.orchestra.run.smithi133.stdout:Total 21 MB/s | 18 MB 00:00 2023-07-14T18:01:46.981 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2023-07-14T18:01:47.061 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-14T18:01:47.198 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-14T18:01:47.222 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2023-07-14T18:01:47.245 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2023-07-14T18:01:47.245 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2023-07-14T18:01:47.372 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T18:01:47.557 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-14T18:01:47.745 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-14T18:01:47.784 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2023-07-14T18:01:47.789 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2023-07-14T18:01:47.942 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-14T18:01:48.099 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-14T18:01:48.199 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-14T18:01:48.372 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2023-07-14T18:01:48.375 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-14T18:01:48.552 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T18:01:48.554 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-14T18:01:48.731 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T18:01:48.824 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T18:01:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-14T18:01:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T18:01:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-14T18:01:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-14T18:01:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-14T18:01:48.825 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-14T18:01:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-14T18:01:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-14T18:01:48.826 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-14T18:01:48.845 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-14T18:01:48.974 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-14T18:01:49.199 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-14T18:01:49.607 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-14T18:01:49.754 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-14T18:01:50.001 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-14T18:01:50.002 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:05:31 ago on Fri 14 Jul 2023 05:56:18 PM UTC. 2023-07-14T18:01:50.024 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-14T18:01:50.195 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-14T18:01:50.347 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T18:01:50.448 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-14T18:01:50.618 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T18:01:50.618 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-14T18:01:50.619 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T18:01:50.619 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-14T18:01:50.619 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-14T18:01:50.619 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-14T18:01:50.619 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-14T18:01:50.619 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-14T18:01:50.619 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-14T18:01:50.620 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-14T18:01:51.404 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-14T18:01:51.657 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-14T18:01:51.824 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-14T18:01:51.824 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:05:41 ago on Fri 14 Jul 2023 05:56:10 PM UTC. 2023-07-14T18:01:51.895 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-14T18:01:52.110 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T18:01:52.495 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T18:01:52.495 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-14T18:01:52.495 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T18:01:52.495 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-14T18:01:52.495 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-14T18:01:52.495 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-14T18:01:52.496 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-14T18:01:52.496 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-14T18:01:52.496 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-14T18:01:52.496 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-14T18:01:53.941 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-14T18:01:53.941 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:03:11 ago on Fri 14 Jul 2023 05:58:42 PM UTC. 2023-07-14T18:01:57.658 INFO:teuthology.orchestra.run.smithi171.stdout:Installed products updated. 2023-07-14T18:01:57.858 INFO:teuthology.orchestra.run.smithi171.stdout: 2023-07-14T18:01:57.859 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2023-07-14T18:01:57.859 INFO:teuthology.orchestra.run.smithi171.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T18:01:57.859 INFO:teuthology.orchestra.run.smithi171.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T18:01:57.859 INFO:teuthology.orchestra.run.smithi171.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T18:01:57.859 INFO:teuthology.orchestra.run.smithi171.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-14T18:01:57.860 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-14T18:01:57.860 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-14T18:01:57.860 INFO:teuthology.orchestra.run.smithi171.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T18:01:57.860 INFO:teuthology.orchestra.run.smithi171.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T18:01:57.860 INFO:teuthology.orchestra.run.smithi171.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T18:01:57.860 INFO:teuthology.orchestra.run.smithi171.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-14T18:01:57.860 INFO:teuthology.orchestra.run.smithi171.stdout: 2023-07-14T18:01:57.861 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2023-07-14T18:01:58.013 DEBUG:teuthology.parallel:result is None 2023-07-14T18:02:01.873 INFO:teuthology.orchestra.run.smithi161.stdout:Installed products updated. 2023-07-14T18:02:02.147 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T18:02:02.148 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2023-07-14T18:02:02.148 INFO:teuthology.orchestra.run.smithi161.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T18:02:02.148 INFO:teuthology.orchestra.run.smithi161.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T18:02:02.148 INFO:teuthology.orchestra.run.smithi161.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T18:02:02.148 INFO:teuthology.orchestra.run.smithi161.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-14T18:02:02.149 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-14T18:02:02.149 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-14T18:02:02.149 INFO:teuthology.orchestra.run.smithi161.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T18:02:02.149 INFO:teuthology.orchestra.run.smithi161.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T18:02:02.149 INFO:teuthology.orchestra.run.smithi161.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T18:02:02.149 INFO:teuthology.orchestra.run.smithi161.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-14T18:02:02.150 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T18:02:02.150 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-07-14T18:02:02.323 DEBUG:teuthology.parallel:result is None 2023-07-14T18:02:02.592 INFO:teuthology.orchestra.run.smithi133.stdout:Installed products updated. 2023-07-14T18:02:02.812 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:02:02.812 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2023-07-14T18:02:02.812 INFO:teuthology.orchestra.run.smithi133.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T18:02:02.812 INFO:teuthology.orchestra.run.smithi133.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T18:02:02.812 INFO:teuthology.orchestra.run.smithi133.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T18:02:02.812 INFO:teuthology.orchestra.run.smithi133.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-14T18:02:02.813 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-14T18:02:02.813 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-14T18:02:02.813 INFO:teuthology.orchestra.run.smithi133.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T18:02:02.813 INFO:teuthology.orchestra.run.smithi133.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T18:02:02.813 INFO:teuthology.orchestra.run.smithi133.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T18:02:02.813 INFO:teuthology.orchestra.run.smithi133.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-14T18:02:02.813 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:02:02.814 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2023-07-14T18:02:03.080 DEBUG:teuthology.parallel:result is None 2023-07-14T18:02:03.081 INFO:teuthology.run_tasks:Running task install... 2023-07-14T18:02:03.089 DEBUG:teuthology.task.install:project ceph 2023-07-14T18:02:03.089 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '81c9387d2e7576da065747ec9c16d1c7d69016ec'}} 2023-07-14T18:02:03.090 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '81c9387d2e7576da065747ec9c16d1c7d69016ec'} 2023-07-14T18:02:03.090 INFO:teuthology.task.install:Using flavor: default 2023-07-14T18:02:03.096 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-07-14T18:02:03.097 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-07-14T18:02:03.097 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81c9387d2e7576da065747ec9c16d1c7d69016ec', 'tag': None, 'wait_for_package': False} 2023-07-14T18:02:03.098 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81c9387d2e7576da065747ec9c16d1c7d69016ec 2023-07-14T18:02:03.100 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81c9387d2e7576da065747ec9c16d1c7d69016ec', 'tag': None, 'wait_for_package': False} 2023-07-14T18:02:03.100 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81c9387d2e7576da065747ec9c16d1c7d69016ec 2023-07-14T18:02:03.102 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81c9387d2e7576da065747ec9c16d1c7d69016ec', 'tag': None, 'wait_for_package': False} 2023-07-14T18:02:03.102 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81c9387d2e7576da065747ec9c16d1c7d69016ec 2023-07-14T18:02:18.410 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/ 2023-07-14T18:02:18.410 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4940.g81c9387d 2023-07-14T18:02:23.421 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/ 2023-07-14T18:02:23.421 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4940.g81c9387d 2023-07-14T18:02:23.423 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/ 2023-07-14T18:02:23.424 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4940.g81c9387d 2023-07-14T18:02:28.597 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-14T18:02:28.597 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T18:02:28.598 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-14T18:02:28.633 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-14T18:02:28.633 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2023-07-14T18:02:28.634 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-14T18:02:28.637 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-14T18:02:28.637 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T18:02:28.637 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-14T18:02:28.667 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-14T18:02:28.668 DEBUG:teuthology.orchestra.run.smithi171:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81c9387d2e7576da065747ec9c16d1c7d69016ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-14T18:02:28.669 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-14T18:02:28.669 DEBUG:teuthology.orchestra.run.smithi133:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81c9387d2e7576da065747ec9c16d1c7d69016ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-14T18:02:28.672 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-14T18:02:28.672 DEBUG:teuthology.orchestra.run.smithi161:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81c9387d2e7576da065747ec9c16d1c7d69016ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-14T18:02:28.738 DEBUG:teuthology.orchestra.run.smithi171:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-14T18:02:28.743 DEBUG:teuthology.orchestra.run.smithi161:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-14T18:02:28.780 DEBUG:teuthology.orchestra.run.smithi133:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-14T18:02:28.818 DEBUG:teuthology.orchestra.run.smithi171:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-14T18:02:28.823 DEBUG:teuthology.orchestra.run.smithi161:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-14T18:02:28.875 DEBUG:teuthology.orchestra.run.smithi133:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-14T18:02:28.885 INFO:teuthology.orchestra.run.smithi171.stdout:check_obsoletes = 1 2023-07-14T18:02:28.886 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum clean all 2023-07-14T18:02:28.891 INFO:teuthology.orchestra.run.smithi161.stdout:check_obsoletes = 1 2023-07-14T18:02:28.892 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum clean all 2023-07-14T18:02:28.946 INFO:teuthology.orchestra.run.smithi133.stdout:check_obsoletes = 1 2023-07-14T18:02:28.949 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum clean all 2023-07-14T18:02:29.264 INFO:teuthology.orchestra.run.smithi171.stdout:Updating Subscription Management repositories. 2023-07-14T18:02:29.268 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2023-07-14T18:02:29.424 INFO:teuthology.orchestra.run.smithi133.stdout:Updating Subscription Management repositories. 2023-07-14T18:02:30.331 INFO:teuthology.orchestra.run.smithi171.stdout:44 files removed 2023-07-14T18:02:30.371 INFO:teuthology.orchestra.run.smithi161.stdout:44 files removed 2023-07-14T18:02:30.382 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-14T18:02:30.425 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-14T18:02:30.449 INFO:teuthology.orchestra.run.smithi133.stdout:44 files removed 2023-07-14T18:02:30.512 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-14T18:02:30.715 INFO:teuthology.orchestra.run.smithi171.stdout:Updating Subscription Management repositories. 2023-07-14T18:02:30.759 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2023-07-14T18:02:30.977 INFO:teuthology.orchestra.run.smithi133.stdout:Updating Subscription Management repositories. 2023-07-14T18:02:32.782 INFO:teuthology.orchestra.run.smithi171.stdout:ceph packages for x86_64 98 kB/s | 81 kB 00:00 2023-07-14T18:02:32.994 INFO:teuthology.orchestra.run.smithi161.stdout:ceph packages for x86_64 95 kB/s | 81 kB 00:00 2023-07-14T18:02:33.275 INFO:teuthology.orchestra.run.smithi133.stdout:ceph packages for x86_64 93 kB/s | 81 kB 00:00 2023-07-14T18:02:33.566 INFO:teuthology.orchestra.run.smithi171.stdout:ceph noarch packages 13 kB/s | 9.4 kB 00:00 2023-07-14T18:02:33.786 INFO:teuthology.orchestra.run.smithi161.stdout:ceph noarch packages 12 kB/s | 9.4 kB 00:00 2023-07-14T18:02:34.101 INFO:teuthology.orchestra.run.smithi133.stdout:ceph noarch packages 12 kB/s | 9.4 kB 00:00 2023-07-14T18:02:34.391 INFO:teuthology.orchestra.run.smithi171.stdout:ceph source packages 1.9 kB/s | 1.5 kB 00:00 2023-07-14T18:02:34.628 INFO:teuthology.orchestra.run.smithi161.stdout:ceph source packages 1.8 kB/s | 1.5 kB 00:00 2023-07-14T18:02:34.759 INFO:teuthology.orchestra.run.smithi171.stdout:Copr repo for python3-asyncssh owned by ceph 10 kB/s | 3.5 kB 00:00 2023-07-14T18:02:34.926 INFO:teuthology.orchestra.run.smithi133.stdout:ceph source packages 1.9 kB/s | 1.5 kB 00:00 2023-07-14T18:02:34.946 INFO:teuthology.orchestra.run.smithi161.stdout:Copr repo for python3-asyncssh owned by ceph 12 kB/s | 3.5 kB 00:00 2023-07-14T18:02:35.285 INFO:teuthology.orchestra.run.smithi133.stdout:Copr repo for python3-asyncssh owned by ceph 10 kB/s | 3.5 kB 00:00 2023-07-14T18:02:36.222 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2023-07-14T18:02:37.352 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 7.6 MB/s | 16 MB 00:02 2023-07-14T18:02:37.776 INFO:teuthology.orchestra.run.smithi171.stdout:Extra Packages for Enterprise Linux 5.2 MB/s | 16 MB 00:02 2023-07-14T18:02:39.730 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 496 kB/s | 24 kB 00:00 2023-07-14T18:02:40.924 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 42 MB/s | 47 MB 00:01 2023-07-14T18:02:41.251 INFO:teuthology.orchestra.run.smithi171.stdout:lab-extras 410 kB/s | 24 kB 00:00 2023-07-14T18:02:41.344 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 494 kB/s | 24 kB 00:00 2023-07-14T18:02:42.337 INFO:teuthology.orchestra.run.smithi171.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 47 MB/s | 47 MB 00:01 2023-07-14T18:02:42.571 INFO:teuthology.orchestra.run.smithi133.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 46 MB/s | 53 MB 00:01 2023-07-14T18:02:48.975 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 50 MB/s | 53 MB 00:01 2023-07-14T18:02:50.446 INFO:teuthology.orchestra.run.smithi171.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 48 MB/s | 53 MB 00:01 2023-07-14T18:02:51.555 INFO:teuthology.orchestra.run.smithi133.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 34 MB/s | 47 MB 00:01 2023-07-14T18:02:59.981 INFO:teuthology.orchestra.run.smithi161.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T18:02:59.982 INFO:teuthology.orchestra.run.smithi161.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T18:02:59.997 INFO:teuthology.orchestra.run.smithi161.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-14T18:02:59.999 INFO:teuthology.orchestra.run.smithi161.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-14T18:03:00.000 INFO:teuthology.orchestra.run.smithi161.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-14T18:03:00.001 INFO:teuthology.orchestra.run.smithi161.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-14T18:03:00.003 INFO:teuthology.orchestra.run.smithi161.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-14T18:03:00.007 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-14T18:03:00.008 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-14T18:03:00.142 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-07-14T18:03:00.154 INFO:teuthology.orchestra.run.smithi161.stdout:=========================================================================================================================== 2023-07-14T18:03:00.155 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2023-07-14T18:03:00.155 INFO:teuthology.orchestra.run.smithi161.stdout:=========================================================================================================================== 2023-07-14T18:03:00.155 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2023-07-14T18:03:00.155 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-14T18:03:00.155 INFO:teuthology.orchestra.run.smithi161.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-14T18:03:00.155 INFO:teuthology.orchestra.run.smithi161.stdout: ceph x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 6.4 k 2023-07-14T18:03:00.156 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 5.4 M 2023-07-14T18:03:00.156 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 850 k 2023-07-14T18:03:00.156 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 167 k 2023-07-14T18:03:00.156 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 1.6 M 2023-07-14T18:03:00.156 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 129 k 2023-07-14T18:03:00.156 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 1.7 M 2023-07-14T18:03:00.156 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 7.4 M 2023-07-14T18:03:00.157 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 47 k 2023-07-14T18:03:00.157 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 14 M 2023-07-14T18:03:00.157 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 48 M 2023-07-14T18:03:00.157 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 258 k 2023-07-14T18:03:00.157 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 165 k 2023-07-14T18:03:00.157 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 260 k 2023-07-14T18:03:00.158 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 18 k 2023-07-14T18:03:00.158 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-14T18:03:00.158 INFO:teuthology.orchestra.run.smithi161.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-14T18:03:00.158 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-14T18:03:00.158 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 31 k 2023-07-14T18:03:00.158 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 772 k 2023-07-14T18:03:00.158 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 139 k 2023-07-14T18:03:00.159 INFO:teuthology.orchestra.run.smithi161.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-14T18:03:00.159 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-14T18:03:00.159 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-14T18:03:00.159 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-14T18:03:00.160 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-14T18:03:00.160 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-14T18:03:00.160 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 197 k 2023-07-14T18:03:00.161 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 371 k 2023-07-14T18:03:00.161 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 364 k 2023-07-14T18:03:00.161 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 107 k 2023-07-14T18:03:00.161 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 94 k 2023-07-14T18:03:00.161 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 3.5 M 2023-07-14T18:03:00.162 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 179 k 2023-07-14T18:03:00.162 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-14T18:03:00.162 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrading: 2023-07-14T18:03:00.162 INFO:teuthology.orchestra.run.smithi161.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-14T18:03:00.162 INFO:teuthology.orchestra.run.smithi161.stdout: librados2 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 3.7 M 2023-07-14T18:03:00.162 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 3.6 M 2023-07-14T18:03:00.162 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2023-07-14T18:03:00.163 INFO:teuthology.orchestra.run.smithi161.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-14T18:03:00.163 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 22 M 2023-07-14T18:03:00.163 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 23 k 2023-07-14T18:03:00.163 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 2.3 M 2023-07-14T18:03:00.163 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 241 k 2023-07-14T18:03:00.163 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 4.6 M 2023-07-14T18:03:00.163 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 18 M 2023-07-14T18:03:00.163 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 15 k 2023-07-14T18:03:00.164 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 24 k 2023-07-14T18:03:00.164 INFO:teuthology.orchestra.run.smithi161.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-14T18:03:00.164 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-14T18:03:00.164 INFO:teuthology.orchestra.run.smithi161.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-14T18:03:00.164 INFO:teuthology.orchestra.run.smithi161.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-14T18:03:00.165 INFO:teuthology.orchestra.run.smithi161.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-14T18:03:00.165 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 135 k 2023-07-14T18:03:00.165 INFO:teuthology.orchestra.run.smithi161.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-14T18:03:00.165 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-14T18:03:00.166 INFO:teuthology.orchestra.run.smithi161.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-14T18:03:00.166 INFO:teuthology.orchestra.run.smithi161.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-14T18:03:00.166 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-14T18:03:00.166 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-14T18:03:00.167 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 473 k 2023-07-14T18:03:00.167 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-14T18:03:00.167 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 6.8 M 2023-07-14T18:03:00.167 INFO:teuthology.orchestra.run.smithi161.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-14T18:03:00.168 INFO:teuthology.orchestra.run.smithi161.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-14T18:03:00.168 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-14T18:03:00.168 INFO:teuthology.orchestra.run.smithi161.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-14T18:03:00.168 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-14T18:03:00.169 INFO:teuthology.orchestra.run.smithi161.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-14T18:03:00.169 INFO:teuthology.orchestra.run.smithi161.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-14T18:03:00.169 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-14T18:03:00.169 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-14T18:03:00.170 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-14T18:03:00.170 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-14T18:03:00.170 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-14T18:03:00.170 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-14T18:03:00.171 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-14T18:03:00.171 INFO:teuthology.orchestra.run.smithi161.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-14T18:03:00.171 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-14T18:03:00.171 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 45 k 2023-07-14T18:03:00.172 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 122 k 2023-07-14T18:03:00.172 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-14T18:03:00.172 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-14T18:03:00.172 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-14T18:03:00.173 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-14T18:03:00.173 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-14T18:03:00.173 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-14T18:03:00.173 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-14T18:03:00.174 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-14T18:03:00.174 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-14T18:03:00.174 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-14T18:03:00.174 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-14T18:03:00.175 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T18:03:00.175 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-14T18:03:00.175 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-14T18:03:00.175 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-14T18:03:00.176 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-14T18:03:00.176 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-14T18:03:00.176 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-14T18:03:00.176 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-14T18:03:00.180 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-14T18:03:00.181 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-14T18:03:00.181 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-14T18:03:00.181 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-14T18:03:00.181 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-14T18:03:00.181 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-14T18:03:00.181 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-14T18:03:00.182 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-14T18:03:00.182 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-14T18:03:00.182 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-14T18:03:00.182 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-14T18:03:00.182 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T18:03:00.182 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-14T18:03:00.182 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-14T18:03:00.182 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-14T18:03:00.183 INFO:teuthology.orchestra.run.smithi161.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-14T18:03:00.183 INFO:teuthology.orchestra.run.smithi161.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-14T18:03:00.183 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-14T18:03:00.183 INFO:teuthology.orchestra.run.smithi161.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-14T18:03:00.183 INFO:teuthology.orchestra.run.smithi161.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-14T18:03:00.183 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-14T18:03:00.183 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-14T18:03:00.184 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-14T18:03:00.184 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-14T18:03:00.184 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-14T18:03:00.184 INFO:teuthology.orchestra.run.smithi161.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-14T18:03:00.184 INFO:teuthology.orchestra.run.smithi161.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-14T18:03:00.184 INFO:teuthology.orchestra.run.smithi161.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-14T18:03:00.184 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-14T18:03:00.185 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-14T18:03:00.185 INFO:teuthology.orchestra.run.smithi161.stdout:Enabling module streams: 2023-07-14T18:03:00.185 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql 10 2023-07-14T18:03:00.185 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T18:03:00.185 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2023-07-14T18:03:00.185 INFO:teuthology.orchestra.run.smithi161.stdout:=========================================================================================================================== 2023-07-14T18:03:00.185 INFO:teuthology.orchestra.run.smithi161.stdout:Install 128 Packages 2023-07-14T18:03:00.186 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrade 3 Packages 2023-07-14T18:03:00.186 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T18:03:00.186 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 206 M 2023-07-14T18:03:00.187 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2023-07-14T18:03:00.392 INFO:teuthology.orchestra.run.smithi161.stdout:(1/131): ceph-18.0.0-4940.g81c9387d.el8.x86_64. 33 kB/s | 6.4 kB 00:00 2023-07-14T18:03:00.801 INFO:teuthology.orchestra.run.smithi161.stdout:(2/131): ceph-fuse-18.0.0-4940.g81c9387d.el8.x8 2.0 MB/s | 850 kB 00:00 2023-07-14T18:03:00.867 INFO:teuthology.orchestra.run.smithi161.stdout:(3/131): ceph-immutable-object-cache-18.0.0-494 2.4 MB/s | 167 kB 00:00 2023-07-14T18:03:01.376 INFO:teuthology.orchestra.run.smithi161.stdout:(4/131): ceph-base-18.0.0-4940.g81c9387d.el8.x8 4.6 MB/s | 5.4 MB 00:01 2023-07-14T18:03:01.527 INFO:teuthology.orchestra.run.smithi161.stdout:(5/131): ceph-mds-18.0.0-4940.g81c9387d.el8.x86 3.5 MB/s | 2.3 MB 00:00 2023-07-14T18:03:01.574 INFO:teuthology.orchestra.run.smithi171.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T18:03:01.575 INFO:teuthology.orchestra.run.smithi171.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T18:03:01.589 INFO:teuthology.orchestra.run.smithi171.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-14T18:03:01.590 INFO:teuthology.orchestra.run.smithi171.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-14T18:03:01.591 INFO:teuthology.orchestra.run.smithi171.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-14T18:03:01.592 INFO:teuthology.orchestra.run.smithi171.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-14T18:03:01.594 INFO:teuthology.orchestra.run.smithi171.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-14T18:03:01.597 INFO:teuthology.orchestra.run.smithi171.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-14T18:03:01.598 INFO:teuthology.orchestra.run.smithi171.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-14T18:03:01.677 INFO:teuthology.orchestra.run.smithi161.stdout:(6/131): ceph-mgr-18.0.0-4940.g81c9387d.el8.x86 5.2 MB/s | 1.6 MB 00:00 2023-07-14T18:03:01.730 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2023-07-14T18:03:01.743 INFO:teuthology.orchestra.run.smithi171.stdout:=========================================================================================================================== 2023-07-14T18:03:01.743 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2023-07-14T18:03:01.743 INFO:teuthology.orchestra.run.smithi171.stdout:=========================================================================================================================== 2023-07-14T18:03:01.743 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2023-07-14T18:03:01.743 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-14T18:03:01.743 INFO:teuthology.orchestra.run.smithi171.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-14T18:03:01.744 INFO:teuthology.orchestra.run.smithi171.stdout: ceph x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 6.4 k 2023-07-14T18:03:01.744 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 5.4 M 2023-07-14T18:03:01.744 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 850 k 2023-07-14T18:03:01.744 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 167 k 2023-07-14T18:03:01.744 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 1.6 M 2023-07-14T18:03:01.744 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 129 k 2023-07-14T18:03:01.744 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 1.7 M 2023-07-14T18:03:01.745 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 7.4 M 2023-07-14T18:03:01.745 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 47 k 2023-07-14T18:03:01.745 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 14 M 2023-07-14T18:03:01.745 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 48 M 2023-07-14T18:03:01.745 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 258 k 2023-07-14T18:03:01.745 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 165 k 2023-07-14T18:03:01.745 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-mirror x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 260 k 2023-07-14T18:03:01.746 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-top noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 18 k 2023-07-14T18:03:01.746 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-14T18:03:01.746 INFO:teuthology.orchestra.run.smithi171.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-14T18:03:01.746 INFO:teuthology.orchestra.run.smithi171.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-14T18:03:01.746 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 31 k 2023-07-14T18:03:01.746 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 772 k 2023-07-14T18:03:01.746 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 139 k 2023-07-14T18:03:01.746 INFO:teuthology.orchestra.run.smithi171.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-14T18:03:01.747 INFO:teuthology.orchestra.run.smithi171.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-14T18:03:01.747 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-14T18:03:01.747 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-14T18:03:01.747 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-14T18:03:01.747 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-14T18:03:01.747 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 197 k 2023-07-14T18:03:01.747 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 371 k 2023-07-14T18:03:01.748 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 364 k 2023-07-14T18:03:01.748 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 107 k 2023-07-14T18:03:01.748 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 94 k 2023-07-14T18:03:01.748 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 3.5 M 2023-07-14T18:03:01.748 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 179 k 2023-07-14T18:03:01.748 INFO:teuthology.orchestra.run.smithi171.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-14T18:03:01.748 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrading: 2023-07-14T18:03:01.749 INFO:teuthology.orchestra.run.smithi171.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-14T18:03:01.749 INFO:teuthology.orchestra.run.smithi171.stdout: librados2 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 3.7 M 2023-07-14T18:03:01.749 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 3.6 M 2023-07-14T18:03:01.749 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2023-07-14T18:03:01.749 INFO:teuthology.orchestra.run.smithi171.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-14T18:03:01.749 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 22 M 2023-07-14T18:03:01.749 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 23 k 2023-07-14T18:03:01.749 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 2.3 M 2023-07-14T18:03:01.750 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 241 k 2023-07-14T18:03:01.750 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 4.6 M 2023-07-14T18:03:01.750 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 18 M 2023-07-14T18:03:01.750 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 15 k 2023-07-14T18:03:01.750 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 24 k 2023-07-14T18:03:01.750 INFO:teuthology.orchestra.run.smithi171.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-14T18:03:01.750 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-14T18:03:01.750 INFO:teuthology.orchestra.run.smithi171.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-14T18:03:01.751 INFO:teuthology.orchestra.run.smithi171.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-14T18:03:01.751 INFO:teuthology.orchestra.run.smithi171.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-14T18:03:01.751 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 135 k 2023-07-14T18:03:01.751 INFO:teuthology.orchestra.run.smithi171.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-14T18:03:01.751 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-14T18:03:01.751 INFO:teuthology.orchestra.run.smithi171.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-14T18:03:01.751 INFO:teuthology.orchestra.run.smithi171.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-14T18:03:01.752 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-14T18:03:01.752 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-14T18:03:01.752 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 473 k 2023-07-14T18:03:01.752 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-14T18:03:01.752 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 6.8 M 2023-07-14T18:03:01.752 INFO:teuthology.orchestra.run.smithi171.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-14T18:03:01.752 INFO:teuthology.orchestra.run.smithi171.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-14T18:03:01.753 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-14T18:03:01.753 INFO:teuthology.orchestra.run.smithi171.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-14T18:03:01.753 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-14T18:03:01.753 INFO:teuthology.orchestra.run.smithi171.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-14T18:03:01.753 INFO:teuthology.orchestra.run.smithi171.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-14T18:03:01.753 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-14T18:03:01.753 INFO:teuthology.orchestra.run.smithi171.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-14T18:03:01.753 INFO:teuthology.orchestra.run.smithi171.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-14T18:03:01.754 INFO:teuthology.orchestra.run.smithi171.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-14T18:03:01.754 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-14T18:03:01.754 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-14T18:03:01.754 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-14T18:03:01.754 INFO:teuthology.orchestra.run.smithi171.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-14T18:03:01.754 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-14T18:03:01.754 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 45 k 2023-07-14T18:03:01.754 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 122 k 2023-07-14T18:03:01.755 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-14T18:03:01.755 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-14T18:03:01.755 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-14T18:03:01.755 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-14T18:03:01.755 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-14T18:03:01.755 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-14T18:03:01.755 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-14T18:03:01.755 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-14T18:03:01.756 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-14T18:03:01.756 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-14T18:03:01.756 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-14T18:03:01.756 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T18:03:01.757 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-14T18:03:01.757 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-14T18:03:01.757 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-14T18:03:01.757 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-14T18:03:01.758 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-14T18:03:01.758 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-14T18:03:01.758 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-14T18:03:01.759 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-14T18:03:01.759 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-14T18:03:01.759 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-14T18:03:01.759 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-14T18:03:01.760 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-14T18:03:01.760 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-14T18:03:01.760 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-14T18:03:01.760 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-14T18:03:01.761 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-14T18:03:01.761 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-14T18:03:01.761 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-14T18:03:01.762 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T18:03:01.762 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-14T18:03:01.762 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-14T18:03:01.763 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-14T18:03:01.763 INFO:teuthology.orchestra.run.smithi171.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-14T18:03:01.763 INFO:teuthology.orchestra.run.smithi171.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-14T18:03:01.763 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-14T18:03:01.764 INFO:teuthology.orchestra.run.smithi171.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-14T18:03:01.764 INFO:teuthology.orchestra.run.smithi171.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-14T18:03:01.764 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-14T18:03:01.765 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-14T18:03:01.765 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-14T18:03:01.765 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-14T18:03:01.765 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-14T18:03:01.765 INFO:teuthology.orchestra.run.smithi171.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-14T18:03:01.766 INFO:teuthology.orchestra.run.smithi171.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-14T18:03:01.766 INFO:teuthology.orchestra.run.smithi171.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-14T18:03:01.766 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-14T18:03:01.767 INFO:teuthology.orchestra.run.smithi171.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-14T18:03:01.767 INFO:teuthology.orchestra.run.smithi171.stdout:Enabling module streams: 2023-07-14T18:03:01.767 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql 10 2023-07-14T18:03:01.768 INFO:teuthology.orchestra.run.smithi171.stdout: 2023-07-14T18:03:01.768 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2023-07-14T18:03:01.768 INFO:teuthology.orchestra.run.smithi171.stdout:=========================================================================================================================== 2023-07-14T18:03:01.768 INFO:teuthology.orchestra.run.smithi171.stdout:Install 128 Packages 2023-07-14T18:03:01.768 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrade 3 Packages 2023-07-14T18:03:01.768 INFO:teuthology.orchestra.run.smithi171.stdout: 2023-07-14T18:03:01.769 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 206 M 2023-07-14T18:03:01.769 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2023-07-14T18:03:02.427 INFO:teuthology.orchestra.run.smithi161.stdout:(7/131): ceph-mon-18.0.0-4940.g81c9387d.el8.x86 5.1 MB/s | 4.6 MB 00:00 2023-07-14T18:03:02.605 INFO:teuthology.orchestra.run.smithi171.stdout:(1/131): ceph-18.0.0-4940.g81c9387d.el8.x86_64. 17 kB/s | 6.4 kB 00:00 2023-07-14T18:03:03.231 INFO:teuthology.orchestra.run.smithi133.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T18:03:03.232 INFO:teuthology.orchestra.run.smithi133.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T18:03:03.247 INFO:teuthology.orchestra.run.smithi133.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-14T18:03:03.248 INFO:teuthology.orchestra.run.smithi133.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-14T18:03:03.249 INFO:teuthology.orchestra.run.smithi133.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-14T18:03:03.251 INFO:teuthology.orchestra.run.smithi133.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-14T18:03:03.253 INFO:teuthology.orchestra.run.smithi133.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-14T18:03:03.256 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-14T18:03:03.258 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-14T18:03:03.327 INFO:teuthology.orchestra.run.smithi161.stdout:(8/131): ceph-common-18.0.0-4940.g81c9387d.el8. 7.1 MB/s | 22 MB 00:03 2023-07-14T18:03:03.400 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2023-07-14T18:03:03.412 INFO:teuthology.orchestra.run.smithi133.stdout:=========================================================================================================================== 2023-07-14T18:03:03.413 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2023-07-14T18:03:03.413 INFO:teuthology.orchestra.run.smithi133.stdout:=========================================================================================================================== 2023-07-14T18:03:03.413 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2023-07-14T18:03:03.413 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-14T18:03:03.413 INFO:teuthology.orchestra.run.smithi133.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-14T18:03:03.413 INFO:teuthology.orchestra.run.smithi133.stdout: ceph x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 6.4 k 2023-07-14T18:03:03.414 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 5.4 M 2023-07-14T18:03:03.414 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 850 k 2023-07-14T18:03:03.414 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 167 k 2023-07-14T18:03:03.414 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 1.6 M 2023-07-14T18:03:03.414 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 129 k 2023-07-14T18:03:03.414 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 1.7 M 2023-07-14T18:03:03.414 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 7.4 M 2023-07-14T18:03:03.414 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 47 k 2023-07-14T18:03:03.415 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 14 M 2023-07-14T18:03:03.415 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 48 M 2023-07-14T18:03:03.415 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 258 k 2023-07-14T18:03:03.415 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 165 k 2023-07-14T18:03:03.415 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 260 k 2023-07-14T18:03:03.415 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 18 k 2023-07-14T18:03:03.416 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-14T18:03:03.416 INFO:teuthology.orchestra.run.smithi133.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-14T18:03:03.416 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-14T18:03:03.416 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 31 k 2023-07-14T18:03:03.416 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 772 k 2023-07-14T18:03:03.416 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 139 k 2023-07-14T18:03:03.416 INFO:teuthology.orchestra.run.smithi133.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-14T18:03:03.417 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-14T18:03:03.417 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-14T18:03:03.417 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-14T18:03:03.417 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-14T18:03:03.418 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-14T18:03:03.418 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 197 k 2023-07-14T18:03:03.418 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 371 k 2023-07-14T18:03:03.418 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 364 k 2023-07-14T18:03:03.418 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 107 k 2023-07-14T18:03:03.418 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 94 k 2023-07-14T18:03:03.418 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 3.5 M 2023-07-14T18:03:03.418 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 179 k 2023-07-14T18:03:03.419 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-14T18:03:03.419 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrading: 2023-07-14T18:03:03.419 INFO:teuthology.orchestra.run.smithi133.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-14T18:03:03.419 INFO:teuthology.orchestra.run.smithi133.stdout: librados2 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 3.7 M 2023-07-14T18:03:03.419 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 3.6 M 2023-07-14T18:03:03.419 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2023-07-14T18:03:03.419 INFO:teuthology.orchestra.run.smithi133.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-14T18:03:03.419 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 22 M 2023-07-14T18:03:03.420 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 23 k 2023-07-14T18:03:03.420 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 2.3 M 2023-07-14T18:03:03.420 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 241 k 2023-07-14T18:03:03.420 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 4.6 M 2023-07-14T18:03:03.420 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 18 M 2023-07-14T18:03:03.420 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 15 k 2023-07-14T18:03:03.420 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 24 k 2023-07-14T18:03:03.421 INFO:teuthology.orchestra.run.smithi133.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-14T18:03:03.421 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-14T18:03:03.421 INFO:teuthology.orchestra.run.smithi133.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-14T18:03:03.421 INFO:teuthology.orchestra.run.smithi133.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-14T18:03:03.421 INFO:teuthology.orchestra.run.smithi133.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-14T18:03:03.421 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 135 k 2023-07-14T18:03:03.421 INFO:teuthology.orchestra.run.smithi133.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-14T18:03:03.421 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-14T18:03:03.422 INFO:teuthology.orchestra.run.smithi133.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-14T18:03:03.422 INFO:teuthology.orchestra.run.smithi133.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-14T18:03:03.422 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-14T18:03:03.422 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-14T18:03:03.422 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 473 k 2023-07-14T18:03:03.422 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-14T18:03:03.422 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 6.8 M 2023-07-14T18:03:03.423 INFO:teuthology.orchestra.run.smithi133.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-14T18:03:03.423 INFO:teuthology.orchestra.run.smithi133.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-14T18:03:03.423 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-14T18:03:03.423 INFO:teuthology.orchestra.run.smithi133.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-14T18:03:03.423 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-14T18:03:03.423 INFO:teuthology.orchestra.run.smithi133.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-14T18:03:03.423 INFO:teuthology.orchestra.run.smithi133.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-14T18:03:03.423 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-14T18:03:03.424 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-14T18:03:03.424 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-14T18:03:03.424 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-14T18:03:03.424 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-14T18:03:03.424 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-14T18:03:03.424 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-14T18:03:03.424 INFO:teuthology.orchestra.run.smithi133.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-14T18:03:03.425 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-14T18:03:03.425 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 45 k 2023-07-14T18:03:03.425 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 122 k 2023-07-14T18:03:03.425 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-14T18:03:03.425 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-14T18:03:03.425 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-14T18:03:03.425 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-14T18:03:03.425 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-14T18:03:03.426 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-14T18:03:03.426 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-14T18:03:03.426 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-14T18:03:03.426 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-14T18:03:03.426 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-14T18:03:03.426 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-14T18:03:03.426 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T18:03:03.426 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-14T18:03:03.427 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-14T18:03:03.427 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-14T18:03:03.427 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-14T18:03:03.427 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-14T18:03:03.427 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-14T18:03:03.427 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-14T18:03:03.427 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-14T18:03:03.428 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-14T18:03:03.428 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-14T18:03:03.428 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-14T18:03:03.428 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-14T18:03:03.428 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-14T18:03:03.428 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-14T18:03:03.428 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-14T18:03:03.429 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-14T18:03:03.429 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-14T18:03:03.429 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-14T18:03:03.429 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T18:03:03.429 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-14T18:03:03.429 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-14T18:03:03.429 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-14T18:03:03.430 INFO:teuthology.orchestra.run.smithi133.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-14T18:03:03.430 INFO:teuthology.orchestra.run.smithi133.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-14T18:03:03.430 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-14T18:03:03.430 INFO:teuthology.orchestra.run.smithi133.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-14T18:03:03.430 INFO:teuthology.orchestra.run.smithi133.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-14T18:03:03.430 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-14T18:03:03.430 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-14T18:03:03.430 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-14T18:03:03.431 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-14T18:03:03.431 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-14T18:03:03.431 INFO:teuthology.orchestra.run.smithi133.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-14T18:03:03.431 INFO:teuthology.orchestra.run.smithi133.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-14T18:03:03.431 INFO:teuthology.orchestra.run.smithi133.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-14T18:03:03.431 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-14T18:03:03.431 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-14T18:03:03.432 INFO:teuthology.orchestra.run.smithi133.stdout:Enabling module streams: 2023-07-14T18:03:03.432 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql 10 2023-07-14T18:03:03.432 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:03:03.432 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2023-07-14T18:03:03.432 INFO:teuthology.orchestra.run.smithi133.stdout:=========================================================================================================================== 2023-07-14T18:03:03.432 INFO:teuthology.orchestra.run.smithi133.stdout:Install 128 Packages 2023-07-14T18:03:03.432 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrade 3 Packages 2023-07-14T18:03:03.433 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:03:03.434 INFO:teuthology.orchestra.run.smithi161.stdout:(9/131): ceph-selinux-18.0.0-4940.g81c9387d.el8 243 kB/s | 24 kB 00:00 2023-07-14T18:03:03.434 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 206 M 2023-07-14T18:03:03.434 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2023-07-14T18:03:03.636 INFO:teuthology.orchestra.run.smithi161.stdout:(10/131): ceph-osd-18.0.0-4940.g81c9387d.el8.x8 9.1 MB/s | 18 MB 00:01 2023-07-14T18:03:03.771 INFO:teuthology.orchestra.run.smithi171.stdout:(2/131): ceph-fuse-18.0.0-4940.g81c9387d.el8.x8 729 kB/s | 850 kB 00:01 2023-07-14T18:03:03.778 INFO:teuthology.orchestra.run.smithi161.stdout:(11/131): cephfs-mirror-18.0.0-4940.g81c9387d.e 1.8 MB/s | 260 kB 00:00 2023-07-14T18:03:03.895 INFO:teuthology.orchestra.run.smithi161.stdout:(12/131): libcephfs-devel-18.0.0-4940.g81c9387d 266 kB/s | 31 kB 00:00 2023-07-14T18:03:03.955 INFO:teuthology.orchestra.run.smithi171.stdout:(3/131): ceph-immutable-object-cache-18.0.0-494 911 kB/s | 167 kB 00:00 2023-07-14T18:03:03.997 INFO:teuthology.orchestra.run.smithi133.stdout:(1/131): ceph-18.0.0-4940.g81c9387d.el8.x86_64. 16 kB/s | 6.4 kB 00:00 2023-07-14T18:03:04.045 INFO:teuthology.orchestra.run.smithi161.stdout:(13/131): libcephfs2-18.0.0-4940.g81c9387d.el8. 5.0 MB/s | 772 kB 00:00 2023-07-14T18:03:04.254 INFO:teuthology.orchestra.run.smithi161.stdout:(14/131): ceph-radosgw-18.0.0-4940.g81c9387d.el 7.4 MB/s | 14 MB 00:01 2023-07-14T18:03:04.279 INFO:teuthology.orchestra.run.smithi161.stdout:(15/131): libcephsqlite-18.0.0-4940.g81c9387d.e 579 kB/s | 135 kB 00:00 2023-07-14T18:03:04.338 INFO:teuthology.orchestra.run.smithi161.stdout:(16/131): librados-devel-18.0.0-4940.g81c9387d. 1.6 MB/s | 139 kB 00:00 2023-07-14T18:03:04.403 INFO:teuthology.orchestra.run.smithi161.stdout:(17/131): libradosstriper1-18.0.0-4940.g81c9387 3.8 MB/s | 473 kB 00:00 2023-07-14T18:03:04.514 INFO:teuthology.orchestra.run.smithi161.stdout:(18/131): python3-ceph-argparse-18.0.0-4940.g81 405 kB/s | 45 kB 00:00 2023-07-14T18:03:04.639 INFO:teuthology.orchestra.run.smithi161.stdout:(19/131): python3-ceph-common-18.0.0-4940.g81c9 980 kB/s | 122 kB 00:00 2023-07-14T18:03:04.781 INFO:teuthology.orchestra.run.smithi161.stdout:(20/131): python3-cephfs-18.0.0-4940.g81c9387d. 1.4 MB/s | 197 kB 00:00 2023-07-14T18:03:04.923 INFO:teuthology.orchestra.run.smithi161.stdout:(21/131): python3-rados-18.0.0-4940.g81c9387d.e 2.6 MB/s | 371 kB 00:00 2023-07-14T18:03:05.006 INFO:teuthology.orchestra.run.smithi161.stdout:(22/131): librgw2-18.0.0-4940.g81c9387d.el8.x86 10 MB/s | 6.8 MB 00:00 2023-07-14T18:03:05.032 INFO:teuthology.orchestra.run.smithi161.stdout:(23/131): python3-rbd-18.0.0-4940.g81c9387d.el8 3.3 MB/s | 364 kB 00:00 2023-07-14T18:03:05.107 INFO:teuthology.orchestra.run.smithi161.stdout:(24/131): python3-rgw-18.0.0-4940.g81c9387d.el8 1.0 MB/s | 107 kB 00:00 2023-07-14T18:03:05.133 INFO:teuthology.orchestra.run.smithi161.stdout:(25/131): rbd-fuse-18.0.0-4940.g81c9387d.el8.x8 932 kB/s | 94 kB 00:00 2023-07-14T18:03:05.283 INFO:teuthology.orchestra.run.smithi161.stdout:(26/131): rbd-nbd-18.0.0-4940.g81c9387d.el8.x86 1.2 MB/s | 179 kB 00:00 2023-07-14T18:03:05.416 INFO:teuthology.orchestra.run.smithi161.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4940.g 173 kB/s | 23 kB 00:00 2023-07-14T18:03:05.483 INFO:teuthology.orchestra.run.smithi161.stdout:(28/131): rbd-mirror-18.0.0-4940.g81c9387d.el8. 9.4 MB/s | 3.5 MB 00:00 2023-07-14T18:03:05.509 INFO:teuthology.orchestra.run.smithi161.stdout:(29/131): ceph-mgr-cephadm-18.0.0-4940.g81c9387 1.4 MB/s | 129 kB 00:00 2023-07-14T18:03:05.614 INFO:teuthology.orchestra.run.smithi133.stdout:(2/131): ceph-base-18.0.0-4940.g81c9387d.el8.x8 2.7 MB/s | 5.4 MB 00:02 2023-07-14T18:03:05.739 INFO:teuthology.orchestra.run.smithi133.stdout:(3/131): ceph-immutable-object-cache-18.0.0-494 1.3 MB/s | 167 kB 00:00 2023-07-14T18:03:05.784 INFO:teuthology.orchestra.run.smithi161.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4940.g81c93 5.7 MB/s | 1.7 MB 00:00 2023-07-14T18:03:05.813 INFO:teuthology.orchestra.run.smithi171.stdout:(4/131): ceph-mds-18.0.0-4940.g81c9387d.el8.x86 1.2 MB/s | 2.3 MB 00:01 2023-07-14T18:03:05.931 INFO:teuthology.orchestra.run.smithi133.stdout:(4/131): ceph-fuse-18.0.0-4940.g81c9387d.el8.x8 440 kB/s | 850 kB 00:01 2023-07-14T18:03:06.268 INFO:teuthology.orchestra.run.smithi161.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4940.g81 498 kB/s | 241 kB 00:00 2023-07-14T18:03:06.281 INFO:teuthology.orchestra.run.smithi133.stdout:(5/131): ceph-mds-18.0.0-4940.g81c9387d.el8.x86 4.2 MB/s | 2.3 MB 00:00 2023-07-14T18:03:06.368 INFO:teuthology.orchestra.run.smithi161.stdout:(32/131): ceph-mgr-rook-18.0.0-4940.g81c9387d.e 475 kB/s | 47 kB 00:00 2023-07-14T18:03:06.452 INFO:teuthology.orchestra.run.smithi161.stdout:(33/131): ceph-mgr-diskprediction-local-18.0.0- 7.9 MB/s | 7.4 MB 00:00 2023-07-14T18:03:06.477 INFO:teuthology.orchestra.run.smithi161.stdout:(34/131): ceph-prometheus-alerts-18.0.0-4940.g8 134 kB/s | 15 kB 00:00 2023-07-14T18:03:06.544 INFO:teuthology.orchestra.run.smithi161.stdout:(35/131): ceph-volume-18.0.0-4940.g81c9387d.el8 2.8 MB/s | 258 kB 00:00 2023-07-14T18:03:06.569 INFO:teuthology.orchestra.run.smithi161.stdout:(36/131): cephadm-18.0.0-4940.g81c9387d.el8.noa 1.8 MB/s | 165 kB 00:00 2023-07-14T18:03:06.628 INFO:teuthology.orchestra.run.smithi161.stdout:(37/131): cephfs-top-18.0.0-4940.g81c9387d.el8. 215 kB/s | 18 kB 00:00 2023-07-14T18:03:06.847 INFO:teuthology.orchestra.run.smithi171.stdout:(5/131): ceph-mgr-18.0.0-4940.g81c9387d.el8.x86 1.5 MB/s | 1.6 MB 00:01 2023-07-14T18:03:07.112 INFO:teuthology.orchestra.run.smithi161.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 122 kB/s | 59 kB 00:00 2023-07-14T18:03:07.190 INFO:teuthology.orchestra.run.smithi133.stdout:(6/131): ceph-mon-18.0.0-4940.g81c9387d.el8.x86 5.1 MB/s | 4.6 MB 00:00 2023-07-14T18:03:07.279 INFO:teuthology.orchestra.run.smithi161.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 432 kB/s | 306 kB 00:00 2023-07-14T18:03:07.304 INFO:teuthology.orchestra.run.smithi161.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 391 kB/s | 75 kB 00:00 2023-07-14T18:03:07.471 INFO:teuthology.orchestra.run.smithi161.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 262 kB/s | 44 kB 00:00 2023-07-14T18:03:07.546 INFO:teuthology.orchestra.run.smithi161.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.7 MB/s | 476 kB 00:00 2023-07-14T18:03:07.688 INFO:teuthology.orchestra.run.smithi161.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 233 kB/s | 33 kB 00:00 2023-07-14T18:03:07.747 INFO:teuthology.orchestra.run.smithi161.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 672 kB/s | 185 kB 00:00 2023-07-14T18:03:07.822 INFO:teuthology.orchestra.run.smithi161.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 122 kB/s | 16 kB 00:00 2023-07-14T18:03:07.939 INFO:teuthology.orchestra.run.smithi161.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 899 kB/s | 173 kB 00:00 2023-07-14T18:03:08.123 INFO:teuthology.orchestra.run.smithi161.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 443 kB/s | 81 kB 00:00 2023-07-14T18:03:08.132 INFO:teuthology.orchestra.run.smithi133.stdout:(7/131): ceph-mgr-18.0.0-4940.g81c9387d.el8.x86 722 kB/s | 1.6 MB 00:02 2023-07-14T18:03:08.148 INFO:teuthology.orchestra.run.smithi161.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.2 MB/s | 384 kB 00:00 2023-07-14T18:03:08.256 INFO:teuthology.orchestra.run.smithi161.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 83 kB/s | 11 kB 00:00 2023-07-14T18:03:08.273 INFO:teuthology.orchestra.run.smithi161.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 140 kB/s | 18 kB 00:00 2023-07-14T18:03:08.407 INFO:teuthology.orchestra.run.smithi161.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 365 kB/s | 49 kB 00:00 2023-07-14T18:03:08.532 INFO:teuthology.orchestra.run.smithi161.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 471 kB/s | 59 kB 00:00 2023-07-14T18:03:08.657 INFO:teuthology.orchestra.run.smithi161.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 482 kB/s | 60 kB 00:00 2023-07-14T18:03:08.700 INFO:teuthology.orchestra.run.smithi161.stdout:(54/131): python3-kubernetes-11.0.0-6.el8.noarc 2.6 MB/s | 1.2 MB 00:00 2023-07-14T18:03:08.808 INFO:teuthology.orchestra.run.smithi161.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.8 MB/s | 283 kB 00:00 2023-07-14T18:03:08.825 INFO:teuthology.orchestra.run.smithi161.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 124 kB/s | 16 kB 00:00 2023-07-14T18:03:08.914 INFO:teuthology.orchestra.run.smithi171.stdout:(6/131): ceph-base-18.0.0-4940.g81c9387d.el8.x8 823 kB/s | 5.4 MB 00:06 2023-07-14T18:03:08.951 INFO:teuthology.orchestra.run.smithi161.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 237 kB/s | 34 kB 00:00 2023-07-14T18:03:08.976 INFO:teuthology.orchestra.run.smithi161.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.3 MB/s | 196 kB 00:00 2023-07-14T18:03:09.110 INFO:teuthology.orchestra.run.smithi161.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 388 kB/s | 61 kB 00:00 2023-07-14T18:03:09.135 INFO:teuthology.orchestra.run.smithi161.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 118 kB/s | 19 kB 00:00 2023-07-14T18:03:09.252 INFO:teuthology.orchestra.run.smithi161.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 169 kB/s | 24 kB 00:00 2023-07-14T18:03:09.277 INFO:teuthology.orchestra.run.smithi161.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 206 kB/s | 29 kB 00:00 2023-07-14T18:03:09.377 INFO:teuthology.orchestra.run.smithi161.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 220 kB/s | 27 kB 00:00 2023-07-14T18:03:09.428 INFO:teuthology.orchestra.run.smithi161.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.7 MB/s | 255 kB 00:00 2023-07-14T18:03:09.536 INFO:teuthology.orchestra.run.smithi161.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.5 MB/s | 251 kB 00:00 2023-07-14T18:03:09.561 INFO:teuthology.orchestra.run.smithi161.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 459 kB/s | 61 kB 00:00 2023-07-14T18:03:09.670 INFO:teuthology.orchestra.run.smithi161.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 639 kB/s | 85 kB 00:00 2023-07-14T18:03:09.695 INFO:teuthology.orchestra.run.smithi161.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 172 kB/s | 23 kB 00:00 2023-07-14T18:03:09.721 INFO:teuthology.orchestra.run.smithi161.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-07-14T18:03:09.814 INFO:teuthology.orchestra.run.smithi171.stdout:(7/131): ceph-mon-18.0.0-4940.g81c9387d.el8.x86 1.6 MB/s | 4.6 MB 00:02 2023-07-14T18:03:10.029 INFO:teuthology.orchestra.run.smithi161.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 4.8 MB/s | 1.7 MB 00:00 2023-07-14T18:03:10.205 INFO:teuthology.orchestra.run.smithi161.stdout:(71/131): python3-scipy-1.0.0-21.module+el8.5.0 28 MB/s | 14 MB 00:00 2023-07-14T18:03:10.230 INFO:teuthology.orchestra.run.smithi161.stdout:(72/131): python3-pyasn1-modules-0.3.7-6.el8.no 551 kB/s | 110 kB 00:00 2023-07-14T18:03:10.289 INFO:teuthology.orchestra.run.smithi161.stdout:(73/131): python3-pytz-2017.2-9.el8.noarch.rpm 650 kB/s | 54 kB 00:00 2023-07-14T18:03:10.347 INFO:teuthology.orchestra.run.smithi161.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.2 MB/s | 259 kB 00:00 2023-07-14T18:03:10.397 INFO:teuthology.orchestra.run.smithi161.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 2.9 MB/s | 323 kB 00:00 2023-07-14T18:03:10.414 INFO:teuthology.orchestra.run.smithi161.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 580 kB/s | 39 kB 00:00 2023-07-14T18:03:10.523 INFO:teuthology.orchestra.run.smithi161.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 4.8 MB/s | 538 kB 00:00 2023-07-14T18:03:10.557 INFO:teuthology.orchestra.run.smithi161.stdout:(78/131): bison-3.0.4-10.el8.x86_64.rpm 4.2 MB/s | 688 kB 00:00 2023-07-14T18:03:10.632 INFO:teuthology.orchestra.run.smithi161.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.4 MB/s | 157 kB 00:00 2023-07-14T18:03:10.732 INFO:teuthology.orchestra.run.smithi161.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2023-07-14T18:03:10.766 INFO:teuthology.orchestra.run.smithi161.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 5.2 MB/s | 713 kB 00:00 2023-07-14T18:03:10.933 INFO:teuthology.orchestra.run.smithi161.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 159 kB/s | 27 kB 00:00 2023-07-14T18:03:11.000 INFO:teuthology.orchestra.run.smithi161.stdout:(83/131): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2023-07-14T18:03:11.015 INFO:teuthology.orchestra.run.smithi133.stdout:(8/131): ceph-osd-18.0.0-4940.g81c9387d.el8.x86 4.6 MB/s | 18 MB 00:03 2023-07-14T18:03:11.075 INFO:teuthology.orchestra.run.smithi161.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 2.2 MB/s | 320 kB 00:00 2023-07-14T18:03:11.100 INFO:teuthology.orchestra.run.smithi161.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 438 kB/s | 44 kB 00:00 2023-07-14T18:03:11.115 INFO:teuthology.orchestra.run.smithi133.stdout:(9/131): ceph-selinux-18.0.0-4940.g81c9387d.el8 244 kB/s | 24 kB 00:00 2023-07-14T18:03:11.359 INFO:teuthology.orchestra.run.smithi161.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 1.7 MB/s | 457 kB 00:00 2023-07-14T18:03:11.443 INFO:teuthology.orchestra.run.smithi161.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 10 MB/s | 3.7 MB 00:00 2023-07-14T18:03:11.477 INFO:teuthology.orchestra.run.smithi161.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.9 MB/s | 225 kB 00:00 2023-07-14T18:03:11.527 INFO:teuthology.orchestra.run.smithi161.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 767 kB/s | 64 kB 00:00 2023-07-14T18:03:11.585 INFO:teuthology.orchestra.run.smithi161.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2023-07-14T18:03:11.652 INFO:teuthology.orchestra.run.smithi161.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 160 kB/s | 20 kB 00:00 2023-07-14T18:03:11.711 INFO:teuthology.orchestra.run.smithi161.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 826 kB/s | 103 kB 00:00 2023-07-14T18:03:11.812 INFO:teuthology.orchestra.run.smithi161.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 4.4 MB/s | 709 kB 00:00 2023-07-14T18:03:11.920 INFO:teuthology.orchestra.run.smithi161.stdout:(94/131): openblas-threads-0.3.15-3.el8.x86_64. 23 MB/s | 4.9 MB 00:00 2023-07-14T18:03:11.954 INFO:teuthology.orchestra.run.smithi161.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 1.4 MB/s | 198 kB 00:00 2023-07-14T18:03:12.137 INFO:teuthology.orchestra.run.smithi161.stdout:(96/131): openblas-0.3.15-3.el8.x86_64.rpm 22 MB/s | 4.7 MB 00:00 2023-07-14T18:03:12.162 INFO:teuthology.orchestra.run.smithi161.stdout:(97/131): jq-1.6-3.el8.x86_64.rpm 969 kB/s | 202 kB 00:00 2023-07-14T18:03:12.296 INFO:teuthology.orchestra.run.smithi161.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.2 MB/s | 354 kB 00:00 2023-07-14T18:03:12.447 INFO:teuthology.orchestra.run.smithi161.stdout:(99/131): postgresql-server-10.21-2.module+el8. 18 MB/s | 5.1 MB 00:00 2023-07-14T18:03:12.480 INFO:teuthology.orchestra.run.smithi161.stdout:(100/131): postgresql-contrib-10.21-2.module+el 4.3 MB/s | 808 kB 00:00 2023-07-14T18:03:12.564 INFO:teuthology.orchestra.run.smithi161.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 13 MB/s | 1.5 MB 00:00 2023-07-14T18:03:12.606 INFO:teuthology.orchestra.run.smithi161.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 373 kB/s | 47 kB 00:00 2023-07-14T18:03:12.748 INFO:teuthology.orchestra.run.smithi161.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 1.1 MB/s | 200 kB 00:00 2023-07-14T18:03:12.773 INFO:teuthology.orchestra.run.smithi161.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 255 kB/s | 43 kB 00:00 2023-07-14T18:03:12.865 INFO:teuthology.orchestra.run.smithi161.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.8 MB/s | 220 kB 00:00 2023-07-14T18:03:12.957 INFO:teuthology.orchestra.run.smithi161.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 262 kB/s | 48 kB 00:00 2023-07-14T18:03:12.999 INFO:teuthology.orchestra.run.smithi161.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 651 kB/s | 87 kB 00:00 2023-07-14T18:03:13.074 INFO:teuthology.orchestra.run.smithi161.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 364 kB/s | 43 kB 00:00 2023-07-14T18:03:13.150 INFO:teuthology.orchestra.run.smithi161.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 725 kB/s | 109 kB 00:00 2023-07-14T18:03:13.192 INFO:teuthology.orchestra.run.smithi161.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.9 MB/s | 229 kB 00:00 2023-07-14T18:03:13.259 INFO:teuthology.orchestra.run.smithi161.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 5.0 MB/s | 560 kB 00:00 2023-07-14T18:03:13.309 INFO:teuthology.orchestra.run.smithi161.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 159 kB/s | 19 kB 00:00 2023-07-14T18:03:13.443 INFO:teuthology.orchestra.run.smithi161.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 844 kB/s | 155 kB 00:00 2023-07-14T18:03:13.476 INFO:teuthology.orchestra.run.smithi161.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 3.3 MB/s | 559 kB 00:00 2023-07-14T18:03:13.569 INFO:teuthology.orchestra.run.smithi161.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 143 kB/s | 18 kB 00:00 2023-07-14T18:03:13.602 INFO:teuthology.orchestra.run.smithi161.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.7 MB/s | 605 kB 00:00 2023-07-14T18:03:13.686 INFO:teuthology.orchestra.run.smithi161.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.0 MB/s | 238 kB 00:00 2023-07-14T18:03:13.795 INFO:teuthology.orchestra.run.smithi161.stdout:(118/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 12 MB/s | 2.3 MB 00:00 2023-07-14T18:03:13.828 INFO:teuthology.orchestra.run.smithi161.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 508 kB/s | 72 kB 00:00 2023-07-14T18:03:13.929 INFO:teuthology.orchestra.run.smithi161.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 619 kB/s | 83 kB 00:00 2023-07-14T18:03:13.954 INFO:teuthology.orchestra.run.smithi161.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 478 kB/s | 60 kB 00:00 2023-07-14T18:03:14.029 INFO:teuthology.orchestra.run.smithi161.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 979 kB/s | 98 kB 00:00 2023-07-14T18:03:14.088 INFO:teuthology.orchestra.run.smithi161.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 432 kB/s | 58 kB 00:00 2023-07-14T18:03:14.147 INFO:teuthology.orchestra.run.smithi161.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 330 kB/s | 39 kB 00:00 2023-07-14T18:03:14.272 INFO:teuthology.orchestra.run.smithi161.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 3.4 MB/s | 644 kB 00:00 2023-07-14T18:03:14.389 INFO:teuthology.orchestra.run.smithi161.stdout:(126/131): NetworkManager-initscripts-updown-1. 1.2 MB/s | 138 kB 00:00 2023-07-14T18:03:14.453 INFO:teuthology.orchestra.run.smithi161.stdout:(127/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 559 kB/s | 171 kB 00:00 2023-07-14T18:03:15.215 INFO:teuthology.orchestra.run.smithi161.stdout:(128/131): librados2-18.0.0-4940.g81c9387d.el8. 4.5 MB/s | 3.7 MB 00:00 2023-07-14T18:03:15.230 INFO:teuthology.orchestra.run.smithi171.stdout:(8/131): ceph-radosgw-18.0.0-4940.g81c9387d.el8 2.5 MB/s | 14 MB 00:05 2023-07-14T18:03:15.323 INFO:teuthology.orchestra.run.smithi171.stdout:(9/131): ceph-selinux-18.0.0-4940.g81c9387d.el8 265 kB/s | 24 kB 00:00 2023-07-14T18:03:15.365 INFO:teuthology.orchestra.run.smithi161.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 5.2 MB/s | 798 kB 00:00 2023-07-14T18:03:15.801 INFO:teuthology.orchestra.run.smithi161.stdout:(130/131): ceph-test-18.0.0-4940.g81c9387d.el8. 3.9 MB/s | 48 MB 00:12 2023-07-14T18:03:15.926 INFO:teuthology.orchestra.run.smithi161.stdout:(131/131): librbd1-18.0.0-4940.g81c9387d.el8.x8 2.5 MB/s | 3.6 MB 00:01 2023-07-14T18:03:15.929 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2023-07-14T18:03:15.930 INFO:teuthology.orchestra.run.smithi161.stdout:Total 13 MB/s | 206 MB 00:15 2023-07-14T18:03:16.257 INFO:teuthology.orchestra.run.smithi133.stdout:(10/131): ceph-radosgw-18.0.0-4940.g81c9387d.el 1.7 MB/s | 14 MB 00:08 2023-07-14T18:03:16.415 INFO:teuthology.orchestra.run.smithi133.stdout:(11/131): cephfs-mirror-18.0.0-4940.g81c9387d.e 1.6 MB/s | 260 kB 00:00 2023-07-14T18:03:16.507 INFO:teuthology.orchestra.run.smithi133.stdout:(12/131): libcephfs-devel-18.0.0-4940.g81c9387d 339 kB/s | 31 kB 00:00 2023-07-14T18:03:16.796 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2023-07-14T18:03:16.816 INFO:teuthology.orchestra.run.smithi133.stdout:(13/131): libcephfs2-18.0.0-4940.g81c9387d.el8. 2.4 MB/s | 772 kB 00:00 2023-07-14T18:03:16.908 INFO:teuthology.orchestra.run.smithi133.stdout:(14/131): libcephsqlite-18.0.0-4940.g81c9387d.e 1.4 MB/s | 135 kB 00:00 2023-07-14T18:03:16.917 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2023-07-14T18:03:16.917 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2023-07-14T18:03:17.016 INFO:teuthology.orchestra.run.smithi133.stdout:(15/131): librados-devel-18.0.0-4940.g81c9387d. 1.3 MB/s | 139 kB 00:00 2023-07-14T18:03:17.250 INFO:teuthology.orchestra.run.smithi133.stdout:(16/131): libradosstriper1-18.0.0-4940.g81c9387 2.0 MB/s | 473 kB 00:00 2023-07-14T18:03:18.044 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2023-07-14T18:03:18.048 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2023-07-14T18:03:18.681 INFO:teuthology.orchestra.run.smithi171.stdout:(10/131): ceph-common-18.0.0-4940.g81c9387d.el8 1.3 MB/s | 22 MB 00:16 2023-07-14T18:03:18.881 INFO:teuthology.orchestra.run.smithi171.stdout:(11/131): cephfs-mirror-18.0.0-4940.g81c9387d.e 1.3 MB/s | 260 kB 00:00 2023-07-14T18:03:18.973 INFO:teuthology.orchestra.run.smithi171.stdout:(12/131): libcephfs-devel-18.0.0-4940.g81c9387d 338 kB/s | 31 kB 00:00 2023-07-14T18:03:19.324 INFO:teuthology.orchestra.run.smithi171.stdout:(13/131): libcephfs2-18.0.0-4940.g81c9387d.el8. 2.2 MB/s | 772 kB 00:00 2023-07-14T18:03:19.334 INFO:teuthology.orchestra.run.smithi133.stdout:(17/131): ceph-common-18.0.0-4940.g81c9387d.el8 1.4 MB/s | 22 MB 00:15 2023-07-14T18:03:19.417 INFO:teuthology.orchestra.run.smithi133.stdout:(18/131): librgw2-18.0.0-4940.g81c9387d.el8.x86 3.1 MB/s | 6.8 MB 00:02 2023-07-14T18:03:19.424 INFO:teuthology.orchestra.run.smithi171.stdout:(14/131): libcephsqlite-18.0.0-4940.g81c9387d.e 1.3 MB/s | 135 kB 00:00 2023-07-14T18:03:19.442 INFO:teuthology.orchestra.run.smithi133.stdout:(19/131): python3-ceph-argparse-18.0.0-4940.g81 412 kB/s | 45 kB 00:00 2023-07-14T18:03:19.524 INFO:teuthology.orchestra.run.smithi171.stdout:(15/131): librados-devel-18.0.0-4940.g81c9387d. 1.4 MB/s | 139 kB 00:00 2023-07-14T18:03:19.858 INFO:teuthology.orchestra.run.smithi171.stdout:(16/131): ceph-osd-18.0.0-4940.g81c9387d.el8.x8 1.6 MB/s | 18 MB 00:10 2023-07-14T18:03:19.891 INFO:teuthology.orchestra.run.smithi171.stdout:(17/131): libradosstriper1-18.0.0-4940.g81c9387 1.3 MB/s | 473 kB 00:00 2023-07-14T18:03:19.935 INFO:teuthology.orchestra.run.smithi133.stdout:(20/131): ceph-test-18.0.0-4940.g81c9387d.el8.x 5.4 MB/s | 48 MB 00:08 2023-07-14T18:03:19.960 INFO:teuthology.orchestra.run.smithi133.stdout:(21/131): python3-ceph-common-18.0.0-4940.g81c9 226 kB/s | 122 kB 00:00 2023-07-14T18:03:19.983 INFO:teuthology.orchestra.run.smithi171.stdout:(18/131): python3-ceph-argparse-18.0.0-4940.g81 488 kB/s | 45 kB 00:00 2023-07-14T18:03:19.985 INFO:teuthology.orchestra.run.smithi133.stdout:(22/131): python3-cephfs-18.0.0-4940.g81c9387d. 363 kB/s | 197 kB 00:00 2023-07-14T18:03:20.067 INFO:teuthology.orchestra.run.smithi171.stdout:(19/131): python3-ceph-common-18.0.0-4940.g81c9 1.4 MB/s | 122 kB 00:00 2023-07-14T18:03:20.102 INFO:teuthology.orchestra.run.smithi133.stdout:(23/131): python3-rados-18.0.0-4940.g81c9387d.e 2.2 MB/s | 371 kB 00:00 2023-07-14T18:03:20.136 INFO:teuthology.orchestra.run.smithi133.stdout:(24/131): python3-rgw-18.0.0-4940.g81c9387d.el8 710 kB/s | 107 kB 00:00 2023-07-14T18:03:20.192 INFO:teuthology.orchestra.run.smithi171.stdout:(20/131): python3-cephfs-18.0.0-4940.g81c9387d. 1.5 MB/s | 197 kB 00:00 2023-07-14T18:03:20.202 INFO:teuthology.orchestra.run.smithi133.stdout:(25/131): python3-rbd-18.0.0-4940.g81c9387d.el8 1.5 MB/s | 364 kB 00:00 2023-07-14T18:03:20.220 INFO:teuthology.orchestra.run.smithi133.stdout:(26/131): rbd-fuse-18.0.0-4940.g81c9387d.el8.x8 800 kB/s | 94 kB 00:00 2023-07-14T18:03:20.342 INFO:teuthology.orchestra.run.smithi133.stdout:(27/131): rbd-nbd-18.0.0-4940.g81c9387d.el8.x86 1.3 MB/s | 179 kB 00:00 2023-07-14T18:03:20.367 INFO:teuthology.orchestra.run.smithi171.stdout:(21/131): python3-rados-18.0.0-4940.g81c9387d.e 2.1 MB/s | 371 kB 00:00 2023-07-14T18:03:20.370 INFO:teuthology.orchestra.run.smithi133.stdout:(28/131): ceph-grafana-dashboards-18.0.0-4940.g 154 kB/s | 23 kB 00:00 2023-07-14T18:03:20.429 INFO:teuthology.orchestra.run.smithi133.stdout:(29/131): ceph-mgr-cephadm-18.0.0-4940.g81c9387 1.5 MB/s | 129 kB 00:00 2023-07-14T18:03:20.534 INFO:teuthology.orchestra.run.smithi171.stdout:(22/131): python3-rbd-18.0.0-4940.g81c9387d.el8 2.1 MB/s | 364 kB 00:00 2023-07-14T18:03:20.635 INFO:teuthology.orchestra.run.smithi171.stdout:(23/131): python3-rgw-18.0.0-4940.g81c9387d.el8 1.0 MB/s | 107 kB 00:00 2023-07-14T18:03:20.735 INFO:teuthology.orchestra.run.smithi171.stdout:(24/131): rbd-fuse-18.0.0-4940.g81c9387d.el8.x8 939 kB/s | 94 kB 00:00 2023-07-14T18:03:20.796 INFO:teuthology.orchestra.run.smithi133.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4940.g81c93 4.0 MB/s | 1.7 MB 00:00 2023-07-14T18:03:20.912 INFO:teuthology.orchestra.run.smithi133.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4940.g81 2.0 MB/s | 241 kB 00:00 2023-07-14T18:03:21.004 INFO:teuthology.orchestra.run.smithi133.stdout:(32/131): ceph-mgr-rook-18.0.0-4940.g81c9387d.e 519 kB/s | 47 kB 00:00 2023-07-14T18:03:21.088 INFO:teuthology.orchestra.run.smithi133.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4940.g8 175 kB/s | 15 kB 00:00 2023-07-14T18:03:21.232 INFO:teuthology.orchestra.run.smithi133.stdout:(34/131): ceph-volume-18.0.0-4940.g81c9387d.el8 1.8 MB/s | 258 kB 00:00 2023-07-14T18:03:21.338 INFO:teuthology.orchestra.run.smithi133.stdout:(35/131): cephadm-18.0.0-4940.g81c9387d.el8.noa 1.5 MB/s | 165 kB 00:00 2023-07-14T18:03:21.343 INFO:teuthology.orchestra.run.smithi171.stdout:(25/131): librgw2-18.0.0-4940.g81c9387d.el8.x86 4.6 MB/s | 6.8 MB 00:01 2023-07-14T18:03:21.430 INFO:teuthology.orchestra.run.smithi133.stdout:(36/131): cephfs-top-18.0.0-4940.g81c9387d.el8. 196 kB/s | 18 kB 00:00 2023-07-14T18:03:21.452 INFO:teuthology.orchestra.run.smithi171.stdout:(26/131): rbd-nbd-18.0.0-4940.g81c9387d.el8.x86 1.6 MB/s | 179 kB 00:00 2023-07-14T18:03:21.544 INFO:teuthology.orchestra.run.smithi171.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4940.g 252 kB/s | 23 kB 00:00 2023-07-14T18:03:21.653 INFO:teuthology.orchestra.run.smithi171.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4940.g81c9387 1.2 MB/s | 129 kB 00:00 2023-07-14T18:03:21.853 INFO:teuthology.orchestra.run.smithi171.stdout:(29/131): rbd-mirror-18.0.0-4940.g81c9387d.el8. 3.2 MB/s | 3.5 MB 00:01 2023-07-14T18:03:21.897 INFO:teuthology.orchestra.run.smithi133.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 656 kB/s | 306 kB 00:00 2023-07-14T18:03:21.997 INFO:teuthology.orchestra.run.smithi133.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 589 kB/s | 59 kB 00:00 2023-07-14T18:03:22.011 INFO:teuthology.orchestra.run.smithi171.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4940.g81c93 4.8 MB/s | 1.7 MB 00:00 2023-07-14T18:03:22.098 INFO:teuthology.orchestra.run.smithi133.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 752 kB/s | 75 kB 00:00 2023-07-14T18:03:22.120 INFO:teuthology.orchestra.run.smithi171.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4940.g81 2.2 MB/s | 241 kB 00:00 2023-07-14T18:03:22.220 INFO:teuthology.orchestra.run.smithi171.stdout:(32/131): ceph-mgr-rook-18.0.0-4940.g81c9387d.e 474 kB/s | 47 kB 00:00 2023-07-14T18:03:22.256 INFO:teuthology.orchestra.run.smithi133.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.9 MB/s | 476 kB 00:00 2023-07-14T18:03:22.312 INFO:teuthology.orchestra.run.smithi171.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4940.g8 158 kB/s | 15 kB 00:00 2023-07-14T18:03:22.315 INFO:teuthology.orchestra.run.smithi133.stdout:(41/131): rbd-mirror-18.0.0-4940.g81c9387d.el8. 1.6 MB/s | 3.5 MB 00:02 2023-07-14T18:03:22.340 INFO:teuthology.orchestra.run.smithi133.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 523 kB/s | 44 kB 00:00 2023-07-14T18:03:22.424 INFO:teuthology.orchestra.run.smithi133.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 396 kB/s | 33 kB 00:00 2023-07-14T18:03:22.524 INFO:teuthology.orchestra.run.smithi133.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 163 kB/s | 16 kB 00:00 2023-07-14T18:03:22.666 INFO:teuthology.orchestra.run.smithi133.stdout:(45/131): ceph-mgr-diskprediction-local-18.0.0- 3.3 MB/s | 7.4 MB 00:02 2023-07-14T18:03:22.691 INFO:teuthology.orchestra.run.smithi133.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.0 MB/s | 173 kB 00:00 2023-07-14T18:03:22.796 INFO:teuthology.orchestra.run.smithi171.stdout:(34/131): ceph-test-18.0.0-4940.g81c9387d.el8.x 6.4 MB/s | 48 MB 00:07 2023-07-14T18:03:22.822 INFO:teuthology.orchestra.run.smithi171.stdout:(35/131): ceph-volume-18.0.0-4940.g81c9387d.el8 507 kB/s | 258 kB 00:00 2023-07-14T18:03:22.845 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2023-07-14T18:03:22.872 INFO:teuthology.orchestra.run.smithi171.stdout:(36/131): cephadm-18.0.0-4940.g81c9387d.el8.noa 2.2 MB/s | 165 kB 00:00 2023-07-14T18:03:22.931 INFO:teuthology.orchestra.run.smithi171.stdout:(37/131): cephfs-top-18.0.0-4940.g81c9387d.el8. 165 kB/s | 18 kB 00:00 2023-07-14T18:03:23.025 INFO:teuthology.orchestra.run.smithi133.stdout:(47/131): python3-beautifulsoup4-4.6.3-2.el8.1. 261 kB/s | 185 kB 00:00 2023-07-14T18:03:23.037 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-14T18:03:23.051 INFO:teuthology.orchestra.run.smithi133.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 226 kB/s | 81 kB 00:00 2023-07-14T18:03:23.092 INFO:teuthology.orchestra.run.smithi133.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 166 kB/s | 11 kB 00:00 2023-07-14T18:03:23.143 INFO:teuthology.orchestra.run.smithi133.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 192 kB/s | 18 kB 00:00 2023-07-14T18:03:23.218 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-14T18:03:23.235 INFO:teuthology.orchestra.run.smithi133.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 532 kB/s | 49 kB 00:00 2023-07-14T18:03:23.327 INFO:teuthology.orchestra.run.smithi133.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 643 kB/s | 59 kB 00:00 2023-07-14T18:03:23.347 INFO:teuthology.orchestra.run.smithi171.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 141 kB/s | 59 kB 00:00 2023-07-14T18:03:23.393 INFO:teuthology.orchestra.run.smithi133.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 3.9 MB/s | 1.2 MB 00:00 2023-07-14T18:03:23.437 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-07-14T18:03:23.444 INFO:teuthology.orchestra.run.smithi133.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 515 kB/s | 60 kB 00:00 2023-07-14T18:03:23.469 INFO:teuthology.orchestra.run.smithi133.stdout:(55/131): python3-cherrypy-18.4.0-1.el8.noarch. 479 kB/s | 384 kB 00:00 2023-07-14T18:03:23.494 INFO:teuthology.orchestra.run.smithi133.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.8 MB/s | 283 kB 00:00 2023-07-14T18:03:23.520 INFO:teuthology.orchestra.run.smithi133.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 207 kB/s | 16 kB 00:00 2023-07-14T18:03:23.553 INFO:teuthology.orchestra.run.smithi133.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 402 kB/s | 34 kB 00:00 2023-07-14T18:03:23.587 INFO:teuthology.orchestra.run.smithi133.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 2.1 MB/s | 196 kB 00:00 2023-07-14T18:03:23.590 INFO:teuthology.orchestra.run.smithi171.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 310 kB/s | 75 kB 00:00 2023-07-14T18:03:23.597 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-14T18:03:23.612 INFO:teuthology.orchestra.run.smithi133.stdout:(60/131): python3-rsa-4.8-1.el8.noarch.rpm 669 kB/s | 61 kB 00:00 2023-07-14T18:03:23.637 INFO:teuthology.orchestra.run.smithi133.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 223 kB/s | 19 kB 00:00 2023-07-14T18:03:23.663 INFO:teuthology.orchestra.run.smithi133.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 316 kB/s | 24 kB 00:00 2023-07-14T18:03:23.673 INFO:teuthology.orchestra.run.smithi171.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 382 kB/s | 306 kB 00:00 2023-07-14T18:03:23.696 INFO:teuthology.orchestra.run.smithi133.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 349 kB/s | 29 kB 00:00 2023-07-14T18:03:23.713 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-14T18:03:23.721 INFO:teuthology.orchestra.run.smithi133.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 329 kB/s | 27 kB 00:00 2023-07-14T18:03:23.763 INFO:teuthology.orchestra.run.smithi133.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 2.5 MB/s | 255 kB 00:00 2023-07-14T18:03:23.806 INFO:teuthology.orchestra.run.smithi133.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.3 MB/s | 251 kB 00:00 2023-07-14T18:03:23.824 INFO:teuthology.orchestra.run.smithi171.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 291 kB/s | 44 kB 00:00 2023-07-14T18:03:23.862 INFO:teuthology.orchestra.run.smithi133.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 436 kB/s | 61 kB 00:00 2023-07-14T18:03:23.889 INFO:teuthology.orchestra.run.smithi133.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 275 kB/s | 23 kB 00:00 2023-07-14T18:03:23.915 INFO:teuthology.orchestra.run.smithi133.stdout:(69/131): python3-webtest-2.0.33-1.el8.noarch.r 566 kB/s | 85 kB 00:00 2023-07-14T18:03:23.939 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-14T18:03:23.940 INFO:teuthology.orchestra.run.smithi133.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-07-14T18:03:24.073 INFO:teuthology.orchestra.run.smithi133.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 296 kB/s | 47 kB 00:00 2023-07-14T18:03:24.107 INFO:teuthology.orchestra.run.smithi133.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 1.2 MB/s | 200 kB 00:00 2023-07-14T18:03:24.149 INFO:teuthology.orchestra.run.smithi171.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 570 kB/s | 185 kB 00:00 2023-07-14T18:03:24.149 INFO:teuthology.orchestra.run.smithi133.stdout:(73/131): thrift-0.13.0-2.el8.x86_64.rpm 6.1 MB/s | 1.7 MB 00:00 2023-07-14T18:03:24.216 INFO:teuthology.orchestra.run.smithi133.stdout:(74/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 723 kB/s | 48 kB 00:00 2023-07-14T18:03:24.241 INFO:teuthology.orchestra.run.smithi133.stdout:(75/131): python3-jwt-1.6.1-2.el8.noarch.rpm 255 kB/s | 43 kB 00:00 2023-07-14T18:03:24.249 INFO:teuthology.orchestra.run.smithi171.stdout:(43/131): ceph-mgr-diskprediction-local-18.0.0- 3.1 MB/s | 7.4 MB 00:02 2023-07-14T18:03:24.266 INFO:teuthology.orchestra.run.smithi171.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 282 kB/s | 33 kB 00:00 2023-07-14T18:03:24.299 INFO:teuthology.orchestra.run.smithi133.stdout:(76/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.1 MB/s | 220 kB 00:00 2023-07-14T18:03:24.324 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-14T18:03:24.342 INFO:teuthology.orchestra.run.smithi133.stdout:(77/131): libsepol-devel-2.9-3.el8.x86_64.rpm 694 kB/s | 87 kB 00:00 2023-07-14T18:03:24.401 INFO:teuthology.orchestra.run.smithi133.stdout:(78/131): python3-requests-oauthlib-1.0.0-1.el8 268 kB/s | 43 kB 00:00 2023-07-14T18:03:24.426 INFO:teuthology.orchestra.run.smithi133.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 864 kB/s | 109 kB 00:00 2023-07-14T18:03:24.484 INFO:teuthology.orchestra.run.smithi133.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.6 MB/s | 229 kB 00:00 2023-07-14T18:03:24.518 INFO:teuthology.orchestra.run.smithi133.stdout:(81/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.7 MB/s | 560 kB 00:00 2023-07-14T18:03:24.525 INFO:teuthology.orchestra.run.smithi171.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 59 kB/s | 16 kB 00:00 2023-07-14T18:03:24.547 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librados2-2:18.0.0-4940.g81c9387d.el8.x86_64 6/134 2023-07-14T18:03:24.558 INFO:teuthology.orchestra.run.smithi171.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 592 kB/s | 173 kB 00:00 2023-07-14T18:03:24.560 INFO:teuthology.orchestra.run.smithi133.stdout:(82/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 140 kB/s | 19 kB 00:00 2023-07-14T18:03:24.644 INFO:teuthology.orchestra.run.smithi133.stdout:(83/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 214 kB/s | 18 kB 00:00 2023-07-14T18:03:24.669 INFO:teuthology.orchestra.run.smithi133.stdout:(84/131): python3-oauthlib-2.1.0-1.el8.noarch.r 842 kB/s | 155 kB 00:00 2023-07-14T18:03:24.675 INFO:teuthology.orchestra.run.smithi171.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 697 kB/s | 81 kB 00:00 2023-07-14T18:03:24.726 INFO:teuthology.orchestra.run.smithi171.stdout:(48/131): python3-asyncssh-2.7.0-2.el8.noarch.r 419 kB/s | 476 kB 00:01 2023-07-14T18:03:24.727 INFO:teuthology.orchestra.run.smithi133.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 2.6 MB/s | 559 kB 00:00 2023-07-14T18:03:24.775 INFO:teuthology.orchestra.run.smithi171.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2023-07-14T18:03:24.786 INFO:teuthology.orchestra.run.smithi133.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.0 MB/s | 238 kB 00:00 2023-07-14T18:03:24.820 INFO:teuthology.orchestra.run.smithi133.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 3.4 MB/s | 605 kB 00:00 2023-07-14T18:03:24.834 INFO:teuthology.orchestra.run.smithi171.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 162 kB/s | 18 kB 00:00 2023-07-14T18:03:24.895 INFO:teuthology.orchestra.run.smithi133.stdout:(88/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 14 MB/s | 2.3 MB 00:00 2023-07-14T18:03:24.920 INFO:teuthology.orchestra.run.smithi133.stdout:(89/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 541 kB/s | 72 kB 00:00 2023-07-14T18:03:24.943 INFO:teuthology.orchestra.run.smithi171.stdout:(51/131): python3-cherrypy-18.4.0-1.el8.noarch. 920 kB/s | 384 kB 00:00 2023-07-14T18:03:24.954 INFO:teuthology.orchestra.run.smithi133.stdout:(90/131): fuse-2.9.7-15.el8.x86_64.rpm 619 kB/s | 83 kB 00:00 2023-07-14T18:03:24.968 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:18.0.0-4940.g81c9387d.el8.x86_64 6/134 2023-07-14T18:03:25.029 INFO:teuthology.orchestra.run.smithi133.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 904 kB/s | 98 kB 00:00 2023-07-14T18:03:25.035 INFO:teuthology.orchestra.run.smithi171.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 243 kB/s | 49 kB 00:00 2023-07-14T18:03:25.052 INFO:teuthology.orchestra.run.smithi171.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 541 kB/s | 59 kB 00:00 2023-07-14T18:03:25.054 INFO:teuthology.orchestra.run.smithi133.stdout:(92/131): elfutils-libelf-devel-0.186-1.el8.x86 376 kB/s | 60 kB 00:00 2023-07-14T18:03:25.121 INFO:teuthology.orchestra.run.smithi133.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 420 kB/s | 39 kB 00:00 2023-07-14T18:03:25.155 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librbd1-2:18.0.0-4940.g81c9387d.el8.x86_64 7/134 2023-07-14T18:03:25.169 INFO:teuthology.orchestra.run.smithi171.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.4 MB/s | 283 kB 00:00 2023-07-14T18:03:25.180 INFO:teuthology.orchestra.run.smithi133.stdout:(94/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 255 kB/s | 58 kB 00:00 2023-07-14T18:03:25.213 INFO:teuthology.orchestra.run.smithi133.stdout:(95/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 4.0 MB/s | 644 kB 00:00 2023-07-14T18:03:25.219 INFO:teuthology.orchestra.run.smithi171.stdout:(55/131): python3-natsort-7.1.1-2.el8.noarch.rp 327 kB/s | 60 kB 00:00 2023-07-14T18:03:25.239 INFO:teuthology.orchestra.run.smithi133.stdout:(96/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 1.4 MB/s | 171 kB 00:00 2023-07-14T18:03:25.269 INFO:teuthology.orchestra.run.smithi171.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 155 kB/s | 16 kB 00:00 2023-07-14T18:03:25.279 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:18.0.0-4940.g81c9387d.el8.x86_64 7/134 2023-07-14T18:03:25.311 INFO:teuthology.orchestra.run.smithi171.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 366 kB/s | 34 kB 00:00 2023-07-14T18:03:25.331 INFO:teuthology.orchestra.run.smithi133.stdout:(97/131): NetworkManager-initscripts-updown-1.3 917 kB/s | 138 kB 00:00 2023-07-14T18:03:25.378 INFO:teuthology.orchestra.run.smithi171.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.8 MB/s | 196 kB 00:00 2023-07-14T18:03:25.381 INFO:teuthology.orchestra.run.smithi133.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 776 kB/s | 110 kB 00:00 2023-07-14T18:03:25.440 INFO:teuthology.orchestra.run.smithi133.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 499 kB/s | 54 kB 00:00 2023-07-14T18:03:25.463 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs2-2:18.0.0-4940.g81c9387d.el8.x86_64 8/134 2023-07-14T18:03:25.487 INFO:teuthology.orchestra.run.smithi171.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 173 kB/s | 19 kB 00:00 2023-07-14T18:03:25.506 INFO:teuthology.orchestra.run.smithi133.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.0 MB/s | 259 kB 00:00 2023-07-14T18:03:25.512 INFO:teuthology.orchestra.run.smithi171.stdout:(60/131): python3-rsa-4.8-1.el8.noarch.rpm 307 kB/s | 61 kB 00:00 2023-07-14T18:03:25.567 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephfs2-2:18.0.0-4940.g81c9387d.el8.x86_64 8/134 2023-07-14T18:03:25.587 INFO:teuthology.orchestra.run.smithi171.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 239 kB/s | 24 kB 00:00 2023-07-14T18:03:25.598 INFO:teuthology.orchestra.run.smithi133.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 2.0 MB/s | 323 kB 00:00 2023-07-14T18:03:25.613 INFO:teuthology.orchestra.run.smithi171.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 291 kB/s | 29 kB 00:00 2023-07-14T18:03:25.640 INFO:teuthology.orchestra.run.smithi133.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 291 kB/s | 39 kB 00:00 2023-07-14T18:03:25.688 INFO:teuthology.orchestra.run.smithi171.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 274 kB/s | 27 kB 00:00 2023-07-14T18:03:25.713 INFO:teuthology.orchestra.run.smithi171.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.5 MB/s | 255 kB 00:00 2023-07-14T18:03:25.752 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rados-2:18.0.0-4940.g81c9387d.el8.x86_64 9/134 2023-07-14T18:03:25.791 INFO:teuthology.orchestra.run.smithi133.stdout:(103/131): python3-scipy-1.0.0-21.module+el8.5. 24 MB/s | 14 MB 00:00 2023-07-14T18:03:25.813 INFO:teuthology.orchestra.run.smithi171.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 612 kB/s | 61 kB 00:00 2023-07-14T18:03:25.824 INFO:teuthology.orchestra.run.smithi133.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 2.9 MB/s | 538 kB 00:00 2023-07-14T18:03:25.858 INFO:teuthology.orchestra.run.smithi133.stdout:(105/131): bison-3.0.4-10.el8.x86_64.rpm 2.6 MB/s | 688 kB 00:00 2023-07-14T18:03:25.872 INFO:teuthology.orchestra.run.smithi171.stdout:(66/131): python3-kubernetes-11.0.0-6.el8.noarc 1.1 MB/s | 1.2 MB 00:01 2023-07-14T18:03:25.914 INFO:teuthology.orchestra.run.smithi171.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 851 kB/s | 85 kB 00:00 2023-07-14T18:03:25.941 INFO:teuthology.orchestra.run.smithi133.stdout:(106/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.0 MB/s | 157 kB 00:00 2023-07-14T18:03:25.968 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-14T18:03:25.973 INFO:teuthology.orchestra.run.smithi171.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 229 kB/s | 23 kB 00:00 2023-07-14T18:03:25.998 INFO:teuthology.orchestra.run.smithi171.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-07-14T18:03:26.050 INFO:teuthology.orchestra.run.smithi133.stdout:(107/131): oniguruma-6.8.2-2.el8.x86_64.rpm 832 kB/s | 187 kB 00:00 2023-07-14T18:03:26.092 INFO:teuthology.orchestra.run.smithi133.stdout:(108/131): automake-1.16.1-7.el8.noarch.rpm 3.0 MB/s | 713 kB 00:00 2023-07-14T18:03:26.150 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-14T18:03:26.173 INFO:teuthology.orchestra.run.smithi171.stdout:(70/131): python3-webob-1.8.5-1.el8.1.noarch.rp 518 kB/s | 251 kB 00:00 2023-07-14T18:03:26.209 INFO:teuthology.orchestra.run.smithi133.stdout:(109/131): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2023-07-14T18:03:26.215 INFO:teuthology.orchestra.run.smithi171.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 5.8 MB/s | 1.7 MB 00:00 2023-07-14T18:03:26.242 INFO:teuthology.orchestra.run.smithi133.stdout:(110/131): flex-2.6.1-9.el8.x86_64.rpm 2.1 MB/s | 320 kB 00:00 2023-07-14T18:03:26.260 INFO:teuthology.orchestra.run.smithi133.stdout:(111/131): python3-jsonpatch-1.21-2.el8.noarch. 127 kB/s | 27 kB 00:00 2023-07-14T18:03:26.326 INFO:teuthology.orchestra.run.smithi133.stdout:(112/131): python3-prettytable-0.7.2-14.el8.noa 376 kB/s | 44 kB 00:00 2023-07-14T18:03:26.332 INFO:teuthology.orchestra.run.smithi171.stdout:(72/131): python3-pyasn1-modules-0.3.7-6.el8.no 695 kB/s | 110 kB 00:00 2023-07-14T18:03:26.360 INFO:teuthology.orchestra.run.smithi133.stdout:(113/131): python3-werkzeug-0.12.2-4.el8.noarch 4.5 MB/s | 457 kB 00:00 2023-07-14T18:03:26.391 INFO:teuthology.orchestra.run.smithi171.stdout:(73/131): python3-pytz-2017.2-9.el8.noarch.rpm 320 kB/s | 54 kB 00:00 2023-07-14T18:03:26.416 INFO:teuthology.orchestra.run.smithi171.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.0 MB/s | 259 kB 00:00 2023-07-14T18:03:26.469 INFO:teuthology.orchestra.run.smithi133.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 17 MB/s | 3.7 MB 00:00 2023-07-14T18:03:26.494 INFO:teuthology.orchestra.run.smithi133.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 479 kB/s | 64 kB 00:00 2023-07-14T18:03:26.517 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-14T18:03:26.519 INFO:teuthology.orchestra.run.smithi133.stdout:(116/131): python3-numpy-f2py-1.14.3-10.el8.x86 1.1 MB/s | 225 kB 00:00 2023-07-14T18:03:26.583 INFO:teuthology.orchestra.run.smithi171.stdout:(75/131): python3-scipy-1.0.0-21.module+el8.5.0 23 MB/s | 14 MB 00:00 2023-07-14T18:03:26.608 INFO:teuthology.orchestra.run.smithi171.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 202 kB/s | 39 kB 00:00 2023-07-14T18:03:26.633 INFO:teuthology.orchestra.run.smithi171.stdout:(77/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.3 MB/s | 323 kB 00:00 2023-07-14T18:03:26.644 INFO:teuthology.orchestra.run.smithi133.stdout:(117/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 722 kB/s | 126 kB 00:00 2023-07-14T18:03:26.711 INFO:teuthology.orchestra.run.smithi133.stdout:(118/131): python3-jsonpointer-1.10-11.el8.noar 92 kB/s | 20 kB 00:00 2023-07-14T18:03:26.753 INFO:teuthology.orchestra.run.smithi171.stdout:(78/131): bison-3.0.4-10.el8.x86_64.rpm 4.0 MB/s | 688 kB 00:00 2023-07-14T18:03:26.762 INFO:teuthology.orchestra.run.smithi133.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 5.9 MB/s | 709 kB 00:00 2023-07-14T18:03:26.776 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-14T18:03:26.784 INFO:teuthology.orchestra.run.smithi171.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.0 MB/s | 157 kB 00:00 2023-07-14T18:03:26.787 INFO:teuthology.orchestra.run.smithi133.stdout:(120/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 387 kB/s | 103 kB 00:00 2023-07-14T18:03:26.818 INFO:teuthology.orchestra.run.smithi171.stdout:(80/131): python3-jinja2-2.10.1-3.el8.noarch.rp 2.5 MB/s | 538 kB 00:00 2023-07-14T18:03:26.884 INFO:teuthology.orchestra.run.smithi171.stdout:(81/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.4 MB/s | 187 kB 00:00 2023-07-14T18:03:26.895 INFO:teuthology.orchestra.run.smithi133.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 1.4 MB/s | 198 kB 00:00 2023-07-14T18:03:26.938 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephsqlite-2:18.0.0-4940.g81c9387d.el8.x86_64 14/134 2023-07-14T18:03:26.951 INFO:teuthology.orchestra.run.smithi171.stdout:(82/131): automake-1.16.1-7.el8.noarch.rpm 4.2 MB/s | 713 kB 00:00 2023-07-14T18:03:27.002 INFO:teuthology.orchestra.run.smithi171.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 227 kB/s | 27 kB 00:00 2023-07-14T18:03:27.046 INFO:teuthology.orchestra.run.smithi133.stdout:(122/131): openblas-threads-0.3.15-3.el8.x86_64 15 MB/s | 4.9 MB 00:00 2023-07-14T18:03:27.049 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4940.g81c9387d.el8.x86_64 14/134 2023-07-14T18:03:27.071 INFO:teuthology.orchestra.run.smithi133.stdout:(123/131): jq-1.6-3.el8.x86_64.rpm 1.1 MB/s | 202 kB 00:00 2023-07-14T18:03:27.102 INFO:teuthology.orchestra.run.smithi171.stdout:(84/131): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2023-07-14T18:03:27.127 INFO:teuthology.orchestra.run.smithi171.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 351 kB/s | 44 kB 00:00 2023-07-14T18:03:27.146 INFO:teuthology.orchestra.run.smithi133.stdout:(124/131): openblas-0.3.15-3.el8.x86_64.rpm 13 MB/s | 4.7 MB 00:00 2023-07-14T18:03:27.153 INFO:teuthology.orchestra.run.smithi171.stdout:(86/131): flex-2.6.1-9.el8.x86_64.rpm 1.6 MB/s | 320 kB 00:00 2023-07-14T18:03:27.180 INFO:teuthology.orchestra.run.smithi133.stdout:(125/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2023-07-14T18:03:27.303 INFO:teuthology.orchestra.run.smithi171.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 19 MB/s | 3.7 MB 00:00 2023-07-14T18:03:27.328 INFO:teuthology.orchestra.run.smithi171.stdout:(88/131): python3-werkzeug-0.12.2-4.el8.noarch. 2.2 MB/s | 457 kB 00:00 2023-07-14T18:03:27.347 INFO:teuthology.orchestra.run.smithi133.stdout:(126/131): postgresql-server-10.21-2.module+el8 18 MB/s | 5.1 MB 00:00 2023-07-14T18:03:27.353 INFO:teuthology.orchestra.run.smithi171.stdout:(89/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.1 MB/s | 225 kB 00:00 2023-07-14T18:03:27.389 INFO:teuthology.orchestra.run.smithi133.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 7.2 MB/s | 1.5 MB 00:00 2023-07-14T18:03:27.431 INFO:teuthology.orchestra.run.smithi133.stdout:(128/131): postgresql-contrib-10.21-2.module+el 2.8 MB/s | 808 kB 00:00 2023-07-14T18:03:27.462 INFO:teuthology.orchestra.run.smithi171.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 945 kB/s | 126 kB 00:00 2023-07-14T18:03:27.487 INFO:teuthology.orchestra.run.smithi171.stdout:(91/131): uuid-1.6.2-43.el8.x86_64.rpm 348 kB/s | 64 kB 00:00 2023-07-14T18:03:27.538 INFO:teuthology.orchestra.run.smithi171.stdout:(92/131): python3-jsonpointer-1.10-11.el8.noarc 108 kB/s | 20 kB 00:00 2023-07-14T18:03:27.596 INFO:teuthology.orchestra.run.smithi171.stdout:(93/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 772 kB/s | 103 kB 00:00 2023-07-14T18:03:27.638 INFO:teuthology.orchestra.run.smithi171.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 4.6 MB/s | 709 kB 00:00 2023-07-14T18:03:27.639 INFO:teuthology.orchestra.run.smithi133.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 3.7 MB/s | 798 kB 00:00 2023-07-14T18:03:27.672 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libradosstriper1-2:18.0.0-4940.g81c9387d.el8.x86 15/134 2023-07-14T18:03:27.689 INFO:teuthology.orchestra.run.smithi171.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 2.1 MB/s | 198 kB 00:00 2023-07-14T18:03:27.748 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4940.g81c9387d.el8.x86 15/134 2023-07-14T18:03:27.772 INFO:teuthology.orchestra.run.smithi171.stdout:(96/131): openblas-threads-0.3.15-3.el8.x86_64. 21 MB/s | 4.9 MB 00:00 2023-07-14T18:03:27.872 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-14T18:03:27.881 INFO:teuthology.orchestra.run.smithi171.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 20 MB/s | 4.7 MB 00:00 2023-07-14T18:03:27.906 INFO:teuthology.orchestra.run.smithi171.stdout:(98/131): jq-1.6-3.el8.x86_64.rpm 931 kB/s | 202 kB 00:00 2023-07-14T18:03:27.931 INFO:teuthology.orchestra.run.smithi171.stdout:(99/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.2 MB/s | 354 kB 00:00 2023-07-14T18:03:28.023 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-14T18:03:28.073 INFO:teuthology.orchestra.run.smithi171.stdout:(100/131): postgresql-server-10.21-2.module+el8 26 MB/s | 5.1 MB 00:00 2023-07-14T18:03:28.115 INFO:teuthology.orchestra.run.smithi171.stdout:(101/131): postgresql-contrib-10.21-2.module+el 3.8 MB/s | 808 kB 00:00 2023-07-14T18:03:28.157 INFO:teuthology.orchestra.run.smithi171.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 6.7 MB/s | 1.5 MB 00:00 2023-07-14T18:03:28.172 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-14T18:03:28.208 INFO:teuthology.orchestra.run.smithi171.stdout:(103/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 350 kB/s | 47 kB 00:00 2023-07-14T18:03:28.258 INFO:teuthology.orchestra.run.smithi171.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 424 kB/s | 43 kB 00:00 2023-07-14T18:03:28.283 INFO:teuthology.orchestra.run.smithi171.stdout:(105/131): libselinux-devel-2.9-5.el8.x86_64.rp 1.2 MB/s | 200 kB 00:00 2023-07-14T18:03:28.342 INFO:teuthology.orchestra.run.smithi171.stdout:(106/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.6 MB/s | 220 kB 00:00 2023-07-14T18:03:28.409 INFO:teuthology.orchestra.run.smithi171.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 695 kB/s | 87 kB 00:00 2023-07-14T18:03:28.434 INFO:teuthology.orchestra.run.smithi171.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 464 kB/s | 43 kB 00:00 2023-07-14T18:03:28.459 INFO:teuthology.orchestra.run.smithi171.stdout:(109/131): keyutils-libs-devel-1.5.10-9.el8.x86 239 kB/s | 48 kB 00:00 2023-07-14T18:03:28.518 INFO:teuthology.orchestra.run.smithi171.stdout:(110/131): python3-pycparser-2.14-14.el8.noarch 1.0 MB/s | 109 kB 00:00 2023-07-14T18:03:28.551 INFO:teuthology.orchestra.run.smithi171.stdout:(111/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.9 MB/s | 229 kB 00:00 2023-07-14T18:03:28.585 INFO:teuthology.orchestra.run.smithi171.stdout:(112/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.4 MB/s | 560 kB 00:00 2023-07-14T18:03:28.619 INFO:teuthology.orchestra.run.smithi171.stdout:(113/131): libaio-devel-0.3.112-1.el8.x86_64.rp 186 kB/s | 19 kB 00:00 2023-07-14T18:03:28.786 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-14T18:03:28.886 INFO:teuthology.orchestra.run.smithi171.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 464 kB/s | 155 kB 00:00 2023-07-14T18:03:28.911 INFO:teuthology.orchestra.run.smithi171.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 61 kB/s | 18 kB 00:00 2023-07-14T18:03:28.945 INFO:teuthology.orchestra.run.smithi171.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 1.5 MB/s | 559 kB 00:00 2023-07-14T18:03:28.971 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-14T18:03:28.987 INFO:teuthology.orchestra.run.smithi171.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.9 MB/s | 605 kB 00:00 2023-07-14T18:03:29.079 INFO:teuthology.orchestra.run.smithi171.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.4 MB/s | 238 kB 00:00 2023-07-14T18:03:29.137 INFO:teuthology.orchestra.run.smithi171.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 12 MB/s | 2.3 MB 00:00 2023-07-14T18:03:29.148 INFO:teuthology.orchestra.run.smithi133.stdout:(130/131): librados2-18.0.0-4940.g81c9387d.el8. 2.0 MB/s | 3.7 MB 00:01 2023-07-14T18:03:29.171 INFO:teuthology.orchestra.run.smithi171.stdout:(120/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 392 kB/s | 72 kB 00:00 2023-07-14T18:03:29.205 INFO:teuthology.orchestra.run.smithi171.stdout:(121/131): fuse-2.9.7-15.el8.x86_64.rpm 658 kB/s | 83 kB 00:00 2023-07-14T18:03:29.272 INFO:teuthology.orchestra.run.smithi171.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 983 kB/s | 98 kB 00:00 2023-07-14T18:03:29.339 INFO:teuthology.orchestra.run.smithi171.stdout:(123/131): elfutils-libelf-devel-0.186-1.el8.x8 298 kB/s | 60 kB 00:00 2023-07-14T18:03:29.439 INFO:teuthology.orchestra.run.smithi171.stdout:(124/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 247 kB/s | 58 kB 00:00 2023-07-14T18:03:29.498 INFO:teuthology.orchestra.run.smithi133.stdout:(131/131): librbd1-18.0.0-4940.g81c9387d.el8.x8 1.7 MB/s | 3.6 MB 00:02 2023-07-14T18:03:29.501 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2023-07-14T18:03:29.502 INFO:teuthology.orchestra.run.smithi133.stdout:Total 7.9 MB/s | 206 MB 00:26 2023-07-14T18:03:29.506 INFO:teuthology.orchestra.run.smithi171.stdout:(125/131): libcom_err-devel-1.45.6-4.el8.x86_64 165 kB/s | 39 kB 00:00 2023-07-14T18:03:29.556 INFO:teuthology.orchestra.run.smithi171.stdout:(126/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 2.9 MB/s | 644 kB 00:00 2023-07-14T18:03:29.569 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-14T18:03:29.582 INFO:teuthology.orchestra.run.smithi171.stdout:(127/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.2 MB/s | 171 kB 00:00 2023-07-14T18:03:29.615 INFO:teuthology.orchestra.run.smithi171.stdout:(128/131): NetworkManager-initscripts-updown-1. 1.2 MB/s | 138 kB 00:00 2023-07-14T18:03:29.771 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-07-14T18:03:29.774 INFO:teuthology.orchestra.run.smithi171.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 4.9 MB/s | 798 kB 00:00 2023-07-14T18:03:30.468 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-07-14T18:03:30.516 INFO:teuthology.orchestra.run.smithi171.stdout:(130/131): librados2-18.0.0-4940.g81c9387d.el8. 3.8 MB/s | 3.7 MB 00:00 2023-07-14T18:03:30.694 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-07-14T18:03:30.849 INFO:teuthology.orchestra.run.smithi171.stdout:(131/131): librbd1-18.0.0-4940.g81c9387d.el8.x8 2.9 MB/s | 3.6 MB 00:01 2023-07-14T18:03:30.852 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2023-07-14T18:03:30.852 INFO:teuthology.orchestra.run.smithi171.stdout:Total 7.1 MB/s | 206 MB 00:29 2023-07-14T18:03:30.906 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-07-14T18:03:31.134 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-07-14T18:03:31.154 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2023-07-14T18:03:31.287 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2023-07-14T18:03:31.287 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2023-07-14T18:03:31.336 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-07-14T18:03:31.520 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-07-14T18:03:31.676 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2023-07-14T18:03:31.740 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-14T18:03:31.796 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2023-07-14T18:03:31.797 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2023-07-14T18:03:31.894 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-common-2:18.0.0-4940.g81c9387d.el8. 26/134 2023-07-14T18:03:32.015 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-argparse-2:18.0.0-4940.g81c9387d.el 27/134 2023-07-14T18:03:32.887 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2023-07-14T18:03:32.892 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2023-07-14T18:03:32.921 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2023-07-14T18:03:32.925 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2023-07-14T18:03:32.963 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cephfs-2:18.0.0-4940.g81c9387d.el8.x86_6 28/134 2023-07-14T18:03:33.278 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-07-14T18:03:33.532 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-07-14T18:03:33.774 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-07-14T18:03:33.979 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-14T18:03:35.764 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-14T18:03:36.073 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-07-14T18:03:36.274 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-14T18:03:36.464 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-14T18:03:36.676 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rbd-2:18.0.0-4940.g81c9387d.el8.x86_64 36/134 2023-07-14T18:03:36.841 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librados-devel-2:18.0.0-4940.g81c9387d.el8.x86_6 37/134 2023-07-14T18:03:37.019 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-07-14T18:03:37.188 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-07-14T18:03:37.390 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-07-14T18:03:37.524 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-07-14T18:03:37.701 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2023-07-14T18:03:37.804 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-07-14T18:03:37.889 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-14T18:03:38.003 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-07-14T18:03:38.030 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2023-07-14T18:03:38.051 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-14T18:03:38.231 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-07-14T18:03:38.246 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-14T18:03:38.362 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-07-14T18:03:38.424 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-14T18:03:38.474 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-14T18:03:38.541 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-14T18:03:38.648 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-14T18:03:38.657 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-07-14T18:03:38.757 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-14T18:03:38.787 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-14T18:03:38.848 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-07-14T18:03:39.039 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-14T18:03:39.066 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-07-14T18:03:39.146 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-14T18:03:39.284 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-07-14T18:03:39.340 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librados2-2:18.0.0-4940.g81c9387d.el8.x86_64 6/134 2023-07-14T18:03:39.509 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-14T18:03:39.588 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-07-14T18:03:39.737 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:18.0.0-4940.g81c9387d.el8.x86_64 6/134 2023-07-14T18:03:39.770 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-07-14T18:03:39.780 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librados2-2:18.0.0-4940.g81c9387d.el8.x86_64 5/134 2023-07-14T18:03:39.916 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librbd1-2:18.0.0-4940.g81c9387d.el8.x86_64 7/134 2023-07-14T18:03:39.929 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-07-14T18:03:40.041 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:18.0.0-4940.g81c9387d.el8.x86_64 7/134 2023-07-14T18:03:40.158 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-07-14T18:03:40.224 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs2-2:18.0.0-4940.g81c9387d.el8.x86_64 8/134 2023-07-14T18:03:40.238 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:18.0.0-4940.g81c9387d.el8.x86_64 5/134 2023-07-14T18:03:40.307 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-07-14T18:03:40.321 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephfs2-2:18.0.0-4940.g81c9387d.el8.x86_64 8/134 2023-07-14T18:03:40.422 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librbd1-2:18.0.0-4940.g81c9387d.el8.x86_64 6/134 2023-07-14T18:03:40.499 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-07-14T18:03:40.500 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rados-2:18.0.0-4940.g81c9387d.el8.x86_64 9/134 2023-07-14T18:03:40.571 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:18.0.0-4940.g81c9387d.el8.x86_64 6/134 2023-07-14T18:03:40.719 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-14T18:03:40.789 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs2-2:18.0.0-4940.g81c9387d.el8.x86_64 7/134 2023-07-14T18:03:40.792 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-07-14T18:03:40.908 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephfs2-2:18.0.0-4940.g81c9387d.el8.x86_64 7/134 2023-07-14T18:03:40.952 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-14T18:03:41.041 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-07-14T18:03:41.103 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rados-2:18.0.0-4940.g81c9387d.el8.x86_64 8/134 2023-07-14T18:03:41.266 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-07-14T18:03:41.269 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-07-14T18:03:41.329 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-14T18:03:41.502 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-07-14T18:03:41.531 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-14T18:03:41.552 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-14T18:03:41.677 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-07-14T18:03:41.698 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephsqlite-2:18.0.0-4940.g81c9387d.el8.x86_64 14/134 2023-07-14T18:03:41.744 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-14T18:03:41.809 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4940.g81c9387d.el8.x86_64 14/134 2023-07-14T18:03:41.821 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-07-14T18:03:41.957 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libradosstriper1-2:18.0.0-4940.g81c9387d.el8.x86 15/134 2023-07-14T18:03:42.011 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-07-14T18:03:42.042 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4940.g81c9387d.el8.x86 15/134 2023-07-14T18:03:42.087 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-07-14T18:03:42.197 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-14T18:03:42.206 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-14T18:03:42.326 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-07-14T18:03:42.347 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-14T18:03:42.500 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-14T18:03:42.506 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-07-14T18:03:42.515 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-14T18:03:42.661 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-14T18:03:42.663 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephsqlite-2:18.0.0-4940.g81c9387d.el8.x86_64 14/134 2023-07-14T18:03:42.739 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-14T18:03:42.818 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4940.g81c9387d.el8.x86_64 14/134 2023-07-14T18:03:42.964 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libradosstriper1-2:18.0.0-4940.g81c9387d.el8.x86 15/134 2023-07-14T18:03:42.965 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-07-14T18:03:43.132 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-14T18:03:43.137 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4940.g81c9387d.el8.x86 15/134 2023-07-14T18:03:43.137 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-07-14T18:03:43.307 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-14T18:03:43.316 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-14T18:03:43.403 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-07-14T18:03:43.480 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-14T18:03:43.607 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-07-14T18:03:43.728 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-14T18:03:43.807 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-07-14T18:03:43.929 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-14T18:03:43.951 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-14T18:03:43.955 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-14T18:03:44.099 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-14T18:03:44.123 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-07-14T18:03:44.165 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-14T18:03:44.238 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-14T18:03:44.338 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-14T18:03:44.468 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-14T18:03:44.663 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-14T18:03:44.810 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-07-14T18:03:44.837 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-14T18:03:44.989 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-14T18:03:45.035 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-14T18:03:45.046 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-07-14T18:03:45.116 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-14T18:03:45.222 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-14T18:03:45.274 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-07-14T18:03:45.335 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-14T18:03:45.510 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-07-14T18:03:45.530 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-14T18:03:45.691 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-14T18:03:45.712 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-07-14T18:03:45.913 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-07-14T18:03:45.917 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-14T18:03:45.926 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-14T18:03:46.113 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-07-14T18:03:46.166 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-14T18:03:46.168 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-14T18:03:46.325 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-14T18:03:46.338 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-common-2:18.0.0-4940.g81c9387d.el8. 26/134 2023-07-14T18:03:46.467 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-argparse-2:18.0.0-4940.g81c9387d.el 27/134 2023-07-14T18:03:46.515 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-14T18:03:46.969 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-07-14T18:03:47.254 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-07-14T18:03:47.393 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cephfs-2:18.0.0-4940.g81c9387d.el8.x86_6 28/134 2023-07-14T18:03:47.464 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-07-14T18:03:47.584 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-14T18:03:47.695 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-07-14T18:03:47.733 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-14T18:03:47.848 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-14T18:03:47.950 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-07-14T18:03:47.954 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-common-2:18.0.0-4940.g81c9387d.el8. 26/134 2023-07-14T18:03:48.102 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-argparse-2:18.0.0-4940.g81c9387d.el 27/134 2023-07-14T18:03:48.184 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-07-14T18:03:48.203 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-14T18:03:48.373 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-14T18:03:48.378 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cephfs-2:18.0.0-4940.g81c9387d.el8.x86_6 28/134 2023-07-14T18:03:48.505 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-14T18:03:48.571 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-14T18:03:48.747 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-14T18:03:48.988 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-14T18:03:49.756 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-14T18:03:49.961 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librgw2-2:18.0.0-4940.g81c9387d.el8.x86_64 92/134 2023-07-14T18:03:50.028 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librgw2-2:18.0.0-4940.g81c9387d.el8.x86_64 92/134 2023-07-14T18:03:50.137 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-14T18:03:50.186 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rgw-2:18.0.0-4940.g81c9387d.el8.x86_64 93/134 2023-07-14T18:03:50.433 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-07-14T18:03:50.609 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-14T18:03:50.635 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-14T18:03:50.791 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-14T18:03:50.953 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rbd-2:18.0.0-4940.g81c9387d.el8.x86_64 36/134 2023-07-14T18:03:51.160 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librados-devel-2:18.0.0-4940.g81c9387d.el8.x86_6 37/134 2023-07-14T18:03:51.304 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-07-14T18:03:51.490 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-07-14T18:03:51.675 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-07-14T18:03:51.825 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-07-14T18:03:51.862 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-07-14T18:03:52.064 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-07-14T18:03:52.217 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-07-14T18:03:52.244 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-07-14T18:03:52.494 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-07-14T18:03:52.653 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-07-14T18:03:52.786 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-07-14T18:03:52.925 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-07-14T18:03:53.016 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-14T18:03:53.098 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-07-14T18:03:53.235 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-14T18:03:53.298 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-07-14T18:03:53.467 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rbd-2:18.0.0-4940.g81c9387d.el8.x86_64 36/134 2023-07-14T18:03:53.518 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-07-14T18:03:53.789 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 94/134 2023-07-14T18:03:53.863 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-07-14T18:03:53.962 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librados-devel-2:18.0.0-4940.g81c9387d.el8.x86_6 37/134 2023-07-14T18:03:53.986 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 94/134 2023-07-14T18:03:54.055 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-07-14T18:03:54.113 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-07-14T18:03:54.256 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-07-14T18:03:54.304 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-07-14T18:03:54.485 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-07-14T18:03:54.528 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-07-14T18:03:54.567 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 94/134 2023-07-14T18:03:54.604 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-07-14T18:03:54.667 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-07-14T18:03:54.829 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-07-14T18:03:54.859 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-base-2:18.0.0-4940.g81c9387d.el8.x86_64 95/134 2023-07-14T18:03:54.891 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-07-14T18:03:54.947 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-base-2:18.0.0-4940.g81c9387d.el8.x86_64 95/134 2023-07-14T18:03:55.093 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-selinux-2:18.0.0-4940.g81c9387d.el8.x86_64 96/134 2023-07-14T18:03:55.111 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-07-14T18:03:55.149 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-07-14T18:03:55.312 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-14T18:03:55.403 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-07-14T18:03:55.415 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-14T18:03:55.660 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-07-14T18:03:55.698 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-07-14T18:03:55.875 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-07-14T18:03:55.895 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-07-14T18:03:56.045 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-07-14T18:03:56.070 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-07-14T18:03:56.217 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-07-14T18:03:56.265 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-07-14T18:03:56.388 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-07-14T18:03:56.457 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-07-14T18:03:56.498 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-07-14T18:03:56.660 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-07-14T18:03:56.682 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-07-14T18:03:56.875 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-07-14T18:03:56.927 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-07-14T18:03:57.021 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-14T18:03:57.086 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-14T18:03:57.160 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-07-14T18:03:57.301 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-07-14T18:03:57.498 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-07-14T18:03:57.630 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-07-14T18:03:57.646 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-14T18:03:57.825 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-14T18:03:57.993 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-07-14T18:03:58.066 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-14T18:03:58.245 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-14T18:03:58.252 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-07-14T18:03:58.384 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-14T18:03:58.499 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-07-14T18:03:58.524 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-14T18:03:58.729 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-14T18:03:58.763 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-07-14T18:03:58.924 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-14T18:03:59.088 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-07-14T18:03:59.090 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-14T18:03:59.249 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-14T18:03:59.296 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-07-14T18:03:59.386 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-14T18:03:59.514 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-07-14T18:03:59.610 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-14T18:03:59.739 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-07-14T18:03:59.817 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-14T18:03:59.952 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-07-14T18:04:00.045 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-14T18:04:00.133 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-07-14T18:04:00.296 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-14T18:04:00.488 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-14T18:04:00.527 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-07-14T18:04:00.636 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-14T18:04:00.806 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-07-14T18:04:00.812 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-14T18:04:01.051 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-07-14T18:04:01.314 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-07-14T18:04:01.505 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-14T18:04:01.729 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-14T18:04:01.887 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-14T18:04:02.005 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-14T18:04:02.150 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-14T18:04:02.261 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-14T18:04:02.395 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-14T18:04:02.416 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-14T18:04:02.616 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-14T18:04:02.755 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-14T18:04:02.820 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-14T18:04:03.013 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-14T18:04:03.054 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-14T18:04:03.198 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-14T18:04:03.253 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-14T18:04:03.473 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-14T18:04:03.619 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-14T18:04:03.966 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-14T18:04:03.970 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-14T18:04:04.129 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librgw2-2:18.0.0-4940.g81c9387d.el8.x86_64 92/134 2023-07-14T18:04:04.189 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librgw2-2:18.0.0-4940.g81c9387d.el8.x86_64 92/134 2023-07-14T18:04:04.200 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-14T18:04:04.346 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rgw-2:18.0.0-4940.g81c9387d.el8.x86_64 93/134 2023-07-14T18:04:04.402 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-14T18:04:04.652 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-14T18:04:04.887 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-14T18:04:05.081 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-14T18:04:05.339 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-14T18:04:06.707 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-14T18:04:07.023 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-14T18:04:07.307 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-14T18:04:07.666 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-14T18:04:07.977 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-14T18:04:08.045 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 94/134 2023-07-14T18:04:08.246 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 94/134 2023-07-14T18:04:08.252 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-14T18:04:08.825 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 94/134 2023-07-14T18:04:09.104 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-14T18:04:09.118 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-base-2:18.0.0-4940.g81c9387d.el8.x86_64 95/134 2023-07-14T18:04:09.198 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-base-2:18.0.0-4940.g81c9387d.el8.x86_64 95/134 2023-07-14T18:04:09.293 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librgw2-2:18.0.0-4940.g81c9387d.el8.x86_64 92/134 2023-07-14T18:04:09.345 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-selinux-2:18.0.0-4940.g81c9387d.el8.x86_64 96/134 2023-07-14T18:04:09.419 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librgw2-2:18.0.0-4940.g81c9387d.el8.x86_64 92/134 2023-07-14T18:04:09.602 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rgw-2:18.0.0-4940.g81c9387d.el8.x86_64 93/134 2023-07-14T18:04:13.692 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4940.g81c9387d.el8.x86_64 96/134 2023-07-14T18:04:13.693 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /sys 2023-07-14T18:04:13.693 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /proc 2023-07-14T18:04:13.693 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /dev 2023-07-14T18:04:13.693 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /run 2023-07-14T18:04:13.693 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /mnt 2023-07-14T18:04:13.693 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /var/tmp 2023-07-14T18:04:13.694 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /home 2023-07-14T18:04:13.694 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /tmp 2023-07-14T18:04:13.694 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /dev 2023-07-14T18:04:13.694 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T18:04:14.008 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-volume-2:18.0.0-4940.g81c9387d.el8.noarch 97/134 2023-07-14T18:04:14.462 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 94/134 2023-07-14T18:04:14.701 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 94/134 2023-07-14T18:04:15.377 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 94/134 2023-07-14T18:04:15.600 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-volume-2:18.0.0-4940.g81c9387d.el8.noarch 97/134 2023-07-14T18:04:15.650 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-base-2:18.0.0-4940.g81c9387d.el8.x86_64 95/134 2023-07-14T18:04:15.781 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-base-2:18.0.0-4940.g81c9387d.el8.x86_64 95/134 2023-07-14T18:04:15.798 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-osd-2:18.0.0-4940.g81c9387d.el8.x86_64 98/134 2023-07-14T18:04:15.959 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-selinux-2:18.0.0-4940.g81c9387d.el8.x86_64 96/134 2023-07-14T18:04:16.018 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-osd-2:18.0.0-4940.g81c9387d.el8.x86_64 98/134 2023-07-14T18:04:16.207 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mds-2:18.0.0-4940.g81c9387d.el8.x86_64 99/134 2023-07-14T18:04:16.661 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mds-2:18.0.0-4940.g81c9387d.el8.x86_64 99/134 2023-07-14T18:04:16.825 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mon-2:18.0.0-4940.g81c9387d.el8.x86_64 100/134 2023-07-14T18:04:16.842 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mon-2:18.0.0-4940.g81c9387d.el8.x86_64 100/134 2023-07-14T18:04:18.425 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 101/134 2023-07-14T18:04:18.568 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 101/134 2023-07-14T18:04:18.680 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4940.g81c9387d.e 102/134 2023-07-14T18:04:18.862 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4940.g81c9387d. 103/134 2023-07-14T18:04:19.015 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4940.g81c9387d.el8.noa 104/134 2023-07-14T18:04:19.233 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4940.g81c9387d.el8.noa 104/134 2023-07-14T18:04:19.389 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4940.g81c9387d.el8.n 105/134 2023-07-14T18:04:20.351 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4940.g81c9387d.el8.n 105/134 2023-07-14T18:04:20.490 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4940.g81c 106/134 2023-07-14T18:04:20.581 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4940.g81c 106/134 2023-07-14T18:04:20.926 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4940.g81c9387d.el 107/134 2023-07-14T18:04:21.166 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-2:18.0.0-4940.g81c9387d.el8.x86_64 108/134 2023-07-14T18:04:21.201 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4940.g81c9387d.el8.x86_64 108/134 2023-07-14T18:04:21.332 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-rook-2:18.0.0-4940.g81c9387d.el8.noarch 109/134 2023-07-14T18:04:21.343 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4940.g81c9387d.el8.noarch 109/134 2023-07-14T18:04:22.627 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-2:18.0.0-4940.g81c9387d.el8.x86_64 110/134 2023-07-14T18:04:22.840 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-radosgw-2:18.0.0-4940.g81c9387d.el8.x86_64 111/134 2023-07-14T18:04:22.875 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4940.g81c9387d.el8.x86_64 111/134 2023-07-14T18:04:23.032 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4940.g81c93 112/134 2023-07-14T18:04:23.074 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4940.g81c93 112/134 2023-07-14T18:04:23.232 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 113/134 2023-07-14T18:04:23.584 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 113/134 2023-07-14T18:04:23.773 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 114/134 2023-07-14T18:04:28.172 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4940.g81c9387d.el8.x86_64 96/134 2023-07-14T18:04:28.172 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /sys 2023-07-14T18:04:28.172 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /proc 2023-07-14T18:04:28.173 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /dev 2023-07-14T18:04:28.173 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /run 2023-07-14T18:04:28.173 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /mnt 2023-07-14T18:04:28.173 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /var/tmp 2023-07-14T18:04:28.173 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /home 2023-07-14T18:04:28.174 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /tmp 2023-07-14T18:04:28.174 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /dev 2023-07-14T18:04:28.174 INFO:teuthology.orchestra.run.smithi171.stdout: 2023-07-14T18:04:28.326 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 114/134 2023-07-14T18:04:28.551 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-volume-2:18.0.0-4940.g81c9387d.el8.noarch 97/134 2023-07-14T18:04:28.604 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-test-2:18.0.0-4940.g81c9387d.el8.x86_64 115/134 2023-07-14T18:04:28.756 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-07-14T18:04:28.972 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-07-14T18:04:29.892 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-07-14T18:04:30.139 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-volume-2:18.0.0-4940.g81c9387d.el8.noarch 97/134 2023-07-14T18:04:30.246 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-07-14T18:04:30.332 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-osd-2:18.0.0-4940.g81c9387d.el8.x86_64 98/134 2023-07-14T18:04:30.450 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 119/134 2023-07-14T18:04:30.552 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-osd-2:18.0.0-4940.g81c9387d.el8.x86_64 98/134 2023-07-14T18:04:30.643 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-14T18:04:30.743 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mds-2:18.0.0-4940.g81c9387d.el8.x86_64 99/134 2023-07-14T18:04:30.810 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs-devel-2:18.0.0-4940.g81c9387d.el8.x86_ 121/134 2023-07-14T18:04:30.957 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-14T18:04:31.199 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mds-2:18.0.0-4940.g81c9387d.el8.x86_64 99/134 2023-07-14T18:04:31.393 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mon-2:18.0.0-4940.g81c9387d.el8.x86_64 100/134 2023-07-14T18:04:31.409 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mon-2:18.0.0-4940.g81c9387d.el8.x86_64 100/134 2023-07-14T18:04:33.040 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 101/134 2023-07-14T18:04:33.086 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T18:04:33.201 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 101/134 2023-07-14T18:04:33.323 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4940.g81c9387d.e 102/134 2023-07-14T18:04:33.339 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T18:04:33.356 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T18:04:33.497 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-top-2:18.0.0-4940.g81c9387d.el8.noarch 124/134 2023-07-14T18:04:33.515 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4940.g81c9387d. 103/134 2023-07-14T18:04:33.708 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4940.g81c9387d.el8.noa 104/134 2023-07-14T18:04:33.780 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 125/134 2023-07-14T18:04:33.926 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4940.g81c9387d.el8.noa 104/134 2023-07-14T18:04:34.109 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-nbd-2:18.0.0-4940.g81c9387d.el8.x86_64 126/134 2023-07-14T18:04:34.204 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4940.g81c9387d.el8.n 105/134 2023-07-14T18:04:34.415 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-14T18:04:34.563 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-14T18:04:34.572 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-14T18:04:34.745 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-07-14T18:04:34.946 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-07-14T18:04:35.027 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-07-14T18:04:35.167 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4940.g81c9387d.el8.n 105/134 2023-07-14T18:04:35.213 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-07-14T18:04:35.226 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-07-14T18:04:35.226 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T18:04:35.334 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4940.g81c 106/134 2023-07-14T18:04:35.425 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T18:04:35.425 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T18:04:35.425 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4940.g81c 106/134 2023-07-14T18:04:35.654 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T18:04:35.777 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4940.g81c9387d.el 107/134 2023-07-14T18:04:35.878 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T18:04:35.977 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-2:18.0.0-4940.g81c9387d.el8.x86_64 108/134 2023-07-14T18:04:36.012 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4940.g81c9387d.el8.x86_64 108/134 2023-07-14T18:04:36.192 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-rook-2:18.0.0-4940.g81c9387d.el8.noarch 109/134 2023-07-14T18:04:36.203 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4940.g81c9387d.el8.noarch 109/134 2023-07-14T18:04:36.324 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4940.g81c9387d.el8.x86_64 96/134 2023-07-14T18:04:36.324 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /sys 2023-07-14T18:04:36.325 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /proc 2023-07-14T18:04:36.325 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /dev 2023-07-14T18:04:36.325 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /run 2023-07-14T18:04:36.325 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /mnt 2023-07-14T18:04:36.325 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /var/tmp 2023-07-14T18:04:36.325 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /home 2023-07-14T18:04:36.326 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /tmp 2023-07-14T18:04:36.326 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /dev 2023-07-14T18:04:36.326 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:04:36.417 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T18:04:36.417 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-2:18.0.0-4940.g81c9387d.el8.x86_64 1/134 2023-07-14T18:04:36.418 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-base-2:18.0.0-4940.g81c9387d.el8.x86_64 2/134 2023-07-14T18:04:36.418 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 3/134 2023-07-14T18:04:36.418 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 4/134 2023-07-14T18:04:36.418 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4940.g81c93 5/134 2023-07-14T18:04:36.418 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mds-2:18.0.0-4940.g81c9387d.el8.x86_64 6/134 2023-07-14T18:04:36.418 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-2:18.0.0-4940.g81c9387d.el8.x86_64 7/134 2023-07-14T18:04:36.418 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mon-2:18.0.0-4940.g81c9387d.el8.x86_64 8/134 2023-07-14T18:04:36.419 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-osd-2:18.0.0-4940.g81c9387d.el8.x86_64 9/134 2023-07-14T18:04:36.419 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-radosgw-2:18.0.0-4940.g81c9387d.el8.x86_64 10/134 2023-07-14T18:04:36.419 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-selinux-2:18.0.0-4940.g81c9387d.el8.x86_64 11/134 2023-07-14T18:04:36.419 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-test-2:18.0.0-4940.g81c9387d.el8.x86_64 12/134 2023-07-14T18:04:36.419 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 13/134 2023-07-14T18:04:36.419 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs-devel-2:18.0.0-4940.g81c9387d.el8.x86_ 14/134 2023-07-14T18:04:36.419 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs2-2:18.0.0-4940.g81c9387d.el8.x86_64 15/134 2023-07-14T18:04:36.419 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephsqlite-2:18.0.0-4940.g81c9387d.el8.x86_64 16/134 2023-07-14T18:04:36.420 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados-devel-2:18.0.0-4940.g81c9387d.el8.x86_6 17/134 2023-07-14T18:04:36.420 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libradosstriper1-2:18.0.0-4940.g81c9387d.el8.x86 18/134 2023-07-14T18:04:36.420 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librgw2-2:18.0.0-4940.g81c9387d.el8.x86_64 19/134 2023-07-14T18:04:36.420 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4940.g81c9387d.el 20/134 2023-07-14T18:04:36.420 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-common-2:18.0.0-4940.g81c9387d.el8. 21/134 2023-07-14T18:04:36.420 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cephfs-2:18.0.0-4940.g81c9387d.el8.x86_6 22/134 2023-07-14T18:04:36.420 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rados-2:18.0.0-4940.g81c9387d.el8.x86_64 23/134 2023-07-14T18:04:36.421 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rbd-2:18.0.0-4940.g81c9387d.el8.x86_64 24/134 2023-07-14T18:04:36.421 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rgw-2:18.0.0-4940.g81c9387d.el8.x86_64 25/134 2023-07-14T18:04:36.421 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 26/134 2023-07-14T18:04:36.421 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 27/134 2023-07-14T18:04:36.428 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-nbd-2:18.0.0-4940.g81c9387d.el8.x86_64 28/134 2023-07-14T18:04:36.428 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4940.g81c9387d. 29/134 2023-07-14T18:04:36.428 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4940.g81c9387d.el8.noa 30/134 2023-07-14T18:04:36.428 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4940.g81c9387d.el8.n 31/134 2023-07-14T18:04:36.428 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4940.g81c 32/134 2023-07-14T18:04:36.428 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4940.g81c9387d.el 33/134 2023-07-14T18:04:36.429 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4940.g81c9387d.el8.noarch 34/134 2023-07-14T18:04:36.429 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4940.g81c9387d.e 35/134 2023-07-14T18:04:36.429 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-volume-2:18.0.0-4940.g81c9387d.el8.noarch 36/134 2023-07-14T18:04:36.429 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 37/134 2023-07-14T18:04:36.429 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-top-2:18.0.0-4940.g81c9387d.el8.noarch 38/134 2023-07-14T18:04:36.429 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-14T18:04:36.429 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-14T18:04:36.430 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-14T18:04:36.430 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-14T18:04:36.430 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-14T18:04:36.430 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-14T18:04:36.430 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-14T18:04:36.430 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-14T18:04:36.430 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-14T18:04:36.430 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-14T18:04:36.431 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-14T18:04:36.431 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-14T18:04:36.431 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-14T18:04:36.431 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-14T18:04:36.431 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-14T18:04:36.431 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-14T18:04:36.431 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-14T18:04:36.431 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-14T18:04:36.432 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-14T18:04:36.432 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-14T18:04:36.432 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-14T18:04:36.432 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-14T18:04:36.432 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-14T18:04:36.432 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-14T18:04:36.432 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-14T18:04:36.433 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-14T18:04:36.433 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-14T18:04:36.433 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-14T18:04:36.433 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-14T18:04:36.433 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-14T18:04:36.433 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-14T18:04:36.433 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-14T18:04:36.434 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-14T18:04:36.434 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-07-14T18:04:36.434 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-07-14T18:04:36.434 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-07-14T18:04:36.434 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-07-14T18:04:36.434 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-07-14T18:04:36.434 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-07-14T18:04:36.434 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-07-14T18:04:36.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-07-14T18:04:36.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-07-14T18:04:36.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-07-14T18:04:36.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-07-14T18:04:36.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-07-14T18:04:36.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-07-14T18:04:36.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-07-14T18:04:36.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-07-14T18:04:36.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-07-14T18:04:36.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-07-14T18:04:36.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-07-14T18:04:36.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-07-14T18:04:36.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-07-14T18:04:36.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-07-14T18:04:36.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-07-14T18:04:36.437 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-07-14T18:04:36.437 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-07-14T18:04:36.437 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-07-14T18:04:36.437 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-07-14T18:04:36.437 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-07-14T18:04:36.437 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-07-14T18:04:36.437 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-07-14T18:04:36.438 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-07-14T18:04:36.438 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-07-14T18:04:36.438 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-07-14T18:04:36.438 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-07-14T18:04:36.438 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-07-14T18:04:36.438 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-07-14T18:04:36.438 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-07-14T18:04:36.438 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-07-14T18:04:36.439 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-07-14T18:04:36.439 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-07-14T18:04:36.439 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-07-14T18:04:36.439 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-07-14T18:04:36.439 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-07-14T18:04:36.439 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-07-14T18:04:36.439 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-07-14T18:04:36.439 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-07-14T18:04:36.440 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-07-14T18:04:36.440 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-07-14T18:04:36.440 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-07-14T18:04:36.440 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-14T18:04:36.440 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-07-14T18:04:36.440 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-14T18:04:36.440 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-07-14T18:04:36.440 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-07-14T18:04:36.441 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-07-14T18:04:36.441 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-07-14T18:04:36.441 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-07-14T18:04:36.441 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-14T18:04:36.441 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:18.0.0-4940.g81c9387d.el8.x86_64 129/134 2023-07-14T18:04:36.441 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-14T18:04:36.441 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:18.0.0-4940.g81c9387d.el8.x86_64 131/134 2023-07-14T18:04:36.442 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T18:04:36.442 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-14T18:04:36.707 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-volume-2:18.0.0-4940.g81c9387d.el8.noarch 97/134 2023-07-14T18:04:37.504 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-2:18.0.0-4940.g81c9387d.el8.x86_64 110/134 2023-07-14T18:04:37.702 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-radosgw-2:18.0.0-4940.g81c9387d.el8.x86_64 111/134 2023-07-14T18:04:37.737 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4940.g81c9387d.el8.x86_64 111/134 2023-07-14T18:04:37.892 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4940.g81c93 112/134 2023-07-14T18:04:37.934 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4940.g81c93 112/134 2023-07-14T18:04:38.100 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 113/134 2023-07-14T18:04:38.452 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 113/134 2023-07-14T18:04:38.454 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-volume-2:18.0.0-4940.g81c9387d.el8.noarch 97/134 2023-07-14T18:04:38.608 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 114/134 2023-07-14T18:04:38.688 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-osd-2:18.0.0-4940.g81c9387d.el8.x86_64 98/134 2023-07-14T18:04:38.900 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T18:04:38.900 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:01:50 ago on Fri 14 Jul 2023 06:02:48 PM UTC. 2023-07-14T18:04:38.958 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-osd-2:18.0.0-4940.g81c9387d.el8.x86_64 98/134 2023-07-14T18:04:39.206 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mds-2:18.0.0-4940.g81c9387d.el8.x86_64 99/134 2023-07-14T18:04:39.735 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mds-2:18.0.0-4940.g81c9387d.el8.x86_64 99/134 2023-07-14T18:04:39.956 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mon-2:18.0.0-4940.g81c9387d.el8.x86_64 100/134 2023-07-14T18:04:39.998 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mon-2:18.0.0-4940.g81c9387d.el8.x86_64 100/134 2023-07-14T18:04:42.690 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 101/134 2023-07-14T18:04:42.899 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 101/134 2023-07-14T18:04:43.013 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4940.g81c9387d.e 102/134 2023-07-14T18:04:43.181 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 114/134 2023-07-14T18:04:43.211 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4940.g81c9387d. 103/134 2023-07-14T18:04:43.421 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4940.g81c9387d.el8.noa 104/134 2023-07-14T18:04:43.456 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-test-2:18.0.0-4940.g81c9387d.el8.x86_64 115/134 2023-07-14T18:04:43.615 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-07-14T18:04:43.687 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4940.g81c9387d.el8.noa 104/134 2023-07-14T18:04:43.816 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-07-14T18:04:43.904 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4940.g81c9387d.el8.n 105/134 2023-07-14T18:04:44.693 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-07-14T18:04:44.997 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-07-14T18:04:45.016 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4940.g81c9387d.el8.n 105/134 2023-07-14T18:04:45.219 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 119/134 2023-07-14T18:04:45.221 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4940.g81c 106/134 2023-07-14T18:04:45.350 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4940.g81c 106/134 2023-07-14T18:04:45.411 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-14T18:04:45.578 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs-devel-2:18.0.0-4940.g81c9387d.el8.x86_ 121/134 2023-07-14T18:04:45.725 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-14T18:04:45.753 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4940.g81c9387d.el 107/134 2023-07-14T18:04:46.035 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-2:18.0.0-4940.g81c9387d.el8.x86_64 108/134 2023-07-14T18:04:46.100 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4940.g81c9387d.el8.x86_64 108/134 2023-07-14T18:04:46.304 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-rook-2:18.0.0-4940.g81c9387d.el8.noarch 109/134 2023-07-14T18:04:46.333 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4940.g81c9387d.el8.noarch 109/134 2023-07-14T18:04:47.776 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-2:18.0.0-4940.g81c9387d.el8.x86_64 110/134 2023-07-14T18:04:47.865 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T18:04:48.006 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-radosgw-2:18.0.0-4940.g81c9387d.el8.x86_64 111/134 2023-07-14T18:04:48.069 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4940.g81c9387d.el8.x86_64 111/134 2023-07-14T18:04:48.082 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T18:04:48.099 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T18:04:48.255 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4940.g81c93 112/134 2023-07-14T18:04:48.257 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-top-2:18.0.0-4940.g81c9387d.el8.noarch 124/134 2023-07-14T18:04:48.327 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4940.g81c93 112/134 2023-07-14T18:04:48.538 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 113/134 2023-07-14T18:04:48.548 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 125/134 2023-07-14T18:04:48.809 INFO:teuthology.orchestra.run.smithi161.stdout:Installed products updated. 2023-07-14T18:04:48.867 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-nbd-2:18.0.0-4940.g81c9387d.el8.x86_64 126/134 2023-07-14T18:04:48.956 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 113/134 2023-07-14T18:04:49.024 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T18:04:49.024 INFO:teuthology.orchestra.run.smithi161.stdout:Upgraded: 2023-07-14T18:04:49.024 INFO:teuthology.orchestra.run.smithi161.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-14T18:04:49.024 INFO:teuthology.orchestra.run.smithi161.stdout: librados2-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.025 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.025 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2023-07-14T18:04:49.025 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-14T18:04:49.025 INFO:teuthology.orchestra.run.smithi161.stdout: automake-1.16.1-7.el8.noarch 2023-07-14T18:04:49.025 INFO:teuthology.orchestra.run.smithi161.stdout: bison-3.0.4-10.el8.x86_64 2023-07-14T18:04:49.025 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.025 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.025 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.026 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.026 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:04:49.026 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.026 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.026 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.026 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:04:49.026 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:04:49.027 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:04:49.027 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:04:49.027 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:04:49.027 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.027 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.027 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:04:49.027 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.027 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.028 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.028 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:04:49.028 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:04:49.028 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.028 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:04:49.028 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-14T18:04:49.028 INFO:teuthology.orchestra.run.smithi161.stdout: flex-2.6.1-9.el8.x86_64 2023-07-14T18:04:49.028 INFO:teuthology.orchestra.run.smithi161.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-14T18:04:49.029 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-14T18:04:49.029 INFO:teuthology.orchestra.run.smithi161.stdout: jq-1.6-3.el8.x86_64 2023-07-14T18:04:49.029 INFO:teuthology.orchestra.run.smithi161.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-14T18:04:49.029 INFO:teuthology.orchestra.run.smithi161.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-14T18:04:49.029 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-14T18:04:49.029 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.029 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.029 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.030 INFO:teuthology.orchestra.run.smithi161.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-14T18:04:49.030 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-14T18:04:49.030 INFO:teuthology.orchestra.run.smithi161.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-14T18:04:49.030 INFO:teuthology.orchestra.run.smithi161.stdout: libpq-13.5-1.el8.x86_64 2023-07-14T18:04:49.030 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-14T18:04:49.030 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-14T18:04:49.030 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.030 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.031 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-14T18:04:49.031 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.031 INFO:teuthology.orchestra.run.smithi161.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-14T18:04:49.031 INFO:teuthology.orchestra.run.smithi161.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-14T18:04:49.031 INFO:teuthology.orchestra.run.smithi161.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-14T18:04:49.031 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-14T18:04:49.031 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-14T18:04:49.032 INFO:teuthology.orchestra.run.smithi161.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-14T18:04:49.032 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-14T18:04:49.032 INFO:teuthology.orchestra.run.smithi161.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-14T18:04:49.032 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-14T18:04:49.032 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-14T18:04:49.032 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-14T18:04:49.032 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-14T18:04:49.032 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-14T18:04:49.033 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-14T18:04:49.033 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T18:04:49.033 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T18:04:49.033 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T18:04:49.033 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-14T18:04:49.033 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-14T18:04:49.033 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-14T18:04:49.033 INFO:teuthology.orchestra.run.smithi161.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-14T18:04:49.034 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-14T18:04:49.034 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.034 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.034 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.034 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-14T18:04:49.034 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-14T18:04:49.034 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-14T18:04:49.034 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-14T18:04:49.035 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-14T18:04:49.035 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-14T18:04:49.035 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-14T18:04:49.035 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-14T18:04:49.035 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-14T18:04:49.035 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-14T18:04:49.035 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-14T18:04:49.035 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-14T18:04:49.036 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-14T18:04:49.036 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-14T18:04:49.036 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-14T18:04:49.036 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-14T18:04:49.036 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-14T18:04:49.036 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-14T18:04:49.036 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-14T18:04:49.036 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-14T18:04:49.037 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-14T18:04:49.037 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-14T18:04:49.037 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend-2.6-1.el8.noarch 2023-07-14T18:04:49.037 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-14T18:04:49.037 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-14T18:04:49.037 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-14T18:04:49.037 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-14T18:04:49.038 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-14T18:04:49.038 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-14T18:04:49.038 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.038 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.038 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-14T18:04:49.038 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-14T18:04:49.038 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.038 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-14T18:04:49.039 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-14T18:04:49.039 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-14T18:04:49.039 INFO:teuthology.orchestra.run.smithi161.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-14T18:04:49.039 INFO:teuthology.orchestra.run.smithi161.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-14T18:04:49.039 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-14T18:04:49.039 INFO:teuthology.orchestra.run.smithi161.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-14T18:04:49.039 INFO:teuthology.orchestra.run.smithi161.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-14T18:04:49.039 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-14T18:04:49.040 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-14T18:04:49.040 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-14T18:04:49.040 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-14T18:04:49.040 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-14T18:04:49.040 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.040 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.040 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:04:49.040 INFO:teuthology.orchestra.run.smithi161.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-14T18:04:49.041 INFO:teuthology.orchestra.run.smithi161.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-14T18:04:49.041 INFO:teuthology.orchestra.run.smithi161.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-14T18:04:49.041 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-14T18:04:49.041 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-14T18:04:49.041 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-14T18:04:49.042 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T18:04:49.042 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-07-14T18:04:49.130 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 114/134 2023-07-14T18:04:49.175 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-14T18:04:49.190 DEBUG:teuthology.parallel:result is None 2023-07-14T18:04:49.298 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-14T18:04:49.306 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-14T18:04:49.492 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-07-14T18:04:49.698 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-07-14T18:04:49.781 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-07-14T18:04:49.981 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-07-14T18:04:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-07-14T18:04:49.994 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T18:04:50.226 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T18:04:50.226 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T18:04:50.481 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T18:04:50.680 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T18:04:51.193 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T18:04:51.193 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-2:18.0.0-4940.g81c9387d.el8.x86_64 1/134 2023-07-14T18:04:51.193 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-base-2:18.0.0-4940.g81c9387d.el8.x86_64 2/134 2023-07-14T18:04:51.193 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 3/134 2023-07-14T18:04:51.193 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 4/134 2023-07-14T18:04:51.194 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4940.g81c93 5/134 2023-07-14T18:04:51.194 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mds-2:18.0.0-4940.g81c9387d.el8.x86_64 6/134 2023-07-14T18:04:51.194 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-2:18.0.0-4940.g81c9387d.el8.x86_64 7/134 2023-07-14T18:04:51.194 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mon-2:18.0.0-4940.g81c9387d.el8.x86_64 8/134 2023-07-14T18:04:51.194 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-osd-2:18.0.0-4940.g81c9387d.el8.x86_64 9/134 2023-07-14T18:04:51.194 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-radosgw-2:18.0.0-4940.g81c9387d.el8.x86_64 10/134 2023-07-14T18:04:51.194 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-selinux-2:18.0.0-4940.g81c9387d.el8.x86_64 11/134 2023-07-14T18:04:51.194 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-test-2:18.0.0-4940.g81c9387d.el8.x86_64 12/134 2023-07-14T18:04:51.195 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 13/134 2023-07-14T18:04:51.195 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs-devel-2:18.0.0-4940.g81c9387d.el8.x86_ 14/134 2023-07-14T18:04:51.195 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs2-2:18.0.0-4940.g81c9387d.el8.x86_64 15/134 2023-07-14T18:04:51.195 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephsqlite-2:18.0.0-4940.g81c9387d.el8.x86_64 16/134 2023-07-14T18:04:51.195 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados-devel-2:18.0.0-4940.g81c9387d.el8.x86_6 17/134 2023-07-14T18:04:51.195 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libradosstriper1-2:18.0.0-4940.g81c9387d.el8.x86 18/134 2023-07-14T18:04:51.195 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librgw2-2:18.0.0-4940.g81c9387d.el8.x86_64 19/134 2023-07-14T18:04:51.196 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4940.g81c9387d.el 20/134 2023-07-14T18:04:51.196 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-common-2:18.0.0-4940.g81c9387d.el8. 21/134 2023-07-14T18:04:51.196 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cephfs-2:18.0.0-4940.g81c9387d.el8.x86_6 22/134 2023-07-14T18:04:51.196 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rados-2:18.0.0-4940.g81c9387d.el8.x86_64 23/134 2023-07-14T18:04:51.196 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rbd-2:18.0.0-4940.g81c9387d.el8.x86_64 24/134 2023-07-14T18:04:51.196 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rgw-2:18.0.0-4940.g81c9387d.el8.x86_64 25/134 2023-07-14T18:04:51.196 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 26/134 2023-07-14T18:04:51.196 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 27/134 2023-07-14T18:04:51.197 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-nbd-2:18.0.0-4940.g81c9387d.el8.x86_64 28/134 2023-07-14T18:04:51.197 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4940.g81c9387d. 29/134 2023-07-14T18:04:51.197 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4940.g81c9387d.el8.noa 30/134 2023-07-14T18:04:51.203 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4940.g81c9387d.el8.n 31/134 2023-07-14T18:04:51.204 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4940.g81c 32/134 2023-07-14T18:04:51.204 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4940.g81c9387d.el 33/134 2023-07-14T18:04:51.204 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4940.g81c9387d.el8.noarch 34/134 2023-07-14T18:04:51.204 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4940.g81c9387d.e 35/134 2023-07-14T18:04:51.204 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-volume-2:18.0.0-4940.g81c9387d.el8.noarch 36/134 2023-07-14T18:04:51.204 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 37/134 2023-07-14T18:04:51.204 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-top-2:18.0.0-4940.g81c9387d.el8.noarch 38/134 2023-07-14T18:04:51.204 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-14T18:04:51.205 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-14T18:04:51.205 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-14T18:04:51.205 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-14T18:04:51.205 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-14T18:04:51.205 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-14T18:04:51.205 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-14T18:04:51.205 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-14T18:04:51.206 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-14T18:04:51.206 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-14T18:04:51.206 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-14T18:04:51.206 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-14T18:04:51.206 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-14T18:04:51.206 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-14T18:04:51.206 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-14T18:04:51.206 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-14T18:04:51.207 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-14T18:04:51.207 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-14T18:04:51.207 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-14T18:04:51.207 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-14T18:04:51.207 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-14T18:04:51.207 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-14T18:04:51.207 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-14T18:04:51.207 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-14T18:04:51.208 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-14T18:04:51.208 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-14T18:04:51.208 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-14T18:04:51.208 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-14T18:04:51.208 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-14T18:04:51.208 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-14T18:04:51.208 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-14T18:04:51.208 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-14T18:04:51.209 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-14T18:04:51.209 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-07-14T18:04:51.209 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-07-14T18:04:51.209 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-07-14T18:04:51.209 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-07-14T18:04:51.209 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-07-14T18:04:51.209 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-07-14T18:04:51.209 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-07-14T18:04:51.210 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-07-14T18:04:51.210 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-07-14T18:04:51.210 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-07-14T18:04:51.210 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-07-14T18:04:51.210 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-07-14T18:04:51.210 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-07-14T18:04:51.210 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-07-14T18:04:51.211 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-07-14T18:04:51.211 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-07-14T18:04:51.211 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-07-14T18:04:51.211 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-07-14T18:04:51.211 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-07-14T18:04:51.211 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-07-14T18:04:51.211 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-07-14T18:04:51.211 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-07-14T18:04:51.212 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-07-14T18:04:51.212 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-07-14T18:04:51.212 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-07-14T18:04:51.212 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-07-14T18:04:51.212 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-07-14T18:04:51.212 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-07-14T18:04:51.212 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-07-14T18:04:51.212 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-07-14T18:04:51.213 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-07-14T18:04:51.213 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-07-14T18:04:51.213 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-07-14T18:04:51.213 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-07-14T18:04:51.213 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-07-14T18:04:51.213 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-07-14T18:04:51.213 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-07-14T18:04:51.213 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-07-14T18:04:51.214 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-07-14T18:04:51.214 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-07-14T18:04:51.214 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-07-14T18:04:51.214 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-07-14T18:04:51.214 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-07-14T18:04:51.214 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-07-14T18:04:51.214 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-07-14T18:04:51.214 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-07-14T18:04:51.215 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-07-14T18:04:51.215 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-07-14T18:04:51.215 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-14T18:04:51.215 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-07-14T18:04:51.215 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-14T18:04:51.215 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-07-14T18:04:51.215 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-07-14T18:04:51.215 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-07-14T18:04:51.216 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-07-14T18:04:51.216 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-07-14T18:04:51.216 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-14T18:04:51.216 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:18.0.0-4940.g81c9387d.el8.x86_64 129/134 2023-07-14T18:04:51.216 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-14T18:04:51.216 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:18.0.0-4940.g81c9387d.el8.x86_64 131/134 2023-07-14T18:04:51.216 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T18:04:51.217 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-14T18:04:53.717 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T18:04:53.718 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:02:03 ago on Fri 14 Jul 2023 06:02:50 PM UTC. 2023-07-14T18:04:54.175 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 114/134 2023-07-14T18:04:55.391 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-test-2:18.0.0-4940.g81c9387d.el8.x86_64 115/134 2023-07-14T18:04:55.708 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-07-14T18:04:55.910 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 117/134 2023-07-14T18:04:56.152 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-07-14T18:04:56.303 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-07-14T18:04:56.554 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-07-14T18:04:56.817 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-07-14T18:04:57.033 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs-devel-2:18.0.0-4940.g81c9387d.el8.x86_ 121/134 2023-07-14T18:04:57.188 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-14T18:05:00.422 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T18:05:00.711 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T18:05:00.751 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T18:05:01.046 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-top-2:18.0.0-4940.g81c9387d.el8.noarch 124/134 2023-07-14T18:05:01.337 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 125/134 2023-07-14T18:05:01.678 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-nbd-2:18.0.0-4940.g81c9387d.el8.x86_64 126/134 2023-07-14T18:05:02.055 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-14T18:05:02.277 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-07-14T18:05:02.386 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-07-14T18:05:02.408 INFO:teuthology.orchestra.run.smithi171.stdout:Installed products updated. 2023-07-14T18:05:02.624 INFO:teuthology.orchestra.run.smithi171.stdout: 2023-07-14T18:05:02.624 INFO:teuthology.orchestra.run.smithi171.stdout:Upgraded: 2023-07-14T18:05:02.624 INFO:teuthology.orchestra.run.smithi171.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-14T18:05:02.624 INFO:teuthology.orchestra.run.smithi171.stdout: librados2-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.624 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.625 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2023-07-14T18:05:02.625 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-14T18:05:02.625 INFO:teuthology.orchestra.run.smithi171.stdout: automake-1.16.1-7.el8.noarch 2023-07-14T18:05:02.625 INFO:teuthology.orchestra.run.smithi171.stdout: bison-3.0.4-10.el8.x86_64 2023-07-14T18:05:02.625 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.625 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.625 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.626 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.626 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:05:02.626 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.626 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.626 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.626 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:05:02.626 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:05:02.626 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:05:02.627 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:05:02.627 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:05:02.627 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.627 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.627 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:05:02.627 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.627 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.627 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.628 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:05:02.628 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:05:02.628 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.628 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-top-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:05:02.628 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-14T18:05:02.628 INFO:teuthology.orchestra.run.smithi171.stdout: flex-2.6.1-9.el8.x86_64 2023-07-14T18:05:02.628 INFO:teuthology.orchestra.run.smithi171.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-14T18:05:02.628 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-14T18:05:02.629 INFO:teuthology.orchestra.run.smithi171.stdout: jq-1.6-3.el8.x86_64 2023-07-14T18:05:02.629 INFO:teuthology.orchestra.run.smithi171.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-14T18:05:02.629 INFO:teuthology.orchestra.run.smithi171.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-14T18:05:02.629 INFO:teuthology.orchestra.run.smithi171.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-14T18:05:02.629 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.629 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.629 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.630 INFO:teuthology.orchestra.run.smithi171.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-14T18:05:02.630 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-14T18:05:02.630 INFO:teuthology.orchestra.run.smithi171.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-14T18:05:02.630 INFO:teuthology.orchestra.run.smithi171.stdout: libpq-13.5-1.el8.x86_64 2023-07-14T18:05:02.630 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-14T18:05:02.630 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-14T18:05:02.630 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.630 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.631 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-14T18:05:02.631 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.631 INFO:teuthology.orchestra.run.smithi171.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-14T18:05:02.631 INFO:teuthology.orchestra.run.smithi171.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-14T18:05:02.631 INFO:teuthology.orchestra.run.smithi171.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-14T18:05:02.631 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-14T18:05:02.631 INFO:teuthology.orchestra.run.smithi171.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-14T18:05:02.631 INFO:teuthology.orchestra.run.smithi171.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-14T18:05:02.632 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-14T18:05:02.632 INFO:teuthology.orchestra.run.smithi171.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-14T18:05:02.632 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-14T18:05:02.632 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-14T18:05:02.632 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-14T18:05:02.632 INFO:teuthology.orchestra.run.smithi171.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-14T18:05:02.632 INFO:teuthology.orchestra.run.smithi171.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-14T18:05:02.632 INFO:teuthology.orchestra.run.smithi171.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-14T18:05:02.633 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T18:05:02.633 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T18:05:02.633 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T18:05:02.633 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-14T18:05:02.633 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-14T18:05:02.633 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-14T18:05:02.633 INFO:teuthology.orchestra.run.smithi171.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-14T18:05:02.634 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-14T18:05:02.634 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.634 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.634 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.634 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-14T18:05:02.634 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-14T18:05:02.634 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-14T18:05:02.634 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-14T18:05:02.635 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-14T18:05:02.635 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-14T18:05:02.635 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-14T18:05:02.635 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-14T18:05:02.635 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-14T18:05:02.635 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-14T18:05:02.635 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-14T18:05:02.635 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-14T18:05:02.636 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-14T18:05:02.636 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-14T18:05:02.636 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-14T18:05:02.636 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-14T18:05:02.636 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-14T18:05:02.636 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-14T18:05:02.636 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-14T18:05:02.636 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-14T18:05:02.637 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-14T18:05:02.637 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-14T18:05:02.637 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend-2.6-1.el8.noarch 2023-07-14T18:05:02.637 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-14T18:05:02.637 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-14T18:05:02.637 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-14T18:05:02.637 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-14T18:05:02.638 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-14T18:05:02.638 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-14T18:05:02.638 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.638 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.638 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-14T18:05:02.638 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-14T18:05:02.638 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.638 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-14T18:05:02.639 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-14T18:05:02.639 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-14T18:05:02.639 INFO:teuthology.orchestra.run.smithi171.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-14T18:05:02.639 INFO:teuthology.orchestra.run.smithi171.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-14T18:05:02.639 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-14T18:05:02.639 INFO:teuthology.orchestra.run.smithi171.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-14T18:05:02.639 INFO:teuthology.orchestra.run.smithi171.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-14T18:05:02.639 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-14T18:05:02.640 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-14T18:05:02.640 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-14T18:05:02.640 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-14T18:05:02.640 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-14T18:05:02.640 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.640 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.640 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:02.640 INFO:teuthology.orchestra.run.smithi171.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-14T18:05:02.641 INFO:teuthology.orchestra.run.smithi171.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-14T18:05:02.641 INFO:teuthology.orchestra.run.smithi171.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-14T18:05:02.641 INFO:teuthology.orchestra.run.smithi171.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-14T18:05:02.641 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-14T18:05:02.641 INFO:teuthology.orchestra.run.smithi171.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-14T18:05:02.642 INFO:teuthology.orchestra.run.smithi171.stdout: 2023-07-14T18:05:02.642 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2023-07-14T18:05:02.644 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-07-14T18:05:02.672 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-07-14T18:05:02.804 DEBUG:teuthology.parallel:result is None 2023-07-14T18:05:02.869 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-14T18:05:02.891 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-14T18:05:03.114 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-07-14T18:05:03.114 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T18:05:03.350 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T18:05:03.350 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T18:05:03.652 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T18:05:03.870 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T18:05:04.167 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T18:05:04.167 INFO:teuthology.orchestra.run.smithi133.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-07-14T18:05:04.167 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:05:04.775 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-2:18.0.0-4940.g81c9387d.el8.x86_64 1/134 2023-07-14T18:05:04.775 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-base-2:18.0.0-4940.g81c9387d.el8.x86_64 2/134 2023-07-14T18:05:04.775 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 3/134 2023-07-14T18:05:04.776 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 4/134 2023-07-14T18:05:04.776 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4940.g81c93 5/134 2023-07-14T18:05:04.776 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mds-2:18.0.0-4940.g81c9387d.el8.x86_64 6/134 2023-07-14T18:05:04.776 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-2:18.0.0-4940.g81c9387d.el8.x86_64 7/134 2023-07-14T18:05:04.776 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mon-2:18.0.0-4940.g81c9387d.el8.x86_64 8/134 2023-07-14T18:05:04.776 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-osd-2:18.0.0-4940.g81c9387d.el8.x86_64 9/134 2023-07-14T18:05:04.776 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-radosgw-2:18.0.0-4940.g81c9387d.el8.x86_64 10/134 2023-07-14T18:05:04.777 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-selinux-2:18.0.0-4940.g81c9387d.el8.x86_64 11/134 2023-07-14T18:05:04.777 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-test-2:18.0.0-4940.g81c9387d.el8.x86_64 12/134 2023-07-14T18:05:04.777 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 13/134 2023-07-14T18:05:04.777 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs-devel-2:18.0.0-4940.g81c9387d.el8.x86_ 14/134 2023-07-14T18:05:04.777 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs2-2:18.0.0-4940.g81c9387d.el8.x86_64 15/134 2023-07-14T18:05:04.777 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephsqlite-2:18.0.0-4940.g81c9387d.el8.x86_64 16/134 2023-07-14T18:05:04.777 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados-devel-2:18.0.0-4940.g81c9387d.el8.x86_6 17/134 2023-07-14T18:05:04.778 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libradosstriper1-2:18.0.0-4940.g81c9387d.el8.x86 18/134 2023-07-14T18:05:04.778 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librgw2-2:18.0.0-4940.g81c9387d.el8.x86_64 19/134 2023-07-14T18:05:04.778 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4940.g81c9387d.el 20/134 2023-07-14T18:05:04.778 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-common-2:18.0.0-4940.g81c9387d.el8. 21/134 2023-07-14T18:05:04.778 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cephfs-2:18.0.0-4940.g81c9387d.el8.x86_6 22/134 2023-07-14T18:05:04.778 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rados-2:18.0.0-4940.g81c9387d.el8.x86_64 23/134 2023-07-14T18:05:04.778 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rbd-2:18.0.0-4940.g81c9387d.el8.x86_64 24/134 2023-07-14T18:05:04.779 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rgw-2:18.0.0-4940.g81c9387d.el8.x86_64 25/134 2023-07-14T18:05:04.779 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 26/134 2023-07-14T18:05:04.779 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 27/134 2023-07-14T18:05:04.779 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-nbd-2:18.0.0-4940.g81c9387d.el8.x86_64 28/134 2023-07-14T18:05:04.779 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4940.g81c9387d. 29/134 2023-07-14T18:05:04.779 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4940.g81c9387d.el8.noa 30/134 2023-07-14T18:05:04.779 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4940.g81c9387d.el8.n 31/134 2023-07-14T18:05:04.780 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4940.g81c 32/134 2023-07-14T18:05:04.780 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4940.g81c9387d.el 33/134 2023-07-14T18:05:04.780 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4940.g81c9387d.el8.noarch 34/134 2023-07-14T18:05:04.780 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4940.g81c9387d.e 35/134 2023-07-14T18:05:04.780 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-volume-2:18.0.0-4940.g81c9387d.el8.noarch 36/134 2023-07-14T18:05:04.780 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 37/134 2023-07-14T18:05:04.780 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-top-2:18.0.0-4940.g81c9387d.el8.noarch 38/134 2023-07-14T18:05:04.781 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-14T18:05:04.781 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-14T18:05:04.781 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-14T18:05:04.781 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-14T18:05:04.781 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-14T18:05:04.781 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-14T18:05:04.781 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-14T18:05:04.781 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-14T18:05:04.782 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-14T18:05:04.782 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-14T18:05:04.782 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-14T18:05:04.782 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-14T18:05:04.782 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-14T18:05:04.782 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-14T18:05:04.782 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-14T18:05:04.783 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-14T18:05:04.783 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-14T18:05:04.783 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-14T18:05:04.783 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-14T18:05:04.783 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-14T18:05:04.783 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-14T18:05:04.783 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-14T18:05:04.784 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-14T18:05:04.784 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-14T18:05:04.784 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-14T18:05:04.784 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-14T18:05:04.784 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-14T18:05:04.784 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-14T18:05:04.784 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-14T18:05:04.785 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-14T18:05:04.785 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-14T18:05:04.785 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-14T18:05:04.785 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-14T18:05:04.785 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-07-14T18:05:04.785 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-07-14T18:05:04.785 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-07-14T18:05:04.785 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-07-14T18:05:04.786 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-07-14T18:05:04.786 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-07-14T18:05:04.786 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-07-14T18:05:04.786 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-07-14T18:05:04.786 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-07-14T18:05:04.786 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-07-14T18:05:04.786 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-07-14T18:05:04.787 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-07-14T18:05:04.787 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-07-14T18:05:04.787 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-07-14T18:05:04.787 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-07-14T18:05:04.787 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-07-14T18:05:04.787 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-07-14T18:05:04.787 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-07-14T18:05:04.788 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-07-14T18:05:04.788 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-07-14T18:05:04.788 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-07-14T18:05:04.788 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-07-14T18:05:04.788 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-07-14T18:05:04.788 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-07-14T18:05:04.788 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-07-14T18:05:04.789 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-07-14T18:05:04.789 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-07-14T18:05:04.789 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-07-14T18:05:04.789 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-07-14T18:05:04.789 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-07-14T18:05:04.789 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-07-14T18:05:04.789 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-07-14T18:05:04.790 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-07-14T18:05:04.790 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-07-14T18:05:04.790 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-07-14T18:05:04.790 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-07-14T18:05:04.790 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-07-14T18:05:04.790 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-07-14T18:05:04.790 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-07-14T18:05:04.790 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-07-14T18:05:04.791 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-07-14T18:05:04.791 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-07-14T18:05:04.791 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-07-14T18:05:04.791 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-07-14T18:05:04.791 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-07-14T18:05:04.791 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-07-14T18:05:04.791 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-07-14T18:05:04.792 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-07-14T18:05:04.792 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-07-14T18:05:04.792 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-07-14T18:05:04.792 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-07-14T18:05:04.792 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-07-14T18:05:04.792 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-07-14T18:05:04.792 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-07-14T18:05:04.793 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-07-14T18:05:04.793 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-07-14T18:05:04.793 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-07-14T18:05:04.793 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:18.0.0-4940.g81c9387d.el8.x86_64 129/134 2023-07-14T18:05:04.793 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-14T18:05:04.793 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:18.0.0-4940.g81c9387d.el8.x86_64 131/134 2023-07-14T18:05:04.793 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T18:05:04.794 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-14T18:05:07.521 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T18:05:07.521 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:02:17 ago on Fri 14 Jul 2023 06:02:50 PM UTC. 2023-07-14T18:05:16.098 INFO:teuthology.orchestra.run.smithi133.stdout:Installed products updated. 2023-07-14T18:05:16.319 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:05:16.319 INFO:teuthology.orchestra.run.smithi133.stdout:Upgraded: 2023-07-14T18:05:16.319 INFO:teuthology.orchestra.run.smithi133.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-14T18:05:16.319 INFO:teuthology.orchestra.run.smithi133.stdout: librados2-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.319 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.320 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2023-07-14T18:05:16.320 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-14T18:05:16.320 INFO:teuthology.orchestra.run.smithi133.stdout: automake-1.16.1-7.el8.noarch 2023-07-14T18:05:16.320 INFO:teuthology.orchestra.run.smithi133.stdout: bison-3.0.4-10.el8.x86_64 2023-07-14T18:05:16.320 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.320 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.320 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.321 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.321 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:05:16.321 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.321 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.321 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.321 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:05:16.321 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:05:16.321 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:05:16.322 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:05:16.322 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:05:16.322 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.322 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.322 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:05:16.322 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.322 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.323 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.323 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:05:16.323 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:05:16.323 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.323 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T18:05:16.323 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-14T18:05:16.323 INFO:teuthology.orchestra.run.smithi133.stdout: flex-2.6.1-9.el8.x86_64 2023-07-14T18:05:16.323 INFO:teuthology.orchestra.run.smithi133.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-14T18:05:16.324 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-14T18:05:16.324 INFO:teuthology.orchestra.run.smithi133.stdout: jq-1.6-3.el8.x86_64 2023-07-14T18:05:16.324 INFO:teuthology.orchestra.run.smithi133.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-14T18:05:16.324 INFO:teuthology.orchestra.run.smithi133.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-14T18:05:16.324 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-14T18:05:16.324 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.324 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.324 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.325 INFO:teuthology.orchestra.run.smithi133.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-14T18:05:16.325 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-14T18:05:16.325 INFO:teuthology.orchestra.run.smithi133.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-14T18:05:16.325 INFO:teuthology.orchestra.run.smithi133.stdout: libpq-13.5-1.el8.x86_64 2023-07-14T18:05:16.325 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-14T18:05:16.325 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-14T18:05:16.325 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.326 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.326 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-14T18:05:16.326 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.326 INFO:teuthology.orchestra.run.smithi133.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-14T18:05:16.326 INFO:teuthology.orchestra.run.smithi133.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-14T18:05:16.326 INFO:teuthology.orchestra.run.smithi133.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-14T18:05:16.326 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-14T18:05:16.326 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-14T18:05:16.327 INFO:teuthology.orchestra.run.smithi133.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-14T18:05:16.327 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-14T18:05:16.327 INFO:teuthology.orchestra.run.smithi133.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-14T18:05:16.327 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-14T18:05:16.327 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-14T18:05:16.327 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-14T18:05:16.327 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-14T18:05:16.327 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-14T18:05:16.328 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-14T18:05:16.328 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T18:05:16.328 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T18:05:16.328 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T18:05:16.328 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-14T18:05:16.328 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-14T18:05:16.328 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-14T18:05:16.328 INFO:teuthology.orchestra.run.smithi133.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-14T18:05:16.329 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-14T18:05:16.329 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.329 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.329 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.329 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-14T18:05:16.329 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-14T18:05:16.329 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-14T18:05:16.329 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-14T18:05:16.330 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-14T18:05:16.330 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-14T18:05:16.330 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-14T18:05:16.330 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-14T18:05:16.330 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-14T18:05:16.330 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-14T18:05:16.330 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-14T18:05:16.330 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-14T18:05:16.331 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-14T18:05:16.331 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-14T18:05:16.331 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-14T18:05:16.331 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-14T18:05:16.331 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-14T18:05:16.331 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-14T18:05:16.331 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-14T18:05:16.332 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-14T18:05:16.332 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-14T18:05:16.332 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-14T18:05:16.332 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend-2.6-1.el8.noarch 2023-07-14T18:05:16.332 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-14T18:05:16.332 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-14T18:05:16.332 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-14T18:05:16.332 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-14T18:05:16.333 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-14T18:05:16.333 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-14T18:05:16.333 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.333 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.333 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-14T18:05:16.333 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-14T18:05:16.333 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.333 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-14T18:05:16.334 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-14T18:05:16.334 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-14T18:05:16.334 INFO:teuthology.orchestra.run.smithi133.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-14T18:05:16.334 INFO:teuthology.orchestra.run.smithi133.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-14T18:05:16.334 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-14T18:05:16.334 INFO:teuthology.orchestra.run.smithi133.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-14T18:05:16.334 INFO:teuthology.orchestra.run.smithi133.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-14T18:05:16.334 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-14T18:05:16.335 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-14T18:05:16.335 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-14T18:05:16.335 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-14T18:05:16.335 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-14T18:05:16.335 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.335 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.335 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T18:05:16.335 INFO:teuthology.orchestra.run.smithi133.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-14T18:05:16.336 INFO:teuthology.orchestra.run.smithi133.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-14T18:05:16.336 INFO:teuthology.orchestra.run.smithi133.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-14T18:05:16.336 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-14T18:05:16.336 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-14T18:05:16.336 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-14T18:05:16.337 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:05:16.337 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2023-07-14T18:05:16.641 DEBUG:teuthology.parallel:result is None 2023-07-14T18:05:16.641 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81c9387d2e7576da065747ec9c16d1c7d69016ec 2023-07-14T18:05:21.926 DEBUG:teuthology.orchestra.run.smithi133:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-14T18:05:21.985 INFO:teuthology.orchestra.run.smithi133.stdout:18.0.0-4940.g81c9387d.el8 2023-07-14T18:05:21.986 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4940.g81c9387d.el8 2023-07-14T18:05:21.987 INFO:teuthology.task.install:The correct ceph version 18.0.0-4940.g81c9387d is installed. 2023-07-14T18:05:21.987 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81c9387d2e7576da065747ec9c16d1c7d69016ec 2023-07-14T18:05:27.297 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-14T18:05:27.326 INFO:teuthology.orchestra.run.smithi161.stdout:18.0.0-4940.g81c9387d.el8 2023-07-14T18:05:27.326 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4940.g81c9387d.el8 2023-07-14T18:05:27.326 INFO:teuthology.task.install:The correct ceph version 18.0.0-4940.g81c9387d is installed. 2023-07-14T18:05:27.327 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81c9387d2e7576da065747ec9c16d1c7d69016ec 2023-07-14T18:05:32.158 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-14T18:05:32.186 INFO:teuthology.orchestra.run.smithi171.stdout:18.0.0-4940.g81c9387d.el8 2023-07-14T18:05:32.186 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4940.g81c9387d.el8 2023-07-14T18:05:32.187 INFO:teuthology.task.install:The correct ceph version 18.0.0-4940.g81c9387d is installed. 2023-07-14T18:05:32.187 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-07-14T18:05:32.187 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T18:05:32.188 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-14T18:05:32.284 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T18:05:32.285 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-14T18:05:32.333 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2023-07-14T18:05:32.333 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-14T18:05:32.380 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-07-14T18:05:32.381 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T18:05:32.381 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/daemon-helper 2023-07-14T18:05:32.450 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-14T18:05:32.556 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T18:05:32.557 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/daemon-helper 2023-07-14T18:05:32.589 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-14T18:05:32.659 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2023-07-14T18:05:32.660 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/daemon-helper 2023-07-14T18:05:32.691 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-14T18:05:32.761 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-07-14T18:05:32.762 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T18:05:32.762 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-14T18:05:32.828 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-14T18:05:32.934 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T18:05:32.935 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-14T18:05:32.966 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-14T18:05:33.035 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2023-07-14T18:05:33.036 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-14T18:05:33.067 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-14T18:05:33.137 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-07-14T18:05:33.138 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T18:05:33.138 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/stdin-killer 2023-07-14T18:05:33.204 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-14T18:05:33.310 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T18:05:33.311 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/stdin-killer 2023-07-14T18:05:33.342 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-14T18:05:33.413 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2023-07-14T18:05:33.413 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/stdin-killer 2023-07-14T18:05:33.444 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-14T18:05:33.513 INFO:teuthology.run_tasks:Running task cephadm... 2023-07-14T18:05:33.597 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '81c9387d2e7576da065747ec9c16d1c7d69016ec', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-07-14T18:05:33.598 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec 2023-07-14T18:05:33.607 INFO:tasks.cephadm:Cluster fsid is 06d0f5fa-2271-11ee-9b33-001a4aab830c 2023-07-14T18:05:33.607 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-07-14T18:05:33.608 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.133', 'mon.b': '172.21.15.161', 'mon.c': '172.21.15.171'} 2023-07-14T18:05:33.608 INFO:tasks.cephadm:First mon is mon.a on smithi133 2023-07-14T18:05:33.608 INFO:tasks.cephadm:First mgr is x 2023-07-14T18:05:33.609 INFO:tasks.cephadm:Normalizing hostnames... 2023-07-14T18:05:33.609 DEBUG:teuthology.orchestra.run.smithi133:> sudo hostname $(hostname -s) 2023-07-14T18:05:33.680 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname $(hostname -s) 2023-07-14T18:05:33.712 DEBUG:teuthology.orchestra.run.smithi171:> sudo hostname $(hostname -s) 2023-07-14T18:05:33.745 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-07-14T18:05:33.745 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81c9387d2e7576da065747ec9c16d1c7d69016ec 2023-07-14T18:05:34.019 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '81c9387d2e7576da065747ec9c16d1c7d69016ec', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/71431/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-4940-g81c9387d', 'node_name': '172.21.4.65+confusa03', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-4940.g81c9387d'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-07-14 16:50:26.249583', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20230714.134335', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/', 'archs': ['source', 'x86_64', 'arm64'], 'distro': 'centos'}] 2023-07-14T18:05:34.400 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-pdonnell-testing-20230714.134335, sha1 81c9387d2e7576da065747ec9c16d1c7d69016ec from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=81c9387d2e7576da065747ec9c16d1c7d69016ec 2023-07-14T18:05:34.401 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/x86_64/flavors/default/cephadm 2023-07-14T18:05:34.401 DEBUG:teuthology.orchestra.run.smithi133:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-14T18:05:36.477 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-rw-r--. 1 ubuntu ubuntu 670007 Jul 14 18:05 /home/ubuntu/cephtest/cephadm 2023-07-14T18:05:36.478 DEBUG:teuthology.orchestra.run.smithi161:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-14T18:05:38.230 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-rw-r--. 1 ubuntu ubuntu 670007 Jul 14 18:05 /home/ubuntu/cephtest/cephadm 2023-07-14T18:05:38.231 DEBUG:teuthology.orchestra.run.smithi171:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-14T18:05:39.603 INFO:teuthology.orchestra.run.smithi171.stdout:-rw-rw-r--. 1 ubuntu ubuntu 670007 Jul 14 18:05 /home/ubuntu/cephtest/cephadm 2023-07-14T18:05:39.603 DEBUG:teuthology.orchestra.run.smithi133:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-14T18:05:39.654 DEBUG:teuthology.orchestra.run.smithi161:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-14T18:05:39.675 DEBUG:teuthology.orchestra.run.smithi171:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-14T18:05:39.710 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec on all hosts... 2023-07-14T18:05:39.711 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec pull 2023-07-14T18:05:39.715 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec pull 2023-07-14T18:05:39.718 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec pull 2023-07-14T18:05:41.009 INFO:teuthology.orchestra.run.smithi161.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec... 2023-07-14T18:05:41.041 INFO:teuthology.orchestra.run.smithi171.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec... 2023-07-14T18:05:41.267 INFO:teuthology.orchestra.run.smithi133.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec... 2023-07-14T18:06:12.325 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:06:12.325 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 18.0.0-4940-g81c9387d (81c9387d2e7576da065747ec9c16d1c7d69016ec) reef (dev)", 2023-07-14T18:06:12.325 INFO:teuthology.orchestra.run.smithi133.stdout: "image_id": "219b180907a819a02c0d1790ec69e152120a3974adf061aaf947f90f6ccf9705", 2023-07-14T18:06:12.326 INFO:teuthology.orchestra.run.smithi133.stdout: "repo_digests": [ 2023-07-14T18:06:12.326 INFO:teuthology.orchestra.run.smithi133.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1302190083259ae87fcd27babf423c1dd04ae7417edb252031c1d861adeb3110" 2023-07-14T18:06:12.326 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2023-07-14T18:06:12.326 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:06:23.883 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2023-07-14T18:06:23.884 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_version": "ceph version 18.0.0-4940-g81c9387d (81c9387d2e7576da065747ec9c16d1c7d69016ec) reef (dev)", 2023-07-14T18:06:23.884 INFO:teuthology.orchestra.run.smithi171.stdout: "image_id": "219b180907a819a02c0d1790ec69e152120a3974adf061aaf947f90f6ccf9705", 2023-07-14T18:06:23.884 INFO:teuthology.orchestra.run.smithi171.stdout: "repo_digests": [ 2023-07-14T18:06:23.884 INFO:teuthology.orchestra.run.smithi171.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1302190083259ae87fcd27babf423c1dd04ae7417edb252031c1d861adeb3110" 2023-07-14T18:06:23.885 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2023-07-14T18:06:23.885 INFO:teuthology.orchestra.run.smithi171.stdout:} 2023-07-14T18:06:32.409 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T18:06:32.409 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_version": "ceph version 18.0.0-4940-g81c9387d (81c9387d2e7576da065747ec9c16d1c7d69016ec) reef (dev)", 2023-07-14T18:06:32.410 INFO:teuthology.orchestra.run.smithi161.stdout: "image_id": "219b180907a819a02c0d1790ec69e152120a3974adf061aaf947f90f6ccf9705", 2023-07-14T18:06:32.410 INFO:teuthology.orchestra.run.smithi161.stdout: "repo_digests": [ 2023-07-14T18:06:32.410 INFO:teuthology.orchestra.run.smithi161.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1302190083259ae87fcd27babf423c1dd04ae7417edb252031c1d861adeb3110" 2023-07-14T18:06:32.410 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2023-07-14T18:06:32.410 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T18:06:32.430 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph 2023-07-14T18:06:32.510 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /etc/ceph 2023-07-14T18:06:32.541 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /etc/ceph 2023-07-14T18:06:32.579 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 777 /etc/ceph 2023-07-14T18:06:32.648 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 777 /etc/ceph 2023-07-14T18:06:32.675 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 777 /etc/ceph 2023-07-14T18:06:32.707 INFO:tasks.cephadm:Writing seed config... 2023-07-14T18:06:32.708 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-07-14T18:06:32.708 INFO:tasks.cephadm: override: [client] debug client = 20 2023-07-14T18:06:32.708 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-07-14T18:06:32.709 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-07-14T18:06:32.709 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-07-14T18:06:32.709 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-07-14T18:06:32.709 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-07-14T18:06:32.709 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-07-14T18:06:32.710 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-07-14T18:06:32.710 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-07-14T18:06:32.710 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-07-14T18:06:32.710 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-07-14T18:06:32.710 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-07-14T18:06:32.711 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-07-14T18:06:32.711 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-07-14T18:06:32.711 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-07-14T18:06:32.711 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-07-14T18:06:32.711 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-07-14T18:06:32.712 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-07-14T18:06:32.712 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-07-14T18:06:32.712 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-07-14T18:06:32.712 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-07-14T18:06:32.712 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-07-14T18:06:32.713 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-07-14T18:06:32.713 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-07-14T18:06:32.713 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-07-14T18:06:32.713 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-07-14T18:06:32.713 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-07-14T18:06:32.714 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-07-14T18:06:32.714 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2023-07-14T18:06:32.714 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-07-14T18:06:32.714 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-07-14T18:06:32.714 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-07-14T18:06:32.714 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-07-14T18:06:32.715 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-07-14T18:06:32.715 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-07-14T18:06:32.715 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-07-14T18:06:32.715 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-07-14T18:06:32.715 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-07-14T18:06:32.716 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-07-14T18:06:32.716 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-07-14T18:06:32.716 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-07-14T18:06:32.716 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-07-14T18:06:32.716 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-07-14T18:06:32.717 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T18:06:32.718 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-07-14T18:06:32.760 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 06d0f5fa-2271-11ee-9b33-001a4aab830c ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-07-14T18:06:32.760 DEBUG:teuthology.orchestra.run.smithi133:mon.a> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mon.a.service 2023-07-14T18:06:32.805 DEBUG:teuthology.orchestra.run.smithi133:mgr.x> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mgr.x.service 2023-07-14T18:06:32.850 INFO:tasks.cephadm:Bootstrapping... 2023-07-14T18:06:32.850 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec -v bootstrap --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.133 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-07-14T18:06:32.871 INFO:journalctl@ceph.mon.a.smithi133.stdout:-- Logs begin at Fri 2023-07-14 17:46:17 UTC. -- 2023-07-14T18:06:32.918 INFO:journalctl@ceph.mgr.x.smithi133.stdout:-- Logs begin at Fri 2023-07-14 17:46:17 UTC. -- 2023-07-14T18:06:33.072 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2023-07-14T18:06:33.072 INFO:teuthology.orchestra.run.smithi133.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec', '-v', 'bootstrap', '--fsid', '06d0f5fa-2271-11ee-9b33-001a4aab830c', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.133', '--skip-admin-label'] 2023-07-14T18:06:33.247 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stdout 4.1.1 2023-07-14T18:06:33.247 INFO:teuthology.orchestra.run.smithi133.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-07-14T18:06:33.248 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying podman|docker is present... 2023-07-14T18:06:33.418 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stdout 4.1.1 2023-07-14T18:06:33.418 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying lvm2 is present... 2023-07-14T18:06:33.418 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying time synchronization is in place... 2023-07-14T18:06:33.432 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-14T18:06:33.432 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-14T18:06:33.444 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-14T18:06:33.445 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout inactive 2023-07-14T18:06:33.458 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout enabled 2023-07-14T18:06:33.470 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout active 2023-07-14T18:06:33.471 INFO:teuthology.orchestra.run.smithi133.stdout:Unit chronyd.service is enabled and running 2023-07-14T18:06:33.471 INFO:teuthology.orchestra.run.smithi133.stdout:Repeating the final host check... 2023-07-14T18:06:33.640 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stdout 4.1.1 2023-07-14T18:06:33.641 INFO:teuthology.orchestra.run.smithi133.stdout:podman (/bin/podman) version 4.1.1 is present 2023-07-14T18:06:33.641 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl is present 2023-07-14T18:06:33.641 INFO:teuthology.orchestra.run.smithi133.stdout:lvcreate is present 2023-07-14T18:06:33.654 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-14T18:06:33.654 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-14T18:06:33.666 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-14T18:06:33.666 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout inactive 2023-07-14T18:06:33.679 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout enabled 2023-07-14T18:06:33.692 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout active 2023-07-14T18:06:33.692 INFO:teuthology.orchestra.run.smithi133.stdout:Unit chronyd.service is enabled and running 2023-07-14T18:06:33.692 INFO:teuthology.orchestra.run.smithi133.stdout:Host looks OK 2023-07-14T18:06:33.692 INFO:teuthology.orchestra.run.smithi133.stdout:Cluster fsid: 06d0f5fa-2271-11ee-9b33-001a4aab830c 2023-07-14T18:06:33.693 INFO:teuthology.orchestra.run.smithi133.stdout:Acquiring lock 140550968998376 on /run/cephadm/06d0f5fa-2271-11ee-9b33-001a4aab830c.lock 2023-07-14T18:06:33.693 INFO:teuthology.orchestra.run.smithi133.stdout:Lock 140550968998376 acquired on /run/cephadm/06d0f5fa-2271-11ee-9b33-001a4aab830c.lock 2023-07-14T18:06:33.693 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying IP 172.21.15.133 port 3300 ... 2023-07-14T18:06:33.694 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying IP 172.21.15.133 port 6789 ... 2023-07-14T18:06:33.694 INFO:teuthology.orchestra.run.smithi133.stdout:Base mon IP(s) is [172.21.15.133:3300, 172.21.15.133:6789], mon addrv is [v2:172.21.15.133:3300,v1:172.21.15.133:6789] 2023-07-14T18:06:33.701 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2023-07-14T18:06:33.702 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.133 metric 100 2023-07-14T18:06:33.708 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-07-14T18:06:33.708 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2023-07-14T18:06:33.709 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2023-07-14T18:06:33.715 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-07-14T18:06:33.715 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-07-14T18:06:33.715 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-14T18:06:33.715 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-07-14T18:06:33.715 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout inet6 fe80::1cfd:5bbf:6db9:2d9/64 scope link noprefixroute 2023-07-14T18:06:33.715 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-14T18:06:33.716 INFO:teuthology.orchestra.run.smithi133.stdout:Mon IP `172.21.15.133` is in CIDR network `172.21.0.0/20` 2023-07-14T18:06:33.716 INFO:teuthology.orchestra.run.smithi133.stdout:Mon IP `172.21.15.133` is in CIDR network `172.21.0.0/20` 2023-07-14T18:06:33.716 INFO:teuthology.orchestra.run.smithi133.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-07-14T18:06:33.717 INFO:teuthology.orchestra.run.smithi133.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-07-14T18:06:33.718 INFO:teuthology.orchestra.run.smithi133.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec... 2023-07-14T18:06:34.190 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stdout 219b180907a819a02c0d1790ec69e152120a3974adf061aaf947f90f6ccf9705 2023-07-14T18:06:34.190 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec... 2023-07-14T18:06:34.191 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stderr Getting image source signatures 2023-07-14T18:06:34.191 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stderr Copying blob sha256:0e454b422ba90e2ad13e636c40610401086a3576fe973b0460943839fe5f2508 2023-07-14T18:06:34.191 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stderr Copying blob sha256:6022c611d474f2b306ddd1edfc8360123ece5266119096f25fde843d7967e636 2023-07-14T18:06:34.191 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stderr Copying config sha256:219b180907a819a02c0d1790ec69e152120a3974adf061aaf947f90f6ccf9705 2023-07-14T18:06:34.191 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stderr Writing manifest to image destination 2023-07-14T18:06:34.191 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stderr Storing signatures 2023-07-14T18:06:35.062 INFO:teuthology.orchestra.run.smithi133.stdout:ceph: stdout ceph version 18.0.0-4940-g81c9387d (81c9387d2e7576da065747ec9c16d1c7d69016ec) reef (dev) 2023-07-14T18:06:35.063 INFO:teuthology.orchestra.run.smithi133.stdout:Ceph version: ceph version 18.0.0-4940-g81c9387d (81c9387d2e7576da065747ec9c16d1c7d69016ec) reef (dev) 2023-07-14T18:06:35.063 INFO:teuthology.orchestra.run.smithi133.stdout:Extracting ceph user uid/gid from container image... 2023-07-14T18:06:35.836 INFO:teuthology.orchestra.run.smithi133.stdout:stat: stdout 167 167 2023-07-14T18:06:35.837 INFO:teuthology.orchestra.run.smithi133.stdout:Creating initial keys... 2023-07-14T18:06:36.613 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph-authtool: stdout AQCsjrFkoVjhCxAAgDauy0jmWgCFtRbGtCitfg== 2023-07-14T18:06:41.400 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph-authtool: stdout AQCsjrFk2lb9OBAAb6NhR8J7Whtdua+Ae2aZxg== 2023-07-14T18:06:45.992 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph-authtool: stdout AQC1jrFkZeukHxAATSNfwADL0LMf4Tl78eBECw== 2023-07-14T18:06:45.993 INFO:teuthology.orchestra.run.smithi133.stdout:Creating initial monmap... 2023-07-14T18:06:46.764 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-14T18:06:46.765 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-07-14T18:06:46.765 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 06d0f5fa-2271-11ee-9b33-001a4aab830c 2023-07-14T18:06:46.765 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-14T18:06:46.765 INFO:teuthology.orchestra.run.smithi133.stdout:monmaptool for a [v2:172.21.15.133:3300,v1:172.21.15.133:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-14T18:06:46.765 INFO:teuthology.orchestra.run.smithi133.stdout:setting min_mon_release = pacific 2023-07-14T18:06:46.765 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/monmaptool: set fsid to 06d0f5fa-2271-11ee-9b33-001a4aab830c 2023-07-14T18:06:46.765 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-14T18:06:46.766 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:06:46.766 INFO:teuthology.orchestra.run.smithi133.stdout:Creating mon... 2023-07-14T18:06:47.543 INFO:teuthology.orchestra.run.smithi133.stdout:create mon.a on 2023-07-14T18:06:47.808 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-07-14T18:06:48.013 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-07-14T18:06:48.256 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c.target → /etc/systemd/system/ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c.target. 2023-07-14T18:06:48.256 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c.target → /etc/systemd/system/ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c.target. 2023-07-14T18:06:48.613 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mon.a 2023-07-14T18:06:48.614 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Failed to reset failed state of unit ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mon.a.service: Unit ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mon.a.service not loaded. 2023-07-14T18:06:48.809 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c.target.wants/ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mon.a.service → /etc/systemd/system/ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@.service. 2023-07-14T18:06:48.895 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:48 smithi133 systemd[1]: Starting Ceph mon.a for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:06:49.594 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:49 smithi133 podman[125833]: 2023-07-14T18:06:49.594 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:49 smithi133 bash[125833]: 0a94c68630c14bcc758495cca25a3e97141c8c07e30b0a676760ed4376bd2873 2023-07-14T18:06:49.617 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-14T18:06:49.617 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout disabled 2023-07-14T18:06:49.633 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-14T18:06:49.633 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout inactive 2023-07-14T18:06:49.633 INFO:teuthology.orchestra.run.smithi133.stdout:firewalld.service is not enabled 2023-07-14T18:06:49.633 INFO:teuthology.orchestra.run.smithi133.stdout:Not possible to enable service . firewalld.service is not available 2023-07-14T18:06:49.634 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for mon to start... 2023-07-14T18:06:49.634 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for mon... 2023-07-14T18:06:49.895 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:49 smithi133 systemd[1]: Started Ceph mon.a for 06d0f5fa-2271-11ee-9b33-001a4aab830c. 2023-07-14T18:06:49.895 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:49 smithi133 ceph-mon[125873]: mkfs 06d0f5fa-2271-11ee-9b33-001a4aab830c 2023-07-14T18:06:49.895 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:49 smithi133 ceph-mon[125873]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-14T18:06:50.728 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout cluster: 2023-07-14T18:06:50.728 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout id: 06d0f5fa-2271-11ee-9b33-001a4aab830c 2023-07-14T18:06:50.728 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-07-14T18:06:50.729 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2023-07-14T18:06:50.729 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout services: 2023-07-14T18:06:50.729 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.670401s) 2023-07-14T18:06:50.729 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-07-14T18:06:50.729 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-07-14T18:06:50.729 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2023-07-14T18:06:50.729 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout data: 2023-07-14T18:06:50.729 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-07-14T18:06:50.730 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-07-14T18:06:50.730 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-07-14T18:06:50.730 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout pgs: 2023-07-14T18:06:50.730 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2023-07-14T18:06:50.730 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.148+0000 7fdf2f78d700 1 Processor -- start 2023-07-14T18:06:50.730 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.149+0000 7fdf2f78d700 1 -- start start 2023-07-14T18:06:50.730 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.149+0000 7fdf2f78d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf2807bdf0 0x7fdf2807a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:06:50.730 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.149+0000 7fdf2f78d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2807a830 con 0x7fdf2807bdf0 2023-07-14T18:06:50.731 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.150+0000 7fdf2d529700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf2807bdf0 0x7fdf2807a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:06:50.731 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.150+0000 7fdf2d529700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf2807bdf0 0x7fdf2807a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54622/0 (socket says 172.21.15.133:54622) 2023-07-14T18:06:50.731 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.150+0000 7fdf2d529700 1 -- 172.21.15.133:0/581505346 learned_addr learned my addr 172.21.15.133:0/581505346 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:06:50.731 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.150+0000 7fdf2d529700 1 -- 172.21.15.133:0/581505346 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2807a970 con 0x7fdf2807bdf0 2023-07-14T18:06:50.731 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.150+0000 7fdf2d529700 1 --2- 172.21.15.133:0/581505346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf2807bdf0 0x7fdf2807a2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdf240098a0 tx=0x7fdf24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e182811b77cddc41 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:06:50.731 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.151+0000 7fdf1ffff700 1 -- 172.21.15.133:0/581505346 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf24004160 con 0x7fdf2807bdf0 2023-07-14T18:06:50.731 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.151+0000 7fdf1ffff700 1 -- 172.21.15.133:0/581505346 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf240042c0 con 0x7fdf2807bdf0 2023-07-14T18:06:50.732 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.151+0000 7fdf1ffff700 1 -- 172.21.15.133:0/581505346 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf240045a0 con 0x7fdf2807bdf0 2023-07-14T18:06:50.732 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.151+0000 7fdf2f78d700 1 -- 172.21.15.133:0/581505346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf2807bdf0 msgr2=0x7fdf2807a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:06:50.732 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.151+0000 7fdf2f78d700 1 --2- 172.21.15.133:0/581505346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf2807bdf0 0x7fdf2807a2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdf240098a0 tx=0x7fdf24009bb0 comp rx=0 tx=0).stop 2023-07-14T18:06:50.732 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.151+0000 7fdf2f78d700 1 -- 172.21.15.133:0/581505346 shutdown_connections 2023-07-14T18:06:50.732 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.151+0000 7fdf2f78d700 1 --2- 172.21.15.133:0/581505346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf2807bdf0 0x7fdf2807a2f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:06:50.732 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.151+0000 7fdf2f78d700 1 -- 172.21.15.133:0/581505346 >> 172.21.15.133:0/581505346 conn(0x7fdf28100ab0 msgr2=0x7fdf28102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:06:50.732 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.151+0000 7fdf2f78d700 1 -- 172.21.15.133:0/581505346 shutdown_connections 2023-07-14T18:06:50.733 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.151+0000 7fdf2f78d700 1 -- 172.21.15.133:0/581505346 wait complete. 2023-07-14T18:06:50.733 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.152+0000 7fdf2f78d700 1 Processor -- start 2023-07-14T18:06:50.733 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.153+0000 7fdf2f78d700 1 -- start start 2023-07-14T18:06:50.733 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.153+0000 7fdf2f78d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf2807bdf0 0x7fdf2819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:06:50.733 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.153+0000 7fdf2f78d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2819b0c0 con 0x7fdf2807bdf0 2023-07-14T18:06:50.733 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.153+0000 7fdf2d529700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf2807bdf0 0x7fdf2819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:06:50.733 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.153+0000 7fdf2d529700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf2807bdf0 0x7fdf2819ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54632/0 (socket says 172.21.15.133:54632) 2023-07-14T18:06:50.733 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.153+0000 7fdf2d529700 1 -- 172.21.15.133:0/1645855209 learned_addr learned my addr 172.21.15.133:0/1645855209 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:06:50.734 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.153+0000 7fdf2d529700 1 -- 172.21.15.133:0/1645855209 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf24009550 con 0x7fdf2807bdf0 2023-07-14T18:06:50.734 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.154+0000 7fdf2d529700 1 --2- 172.21.15.133:0/1645855209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf2807bdf0 0x7fdf2819ab80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdf24004000 tx=0x7fdf24004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:06:50.734 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.154+0000 7fdf1e7fc700 1 -- 172.21.15.133:0/1645855209 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf24011630 con 0x7fdf2807bdf0 2023-07-14T18:06:50.734 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.154+0000 7fdf1e7fc700 1 -- 172.21.15.133:0/1645855209 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf24011790 con 0x7fdf2807bdf0 2023-07-14T18:06:50.734 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.154+0000 7fdf2f78d700 1 -- 172.21.15.133:0/1645855209 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2819b2c0 con 0x7fdf2807bdf0 2023-07-14T18:06:50.734 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.154+0000 7fdf1e7fc700 1 -- 172.21.15.133:0/1645855209 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf24011a30 con 0x7fdf2807bdf0 2023-07-14T18:06:50.734 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.154+0000 7fdf2f78d700 1 -- 172.21.15.133:0/1645855209 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2819b6e0 con 0x7fdf2807bdf0 2023-07-14T18:06:50.734 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.154+0000 7fdf1e7fc700 1 -- 172.21.15.133:0/1645855209 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fdf24022070 con 0x7fdf2807bdf0 2023-07-14T18:06:50.735 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.154+0000 7fdf1e7fc700 1 -- 172.21.15.133:0/1645855209 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdf2401c450 con 0x7fdf2807bdf0 2023-07-14T18:06:50.735 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.155+0000 7fdf2f78d700 1 -- 172.21.15.133:0/1645855209 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf28194990 con 0x7fdf2807bdf0 2023-07-14T18:06:50.735 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.158+0000 7fdf1e7fc700 1 -- 172.21.15.133:0/1645855209 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fdf2401b450 con 0x7fdf2807bdf0 2023-07-14T18:06:50.735 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.229+0000 7fdf2f78d700 1 -- 172.21.15.133:0/1645855209 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fdf28061250 con 0x7fdf2807bdf0 2023-07-14T18:06:50.735 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.229+0000 7fdf1e7fc700 1 -- 172.21.15.133:0/1645855209 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fdf2401b5f0 con 0x7fdf2807bdf0 2023-07-14T18:06:50.735 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.230+0000 7fdf2f78d700 1 -- 172.21.15.133:0/1645855209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf2807bdf0 msgr2=0x7fdf2819ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:06:50.735 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.230+0000 7fdf2f78d700 1 --2- 172.21.15.133:0/1645855209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf2807bdf0 0x7fdf2819ab80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdf24004000 tx=0x7fdf24004750 comp rx=0 tx=0).stop 2023-07-14T18:06:50.736 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.230+0000 7fdf2f78d700 1 -- 172.21.15.133:0/1645855209 shutdown_connections 2023-07-14T18:06:50.736 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.230+0000 7fdf2f78d700 1 --2- 172.21.15.133:0/1645855209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf2807bdf0 0x7fdf2819ab80 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:06:50.736 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.230+0000 7fdf2f78d700 1 -- 172.21.15.133:0/1645855209 >> 172.21.15.133:0/1645855209 conn(0x7fdf28100ab0 msgr2=0x7fdf281023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:06:50.736 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.231+0000 7fdf2f78d700 1 -- 172.21.15.133:0/1645855209 shutdown_connections 2023-07-14T18:06:50.736 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:50.231+0000 7fdf2f78d700 1 -- 172.21.15.133:0/1645855209 wait complete. 2023-07-14T18:06:50.736 INFO:teuthology.orchestra.run.smithi133.stdout:mon is available 2023-07-14T18:06:50.736 INFO:teuthology.orchestra.run.smithi133.stdout:Assimilating anything we can from ceph.conf... 2023-07-14T18:06:50.894 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:50 smithi133 ceph-mon[125873]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-14T18:06:50.895 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:50 smithi133 ceph-mon[125873]: monmap e1: 1 mons at {a=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0]} removed_ranks: {} 2023-07-14T18:06:50.895 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:50 smithi133 ceph-mon[125873]: fsmap 2023-07-14T18:06:50.895 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:50 smithi133 ceph-mon[125873]: osdmap e1: 0 total, 0 up, 0 in 2023-07-14T18:06:50.895 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:50 smithi133 ceph-mon[125873]: mgrmap e1: no daemons active 2023-07-14T18:06:50.895 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:50 smithi133 ceph-mon[125873]: from='client.? 172.21.15.133:0/1645855209' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-07-14T18:06:51.805 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2023-07-14T18:06:51.805 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout [global] 2023-07-14T18:06:51.805 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout fsid = 06d0f5fa-2271-11ee-9b33-001a4aab830c 2023-07-14T18:06:51.805 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.133:3300,v1:172.21.15.133:6789] 2023-07-14T18:06:51.805 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-14T18:06:51.805 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-14T18:06:51.806 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-14T18:06:51.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-14T18:06:51.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2023-07-14T18:06:51.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-14T18:06:51.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-14T18:06:51.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2023-07-14T18:06:51.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout [osd] 2023-07-14T18:06:51.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-14T18:06:51.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-14T18:06:51.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-14T18:06:51.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-14T18:06:51.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-14T18:06:51.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.205+0000 7f6344730700 1 Processor -- start 2023-07-14T18:06:51.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.205+0000 7f6344730700 1 -- start start 2023-07-14T18:06:51.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.205+0000 7f6344730700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f633c105440 0x7f633c105850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:06:51.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.205+0000 7f6344730700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c105d90 con 0x7f633c105440 2023-07-14T18:06:51.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.206+0000 7f63424cc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f633c105440 0x7f633c105850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:06:51.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.206+0000 7f63424cc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f633c105440 0x7f633c105850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54644/0 (socket says 172.21.15.133:54644) 2023-07-14T18:06:51.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.206+0000 7f63424cc700 1 -- 172.21.15.133:0/3839423824 learned_addr learned my addr 172.21.15.133:0/3839423824 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:06:51.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.206+0000 7f63424cc700 1 -- 172.21.15.133:0/3839423824 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633c105ed0 con 0x7f633c105440 2023-07-14T18:06:51.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.207+0000 7f63424cc700 1 --2- 172.21.15.133:0/3839423824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f633c105440 0x7f633c105850 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f63380098a0 tx=0x7f6338009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa7ca2430f49d3d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:06:51.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.207+0000 7f63414ca700 1 -- 172.21.15.133:0/3839423824 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6338004160 con 0x7f633c105440 2023-07-14T18:06:51.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.207+0000 7f63414ca700 1 -- 172.21.15.133:0/3839423824 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63380042c0 con 0x7f633c105440 2023-07-14T18:06:51.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.207+0000 7f63414ca700 1 -- 172.21.15.133:0/3839423824 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63380045a0 con 0x7f633c105440 2023-07-14T18:06:51.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.207+0000 7f6344730700 1 -- 172.21.15.133:0/3839423824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f633c105440 msgr2=0x7f633c105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:06:51.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.207+0000 7f6344730700 1 --2- 172.21.15.133:0/3839423824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f633c105440 0x7f633c105850 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f63380098a0 tx=0x7f6338009bb0 comp rx=0 tx=0).stop 2023-07-14T18:06:51.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.207+0000 7f6344730700 1 -- 172.21.15.133:0/3839423824 shutdown_connections 2023-07-14T18:06:51.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.207+0000 7f6344730700 1 --2- 172.21.15.133:0/3839423824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f633c105440 0x7f633c105850 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:06:51.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.207+0000 7f6344730700 1 -- 172.21.15.133:0/3839423824 >> 172.21.15.133:0/3839423824 conn(0x7f633c100ab0 msgr2=0x7f633c102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:06:51.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.208+0000 7f6344730700 1 -- 172.21.15.133:0/3839423824 shutdown_connections 2023-07-14T18:06:51.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.208+0000 7f6344730700 1 -- 172.21.15.133:0/3839423824 wait complete. 2023-07-14T18:06:51.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.209+0000 7f6344730700 1 Processor -- start 2023-07-14T18:06:51.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.209+0000 7f6344730700 1 -- start start 2023-07-14T18:06:51.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.209+0000 7f6344730700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f633c105440 0x7f633c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:06:51.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.209+0000 7f6344730700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c19b090 con 0x7f633c105440 2023-07-14T18:06:51.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.209+0000 7f63424cc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f633c105440 0x7f633c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:06:51.815 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.209+0000 7f63424cc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f633c105440 0x7f633c19ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54652/0 (socket says 172.21.15.133:54652) 2023-07-14T18:06:51.815 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.209+0000 7f63424cc700 1 -- 172.21.15.133:0/3289867576 learned_addr learned my addr 172.21.15.133:0/3289867576 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:06:51.815 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.210+0000 7f63424cc700 1 -- 172.21.15.133:0/3289867576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6338009550 con 0x7f633c105440 2023-07-14T18:06:51.815 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.210+0000 7f63424cc700 1 --2- 172.21.15.133:0/3289867576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f633c105440 0x7f633c19ab50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f633c106170 tx=0x7f6338004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:06:51.815 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.210+0000 7f63337fe700 1 -- 172.21.15.133:0/3289867576 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6338011630 con 0x7f633c105440 2023-07-14T18:06:51.815 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.210+0000 7f63337fe700 1 -- 172.21.15.133:0/3289867576 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6338011790 con 0x7f633c105440 2023-07-14T18:06:51.815 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.210+0000 7f6344730700 1 -- 172.21.15.133:0/3289867576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633c19b290 con 0x7f633c105440 2023-07-14T18:06:51.816 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.210+0000 7f63337fe700 1 -- 172.21.15.133:0/3289867576 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6338011a30 con 0x7f633c105440 2023-07-14T18:06:51.816 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.210+0000 7f6344730700 1 -- 172.21.15.133:0/3289867576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633c19b6b0 con 0x7f633c105440 2023-07-14T18:06:51.816 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.210+0000 7f63337fe700 1 -- 172.21.15.133:0/3289867576 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6338022070 con 0x7f633c105440 2023-07-14T18:06:51.816 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.211+0000 7f63337fe700 1 -- 172.21.15.133:0/3289867576 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f633801c450 con 0x7f633c105440 2023-07-14T18:06:51.816 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.211+0000 7f6344730700 1 -- 172.21.15.133:0/3289867576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633c061c70 con 0x7f633c105440 2023-07-14T18:06:51.816 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.214+0000 7f63337fe700 1 -- 172.21.15.133:0/3289867576 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f633801b450 con 0x7f633c105440 2023-07-14T18:06:51.816 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.282+0000 7f6344730700 1 -- 172.21.15.133:0/3289867576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f633c19e0f0 con 0x7f633c105440 2023-07-14T18:06:51.816 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.287+0000 7f63337fe700 1 -- 172.21.15.133:0/3289867576 <== mon.0 v2:172.21.15.133:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f633801b5f0 con 0x7f633c105440 2023-07-14T18:06:51.817 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.287+0000 7f63337fe700 1 -- 172.21.15.133:0/3289867576 <== mon.0 v2:172.21.15.133:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7f63380375f0 con 0x7f633c105440 2023-07-14T18:06:51.817 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.288+0000 7f6344730700 1 -- 172.21.15.133:0/3289867576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f633c105440 msgr2=0x7f633c19ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:06:51.817 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.288+0000 7f6344730700 1 --2- 172.21.15.133:0/3289867576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f633c105440 0x7f633c19ab50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f633c106170 tx=0x7f6338004750 comp rx=0 tx=0).stop 2023-07-14T18:06:51.817 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.288+0000 7f6344730700 1 -- 172.21.15.133:0/3289867576 shutdown_connections 2023-07-14T18:06:51.817 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.288+0000 7f6344730700 1 --2- 172.21.15.133:0/3289867576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f633c105440 0x7f633c19ab50 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:06:51.817 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.288+0000 7f6344730700 1 -- 172.21.15.133:0/3289867576 >> 172.21.15.133:0/3289867576 conn(0x7f633c100ab0 msgr2=0x7f633c07a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:06:51.817 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.288+0000 7f6344730700 1 -- 172.21.15.133:0/3289867576 shutdown_connections 2023-07-14T18:06:51.817 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:51.288+0000 7f6344730700 1 -- 172.21.15.133:0/3289867576 wait complete. 2023-07-14T18:06:51.818 INFO:teuthology.orchestra.run.smithi133.stdout:Generating new minimal ceph.conf... 2023-07-14T18:06:51.894 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:51 smithi133 ceph-mon[125873]: from='client.? 172.21.15.133:0/3289867576' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-14T18:06:51.895 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:51 smithi133 ceph-mon[125873]: from='client.? 172.21.15.133:0/3289867576' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-07-14T18:06:52.825 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:52 smithi133 ceph-mon[125873]: from='client.? 172.21.15.133:0/165694447' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:06:52.886 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.289+0000 7f2232c25700 1 Processor -- start 2023-07-14T18:06:52.887 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.290+0000 7f2232c25700 1 -- start start 2023-07-14T18:06:52.887 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.290+0000 7f2232c25700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f222c07be10 0x7f222c07a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:06:52.887 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.290+0000 7f2232c25700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c07a850 con 0x7f222c07be10 2023-07-14T18:06:52.887 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.290+0000 7f22309c1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f222c07be10 0x7f222c07a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:06:52.887 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.290+0000 7f22309c1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f222c07be10 0x7f222c07a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54658/0 (socket says 172.21.15.133:54658) 2023-07-14T18:06:52.887 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.290+0000 7f22309c1700 1 -- 172.21.15.133:0/376901598 learned_addr learned my addr 172.21.15.133:0/376901598 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:06:52.888 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.291+0000 7f22309c1700 1 -- 172.21.15.133:0/376901598 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222c07a990 con 0x7f222c07be10 2023-07-14T18:06:52.888 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.291+0000 7f22309c1700 1 --2- 172.21.15.133:0/376901598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f222c07be10 0x7f222c07a310 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f22200098a0 tx=0x7f2220009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2620aad9bfb241 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:06:52.888 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.291+0000 7f222b7fe700 1 -- 172.21.15.133:0/376901598 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2220004160 con 0x7f222c07be10 2023-07-14T18:06:52.888 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.291+0000 7f222b7fe700 1 -- 172.21.15.133:0/376901598 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f22200042c0 con 0x7f222c07be10 2023-07-14T18:06:52.888 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.291+0000 7f222b7fe700 1 -- 172.21.15.133:0/376901598 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2220011a00 con 0x7f222c07be10 2023-07-14T18:06:52.888 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.292+0000 7f2232c25700 1 -- 172.21.15.133:0/376901598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f222c07be10 msgr2=0x7f222c07a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:06:52.888 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.292+0000 7f2232c25700 1 --2- 172.21.15.133:0/376901598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f222c07be10 0x7f222c07a310 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f22200098a0 tx=0x7f2220009bb0 comp rx=0 tx=0).stop 2023-07-14T18:06:52.889 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.292+0000 7f2232c25700 1 -- 172.21.15.133:0/376901598 shutdown_connections 2023-07-14T18:06:52.889 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.292+0000 7f2232c25700 1 --2- 172.21.15.133:0/376901598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f222c07be10 0x7f222c07a310 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:06:52.889 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.292+0000 7f2232c25700 1 -- 172.21.15.133:0/376901598 >> 172.21.15.133:0/376901598 conn(0x7f222c100ab0 msgr2=0x7f222c102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:06:52.889 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.292+0000 7f2232c25700 1 -- 172.21.15.133:0/376901598 shutdown_connections 2023-07-14T18:06:52.889 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.292+0000 7f2232c25700 1 -- 172.21.15.133:0/376901598 wait complete. 2023-07-14T18:06:52.889 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.293+0000 7f2232c25700 1 Processor -- start 2023-07-14T18:06:52.889 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.293+0000 7f2232c25700 1 -- start start 2023-07-14T18:06:52.889 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.293+0000 7f2232c25700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f222c07be10 0x7f222c19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:06:52.890 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.293+0000 7f2232c25700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c19b8d0 con 0x7f222c07be10 2023-07-14T18:06:52.890 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.293+0000 7f22309c1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f222c07be10 0x7f222c19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:06:52.890 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.294+0000 7f22309c1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f222c07be10 0x7f222c19b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54662/0 (socket says 172.21.15.133:54662) 2023-07-14T18:06:52.890 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.294+0000 7f22309c1700 1 -- 172.21.15.133:0/165694447 learned_addr learned my addr 172.21.15.133:0/165694447 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:06:52.890 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.294+0000 7f22309c1700 1 -- 172.21.15.133:0/165694447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2220009550 con 0x7f222c07be10 2023-07-14T18:06:52.890 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.294+0000 7f22309c1700 1 --2- 172.21.15.133:0/165694447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f222c07be10 0x7f222c19b390 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2220009fb0 tx=0x7f2220011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:06:52.890 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.294+0000 7f2229ffb700 1 -- 172.21.15.133:0/165694447 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2220003b70 con 0x7f222c07be10 2023-07-14T18:06:52.891 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.294+0000 7f2229ffb700 1 -- 172.21.15.133:0/165694447 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2220003cd0 con 0x7f222c07be10 2023-07-14T18:06:52.891 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.294+0000 7f2229ffb700 1 -- 172.21.15.133:0/165694447 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22200247a0 con 0x7f222c07be10 2023-07-14T18:06:52.891 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.294+0000 7f2232c25700 1 -- 172.21.15.133:0/165694447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f222c19bad0 con 0x7f222c07be10 2023-07-14T18:06:52.891 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.294+0000 7f2232c25700 1 -- 172.21.15.133:0/165694447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222c19bef0 con 0x7f222c07be10 2023-07-14T18:06:52.891 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.295+0000 7f2229ffb700 1 -- 172.21.15.133:0/165694447 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2220024d10 con 0x7f222c07be10 2023-07-14T18:06:52.891 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.295+0000 7f2229ffb700 1 -- 172.21.15.133:0/165694447 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f22200193c0 con 0x7f222c07be10 2023-07-14T18:06:52.891 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.296+0000 7f2232c25700 1 -- 172.21.15.133:0/165694447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f222c061c70 con 0x7f222c07be10 2023-07-14T18:06:52.891 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.298+0000 7f2229ffb700 1 -- 172.21.15.133:0/165694447 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f222001ab40 con 0x7f222c07be10 2023-07-14T18:06:52.892 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.367+0000 7f2232c25700 1 -- 172.21.15.133:0/165694447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f222c19ea20 con 0x7f222c07be10 2023-07-14T18:06:52.892 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.368+0000 7f2229ffb700 1 -- 172.21.15.133:0/165694447 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7f222001ace0 con 0x7f222c07be10 2023-07-14T18:06:52.892 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.369+0000 7f2232c25700 1 -- 172.21.15.133:0/165694447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f222c07be10 msgr2=0x7f222c19b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:06:52.892 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.369+0000 7f2232c25700 1 --2- 172.21.15.133:0/165694447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f222c07be10 0x7f222c19b390 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2220009fb0 tx=0x7f2220011ad0 comp rx=0 tx=0).stop 2023-07-14T18:06:52.892 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.369+0000 7f2232c25700 1 -- 172.21.15.133:0/165694447 shutdown_connections 2023-07-14T18:06:52.892 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.369+0000 7f2232c25700 1 --2- 172.21.15.133:0/165694447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f222c07be10 0x7f222c19b390 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:06:52.892 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.369+0000 7f2232c25700 1 -- 172.21.15.133:0/165694447 >> 172.21.15.133:0/165694447 conn(0x7f222c100ab0 msgr2=0x7f222c101880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:06:52.893 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.369+0000 7f2232c25700 1 -- 172.21.15.133:0/165694447 shutdown_connections 2023-07-14T18:06:52.893 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:52.369+0000 7f2232c25700 1 -- 172.21.15.133:0/165694447 wait complete. 2023-07-14T18:06:52.893 INFO:teuthology.orchestra.run.smithi133.stdout:Restarting the monitor... 2023-07-14T18:06:53.145 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:52 smithi133 systemd[1]: Stopping Ceph mon.a for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:06:53.626 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:53 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mon-a[125849]: 2023-07-14T18:06:53.295+0000 7fa88db82700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:06:53.627 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:53 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mon-a[125849]: 2023-07-14T18:06:53.295+0000 7fa88db82700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-07-14T18:06:53.895 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:53 smithi133 bash[126212]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mon-a 2023-07-14T18:06:54.395 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 systemd[1]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mon.a.service: Succeeded. 2023-07-14T18:06:54.395 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 systemd[1]: Stopped Ceph mon.a for 06d0f5fa-2271-11ee-9b33-001a4aab830c. 2023-07-14T18:06:54.395 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 systemd[1]: Starting Ceph mon.a for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:06:54.767 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 podman[126344]: 2023-07-14T18:06:54.768 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 bash[126344]: e06732b371066489ee84e78df8f00db7853743812fa5bca8ef5c2ce92725e1b1 2023-07-14T18:06:54.768 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: set uid:gid to 167:167 (ceph:ceph) 2023-07-14T18:06:54.841 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: ceph version 18.0.0-4940-g81c9387d (81c9387d2e7576da065747ec9c16d1c7d69016ec) reef (dev), process ceph-mon, pid 7 2023-07-14T18:06:54.842 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: pidfile_write: ignore empty --pid-file 2023-07-14T18:06:54.842 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: load: jerasure load: lrc 2023-07-14T18:06:54.842 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: RocksDB version: 7.9.2 2023-07-14T18:06:54.842 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Git sha 0 2023-07-14T18:06:54.842 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Compile date 2023-07-14 14:45:42 2023-07-14T18:06:54.843 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: DB SUMMARY 2023-07-14T18:06:54.843 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: DB Session ID: MHVKFR0R8IKS4KHXPC4O 2023-07-14T18:06:54.843 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: CURRENT file: CURRENT 2023-07-14T18:06:54.843 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: IDENTITY file: IDENTITY 2023-07-14T18:06:54.843 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-07-14T18:06:54.843 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-07-14T18:06:54.843 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98881 ; 2023-07-14T18:06:54.844 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.error_if_exists: 0 2023-07-14T18:06:54.844 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.create_if_missing: 0 2023-07-14T18:06:54.844 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.paranoid_checks: 1 2023-07-14T18:06:54.844 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-14T18:06:54.844 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-14T18:06:54.844 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-14T18:06:54.844 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.env: 0x55ae7b107c60 2023-07-14T18:06:54.845 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.fs: PosixFileSystem 2023-07-14T18:06:54.845 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.info_log: 0x55ae7cb0b2a0 2023-07-14T18:06:54.845 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.max_file_opening_threads: 16 2023-07-14T18:06:54.845 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.statistics: (nil) 2023-07-14T18:06:54.845 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.use_fsync: 0 2023-07-14T18:06:54.845 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.max_log_file_size: 0 2023-07-14T18:06:54.845 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-14T18:06:54.846 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-14T18:06:54.846 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.keep_log_file_num: 1000 2023-07-14T18:06:54.846 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.recycle_log_file_num: 0 2023-07-14T18:06:54.846 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.allow_fallocate: 1 2023-07-14T18:06:54.846 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.allow_mmap_reads: 0 2023-07-14T18:06:54.846 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.allow_mmap_writes: 0 2023-07-14T18:06:54.847 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.use_direct_reads: 0 2023-07-14T18:06:54.847 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-14T18:06:54.847 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.create_missing_column_families: 0 2023-07-14T18:06:54.847 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.db_log_dir: 2023-07-14T18:06:54.847 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.wal_dir: 2023-07-14T18:06:54.847 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-14T18:06:54.847 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-14T18:06:54.847 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-14T18:06:54.848 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-14T18:06:54.848 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-14T18:06:54.848 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-14T18:06:54.848 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.advise_random_on_open: 1 2023-07-14T18:06:54.848 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.db_write_buffer_size: 0 2023-07-14T18:06:54.848 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.write_buffer_manager: 0x55ae7ca2c3c0 2023-07-14T18:06:54.849 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-14T18:06:54.849 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-14T18:06:54.849 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-14T18:06:54.849 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.rate_limiter: (nil) 2023-07-14T18:06:54.849 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-14T18:06:54.849 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.wal_recovery_mode: 2 2023-07-14T18:06:54.849 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.enable_thread_tracking: 0 2023-07-14T18:06:54.849 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.enable_pipelined_write: 0 2023-07-14T18:06:54.850 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.unordered_write: 0 2023-07-14T18:06:54.850 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-14T18:06:54.850 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-14T18:06:54.850 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-14T18:06:54.850 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-14T18:06:54.850 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.row_cache: None 2023-07-14T18:06:54.850 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.wal_filter: None 2023-07-14T18:06:54.850 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-14T18:06:54.851 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.allow_ingest_behind: 0 2023-07-14T18:06:54.851 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.two_write_queues: 0 2023-07-14T18:06:54.851 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.manual_wal_flush: 0 2023-07-14T18:06:54.851 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.wal_compression: 0 2023-07-14T18:06:54.851 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.atomic_flush: 0 2023-07-14T18:06:54.851 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-14T18:06:54.851 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-14T18:06:54.851 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-14T18:06:54.852 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.log_readahead_size: 0 2023-07-14T18:06:54.852 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-14T18:06:54.852 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.best_efforts_recovery: 0 2023-07-14T18:06:54.852 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-14T18:06:54.852 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-14T18:06:54.852 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.allow_data_in_errors: 0 2023-07-14T18:06:54.852 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.db_host_id: __hostname__ 2023-07-14T18:06:54.853 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-14T18:06:54.854 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.max_background_jobs: 2 2023-07-14T18:06:54.854 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.max_background_compactions: -1 2023-07-14T18:06:54.854 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.max_subcompactions: 1 2023-07-14T18:06:54.854 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-14T18:06:54.854 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-14T18:06:54.855 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-14T18:06:54.855 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.max_total_wal_size: 0 2023-07-14T18:06:54.855 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-14T18:06:54.855 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-14T18:06:54.855 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-14T18:06:54.855 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-14T18:06:54.855 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.max_open_files: -1 2023-07-14T18:06:54.856 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.bytes_per_sync: 0 2023-07-14T18:06:54.856 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-14T18:06:54.856 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-14T18:06:54.856 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.compaction_readahead_size: 0 2023-07-14T18:06:54.856 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.max_background_flushes: -1 2023-07-14T18:06:54.856 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Compression algorithms supported: 2023-07-14T18:06:54.856 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-14T18:06:54.857 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: kZSTD supported: 0 2023-07-14T18:06:54.857 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: kXpressCompression supported: 0 2023-07-14T18:06:54.857 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: kLZ4HCCompression supported: 1 2023-07-14T18:06:54.857 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: kZlibCompression supported: 1 2023-07-14T18:06:54.857 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: kSnappyCompression supported: 1 2023-07-14T18:06:54.857 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: kLZ4Compression supported: 1 2023-07-14T18:06:54.857 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: kBZip2Compression supported: 0 2023-07-14T18:06:54.858 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-14T18:06:54.858 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-14T18:06:54.858 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-07-14T18:06:54.858 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-14T18:06:54.858 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-14T18:06:54.858 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.merge_operator: 2023-07-14T18:06:54.858 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.compaction_filter: None 2023-07-14T18:06:54.858 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.compaction_filter_factory: None 2023-07-14T18:06:54.859 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.sst_partitioner_factory: None 2023-07-14T18:06:54.859 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-14T18:06:54.859 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-14T18:06:54.859 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ae7cb0b3a0) 2023-07-14T18:06:54.859 INFO:journalctl@ceph.mon.a.smithi133.stdout: cache_index_and_filter_blocks: 1 2023-07-14T18:06:54.859 INFO:journalctl@ceph.mon.a.smithi133.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-14T18:06:54.860 INFO:journalctl@ceph.mon.a.smithi133.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-14T18:06:54.860 INFO:journalctl@ceph.mon.a.smithi133.stdout: pin_top_level_index_and_filter: 1 2023-07-14T18:06:54.860 INFO:journalctl@ceph.mon.a.smithi133.stdout: index_type: 0 2023-07-14T18:06:54.860 INFO:journalctl@ceph.mon.a.smithi133.stdout: data_block_index_type: 0 2023-07-14T18:06:54.860 INFO:journalctl@ceph.mon.a.smithi133.stdout: index_shortening: 1 2023-07-14T18:06:54.860 INFO:journalctl@ceph.mon.a.smithi133.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-14T18:06:54.860 INFO:journalctl@ceph.mon.a.smithi133.stdout: checksum: 4 2023-07-14T18:06:54.861 INFO:journalctl@ceph.mon.a.smithi133.stdout: no_block_cache: 0 2023-07-14T18:06:54.861 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_cache: 0x55ae7cacb1f0 2023-07-14T18:06:54.861 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_cache_name: BinnedLRUCache 2023-07-14T18:06:54.861 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_cache_options: 2023-07-14T18:06:54.861 INFO:journalctl@ceph.mon.a.smithi133.stdout: capacity : 536870912 2023-07-14T18:06:54.861 INFO:journalctl@ceph.mon.a.smithi133.stdout: num_shard_bits : 4 2023-07-14T18:06:54.861 INFO:journalctl@ceph.mon.a.smithi133.stdout: strict_capacity_limit : 0 2023-07-14T18:06:54.861 INFO:journalctl@ceph.mon.a.smithi133.stdout: high_pri_pool_ratio: 0.000 2023-07-14T18:06:54.862 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_cache_compressed: (nil) 2023-07-14T18:06:54.862 INFO:journalctl@ceph.mon.a.smithi133.stdout: persistent_cache: (nil) 2023-07-14T18:06:54.862 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_size: 4096 2023-07-14T18:06:54.862 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_size_deviation: 10 2023-07-14T18:06:54.862 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_restart_interval: 16 2023-07-14T18:06:54.862 INFO:journalctl@ceph.mon.a.smithi133.stdout: index_block_restart_interval: 1 2023-07-14T18:06:54.862 INFO:journalctl@ceph.mon.a.smithi133.stdout: metadata_block_size: 4096 2023-07-14T18:06:54.862 INFO:journalctl@ceph.mon.a.smithi133.stdout: partition_filters: 0 2023-07-14T18:06:54.863 INFO:journalctl@ceph.mon.a.smithi133.stdout: use_delta_encoding: 1 2023-07-14T18:06:54.863 INFO:journalctl@ceph.mon.a.smithi133.stdout: filter_policy: bloomfilter 2023-07-14T18:06:54.863 INFO:journalctl@ceph.mon.a.smithi133.stdout: whole_key_filtering: 1 2023-07-14T18:06:54.863 INFO:journalctl@ceph.mon.a.smithi133.stdout: verify_compression: 0 2023-07-14T18:06:54.863 INFO:journalctl@ceph.mon.a.smithi133.stdout: read_amp_bytes_per_bit: 0 2023-07-14T18:06:54.863 INFO:journalctl@ceph.mon.a.smithi133.stdout: format_version: 5 2023-07-14T18:06:54.863 INFO:journalctl@ceph.mon.a.smithi133.stdout: enable_index_compression: 1 2023-07-14T18:06:54.863 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_align: 0 2023-07-14T18:06:54.864 INFO:journalctl@ceph.mon.a.smithi133.stdout: max_auto_readahead_size: 262144 2023-07-14T18:06:54.864 INFO:journalctl@ceph.mon.a.smithi133.stdout: prepopulate_block_cache: 0 2023-07-14T18:06:54.864 INFO:journalctl@ceph.mon.a.smithi133.stdout: initial_auto_readahead_size: 8192 2023-07-14T18:06:54.864 INFO:journalctl@ceph.mon.a.smithi133.stdout: num_file_reads_for_auto_readahead: 2 2023-07-14T18:06:54.864 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.write_buffer_size: 33554432 2023-07-14T18:06:54.864 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.max_write_buffer_number: 2 2023-07-14T18:06:54.864 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.compression: NoCompression 2023-07-14T18:06:54.865 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.bottommost_compression: Disabled 2023-07-14T18:06:54.865 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.prefix_extractor: nullptr 2023-07-14T18:06:54.865 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-14T18:06:54.865 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.num_levels: 7 2023-07-14T18:06:54.865 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-14T18:06:54.865 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-14T18:06:54.865 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-14T18:06:54.865 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-14T18:06:54.866 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-14T18:06:54.866 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-14T18:06:54.866 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-14T18:06:54.866 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-14T18:06:54.866 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-14T18:06:54.866 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-14T18:06:54.866 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-14T18:06:54.867 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-14T18:06:54.867 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-14T18:06:54.867 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.compression_opts.level: 32767 2023-07-14T18:06:54.867 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.compression_opts.strategy: 0 2023-07-14T18:06:54.867 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-14T18:06:54.867 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-14T18:06:54.868 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-14T18:06:54.868 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-14T18:06:54.868 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.compression_opts.enabled: false 2023-07-14T18:06:54.868 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-14T18:06:54.868 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-14T18:06:54.868 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-14T18:06:54.868 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-14T18:06:54.869 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.target_file_size_base: 67108864 2023-07-14T18:06:54.869 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-14T18:06:54.869 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-14T18:06:54.869 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-14T18:06:54.869 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-14T18:06:54.869 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-14T18:06:54.869 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-14T18:06:54.870 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-14T18:06:54.870 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-14T18:06:54.870 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-14T18:06:54.870 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-14T18:06:54.870 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-14T18:06:54.870 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-14T18:06:54.870 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-14T18:06:54.870 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-14T18:06:54.871 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.arena_block_size: 1048576 2023-07-14T18:06:54.871 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-14T18:06:54.871 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-14T18:06:54.871 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.disable_auto_compactions: 0 2023-07-14T18:06:54.871 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-14T18:06:54.871 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-14T18:06:54.871 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-14T18:06:54.872 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-14T18:06:54.872 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-14T18:06:54.872 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-14T18:06:54.872 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-14T18:06:54.872 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-14T18:06:54.872 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-14T18:06:54.872 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-14T18:06:54.872 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.table_properties_collectors: 2023-07-14T18:06:54.873 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.inplace_update_support: 0 2023-07-14T18:06:54.873 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-14T18:06:54.873 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-14T18:06:54.873 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-14T18:06:54.873 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-14T18:06:54.873 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.bloom_locality: 0 2023-07-14T18:06:54.873 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.max_successive_merges: 0 2023-07-14T18:06:54.873 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-14T18:06:54.874 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.paranoid_file_checks: 0 2023-07-14T18:06:54.874 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.force_consistency_checks: 1 2023-07-14T18:06:54.874 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.report_bg_io_stats: 0 2023-07-14T18:06:54.874 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.ttl: 2592000 2023-07-14T18:06:54.874 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-14T18:06:54.874 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-14T18:06:54.874 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-14T18:06:54.875 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.enable_blob_files: false 2023-07-14T18:06:54.875 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.min_blob_size: 0 2023-07-14T18:06:54.875 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.blob_file_size: 268435456 2023-07-14T18:06:54.875 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-14T18:06:54.875 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-14T18:06:54.875 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-14T18:06:54.875 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-14T18:06:54.875 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-14T18:06:54.876 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.blob_file_starting_level: 0 2023-07-14T18:06:54.876 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-14T18:06:54.876 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-07-14T18:06:54.876 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-07-14T18:06:54.876 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 31044754-a8c3-4353-8ada-7480f9d4af9d 2023-07-14T18:06:54.876 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689358014744852, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-07-14T18:06:54.876 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-07-14T18:06:54.877 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689358014746757, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94086, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 314, "table_properties": {"data_size": 92097, "index_size": 271, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13662, "raw_average_key_size": 50, "raw_value_size": 84670, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 273, "num_filter_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689358014, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "31044754-a8c3-4353-8ada-7480f9d4af9d", "db_session_id": "MHVKFR0R8IKS4KHXPC4O", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-07-14T18:06:54.877 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689358014746904, "job": 1, "event": "recovery_finished"} 2023-07-14T18:06:54.877 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-07-14T18:06:54.877 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-14T18:06:54.877 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ae7cb64000 2023-07-14T18:06:54.877 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: DB pointer 0x55ae7cb50000 2023-07-14T18:06:54.877 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-14T18:06:54.878 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-14T18:06:54.878 INFO:journalctl@ceph.mon.a.smithi133.stdout: ** DB Stats ** 2023-07-14T18:06:54.878 INFO:journalctl@ceph.mon.a.smithi133.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-14T18:06:54.878 INFO:journalctl@ceph.mon.a.smithi133.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-14T18:06:54.878 INFO:journalctl@ceph.mon.a.smithi133.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-14T18:06:54.878 INFO:journalctl@ceph.mon.a.smithi133.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-14T18:06:54.878 INFO:journalctl@ceph.mon.a.smithi133.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-14T18:06:54.879 INFO:journalctl@ceph.mon.a.smithi133.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-14T18:06:54.879 INFO:journalctl@ceph.mon.a.smithi133.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-14T18:06:54.879 INFO:journalctl@ceph.mon.a.smithi133.stdout: 2023-07-14T18:06:54.879 INFO:journalctl@ceph.mon.a.smithi133.stdout: ** Compaction Stats [default] ** 2023-07-14T18:06:54.879 INFO:journalctl@ceph.mon.a.smithi133.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-14T18:06:54.879 INFO:journalctl@ceph.mon.a.smithi133.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-14T18:06:54.879 INFO:journalctl@ceph.mon.a.smithi133.stdout: L0 2/0 93.72 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 70.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T18:06:54.879 INFO:journalctl@ceph.mon.a.smithi133.stdout: Sum 2/0 93.72 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 70.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T18:06:54.880 INFO:journalctl@ceph.mon.a.smithi133.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 70.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T18:06:54.880 INFO:journalctl@ceph.mon.a.smithi133.stdout: 2023-07-14T18:06:54.880 INFO:journalctl@ceph.mon.a.smithi133.stdout: ** Compaction Stats [default] ** 2023-07-14T18:06:54.880 INFO:journalctl@ceph.mon.a.smithi133.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-14T18:06:54.880 INFO:journalctl@ceph.mon.a.smithi133.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-14T18:06:54.880 INFO:journalctl@ceph.mon.a.smithi133.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 70.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T18:06:54.880 INFO:journalctl@ceph.mon.a.smithi133.stdout: 2023-07-14T18:06:54.881 INFO:journalctl@ceph.mon.a.smithi133.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-14T18:06:54.881 INFO:journalctl@ceph.mon.a.smithi133.stdout: 2023-07-14T18:06:54.881 INFO:journalctl@ceph.mon.a.smithi133.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-14T18:06:54.881 INFO:journalctl@ceph.mon.a.smithi133.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-14T18:06:54.881 INFO:journalctl@ceph.mon.a.smithi133.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-14T18:06:54.881 INFO:journalctl@ceph.mon.a.smithi133.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-14T18:06:54.881 INFO:journalctl@ceph.mon.a.smithi133.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-14T18:06:54.881 INFO:journalctl@ceph.mon.a.smithi133.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-14T18:06:54.882 INFO:journalctl@ceph.mon.a.smithi133.stdout: Cumulative compaction: 0.00 GB write, 12.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-14T18:06:54.882 INFO:journalctl@ceph.mon.a.smithi133.stdout: Interval compaction: 0.00 GB write, 12.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-14T18:06:54.882 INFO:journalctl@ceph.mon.a.smithi133.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-14T18:06:54.882 INFO:journalctl@ceph.mon.a.smithi133.stdout: Block cache BinnedLRUCache@0x55ae7cacb1f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-07-14T18:06:54.882 INFO:journalctl@ceph.mon.a.smithi133.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-07-14T18:06:54.882 INFO:journalctl@ceph.mon.a.smithi133.stdout: 2023-07-14T18:06:54.882 INFO:journalctl@ceph.mon.a.smithi133.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-14T18:06:54.882 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: starting mon.a rank 0 at public addrs [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] at bind addrs [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c 2023-07-14T18:06:54.883 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: mon.a@-1(???) e1 preinit fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c 2023-07-14T18:06:54.883 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: mon.a@-1(???).mds e1 new map 2023-07-14T18:06:54.883 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: mon.a@-1(???).mds e1 print_map 2023-07-14T18:06:54.883 INFO:journalctl@ceph.mon.a.smithi133.stdout: e1 2023-07-14T18:06:54.883 INFO:journalctl@ceph.mon.a.smithi133.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-14T18:06:54.883 INFO:journalctl@ceph.mon.a.smithi133.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-14T18:06:54.883 INFO:journalctl@ceph.mon.a.smithi133.stdout: legacy client fscid: -1 2023-07-14T18:06:54.884 INFO:journalctl@ceph.mon.a.smithi133.stdout: 2023-07-14T18:06:54.884 INFO:journalctl@ceph.mon.a.smithi133.stdout: No filesystems configured 2023-07-14T18:06:54.884 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-14T18:06:54.884 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T18:06:54.884 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T18:06:54.884 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T18:06:54.884 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-07-14T18:06:54.884 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: expand_channel_meta expand map: {default=false} 2023-07-14T18:06:54.885 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: expand_channel_meta from 'false' to 'false' 2023-07-14T18:06:54.885 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: expand_channel_meta expanded map: {default=false} 2023-07-14T18:06:54.885 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: expand_channel_meta expand map: {default=info} 2023-07-14T18:06:54.885 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: expand_channel_meta from 'info' to 'info' 2023-07-14T18:06:54.885 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: expand_channel_meta expanded map: {default=info} 2023-07-14T18:06:54.885 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: expand_channel_meta expand map: {default=daemon} 2023-07-14T18:06:54.885 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-14T18:06:54.886 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: expand_channel_meta expanded map: {default=daemon} 2023-07-14T18:06:54.886 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: expand_channel_meta expand map: {default=debug} 2023-07-14T18:06:54.886 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: expand_channel_meta from 'debug' to 'debug' 2023-07-14T18:06:54.886 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: expand_channel_meta expanded map: {default=debug} 2023-07-14T18:06:54.886 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-14T18:06:54.886 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: monmap e1: 1 mons at {a=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0]} removed_ranks: {} 2023-07-14T18:06:54.886 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: fsmap 2023-07-14T18:06:54.886 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: osdmap e1: 0 total, 0 up, 0 in 2023-07-14T18:06:54.887 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 ceph-mon[126384]: mgrmap e1: no daemons active 2023-07-14T18:06:54.887 INFO:teuthology.orchestra.run.smithi133.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-07-14T18:06:55.104 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:54 smithi133 systemd[1]: Started Ceph mon.a for 06d0f5fa-2271-11ee-9b33-001a4aab830c. 2023-07-14T18:06:56.244 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.632+0000 7fe7556d5700 1 Processor -- start 2023-07-14T18:06:56.245 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.632+0000 7fe7556d5700 1 -- start start 2023-07-14T18:06:56.245 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.633+0000 7fe7556d5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe75007bdf0 0x7fe75007a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:06:56.245 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.633+0000 7fe7556d5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75007a830 con 0x7fe75007bdf0 2023-07-14T18:06:56.245 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.634+0000 7fe74effd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe75007bdf0 0x7fe75007a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:06:56.245 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.634+0000 7fe74effd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe75007bdf0 0x7fe75007a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54666/0 (socket says 172.21.15.133:54666) 2023-07-14T18:06:56.245 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.634+0000 7fe74effd700 1 -- 172.21.15.133:0/2411864738 learned_addr learned my addr 172.21.15.133:0/2411864738 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:06:56.245 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.634+0000 7fe74effd700 1 -- 172.21.15.133:0/2411864738 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75007a970 con 0x7fe75007bdf0 2023-07-14T18:06:56.246 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.634+0000 7fe74effd700 1 --2- 172.21.15.133:0/2411864738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe75007bdf0 0x7fe75007a2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe7400098a0 tx=0x7fe740009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99973a5e4897a3e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:06:56.246 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.635+0000 7fe74dffb700 1 -- 172.21.15.133:0/2411864738 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe740004160 con 0x7fe75007bdf0 2023-07-14T18:06:56.246 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.635+0000 7fe74dffb700 1 -- 172.21.15.133:0/2411864738 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe7400042c0 con 0x7fe75007bdf0 2023-07-14T18:06:56.246 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.635+0000 7fe74dffb700 1 -- 172.21.15.133:0/2411864738 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe740011a00 con 0x7fe75007bdf0 2023-07-14T18:06:56.246 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.635+0000 7fe7556d5700 1 -- 172.21.15.133:0/2411864738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe75007bdf0 msgr2=0x7fe75007a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:06:56.246 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.635+0000 7fe7556d5700 1 --2- 172.21.15.133:0/2411864738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe75007bdf0 0x7fe75007a2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe7400098a0 tx=0x7fe740009bb0 comp rx=0 tx=0).stop 2023-07-14T18:06:56.246 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.635+0000 7fe7556d5700 1 -- 172.21.15.133:0/2411864738 shutdown_connections 2023-07-14T18:06:56.247 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.635+0000 7fe7556d5700 1 --2- 172.21.15.133:0/2411864738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe75007bdf0 0x7fe75007a2f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:06:56.247 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.635+0000 7fe7556d5700 1 -- 172.21.15.133:0/2411864738 >> 172.21.15.133:0/2411864738 conn(0x7fe750100ab0 msgr2=0x7fe750102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:06:56.247 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.635+0000 7fe7556d5700 1 -- 172.21.15.133:0/2411864738 shutdown_connections 2023-07-14T18:06:56.247 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.636+0000 7fe7556d5700 1 -- 172.21.15.133:0/2411864738 wait complete. 2023-07-14T18:06:56.247 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.636+0000 7fe7556d5700 1 Processor -- start 2023-07-14T18:06:56.247 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.637+0000 7fe7556d5700 1 -- start start 2023-07-14T18:06:56.247 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.637+0000 7fe7556d5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe75007bdf0 0x7fe75019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:06:56.247 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.637+0000 7fe7556d5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75019b890 con 0x7fe75007bdf0 2023-07-14T18:06:56.248 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.637+0000 7fe74effd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe75007bdf0 0x7fe75019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:06:56.248 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.637+0000 7fe74effd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe75007bdf0 0x7fe75019b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54672/0 (socket says 172.21.15.133:54672) 2023-07-14T18:06:56.248 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.637+0000 7fe74effd700 1 -- 172.21.15.133:0/2438834604 learned_addr learned my addr 172.21.15.133:0/2438834604 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:06:56.248 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.637+0000 7fe74effd700 1 -- 172.21.15.133:0/2438834604 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe740009550 con 0x7fe75007bdf0 2023-07-14T18:06:56.248 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.638+0000 7fe74effd700 1 --2- 172.21.15.133:0/2438834604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe75007bdf0 0x7fe75019b350 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe740006af0 tx=0x7fe740011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:06:56.248 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.638+0000 7fe737fff700 1 -- 172.21.15.133:0/2438834604 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe740003a40 con 0x7fe75007bdf0 2023-07-14T18:06:56.249 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.638+0000 7fe737fff700 1 -- 172.21.15.133:0/2438834604 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe740003ba0 con 0x7fe75007bdf0 2023-07-14T18:06:56.249 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.638+0000 7fe7556d5700 1 -- 172.21.15.133:0/2438834604 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe75019ba90 con 0x7fe75007bdf0 2023-07-14T18:06:56.249 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.638+0000 7fe737fff700 1 -- 172.21.15.133:0/2438834604 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe74001a700 con 0x7fe75007bdf0 2023-07-14T18:06:56.249 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.638+0000 7fe7556d5700 1 -- 172.21.15.133:0/2438834604 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe75019beb0 con 0x7fe75007bdf0 2023-07-14T18:06:56.249 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.638+0000 7fe737fff700 1 -- 172.21.15.133:0/2438834604 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe74001ac80 con 0x7fe75007bdf0 2023-07-14T18:06:56.249 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.639+0000 7fe737fff700 1 -- 172.21.15.133:0/2438834604 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe740019700 con 0x7fe75007bdf0 2023-07-14T18:06:56.249 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.639+0000 7fe7556d5700 1 -- 172.21.15.133:0/2438834604 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe750061c70 con 0x7fe75007bdf0 2023-07-14T18:06:56.250 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.642+0000 7fe737fff700 1 -- 172.21.15.133:0/2438834604 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fe740024ad0 con 0x7fe75007bdf0 2023-07-14T18:06:56.250 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.710+0000 7fe7556d5700 1 -- 172.21.15.133:0/2438834604 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fe75019e8f0 con 0x7fe75007bdf0 2023-07-14T18:06:56.250 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.713+0000 7fe737fff700 1 -- 172.21.15.133:0/2438834604 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fe740024c70 con 0x7fe75007bdf0 2023-07-14T18:06:56.250 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.714+0000 7fe7556d5700 1 -- 172.21.15.133:0/2438834604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe75007bdf0 msgr2=0x7fe75019b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:06:56.250 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.714+0000 7fe7556d5700 1 --2- 172.21.15.133:0/2438834604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe75007bdf0 0x7fe75019b350 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe740006af0 tx=0x7fe740011ad0 comp rx=0 tx=0).stop 2023-07-14T18:06:56.250 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.714+0000 7fe7556d5700 1 -- 172.21.15.133:0/2438834604 shutdown_connections 2023-07-14T18:06:56.250 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.714+0000 7fe7556d5700 1 --2- 172.21.15.133:0/2438834604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe75007bdf0 0x7fe75019b350 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:06:56.250 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.714+0000 7fe7556d5700 1 -- 172.21.15.133:0/2438834604 >> 172.21.15.133:0/2438834604 conn(0x7fe750100ab0 msgr2=0x7fe750102150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:06:56.251 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.714+0000 7fe7556d5700 1 -- 172.21.15.133:0/2438834604 shutdown_connections 2023-07-14T18:06:56.251 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:55.714+0000 7fe7556d5700 1 -- 172.21.15.133:0/2438834604 wait complete. 2023-07-14T18:06:56.251 INFO:teuthology.orchestra.run.smithi133.stdout:Wrote config to /etc/ceph/ceph.conf 2023-07-14T18:06:56.267 INFO:teuthology.orchestra.run.smithi133.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-07-14T18:06:56.268 INFO:teuthology.orchestra.run.smithi133.stdout:Creating mgr... 2023-07-14T18:06:56.268 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying port 9283 ... 2023-07-14T18:06:56.268 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying port 8765 ... 2023-07-14T18:06:56.649 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mgr.x 2023-07-14T18:06:56.650 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Failed to reset failed state of unit ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mgr.x.service: Unit ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mgr.x.service not loaded. 2023-07-14T18:06:56.854 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c.target.wants/ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mgr.x.service → /etc/systemd/system/ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@.service. 2023-07-14T18:06:57.145 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:56 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2438834604' entity='client.admin' 2023-07-14T18:06:57.145 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:06:56 smithi133 systemd[1]: Starting Ceph mgr.x for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:06:57.637 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:06:57 smithi133 podman[126618]: 2023-07-14T18:06:57.638 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:06:57 smithi133 bash[126618]: 4dbf6a4c75960f5499bd65d0a23dff97510930f5809d1f6732b37c54962679a3 2023-07-14T18:06:57.660 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-14T18:06:57.660 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout disabled 2023-07-14T18:06:57.677 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-14T18:06:57.677 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout inactive 2023-07-14T18:06:57.678 INFO:teuthology.orchestra.run.smithi133.stdout:firewalld.service is not enabled 2023-07-14T18:06:57.678 INFO:teuthology.orchestra.run.smithi133.stdout:Not possible to enable service . firewalld.service is not available 2023-07-14T18:06:57.697 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-14T18:06:57.697 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout disabled 2023-07-14T18:06:57.715 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-14T18:06:57.715 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout inactive 2023-07-14T18:06:57.715 INFO:teuthology.orchestra.run.smithi133.stdout:firewalld.service is not enabled 2023-07-14T18:06:57.715 INFO:teuthology.orchestra.run.smithi133.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-07-14T18:06:57.715 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for mgr to start... 2023-07-14T18:06:57.716 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for mgr... 2023-07-14T18:06:57.895 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:06:57 smithi133 systemd[1]: Started Ceph mgr.x for 06d0f5fa-2271-11ee-9b33-001a4aab830c. 2023-07-14T18:06:57.895 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:06:57 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:06:57.716+0000 7fb2590c5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T18:06:57.895 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:06:57 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:06:57.825+0000 7fb2590c5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T18:06:58.336 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:06:57 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:06:57.946+0000 7fb2590c5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T18:06:58.336 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:06:58 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:06:58.073+0000 7fb2590c5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T18:06:58.645 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:06:58 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3263394869' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-14T18:06:58.645 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:06:58 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:06:58.301+0000 7fb2590c5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T18:06:58.799 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2023-07-14T18:06:58.801 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2023-07-14T18:06:58.802 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsid": "06d0f5fa-2271-11ee-9b33-001a4aab830c", 2023-07-14T18:06:58.802 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "health": { 2023-07-14T18:06:58.802 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-14T18:06:58.802 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-14T18:06:58.802 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-14T18:06:58.802 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:06:58.803 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-14T18:06:58.803 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-14T18:06:58.803 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 0 2023-07-14T18:06:58.803 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2023-07-14T18:06:58.803 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-14T18:06:58.803 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "a" 2023-07-14T18:06:58.803 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2023-07-14T18:06:58.803 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-07-14T18:06:58.804 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-14T18:06:58.804 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T18:06:58.804 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-14T18:06:58.804 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-14T18:06:58.804 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:06:58.804 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-14T18:06:58.804 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T18:06:58.804 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-14T18:06:58.805 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-14T18:06:58.805 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-14T18:06:58.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-14T18:06:58.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-14T18:06:58.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-14T18:06:58.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:06:58.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-14T18:06:58.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-14T18:06:58.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-14T18:06:58.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-14T18:06:58.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-14T18:06:58.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-14T18:06:58.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-14T18:06:58.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-14T18:06:58.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-14T18:06:58.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:06:58.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-14T18:06:58.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T18:06:58.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-14T18:06:58.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-14T18:06:58.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:06:58.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-14T18:06:58.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-14T18:06:58.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-14T18:06:58.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-14T18:06:58.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-14T18:06:58.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-14T18:06:58.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "restful" 2023-07-14T18:06:58.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2023-07-14T18:06:58.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T18:06:58.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:06:58.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-14T18:06:58.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T18:06:58.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modified": "2023-07-14T18:06:49.562361+0000", 2023-07-14T18:06:58.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T18:06:58.815 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:06:58.815 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-14T18:06:58.815 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2023-07-14T18:06:58.815 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.234+0000 7feb6a7c8700 1 Processor -- start 2023-07-14T18:06:58.815 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.234+0000 7feb6a7c8700 1 -- start start 2023-07-14T18:06:58.815 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.235+0000 7feb6a7c8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb6407bdf0 0x7feb6407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:06:58.815 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.235+0000 7feb6a7c8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb6407a830 con 0x7feb6407bdf0 2023-07-14T18:06:58.816 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.235+0000 7feb63fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb6407bdf0 0x7feb6407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:06:58.816 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.235+0000 7feb63fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb6407bdf0 0x7feb6407a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46682/0 (socket says 172.21.15.133:46682) 2023-07-14T18:06:58.816 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.235+0000 7feb63fff700 1 -- 172.21.15.133:0/3035468175 learned_addr learned my addr 172.21.15.133:0/3035468175 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:06:58.816 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.236+0000 7feb63fff700 1 -- 172.21.15.133:0/3035468175 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb6407a970 con 0x7feb6407bdf0 2023-07-14T18:06:58.816 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.236+0000 7feb63fff700 1 --2- 172.21.15.133:0/3035468175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb6407bdf0 0x7feb6407a2f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feb540098a0 tx=0x7feb54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a651ff400954404b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:06:58.816 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.236+0000 7feb62ffd700 1 -- 172.21.15.133:0/3035468175 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb54004160 con 0x7feb6407bdf0 2023-07-14T18:06:58.816 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.236+0000 7feb62ffd700 1 -- 172.21.15.133:0/3035468175 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feb540042c0 con 0x7feb6407bdf0 2023-07-14T18:06:58.817 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.236+0000 7feb62ffd700 1 -- 172.21.15.133:0/3035468175 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb54011a00 con 0x7feb6407bdf0 2023-07-14T18:06:58.817 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.237+0000 7feb6a7c8700 1 -- 172.21.15.133:0/3035468175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb6407bdf0 msgr2=0x7feb6407a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:06:58.817 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.237+0000 7feb6a7c8700 1 --2- 172.21.15.133:0/3035468175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb6407bdf0 0x7feb6407a2f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feb540098a0 tx=0x7feb54009bb0 comp rx=0 tx=0).stop 2023-07-14T18:06:58.817 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.237+0000 7feb6a7c8700 1 -- 172.21.15.133:0/3035468175 shutdown_connections 2023-07-14T18:06:58.817 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.237+0000 7feb6a7c8700 1 --2- 172.21.15.133:0/3035468175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb6407bdf0 0x7feb6407a2f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:06:58.817 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.237+0000 7feb6a7c8700 1 -- 172.21.15.133:0/3035468175 >> 172.21.15.133:0/3035468175 conn(0x7feb64100ab0 msgr2=0x7feb64102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:06:58.817 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.237+0000 7feb6a7c8700 1 -- 172.21.15.133:0/3035468175 shutdown_connections 2023-07-14T18:06:58.817 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.237+0000 7feb6a7c8700 1 -- 172.21.15.133:0/3035468175 wait complete. 2023-07-14T18:06:58.818 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.238+0000 7feb6a7c8700 1 Processor -- start 2023-07-14T18:06:58.818 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.238+0000 7feb6a7c8700 1 -- start start 2023-07-14T18:06:58.818 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.238+0000 7feb6a7c8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb6407bdf0 0x7feb6419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:06:58.818 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.238+0000 7feb6a7c8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb6419b8a0 con 0x7feb6407bdf0 2023-07-14T18:06:58.818 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.239+0000 7feb63fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb6407bdf0 0x7feb6419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:06:58.818 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.239+0000 7feb63fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb6407bdf0 0x7feb6419b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46688/0 (socket says 172.21.15.133:46688) 2023-07-14T18:06:58.818 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.239+0000 7feb63fff700 1 -- 172.21.15.133:0/3263394869 learned_addr learned my addr 172.21.15.133:0/3263394869 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:06:58.819 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.239+0000 7feb63fff700 1 -- 172.21.15.133:0/3263394869 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb54009550 con 0x7feb6407bdf0 2023-07-14T18:06:58.819 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.239+0000 7feb63fff700 1 --2- 172.21.15.133:0/3263394869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb6407bdf0 0x7feb6419b360 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feb54009fb0 tx=0x7feb54011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:06:58.819 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.240+0000 7feb617fa700 1 -- 172.21.15.133:0/3263394869 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb54003930 con 0x7feb6407bdf0 2023-07-14T18:06:58.819 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.240+0000 7feb617fa700 1 -- 172.21.15.133:0/3263394869 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feb54003a90 con 0x7feb6407bdf0 2023-07-14T18:06:58.819 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.240+0000 7feb6a7c8700 1 -- 172.21.15.133:0/3263394869 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb6419baa0 con 0x7feb6407bdf0 2023-07-14T18:06:58.819 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.240+0000 7feb617fa700 1 -- 172.21.15.133:0/3263394869 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb540195e0 con 0x7feb6407bdf0 2023-07-14T18:06:58.819 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.240+0000 7feb6a7c8700 1 -- 172.21.15.133:0/3263394869 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb6419bec0 con 0x7feb6407bdf0 2023-07-14T18:06:58.820 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.240+0000 7feb617fa700 1 -- 172.21.15.133:0/3263394869 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7feb54019b50 con 0x7feb6407bdf0 2023-07-14T18:06:58.820 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.240+0000 7feb617fa700 1 -- 172.21.15.133:0/3263394869 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7feb540243f0 con 0x7feb6407bdf0 2023-07-14T18:06:58.820 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.241+0000 7feb6a7c8700 1 -- 172.21.15.133:0/3263394869 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb64061c70 con 0x7feb6407bdf0 2023-07-14T18:06:58.820 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.244+0000 7feb617fa700 1 -- 172.21.15.133:0/3263394869 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7feb5402de30 con 0x7feb6407bdf0 2023-07-14T18:06:58.820 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.320+0000 7feb6a7c8700 1 -- 172.21.15.133:0/3263394869 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7feb6419e9f0 con 0x7feb6407bdf0 2023-07-14T18:06:58.820 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.320+0000 7feb617fa700 1 -- 172.21.15.133:0/3263394869 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7feb54003c00 con 0x7feb6407bdf0 2023-07-14T18:06:58.820 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.321+0000 7feb6a7c8700 1 -- 172.21.15.133:0/3263394869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb6407bdf0 msgr2=0x7feb6419b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:06:58.821 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.321+0000 7feb6a7c8700 1 --2- 172.21.15.133:0/3263394869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb6407bdf0 0x7feb6419b360 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feb54009fb0 tx=0x7feb54011ad0 comp rx=0 tx=0).stop 2023-07-14T18:06:58.821 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.321+0000 7feb6a7c8700 1 -- 172.21.15.133:0/3263394869 shutdown_connections 2023-07-14T18:06:58.821 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.321+0000 7feb6a7c8700 1 --2- 172.21.15.133:0/3263394869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb6407bdf0 0x7feb6419b360 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:06:58.821 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.321+0000 7feb6a7c8700 1 -- 172.21.15.133:0/3263394869 >> 172.21.15.133:0/3263394869 conn(0x7feb64100ab0 msgr2=0x7feb64101850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:06:58.821 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.321+0000 7feb6a7c8700 1 -- 172.21.15.133:0/3263394869 shutdown_connections 2023-07-14T18:06:58.821 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:06:58.321+0000 7feb6a7c8700 1 -- 172.21.15.133:0/3263394869 wait complete. 2023-07-14T18:06:58.821 INFO:teuthology.orchestra.run.smithi133.stdout:mgr not available, waiting (1/15)... 2023-07-14T18:06:59.027 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:06:58 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:06:58.714+0000 7fb2590c5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T18:06:59.027 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:06:58 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:06:58.833+0000 7fb2590c5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T18:06:59.028 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:06:58 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:06:58.994+0000 7fb2590c5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T18:06:59.395 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:06:59 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:06:59.116+0000 7fb2590c5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T18:07:00.145 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:06:59 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:06:59.651+0000 7fb2590c5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T18:07:00.895 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:00 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:00.417+0000 7fb2590c5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T18:07:00.895 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:00 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:00.624+0000 7fb2590c5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T18:07:01.811 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:01 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3334878145' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-14T18:07:01.811 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:01 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:01.658+0000 7fb2590c5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T18:07:01.910 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2023-07-14T18:07:01.910 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2023-07-14T18:07:01.910 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsid": "06d0f5fa-2271-11ee-9b33-001a4aab830c", 2023-07-14T18:07:01.910 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "health": { 2023-07-14T18:07:01.910 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-14T18:07:01.910 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-14T18:07:01.911 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-14T18:07:01.911 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:07:01.911 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-14T18:07:01.911 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-14T18:07:01.911 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 0 2023-07-14T18:07:01.911 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2023-07-14T18:07:01.911 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-14T18:07:01.912 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "a" 2023-07-14T18:07:01.912 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2023-07-14T18:07:01.912 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-07-14T18:07:01.912 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-14T18:07:01.912 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T18:07:01.912 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-14T18:07:01.912 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-14T18:07:01.912 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:07:01.913 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-14T18:07:01.913 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T18:07:01.913 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-14T18:07:01.913 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-14T18:07:01.913 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-14T18:07:01.919 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-14T18:07:01.919 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-14T18:07:01.919 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-14T18:07:01.919 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:07:01.920 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-14T18:07:01.920 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-14T18:07:01.920 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-14T18:07:01.920 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-14T18:07:01.920 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-14T18:07:01.920 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-14T18:07:01.920 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-14T18:07:01.921 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-14T18:07:01.921 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-14T18:07:01.921 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:07:01.921 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-14T18:07:01.921 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T18:07:01.921 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-14T18:07:01.921 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-14T18:07:01.922 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:07:01.922 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-14T18:07:01.922 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-14T18:07:01.922 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-14T18:07:01.922 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-14T18:07:01.922 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-14T18:07:01.922 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-14T18:07:01.922 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "restful" 2023-07-14T18:07:01.923 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2023-07-14T18:07:01.923 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T18:07:01.923 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:07:01.923 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-14T18:07:01.923 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T18:07:01.923 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modified": "2023-07-14T18:06:49.562361+0000", 2023-07-14T18:07:01.923 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T18:07:01.923 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:07:01.924 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-14T18:07:01.924 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2023-07-14T18:07:01.924 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.332+0000 7f133a99a700 1 Processor -- start 2023-07-14T18:07:01.924 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.332+0000 7f133a99a700 1 -- start start 2023-07-14T18:07:01.924 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.332+0000 7f133a99a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f133407bdf0 0x7f133407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:01.924 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.332+0000 7f133a99a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133407a830 con 0x7f133407bdf0 2023-07-14T18:07:01.924 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.332+0000 7f1339998700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f133407bdf0 0x7f133407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:01.925 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.332+0000 7f1339998700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f133407bdf0 0x7f133407a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46690/0 (socket says 172.21.15.133:46690) 2023-07-14T18:07:01.925 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.332+0000 7f1339998700 1 -- 172.21.15.133:0/924275962 learned_addr learned my addr 172.21.15.133:0/924275962 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:01.925 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.333+0000 7f1339998700 1 -- 172.21.15.133:0/924275962 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133407a970 con 0x7f133407bdf0 2023-07-14T18:07:01.925 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.333+0000 7f1339998700 1 --2- 172.21.15.133:0/924275962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f133407bdf0 0x7f133407a2f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f13300098a0 tx=0x7f1330009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc561eeb82ba40f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:01.925 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.333+0000 7f1338996700 1 -- 172.21.15.133:0/924275962 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1330004160 con 0x7f133407bdf0 2023-07-14T18:07:01.925 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.333+0000 7f1338996700 1 -- 172.21.15.133:0/924275962 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f13300042c0 con 0x7f133407bdf0 2023-07-14T18:07:01.925 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.333+0000 7f1338996700 1 -- 172.21.15.133:0/924275962 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1330011a00 con 0x7f133407bdf0 2023-07-14T18:07:01.926 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.334+0000 7f133a99a700 1 -- 172.21.15.133:0/924275962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f133407bdf0 msgr2=0x7f133407a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:01.926 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.334+0000 7f133a99a700 1 --2- 172.21.15.133:0/924275962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f133407bdf0 0x7f133407a2f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f13300098a0 tx=0x7f1330009bb0 comp rx=0 tx=0).stop 2023-07-14T18:07:01.926 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.334+0000 7f133a99a700 1 -- 172.21.15.133:0/924275962 shutdown_connections 2023-07-14T18:07:01.926 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.334+0000 7f133a99a700 1 --2- 172.21.15.133:0/924275962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f133407bdf0 0x7f133407a2f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:01.926 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.334+0000 7f133a99a700 1 -- 172.21.15.133:0/924275962 >> 172.21.15.133:0/924275962 conn(0x7f1334100900 msgr2=0x7f1334102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:01.926 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.334+0000 7f133a99a700 1 -- 172.21.15.133:0/924275962 shutdown_connections 2023-07-14T18:07:01.926 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.334+0000 7f133a99a700 1 -- 172.21.15.133:0/924275962 wait complete. 2023-07-14T18:07:01.926 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.335+0000 7f133a99a700 1 Processor -- start 2023-07-14T18:07:01.927 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.335+0000 7f133a99a700 1 -- start start 2023-07-14T18:07:01.927 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.336+0000 7f133a99a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f133407bdf0 0x7f133419b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:01.927 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.336+0000 7f133a99a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133419b700 con 0x7f133407bdf0 2023-07-14T18:07:01.927 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.336+0000 7f1339998700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f133407bdf0 0x7f133419b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:01.927 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.336+0000 7f1339998700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f133407bdf0 0x7f133419b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46702/0 (socket says 172.21.15.133:46702) 2023-07-14T18:07:01.927 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.336+0000 7f1339998700 1 -- 172.21.15.133:0/3334878145 learned_addr learned my addr 172.21.15.133:0/3334878145 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:01.927 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.336+0000 7f1339998700 1 -- 172.21.15.133:0/3334878145 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1330009550 con 0x7f133407bdf0 2023-07-14T18:07:01.927 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.337+0000 7f1339998700 1 --2- 172.21.15.133:0/3334878145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f133407bdf0 0x7f133419b1c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1330006af0 tx=0x7f1330011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:01.928 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.337+0000 7f132affd700 1 -- 172.21.15.133:0/3334878145 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1330003a40 con 0x7f133407bdf0 2023-07-14T18:07:01.928 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.337+0000 7f132affd700 1 -- 172.21.15.133:0/3334878145 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1330003ba0 con 0x7f133407bdf0 2023-07-14T18:07:01.928 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.337+0000 7f133a99a700 1 -- 172.21.15.133:0/3334878145 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f133419b900 con 0x7f133407bdf0 2023-07-14T18:07:01.928 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.337+0000 7f132affd700 1 -- 172.21.15.133:0/3334878145 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f133001a700 con 0x7f133407bdf0 2023-07-14T18:07:01.928 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.337+0000 7f133a99a700 1 -- 172.21.15.133:0/3334878145 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f133419bd20 con 0x7f133407bdf0 2023-07-14T18:07:01.928 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.338+0000 7f132affd700 1 -- 172.21.15.133:0/3334878145 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f133001a980 con 0x7f133407bdf0 2023-07-14T18:07:01.929 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.338+0000 7f132affd700 1 -- 172.21.15.133:0/3334878145 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f133002dd60 con 0x7f133407bdf0 2023-07-14T18:07:01.929 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.338+0000 7f133a99a700 1 -- 172.21.15.133:0/3334878145 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f133410b5d0 con 0x7f133407bdf0 2023-07-14T18:07:01.929 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.341+0000 7f132affd700 1 -- 172.21.15.133:0/3334878145 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f13300248d0 con 0x7f133407bdf0 2023-07-14T18:07:01.929 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.416+0000 7f133a99a700 1 -- 172.21.15.133:0/3334878145 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f133419ed60 con 0x7f133407bdf0 2023-07-14T18:07:01.929 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.416+0000 7f132affd700 1 -- 172.21.15.133:0/3334878145 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f133001ad70 con 0x7f133407bdf0 2023-07-14T18:07:01.929 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.417+0000 7f133a99a700 1 -- 172.21.15.133:0/3334878145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f133407bdf0 msgr2=0x7f133419b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:01.929 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.417+0000 7f133a99a700 1 --2- 172.21.15.133:0/3334878145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f133407bdf0 0x7f133419b1c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1330006af0 tx=0x7f1330011ad0 comp rx=0 tx=0).stop 2023-07-14T18:07:01.930 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.418+0000 7f133a99a700 1 -- 172.21.15.133:0/3334878145 shutdown_connections 2023-07-14T18:07:01.930 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.418+0000 7f133a99a700 1 --2- 172.21.15.133:0/3334878145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f133407bdf0 0x7f133419b1c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:01.930 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.418+0000 7f133a99a700 1 -- 172.21.15.133:0/3334878145 >> 172.21.15.133:0/3334878145 conn(0x7f1334100900 msgr2=0x7f1334101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:01.930 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.418+0000 7f133a99a700 1 -- 172.21.15.133:0/3334878145 shutdown_connections 2023-07-14T18:07:01.930 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:01.418+0000 7f133a99a700 1 -- 172.21.15.133:0/3334878145 wait complete. 2023-07-14T18:07:01.930 INFO:teuthology.orchestra.run.smithi133.stdout:mgr not available, waiting (2/15)... 2023-07-14T18:07:02.145 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:01 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:01.778+0000 7fb2590c5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T18:07:02.145 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:02 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:02.003+0000 7fb2590c5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T18:07:02.645 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:02 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:02.356+0000 7fb2590c5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T18:07:03.145 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:02 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:02.634+0000 7fb2590c5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T18:07:03.145 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:02 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:02.825+0000 7fb2590c5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T18:07:03.645 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:03 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:03.159+0000 7fb2590c5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T18:07:04.395 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:04 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:04.086+0000 7fb2590c5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T18:07:04.395 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:04 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:04.268+0000 7fb2590c5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T18:07:04.886 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:04 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3305679594' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-14T18:07:04.886 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:04 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:04.522+0000 7fb2590c5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T18:07:04.887 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:04 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:04.693+0000 7fb2590c5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T18:07:04.894 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2023-07-14T18:07:04.895 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2023-07-14T18:07:04.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsid": "06d0f5fa-2271-11ee-9b33-001a4aab830c", 2023-07-14T18:07:04.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "health": { 2023-07-14T18:07:04.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-14T18:07:04.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-14T18:07:04.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-14T18:07:04.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:07:04.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-14T18:07:04.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-14T18:07:04.897 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 0 2023-07-14T18:07:04.898 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2023-07-14T18:07:04.898 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-14T18:07:04.898 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "a" 2023-07-14T18:07:04.898 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2023-07-14T18:07:04.898 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2023-07-14T18:07:04.898 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-14T18:07:04.899 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T18:07:04.899 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-14T18:07:04.899 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-14T18:07:04.899 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:07:04.899 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-14T18:07:04.899 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T18:07:04.899 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-14T18:07:04.905 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-14T18:07:04.905 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-14T18:07:04.905 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-14T18:07:04.905 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-14T18:07:04.906 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-14T18:07:04.906 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:07:04.906 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-14T18:07:04.906 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-14T18:07:04.906 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-14T18:07:04.906 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-14T18:07:04.906 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-14T18:07:04.906 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-14T18:07:04.907 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-14T18:07:04.907 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-14T18:07:04.907 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-14T18:07:04.907 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:07:04.907 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-14T18:07:04.907 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T18:07:04.907 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-14T18:07:04.907 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-14T18:07:04.908 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:07:04.908 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-14T18:07:04.908 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-14T18:07:04.908 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-14T18:07:04.908 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-14T18:07:04.908 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-14T18:07:04.908 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-14T18:07:04.908 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "restful" 2023-07-14T18:07:04.909 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2023-07-14T18:07:04.909 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T18:07:04.909 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:07:04.909 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-14T18:07:04.909 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T18:07:04.909 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modified": "2023-07-14T18:06:49.562361+0000", 2023-07-14T18:07:04.909 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T18:07:04.909 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:07:04.910 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-14T18:07:04.910 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2023-07-14T18:07:04.910 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.432+0000 7fc619e3c700 1 Processor -- start 2023-07-14T18:07:04.910 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.432+0000 7fc619e3c700 1 -- start start 2023-07-14T18:07:04.910 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.432+0000 7fc619e3c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc61407bdd0 0x7fc61407a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:04.910 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.432+0000 7fc619e3c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61407a810 con 0x7fc61407bdd0 2023-07-14T18:07:04.910 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.432+0000 7fc618e3a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc61407bdd0 0x7fc61407a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:04.911 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.432+0000 7fc618e3a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc61407bdd0 0x7fc61407a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46706/0 (socket says 172.21.15.133:46706) 2023-07-14T18:07:04.911 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.433+0000 7fc618e3a700 1 -- 172.21.15.133:0/3485403308 learned_addr learned my addr 172.21.15.133:0/3485403308 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:04.911 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.433+0000 7fc618e3a700 1 -- 172.21.15.133:0/3485403308 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc61407a950 con 0x7fc61407bdd0 2023-07-14T18:07:04.911 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.433+0000 7fc618e3a700 1 --2- 172.21.15.133:0/3485403308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc61407bdd0 0x7fc61407a2d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc6080098a0 tx=0x7fc608009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b6c678c0653424f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:04.911 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.433+0000 7fc6137fe700 1 -- 172.21.15.133:0/3485403308 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc608004160 con 0x7fc61407bdd0 2023-07-14T18:07:04.911 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.433+0000 7fc6137fe700 1 -- 172.21.15.133:0/3485403308 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc6080042c0 con 0x7fc61407bdd0 2023-07-14T18:07:04.911 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.433+0000 7fc6137fe700 1 -- 172.21.15.133:0/3485403308 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc608011a00 con 0x7fc61407bdd0 2023-07-14T18:07:04.911 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.434+0000 7fc619e3c700 1 -- 172.21.15.133:0/3485403308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc61407bdd0 msgr2=0x7fc61407a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:04.912 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.434+0000 7fc619e3c700 1 --2- 172.21.15.133:0/3485403308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc61407bdd0 0x7fc61407a2d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc6080098a0 tx=0x7fc608009bb0 comp rx=0 tx=0).stop 2023-07-14T18:07:04.912 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.434+0000 7fc619e3c700 1 -- 172.21.15.133:0/3485403308 shutdown_connections 2023-07-14T18:07:04.912 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.434+0000 7fc619e3c700 1 --2- 172.21.15.133:0/3485403308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc61407bdd0 0x7fc61407a2d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:04.912 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.434+0000 7fc619e3c700 1 -- 172.21.15.133:0/3485403308 >> 172.21.15.133:0/3485403308 conn(0x7fc614100900 msgr2=0x7fc614102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:04.912 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.434+0000 7fc619e3c700 1 -- 172.21.15.133:0/3485403308 shutdown_connections 2023-07-14T18:07:04.912 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.434+0000 7fc619e3c700 1 -- 172.21.15.133:0/3485403308 wait complete. 2023-07-14T18:07:04.912 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.435+0000 7fc619e3c700 1 Processor -- start 2023-07-14T18:07:04.913 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.435+0000 7fc619e3c700 1 -- start start 2023-07-14T18:07:04.913 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.436+0000 7fc619e3c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc61407bdd0 0x7fc61419b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:04.913 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.436+0000 7fc619e3c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61419b700 con 0x7fc61407bdd0 2023-07-14T18:07:04.913 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.436+0000 7fc618e3a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc61407bdd0 0x7fc61419b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:04.913 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.436+0000 7fc618e3a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc61407bdd0 0x7fc61419b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46714/0 (socket says 172.21.15.133:46714) 2023-07-14T18:07:04.913 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.436+0000 7fc618e3a700 1 -- 172.21.15.133:0/3305679594 learned_addr learned my addr 172.21.15.133:0/3305679594 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:04.913 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.436+0000 7fc618e3a700 1 -- 172.21.15.133:0/3305679594 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc608009550 con 0x7fc61407bdd0 2023-07-14T18:07:04.913 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.436+0000 7fc618e3a700 1 --2- 172.21.15.133:0/3305679594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc61407bdd0 0x7fc61419b1c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc608009fb0 tx=0x7fc608011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:04.914 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.437+0000 7fc611ffb700 1 -- 172.21.15.133:0/3305679594 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc608003930 con 0x7fc61407bdd0 2023-07-14T18:07:04.914 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.437+0000 7fc611ffb700 1 -- 172.21.15.133:0/3305679594 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc608003a90 con 0x7fc61407bdd0 2023-07-14T18:07:04.914 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.437+0000 7fc619e3c700 1 -- 172.21.15.133:0/3305679594 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc61419b900 con 0x7fc61407bdd0 2023-07-14T18:07:04.914 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.437+0000 7fc611ffb700 1 -- 172.21.15.133:0/3305679594 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6080195e0 con 0x7fc61407bdd0 2023-07-14T18:07:04.914 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.437+0000 7fc619e3c700 1 -- 172.21.15.133:0/3305679594 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc61419bd20 con 0x7fc61407bdd0 2023-07-14T18:07:04.914 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.437+0000 7fc611ffb700 1 -- 172.21.15.133:0/3305679594 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc608019ba0 con 0x7fc61407bdd0 2023-07-14T18:07:04.914 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.437+0000 7fc611ffb700 1 -- 172.21.15.133:0/3305679594 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc6080243f0 con 0x7fc61407bdd0 2023-07-14T18:07:04.915 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.438+0000 7fc619e3c700 1 -- 172.21.15.133:0/3305679594 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc614061c70 con 0x7fc61407bdd0 2023-07-14T18:07:04.915 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.440+0000 7fc611ffb700 1 -- 172.21.15.133:0/3305679594 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fc608003c00 con 0x7fc61407bdd0 2023-07-14T18:07:04.915 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.516+0000 7fc619e3c700 1 -- 172.21.15.133:0/3305679594 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc61419e850 con 0x7fc61407bdd0 2023-07-14T18:07:04.915 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.517+0000 7fc611ffb700 1 -- 172.21.15.133:0/3305679594 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fc60801a9d0 con 0x7fc61407bdd0 2023-07-14T18:07:04.915 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.518+0000 7fc619e3c700 1 -- 172.21.15.133:0/3305679594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc61407bdd0 msgr2=0x7fc61419b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:04.915 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.518+0000 7fc619e3c700 1 --2- 172.21.15.133:0/3305679594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc61407bdd0 0x7fc61419b1c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc608009fb0 tx=0x7fc608011ad0 comp rx=0 tx=0).stop 2023-07-14T18:07:04.915 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.518+0000 7fc619e3c700 1 -- 172.21.15.133:0/3305679594 shutdown_connections 2023-07-14T18:07:04.916 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.518+0000 7fc619e3c700 1 --2- 172.21.15.133:0/3305679594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc61407bdd0 0x7fc61419b1c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:04.916 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.518+0000 7fc619e3c700 1 -- 172.21.15.133:0/3305679594 >> 172.21.15.133:0/3305679594 conn(0x7fc614100900 msgr2=0x7fc614101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:04.916 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.518+0000 7fc619e3c700 1 -- 172.21.15.133:0/3305679594 shutdown_connections 2023-07-14T18:07:04.916 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:04.518+0000 7fc619e3c700 1 -- 172.21.15.133:0/3305679594 wait complete. 2023-07-14T18:07:04.916 INFO:teuthology.orchestra.run.smithi133.stdout:mgr not available, waiting (3/15)... 2023-07-14T18:07:05.145 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:04 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:04.976+0000 7fb2590c5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T18:07:06.395 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:06 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:06.028+0000 7fb2590c5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T18:07:06.395 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:06 smithi133 ceph-mon[126384]: Activating manager daemon x 2023-07-14T18:07:06.396 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:06 smithi133 ceph-mon[126384]: mgrmap e2: x(active, starting, since 0.00446645s) 2023-07-14T18:07:06.396 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:06 smithi133 ceph-mon[126384]: from='mgr.14100 172.21.15.133:0/2134864737' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-14T18:07:06.396 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:06 smithi133 ceph-mon[126384]: from='mgr.14100 172.21.15.133:0/2134864737' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-14T18:07:06.396 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:06 smithi133 ceph-mon[126384]: from='mgr.14100 172.21.15.133:0/2134864737' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-14T18:07:06.396 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:06 smithi133 ceph-mon[126384]: from='mgr.14100 172.21.15.133:0/2134864737' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T18:07:06.396 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:06 smithi133 ceph-mon[126384]: from='mgr.14100 172.21.15.133:0/2134864737' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-14T18:07:06.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:06 smithi133 ceph-mon[126384]: Manager daemon x is now available 2023-07-14T18:07:06.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:06 smithi133 ceph-mon[126384]: from='mgr.14100 172.21.15.133:0/2134864737' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-14T18:07:06.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:06 smithi133 ceph-mon[126384]: from='mgr.14100 172.21.15.133:0/2134864737' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-14T18:07:06.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:06 smithi133 ceph-mon[126384]: from='mgr.14100 172.21.15.133:0/2134864737' entity='mgr.x' 2023-07-14T18:07:06.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:06 smithi133 ceph-mon[126384]: from='mgr.14100 172.21.15.133:0/2134864737' entity='mgr.x' 2023-07-14T18:07:06.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:06 smithi133 ceph-mon[126384]: from='mgr.14100 172.21.15.133:0/2134864737' entity='mgr.x' 2023-07-14T18:07:08.189 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2023-07-14T18:07:08.189 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2023-07-14T18:07:08.190 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsid": "06d0f5fa-2271-11ee-9b33-001a4aab830c", 2023-07-14T18:07:08.190 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "health": { 2023-07-14T18:07:08.190 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-14T18:07:08.190 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-14T18:07:08.190 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-14T18:07:08.190 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:07:08.190 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-14T18:07:08.190 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-14T18:07:08.191 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 0 2023-07-14T18:07:08.191 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2023-07-14T18:07:08.191 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-14T18:07:08.191 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "a" 2023-07-14T18:07:08.191 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2023-07-14T18:07:08.191 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2023-07-14T18:07:08.191 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-14T18:07:08.191 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T18:07:08.192 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-14T18:07:08.192 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-14T18:07:08.192 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:07:08.192 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-14T18:07:08.192 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T18:07:08.192 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-14T18:07:08.192 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-14T18:07:08.193 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-14T18:07:08.198 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-14T18:07:08.199 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-14T18:07:08.199 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-14T18:07:08.199 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:07:08.199 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-14T18:07:08.199 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-14T18:07:08.199 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-14T18:07:08.199 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-14T18:07:08.200 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-14T18:07:08.200 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-14T18:07:08.200 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-14T18:07:08.200 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-14T18:07:08.200 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-14T18:07:08.200 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:07:08.200 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-14T18:07:08.201 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T18:07:08.201 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-14T18:07:08.201 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-14T18:07:08.201 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:07:08.201 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-14T18:07:08.201 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-14T18:07:08.201 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-14T18:07:08.201 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-14T18:07:08.202 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-14T18:07:08.202 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-14T18:07:08.202 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "restful" 2023-07-14T18:07:08.202 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2023-07-14T18:07:08.202 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T18:07:08.202 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:07:08.202 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-14T18:07:08.202 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T18:07:08.203 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modified": "2023-07-14T18:06:49.562361+0000", 2023-07-14T18:07:08.203 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T18:07:08.203 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-07-14T18:07:08.203 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-14T18:07:08.203 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2023-07-14T18:07:08.203 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.436+0000 7f97d9e85700 1 Processor -- start 2023-07-14T18:07:08.203 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.436+0000 7f97d9e85700 1 -- start start 2023-07-14T18:07:08.203 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.437+0000 7f97d9e85700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d407be30 0x7f97d407a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:08.204 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.437+0000 7f97d9e85700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d407a870 con 0x7f97d407be30 2023-07-14T18:07:08.204 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.437+0000 7f97d37fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d407be30 0x7f97d407a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:08.204 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.437+0000 7f97d37fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d407be30 0x7f97d407a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38384/0 (socket says 172.21.15.133:38384) 2023-07-14T18:07:08.204 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.437+0000 7f97d37fe700 1 -- 172.21.15.133:0/138828792 learned_addr learned my addr 172.21.15.133:0/138828792 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:08.204 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.437+0000 7f97d37fe700 1 -- 172.21.15.133:0/138828792 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d407a9b0 con 0x7f97d407be30 2023-07-14T18:07:08.204 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.437+0000 7f97d37fe700 1 --2- 172.21.15.133:0/138828792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d407be30 0x7f97d407a330 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f97c40098a0 tx=0x7f97c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d663e4311a99ce server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:08.204 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.438+0000 7f97d27fc700 1 -- 172.21.15.133:0/138828792 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97c4004160 con 0x7f97d407be30 2023-07-14T18:07:08.205 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.438+0000 7f97d27fc700 1 -- 172.21.15.133:0/138828792 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f97c40042c0 con 0x7f97d407be30 2023-07-14T18:07:08.205 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.438+0000 7f97d27fc700 1 -- 172.21.15.133:0/138828792 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97c4011a00 con 0x7f97d407be30 2023-07-14T18:07:08.205 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.438+0000 7f97d9e85700 1 -- 172.21.15.133:0/138828792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d407be30 msgr2=0x7f97d407a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:08.205 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.438+0000 7f97d9e85700 1 --2- 172.21.15.133:0/138828792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d407be30 0x7f97d407a330 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f97c40098a0 tx=0x7f97c4009bb0 comp rx=0 tx=0).stop 2023-07-14T18:07:08.205 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.438+0000 7f97d9e85700 1 -- 172.21.15.133:0/138828792 shutdown_connections 2023-07-14T18:07:08.205 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.438+0000 7f97d9e85700 1 --2- 172.21.15.133:0/138828792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d407be30 0x7f97d407a330 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:08.205 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.438+0000 7f97d9e85700 1 -- 172.21.15.133:0/138828792 >> 172.21.15.133:0/138828792 conn(0x7f97d4100ab0 msgr2=0x7f97d4102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:08.205 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.438+0000 7f97d9e85700 1 -- 172.21.15.133:0/138828792 shutdown_connections 2023-07-14T18:07:08.206 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.439+0000 7f97d9e85700 1 -- 172.21.15.133:0/138828792 wait complete. 2023-07-14T18:07:08.206 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.439+0000 7f97d9e85700 1 Processor -- start 2023-07-14T18:07:08.206 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.440+0000 7f97d9e85700 1 -- start start 2023-07-14T18:07:08.206 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.440+0000 7f97d9e85700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d407be30 0x7f97d419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:08.206 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.440+0000 7f97d9e85700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d419b8b0 con 0x7f97d407be30 2023-07-14T18:07:08.206 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.440+0000 7f97d37fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d407be30 0x7f97d419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:08.206 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.440+0000 7f97d37fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d407be30 0x7f97d419b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38400/0 (socket says 172.21.15.133:38400) 2023-07-14T18:07:08.206 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.440+0000 7f97d37fe700 1 -- 172.21.15.133:0/1235823845 learned_addr learned my addr 172.21.15.133:0/1235823845 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:08.207 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.440+0000 7f97d37fe700 1 -- 172.21.15.133:0/1235823845 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c4009550 con 0x7f97d407be30 2023-07-14T18:07:08.207 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.441+0000 7f97d37fe700 1 --2- 172.21.15.133:0/1235823845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d407be30 0x7f97d419b370 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f97c4006af0 tx=0x7f97c4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:08.207 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.441+0000 7f97d0ff9700 1 -- 172.21.15.133:0/1235823845 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97c4003a40 con 0x7f97d407be30 2023-07-14T18:07:08.207 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.441+0000 7f97d0ff9700 1 -- 172.21.15.133:0/1235823845 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f97c4003ba0 con 0x7f97d407be30 2023-07-14T18:07:08.207 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.441+0000 7f97d0ff9700 1 -- 172.21.15.133:0/1235823845 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97c401a700 con 0x7f97d407be30 2023-07-14T18:07:08.207 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.441+0000 7f97d9e85700 1 -- 172.21.15.133:0/1235823845 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d419bab0 con 0x7f97d407be30 2023-07-14T18:07:08.207 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.441+0000 7f97d9e85700 1 -- 172.21.15.133:0/1235823845 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d419bed0 con 0x7f97d407be30 2023-07-14T18:07:08.208 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.442+0000 7f97d0ff9700 1 -- 172.21.15.133:0/1235823845 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 3) v1 ==== 45366+0+0 (secure 0 0 0) 0x7f97c4024ad0 con 0x7f97d407be30 2023-07-14T18:07:08.208 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.442+0000 7f97d9e85700 1 -- 172.21.15.133:0/1235823845 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97d4061c70 con 0x7f97d407be30 2023-07-14T18:07:08.208 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.443+0000 7f97d0ff9700 1 --2- 172.21.15.133:0/1235823845 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7f97bc0385e0 0x7f97bc03aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:08.208 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.443+0000 7f97d0ff9700 1 -- 172.21.15.133:0/1235823845 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f97c404c420 con 0x7f97d407be30 2023-07-14T18:07:08.208 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.443+0000 7f97d2ffd700 1 --2- 172.21.15.133:0/1235823845 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7f97bc0385e0 0x7f97bc03aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:08.208 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.443+0000 7f97d2ffd700 1 --2- 172.21.15.133:0/1235823845 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7f97bc0385e0 0x7f97bc03aa90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f97c8006fd0 tx=0x7f97c8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:08.208 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.448+0000 7f97d0ff9700 1 -- 172.21.15.133:0/1235823845 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f97c4024430 con 0x7f97d407be30 2023-07-14T18:07:08.208 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.716+0000 7f97d9e85700 1 -- 172.21.15.133:0/1235823845 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f97d419f050 con 0x7f97d407be30 2023-07-14T18:07:08.209 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.716+0000 7f97d0ff9700 1 -- 172.21.15.133:0/1235823845 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f97c4018350 con 0x7f97d407be30 2023-07-14T18:07:08.209 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.719+0000 7f97d9e85700 1 -- 172.21.15.133:0/1235823845 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7f97bc0385e0 msgr2=0x7f97bc03aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:08.209 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.719+0000 7f97d9e85700 1 --2- 172.21.15.133:0/1235823845 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7f97bc0385e0 0x7f97bc03aa90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f97c8006fd0 tx=0x7f97c8006e40 comp rx=0 tx=0).stop 2023-07-14T18:07:08.209 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.719+0000 7f97d9e85700 1 -- 172.21.15.133:0/1235823845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d407be30 msgr2=0x7f97d419b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:08.209 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.719+0000 7f97d9e85700 1 --2- 172.21.15.133:0/1235823845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d407be30 0x7f97d419b370 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f97c4006af0 tx=0x7f97c4011ad0 comp rx=0 tx=0).stop 2023-07-14T18:07:08.209 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.719+0000 7f97d9e85700 1 -- 172.21.15.133:0/1235823845 shutdown_connections 2023-07-14T18:07:08.209 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.719+0000 7f97d9e85700 1 --2- 172.21.15.133:0/1235823845 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7f97bc0385e0 0x7f97bc03aa90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:08.209 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.719+0000 7f97d9e85700 1 --2- 172.21.15.133:0/1235823845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d407be30 0x7f97d419b370 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:08.210 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.719+0000 7f97d9e85700 1 -- 172.21.15.133:0/1235823845 >> 172.21.15.133:0/1235823845 conn(0x7f97d4100ab0 msgr2=0x7f97d4101880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:08.210 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.719+0000 7f97d9e85700 1 -- 172.21.15.133:0/1235823845 shutdown_connections 2023-07-14T18:07:08.210 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:07.719+0000 7f97d9e85700 1 -- 172.21.15.133:0/1235823845 wait complete. 2023-07-14T18:07:08.210 INFO:teuthology.orchestra.run.smithi133.stdout:mgr is available 2023-07-14T18:07:08.395 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:08 smithi133 ceph-mon[126384]: mgrmap e3: x(active, since 1.00878s) 2023-07-14T18:07:08.395 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:08 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1235823845' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-14T18:07:09.353 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:09 smithi133 ceph-mon[126384]: mgrmap e4: x(active, since 2s) 2023-07-14T18:07:09.353 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:09 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1646846652' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-14T18:07:09.374 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2023-07-14T18:07:09.375 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout [global] 2023-07-14T18:07:09.375 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout fsid = 06d0f5fa-2271-11ee-9b33-001a4aab830c 2023-07-14T18:07:09.375 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-14T18:07:09.375 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-14T18:07:09.375 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-14T18:07:09.375 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-14T18:07:09.376 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2023-07-14T18:07:09.376 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-14T18:07:09.376 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-14T18:07:09.376 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2023-07-14T18:07:09.376 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout [osd] 2023-07-14T18:07:09.376 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-14T18:07:09.376 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-14T18:07:09.377 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-14T18:07:09.377 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-14T18:07:09.377 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-14T18:07:09.377 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.694+0000 7f91ed89b700 1 Processor -- start 2023-07-14T18:07:09.377 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.694+0000 7f91ed89b700 1 -- start start 2023-07-14T18:07:09.377 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.695+0000 7f91ed89b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91e807be30 0x7f91e807a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:09.377 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.695+0000 7f91ed89b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e807a870 con 0x7f91e807be30 2023-07-14T18:07:09.377 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.695+0000 7f91e6ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91e807be30 0x7f91e807a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:09.378 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.695+0000 7f91e6ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91e807be30 0x7f91e807a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38414/0 (socket says 172.21.15.133:38414) 2023-07-14T18:07:09.378 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.695+0000 7f91e6ffd700 1 -- 172.21.15.133:0/2494921297 learned_addr learned my addr 172.21.15.133:0/2494921297 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:09.378 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.695+0000 7f91e6ffd700 1 -- 172.21.15.133:0/2494921297 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e807a9b0 con 0x7f91e807be30 2023-07-14T18:07:09.378 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.696+0000 7f91e6ffd700 1 --2- 172.21.15.133:0/2494921297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91e807be30 0x7f91e807a330 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f91d80098a0 tx=0x7f91d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b577d460c6762b5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:09.378 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.696+0000 7f91e5ffb700 1 -- 172.21.15.133:0/2494921297 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91d8004160 con 0x7f91e807be30 2023-07-14T18:07:09.378 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.696+0000 7f91e5ffb700 1 -- 172.21.15.133:0/2494921297 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f91d80042c0 con 0x7f91e807be30 2023-07-14T18:07:09.378 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.696+0000 7f91e5ffb700 1 -- 172.21.15.133:0/2494921297 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91d8011a00 con 0x7f91e807be30 2023-07-14T18:07:09.379 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.697+0000 7f91ed89b700 1 -- 172.21.15.133:0/2494921297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91e807be30 msgr2=0x7f91e807a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:09.379 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.697+0000 7f91ed89b700 1 --2- 172.21.15.133:0/2494921297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91e807be30 0x7f91e807a330 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f91d80098a0 tx=0x7f91d8009bb0 comp rx=0 tx=0).stop 2023-07-14T18:07:09.379 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.697+0000 7f91ed89b700 1 -- 172.21.15.133:0/2494921297 shutdown_connections 2023-07-14T18:07:09.379 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.697+0000 7f91ed89b700 1 --2- 172.21.15.133:0/2494921297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91e807be30 0x7f91e807a330 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:09.379 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.697+0000 7f91ed89b700 1 -- 172.21.15.133:0/2494921297 >> 172.21.15.133:0/2494921297 conn(0x7f91e8100ab0 msgr2=0x7f91e8102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:09.379 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.697+0000 7f91ed89b700 1 -- 172.21.15.133:0/2494921297 shutdown_connections 2023-07-14T18:07:09.379 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.697+0000 7f91ed89b700 1 -- 172.21.15.133:0/2494921297 wait complete. 2023-07-14T18:07:09.379 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.698+0000 7f91ed89b700 1 Processor -- start 2023-07-14T18:07:09.380 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.698+0000 7f91ed89b700 1 -- start start 2023-07-14T18:07:09.380 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.698+0000 7f91ed89b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91e807be30 0x7f91e819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:09.380 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.698+0000 7f91ed89b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e819b8b0 con 0x7f91e807be30 2023-07-14T18:07:09.380 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.698+0000 7f91e6ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91e807be30 0x7f91e819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:09.380 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.698+0000 7f91e6ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91e807be30 0x7f91e819b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38426/0 (socket says 172.21.15.133:38426) 2023-07-14T18:07:09.380 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.698+0000 7f91e6ffd700 1 -- 172.21.15.133:0/1646846652 learned_addr learned my addr 172.21.15.133:0/1646846652 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:09.381 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.699+0000 7f91e6ffd700 1 -- 172.21.15.133:0/1646846652 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d8009550 con 0x7f91e807be30 2023-07-14T18:07:09.381 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.699+0000 7f91e6ffd700 1 --2- 172.21.15.133:0/1646846652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91e807be30 0x7f91e819b370 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f91d8006af0 tx=0x7f91d8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:09.381 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.699+0000 7f91ec899700 1 -- 172.21.15.133:0/1646846652 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91d8003a40 con 0x7f91e807be30 2023-07-14T18:07:09.381 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.699+0000 7f91ec899700 1 -- 172.21.15.133:0/1646846652 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f91d8003ba0 con 0x7f91e807be30 2023-07-14T18:07:09.381 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.699+0000 7f91ec899700 1 -- 172.21.15.133:0/1646846652 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91d801a700 con 0x7f91e807be30 2023-07-14T18:07:09.381 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.699+0000 7f91ed89b700 1 -- 172.21.15.133:0/1646846652 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e819bab0 con 0x7f91e807be30 2023-07-14T18:07:09.381 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.699+0000 7f91ed89b700 1 -- 172.21.15.133:0/1646846652 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e819bed0 con 0x7f91e807be30 2023-07-14T18:07:09.381 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.700+0000 7f91ec899700 1 -- 172.21.15.133:0/1646846652 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f91d8024ad0 con 0x7f91e807be30 2023-07-14T18:07:09.382 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.701+0000 7f91ce7fc700 1 -- 172.21.15.133:0/1646846652 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e8061c70 con 0x7f91e807be30 2023-07-14T18:07:09.382 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.701+0000 7f91ec899700 1 --2- 172.21.15.133:0/1646846652 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7f91d0038710 0x7f91d003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:09.382 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.701+0000 7f91ec899700 1 -- 172.21.15.133:0/1646846652 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f91d804c640 con 0x7f91e807be30 2023-07-14T18:07:09.382 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.701+0000 7f91e67fc700 1 --2- 172.21.15.133:0/1646846652 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7f91d0038710 0x7f91d003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:09.382 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.702+0000 7f91e67fc700 1 --2- 172.21.15.133:0/1646846652 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7f91d0038710 0x7f91d003abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f91dc006fd0 tx=0x7f91dc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:09.382 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.706+0000 7f91ec899700 1 -- 172.21.15.133:0/1646846652 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f91d8024430 con 0x7f91e807be30 2023-07-14T18:07:09.382 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.900+0000 7f91ce7fc700 1 -- 172.21.15.133:0/1646846652 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f91e8061480 con 0x7f91e807be30 2023-07-14T18:07:09.382 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.901+0000 7f91ec899700 1 -- 172.21.15.133:0/1646846652 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f91d8018350 con 0x7f91e807be30 2023-07-14T18:07:09.383 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.903+0000 7f91ce7fc700 1 -- 172.21.15.133:0/1646846652 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7f91d0038710 msgr2=0x7f91d003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:09.383 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.903+0000 7f91ce7fc700 1 --2- 172.21.15.133:0/1646846652 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7f91d0038710 0x7f91d003abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f91dc006fd0 tx=0x7f91dc006e40 comp rx=0 tx=0).stop 2023-07-14T18:07:09.383 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.903+0000 7f91ce7fc700 1 -- 172.21.15.133:0/1646846652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91e807be30 msgr2=0x7f91e819b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:09.383 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.903+0000 7f91ce7fc700 1 --2- 172.21.15.133:0/1646846652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91e807be30 0x7f91e819b370 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f91d8006af0 tx=0x7f91d8011ad0 comp rx=0 tx=0).stop 2023-07-14T18:07:09.383 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.903+0000 7f91ce7fc700 1 -- 172.21.15.133:0/1646846652 shutdown_connections 2023-07-14T18:07:09.383 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.903+0000 7f91ce7fc700 1 --2- 172.21.15.133:0/1646846652 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7f91d0038710 0x7f91d003abc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:09.383 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.903+0000 7f91ce7fc700 1 --2- 172.21.15.133:0/1646846652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91e807be30 0x7f91e819b370 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:09.384 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.903+0000 7f91ce7fc700 1 -- 172.21.15.133:0/1646846652 >> 172.21.15.133:0/1646846652 conn(0x7f91e8100ab0 msgr2=0x7f91e8101880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:09.384 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.903+0000 7f91ce7fc700 1 -- 172.21.15.133:0/1646846652 shutdown_connections 2023-07-14T18:07:09.384 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:08.903+0000 7f91ce7fc700 1 -- 172.21.15.133:0/1646846652 wait complete. 2023-07-14T18:07:09.384 INFO:teuthology.orchestra.run.smithi133.stdout:Enabling cephadm module... 2023-07-14T18:07:11.057 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:10 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/281896936' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-07-14T18:07:11.058 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:10 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: ignoring --setuser ceph since I am not root 2023-07-14T18:07:11.058 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:10 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: ignoring --setgroup ceph since I am not root 2023-07-14T18:07:11.251 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.844+0000 7fc88bfa8700 1 Processor -- start 2023-07-14T18:07:11.251 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.844+0000 7fc88bfa8700 1 -- start start 2023-07-14T18:07:11.252 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.845+0000 7fc88bfa8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc884105440 0x7fc884105850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:11.252 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.845+0000 7fc88bfa8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc884105d90 con 0x7fc884105440 2023-07-14T18:07:11.252 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.845+0000 7fc889d44700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc884105440 0x7fc884105850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:11.252 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.845+0000 7fc889d44700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc884105440 0x7fc884105850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38438/0 (socket says 172.21.15.133:38438) 2023-07-14T18:07:11.252 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.845+0000 7fc889d44700 1 -- 172.21.15.133:0/2071648346 learned_addr learned my addr 172.21.15.133:0/2071648346 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:11.252 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.845+0000 7fc889d44700 1 -- 172.21.15.133:0/2071648346 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc884105ed0 con 0x7fc884105440 2023-07-14T18:07:11.252 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.846+0000 7fc889d44700 1 --2- 172.21.15.133:0/2071648346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc884105440 0x7fc884105850 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc8800098a0 tx=0x7fc880009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74da090ad9a8fe49 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:11.253 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.846+0000 7fc888d42700 1 -- 172.21.15.133:0/2071648346 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc880004160 con 0x7fc884105440 2023-07-14T18:07:11.253 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.846+0000 7fc888d42700 1 -- 172.21.15.133:0/2071648346 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc8800042c0 con 0x7fc884105440 2023-07-14T18:07:11.253 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.846+0000 7fc888d42700 1 -- 172.21.15.133:0/2071648346 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc880011a00 con 0x7fc884105440 2023-07-14T18:07:11.253 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.846+0000 7fc88bfa8700 1 -- 172.21.15.133:0/2071648346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc884105440 msgr2=0x7fc884105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:11.253 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.846+0000 7fc88bfa8700 1 --2- 172.21.15.133:0/2071648346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc884105440 0x7fc884105850 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc8800098a0 tx=0x7fc880009bb0 comp rx=0 tx=0).stop 2023-07-14T18:07:11.253 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.846+0000 7fc88bfa8700 1 -- 172.21.15.133:0/2071648346 shutdown_connections 2023-07-14T18:07:11.253 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.846+0000 7fc88bfa8700 1 --2- 172.21.15.133:0/2071648346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc884105440 0x7fc884105850 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:11.254 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.846+0000 7fc88bfa8700 1 -- 172.21.15.133:0/2071648346 >> 172.21.15.133:0/2071648346 conn(0x7fc884100ab0 msgr2=0x7fc884102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:11.254 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.846+0000 7fc88bfa8700 1 -- 172.21.15.133:0/2071648346 shutdown_connections 2023-07-14T18:07:11.254 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.847+0000 7fc88bfa8700 1 -- 172.21.15.133:0/2071648346 wait complete. 2023-07-14T18:07:11.254 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.848+0000 7fc88bfa8700 1 Processor -- start 2023-07-14T18:07:11.254 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.848+0000 7fc88bfa8700 1 -- start start 2023-07-14T18:07:11.254 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.849+0000 7fc88bfa8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc884105440 0x7fc88407bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:11.254 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.849+0000 7fc88bfa8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88407c1a0 con 0x7fc884105440 2023-07-14T18:07:11.254 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.849+0000 7fc889d44700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc884105440 0x7fc88407bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:11.255 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.849+0000 7fc889d44700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc884105440 0x7fc88407bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38444/0 (socket says 172.21.15.133:38444) 2023-07-14T18:07:11.255 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.849+0000 7fc889d44700 1 -- 172.21.15.133:0/281896936 learned_addr learned my addr 172.21.15.133:0/281896936 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:11.255 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.849+0000 7fc889d44700 1 -- 172.21.15.133:0/281896936 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc880009550 con 0x7fc884105440 2023-07-14T18:07:11.255 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.849+0000 7fc889d44700 1 --2- 172.21.15.133:0/281896936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc884105440 0x7fc88407bc60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc880006af0 tx=0x7fc880011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:11.255 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.850+0000 7fc87affd700 1 -- 172.21.15.133:0/281896936 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc880003a40 con 0x7fc884105440 2023-07-14T18:07:11.255 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.850+0000 7fc87affd700 1 -- 172.21.15.133:0/281896936 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc880003ba0 con 0x7fc884105440 2023-07-14T18:07:11.255 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.850+0000 7fc88bfa8700 1 -- 172.21.15.133:0/281896936 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc88407a2b0 con 0x7fc884105440 2023-07-14T18:07:11.256 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.850+0000 7fc87affd700 1 -- 172.21.15.133:0/281896936 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc88001a6e0 con 0x7fc884105440 2023-07-14T18:07:11.256 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.850+0000 7fc88bfa8700 1 -- 172.21.15.133:0/281896936 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc88407a6d0 con 0x7fc884105440 2023-07-14T18:07:11.256 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.851+0000 7fc87affd700 1 -- 172.21.15.133:0/281896936 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7fc880024a60 con 0x7fc884105440 2023-07-14T18:07:11.256 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.851+0000 7fc88bfa8700 1 -- 172.21.15.133:0/281896936 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc884061c70 con 0x7fc884105440 2023-07-14T18:07:11.256 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.851+0000 7fc87affd700 1 --2- 172.21.15.133:0/281896936 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7fc870038710 0x7fc87003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:11.256 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.851+0000 7fc87affd700 1 -- 172.21.15.133:0/281896936 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc88004c690 con 0x7fc884105440 2023-07-14T18:07:11.256 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.852+0000 7fc889543700 1 --2- 172.21.15.133:0/281896936 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7fc870038710 0x7fc87003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:11.256 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.852+0000 7fc889543700 1 --2- 172.21.15.133:0/281896936 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7fc870038710 0x7fc87003abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc874006fd0 tx=0x7fc874006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:11.257 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:09.856+0000 7fc87affd700 1 -- 172.21.15.133:0/281896936 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fc880024430 con 0x7fc884105440 2023-07-14T18:07:11.257 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:10.101+0000 7fc88bfa8700 1 -- 172.21.15.133:0/281896936 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fc884107350 con 0x7fc884105440 2023-07-14T18:07:11.257 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:10.762+0000 7fc87affd700 1 -- 172.21.15.133:0/281896936 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fc88002d910 con 0x7fc884105440 2023-07-14T18:07:11.257 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:10.764+0000 7fc87affd700 1 -- 172.21.15.133:0/281896936 <== mon.0 v2:172.21.15.133:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fc88002c350 con 0x7fc884105440 2023-07-14T18:07:11.257 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:10.767+0000 7fc88bfa8700 1 -- 172.21.15.133:0/281896936 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7fc870038710 msgr2=0x7fc87003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:11.257 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:10.767+0000 7fc88bfa8700 1 --2- 172.21.15.133:0/281896936 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7fc870038710 0x7fc87003abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc874006fd0 tx=0x7fc874006e40 comp rx=0 tx=0).stop 2023-07-14T18:07:11.257 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:10.767+0000 7fc88bfa8700 1 -- 172.21.15.133:0/281896936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc884105440 msgr2=0x7fc88407bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:11.258 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:10.767+0000 7fc88bfa8700 1 --2- 172.21.15.133:0/281896936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc884105440 0x7fc88407bc60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc880006af0 tx=0x7fc880011ad0 comp rx=0 tx=0).stop 2023-07-14T18:07:11.258 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:10.767+0000 7fc88bfa8700 1 -- 172.21.15.133:0/281896936 shutdown_connections 2023-07-14T18:07:11.258 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:10.767+0000 7fc88bfa8700 1 --2- 172.21.15.133:0/281896936 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7fc870038710 0x7fc87003abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:11.258 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:10.767+0000 7fc88bfa8700 1 --2- 172.21.15.133:0/281896936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc884105440 0x7fc88407bc60 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:11.258 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:10.767+0000 7fc88bfa8700 1 -- 172.21.15.133:0/281896936 >> 172.21.15.133:0/281896936 conn(0x7fc884100ab0 msgr2=0x7fc884101810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:11.258 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:10.768+0000 7fc88bfa8700 1 -- 172.21.15.133:0/281896936 shutdown_connections 2023-07-14T18:07:11.258 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:10.768+0000 7fc88bfa8700 1 -- 172.21.15.133:0/281896936 wait complete. 2023-07-14T18:07:11.395 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:11 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:11.024+0000 7fd278877200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T18:07:11.395 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:11 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:11.128+0000 7fd278877200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T18:07:11.396 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:11 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:11.243+0000 7fd278877200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T18:07:11.794 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:11 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:11.373+0000 7fd278877200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T18:07:11.795 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:11 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:11.600+0000 7fd278877200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T18:07:12.065 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:12 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:12.000+0000 7fd278877200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T18:07:12.066 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:11 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/281896936' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-07-14T18:07:12.066 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:11 smithi133 ceph-mon[126384]: mgrmap e5: x(active, since 4s) 2023-07-14T18:07:12.336 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:12 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:12.136+0000 7fd278877200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T18:07:12.538 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2023-07-14T18:07:12.538 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-07-14T18:07:12.538 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-14T18:07:12.538 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-14T18:07:12.538 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-14T18:07:12.538 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2023-07-14T18:07:12.539 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.741+0000 7f3fe336b700 1 Processor -- start 2023-07-14T18:07:12.539 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.742+0000 7f3fe336b700 1 -- start start 2023-07-14T18:07:12.539 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.742+0000 7f3fe336b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fdc07bdf0 0x7f3fdc07a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:12.539 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.742+0000 7f3fe336b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc07a830 con 0x7f3fdc07bdf0 2023-07-14T18:07:12.539 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.742+0000 7f3fe1107700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fdc07bdf0 0x7f3fdc07a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:12.539 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.742+0000 7f3fe1107700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fdc07bdf0 0x7f3fdc07a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38460/0 (socket says 172.21.15.133:38460) 2023-07-14T18:07:12.539 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.742+0000 7f3fe1107700 1 -- 172.21.15.133:0/1266420432 learned_addr learned my addr 172.21.15.133:0/1266420432 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:12.539 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.743+0000 7f3fe1107700 1 -- 172.21.15.133:0/1266420432 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fdc07a970 con 0x7f3fdc07bdf0 2023-07-14T18:07:12.540 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.743+0000 7f3fe1107700 1 --2- 172.21.15.133:0/1266420432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fdc07bdf0 0x7f3fdc07a2f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80098a0 tx=0x7f3fd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349e848dd3a7b452 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:12.540 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.743+0000 7f3fd3fff700 1 -- 172.21.15.133:0/1266420432 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fd8004160 con 0x7f3fdc07bdf0 2023-07-14T18:07:12.540 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.743+0000 7f3fd3fff700 1 -- 172.21.15.133:0/1266420432 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3fd80042c0 con 0x7f3fdc07bdf0 2023-07-14T18:07:12.540 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.743+0000 7f3fd3fff700 1 -- 172.21.15.133:0/1266420432 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fd8011a00 con 0x7f3fdc07bdf0 2023-07-14T18:07:12.540 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.744+0000 7f3fe336b700 1 -- 172.21.15.133:0/1266420432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fdc07bdf0 msgr2=0x7f3fdc07a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:12.540 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.744+0000 7f3fe336b700 1 --2- 172.21.15.133:0/1266420432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fdc07bdf0 0x7f3fdc07a2f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80098a0 tx=0x7f3fd8009bb0 comp rx=0 tx=0).stop 2023-07-14T18:07:12.541 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.744+0000 7f3fe336b700 1 -- 172.21.15.133:0/1266420432 shutdown_connections 2023-07-14T18:07:12.541 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.744+0000 7f3fe336b700 1 --2- 172.21.15.133:0/1266420432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fdc07bdf0 0x7f3fdc07a2f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:12.541 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.744+0000 7f3fe336b700 1 -- 172.21.15.133:0/1266420432 >> 172.21.15.133:0/1266420432 conn(0x7f3fdc100ab0 msgr2=0x7f3fdc102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:12.541 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.744+0000 7f3fe336b700 1 -- 172.21.15.133:0/1266420432 shutdown_connections 2023-07-14T18:07:12.541 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.744+0000 7f3fe336b700 1 -- 172.21.15.133:0/1266420432 wait complete. 2023-07-14T18:07:12.541 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.745+0000 7f3fe336b700 1 Processor -- start 2023-07-14T18:07:12.541 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.745+0000 7f3fe336b700 1 -- start start 2023-07-14T18:07:12.541 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.745+0000 7f3fe336b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fdc07bdf0 0x7f3fdc19b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:12.542 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.745+0000 7f3fe336b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc19b8a0 con 0x7f3fdc07bdf0 2023-07-14T18:07:12.542 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.746+0000 7f3fe1107700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fdc07bdf0 0x7f3fdc19b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:12.542 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.746+0000 7f3fe1107700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fdc07bdf0 0x7f3fdc19b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38466/0 (socket says 172.21.15.133:38466) 2023-07-14T18:07:12.542 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.746+0000 7f3fe1107700 1 -- 172.21.15.133:0/2194945690 learned_addr learned my addr 172.21.15.133:0/2194945690 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:12.542 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.746+0000 7f3fe1107700 1 -- 172.21.15.133:0/2194945690 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd8009550 con 0x7f3fdc07bdf0 2023-07-14T18:07:12.542 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.746+0000 7f3fe1107700 1 --2- 172.21.15.133:0/2194945690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fdc07bdf0 0x7f3fdc19b360 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8006af0 tx=0x7f3fd8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:12.542 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.746+0000 7f3fd27fc700 1 -- 172.21.15.133:0/2194945690 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fd8003a40 con 0x7f3fdc07bdf0 2023-07-14T18:07:12.542 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.747+0000 7f3fd27fc700 1 -- 172.21.15.133:0/2194945690 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3fd8003ba0 con 0x7f3fdc07bdf0 2023-07-14T18:07:12.543 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.747+0000 7f3fe336b700 1 -- 172.21.15.133:0/2194945690 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fdc19baa0 con 0x7f3fdc07bdf0 2023-07-14T18:07:12.543 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.747+0000 7f3fd27fc700 1 -- 172.21.15.133:0/2194945690 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fd801a700 con 0x7f3fdc07bdf0 2023-07-14T18:07:12.543 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.747+0000 7f3fe336b700 1 -- 172.21.15.133:0/2194945690 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fdc19bec0 con 0x7f3fdc07bdf0 2023-07-14T18:07:12.543 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.748+0000 7f3fd27fc700 1 -- 172.21.15.133:0/2194945690 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f3fd801a930 con 0x7f3fdc07bdf0 2023-07-14T18:07:12.544 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.748+0000 7f3fe336b700 1 -- 172.21.15.133:0/2194945690 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fdc061c70 con 0x7f3fdc07bdf0 2023-07-14T18:07:12.544 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.748+0000 7f3fd27fc700 1 --2- 172.21.15.133:0/2194945690 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7f3fc8038760 0x7f3fc803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:12.544 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.748+0000 7f3fd27fc700 1 -- 172.21.15.133:0/2194945690 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3fd804c6e0 con 0x7f3fdc07bdf0 2023-07-14T18:07:12.544 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.748+0000 7f3fe0906700 1 -- 172.21.15.133:0/2194945690 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7f3fc8038760 msgr2=0x7f3fc803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/58641721 2023-07-14T18:07:12.545 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.749+0000 7f3fe0906700 1 --2- 172.21.15.133:0/2194945690 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7f3fc8038760 0x7f3fc803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T18:07:12.545 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.753+0000 7f3fd27fc700 1 -- 172.21.15.133:0/2194945690 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f3fd8018b80 con 0x7f3fdc07bdf0 2023-07-14T18:07:12.545 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.950+0000 7f3fe0906700 1 -- 172.21.15.133:0/2194945690 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7f3fc8038760 msgr2=0x7f3fc803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/58641721 2023-07-14T18:07:12.545 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:11.950+0000 7f3fe0906700 1 --2- 172.21.15.133:0/2194945690 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7f3fc8038760 0x7f3fc803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-14T18:07:12.545 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:12.008+0000 7f3fe336b700 1 -- 172.21.15.133:0/2194945690 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f3fdc19f040 con 0x7f3fdc07bdf0 2023-07-14T18:07:12.545 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:12.008+0000 7f3fd27fc700 1 -- 172.21.15.133:0/2194945690 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f3fd8018350 con 0x7f3fdc07bdf0 2023-07-14T18:07:12.545 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:12.010+0000 7f3fe336b700 1 -- 172.21.15.133:0/2194945690 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7f3fc8038760 msgr2=0x7f3fc803ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T18:07:12.546 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:12.010+0000 7f3fe336b700 1 --2- 172.21.15.133:0/2194945690 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7f3fc8038760 0x7f3fc803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:12.547 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:12.010+0000 7f3fe336b700 1 -- 172.21.15.133:0/2194945690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fdc07bdf0 msgr2=0x7f3fdc19b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:12.547 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:12.010+0000 7f3fe336b700 1 --2- 172.21.15.133:0/2194945690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fdc07bdf0 0x7f3fdc19b360 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8006af0 tx=0x7f3fd8011ad0 comp rx=0 tx=0).stop 2023-07-14T18:07:12.547 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:12.010+0000 7f3fe336b700 1 -- 172.21.15.133:0/2194945690 shutdown_connections 2023-07-14T18:07:12.547 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:12.010+0000 7f3fe336b700 1 --2- 172.21.15.133:0/2194945690 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7f3fc8038760 0x7f3fc803ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:12.547 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:12.010+0000 7f3fe336b700 1 --2- 172.21.15.133:0/2194945690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fdc07bdf0 0x7f3fdc19b360 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:12.547 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:12.010+0000 7f3fe336b700 1 -- 172.21.15.133:0/2194945690 >> 172.21.15.133:0/2194945690 conn(0x7f3fdc100ab0 msgr2=0x7f3fdc101850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:12.548 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:12.011+0000 7f3fe336b700 1 -- 172.21.15.133:0/2194945690 shutdown_connections 2023-07-14T18:07:12.548 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:12.011+0000 7f3fe336b700 1 -- 172.21.15.133:0/2194945690 wait complete. 2023-07-14T18:07:12.548 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for the mgr to restart... 2023-07-14T18:07:12.548 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for mgr epoch 5... 2023-07-14T18:07:12.645 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:12 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:12.302+0000 7fd278877200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T18:07:12.646 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:12 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:12.424+0000 7fd278877200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T18:07:13.145 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:12 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2194945690' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-14T18:07:13.146 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:13 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:13.000+0000 7fd278877200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T18:07:14.145 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:13 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:13.736+0000 7fd278877200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T18:07:14.146 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:13 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:13.942+0000 7fd278877200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T18:07:15.229 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:14 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:14.865+0000 7fd278877200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T18:07:15.229 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:14 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:14.980+0000 7fd278877200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T18:07:15.230 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:15 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:15.195+0000 7fd278877200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T18:07:15.856 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:15 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:15.546+0000 7fd278877200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T18:07:16.145 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:15 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:15.822+0000 7fd278877200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T18:07:16.145 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:16 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:16.009+0000 7fd278877200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T18:07:16.645 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:16 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:16.341+0000 7fd278877200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T18:07:17.645 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:17 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:17.244+0000 7fd278877200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T18:07:17.645 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:17 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:17.411+0000 7fd278877200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T18:07:18.095 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:17 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:17.651+0000 7fd278877200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T18:07:18.095 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:17 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:17.792+0000 7fd278877200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T18:07:18.395 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:18 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:18.061+0000 7fd278877200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T18:07:19.395 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:19 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:19.107+0000 7fd278877200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T18:07:19.396 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:19 smithi133 ceph-mon[126384]: Active manager daemon x restarted 2023-07-14T18:07:19.396 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:19 smithi133 ceph-mon[126384]: Activating manager daemon x 2023-07-14T18:07:19.396 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:19 smithi133 ceph-mon[126384]: osdmap e2: 0 total, 0 up, 0 in 2023-07-14T18:07:19.396 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:19 smithi133 ceph-mon[126384]: mgrmap e6: x(active, starting, since 0.00362234s) 2023-07-14T18:07:19.396 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:19 smithi133 ceph-mon[126384]: from='mgr.14120 172.21.15.133:0/1173332444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T18:07:19.396 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:19 smithi133 ceph-mon[126384]: from='mgr.14120 172.21.15.133:0/1173332444' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-14T18:07:19.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:19 smithi133 ceph-mon[126384]: from='mgr.14120 172.21.15.133:0/1173332444' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-14T18:07:19.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:19 smithi133 ceph-mon[126384]: from='mgr.14120 172.21.15.133:0/1173332444' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-14T18:07:19.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:19 smithi133 ceph-mon[126384]: from='mgr.14120 172.21.15.133:0/1173332444' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-14T18:07:19.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:19 smithi133 ceph-mon[126384]: Manager daemon x is now available 2023-07-14T18:07:19.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:19 smithi133 ceph-mon[126384]: from='mgr.14120 172.21.15.133:0/1173332444' entity='mgr.x' 2023-07-14T18:07:19.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:19 smithi133 ceph-mon[126384]: from='mgr.14120 172.21.15.133:0/1173332444' entity='mgr.x' 2023-07-14T18:07:19.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:19 smithi133 ceph-mon[126384]: from='mgr.14120 172.21.15.133:0/1173332444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:07:19.398 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:19 smithi133 ceph-mon[126384]: from='mgr.14120 172.21.15.133:0/1173332444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:07:20.634 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2023-07-14T18:07:20.635 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-07-14T18:07:20.635 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-14T18:07:20.635 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2023-07-14T18:07:20.635 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.074+0000 7fda653c3700 1 Processor -- start 2023-07-14T18:07:20.635 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.075+0000 7fda653c3700 1 -- start start 2023-07-14T18:07:20.635 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.075+0000 7fda653c3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda60105440 0x7fda60105850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:20.636 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.075+0000 7fda653c3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda60105d90 con 0x7fda60105440 2023-07-14T18:07:20.636 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.075+0000 7fda5effd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda60105440 0x7fda60105850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:20.636 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.075+0000 7fda5effd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda60105440 0x7fda60105850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38470/0 (socket says 172.21.15.133:38470) 2023-07-14T18:07:20.636 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.075+0000 7fda5effd700 1 -- 172.21.15.133:0/1787923357 learned_addr learned my addr 172.21.15.133:0/1787923357 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:20.640 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.076+0000 7fda5effd700 1 -- 172.21.15.133:0/1787923357 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda60105ed0 con 0x7fda60105440 2023-07-14T18:07:20.641 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.076+0000 7fda5effd700 1 --2- 172.21.15.133:0/1787923357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda60105440 0x7fda60105850 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fda500098a0 tx=0x7fda50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939949431aced94f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:20.641 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.076+0000 7fda5dffb700 1 -- 172.21.15.133:0/1787923357 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fda50004160 con 0x7fda60105440 2023-07-14T18:07:20.641 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.076+0000 7fda5dffb700 1 -- 172.21.15.133:0/1787923357 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fda500042c0 con 0x7fda60105440 2023-07-14T18:07:20.641 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.076+0000 7fda5dffb700 1 -- 172.21.15.133:0/1787923357 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fda50011a00 con 0x7fda60105440 2023-07-14T18:07:20.641 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.077+0000 7fda653c3700 1 -- 172.21.15.133:0/1787923357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda60105440 msgr2=0x7fda60105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:20.641 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.077+0000 7fda653c3700 1 --2- 172.21.15.133:0/1787923357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda60105440 0x7fda60105850 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fda500098a0 tx=0x7fda50009bb0 comp rx=0 tx=0).stop 2023-07-14T18:07:20.641 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.077+0000 7fda653c3700 1 -- 172.21.15.133:0/1787923357 shutdown_connections 2023-07-14T18:07:20.642 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.077+0000 7fda653c3700 1 --2- 172.21.15.133:0/1787923357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda60105440 0x7fda60105850 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:20.642 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.077+0000 7fda653c3700 1 -- 172.21.15.133:0/1787923357 >> 172.21.15.133:0/1787923357 conn(0x7fda60100ab0 msgr2=0x7fda60102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:20.642 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.077+0000 7fda653c3700 1 -- 172.21.15.133:0/1787923357 shutdown_connections 2023-07-14T18:07:20.642 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.077+0000 7fda653c3700 1 -- 172.21.15.133:0/1787923357 wait complete. 2023-07-14T18:07:20.642 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.078+0000 7fda653c3700 1 Processor -- start 2023-07-14T18:07:20.642 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.079+0000 7fda653c3700 1 -- start start 2023-07-14T18:07:20.642 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.079+0000 7fda653c3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda60105440 0x7fda60199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:20.642 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.079+0000 7fda653c3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda60199650 con 0x7fda60105440 2023-07-14T18:07:20.643 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.079+0000 7fda5effd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda60105440 0x7fda60199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:20.643 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.079+0000 7fda5effd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda60105440 0x7fda60199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38476/0 (socket says 172.21.15.133:38476) 2023-07-14T18:07:20.643 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.079+0000 7fda5effd700 1 -- 172.21.15.133:0/4111039351 learned_addr learned my addr 172.21.15.133:0/4111039351 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:20.643 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.079+0000 7fda5effd700 1 -- 172.21.15.133:0/4111039351 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda50009550 con 0x7fda60105440 2023-07-14T18:07:20.643 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.080+0000 7fda5effd700 1 --2- 172.21.15.133:0/4111039351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda60105440 0x7fda60199110 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fda50006af0 tx=0x7fda50011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:20.643 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.080+0000 7fda47fff700 1 -- 172.21.15.133:0/4111039351 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fda50003a40 con 0x7fda60105440 2023-07-14T18:07:20.643 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.080+0000 7fda653c3700 1 -- 172.21.15.133:0/4111039351 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda60199850 con 0x7fda60105440 2023-07-14T18:07:20.643 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.080+0000 7fda47fff700 1 -- 172.21.15.133:0/4111039351 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fda50003ba0 con 0x7fda60105440 2023-07-14T18:07:20.644 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.080+0000 7fda47fff700 1 -- 172.21.15.133:0/4111039351 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fda5001a6e0 con 0x7fda60105440 2023-07-14T18:07:20.644 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.080+0000 7fda653c3700 1 -- 172.21.15.133:0/4111039351 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda60199c70 con 0x7fda60105440 2023-07-14T18:07:20.644 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.081+0000 7fda47fff700 1 -- 172.21.15.133:0/4111039351 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fda5001a950 con 0x7fda60105440 2023-07-14T18:07:20.644 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.082+0000 7fda47fff700 1 --2- 172.21.15.133:0/4111039351 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7fda48038710 0x7fda4803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:20.644 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.082+0000 7fda47fff700 1 -- 172.21.15.133:0/4111039351 --> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fda4803b2d0 con 0x7fda48038710 2023-07-14T18:07:20.644 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.082+0000 7fda47fff700 1 -- 172.21.15.133:0/4111039351 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fda5004c920 con 0x7fda60105440 2023-07-14T18:07:20.644 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.082+0000 7fda5e7fc700 1 -- 172.21.15.133:0/4111039351 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7fda48038710 msgr2=0x7fda4803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/58641721 2023-07-14T18:07:20.645 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.082+0000 7fda5e7fc700 1 --2- 172.21.15.133:0/4111039351 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7fda48038710 0x7fda4803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T18:07:20.645 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.283+0000 7fda5e7fc700 1 -- 172.21.15.133:0/4111039351 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7fda48038710 msgr2=0x7fda4803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/58641721 2023-07-14T18:07:20.645 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.283+0000 7fda5e7fc700 1 --2- 172.21.15.133:0/4111039351 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7fda48038710 0x7fda4803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-14T18:07:20.645 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.684+0000 7fda5e7fc700 1 -- 172.21.15.133:0/4111039351 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7fda48038710 msgr2=0x7fda4803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/58641721 2023-07-14T18:07:20.645 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:13.684+0000 7fda5e7fc700 1 --2- 172.21.15.133:0/4111039351 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7fda48038710 0x7fda4803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-14T18:07:20.645 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:14.485+0000 7fda5e7fc700 1 -- 172.21.15.133:0/4111039351 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7fda48038710 msgr2=0x7fda4803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/58641721 2023-07-14T18:07:20.645 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:14.485+0000 7fda5e7fc700 1 --2- 172.21.15.133:0/4111039351 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7fda48038710 0x7fda4803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-14T18:07:20.645 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:16.086+0000 7fda5e7fc700 1 -- 172.21.15.133:0/4111039351 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7fda48038710 msgr2=0x7fda4803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/58641721 2023-07-14T18:07:20.646 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:16.086+0000 7fda5e7fc700 1 --2- 172.21.15.133:0/4111039351 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7fda48038710 0x7fda4803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-14T18:07:20.646 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:19.111+0000 7fda47fff700 1 -- 172.21.15.133:0/4111039351 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mgrmap(e 6) v1 ==== 45250+0+0 (secure 0 0 0) 0x7fda5002d960 con 0x7fda60105440 2023-07-14T18:07:20.646 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:19.111+0000 7fda47fff700 1 -- 172.21.15.133:0/4111039351 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7fda48038710 msgr2=0x7fda4803abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T18:07:20.646 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:19.111+0000 7fda47fff700 1 --2- 172.21.15.133:0/4111039351 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7fda48038710 0x7fda4803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:20.646 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:20.113+0000 7fda47fff700 1 -- 172.21.15.133:0/4111039351 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7fda5004e2b0 con 0x7fda60105440 2023-07-14T18:07:20.646 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:20.114+0000 7fda47fff700 1 --2- 172.21.15.133:0/4111039351 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7fda4803bfe0 0x7fda4803e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:20.646 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:20.114+0000 7fda47fff700 1 -- 172.21.15.133:0/4111039351 --> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fda4803b2d0 con 0x7fda4803bfe0 2023-07-14T18:07:20.647 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:20.114+0000 7fda5e7fc700 1 --2- 172.21.15.133:0/4111039351 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7fda4803bfe0 0x7fda4803e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:20.647 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:20.116+0000 7fda5e7fc700 1 --2- 172.21.15.133:0/4111039351 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7fda4803bfe0 0x7fda4803e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fda54003a10 tx=0x7fda54009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:20.647 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:20.116+0000 7fda47fff700 1 -- 172.21.15.133:0/4111039351 <== mgr.14120 v2:172.21.15.133:6800/2986586981 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fda4803b2d0 con 0x7fda4803bfe0 2023-07-14T18:07:20.647 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:20.123+0000 7fda653c3700 1 -- 172.21.15.133:0/4111039351 --> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fda6019a450 con 0x7fda4803bfe0 2023-07-14T18:07:20.647 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:20.123+0000 7fda47fff700 1 -- 172.21.15.133:0/4111039351 <== mgr.14120 v2:172.21.15.133:6800/2986586981 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fda6019a450 con 0x7fda4803bfe0 2023-07-14T18:07:20.647 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:20.124+0000 7fda653c3700 1 -- 172.21.15.133:0/4111039351 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7fda4803bfe0 msgr2=0x7fda4803e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:20.647 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:20.124+0000 7fda653c3700 1 --2- 172.21.15.133:0/4111039351 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7fda4803bfe0 0x7fda4803e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fda54003a10 tx=0x7fda54009290 comp rx=0 tx=0).stop 2023-07-14T18:07:20.647 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:20.124+0000 7fda653c3700 1 -- 172.21.15.133:0/4111039351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda60105440 msgr2=0x7fda60199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:20.648 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:20.124+0000 7fda653c3700 1 --2- 172.21.15.133:0/4111039351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda60105440 0x7fda60199110 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fda50006af0 tx=0x7fda50011ad0 comp rx=0 tx=0).stop 2023-07-14T18:07:20.648 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:20.124+0000 7fda653c3700 1 -- 172.21.15.133:0/4111039351 shutdown_connections 2023-07-14T18:07:20.648 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:20.124+0000 7fda653c3700 1 --2- 172.21.15.133:0/4111039351 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7fda4803bfe0 0x7fda4803e3c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:20.648 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:20.124+0000 7fda653c3700 1 --2- 172.21.15.133:0/4111039351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda60105440 0x7fda60199110 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:20.648 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:20.124+0000 7fda653c3700 1 --2- 172.21.15.133:0/4111039351 >> [v2:172.21.15.133:6800/58641721,v1:172.21.15.133:6801/58641721] conn(0x7fda48038710 0x7fda4803abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:20.648 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:20.124+0000 7fda653c3700 1 -- 172.21.15.133:0/4111039351 >> 172.21.15.133:0/4111039351 conn(0x7fda60100ab0 msgr2=0x7fda60101800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:20.648 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:20.124+0000 7fda653c3700 1 -- 172.21.15.133:0/4111039351 shutdown_connections 2023-07-14T18:07:20.648 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:20.124+0000 7fda653c3700 1 -- 172.21.15.133:0/4111039351 wait complete. 2023-07-14T18:07:20.649 INFO:teuthology.orchestra.run.smithi133.stdout:mgr epoch 5 is available 2023-07-14T18:07:20.649 INFO:teuthology.orchestra.run.smithi133.stdout:Setting orchestrator backend to cephadm... 2023-07-14T18:07:20.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:20 smithi133 ceph-mon[126384]: Found migration_current of "None". Setting to last migration. 2023-07-14T18:07:20.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:20 smithi133 ceph-mon[126384]: from='mgr.14120 172.21.15.133:0/1173332444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:07:20.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:20 smithi133 ceph-mon[126384]: from='mgr.14120 172.21.15.133:0/1173332444' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-14T18:07:20.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:20 smithi133 ceph-mon[126384]: from='mgr.14120 172.21.15.133:0/1173332444' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-14T18:07:20.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:20 smithi133 ceph-mon[126384]: from='mgr.14120 172.21.15.133:0/1173332444' entity='mgr.x' 2023-07-14T18:07:20.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:20 smithi133 ceph-mon[126384]: from='mgr.14120 172.21.15.133:0/1173332444' entity='mgr.x' 2023-07-14T18:07:20.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:20 smithi133 ceph-mon[126384]: mgrmap e7: x(active, since 1.00766s) 2023-07-14T18:07:21.456 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:21 smithi133 ceph-mon[126384]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-07-14T18:07:21.457 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:21 smithi133 ceph-mon[126384]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-07-14T18:07:21.457 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:21 smithi133 ceph-mon[126384]: [14/Jul/2023:18:07:20] ENGINE Bus STARTING 2023-07-14T18:07:21.457 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:21 smithi133 ceph-mon[126384]: [14/Jul/2023:18:07:20] ENGINE Serving on http://172.21.15.133:8765 2023-07-14T18:07:21.457 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:21 smithi133 ceph-mon[126384]: [14/Jul/2023:18:07:20] ENGINE Serving on https://172.21.15.133:7150 2023-07-14T18:07:21.457 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:21 smithi133 ceph-mon[126384]: [14/Jul/2023:18:07:20] ENGINE Bus STARTED 2023-07-14T18:07:21.457 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:21 smithi133 ceph-mon[126384]: from='mgr.14120 172.21.15.133:0/1173332444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:07:21.818 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.171+0000 7f09db9d1700 1 Processor -- start 2023-07-14T18:07:21.823 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.171+0000 7f09db9d1700 1 -- start start 2023-07-14T18:07:21.823 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.171+0000 7f09db9d1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d4079cb0 0x7f09d4108030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:21.824 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.171+0000 7f09db9d1700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d4108570 con 0x7f09d4079cb0 2023-07-14T18:07:21.824 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.171+0000 7f09d976d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d4079cb0 0x7f09d4108030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:21.824 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.171+0000 7f09d976d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d4079cb0 0x7f09d4108030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57850/0 (socket says 172.21.15.133:57850) 2023-07-14T18:07:21.825 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.171+0000 7f09d976d700 1 -- 172.21.15.133:0/2418936364 learned_addr learned my addr 172.21.15.133:0/2418936364 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:21.825 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.172+0000 7f09d976d700 1 -- 172.21.15.133:0/2418936364 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d41086b0 con 0x7f09d4079cb0 2023-07-14T18:07:21.825 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.172+0000 7f09d976d700 1 --2- 172.21.15.133:0/2418936364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d4079cb0 0x7f09d4108030 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f09d00098a0 tx=0x7f09d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50650fbcf47c7d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:21.825 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.172+0000 7f09cbfff700 1 -- 172.21.15.133:0/2418936364 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09d0004160 con 0x7f09d4079cb0 2023-07-14T18:07:21.826 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.172+0000 7f09cbfff700 1 -- 172.21.15.133:0/2418936364 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f09d00042c0 con 0x7f09d4079cb0 2023-07-14T18:07:21.826 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.172+0000 7f09cbfff700 1 -- 172.21.15.133:0/2418936364 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09d0011a00 con 0x7f09d4079cb0 2023-07-14T18:07:21.826 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.173+0000 7f09db9d1700 1 -- 172.21.15.133:0/2418936364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d4079cb0 msgr2=0x7f09d4108030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:21.826 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.173+0000 7f09db9d1700 1 --2- 172.21.15.133:0/2418936364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d4079cb0 0x7f09d4108030 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f09d00098a0 tx=0x7f09d0009bb0 comp rx=0 tx=0).stop 2023-07-14T18:07:21.827 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.173+0000 7f09db9d1700 1 -- 172.21.15.133:0/2418936364 shutdown_connections 2023-07-14T18:07:21.827 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.173+0000 7f09db9d1700 1 --2- 172.21.15.133:0/2418936364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d4079cb0 0x7f09d4108030 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:21.827 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.173+0000 7f09db9d1700 1 -- 172.21.15.133:0/2418936364 >> 172.21.15.133:0/2418936364 conn(0x7f09d40ff980 msgr2=0x7f09d4101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:21.828 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.173+0000 7f09db9d1700 1 -- 172.21.15.133:0/2418936364 shutdown_connections 2023-07-14T18:07:21.828 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.173+0000 7f09db9d1700 1 -- 172.21.15.133:0/2418936364 wait complete. 2023-07-14T18:07:21.828 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.174+0000 7f09db9d1700 1 Processor -- start 2023-07-14T18:07:21.828 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.174+0000 7f09db9d1700 1 -- start start 2023-07-14T18:07:21.828 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.175+0000 7f09db9d1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d4079cb0 0x7f09d4199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:21.829 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.175+0000 7f09db9d1700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d41996a0 con 0x7f09d4079cb0 2023-07-14T18:07:21.829 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.175+0000 7f09d976d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d4079cb0 0x7f09d4199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:21.829 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.175+0000 7f09d976d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d4079cb0 0x7f09d4199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57866/0 (socket says 172.21.15.133:57866) 2023-07-14T18:07:21.829 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.175+0000 7f09d976d700 1 -- 172.21.15.133:0/2958896673 learned_addr learned my addr 172.21.15.133:0/2958896673 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:21.829 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.175+0000 7f09d976d700 1 -- 172.21.15.133:0/2958896673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d0009550 con 0x7f09d4079cb0 2023-07-14T18:07:21.829 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.175+0000 7f09d976d700 1 --2- 172.21.15.133:0/2958896673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d4079cb0 0x7f09d4199160 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f09d0006af0 tx=0x7f09d0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:21.829 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.176+0000 7f09ca7fc700 1 -- 172.21.15.133:0/2958896673 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09d0003a40 con 0x7f09d4079cb0 2023-07-14T18:07:21.830 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.176+0000 7f09ca7fc700 1 -- 172.21.15.133:0/2958896673 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f09d0003ba0 con 0x7f09d4079cb0 2023-07-14T18:07:21.830 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.176+0000 7f09db9d1700 1 -- 172.21.15.133:0/2958896673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d41998a0 con 0x7f09d4079cb0 2023-07-14T18:07:21.830 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.176+0000 7f09ca7fc700 1 -- 172.21.15.133:0/2958896673 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09d001a700 con 0x7f09d4079cb0 2023-07-14T18:07:21.830 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.176+0000 7f09db9d1700 1 -- 172.21.15.133:0/2958896673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d4199cc0 con 0x7f09d4079cb0 2023-07-14T18:07:21.830 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.177+0000 7f09ca7fc700 1 -- 172.21.15.133:0/2958896673 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f09d0024ad0 con 0x7f09d4079cb0 2023-07-14T18:07:21.830 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.177+0000 7f09db9d1700 1 -- 172.21.15.133:0/2958896673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d4061c70 con 0x7f09d4079cb0 2023-07-14T18:07:21.830 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.177+0000 7f09ca7fc700 1 --2- 172.21.15.133:0/2958896673 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f09c0038760 0x7f09c003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:21.830 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.177+0000 7f09ca7fc700 1 -- 172.21.15.133:0/2958896673 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f09d004c770 con 0x7f09d4079cb0 2023-07-14T18:07:21.831 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.178+0000 7f09d8f6c700 1 --2- 172.21.15.133:0/2958896673 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f09c0038760 0x7f09c003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:21.831 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.178+0000 7f09d8f6c700 1 --2- 172.21.15.133:0/2958896673 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f09c0038760 0x7f09c003ac10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f09c4006fd0 tx=0x7f09c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:21.831 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.183+0000 7f09ca7fc700 1 -- 172.21.15.133:0/2958896673 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f09d0024430 con 0x7f09d4079cb0 2023-07-14T18:07:21.831 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.392+0000 7f09db9d1700 1 -- 172.21.15.133:0/2958896673 --> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f09d4101c40 con 0x7f09c0038760 2023-07-14T18:07:21.831 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.399+0000 7f09ca7fc700 1 -- 172.21.15.133:0/2958896673 <== mgr.14120 v2:172.21.15.133:6800/2986586981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f09d4101c40 con 0x7f09c0038760 2023-07-14T18:07:21.831 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.401+0000 7f09db9d1700 1 -- 172.21.15.133:0/2958896673 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f09c0038760 msgr2=0x7f09c003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:21.831 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.401+0000 7f09db9d1700 1 --2- 172.21.15.133:0/2958896673 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f09c0038760 0x7f09c003ac10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f09c4006fd0 tx=0x7f09c4006e40 comp rx=0 tx=0).stop 2023-07-14T18:07:21.832 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.401+0000 7f09db9d1700 1 -- 172.21.15.133:0/2958896673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d4079cb0 msgr2=0x7f09d4199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:21.832 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.401+0000 7f09db9d1700 1 --2- 172.21.15.133:0/2958896673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d4079cb0 0x7f09d4199160 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f09d0006af0 tx=0x7f09d0011ad0 comp rx=0 tx=0).stop 2023-07-14T18:07:21.832 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.401+0000 7f09db9d1700 1 -- 172.21.15.133:0/2958896673 shutdown_connections 2023-07-14T18:07:21.832 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.401+0000 7f09db9d1700 1 --2- 172.21.15.133:0/2958896673 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f09c0038760 0x7f09c003ac10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:21.832 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.401+0000 7f09db9d1700 1 --2- 172.21.15.133:0/2958896673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d4079cb0 0x7f09d4199160 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:21.832 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.401+0000 7f09db9d1700 1 -- 172.21.15.133:0/2958896673 >> 172.21.15.133:0/2958896673 conn(0x7f09d40ff980 msgr2=0x7f09d4101020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:21.832 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.401+0000 7f09db9d1700 1 -- 172.21.15.133:0/2958896673 shutdown_connections 2023-07-14T18:07:21.833 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:21.402+0000 7f09db9d1700 1 -- 172.21.15.133:0/2958896673 wait complete. 2023-07-14T18:07:22.395 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:22 smithi133 ceph-mon[126384]: mgrmap e8: x(active, since 2s) 2023-07-14T18:07:22.396 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:22 smithi133 ceph-mon[126384]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:07:22.396 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:22 smithi133 ceph-mon[126384]: from='mgr.14120 172.21.15.133:0/1173332444' entity='mgr.x' 2023-07-14T18:07:22.396 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:22 smithi133 ceph-mon[126384]: from='mgr.14120 172.21.15.133:0/1173332444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:07:23.003 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout value unchanged 2023-07-14T18:07:23.004 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.318+0000 7f0eedb2b700 1 Processor -- start 2023-07-14T18:07:23.004 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.319+0000 7f0eedb2b700 1 -- start start 2023-07-14T18:07:23.004 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.319+0000 7f0eedb2b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ee8105440 0x7f0ee8105850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:23.004 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.319+0000 7f0eedb2b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee8105d90 con 0x7f0ee8105440 2023-07-14T18:07:23.004 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.319+0000 7f0ee77fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ee8105440 0x7f0ee8105850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:23.004 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.319+0000 7f0ee77fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ee8105440 0x7f0ee8105850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57876/0 (socket says 172.21.15.133:57876) 2023-07-14T18:07:23.005 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.319+0000 7f0ee77fe700 1 -- 172.21.15.133:0/1771535118 learned_addr learned my addr 172.21.15.133:0/1771535118 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:23.005 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.320+0000 7f0ee77fe700 1 -- 172.21.15.133:0/1771535118 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee8105ed0 con 0x7f0ee8105440 2023-07-14T18:07:23.005 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.320+0000 7f0ee77fe700 1 --2- 172.21.15.133:0/1771535118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ee8105440 0x7f0ee8105850 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80098a0 tx=0x7f0ed8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeea3b656c1de30b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:23.005 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.320+0000 7f0ee67fc700 1 -- 172.21.15.133:0/1771535118 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ed8004160 con 0x7f0ee8105440 2023-07-14T18:07:23.005 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.320+0000 7f0ee67fc700 1 -- 172.21.15.133:0/1771535118 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0ed80042c0 con 0x7f0ee8105440 2023-07-14T18:07:23.005 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.320+0000 7f0ee67fc700 1 -- 172.21.15.133:0/1771535118 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ed8011a00 con 0x7f0ee8105440 2023-07-14T18:07:23.006 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.321+0000 7f0eedb2b700 1 -- 172.21.15.133:0/1771535118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ee8105440 msgr2=0x7f0ee8105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:23.006 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.321+0000 7f0eedb2b700 1 --2- 172.21.15.133:0/1771535118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ee8105440 0x7f0ee8105850 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80098a0 tx=0x7f0ed8009bb0 comp rx=0 tx=0).stop 2023-07-14T18:07:23.006 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.321+0000 7f0eedb2b700 1 -- 172.21.15.133:0/1771535118 shutdown_connections 2023-07-14T18:07:23.006 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.321+0000 7f0eedb2b700 1 --2- 172.21.15.133:0/1771535118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ee8105440 0x7f0ee8105850 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:23.006 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.321+0000 7f0eedb2b700 1 -- 172.21.15.133:0/1771535118 >> 172.21.15.133:0/1771535118 conn(0x7f0ee8100ab0 msgr2=0x7f0ee8102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:23.006 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.321+0000 7f0eedb2b700 1 -- 172.21.15.133:0/1771535118 shutdown_connections 2023-07-14T18:07:23.006 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.321+0000 7f0eedb2b700 1 -- 172.21.15.133:0/1771535118 wait complete. 2023-07-14T18:07:23.007 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.322+0000 7f0eedb2b700 1 Processor -- start 2023-07-14T18:07:23.007 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.322+0000 7f0eedb2b700 1 -- start start 2023-07-14T18:07:23.007 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.322+0000 7f0eedb2b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ee8105440 0x7f0ee807bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:23.007 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.322+0000 7f0eedb2b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee807c1a0 con 0x7f0ee8105440 2023-07-14T18:07:23.007 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.323+0000 7f0ee77fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ee8105440 0x7f0ee807bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:23.007 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.323+0000 7f0ee77fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ee8105440 0x7f0ee807bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57884/0 (socket says 172.21.15.133:57884) 2023-07-14T18:07:23.007 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.323+0000 7f0ee77fe700 1 -- 172.21.15.133:0/142014287 learned_addr learned my addr 172.21.15.133:0/142014287 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:23.007 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.323+0000 7f0ee77fe700 1 -- 172.21.15.133:0/142014287 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed8009550 con 0x7f0ee8105440 2023-07-14T18:07:23.008 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.323+0000 7f0ee77fe700 1 --2- 172.21.15.133:0/142014287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ee8105440 0x7f0ee807bc60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8006af0 tx=0x7f0ed8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:23.008 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.323+0000 7f0ee4ff9700 1 -- 172.21.15.133:0/142014287 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ed8003a40 con 0x7f0ee8105440 2023-07-14T18:07:23.008 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.323+0000 7f0ee4ff9700 1 -- 172.21.15.133:0/142014287 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0ed8003ba0 con 0x7f0ee8105440 2023-07-14T18:07:23.008 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.323+0000 7f0ee4ff9700 1 -- 172.21.15.133:0/142014287 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ed801a6e0 con 0x7f0ee8105440 2023-07-14T18:07:23.008 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.323+0000 7f0eedb2b700 1 -- 172.21.15.133:0/142014287 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee807a2b0 con 0x7f0ee8105440 2023-07-14T18:07:23.008 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.324+0000 7f0eedb2b700 1 -- 172.21.15.133:0/142014287 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee807a6d0 con 0x7f0ee8105440 2023-07-14T18:07:23.008 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.324+0000 7f0ee4ff9700 1 -- 172.21.15.133:0/142014287 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f0ed8024a60 con 0x7f0ee8105440 2023-07-14T18:07:23.008 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.324+0000 7f0eedb2b700 1 -- 172.21.15.133:0/142014287 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ee8061c70 con 0x7f0ee8105440 2023-07-14T18:07:23.009 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.325+0000 7f0ee4ff9700 1 --2- 172.21.15.133:0/142014287 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f0ed0038760 0x7f0ed003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:23.009 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.325+0000 7f0ee4ff9700 1 -- 172.21.15.133:0/142014287 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0ed804c7e0 con 0x7f0ee8105440 2023-07-14T18:07:23.009 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.325+0000 7f0ee6ffd700 1 --2- 172.21.15.133:0/142014287 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f0ed0038760 0x7f0ed003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:23.009 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.326+0000 7f0ee6ffd700 1 --2- 172.21.15.133:0/142014287 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f0ed0038760 0x7f0ed003ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0edc006fd0 tx=0x7f0edc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:23.009 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.330+0000 7f0ee4ff9700 1 -- 172.21.15.133:0/142014287 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f0ed8024430 con 0x7f0ee8105440 2023-07-14T18:07:23.009 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.523+0000 7f0eedb2b700 1 -- 172.21.15.133:0/142014287 --> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ee8101f10 con 0x7f0ed0038760 2023-07-14T18:07:23.009 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.524+0000 7f0ee4ff9700 1 -- 172.21.15.133:0/142014287 <== mgr.14120 v2:172.21.15.133:6800/2986586981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f0ee8101f10 con 0x7f0ed0038760 2023-07-14T18:07:23.010 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.526+0000 7f0eedb2b700 1 -- 172.21.15.133:0/142014287 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f0ed0038760 msgr2=0x7f0ed003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:23.010 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.526+0000 7f0eedb2b700 1 --2- 172.21.15.133:0/142014287 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f0ed0038760 0x7f0ed003ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0edc006fd0 tx=0x7f0edc006e40 comp rx=0 tx=0).stop 2023-07-14T18:07:23.010 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.526+0000 7f0eedb2b700 1 -- 172.21.15.133:0/142014287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ee8105440 msgr2=0x7f0ee807bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:23.010 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.526+0000 7f0eedb2b700 1 --2- 172.21.15.133:0/142014287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ee8105440 0x7f0ee807bc60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8006af0 tx=0x7f0ed8011ad0 comp rx=0 tx=0).stop 2023-07-14T18:07:23.010 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.526+0000 7f0eedb2b700 1 -- 172.21.15.133:0/142014287 shutdown_connections 2023-07-14T18:07:23.010 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.526+0000 7f0eedb2b700 1 --2- 172.21.15.133:0/142014287 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f0ed0038760 0x7f0ed003ac10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:23.010 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.526+0000 7f0eedb2b700 1 --2- 172.21.15.133:0/142014287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ee8105440 0x7f0ee807bc60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:23.010 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.526+0000 7f0eedb2b700 1 -- 172.21.15.133:0/142014287 >> 172.21.15.133:0/142014287 conn(0x7f0ee8100ab0 msgr2=0x7f0ee8101800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:23.011 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.526+0000 7f0eedb2b700 1 -- 172.21.15.133:0/142014287 shutdown_connections 2023-07-14T18:07:23.011 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:22.526+0000 7f0eedb2b700 1 -- 172.21.15.133:0/142014287 wait complete. 2023-07-14T18:07:23.011 INFO:teuthology.orchestra.run.smithi133.stdout:Generating ssh key... 2023-07-14T18:07:23.256 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:23 smithi133 ceph-mon[126384]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:07:24.146 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:23 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: Generating public/private rsa key pair. 2023-07-14T18:07:24.146 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:23 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: Your identification has been saved in /tmp/tmpg66c5lyt/key. 2023-07-14T18:07:24.146 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:23 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: Your public key has been saved in /tmp/tmpg66c5lyt/key.pub. 2023-07-14T18:07:24.146 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:23 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: The key fingerprint is: 2023-07-14T18:07:24.147 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:23 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: SHA256:4yEhFThsFV+uQ8Olne6kxMM0+AYTAOgekyED7wizrSs ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c 2023-07-14T18:07:24.147 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:23 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: The key's randomart image is: 2023-07-14T18:07:24.147 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:23 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: +---[RSA 3072]----+ 2023-07-14T18:07:24.147 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:23 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: |o..o.+*o o | 2023-07-14T18:07:24.147 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:23 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: |+o =. = * . | 2023-07-14T18:07:24.147 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:23 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: |=.+...= O + | 2023-07-14T18:07:24.147 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:23 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: |.@ . X = | 2023-07-14T18:07:24.148 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:23 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: |+ = . S o | 2023-07-14T18:07:24.148 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:23 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: | o = O | 2023-07-14T18:07:24.148 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:23 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: |. o . | 2023-07-14T18:07:24.148 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:23 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: |E. | 2023-07-14T18:07:24.148 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:23 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: |o | 2023-07-14T18:07:24.148 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:23 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: +----[SHA256]-----+ 2023-07-14T18:07:24.282 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.483+0000 7f34998f3700 1 Processor -- start 2023-07-14T18:07:24.282 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.483+0000 7f34998f3700 1 -- start start 2023-07-14T18:07:24.282 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.484+0000 7f34998f3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f349407bdf0 0x7f349407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:24.282 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.484+0000 7f34998f3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349407a830 con 0x7f349407bdf0 2023-07-14T18:07:24.283 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.484+0000 7f3492ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f349407bdf0 0x7f349407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:24.283 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.484+0000 7f3492ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f349407bdf0 0x7f349407a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57898/0 (socket says 172.21.15.133:57898) 2023-07-14T18:07:24.283 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.484+0000 7f3492ffd700 1 -- 172.21.15.133:0/351066316 learned_addr learned my addr 172.21.15.133:0/351066316 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:24.283 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.484+0000 7f3492ffd700 1 -- 172.21.15.133:0/351066316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f349407a970 con 0x7f349407bdf0 2023-07-14T18:07:24.284 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.485+0000 7f3492ffd700 1 --2- 172.21.15.133:0/351066316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f349407bdf0 0x7f349407a2f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f34840098a0 tx=0x7f3484009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877840fe320719ec server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:24.284 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.485+0000 7f3491ffb700 1 -- 172.21.15.133:0/351066316 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3484004160 con 0x7f349407bdf0 2023-07-14T18:07:24.284 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.485+0000 7f3491ffb700 1 -- 172.21.15.133:0/351066316 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f34840042c0 con 0x7f349407bdf0 2023-07-14T18:07:24.284 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.485+0000 7f3491ffb700 1 -- 172.21.15.133:0/351066316 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3484011a00 con 0x7f349407bdf0 2023-07-14T18:07:24.285 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.485+0000 7f34998f3700 1 -- 172.21.15.133:0/351066316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f349407bdf0 msgr2=0x7f349407a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:24.285 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.485+0000 7f34998f3700 1 --2- 172.21.15.133:0/351066316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f349407bdf0 0x7f349407a2f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f34840098a0 tx=0x7f3484009bb0 comp rx=0 tx=0).stop 2023-07-14T18:07:24.285 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.485+0000 7f34998f3700 1 -- 172.21.15.133:0/351066316 shutdown_connections 2023-07-14T18:07:24.286 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.485+0000 7f34998f3700 1 --2- 172.21.15.133:0/351066316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f349407bdf0 0x7f349407a2f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:24.286 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.485+0000 7f34998f3700 1 -- 172.21.15.133:0/351066316 >> 172.21.15.133:0/351066316 conn(0x7f3494100ab0 msgr2=0x7f3494102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:24.286 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.486+0000 7f34998f3700 1 -- 172.21.15.133:0/351066316 shutdown_connections 2023-07-14T18:07:24.287 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.486+0000 7f34998f3700 1 -- 172.21.15.133:0/351066316 wait complete. 2023-07-14T18:07:24.287 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.487+0000 7f34998f3700 1 Processor -- start 2023-07-14T18:07:24.287 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.487+0000 7f34998f3700 1 -- start start 2023-07-14T18:07:24.287 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.488+0000 7f34998f3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f349407bdf0 0x7f349419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:24.288 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.488+0000 7f34998f3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349419b8a0 con 0x7f349407bdf0 2023-07-14T18:07:24.288 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.488+0000 7f3492ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f349407bdf0 0x7f349419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:24.288 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.488+0000 7f3492ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f349407bdf0 0x7f349419b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57910/0 (socket says 172.21.15.133:57910) 2023-07-14T18:07:24.289 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.488+0000 7f3492ffd700 1 -- 172.21.15.133:0/2651130498 learned_addr learned my addr 172.21.15.133:0/2651130498 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:24.289 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.488+0000 7f3492ffd700 1 -- 172.21.15.133:0/2651130498 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3484009550 con 0x7f349407bdf0 2023-07-14T18:07:24.289 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.488+0000 7f3492ffd700 1 --2- 172.21.15.133:0/2651130498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f349407bdf0 0x7f349419b360 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3484006af0 tx=0x7f3484011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:24.290 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.489+0000 7f34988f1700 1 -- 172.21.15.133:0/2651130498 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3484003a40 con 0x7f349407bdf0 2023-07-14T18:07:24.290 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.489+0000 7f34988f1700 1 -- 172.21.15.133:0/2651130498 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3484003ba0 con 0x7f349407bdf0 2023-07-14T18:07:24.290 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.489+0000 7f34998f3700 1 -- 172.21.15.133:0/2651130498 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f349419baa0 con 0x7f349407bdf0 2023-07-14T18:07:24.290 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.489+0000 7f34988f1700 1 -- 172.21.15.133:0/2651130498 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f348401a700 con 0x7f349407bdf0 2023-07-14T18:07:24.291 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.489+0000 7f34998f3700 1 -- 172.21.15.133:0/2651130498 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f349419bec0 con 0x7f349407bdf0 2023-07-14T18:07:24.291 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.490+0000 7f34988f1700 1 -- 172.21.15.133:0/2651130498 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f3484024430 con 0x7f349407bdf0 2023-07-14T18:07:24.291 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.490+0000 7f34998f3700 1 -- 172.21.15.133:0/2651130498 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3494061c70 con 0x7f349407bdf0 2023-07-14T18:07:24.292 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.490+0000 7f34988f1700 1 --2- 172.21.15.133:0/2651130498 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f347c038710 0x7f347c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:24.292 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.490+0000 7f34988f1700 1 -- 172.21.15.133:0/2651130498 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f348404c7a0 con 0x7f349407bdf0 2023-07-14T18:07:24.292 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.491+0000 7f34927fc700 1 --2- 172.21.15.133:0/2651130498 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f347c038710 0x7f347c03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:24.293 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.491+0000 7f34927fc700 1 --2- 172.21.15.133:0/2651130498 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f347c038710 0x7f347c03abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3488006fd0 tx=0x7f3488006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:24.293 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.495+0000 7f34988f1700 1 -- 172.21.15.133:0/2651130498 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f34840189a0 con 0x7f349407bdf0 2023-07-14T18:07:24.293 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.689+0000 7f34998f3700 1 -- 172.21.15.133:0/2651130498 --> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f3494101f60 con 0x7f347c038710 2023-07-14T18:07:24.293 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.796+0000 7f34988f1700 1 -- 172.21.15.133:0/2651130498 <== mgr.14120 v2:172.21.15.133:6800/2986586981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3494101f60 con 0x7f347c038710 2023-07-14T18:07:24.293 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.799+0000 7f34998f3700 1 -- 172.21.15.133:0/2651130498 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f347c038710 msgr2=0x7f347c03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:24.294 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.799+0000 7f34998f3700 1 --2- 172.21.15.133:0/2651130498 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f347c038710 0x7f347c03abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3488006fd0 tx=0x7f3488006e40 comp rx=0 tx=0).stop 2023-07-14T18:07:24.294 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.799+0000 7f34998f3700 1 -- 172.21.15.133:0/2651130498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f349407bdf0 msgr2=0x7f349419b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:24.294 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.799+0000 7f34998f3700 1 --2- 172.21.15.133:0/2651130498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f349407bdf0 0x7f349419b360 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3484006af0 tx=0x7f3484011ad0 comp rx=0 tx=0).stop 2023-07-14T18:07:24.294 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.799+0000 7f34998f3700 1 -- 172.21.15.133:0/2651130498 shutdown_connections 2023-07-14T18:07:24.294 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.799+0000 7f34998f3700 1 --2- 172.21.15.133:0/2651130498 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f347c038710 0x7f347c03abc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:24.294 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.799+0000 7f34998f3700 1 --2- 172.21.15.133:0/2651130498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f349407bdf0 0x7f349419b360 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:24.294 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.799+0000 7f34998f3700 1 -- 172.21.15.133:0/2651130498 >> 172.21.15.133:0/2651130498 conn(0x7f3494100ab0 msgr2=0x7f3494101850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:24.295 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.799+0000 7f34998f3700 1 -- 172.21.15.133:0/2651130498 shutdown_connections 2023-07-14T18:07:24.295 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:23.799+0000 7f34998f3700 1 -- 172.21.15.133:0/2651130498 wait complete. 2023-07-14T18:07:25.146 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:24 smithi133 ceph-mon[126384]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:07:25.146 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:24 smithi133 ceph-mon[126384]: Generating ssh key... 2023-07-14T18:07:25.146 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:24 smithi133 ceph-mon[126384]: from='mgr.14120 172.21.15.133:0/1173332444' entity='mgr.x' 2023-07-14T18:07:25.146 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:24 smithi133 ceph-mon[126384]: from='mgr.14120 172.21.15.133:0/1173332444' entity='mgr.x' 2023-07-14T18:07:25.558 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrFlTG0nyEvUyDLAOrZfbfgyWYxB55ugggPTkX1pNAXRJIxVvjXLaa/ghiJ/h4lmU069hgLyCYatnO8RtEsgSYdTPlDI/ecVmJgprQV5qwYx0nR8vGQSQJIAQDbUQ0LIW9k1zVQp4ZvNCYx3W60+vD2pVA+/9Qe0IevJqiyjchDJGrqJMfYk4vCVNLkl0v5HNuJqZa/ygE4a/tH8YyyF8s67sbA/dxNQC8dRjkc0L3L/UpX42g/+jX92b4vgqikvyXEOR0GMiA89Yyry8ysTBn2VfIPAi53EJ2B6/4txeX06xM7DQ3NzFrk7cH7+3ir4d2QHhn4QyP9yE+BU7eFRhrLz9udTHw/5/0CDKfMUqARb89N+L5lwtC6YIvdiq3be5QlbuvBKIRnZlhq4+Gk7kchQvafaLOyC2Kafo30YqHP/B+TiDmw4qIpcfofGDqW6xoQ5dYJ+ziQCNnqemZp1ROuawGoOj9hRlZ8SUF9aOP/3dEblsyfYrRXHPBSQJQGfk= ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c 2023-07-14T18:07:25.558 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.758+0000 7f4560335700 1 Processor -- start 2023-07-14T18:07:25.558 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.758+0000 7f4560335700 1 -- start start 2023-07-14T18:07:25.558 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.759+0000 7f4560335700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4558108b50 0x7f4558108f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:25.558 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.759+0000 7f4560335700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4558109530 con 0x7f4558108b50 2023-07-14T18:07:25.559 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.759+0000 7f455e0d1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4558108b50 0x7f4558108f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:25.559 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.759+0000 7f455e0d1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4558108b50 0x7f4558108f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57920/0 (socket says 172.21.15.133:57920) 2023-07-14T18:07:25.559 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.759+0000 7f455e0d1700 1 -- 172.21.15.133:0/73800576 learned_addr learned my addr 172.21.15.133:0/73800576 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:25.559 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.759+0000 7f455e0d1700 1 -- 172.21.15.133:0/73800576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4558109670 con 0x7f4558108b50 2023-07-14T18:07:25.559 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.760+0000 7f455e0d1700 1 --2- 172.21.15.133:0/73800576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4558108b50 0x7f4558108f60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f45540098a0 tx=0x7f4554009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3eabb279fd5e2b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:25.559 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.760+0000 7f455d0cf700 1 -- 172.21.15.133:0/73800576 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4554004160 con 0x7f4558108b50 2023-07-14T18:07:25.559 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.760+0000 7f455d0cf700 1 -- 172.21.15.133:0/73800576 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f45540042c0 con 0x7f4558108b50 2023-07-14T18:07:25.560 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.760+0000 7f455d0cf700 1 -- 172.21.15.133:0/73800576 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4554011a00 con 0x7f4558108b50 2023-07-14T18:07:25.560 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.760+0000 7f4560335700 1 -- 172.21.15.133:0/73800576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4558108b50 msgr2=0x7f4558108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:25.560 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.760+0000 7f4560335700 1 --2- 172.21.15.133:0/73800576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4558108b50 0x7f4558108f60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f45540098a0 tx=0x7f4554009bb0 comp rx=0 tx=0).stop 2023-07-14T18:07:25.560 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.761+0000 7f4560335700 1 -- 172.21.15.133:0/73800576 shutdown_connections 2023-07-14T18:07:25.563 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.761+0000 7f4560335700 1 --2- 172.21.15.133:0/73800576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4558108b50 0x7f4558108f60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:25.563 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.761+0000 7f4560335700 1 -- 172.21.15.133:0/73800576 >> 172.21.15.133:0/73800576 conn(0x7f455807aab0 msgr2=0x7f455807aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:25.563 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.761+0000 7f4560335700 1 -- 172.21.15.133:0/73800576 shutdown_connections 2023-07-14T18:07:25.563 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.761+0000 7f4560335700 1 -- 172.21.15.133:0/73800576 wait complete. 2023-07-14T18:07:25.564 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.762+0000 7f4560335700 1 Processor -- start 2023-07-14T18:07:25.564 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.762+0000 7f4560335700 1 -- start start 2023-07-14T18:07:25.564 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.762+0000 7f4560335700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4558108b50 0x7f455819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:25.564 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.762+0000 7f4560335700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455819b8a0 con 0x7f4558108b50 2023-07-14T18:07:25.564 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.762+0000 7f455e0d1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4558108b50 0x7f455819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:25.564 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.762+0000 7f455e0d1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4558108b50 0x7f455819b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57930/0 (socket says 172.21.15.133:57930) 2023-07-14T18:07:25.564 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.762+0000 7f455e0d1700 1 -- 172.21.15.133:0/3115152268 learned_addr learned my addr 172.21.15.133:0/3115152268 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:25.565 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.763+0000 7f455e0d1700 1 -- 172.21.15.133:0/3115152268 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4554009550 con 0x7f4558108b50 2023-07-14T18:07:25.565 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.763+0000 7f455e0d1700 1 --2- 172.21.15.133:0/3115152268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4558108b50 0x7f455819b360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4554006af0 tx=0x7f4554011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:25.565 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.763+0000 7f454f7fe700 1 -- 172.21.15.133:0/3115152268 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4554003a40 con 0x7f4558108b50 2023-07-14T18:07:25.565 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.763+0000 7f454f7fe700 1 -- 172.21.15.133:0/3115152268 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4554003ba0 con 0x7f4558108b50 2023-07-14T18:07:25.565 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.763+0000 7f454f7fe700 1 -- 172.21.15.133:0/3115152268 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f455401a700 con 0x7f4558108b50 2023-07-14T18:07:25.565 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.763+0000 7f4560335700 1 -- 172.21.15.133:0/3115152268 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f455819baa0 con 0x7f4558108b50 2023-07-14T18:07:25.566 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.763+0000 7f4560335700 1 -- 172.21.15.133:0/3115152268 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f455819bec0 con 0x7f4558108b50 2023-07-14T18:07:25.566 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.764+0000 7f454f7fe700 1 -- 172.21.15.133:0/3115152268 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f4554024ad0 con 0x7f4558108b50 2023-07-14T18:07:25.566 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.764+0000 7f4560335700 1 -- 172.21.15.133:0/3115152268 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f455810bae0 con 0x7f4558108b50 2023-07-14T18:07:25.566 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.765+0000 7f454f7fe700 1 --2- 172.21.15.133:0/3115152268 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f4544038710 0x7f454403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:25.566 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.765+0000 7f454f7fe700 1 -- 172.21.15.133:0/3115152268 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f455404c7f0 con 0x7f4558108b50 2023-07-14T18:07:25.567 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.765+0000 7f455d8d0700 1 --2- 172.21.15.133:0/3115152268 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f4544038710 0x7f454403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:25.567 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.765+0000 7f455d8d0700 1 --2- 172.21.15.133:0/3115152268 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f4544038710 0x7f454403abc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4548006fd0 tx=0x7f4548006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:25.567 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.770+0000 7f454f7fe700 1 -- 172.21.15.133:0/3115152268 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f4554018b80 con 0x7f4558108b50 2023-07-14T18:07:25.567 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.966+0000 7f4560335700 1 -- 172.21.15.133:0/3115152268 --> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f4558105ec0 con 0x7f4544038710 2023-07-14T18:07:25.567 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.967+0000 7f454f7fe700 1 -- 172.21.15.133:0/3115152268 <== mgr.14120 v2:172.21.15.133:6800/2986586981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f4558105ec0 con 0x7f4544038710 2023-07-14T18:07:25.567 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.969+0000 7f4560335700 1 -- 172.21.15.133:0/3115152268 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f4544038710 msgr2=0x7f454403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:25.567 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.969+0000 7f4560335700 1 --2- 172.21.15.133:0/3115152268 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f4544038710 0x7f454403abc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4548006fd0 tx=0x7f4548006e40 comp rx=0 tx=0).stop 2023-07-14T18:07:25.568 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.970+0000 7f4560335700 1 -- 172.21.15.133:0/3115152268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4558108b50 msgr2=0x7f455819b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:25.568 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.970+0000 7f4560335700 1 --2- 172.21.15.133:0/3115152268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4558108b50 0x7f455819b360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4554006af0 tx=0x7f4554011ad0 comp rx=0 tx=0).stop 2023-07-14T18:07:25.568 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.970+0000 7f4560335700 1 -- 172.21.15.133:0/3115152268 shutdown_connections 2023-07-14T18:07:25.568 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.970+0000 7f4560335700 1 --2- 172.21.15.133:0/3115152268 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f4544038710 0x7f454403abc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:25.568 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.970+0000 7f4560335700 1 --2- 172.21.15.133:0/3115152268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4558108b50 0x7f455819b360 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:25.568 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.970+0000 7f4560335700 1 -- 172.21.15.133:0/3115152268 >> 172.21.15.133:0/3115152268 conn(0x7f455807aab0 msgr2=0x7f45581057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:25.568 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.970+0000 7f4560335700 1 -- 172.21.15.133:0/3115152268 shutdown_connections 2023-07-14T18:07:25.568 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:24.970+0000 7f4560335700 1 -- 172.21.15.133:0/3115152268 wait complete. 2023-07-14T18:07:25.569 INFO:teuthology.orchestra.run.smithi133.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-07-14T18:07:25.569 INFO:teuthology.orchestra.run.smithi133.stdout:Adding key to root@localhost authorized_keys... 2023-07-14T18:07:25.569 INFO:teuthology.orchestra.run.smithi133.stdout:Adding host smithi133... 2023-07-14T18:07:26.146 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:25 smithi133 ceph-mon[126384]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:07:27.119 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:26 smithi133 ceph-mon[126384]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi133", "addr": "172.21.15.133", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:07:28.145 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:27 smithi133 ceph-mon[126384]: Deploying cephadm binary to smithi133 2023-07-14T18:07:32.053 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout Added host 'smithi133' with addr '172.21.15.133' 2023-07-14T18:07:32.053 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.181+0000 7f4ebe23f700 1 Processor -- start 2023-07-14T18:07:32.053 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.182+0000 7f4ebe23f700 1 -- start start 2023-07-14T18:07:32.053 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.182+0000 7f4ebe23f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4eb8105f40 0x7f4eb8108320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:32.053 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.182+0000 7f4ebe23f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb8079d90 con 0x7f4eb8105f40 2023-07-14T18:07:32.053 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.183+0000 7f4eb77fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4eb8105f40 0x7f4eb8108320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:32.054 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.183+0000 7f4eb77fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4eb8105f40 0x7f4eb8108320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57942/0 (socket says 172.21.15.133:57942) 2023-07-14T18:07:32.054 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.183+0000 7f4eb77fe700 1 -- 172.21.15.133:0/526372164 learned_addr learned my addr 172.21.15.133:0/526372164 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:32.054 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.183+0000 7f4eb77fe700 1 -- 172.21.15.133:0/526372164 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb8079ed0 con 0x7f4eb8105f40 2023-07-14T18:07:32.054 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.183+0000 7f4eb77fe700 1 --2- 172.21.15.133:0/526372164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4eb8105f40 0x7f4eb8108320 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80098a0 tx=0x7f4ea8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40817eece962fea0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:32.054 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.183+0000 7f4eb67fc700 1 -- 172.21.15.133:0/526372164 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ea8004160 con 0x7f4eb8105f40 2023-07-14T18:07:32.054 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.184+0000 7f4eb67fc700 1 -- 172.21.15.133:0/526372164 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4ea80042c0 con 0x7f4eb8105f40 2023-07-14T18:07:32.054 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.184+0000 7f4eb67fc700 1 -- 172.21.15.133:0/526372164 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ea8011a00 con 0x7f4eb8105f40 2023-07-14T18:07:32.055 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.184+0000 7f4ebe23f700 1 -- 172.21.15.133:0/526372164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4eb8105f40 msgr2=0x7f4eb8108320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:32.055 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.184+0000 7f4ebe23f700 1 --2- 172.21.15.133:0/526372164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4eb8105f40 0x7f4eb8108320 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80098a0 tx=0x7f4ea8009bb0 comp rx=0 tx=0).stop 2023-07-14T18:07:32.055 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.184+0000 7f4ebe23f700 1 -- 172.21.15.133:0/526372164 shutdown_connections 2023-07-14T18:07:32.055 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.184+0000 7f4ebe23f700 1 --2- 172.21.15.133:0/526372164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4eb8105f40 0x7f4eb8108320 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:32.055 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.184+0000 7f4ebe23f700 1 -- 172.21.15.133:0/526372164 >> 172.21.15.133:0/526372164 conn(0x7f4eb80ff980 msgr2=0x7f4eb8101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:32.055 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.184+0000 7f4ebe23f700 1 -- 172.21.15.133:0/526372164 shutdown_connections 2023-07-14T18:07:32.055 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.184+0000 7f4ebe23f700 1 -- 172.21.15.133:0/526372164 wait complete. 2023-07-14T18:07:32.056 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.185+0000 7f4ebe23f700 1 Processor -- start 2023-07-14T18:07:32.056 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.185+0000 7f4ebe23f700 1 -- start start 2023-07-14T18:07:32.056 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.186+0000 7f4ebe23f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4eb8105f40 0x7f4eb819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:32.056 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.186+0000 7f4ebe23f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb819b840 con 0x7f4eb8105f40 2023-07-14T18:07:32.056 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.186+0000 7f4eb77fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4eb8105f40 0x7f4eb819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:32.056 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.186+0000 7f4eb77fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4eb8105f40 0x7f4eb819b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57950/0 (socket says 172.21.15.133:57950) 2023-07-14T18:07:32.056 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.186+0000 7f4eb77fe700 1 -- 172.21.15.133:0/3856215480 learned_addr learned my addr 172.21.15.133:0/3856215480 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:32.056 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.186+0000 7f4eb77fe700 1 -- 172.21.15.133:0/3856215480 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea8009550 con 0x7f4eb8105f40 2023-07-14T18:07:32.057 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.187+0000 7f4eb77fe700 1 --2- 172.21.15.133:0/3856215480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4eb8105f40 0x7f4eb819b300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009fb0 tx=0x7f4ea8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:32.057 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.187+0000 7f4eb4ff9700 1 -- 172.21.15.133:0/3856215480 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ea8003930 con 0x7f4eb8105f40 2023-07-14T18:07:32.057 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.187+0000 7f4eb4ff9700 1 -- 172.21.15.133:0/3856215480 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4ea8003a90 con 0x7f4eb8105f40 2023-07-14T18:07:32.057 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.187+0000 7f4ebe23f700 1 -- 172.21.15.133:0/3856215480 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb819ba40 con 0x7f4eb8105f40 2023-07-14T18:07:32.057 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.187+0000 7f4eb4ff9700 1 -- 172.21.15.133:0/3856215480 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ea80195e0 con 0x7f4eb8105f40 2023-07-14T18:07:32.057 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.187+0000 7f4ebe23f700 1 -- 172.21.15.133:0/3856215480 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb819be60 con 0x7f4eb8105f40 2023-07-14T18:07:32.057 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.188+0000 7f4ebe23f700 1 -- 172.21.15.133:0/3856215480 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb8109530 con 0x7f4eb8105f40 2023-07-14T18:07:32.057 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.188+0000 7f4eb4ff9700 1 -- 172.21.15.133:0/3856215480 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f4ea801a430 con 0x7f4eb8105f40 2023-07-14T18:07:32.058 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.188+0000 7f4eb4ff9700 1 --2- 172.21.15.133:0/3856215480 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f4ea0038710 0x7f4ea003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:32.058 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.188+0000 7f4eb4ff9700 1 -- 172.21.15.133:0/3856215480 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4ea804c860 con 0x7f4eb8105f40 2023-07-14T18:07:32.058 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.189+0000 7f4eb6ffd700 1 --2- 172.21.15.133:0/3856215480 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f4ea0038710 0x7f4ea003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:32.058 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.189+0000 7f4eb6ffd700 1 --2- 172.21.15.133:0/3856215480 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f4ea0038710 0x7f4ea003abc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4eac006fd0 tx=0x7f4eac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:32.058 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.193+0000 7f4eb4ff9700 1 -- 172.21.15.133:0/3856215480 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f4ea8023840 con 0x7f4eb8105f40 2023-07-14T18:07:32.058 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:26.391+0000 7f4ebe23f700 1 -- 172.21.15.133:0/3856215480 --> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi133", "addr": "172.21.15.133", "target": ["mon-mgr", ""]}) v1 -- 0x7f4eb819ecc0 con 0x7f4ea0038710 2023-07-14T18:07:32.058 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:31.593+0000 7f4eb4ff9700 1 -- 172.21.15.133:0/3856215480 <== mgr.14120 v2:172.21.15.133:6800/2986586981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f4eb819ecc0 con 0x7f4ea0038710 2023-07-14T18:07:32.059 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:31.596+0000 7f4ebe23f700 1 -- 172.21.15.133:0/3856215480 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f4ea0038710 msgr2=0x7f4ea003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:32.059 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:31.596+0000 7f4ebe23f700 1 --2- 172.21.15.133:0/3856215480 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f4ea0038710 0x7f4ea003abc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4eac006fd0 tx=0x7f4eac006e40 comp rx=0 tx=0).stop 2023-07-14T18:07:32.059 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:31.596+0000 7f4ebe23f700 1 -- 172.21.15.133:0/3856215480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4eb8105f40 msgr2=0x7f4eb819b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:32.059 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:31.596+0000 7f4ebe23f700 1 --2- 172.21.15.133:0/3856215480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4eb8105f40 0x7f4eb819b300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009fb0 tx=0x7f4ea8011ad0 comp rx=0 tx=0).stop 2023-07-14T18:07:32.059 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:31.596+0000 7f4ebe23f700 1 -- 172.21.15.133:0/3856215480 shutdown_connections 2023-07-14T18:07:32.059 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:31.596+0000 7f4ebe23f700 1 --2- 172.21.15.133:0/3856215480 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f4ea0038710 0x7f4ea003abc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:32.059 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:31.596+0000 7f4ebe23f700 1 --2- 172.21.15.133:0/3856215480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4eb8105f40 0x7f4eb819b300 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:32.059 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:31.596+0000 7f4ebe23f700 1 -- 172.21.15.133:0/3856215480 >> 172.21.15.133:0/3856215480 conn(0x7f4eb80ff980 msgr2=0x7f4eb8100710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:32.060 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:31.597+0000 7f4ebe23f700 1 -- 172.21.15.133:0/3856215480 shutdown_connections 2023-07-14T18:07:32.060 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:31.597+0000 7f4ebe23f700 1 -- 172.21.15.133:0/3856215480 wait complete. 2023-07-14T18:07:32.060 INFO:teuthology.orchestra.run.smithi133.stdout:Deploying unmanaged mon service... 2023-07-14T18:07:32.896 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:32 smithi133 ceph-mon[126384]: from='mgr.14120 172.21.15.133:0/1173332444' entity='mgr.x' 2023-07-14T18:07:32.896 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:32 smithi133 ceph-mon[126384]: Added host smithi133 2023-07-14T18:07:32.896 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:32 smithi133 ceph-mon[126384]: from='mgr.14120 172.21.15.133:0/1173332444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:07:33.328 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-07-14T18:07:33.328 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.558+0000 7ff51c1fd700 1 Processor -- start 2023-07-14T18:07:33.328 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.558+0000 7ff51c1fd700 1 -- start start 2023-07-14T18:07:33.328 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.558+0000 7ff51c1fd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff514105440 0x7ff514105850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:33.328 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.558+0000 7ff51c1fd700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff514105d90 con 0x7ff514105440 2023-07-14T18:07:33.329 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.559+0000 7ff519f99700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff514105440 0x7ff514105850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:33.329 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.559+0000 7ff519f99700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff514105440 0x7ff514105850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42688/0 (socket says 172.21.15.133:42688) 2023-07-14T18:07:33.329 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.559+0000 7ff519f99700 1 -- 172.21.15.133:0/1501689270 learned_addr learned my addr 172.21.15.133:0/1501689270 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:33.329 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.559+0000 7ff519f99700 1 -- 172.21.15.133:0/1501689270 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff514105ed0 con 0x7ff514105440 2023-07-14T18:07:33.329 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.559+0000 7ff519f99700 1 --2- 172.21.15.133:0/1501689270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff514105440 0x7ff514105850 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff5100098a0 tx=0x7ff510009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699c494980b18d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:33.329 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.559+0000 7ff518f97700 1 -- 172.21.15.133:0/1501689270 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff510004160 con 0x7ff514105440 2023-07-14T18:07:33.329 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.560+0000 7ff518f97700 1 -- 172.21.15.133:0/1501689270 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff5100042c0 con 0x7ff514105440 2023-07-14T18:07:33.329 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.560+0000 7ff518f97700 1 -- 172.21.15.133:0/1501689270 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff510011a00 con 0x7ff514105440 2023-07-14T18:07:33.330 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.560+0000 7ff51c1fd700 1 -- 172.21.15.133:0/1501689270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff514105440 msgr2=0x7ff514105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:33.330 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.560+0000 7ff51c1fd700 1 --2- 172.21.15.133:0/1501689270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff514105440 0x7ff514105850 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff5100098a0 tx=0x7ff510009bb0 comp rx=0 tx=0).stop 2023-07-14T18:07:33.330 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.560+0000 7ff51c1fd700 1 -- 172.21.15.133:0/1501689270 shutdown_connections 2023-07-14T18:07:33.330 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.560+0000 7ff51c1fd700 1 --2- 172.21.15.133:0/1501689270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff514105440 0x7ff514105850 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:33.330 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.560+0000 7ff51c1fd700 1 -- 172.21.15.133:0/1501689270 >> 172.21.15.133:0/1501689270 conn(0x7ff514100ab0 msgr2=0x7ff514102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:33.330 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.560+0000 7ff51c1fd700 1 -- 172.21.15.133:0/1501689270 shutdown_connections 2023-07-14T18:07:33.330 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.560+0000 7ff51c1fd700 1 -- 172.21.15.133:0/1501689270 wait complete. 2023-07-14T18:07:33.331 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.561+0000 7ff51c1fd700 1 Processor -- start 2023-07-14T18:07:33.331 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.562+0000 7ff51c1fd700 1 -- start start 2023-07-14T18:07:33.331 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.562+0000 7ff51c1fd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff514105440 0x7ff5141990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:33.331 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.562+0000 7ff51c1fd700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff514199630 con 0x7ff514105440 2023-07-14T18:07:33.331 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.562+0000 7ff519f99700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff514105440 0x7ff5141990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:33.331 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.562+0000 7ff519f99700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff514105440 0x7ff5141990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42698/0 (socket says 172.21.15.133:42698) 2023-07-14T18:07:33.331 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.562+0000 7ff519f99700 1 -- 172.21.15.133:0/346909474 learned_addr learned my addr 172.21.15.133:0/346909474 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:33.331 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.562+0000 7ff519f99700 1 -- 172.21.15.133:0/346909474 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff510009550 con 0x7ff514105440 2023-07-14T18:07:33.332 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.563+0000 7ff519f99700 1 --2- 172.21.15.133:0/346909474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff514105440 0x7ff5141990f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff510006af0 tx=0x7ff510011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:33.332 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.563+0000 7ff50affd700 1 -- 172.21.15.133:0/346909474 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff510003a40 con 0x7ff514105440 2023-07-14T18:07:33.332 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.563+0000 7ff50affd700 1 -- 172.21.15.133:0/346909474 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff510003ba0 con 0x7ff514105440 2023-07-14T18:07:33.332 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.563+0000 7ff50affd700 1 -- 172.21.15.133:0/346909474 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff51001a6e0 con 0x7ff514105440 2023-07-14T18:07:33.332 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.563+0000 7ff51c1fd700 1 -- 172.21.15.133:0/346909474 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff514199830 con 0x7ff514105440 2023-07-14T18:07:33.332 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.563+0000 7ff51c1fd700 1 -- 172.21.15.133:0/346909474 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff514199c50 con 0x7ff514105440 2023-07-14T18:07:33.332 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.564+0000 7ff50affd700 1 -- 172.21.15.133:0/346909474 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7ff510024a60 con 0x7ff514105440 2023-07-14T18:07:33.332 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.564+0000 7ff508ff9700 1 -- 172.21.15.133:0/346909474 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff514061c70 con 0x7ff514105440 2023-07-14T18:07:33.333 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.565+0000 7ff50affd700 1 --2- 172.21.15.133:0/346909474 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7ff500038760 0x7ff50003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:33.333 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.565+0000 7ff50affd700 1 -- 172.21.15.133:0/346909474 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff51004c710 con 0x7ff514105440 2023-07-14T18:07:33.333 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.565+0000 7ff519798700 1 --2- 172.21.15.133:0/346909474 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7ff500038760 0x7ff50003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:33.333 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.566+0000 7ff519798700 1 --2- 172.21.15.133:0/346909474 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7ff500038760 0x7ff50003ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff504006fd0 tx=0x7ff504006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:33.333 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.570+0000 7ff50affd700 1 -- 172.21.15.133:0/346909474 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7ff510024430 con 0x7ff514105440 2023-07-14T18:07:33.333 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.787+0000 7ff508ff9700 1 -- 172.21.15.133:0/346909474 --> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ff5141073e0 con 0x7ff500038760 2023-07-14T18:07:33.333 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.793+0000 7ff50affd700 1 -- 172.21.15.133:0/346909474 <== mgr.14120 v2:172.21.15.133:6800/2986586981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7ff5141073e0 con 0x7ff500038760 2023-07-14T18:07:33.334 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.796+0000 7ff508ff9700 1 -- 172.21.15.133:0/346909474 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7ff500038760 msgr2=0x7ff50003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:33.334 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.796+0000 7ff508ff9700 1 --2- 172.21.15.133:0/346909474 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7ff500038760 0x7ff50003ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff504006fd0 tx=0x7ff504006e40 comp rx=0 tx=0).stop 2023-07-14T18:07:33.334 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.796+0000 7ff508ff9700 1 -- 172.21.15.133:0/346909474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff514105440 msgr2=0x7ff5141990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:33.334 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.796+0000 7ff508ff9700 1 --2- 172.21.15.133:0/346909474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff514105440 0x7ff5141990f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff510006af0 tx=0x7ff510011ad0 comp rx=0 tx=0).stop 2023-07-14T18:07:33.334 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.796+0000 7ff508ff9700 1 -- 172.21.15.133:0/346909474 shutdown_connections 2023-07-14T18:07:33.334 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.796+0000 7ff508ff9700 1 --2- 172.21.15.133:0/346909474 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7ff500038760 0x7ff50003ac10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:33.334 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.796+0000 7ff508ff9700 1 --2- 172.21.15.133:0/346909474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff514105440 0x7ff5141990f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:33.334 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.796+0000 7ff508ff9700 1 -- 172.21.15.133:0/346909474 >> 172.21.15.133:0/346909474 conn(0x7ff514100ab0 msgr2=0x7ff514101810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:33.335 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.796+0000 7ff508ff9700 1 -- 172.21.15.133:0/346909474 shutdown_connections 2023-07-14T18:07:33.335 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:32.796+0000 7ff508ff9700 1 -- 172.21.15.133:0/346909474 wait complete. 2023-07-14T18:07:33.335 INFO:teuthology.orchestra.run.smithi133.stdout:Deploying unmanaged mgr service... 2023-07-14T18:07:34.053 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:33 smithi133 ceph-mon[126384]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:07:34.054 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:33 smithi133 ceph-mon[126384]: Saving service mon spec with placement count:5 2023-07-14T18:07:34.054 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:33 smithi133 ceph-mon[126384]: from='mgr.14120 172.21.15.133:0/1173332444' entity='mgr.x' 2023-07-14T18:07:34.809 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-07-14T18:07:34.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.910+0000 7f97b424e700 1 Processor -- start 2023-07-14T18:07:34.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.911+0000 7f97b424e700 1 -- start start 2023-07-14T18:07:34.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.911+0000 7f97b424e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ac105f40 0x7f97ac108320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:34.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.911+0000 7f97b424e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac079d90 con 0x7f97ac105f40 2023-07-14T18:07:34.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.911+0000 7f97b1fea700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ac105f40 0x7f97ac108320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:34.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.911+0000 7f97b1fea700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ac105f40 0x7f97ac108320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42706/0 (socket says 172.21.15.133:42706) 2023-07-14T18:07:34.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.911+0000 7f97b1fea700 1 -- 172.21.15.133:0/1413856638 learned_addr learned my addr 172.21.15.133:0/1413856638 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:34.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.912+0000 7f97b1fea700 1 -- 172.21.15.133:0/1413856638 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac079ed0 con 0x7f97ac105f40 2023-07-14T18:07:34.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.912+0000 7f97b1fea700 1 --2- 172.21.15.133:0/1413856638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ac105f40 0x7f97ac108320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f97a80098a0 tx=0x7f97a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e090c12500aef server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:34.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.912+0000 7f97b0fe8700 1 -- 172.21.15.133:0/1413856638 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97a8004160 con 0x7f97ac105f40 2023-07-14T18:07:34.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.912+0000 7f97b0fe8700 1 -- 172.21.15.133:0/1413856638 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f97a80042c0 con 0x7f97ac105f40 2023-07-14T18:07:34.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.912+0000 7f97b0fe8700 1 -- 172.21.15.133:0/1413856638 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97a8011a00 con 0x7f97ac105f40 2023-07-14T18:07:34.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.913+0000 7f97b424e700 1 -- 172.21.15.133:0/1413856638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ac105f40 msgr2=0x7f97ac108320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:34.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.913+0000 7f97b424e700 1 --2- 172.21.15.133:0/1413856638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ac105f40 0x7f97ac108320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f97a80098a0 tx=0x7f97a8009bb0 comp rx=0 tx=0).stop 2023-07-14T18:07:34.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.913+0000 7f97b424e700 1 -- 172.21.15.133:0/1413856638 shutdown_connections 2023-07-14T18:07:34.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.913+0000 7f97b424e700 1 --2- 172.21.15.133:0/1413856638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ac105f40 0x7f97ac108320 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:34.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.913+0000 7f97b424e700 1 -- 172.21.15.133:0/1413856638 >> 172.21.15.133:0/1413856638 conn(0x7f97ac0ff980 msgr2=0x7f97ac101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:34.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.913+0000 7f97b424e700 1 -- 172.21.15.133:0/1413856638 shutdown_connections 2023-07-14T18:07:34.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.913+0000 7f97b424e700 1 -- 172.21.15.133:0/1413856638 wait complete. 2023-07-14T18:07:34.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.914+0000 7f97b424e700 1 Processor -- start 2023-07-14T18:07:34.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.914+0000 7f97b424e700 1 -- start start 2023-07-14T18:07:34.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.914+0000 7f97b424e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ac105f40 0x7f97ac1058a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:34.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.914+0000 7f97b424e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac105de0 con 0x7f97ac105f40 2023-07-14T18:07:34.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.915+0000 7f97b1fea700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ac105f40 0x7f97ac1058a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:34.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.915+0000 7f97b1fea700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ac105f40 0x7f97ac1058a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42718/0 (socket says 172.21.15.133:42718) 2023-07-14T18:07:34.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.915+0000 7f97b1fea700 1 -- 172.21.15.133:0/486964459 learned_addr learned my addr 172.21.15.133:0/486964459 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:34.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.915+0000 7f97b1fea700 1 -- 172.21.15.133:0/486964459 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a8009550 con 0x7f97ac105f40 2023-07-14T18:07:34.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.915+0000 7f97b1fea700 1 --2- 172.21.15.133:0/486964459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ac105f40 0x7f97ac1058a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f97a8009fb0 tx=0x7f97a8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:34.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.915+0000 7f97a2ffd700 1 -- 172.21.15.133:0/486964459 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97a8003930 con 0x7f97ac105f40 2023-07-14T18:07:34.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.915+0000 7f97a2ffd700 1 -- 172.21.15.133:0/486964459 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f97a8003a90 con 0x7f97ac105f40 2023-07-14T18:07:34.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.915+0000 7f97b424e700 1 -- 172.21.15.133:0/486964459 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ac103ef0 con 0x7f97ac105f40 2023-07-14T18:07:34.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.915+0000 7f97a2ffd700 1 -- 172.21.15.133:0/486964459 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97a80195e0 con 0x7f97ac105f40 2023-07-14T18:07:34.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.916+0000 7f97b424e700 1 -- 172.21.15.133:0/486964459 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ac104310 con 0x7f97ac105f40 2023-07-14T18:07:34.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.917+0000 7f97a2ffd700 1 -- 172.21.15.133:0/486964459 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f97a801a430 con 0x7f97ac105f40 2023-07-14T18:07:34.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.917+0000 7f97b424e700 1 -- 172.21.15.133:0/486964459 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ac109530 con 0x7f97ac105f40 2023-07-14T18:07:34.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.917+0000 7f97a2ffd700 1 --2- 172.21.15.133:0/486964459 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f9798038710 0x7f979803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:34.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.917+0000 7f97a2ffd700 1 -- 172.21.15.133:0/486964459 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f97a804c820 con 0x7f97ac105f40 2023-07-14T18:07:34.815 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.917+0000 7f97b17e9700 1 --2- 172.21.15.133:0/486964459 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f9798038710 0x7f979803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:34.815 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.918+0000 7f97b17e9700 1 --2- 172.21.15.133:0/486964459 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f9798038710 0x7f979803abc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f979c006fd0 tx=0x7f979c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:34.815 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:33.922+0000 7f97a2ffd700 1 -- 172.21.15.133:0/486964459 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f97a8023800 con 0x7f97ac105f40 2023-07-14T18:07:34.815 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:34.142+0000 7f97b424e700 1 -- 172.21.15.133:0/486964459 --> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f97ac100e20 con 0x7f9798038710 2023-07-14T18:07:34.815 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:34.150+0000 7f97a2ffd700 1 -- 172.21.15.133:0/486964459 <== mgr.14120 v2:172.21.15.133:6800/2986586981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f97ac100e20 con 0x7f9798038710 2023-07-14T18:07:34.815 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:34.152+0000 7f97b424e700 1 -- 172.21.15.133:0/486964459 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f9798038710 msgr2=0x7f979803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:34.815 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:34.152+0000 7f97b424e700 1 --2- 172.21.15.133:0/486964459 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f9798038710 0x7f979803abc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f979c006fd0 tx=0x7f979c006e40 comp rx=0 tx=0).stop 2023-07-14T18:07:34.816 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:34.152+0000 7f97b424e700 1 -- 172.21.15.133:0/486964459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ac105f40 msgr2=0x7f97ac1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:34.816 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:34.152+0000 7f97b424e700 1 --2- 172.21.15.133:0/486964459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ac105f40 0x7f97ac1058a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f97a8009fb0 tx=0x7f97a8011ad0 comp rx=0 tx=0).stop 2023-07-14T18:07:34.816 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:34.152+0000 7f97b424e700 1 -- 172.21.15.133:0/486964459 shutdown_connections 2023-07-14T18:07:34.816 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:34.152+0000 7f97b424e700 1 --2- 172.21.15.133:0/486964459 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f9798038710 0x7f979803abc0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:34.816 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:34.152+0000 7f97b424e700 1 --2- 172.21.15.133:0/486964459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ac105f40 0x7f97ac1058a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:34.816 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:34.152+0000 7f97b424e700 1 -- 172.21.15.133:0/486964459 >> 172.21.15.133:0/486964459 conn(0x7f97ac0ff980 msgr2=0x7f97ac100710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:34.816 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:34.152+0000 7f97b424e700 1 -- 172.21.15.133:0/486964459 shutdown_connections 2023-07-14T18:07:34.816 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:34.152+0000 7f97b424e700 1 -- 172.21.15.133:0/486964459 wait complete. 2023-07-14T18:07:35.396 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:35 smithi133 ceph-mon[126384]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:07:35.396 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:35 smithi133 ceph-mon[126384]: Saving service mgr spec with placement count:2 2023-07-14T18:07:35.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:35 smithi133 ceph-mon[126384]: from='mgr.14120 172.21.15.133:0/1173332444' entity='mgr.x' 2023-07-14T18:07:35.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:35 smithi133 ceph-mon[126384]: from='mgr.14120 172.21.15.133:0/1173332444' entity='mgr.x' 2023-07-14T18:07:35.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:35 smithi133 ceph-mon[126384]: from='mgr.14120 172.21.15.133:0/1173332444' entity='mgr.x' 2023-07-14T18:07:35.988 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.327+0000 7fc2f5fb7700 1 Processor -- start 2023-07-14T18:07:35.988 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.327+0000 7fc2f5fb7700 1 -- start start 2023-07-14T18:07:35.989 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.328+0000 7fc2f5fb7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2f007bdf0 0x7fc2f007a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:35.989 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.328+0000 7fc2f5fb7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f007a830 con 0x7fc2f007bdf0 2023-07-14T18:07:35.989 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.328+0000 7fc2ef7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2f007bdf0 0x7fc2f007a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:35.989 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.328+0000 7fc2ef7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2f007bdf0 0x7fc2f007a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42734/0 (socket says 172.21.15.133:42734) 2023-07-14T18:07:35.989 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.328+0000 7fc2ef7fe700 1 -- 172.21.15.133:0/335557352 learned_addr learned my addr 172.21.15.133:0/335557352 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:35.989 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.328+0000 7fc2ef7fe700 1 -- 172.21.15.133:0/335557352 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f007a970 con 0x7fc2f007bdf0 2023-07-14T18:07:35.989 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.328+0000 7fc2ef7fe700 1 --2- 172.21.15.133:0/335557352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2f007bdf0 0x7fc2f007a2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00098a0 tx=0x7fc2e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2c65ec52fe3bdc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:35.990 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.329+0000 7fc2ee7fc700 1 -- 172.21.15.133:0/335557352 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2e0004160 con 0x7fc2f007bdf0 2023-07-14T18:07:35.990 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.329+0000 7fc2ee7fc700 1 -- 172.21.15.133:0/335557352 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2e00042c0 con 0x7fc2f007bdf0 2023-07-14T18:07:35.990 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.329+0000 7fc2ee7fc700 1 -- 172.21.15.133:0/335557352 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2e00119b0 con 0x7fc2f007bdf0 2023-07-14T18:07:35.990 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.329+0000 7fc2f5fb7700 1 -- 172.21.15.133:0/335557352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2f007bdf0 msgr2=0x7fc2f007a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:35.990 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.329+0000 7fc2f5fb7700 1 --2- 172.21.15.133:0/335557352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2f007bdf0 0x7fc2f007a2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00098a0 tx=0x7fc2e0009bb0 comp rx=0 tx=0).stop 2023-07-14T18:07:35.990 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.329+0000 7fc2f5fb7700 1 -- 172.21.15.133:0/335557352 shutdown_connections 2023-07-14T18:07:35.990 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.329+0000 7fc2f5fb7700 1 --2- 172.21.15.133:0/335557352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2f007bdf0 0x7fc2f007a2f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:35.991 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.329+0000 7fc2f5fb7700 1 -- 172.21.15.133:0/335557352 >> 172.21.15.133:0/335557352 conn(0x7fc2f0100ab0 msgr2=0x7fc2f0102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:35.991 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.329+0000 7fc2f5fb7700 1 -- 172.21.15.133:0/335557352 shutdown_connections 2023-07-14T18:07:35.991 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.329+0000 7fc2f5fb7700 1 -- 172.21.15.133:0/335557352 wait complete. 2023-07-14T18:07:35.991 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.330+0000 7fc2f5fb7700 1 Processor -- start 2023-07-14T18:07:35.991 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.331+0000 7fc2f5fb7700 1 -- start start 2023-07-14T18:07:35.991 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.331+0000 7fc2f5fb7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2f007bdf0 0x7fc2f019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:35.991 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.331+0000 7fc2f5fb7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f019b8d0 con 0x7fc2f007bdf0 2023-07-14T18:07:35.991 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.331+0000 7fc2ef7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2f007bdf0 0x7fc2f019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:35.992 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.331+0000 7fc2ef7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2f007bdf0 0x7fc2f019b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42742/0 (socket says 172.21.15.133:42742) 2023-07-14T18:07:35.992 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.331+0000 7fc2ef7fe700 1 -- 172.21.15.133:0/3353642607 learned_addr learned my addr 172.21.15.133:0/3353642607 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:35.992 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.331+0000 7fc2ef7fe700 1 -- 172.21.15.133:0/3353642607 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e0009550 con 0x7fc2f007bdf0 2023-07-14T18:07:35.992 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.332+0000 7fc2ef7fe700 1 --2- 172.21.15.133:0/3353642607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2f007bdf0 0x7fc2f019b390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0000c00 tx=0x7fc2e0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:35.992 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.332+0000 7fc2ecff9700 1 -- 172.21.15.133:0/3353642607 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2e00039f0 con 0x7fc2f007bdf0 2023-07-14T18:07:35.992 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.332+0000 7fc2ecff9700 1 -- 172.21.15.133:0/3353642607 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2e0003b50 con 0x7fc2f007bdf0 2023-07-14T18:07:35.992 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.332+0000 7fc2ecff9700 1 -- 172.21.15.133:0/3353642607 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2e001a780 con 0x7fc2f007bdf0 2023-07-14T18:07:35.993 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.332+0000 7fc2f5fb7700 1 -- 172.21.15.133:0/3353642607 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f019bad0 con 0x7fc2f007bdf0 2023-07-14T18:07:35.993 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.332+0000 7fc2f5fb7700 1 -- 172.21.15.133:0/3353642607 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f019bef0 con 0x7fc2f007bdf0 2023-07-14T18:07:35.993 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.333+0000 7fc2ecff9700 1 -- 172.21.15.133:0/3353642607 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fc2e0024440 con 0x7fc2f007bdf0 2023-07-14T18:07:35.993 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.333+0000 7fc2f5fb7700 1 -- 172.21.15.133:0/3353642607 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2f0061c70 con 0x7fc2f007bdf0 2023-07-14T18:07:35.993 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.334+0000 7fc2ecff9700 1 --2- 172.21.15.133:0/3353642607 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7fc2d8038710 0x7fc2d803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:35.993 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.334+0000 7fc2ecff9700 1 -- 172.21.15.133:0/3353642607 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc2e004c880 con 0x7fc2f007bdf0 2023-07-14T18:07:35.993 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.334+0000 7fc2eeffd700 1 --2- 172.21.15.133:0/3353642607 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7fc2d8038710 0x7fc2d803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:35.993 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.334+0000 7fc2eeffd700 1 --2- 172.21.15.133:0/3353642607 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7fc2d8038710 0x7fc2d803abc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4006fd0 tx=0x7fc2e4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:35.994 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.339+0000 7fc2ecff9700 1 -- 172.21.15.133:0/3353642607 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fc2e00246f0 con 0x7fc2f007bdf0 2023-07-14T18:07:35.994 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.540+0000 7fc2f5fb7700 1 -- 172.21.15.133:0/3353642607 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fc2f019ed50 con 0x7fc2f007bdf0 2023-07-14T18:07:35.994 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.543+0000 7fc2ecff9700 1 -- 172.21.15.133:0/3353642607 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fc2e0018350 con 0x7fc2f007bdf0 2023-07-14T18:07:35.994 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.546+0000 7fc2f5fb7700 1 -- 172.21.15.133:0/3353642607 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7fc2d8038710 msgr2=0x7fc2d803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:35.994 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.546+0000 7fc2f5fb7700 1 --2- 172.21.15.133:0/3353642607 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7fc2d8038710 0x7fc2d803abc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4006fd0 tx=0x7fc2e4006e40 comp rx=0 tx=0).stop 2023-07-14T18:07:35.994 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.546+0000 7fc2f5fb7700 1 -- 172.21.15.133:0/3353642607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2f007bdf0 msgr2=0x7fc2f019b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:35.994 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.546+0000 7fc2f5fb7700 1 --2- 172.21.15.133:0/3353642607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2f007bdf0 0x7fc2f019b390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0000c00 tx=0x7fc2e0011a30 comp rx=0 tx=0).stop 2023-07-14T18:07:35.995 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.546+0000 7fc2f5fb7700 1 -- 172.21.15.133:0/3353642607 shutdown_connections 2023-07-14T18:07:35.995 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.546+0000 7fc2f5fb7700 1 --2- 172.21.15.133:0/3353642607 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7fc2d8038710 0x7fc2d803abc0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:35.995 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.546+0000 7fc2f5fb7700 1 --2- 172.21.15.133:0/3353642607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2f007bdf0 0x7fc2f019b390 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:35.995 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.546+0000 7fc2f5fb7700 1 -- 172.21.15.133:0/3353642607 >> 172.21.15.133:0/3353642607 conn(0x7fc2f0100ab0 msgr2=0x7fc2f0101880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:35.995 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.547+0000 7fc2f5fb7700 1 -- 172.21.15.133:0/3353642607 shutdown_connections 2023-07-14T18:07:35.995 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:35.547+0000 7fc2f5fb7700 1 -- 172.21.15.133:0/3353642607 wait complete. 2023-07-14T18:07:36.896 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:36 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3353642607' entity='client.admin' 2023-07-14T18:07:37.271 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.527+0000 7f2c05ec4700 1 Processor -- start 2023-07-14T18:07:37.271 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.527+0000 7f2c05ec4700 1 -- start start 2023-07-14T18:07:37.271 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.528+0000 7f2c05ec4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c0007bdf0 0x7f2c0007a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:37.271 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.528+0000 7f2c05ec4700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0007a830 con 0x7f2c0007bdf0 2023-07-14T18:07:37.272 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.528+0000 7f2bff7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c0007bdf0 0x7f2c0007a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:37.272 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.528+0000 7f2bff7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c0007bdf0 0x7f2c0007a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42750/0 (socket says 172.21.15.133:42750) 2023-07-14T18:07:37.275 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.528+0000 7f2bff7fe700 1 -- 172.21.15.133:0/232616893 learned_addr learned my addr 172.21.15.133:0/232616893 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:37.276 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.528+0000 7f2bff7fe700 1 -- 172.21.15.133:0/232616893 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0007a970 con 0x7f2c0007bdf0 2023-07-14T18:07:37.276 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.528+0000 7f2bff7fe700 1 --2- 172.21.15.133:0/232616893 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c0007bdf0 0x7f2c0007a2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00098a0 tx=0x7f2bf0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b5e40b0e473ece server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:37.276 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.529+0000 7f2bfe7fc700 1 -- 172.21.15.133:0/232616893 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bf0004160 con 0x7f2c0007bdf0 2023-07-14T18:07:37.276 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.529+0000 7f2bfe7fc700 1 -- 172.21.15.133:0/232616893 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bf00042c0 con 0x7f2c0007bdf0 2023-07-14T18:07:37.276 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.529+0000 7f2bfe7fc700 1 -- 172.21.15.133:0/232616893 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bf00119b0 con 0x7f2c0007bdf0 2023-07-14T18:07:37.276 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.529+0000 7f2c05ec4700 1 -- 172.21.15.133:0/232616893 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c0007bdf0 msgr2=0x7f2c0007a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:37.277 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.529+0000 7f2c05ec4700 1 --2- 172.21.15.133:0/232616893 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c0007bdf0 0x7f2c0007a2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00098a0 tx=0x7f2bf0009bb0 comp rx=0 tx=0).stop 2023-07-14T18:07:37.277 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.529+0000 7f2c05ec4700 1 -- 172.21.15.133:0/232616893 shutdown_connections 2023-07-14T18:07:37.277 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.529+0000 7f2c05ec4700 1 --2- 172.21.15.133:0/232616893 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c0007bdf0 0x7f2c0007a2f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:37.277 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.529+0000 7f2c05ec4700 1 -- 172.21.15.133:0/232616893 >> 172.21.15.133:0/232616893 conn(0x7f2c00100ab0 msgr2=0x7f2c00102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:37.277 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.529+0000 7f2c05ec4700 1 -- 172.21.15.133:0/232616893 shutdown_connections 2023-07-14T18:07:37.277 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.529+0000 7f2c05ec4700 1 -- 172.21.15.133:0/232616893 wait complete. 2023-07-14T18:07:37.278 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.530+0000 7f2c05ec4700 1 Processor -- start 2023-07-14T18:07:37.278 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.531+0000 7f2c05ec4700 1 -- start start 2023-07-14T18:07:37.278 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.531+0000 7f2c05ec4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c0007bdf0 0x7f2c0019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:37.278 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.531+0000 7f2c05ec4700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0019b8a0 con 0x7f2c0007bdf0 2023-07-14T18:07:37.278 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.531+0000 7f2bff7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c0007bdf0 0x7f2c0019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:37.278 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.531+0000 7f2bff7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c0007bdf0 0x7f2c0019b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42762/0 (socket says 172.21.15.133:42762) 2023-07-14T18:07:37.278 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.531+0000 7f2bff7fe700 1 -- 172.21.15.133:0/3399090105 learned_addr learned my addr 172.21.15.133:0/3399090105 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:37.278 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.531+0000 7f2bff7fe700 1 -- 172.21.15.133:0/3399090105 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf0009550 con 0x7f2c0007bdf0 2023-07-14T18:07:37.279 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.532+0000 7f2bff7fe700 1 --2- 172.21.15.133:0/3399090105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c0007bdf0 0x7f2c0019b360 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0003e60 tx=0x7f2bf0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:37.279 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.532+0000 7f2bfcff9700 1 -- 172.21.15.133:0/3399090105 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bf0003970 con 0x7f2c0007bdf0 2023-07-14T18:07:37.279 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.532+0000 7f2bfcff9700 1 -- 172.21.15.133:0/3399090105 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bf0003ad0 con 0x7f2c0007bdf0 2023-07-14T18:07:37.279 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.532+0000 7f2c05ec4700 1 -- 172.21.15.133:0/3399090105 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c0019baa0 con 0x7f2c0007bdf0 2023-07-14T18:07:37.279 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.532+0000 7f2bfcff9700 1 -- 172.21.15.133:0/3399090105 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bf00195e0 con 0x7f2c0007bdf0 2023-07-14T18:07:37.279 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.532+0000 7f2c05ec4700 1 -- 172.21.15.133:0/3399090105 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c0019bec0 con 0x7f2c0007bdf0 2023-07-14T18:07:37.279 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.533+0000 7f2bfcff9700 1 -- 172.21.15.133:0/3399090105 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f2bf0019850 con 0x7f2c0007bdf0 2023-07-14T18:07:37.280 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.533+0000 7f2c05ec4700 1 -- 172.21.15.133:0/3399090105 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c00061c70 con 0x7f2c0007bdf0 2023-07-14T18:07:37.280 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.534+0000 7f2bfcff9700 1 --2- 172.21.15.133:0/3399090105 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f2be8038710 0x7f2be803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:37.280 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.534+0000 7f2bfcff9700 1 -- 172.21.15.133:0/3399090105 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2bf004c8a0 con 0x7f2c0007bdf0 2023-07-14T18:07:37.280 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.534+0000 7f2bfeffd700 1 --2- 172.21.15.133:0/3399090105 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f2be8038710 0x7f2be803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:37.280 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.535+0000 7f2bfeffd700 1 --2- 172.21.15.133:0/3399090105 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f2be8038710 0x7f2be803abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4006fd0 tx=0x7f2bf4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:37.280 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.539+0000 7f2bfcff9700 1 -- 172.21.15.133:0/3399090105 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f2bf0003c40 con 0x7f2c0007bdf0 2023-07-14T18:07:37.280 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.744+0000 7f2c05ec4700 1 -- 172.21.15.133:0/3399090105 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f2c0019ed20 con 0x7f2c0007bdf0 2023-07-14T18:07:37.281 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.750+0000 7f2bfcff9700 1 -- 172.21.15.133:0/3399090105 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f2bf0023b80 con 0x7f2c0007bdf0 2023-07-14T18:07:37.281 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.752+0000 7f2c05ec4700 1 -- 172.21.15.133:0/3399090105 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f2be8038710 msgr2=0x7f2be803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:37.281 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.752+0000 7f2c05ec4700 1 --2- 172.21.15.133:0/3399090105 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f2be8038710 0x7f2be803abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4006fd0 tx=0x7f2bf4006e40 comp rx=0 tx=0).stop 2023-07-14T18:07:37.281 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.752+0000 7f2c05ec4700 1 -- 172.21.15.133:0/3399090105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c0007bdf0 msgr2=0x7f2c0019b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:37.281 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.752+0000 7f2c05ec4700 1 --2- 172.21.15.133:0/3399090105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c0007bdf0 0x7f2c0019b360 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0003e60 tx=0x7f2bf0011aa0 comp rx=0 tx=0).stop 2023-07-14T18:07:37.281 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.753+0000 7f2c05ec4700 1 -- 172.21.15.133:0/3399090105 shutdown_connections 2023-07-14T18:07:37.281 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.753+0000 7f2c05ec4700 1 --2- 172.21.15.133:0/3399090105 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f2be8038710 0x7f2be803abc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:37.282 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.753+0000 7f2c05ec4700 1 --2- 172.21.15.133:0/3399090105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c0007bdf0 0x7f2c0019b360 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:37.282 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.753+0000 7f2c05ec4700 1 -- 172.21.15.133:0/3399090105 >> 172.21.15.133:0/3399090105 conn(0x7f2c00100ab0 msgr2=0x7f2c00102130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:37.282 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.753+0000 7f2c05ec4700 1 -- 172.21.15.133:0/3399090105 shutdown_connections 2023-07-14T18:07:37.282 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:36.753+0000 7f2c05ec4700 1 -- 172.21.15.133:0/3399090105 wait complete. 2023-07-14T18:07:37.282 INFO:teuthology.orchestra.run.smithi133.stdout:Enabling the dashboard module... 2023-07-14T18:07:37.896 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:37 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3399090105' entity='client.admin' 2023-07-14T18:07:37.896 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:37 smithi133 ceph-mon[126384]: from='mgr.14120 172.21.15.133:0/1173332444' entity='mgr.x' 2023-07-14T18:07:39.109 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:38 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1582719483' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-07-14T18:07:39.109 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:38 smithi133 ceph-mon[126384]: from='mgr.14120 172.21.15.133:0/1173332444' entity='mgr.x' 2023-07-14T18:07:39.110 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:38 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: ignoring --setuser ceph since I am not root 2023-07-14T18:07:39.110 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:38 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: ignoring --setgroup ceph since I am not root 2023-07-14T18:07:39.110 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:39 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:39.074+0000 7fa6c6e05200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T18:07:39.264 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.802+0000 7f6d0c0c0700 1 Processor -- start 2023-07-14T18:07:39.264 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.802+0000 7f6d0c0c0700 1 -- start start 2023-07-14T18:07:39.265 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.802+0000 7f6d0c0c0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d0407be10 0x7f6d0407a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:39.265 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.802+0000 7f6d0c0c0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0407a850 con 0x7f6d0407be10 2023-07-14T18:07:39.265 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.803+0000 7f6d09e5c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d0407be10 0x7f6d0407a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:39.265 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.803+0000 7f6d09e5c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d0407be10 0x7f6d0407a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47652/0 (socket says 172.21.15.133:47652) 2023-07-14T18:07:39.265 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.803+0000 7f6d09e5c700 1 -- 172.21.15.133:0/547220920 learned_addr learned my addr 172.21.15.133:0/547220920 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:39.265 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.803+0000 7f6d09e5c700 1 -- 172.21.15.133:0/547220920 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d0407a990 con 0x7f6d0407be10 2023-07-14T18:07:39.266 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.803+0000 7f6d09e5c700 1 --2- 172.21.15.133:0/547220920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d0407be10 0x7f6d0407a310 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6d000098a0 tx=0x7f6d00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f516519c443a7a46 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:39.266 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.804+0000 7f6d08e5a700 1 -- 172.21.15.133:0/547220920 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d00004160 con 0x7f6d0407be10 2023-07-14T18:07:39.266 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.804+0000 7f6d08e5a700 1 -- 172.21.15.133:0/547220920 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d000042c0 con 0x7f6d0407be10 2023-07-14T18:07:39.266 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.804+0000 7f6d08e5a700 1 -- 172.21.15.133:0/547220920 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d000119b0 con 0x7f6d0407be10 2023-07-14T18:07:39.266 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.804+0000 7f6d0c0c0700 1 -- 172.21.15.133:0/547220920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d0407be10 msgr2=0x7f6d0407a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:39.266 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.804+0000 7f6d0c0c0700 1 --2- 172.21.15.133:0/547220920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d0407be10 0x7f6d0407a310 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6d000098a0 tx=0x7f6d00009bb0 comp rx=0 tx=0).stop 2023-07-14T18:07:39.266 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.805+0000 7f6d0c0c0700 1 -- 172.21.15.133:0/547220920 shutdown_connections 2023-07-14T18:07:39.267 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.805+0000 7f6d0c0c0700 1 --2- 172.21.15.133:0/547220920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d0407be10 0x7f6d0407a310 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:39.267 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.805+0000 7f6d0c0c0700 1 -- 172.21.15.133:0/547220920 >> 172.21.15.133:0/547220920 conn(0x7f6d04100ab0 msgr2=0x7f6d04102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:39.267 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.805+0000 7f6d0c0c0700 1 -- 172.21.15.133:0/547220920 shutdown_connections 2023-07-14T18:07:39.267 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.805+0000 7f6d0c0c0700 1 -- 172.21.15.133:0/547220920 wait complete. 2023-07-14T18:07:39.267 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.806+0000 7f6d0c0c0700 1 Processor -- start 2023-07-14T18:07:39.268 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.806+0000 7f6d0c0c0700 1 -- start start 2023-07-14T18:07:39.268 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.806+0000 7f6d0c0c0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d0407be10 0x7f6d0419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:39.268 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.806+0000 7f6d0c0c0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0419b8e0 con 0x7f6d0407be10 2023-07-14T18:07:39.268 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.806+0000 7f6d09e5c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d0407be10 0x7f6d0419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:39.268 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.806+0000 7f6d09e5c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d0407be10 0x7f6d0419b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47664/0 (socket says 172.21.15.133:47664) 2023-07-14T18:07:39.269 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.806+0000 7f6d09e5c700 1 -- 172.21.15.133:0/1582719483 learned_addr learned my addr 172.21.15.133:0/1582719483 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:39.269 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.807+0000 7f6d09e5c700 1 -- 172.21.15.133:0/1582719483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d00009550 con 0x7f6d0407be10 2023-07-14T18:07:39.269 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.807+0000 7f6d09e5c700 1 --2- 172.21.15.133:0/1582719483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d0407be10 0x7f6d0419b3a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6d00003e60 tx=0x7f6d00011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:39.269 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.807+0000 7f6cfaffd700 1 -- 172.21.15.133:0/1582719483 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d00003b20 con 0x7f6d0407be10 2023-07-14T18:07:39.269 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.807+0000 7f6cfaffd700 1 -- 172.21.15.133:0/1582719483 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d00003c80 con 0x7f6d0407be10 2023-07-14T18:07:39.269 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.807+0000 7f6cfaffd700 1 -- 172.21.15.133:0/1582719483 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d00024760 con 0x7f6d0407be10 2023-07-14T18:07:39.270 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.807+0000 7f6d0c0c0700 1 -- 172.21.15.133:0/1582719483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d0419bae0 con 0x7f6d0407be10 2023-07-14T18:07:39.270 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.807+0000 7f6d0c0c0700 1 -- 172.21.15.133:0/1582719483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d0419bf00 con 0x7f6d0407be10 2023-07-14T18:07:39.270 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.808+0000 7f6d0c0c0700 1 -- 172.21.15.133:0/1582719483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d04061c70 con 0x7f6d0407be10 2023-07-14T18:07:39.270 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.809+0000 7f6cfaffd700 1 -- 172.21.15.133:0/1582719483 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f6d0001ab10 con 0x7f6d0407be10 2023-07-14T18:07:39.270 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.809+0000 7f6cfaffd700 1 --2- 172.21.15.133:0/1582719483 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f6cf0038760 0x7f6cf003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:39.270 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.809+0000 7f6cfaffd700 1 -- 172.21.15.133:0/1582719483 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6d0004ca00 con 0x7f6d0407be10 2023-07-14T18:07:39.270 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.810+0000 7f6d0965b700 1 --2- 172.21.15.133:0/1582719483 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f6cf0038760 0x7f6cf003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:39.271 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.810+0000 7f6d0965b700 1 --2- 172.21.15.133:0/1582719483 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f6cf0038760 0x7f6cf003ac10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4006fd0 tx=0x7f6cf4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:39.271 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:37.814+0000 7f6cfaffd700 1 -- 172.21.15.133:0/1582719483 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f6d00018920 con 0x7f6d0407be10 2023-07-14T18:07:39.271 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:38.098+0000 7f6d0c0c0700 1 -- 172.21.15.133:0/1582719483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f6d0419eff0 con 0x7f6d0407be10 2023-07-14T18:07:39.271 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:38.759+0000 7f6cfaffd700 1 -- 172.21.15.133:0/1582719483 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f6d0002d770 con 0x7f6d0407be10 2023-07-14T18:07:39.271 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:38.761+0000 7f6cfaffd700 1 -- 172.21.15.133:0/1582719483 <== mon.0 v2:172.21.15.133:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f6d0002c9b0 con 0x7f6d0407be10 2023-07-14T18:07:39.271 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:38.764+0000 7f6d0c0c0700 1 -- 172.21.15.133:0/1582719483 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f6cf0038760 msgr2=0x7f6cf003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:39.272 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:38.764+0000 7f6d0c0c0700 1 --2- 172.21.15.133:0/1582719483 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f6cf0038760 0x7f6cf003ac10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4006fd0 tx=0x7f6cf4006e40 comp rx=0 tx=0).stop 2023-07-14T18:07:39.272 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:38.764+0000 7f6d0c0c0700 1 -- 172.21.15.133:0/1582719483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d0407be10 msgr2=0x7f6d0419b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:39.272 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:38.764+0000 7f6d0c0c0700 1 --2- 172.21.15.133:0/1582719483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d0407be10 0x7f6d0419b3a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6d00003e60 tx=0x7f6d00011a30 comp rx=0 tx=0).stop 2023-07-14T18:07:39.272 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:38.764+0000 7f6d0c0c0700 1 -- 172.21.15.133:0/1582719483 shutdown_connections 2023-07-14T18:07:39.272 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:38.764+0000 7f6d0c0c0700 1 --2- 172.21.15.133:0/1582719483 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f6cf0038760 0x7f6cf003ac10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:39.272 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:38.764+0000 7f6d0c0c0700 1 --2- 172.21.15.133:0/1582719483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d0407be10 0x7f6d0419b3a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:39.273 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:38.764+0000 7f6d0c0c0700 1 -- 172.21.15.133:0/1582719483 >> 172.21.15.133:0/1582719483 conn(0x7f6d04100ab0 msgr2=0x7f6d04101840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:39.273 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:38.765+0000 7f6d0c0c0700 1 -- 172.21.15.133:0/1582719483 shutdown_connections 2023-07-14T18:07:39.273 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:38.765+0000 7f6d0c0c0700 1 -- 172.21.15.133:0/1582719483 wait complete. 2023-07-14T18:07:39.396 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:39 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:39.180+0000 7fa6c6e05200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T18:07:39.396 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:39 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:39.318+0000 7fa6c6e05200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T18:07:39.742 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:39 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:39.449+0000 7fa6c6e05200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T18:07:40.010 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:39 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1582719483' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-07-14T18:07:40.011 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:39 smithi133 ceph-mon[126384]: mgrmap e9: x(active, since 19s) 2023-07-14T18:07:40.011 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:39 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:39.707+0000 7fa6c6e05200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T18:07:40.303 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:40 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:40.138+0000 7fa6c6e05200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T18:07:40.303 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:40 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:40.268+0000 7fa6c6e05200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T18:07:40.554 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:40 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:40.435+0000 7fa6c6e05200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T18:07:40.896 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:40 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2006748758' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-14T18:07:40.896 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:40 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:40.582+0000 7fa6c6e05200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T18:07:40.951 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2023-07-14T18:07:40.952 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-07-14T18:07:40.952 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-14T18:07:40.952 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-14T18:07:40.952 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-14T18:07:40.952 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2023-07-14T18:07:40.952 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.769+0000 7f3973536700 1 Processor -- start 2023-07-14T18:07:40.952 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.769+0000 7f3973536700 1 -- start start 2023-07-14T18:07:40.953 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.770+0000 7f3973536700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f396c07bdf0 0x7f396c07a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:40.953 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.770+0000 7f3973536700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396c07a830 con 0x7f396c07bdf0 2023-07-14T18:07:40.953 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.770+0000 7f39712d2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f396c07bdf0 0x7f396c07a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:40.953 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.770+0000 7f39712d2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f396c07bdf0 0x7f396c07a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47698/0 (socket says 172.21.15.133:47698) 2023-07-14T18:07:40.953 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.770+0000 7f39712d2700 1 -- 172.21.15.133:0/3523751002 learned_addr learned my addr 172.21.15.133:0/3523751002 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:40.953 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.771+0000 7f39712d2700 1 -- 172.21.15.133:0/3523751002 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f396c07a970 con 0x7f396c07bdf0 2023-07-14T18:07:40.953 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.771+0000 7f39712d2700 1 --2- 172.21.15.133:0/3523751002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f396c07bdf0 0x7f396c07a2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f39680098a0 tx=0x7f3968009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39468e4ea9384de3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:40.954 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.771+0000 7f3963fff700 1 -- 172.21.15.133:0/3523751002 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3968004160 con 0x7f396c07bdf0 2023-07-14T18:07:40.954 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.771+0000 7f3963fff700 1 -- 172.21.15.133:0/3523751002 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39680042c0 con 0x7f396c07bdf0 2023-07-14T18:07:40.954 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.771+0000 7f3963fff700 1 -- 172.21.15.133:0/3523751002 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39680119b0 con 0x7f396c07bdf0 2023-07-14T18:07:40.954 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.772+0000 7f3973536700 1 -- 172.21.15.133:0/3523751002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f396c07bdf0 msgr2=0x7f396c07a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:40.954 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.772+0000 7f3973536700 1 --2- 172.21.15.133:0/3523751002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f396c07bdf0 0x7f396c07a2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f39680098a0 tx=0x7f3968009bb0 comp rx=0 tx=0).stop 2023-07-14T18:07:40.954 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.772+0000 7f3973536700 1 -- 172.21.15.133:0/3523751002 shutdown_connections 2023-07-14T18:07:40.954 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.772+0000 7f3973536700 1 --2- 172.21.15.133:0/3523751002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f396c07bdf0 0x7f396c07a2f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:40.954 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.772+0000 7f3973536700 1 -- 172.21.15.133:0/3523751002 >> 172.21.15.133:0/3523751002 conn(0x7f396c100ab0 msgr2=0x7f396c102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:40.955 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.772+0000 7f3973536700 1 -- 172.21.15.133:0/3523751002 shutdown_connections 2023-07-14T18:07:40.955 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.772+0000 7f3973536700 1 -- 172.21.15.133:0/3523751002 wait complete. 2023-07-14T18:07:40.955 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.773+0000 7f3973536700 1 Processor -- start 2023-07-14T18:07:40.955 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.773+0000 7f3973536700 1 -- start start 2023-07-14T18:07:40.955 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.774+0000 7f3973536700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f396c07bdf0 0x7f396c19b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:40.955 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.774+0000 7f3973536700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396c19b8a0 con 0x7f396c07bdf0 2023-07-14T18:07:40.955 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.774+0000 7f39712d2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f396c07bdf0 0x7f396c19b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:40.956 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.774+0000 7f39712d2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f396c07bdf0 0x7f396c19b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47702/0 (socket says 172.21.15.133:47702) 2023-07-14T18:07:40.956 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.774+0000 7f39712d2700 1 -- 172.21.15.133:0/2006748758 learned_addr learned my addr 172.21.15.133:0/2006748758 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:40.956 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.774+0000 7f39712d2700 1 -- 172.21.15.133:0/2006748758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3968009550 con 0x7f396c07bdf0 2023-07-14T18:07:40.956 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.775+0000 7f39712d2700 1 --2- 172.21.15.133:0/2006748758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f396c07bdf0 0x7f396c19b360 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3968000c00 tx=0x7f3968011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:40.956 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.775+0000 7f39627fc700 1 -- 172.21.15.133:0/2006748758 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39680039f0 con 0x7f396c07bdf0 2023-07-14T18:07:40.956 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.775+0000 7f39627fc700 1 -- 172.21.15.133:0/2006748758 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3968003b50 con 0x7f396c07bdf0 2023-07-14T18:07:40.956 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.775+0000 7f39627fc700 1 -- 172.21.15.133:0/2006748758 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f396801a780 con 0x7f396c07bdf0 2023-07-14T18:07:40.956 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.775+0000 7f3973536700 1 -- 172.21.15.133:0/2006748758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f396c19baa0 con 0x7f396c07bdf0 2023-07-14T18:07:40.957 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.775+0000 7f3973536700 1 -- 172.21.15.133:0/2006748758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f396c19bec0 con 0x7f396c07bdf0 2023-07-14T18:07:40.957 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.776+0000 7f3973536700 1 -- 172.21.15.133:0/2006748758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f396c10b7f0 con 0x7f396c07bdf0 2023-07-14T18:07:40.957 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.776+0000 7f39627fc700 1 -- 172.21.15.133:0/2006748758 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f3968024440 con 0x7f396c07bdf0 2023-07-14T18:07:40.957 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.777+0000 7f39627fc700 1 --2- 172.21.15.133:0/2006748758 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f39580387b0 0x7f395803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:40.957 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.777+0000 7f39627fc700 1 -- 172.21.15.133:0/2006748758 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f396804c810 con 0x7f396c07bdf0 2023-07-14T18:07:40.957 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.777+0000 7f3970ad1700 1 -- 172.21.15.133:0/2006748758 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f39580387b0 msgr2=0x7f395803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/2986586981 2023-07-14T18:07:40.957 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.777+0000 7f3970ad1700 1 --2- 172.21.15.133:0/2006748758 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f39580387b0 0x7f395803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T18:07:40.958 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.784+0000 7f39627fc700 1 -- 172.21.15.133:0/2006748758 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f39680189a0 con 0x7f396c07bdf0 2023-07-14T18:07:40.958 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.978+0000 7f3970ad1700 1 -- 172.21.15.133:0/2006748758 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f39580387b0 msgr2=0x7f395803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/2986586981 2023-07-14T18:07:40.958 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:39.978+0000 7f3970ad1700 1 --2- 172.21.15.133:0/2006748758 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f39580387b0 0x7f395803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-14T18:07:40.958 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:40.073+0000 7f3973536700 1 -- 172.21.15.133:0/2006748758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f396c10ba10 con 0x7f396c07bdf0 2023-07-14T18:07:40.958 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:40.073+0000 7f39627fc700 1 -- 172.21.15.133:0/2006748758 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f3968018350 con 0x7f396c07bdf0 2023-07-14T18:07:40.958 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:40.076+0000 7f3973536700 1 -- 172.21.15.133:0/2006748758 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f39580387b0 msgr2=0x7f395803ac60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T18:07:40.958 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:40.076+0000 7f3973536700 1 --2- 172.21.15.133:0/2006748758 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f39580387b0 0x7f395803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:40.959 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:40.076+0000 7f3973536700 1 -- 172.21.15.133:0/2006748758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f396c07bdf0 msgr2=0x7f396c19b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:40.959 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:40.076+0000 7f3973536700 1 --2- 172.21.15.133:0/2006748758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f396c07bdf0 0x7f396c19b360 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3968000c00 tx=0x7f3968011a30 comp rx=0 tx=0).stop 2023-07-14T18:07:40.959 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:40.076+0000 7f3973536700 1 -- 172.21.15.133:0/2006748758 shutdown_connections 2023-07-14T18:07:40.959 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:40.076+0000 7f3973536700 1 --2- 172.21.15.133:0/2006748758 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f39580387b0 0x7f395803ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:40.959 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:40.076+0000 7f3973536700 1 --2- 172.21.15.133:0/2006748758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f396c07bdf0 0x7f396c19b360 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:40.959 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:40.076+0000 7f3973536700 1 -- 172.21.15.133:0/2006748758 >> 172.21.15.133:0/2006748758 conn(0x7f396c100ab0 msgr2=0x7f396c101830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:40.960 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:40.076+0000 7f3973536700 1 -- 172.21.15.133:0/2006748758 shutdown_connections 2023-07-14T18:07:40.960 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:40.076+0000 7f3973536700 1 -- 172.21.15.133:0/2006748758 wait complete. 2023-07-14T18:07:40.960 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for the mgr to restart... 2023-07-14T18:07:40.960 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for mgr epoch 9... 2023-07-14T18:07:41.229 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:41 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:41.147+0000 7fa6c6e05200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T18:07:42.396 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:41 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:41.908+0000 7fa6c6e05200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T18:07:42.396 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:42 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:42.115+0000 7fa6c6e05200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T18:07:43.396 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:43 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:43.044+0000 7fa6c6e05200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T18:07:43.396 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:43 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:43.162+0000 7fa6c6e05200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T18:07:43.771 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:43 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:43.382+0000 7fa6c6e05200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T18:07:44.049 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:43 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:43.736+0000 7fa6c6e05200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T18:07:44.396 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:44 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:44.014+0000 7fa6c6e05200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T18:07:44.396 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:44 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:44.204+0000 7fa6c6e05200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T18:07:44.896 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:44 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:44.541+0000 7fa6c6e05200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T18:07:45.896 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:45 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:45.456+0000 7fa6c6e05200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T18:07:45.896 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:45 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:45.627+0000 7fa6c6e05200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T18:07:46.321 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:45 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:45.869+0000 7fa6c6e05200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T18:07:46.321 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:46 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:46.014+0000 7fa6c6e05200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T18:07:46.646 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:46 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:46.286+0000 7fa6c6e05200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T18:07:47.646 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:07:47 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:07:47.343+0000 7fa6c6e05200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T18:07:47.646 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:47 smithi133 ceph-mon[126384]: Active manager daemon x restarted 2023-07-14T18:07:47.647 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:47 smithi133 ceph-mon[126384]: Activating manager daemon x 2023-07-14T18:07:47.647 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:47 smithi133 ceph-mon[126384]: osdmap e3: 0 total, 0 up, 0 in 2023-07-14T18:07:47.647 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:47 smithi133 ceph-mon[126384]: mgrmap e10: x(active, starting, since 0.00648637s) 2023-07-14T18:07:47.647 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:47 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T18:07:47.647 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:47 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-14T18:07:47.647 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:47 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-14T18:07:47.648 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:47 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-14T18:07:47.648 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:47 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-14T18:07:47.648 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:47 smithi133 ceph-mon[126384]: Manager daemon x is now available 2023-07-14T18:07:47.648 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:47 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:07:47.648 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:47 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:07:48.842 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2023-07-14T18:07:48.851 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-07-14T18:07:48.851 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-14T18:07:48.851 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2023-07-14T18:07:48.851 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.459+0000 7f287c2ea700 1 Processor -- start 2023-07-14T18:07:48.851 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.460+0000 7f287c2ea700 1 -- start start 2023-07-14T18:07:48.852 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.460+0000 7f287c2ea700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f287407bdf0 0x7f287407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:48.852 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.460+0000 7f287c2ea700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287407a830 con 0x7f287407bdf0 2023-07-14T18:07:48.852 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.460+0000 7f287a086700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f287407bdf0 0x7f287407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:48.852 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.460+0000 7f287a086700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f287407bdf0 0x7f287407a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47708/0 (socket says 172.21.15.133:47708) 2023-07-14T18:07:48.852 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.460+0000 7f287a086700 1 -- 172.21.15.133:0/2379496308 learned_addr learned my addr 172.21.15.133:0/2379496308 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:48.852 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.461+0000 7f287a086700 1 -- 172.21.15.133:0/2379496308 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287407a970 con 0x7f287407bdf0 2023-07-14T18:07:48.852 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.461+0000 7f287a086700 1 --2- 172.21.15.133:0/2379496308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f287407bdf0 0x7f287407a2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f28700098a0 tx=0x7f2870009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15077a69e7cdf7dd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:48.853 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.461+0000 7f2879084700 1 -- 172.21.15.133:0/2379496308 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2870004160 con 0x7f287407bdf0 2023-07-14T18:07:48.853 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.461+0000 7f2879084700 1 -- 172.21.15.133:0/2379496308 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28700042c0 con 0x7f287407bdf0 2023-07-14T18:07:48.853 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.461+0000 7f2879084700 1 -- 172.21.15.133:0/2379496308 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28700119b0 con 0x7f287407bdf0 2023-07-14T18:07:48.853 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.462+0000 7f287c2ea700 1 -- 172.21.15.133:0/2379496308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f287407bdf0 msgr2=0x7f287407a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:48.853 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.462+0000 7f287c2ea700 1 --2- 172.21.15.133:0/2379496308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f287407bdf0 0x7f287407a2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f28700098a0 tx=0x7f2870009bb0 comp rx=0 tx=0).stop 2023-07-14T18:07:48.853 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.462+0000 7f287c2ea700 1 -- 172.21.15.133:0/2379496308 shutdown_connections 2023-07-14T18:07:48.853 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.462+0000 7f287c2ea700 1 --2- 172.21.15.133:0/2379496308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f287407bdf0 0x7f287407a2f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:48.854 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.462+0000 7f287c2ea700 1 -- 172.21.15.133:0/2379496308 >> 172.21.15.133:0/2379496308 conn(0x7f2874100ab0 msgr2=0x7f2874102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:48.854 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.462+0000 7f287c2ea700 1 -- 172.21.15.133:0/2379496308 shutdown_connections 2023-07-14T18:07:48.854 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.462+0000 7f287c2ea700 1 -- 172.21.15.133:0/2379496308 wait complete. 2023-07-14T18:07:48.854 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.463+0000 7f287c2ea700 1 Processor -- start 2023-07-14T18:07:48.854 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.464+0000 7f287c2ea700 1 -- start start 2023-07-14T18:07:48.854 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.464+0000 7f287c2ea700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f287407bdf0 0x7f287419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:48.854 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.464+0000 7f287c2ea700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287419b8a0 con 0x7f287407bdf0 2023-07-14T18:07:48.854 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.464+0000 7f287a086700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f287407bdf0 0x7f287419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:48.855 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.464+0000 7f287a086700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f287407bdf0 0x7f287419b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47716/0 (socket says 172.21.15.133:47716) 2023-07-14T18:07:48.855 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.464+0000 7f287a086700 1 -- 172.21.15.133:0/2119297532 learned_addr learned my addr 172.21.15.133:0/2119297532 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:48.855 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.465+0000 7f287a086700 1 -- 172.21.15.133:0/2119297532 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2870009550 con 0x7f287407bdf0 2023-07-14T18:07:48.855 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.465+0000 7f287a086700 1 --2- 172.21.15.133:0/2119297532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f287407bdf0 0x7f287419b360 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2870000c00 tx=0x7f2870011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:48.855 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.465+0000 7f286b7fe700 1 -- 172.21.15.133:0/2119297532 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28700039f0 con 0x7f287407bdf0 2023-07-14T18:07:48.855 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.465+0000 7f286b7fe700 1 -- 172.21.15.133:0/2119297532 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2870003b50 con 0x7f287407bdf0 2023-07-14T18:07:48.855 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.465+0000 7f287c2ea700 1 -- 172.21.15.133:0/2119297532 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f287419baa0 con 0x7f287407bdf0 2023-07-14T18:07:48.855 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.465+0000 7f286b7fe700 1 -- 172.21.15.133:0/2119297532 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f287001a780 con 0x7f287407bdf0 2023-07-14T18:07:48.856 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.465+0000 7f287c2ea700 1 -- 172.21.15.133:0/2119297532 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287419bec0 con 0x7f287407bdf0 2023-07-14T18:07:48.856 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.466+0000 7f286b7fe700 1 -- 172.21.15.133:0/2119297532 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f287001a9f0 con 0x7f287407bdf0 2023-07-14T18:07:48.856 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.467+0000 7f286b7fe700 1 --2- 172.21.15.133:0/2119297532 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f2860038800 0x7f286003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:48.856 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.467+0000 7f287c2ea700 1 -- 172.21.15.133:0/2119297532 --> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2874061c70 con 0x7f2860038800 2023-07-14T18:07:48.856 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.467+0000 7f286b7fe700 1 -- 172.21.15.133:0/2119297532 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f287004ca10 con 0x7f287407bdf0 2023-07-14T18:07:48.857 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.467+0000 7f2879885700 1 -- 172.21.15.133:0/2119297532 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f2860038800 msgr2=0x7f286003acb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/2986586981 2023-07-14T18:07:48.857 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.467+0000 7f2879885700 1 --2- 172.21.15.133:0/2119297532 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f2860038800 0x7f286003acb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T18:07:48.857 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.668+0000 7f2879885700 1 -- 172.21.15.133:0/2119297532 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f2860038800 msgr2=0x7f286003acb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/2986586981 2023-07-14T18:07:48.857 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:41.668+0000 7f2879885700 1 --2- 172.21.15.133:0/2119297532 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f2860038800 0x7f286003acb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-14T18:07:48.857 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:42.069+0000 7f2879885700 1 -- 172.21.15.133:0/2119297532 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f2860038800 msgr2=0x7f286003acb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/2986586981 2023-07-14T18:07:48.857 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:42.069+0000 7f2879885700 1 --2- 172.21.15.133:0/2119297532 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f2860038800 0x7f286003acb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-14T18:07:48.858 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:42.870+0000 7f2879885700 1 -- 172.21.15.133:0/2119297532 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f2860038800 msgr2=0x7f286003acb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/2986586981 2023-07-14T18:07:48.858 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:42.870+0000 7f2879885700 1 --2- 172.21.15.133:0/2119297532 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f2860038800 0x7f286003acb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-14T18:07:48.858 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:44.472+0000 7f2879885700 1 -- 172.21.15.133:0/2119297532 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f2860038800 msgr2=0x7f286003acb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/2986586981 2023-07-14T18:07:48.858 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:44.472+0000 7f2879885700 1 --2- 172.21.15.133:0/2119297532 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f2860038800 0x7f286003acb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-14T18:07:48.858 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:47.350+0000 7f286b7fe700 1 -- 172.21.15.133:0/2119297532 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mgrmap(e 10) v1 ==== 45263+0+0 (secure 0 0 0) 0x7f287002dc90 con 0x7f287407bdf0 2023-07-14T18:07:48.858 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:47.350+0000 7f286b7fe700 1 -- 172.21.15.133:0/2119297532 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f2860038800 msgr2=0x7f286003acb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T18:07:48.858 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:47.350+0000 7f286b7fe700 1 --2- 172.21.15.133:0/2119297532 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f2860038800 0x7f286003acb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:48.858 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:48.351+0000 7f286b7fe700 1 -- 172.21.15.133:0/2119297532 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7f287004e050 con 0x7f287407bdf0 2023-07-14T18:07:48.859 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:48.351+0000 7f286b7fe700 1 --2- 172.21.15.133:0/2119297532 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f286003bfc0 0x7f286003e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:48.859 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:48.351+0000 7f286b7fe700 1 -- 172.21.15.133:0/2119297532 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2874061c70 con 0x7f286003bfc0 2023-07-14T18:07:48.859 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:48.351+0000 7f2879885700 1 --2- 172.21.15.133:0/2119297532 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f286003bfc0 0x7f286003e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:48.859 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:48.352+0000 7f2879885700 1 --2- 172.21.15.133:0/2119297532 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f286003bfc0 0x7f286003e3a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2864003a10 tx=0x7f28640092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:48.859 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:48.353+0000 7f286b7fe700 1 -- 172.21.15.133:0/2119297532 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f2874061c70 con 0x7f286003bfc0 2023-07-14T18:07:48.859 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:48.360+0000 7f287c2ea700 1 -- 172.21.15.133:0/2119297532 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f287419c6a0 con 0x7f286003bfc0 2023-07-14T18:07:48.859 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:48.361+0000 7f286b7fe700 1 -- 172.21.15.133:0/2119297532 <== mgr.14152 v2:172.21.15.133:6800/196197529 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f287419c6a0 con 0x7f286003bfc0 2023-07-14T18:07:48.859 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:48.361+0000 7f287c2ea700 1 -- 172.21.15.133:0/2119297532 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f286003bfc0 msgr2=0x7f286003e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:48.860 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:48.361+0000 7f287c2ea700 1 --2- 172.21.15.133:0/2119297532 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f286003bfc0 0x7f286003e3a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2864003a10 tx=0x7f28640092b0 comp rx=0 tx=0).stop 2023-07-14T18:07:48.860 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:48.361+0000 7f287c2ea700 1 -- 172.21.15.133:0/2119297532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f287407bdf0 msgr2=0x7f287419b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:48.860 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:48.361+0000 7f287c2ea700 1 --2- 172.21.15.133:0/2119297532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f287407bdf0 0x7f287419b360 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2870000c00 tx=0x7f2870011a30 comp rx=0 tx=0).stop 2023-07-14T18:07:48.860 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:48.361+0000 7f287c2ea700 1 -- 172.21.15.133:0/2119297532 shutdown_connections 2023-07-14T18:07:48.860 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:48.361+0000 7f287c2ea700 1 --2- 172.21.15.133:0/2119297532 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f286003bfc0 0x7f286003e3a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:48.860 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:48.361+0000 7f287c2ea700 1 --2- 172.21.15.133:0/2119297532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f287407bdf0 0x7f287419b360 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:48.860 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:48.361+0000 7f287c2ea700 1 --2- 172.21.15.133:0/2119297532 >> [v2:172.21.15.133:6800/2986586981,v1:172.21.15.133:6801/2986586981] conn(0x7f2860038800 0x7f286003acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:48.861 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:48.361+0000 7f287c2ea700 1 -- 172.21.15.133:0/2119297532 >> 172.21.15.133:0/2119297532 conn(0x7f2874100ab0 msgr2=0x7f2874101830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:48.861 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:48.361+0000 7f287c2ea700 1 -- 172.21.15.133:0/2119297532 shutdown_connections 2023-07-14T18:07:48.861 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:48.361+0000 7f287c2ea700 1 -- 172.21.15.133:0/2119297532 wait complete. 2023-07-14T18:07:48.861 INFO:teuthology.orchestra.run.smithi133.stdout:mgr epoch 9 is available 2023-07-14T18:07:48.861 INFO:teuthology.orchestra.run.smithi133.stdout:Generating a dashboard self-signed certificate... 2023-07-14T18:07:48.865 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-14T18:07:48.865 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-14T18:07:48.865 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:48 smithi133 ceph-mon[126384]: mgrmap e11: x(active, since 1.0079s) 2023-07-14T18:07:49.731 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:49 smithi133 ceph-mon[126384]: [14/Jul/2023:18:07:48] ENGINE Bus STARTING 2023-07-14T18:07:49.731 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:49 smithi133 ceph-mon[126384]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-07-14T18:07:49.731 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:49 smithi133 ceph-mon[126384]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-07-14T18:07:49.731 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:49 smithi133 ceph-mon[126384]: [14/Jul/2023:18:07:48] ENGINE Serving on https://172.21.15.133:7150 2023-07-14T18:07:49.731 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:49 smithi133 ceph-mon[126384]: [14/Jul/2023:18:07:48] ENGINE Serving on http://172.21.15.133:8765 2023-07-14T18:07:49.732 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:49 smithi133 ceph-mon[126384]: [14/Jul/2023:18:07:48] ENGINE Bus STARTED 2023-07-14T18:07:50.238 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-07-14T18:07:50.238 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.370+0000 7fd460a28700 1 Processor -- start 2023-07-14T18:07:50.238 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.370+0000 7fd460a28700 1 -- start start 2023-07-14T18:07:50.238 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.371+0000 7fd460a28700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd45c07be70 0x7fd45c07a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:50.239 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.371+0000 7fd460a28700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c07a8b0 con 0x7fd45c07be70 2023-07-14T18:07:50.239 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.372+0000 7fd45a59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd45c07be70 0x7fd45c07a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:50.239 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.372+0000 7fd45a59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd45c07be70 0x7fd45c07a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59420/0 (socket says 172.21.15.133:59420) 2023-07-14T18:07:50.239 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.372+0000 7fd45a59c700 1 -- 172.21.15.133:0/1548738157 learned_addr learned my addr 172.21.15.133:0/1548738157 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:50.239 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.372+0000 7fd45a59c700 1 -- 172.21.15.133:0/1548738157 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd45c07a9f0 con 0x7fd45c07be70 2023-07-14T18:07:50.239 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.372+0000 7fd45a59c700 1 --2- 172.21.15.133:0/1548738157 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd45c07be70 0x7fd45c07a370 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd4540098a0 tx=0x7fd454009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4039dd1d2ce109dc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:50.240 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.373+0000 7fd45959a700 1 -- 172.21.15.133:0/1548738157 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd454004160 con 0x7fd45c07be70 2023-07-14T18:07:50.240 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.373+0000 7fd45959a700 1 -- 172.21.15.133:0/1548738157 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4540042c0 con 0x7fd45c07be70 2023-07-14T18:07:50.240 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.373+0000 7fd45959a700 1 -- 172.21.15.133:0/1548738157 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4540119b0 con 0x7fd45c07be70 2023-07-14T18:07:50.240 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.374+0000 7fd460a28700 1 -- 172.21.15.133:0/1548738157 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd45c07be70 msgr2=0x7fd45c07a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:50.240 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.374+0000 7fd460a28700 1 --2- 172.21.15.133:0/1548738157 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd45c07be70 0x7fd45c07a370 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd4540098a0 tx=0x7fd454009bb0 comp rx=0 tx=0).stop 2023-07-14T18:07:50.240 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.374+0000 7fd460a28700 1 -- 172.21.15.133:0/1548738157 shutdown_connections 2023-07-14T18:07:50.240 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.374+0000 7fd460a28700 1 --2- 172.21.15.133:0/1548738157 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd45c07be70 0x7fd45c07a370 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:50.241 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.374+0000 7fd460a28700 1 -- 172.21.15.133:0/1548738157 >> 172.21.15.133:0/1548738157 conn(0x7fd45c100ab0 msgr2=0x7fd45c102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:50.241 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.374+0000 7fd460a28700 1 -- 172.21.15.133:0/1548738157 shutdown_connections 2023-07-14T18:07:50.241 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.374+0000 7fd460a28700 1 -- 172.21.15.133:0/1548738157 wait complete. 2023-07-14T18:07:50.241 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.375+0000 7fd460a28700 1 Processor -- start 2023-07-14T18:07:50.241 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.376+0000 7fd460a28700 1 -- start start 2023-07-14T18:07:50.241 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.376+0000 7fd460a28700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd45c07be70 0x7fd45c19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:50.241 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.376+0000 7fd460a28700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c19fcc0 con 0x7fd45c07be70 2023-07-14T18:07:50.241 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.376+0000 7fd45a59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd45c07be70 0x7fd45c19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:50.242 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.376+0000 7fd45a59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd45c07be70 0x7fd45c19f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59428/0 (socket says 172.21.15.133:59428) 2023-07-14T18:07:50.242 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.376+0000 7fd45a59c700 1 -- 172.21.15.133:0/1823022472 learned_addr learned my addr 172.21.15.133:0/1823022472 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:50.242 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.377+0000 7fd45a59c700 1 -- 172.21.15.133:0/1823022472 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd454009550 con 0x7fd45c07be70 2023-07-14T18:07:50.242 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.377+0000 7fd45a59c700 1 --2- 172.21.15.133:0/1823022472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd45c07be70 0x7fd45c19f780 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd454003e60 tx=0x7fd454011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:50.242 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.377+0000 7fd44b7fe700 1 -- 172.21.15.133:0/1823022472 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd454003970 con 0x7fd45c07be70 2023-07-14T18:07:50.242 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.377+0000 7fd44b7fe700 1 -- 172.21.15.133:0/1823022472 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd454003ad0 con 0x7fd45c07be70 2023-07-14T18:07:50.242 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.377+0000 7fd44b7fe700 1 -- 172.21.15.133:0/1823022472 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4540195e0 con 0x7fd45c07be70 2023-07-14T18:07:50.242 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.377+0000 7fd460a28700 1 -- 172.21.15.133:0/1823022472 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd45c19fec0 con 0x7fd45c07be70 2023-07-14T18:07:50.243 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.378+0000 7fd460a28700 1 -- 172.21.15.133:0/1823022472 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd45c1a02e0 con 0x7fd45c07be70 2023-07-14T18:07:50.243 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.379+0000 7fd44b7fe700 1 -- 172.21.15.133:0/1823022472 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7fd45401a430 con 0x7fd45c07be70 2023-07-14T18:07:50.243 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.379+0000 7fd460a28700 1 -- 172.21.15.133:0/1823022472 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd45c198f60 con 0x7fd45c07be70 2023-07-14T18:07:50.243 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.379+0000 7fd44b7fe700 1 --2- 172.21.15.133:0/1823022472 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd444038640 0x7fd44403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:50.243 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.379+0000 7fd44b7fe700 1 -- 172.21.15.133:0/1823022472 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd45404c7e0 con 0x7fd45c07be70 2023-07-14T18:07:50.243 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.379+0000 7fd459d9b700 1 --2- 172.21.15.133:0/1823022472 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd444038640 0x7fd44403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:50.243 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.380+0000 7fd459d9b700 1 --2- 172.21.15.133:0/1823022472 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd444038640 0x7fd44403aaf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd450006fd0 tx=0x7fd450006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:50.244 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.384+0000 7fd44b7fe700 1 -- 172.21.15.133:0/1823022472 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fd454023b40 con 0x7fd45c07be70 2023-07-14T18:07:50.244 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.403+0000 7fd44b7fe700 1 -- 172.21.15.133:0/1823022472 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fd45404ec70 con 0x7fd45c07be70 2023-07-14T18:07:50.244 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.636+0000 7fd460a28700 1 -- 172.21.15.133:0/1823022472 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fd45c060a80 con 0x7fd444038640 2023-07-14T18:07:50.244 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.672+0000 7fd44b7fe700 1 -- 172.21.15.133:0/1823022472 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fd45c060a80 con 0x7fd444038640 2023-07-14T18:07:50.244 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.675+0000 7fd460a28700 1 -- 172.21.15.133:0/1823022472 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd444038640 msgr2=0x7fd44403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:50.244 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.675+0000 7fd460a28700 1 --2- 172.21.15.133:0/1823022472 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd444038640 0x7fd44403aaf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd450006fd0 tx=0x7fd450006e40 comp rx=0 tx=0).stop 2023-07-14T18:07:50.244 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.675+0000 7fd460a28700 1 -- 172.21.15.133:0/1823022472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd45c07be70 msgr2=0x7fd45c19f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:50.245 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.675+0000 7fd460a28700 1 --2- 172.21.15.133:0/1823022472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd45c07be70 0x7fd45c19f780 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd454003e60 tx=0x7fd454011aa0 comp rx=0 tx=0).stop 2023-07-14T18:07:50.245 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.675+0000 7fd460a28700 1 -- 172.21.15.133:0/1823022472 shutdown_connections 2023-07-14T18:07:50.245 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.675+0000 7fd460a28700 1 --2- 172.21.15.133:0/1823022472 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd444038640 0x7fd44403aaf0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:50.245 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.675+0000 7fd460a28700 1 --2- 172.21.15.133:0/1823022472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd45c07be70 0x7fd45c19f780 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:50.245 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.675+0000 7fd460a28700 1 -- 172.21.15.133:0/1823022472 >> 172.21.15.133:0/1823022472 conn(0x7fd45c100ab0 msgr2=0x7fd45c102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:50.245 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.675+0000 7fd460a28700 1 -- 172.21.15.133:0/1823022472 shutdown_connections 2023-07-14T18:07:50.245 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:49.675+0000 7fd460a28700 1 -- 172.21.15.133:0/1823022472 wait complete. 2023-07-14T18:07:50.245 INFO:teuthology.orchestra.run.smithi133.stdout:Creating initial admin user... 2023-07-14T18:07:50.488 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:50 smithi133 ceph-mon[126384]: mgrmap e12: x(active, since 2s) 2023-07-14T18:07:50.488 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:50 smithi133 ceph-mon[126384]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:07:50.489 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:50 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:07:50.489 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:50 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:07:50.489 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:50 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:07:51.717 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$DEumJqSuzhpHo5S1o/1.8e5YIJjHlISBWQPNtZ/bw2Ij8zslCDT7a", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1689358071, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-07-14T18:07:51.717 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.757+0000 7efc6eec1700 1 Processor -- start 2023-07-14T18:07:51.717 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.757+0000 7efc6eec1700 1 -- start start 2023-07-14T18:07:51.717 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.757+0000 7efc6eec1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc6807bdf0 0x7efc6807a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:51.718 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.757+0000 7efc6eec1700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6807a830 con 0x7efc6807bdf0 2023-07-14T18:07:51.718 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.757+0000 7efc6cc5d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc6807bdf0 0x7efc6807a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:51.718 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.758+0000 7efc6cc5d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc6807bdf0 0x7efc6807a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59444/0 (socket says 172.21.15.133:59444) 2023-07-14T18:07:51.718 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.758+0000 7efc6cc5d700 1 -- 172.21.15.133:0/4244891708 learned_addr learned my addr 172.21.15.133:0/4244891708 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:51.718 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.758+0000 7efc6cc5d700 1 -- 172.21.15.133:0/4244891708 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc6807a970 con 0x7efc6807bdf0 2023-07-14T18:07:51.718 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.758+0000 7efc6cc5d700 1 --2- 172.21.15.133:0/4244891708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc6807bdf0 0x7efc6807a2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efc5c0098a0 tx=0x7efc5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bda666f4f8e0f8c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:51.718 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.758+0000 7efc677fe700 1 -- 172.21.15.133:0/4244891708 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc5c004160 con 0x7efc6807bdf0 2023-07-14T18:07:51.719 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.758+0000 7efc677fe700 1 -- 172.21.15.133:0/4244891708 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc5c0042c0 con 0x7efc6807bdf0 2023-07-14T18:07:51.719 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.758+0000 7efc677fe700 1 -- 172.21.15.133:0/4244891708 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc5c0119b0 con 0x7efc6807bdf0 2023-07-14T18:07:51.719 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.759+0000 7efc6eec1700 1 -- 172.21.15.133:0/4244891708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc6807bdf0 msgr2=0x7efc6807a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:51.719 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.759+0000 7efc6eec1700 1 --2- 172.21.15.133:0/4244891708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc6807bdf0 0x7efc6807a2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efc5c0098a0 tx=0x7efc5c009bb0 comp rx=0 tx=0).stop 2023-07-14T18:07:51.719 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.759+0000 7efc6eec1700 1 -- 172.21.15.133:0/4244891708 shutdown_connections 2023-07-14T18:07:51.719 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.759+0000 7efc6eec1700 1 --2- 172.21.15.133:0/4244891708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc6807bdf0 0x7efc6807a2f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:51.720 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.759+0000 7efc6eec1700 1 -- 172.21.15.133:0/4244891708 >> 172.21.15.133:0/4244891708 conn(0x7efc68100ab0 msgr2=0x7efc68102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:51.720 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.759+0000 7efc6eec1700 1 -- 172.21.15.133:0/4244891708 shutdown_connections 2023-07-14T18:07:51.720 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.759+0000 7efc6eec1700 1 -- 172.21.15.133:0/4244891708 wait complete. 2023-07-14T18:07:51.720 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.760+0000 7efc6eec1700 1 Processor -- start 2023-07-14T18:07:51.720 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.760+0000 7efc6eec1700 1 -- start start 2023-07-14T18:07:51.720 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.761+0000 7efc6eec1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc6807bdf0 0x7efc68108870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:51.720 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.761+0000 7efc6eec1700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc68108db0 con 0x7efc6807bdf0 2023-07-14T18:07:51.721 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.761+0000 7efc6cc5d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc6807bdf0 0x7efc68108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:51.721 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.761+0000 7efc6cc5d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc6807bdf0 0x7efc68108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59448/0 (socket says 172.21.15.133:59448) 2023-07-14T18:07:51.721 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.761+0000 7efc6cc5d700 1 -- 172.21.15.133:0/3568092034 learned_addr learned my addr 172.21.15.133:0/3568092034 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:51.721 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.761+0000 7efc6cc5d700 1 -- 172.21.15.133:0/3568092034 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5c009550 con 0x7efc6807bdf0 2023-07-14T18:07:51.721 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.762+0000 7efc6cc5d700 1 --2- 172.21.15.133:0/3568092034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc6807bdf0 0x7efc68108870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efc5c003e60 tx=0x7efc5c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:51.721 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.762+0000 7efc65ffb700 1 -- 172.21.15.133:0/3568092034 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc5c003970 con 0x7efc6807bdf0 2023-07-14T18:07:51.721 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.762+0000 7efc65ffb700 1 -- 172.21.15.133:0/3568092034 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc5c003ad0 con 0x7efc6807bdf0 2023-07-14T18:07:51.722 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.762+0000 7efc65ffb700 1 -- 172.21.15.133:0/3568092034 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc5c0195e0 con 0x7efc6807bdf0 2023-07-14T18:07:51.722 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.762+0000 7efc6eec1700 1 -- 172.21.15.133:0/3568092034 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc68108fb0 con 0x7efc6807bdf0 2023-07-14T18:07:51.722 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.762+0000 7efc6eec1700 1 -- 172.21.15.133:0/3568092034 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc681054a0 con 0x7efc6807bdf0 2023-07-14T18:07:51.722 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.763+0000 7efc65ffb700 1 -- 172.21.15.133:0/3568092034 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7efc5c0198a0 con 0x7efc6807bdf0 2023-07-14T18:07:51.722 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.763+0000 7efc6eec1700 1 -- 172.21.15.133:0/3568092034 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc68061c70 con 0x7efc6807bdf0 2023-07-14T18:07:51.722 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.764+0000 7efc65ffb700 1 --2- 172.21.15.133:0/3568092034 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efc58038710 0x7efc5803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:51.723 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.764+0000 7efc65ffb700 1 -- 172.21.15.133:0/3568092034 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efc5c04c9e0 con 0x7efc6807bdf0 2023-07-14T18:07:51.723 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.764+0000 7efc67fff700 1 --2- 172.21.15.133:0/3568092034 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efc58038710 0x7efc5803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:51.723 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.764+0000 7efc67fff700 1 --2- 172.21.15.133:0/3568092034 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efc58038710 0x7efc5803abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efc54006fd0 tx=0x7efc54006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:51.723 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.769+0000 7efc65ffb700 1 -- 172.21.15.133:0/3568092034 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7efc5c003c40 con 0x7efc6807bdf0 2023-07-14T18:07:51.723 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:50.988+0000 7efc6eec1700 1 -- 172.21.15.133:0/3568092034 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7efc68061a90 con 0x7efc58038710 2023-07-14T18:07:51.723 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:51.220+0000 7efc65ffb700 1 -- 172.21.15.133:0/3568092034 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7efc68061a90 con 0x7efc58038710 2023-07-14T18:07:51.723 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:51.223+0000 7efc6eec1700 1 -- 172.21.15.133:0/3568092034 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efc58038710 msgr2=0x7efc5803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:51.724 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:51.223+0000 7efc6eec1700 1 --2- 172.21.15.133:0/3568092034 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efc58038710 0x7efc5803abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efc54006fd0 tx=0x7efc54006e40 comp rx=0 tx=0).stop 2023-07-14T18:07:51.724 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:51.223+0000 7efc6eec1700 1 -- 172.21.15.133:0/3568092034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc6807bdf0 msgr2=0x7efc68108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:51.724 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:51.223+0000 7efc6eec1700 1 --2- 172.21.15.133:0/3568092034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc6807bdf0 0x7efc68108870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efc5c003e60 tx=0x7efc5c011aa0 comp rx=0 tx=0).stop 2023-07-14T18:07:51.724 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:51.223+0000 7efc6eec1700 1 -- 172.21.15.133:0/3568092034 shutdown_connections 2023-07-14T18:07:51.724 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:51.223+0000 7efc6eec1700 1 --2- 172.21.15.133:0/3568092034 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efc58038710 0x7efc5803abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:51.724 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:51.223+0000 7efc6eec1700 1 --2- 172.21.15.133:0/3568092034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc6807bdf0 0x7efc68108870 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:51.724 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:51.223+0000 7efc6eec1700 1 -- 172.21.15.133:0/3568092034 >> 172.21.15.133:0/3568092034 conn(0x7efc68100ab0 msgr2=0x7efc68101880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:51.725 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:51.223+0000 7efc6eec1700 1 -- 172.21.15.133:0/3568092034 shutdown_connections 2023-07-14T18:07:51.725 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:51.223+0000 7efc6eec1700 1 -- 172.21.15.133:0/3568092034 wait complete. 2023-07-14T18:07:51.725 INFO:teuthology.orchestra.run.smithi133.stdout:Fetching dashboard port number... 2023-07-14T18:07:52.396 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:52 smithi133 ceph-mon[126384]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:07:52.396 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:52 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:07:52.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:52 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:07:53.396 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 8443 2023-07-14T18:07:53.396 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.251+0000 7f8f89428700 1 Processor -- start 2023-07-14T18:07:53.397 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.251+0000 7f8f89428700 1 -- start start 2023-07-14T18:07:53.397 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.251+0000 7f8f89428700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f8407bdd0 0x7f8f8407a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:53.397 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.251+0000 7f8f89428700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8407a810 con 0x7f8f8407bdd0 2023-07-14T18:07:53.397 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.252+0000 7f8f82ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f8407bdd0 0x7f8f8407a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:53.397 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.252+0000 7f8f82ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f8407bdd0 0x7f8f8407a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59462/0 (socket says 172.21.15.133:59462) 2023-07-14T18:07:53.397 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.252+0000 7f8f82ffd700 1 -- 172.21.15.133:0/1475328242 learned_addr learned my addr 172.21.15.133:0/1475328242 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:53.397 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.252+0000 7f8f82ffd700 1 -- 172.21.15.133:0/1475328242 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f8407a950 con 0x7f8f8407bdd0 2023-07-14T18:07:53.398 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.252+0000 7f8f82ffd700 1 --2- 172.21.15.133:0/1475328242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f8407bdd0 0x7f8f8407a2d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8f740098a0 tx=0x7f8f74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7a79fd5dd4bc16 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:53.398 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.253+0000 7f8f81ffb700 1 -- 172.21.15.133:0/1475328242 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f74004160 con 0x7f8f8407bdd0 2023-07-14T18:07:53.398 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.253+0000 7f8f81ffb700 1 -- 172.21.15.133:0/1475328242 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f740042c0 con 0x7f8f8407bdd0 2023-07-14T18:07:53.398 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.253+0000 7f8f81ffb700 1 -- 172.21.15.133:0/1475328242 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f740119b0 con 0x7f8f8407bdd0 2023-07-14T18:07:53.398 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.253+0000 7f8f89428700 1 -- 172.21.15.133:0/1475328242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f8407bdd0 msgr2=0x7f8f8407a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:53.398 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.253+0000 7f8f89428700 1 --2- 172.21.15.133:0/1475328242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f8407bdd0 0x7f8f8407a2d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8f740098a0 tx=0x7f8f74009bb0 comp rx=0 tx=0).stop 2023-07-14T18:07:53.398 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.254+0000 7f8f89428700 1 -- 172.21.15.133:0/1475328242 shutdown_connections 2023-07-14T18:07:53.399 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.254+0000 7f8f89428700 1 --2- 172.21.15.133:0/1475328242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f8407bdd0 0x7f8f8407a2d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:53.399 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.254+0000 7f8f89428700 1 -- 172.21.15.133:0/1475328242 >> 172.21.15.133:0/1475328242 conn(0x7f8f84100ab0 msgr2=0x7f8f84102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:53.399 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.254+0000 7f8f89428700 1 -- 172.21.15.133:0/1475328242 shutdown_connections 2023-07-14T18:07:53.399 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.254+0000 7f8f89428700 1 -- 172.21.15.133:0/1475328242 wait complete. 2023-07-14T18:07:53.399 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.255+0000 7f8f89428700 1 Processor -- start 2023-07-14T18:07:53.399 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.255+0000 7f8f89428700 1 -- start start 2023-07-14T18:07:53.399 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.255+0000 7f8f89428700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f8407bdd0 0x7f8f8419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:53.399 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.255+0000 7f8f89428700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8419b8c0 con 0x7f8f8407bdd0 2023-07-14T18:07:53.400 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.255+0000 7f8f82ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f8407bdd0 0x7f8f8419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:53.400 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.255+0000 7f8f82ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f8407bdd0 0x7f8f8419b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59466/0 (socket says 172.21.15.133:59466) 2023-07-14T18:07:53.400 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.255+0000 7f8f82ffd700 1 -- 172.21.15.133:0/3811748447 learned_addr learned my addr 172.21.15.133:0/3811748447 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:53.400 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.256+0000 7f8f82ffd700 1 -- 172.21.15.133:0/3811748447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f74009550 con 0x7f8f8407bdd0 2023-07-14T18:07:53.400 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.256+0000 7f8f82ffd700 1 --2- 172.21.15.133:0/3811748447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f8407bdd0 0x7f8f8419b380 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8f74000c00 tx=0x7f8f74011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:53.400 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.256+0000 7f8f6bfff700 1 -- 172.21.15.133:0/3811748447 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f740039f0 con 0x7f8f8407bdd0 2023-07-14T18:07:53.400 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.256+0000 7f8f89428700 1 -- 172.21.15.133:0/3811748447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f8419bac0 con 0x7f8f8407bdd0 2023-07-14T18:07:53.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:53 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3811748447' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-07-14T18:07:53.402 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.256+0000 7f8f6bfff700 1 -- 172.21.15.133:0/3811748447 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f74003b50 con 0x7f8f8407bdd0 2023-07-14T18:07:53.403 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.256+0000 7f8f6bfff700 1 -- 172.21.15.133:0/3811748447 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f7401a780 con 0x7f8f8407bdd0 2023-07-14T18:07:53.403 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.256+0000 7f8f89428700 1 -- 172.21.15.133:0/3811748447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f8419bee0 con 0x7f8f8407bdd0 2023-07-14T18:07:53.403 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.257+0000 7f8f6bfff700 1 -- 172.21.15.133:0/3811748447 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f8f74024440 con 0x7f8f8407bdd0 2023-07-14T18:07:53.403 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.257+0000 7f8f89428700 1 -- 172.21.15.133:0/3811748447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f84061c70 con 0x7f8f8407bdd0 2023-07-14T18:07:53.403 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.258+0000 7f8f6bfff700 1 --2- 172.21.15.133:0/3811748447 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8f6c038760 0x7f8f6c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:53.404 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.258+0000 7f8f6bfff700 1 -- 172.21.15.133:0/3811748447 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8f7404c9d0 con 0x7f8f8407bdd0 2023-07-14T18:07:53.404 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.258+0000 7f8f827fc700 1 --2- 172.21.15.133:0/3811748447 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8f6c038760 0x7f8f6c03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:53.404 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.259+0000 7f8f827fc700 1 --2- 172.21.15.133:0/3811748447 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8f6c038760 0x7f8f6c03ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8f78006fd0 tx=0x7f8f78006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:53.404 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.264+0000 7f8f6bfff700 1 -- 172.21.15.133:0/3811748447 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f8f740189a0 con 0x7f8f8407bdd0 2023-07-14T18:07:53.404 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.484+0000 7f8f89428700 1 -- 172.21.15.133:0/3811748447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f8f8419ed40 con 0x7f8f8407bdd0 2023-07-14T18:07:53.404 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.484+0000 7f8f6bfff700 1 -- 172.21.15.133:0/3811748447 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f8f74018350 con 0x7f8f8407bdd0 2023-07-14T18:07:53.405 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.487+0000 7f8f89428700 1 -- 172.21.15.133:0/3811748447 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8f6c038760 msgr2=0x7f8f6c03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:53.405 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.487+0000 7f8f89428700 1 --2- 172.21.15.133:0/3811748447 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8f6c038760 0x7f8f6c03ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8f78006fd0 tx=0x7f8f78006e40 comp rx=0 tx=0).stop 2023-07-14T18:07:53.405 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.487+0000 7f8f89428700 1 -- 172.21.15.133:0/3811748447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f8407bdd0 msgr2=0x7f8f8419b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:53.405 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.487+0000 7f8f89428700 1 --2- 172.21.15.133:0/3811748447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f8407bdd0 0x7f8f8419b380 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8f74000c00 tx=0x7f8f74011a30 comp rx=0 tx=0).stop 2023-07-14T18:07:53.405 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.487+0000 7f8f89428700 1 -- 172.21.15.133:0/3811748447 shutdown_connections 2023-07-14T18:07:53.405 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.487+0000 7f8f89428700 1 --2- 172.21.15.133:0/3811748447 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8f6c038760 0x7f8f6c03ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:53.405 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.487+0000 7f8f89428700 1 --2- 172.21.15.133:0/3811748447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f8407bdd0 0x7f8f8419b380 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:53.406 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.487+0000 7f8f89428700 1 -- 172.21.15.133:0/3811748447 >> 172.21.15.133:0/3811748447 conn(0x7f8f84100ab0 msgr2=0x7f8f84102170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:53.406 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.487+0000 7f8f89428700 1 -- 172.21.15.133:0/3811748447 shutdown_connections 2023-07-14T18:07:53.406 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:52.487+0000 7f8f89428700 1 -- 172.21.15.133:0/3811748447 wait complete. 2023-07-14T18:07:53.418 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-14T18:07:53.419 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout disabled 2023-07-14T18:07:53.435 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-14T18:07:53.436 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout inactive 2023-07-14T18:07:53.436 INFO:teuthology.orchestra.run.smithi133.stdout:firewalld.service is not enabled 2023-07-14T18:07:53.436 INFO:teuthology.orchestra.run.smithi133.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-07-14T18:07:53.438 INFO:teuthology.orchestra.run.smithi133.stdout:Ceph Dashboard is now available at: 2023-07-14T18:07:53.438 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:07:53.438 INFO:teuthology.orchestra.run.smithi133.stdout: URL: https://smithi133.front.sepia.ceph.com:8443/ 2023-07-14T18:07:53.439 INFO:teuthology.orchestra.run.smithi133.stdout: User: admin 2023-07-14T18:07:53.439 INFO:teuthology.orchestra.run.smithi133.stdout: Password: ey8hrvvqvj 2023-07-14T18:07:53.439 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:07:53.439 INFO:teuthology.orchestra.run.smithi133.stdout:Saving cluster configuration to /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/config directory 2023-07-14T18:07:53.440 INFO:teuthology.orchestra.run.smithi133.stdout:Enabling autotune for osd_memory_target 2023-07-14T18:07:54.723 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.907+0000 7f5406716700 1 Processor -- start 2023-07-14T18:07:54.723 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.908+0000 7f5406716700 1 -- start start 2023-07-14T18:07:54.723 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.908+0000 7f5406716700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f540007be10 0x7f540007a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:54.723 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.908+0000 7f5406716700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540007a850 con 0x7f540007be10 2023-07-14T18:07:54.723 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.908+0000 7f53fffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f540007be10 0x7f540007a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:54.724 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.908+0000 7f53fffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f540007be10 0x7f540007a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59482/0 (socket says 172.21.15.133:59482) 2023-07-14T18:07:54.724 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.908+0000 7f53fffff700 1 -- 172.21.15.133:0/85429586 learned_addr learned my addr 172.21.15.133:0/85429586 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:54.724 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.909+0000 7f53fffff700 1 -- 172.21.15.133:0/85429586 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540007a990 con 0x7f540007be10 2023-07-14T18:07:54.724 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.909+0000 7f53fffff700 1 --2- 172.21.15.133:0/85429586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f540007be10 0x7f540007a310 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f53f00098a0 tx=0x7f53f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1aa184ca5b8804 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:54.724 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.909+0000 7f53feffd700 1 -- 172.21.15.133:0/85429586 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f53f0004160 con 0x7f540007be10 2023-07-14T18:07:54.724 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.909+0000 7f53feffd700 1 -- 172.21.15.133:0/85429586 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53f00042c0 con 0x7f540007be10 2023-07-14T18:07:54.724 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.909+0000 7f53feffd700 1 -- 172.21.15.133:0/85429586 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f53f00119b0 con 0x7f540007be10 2023-07-14T18:07:54.725 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.910+0000 7f5406716700 1 -- 172.21.15.133:0/85429586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f540007be10 msgr2=0x7f540007a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:54.725 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.910+0000 7f5406716700 1 --2- 172.21.15.133:0/85429586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f540007be10 0x7f540007a310 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f53f00098a0 tx=0x7f53f0009bb0 comp rx=0 tx=0).stop 2023-07-14T18:07:54.725 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.910+0000 7f5406716700 1 -- 172.21.15.133:0/85429586 shutdown_connections 2023-07-14T18:07:54.725 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.910+0000 7f5406716700 1 --2- 172.21.15.133:0/85429586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f540007be10 0x7f540007a310 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:54.725 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.910+0000 7f5406716700 1 -- 172.21.15.133:0/85429586 >> 172.21.15.133:0/85429586 conn(0x7f5400100ab0 msgr2=0x7f5400102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:54.725 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.910+0000 7f5406716700 1 -- 172.21.15.133:0/85429586 shutdown_connections 2023-07-14T18:07:54.725 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.910+0000 7f5406716700 1 -- 172.21.15.133:0/85429586 wait complete. 2023-07-14T18:07:54.725 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.911+0000 7f5406716700 1 Processor -- start 2023-07-14T18:07:54.726 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.911+0000 7f5406716700 1 -- start start 2023-07-14T18:07:54.726 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.911+0000 7f5406716700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f540007be10 0x7f540019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:54.726 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.911+0000 7f5406716700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540019b8e0 con 0x7f540007be10 2023-07-14T18:07:54.726 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.912+0000 7f53fffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f540007be10 0x7f540019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:54.726 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.912+0000 7f53fffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f540007be10 0x7f540019b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59492/0 (socket says 172.21.15.133:59492) 2023-07-14T18:07:54.726 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.912+0000 7f53fffff700 1 -- 172.21.15.133:0/2311748301 learned_addr learned my addr 172.21.15.133:0/2311748301 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:54.726 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.912+0000 7f53fffff700 1 -- 172.21.15.133:0/2311748301 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f0009550 con 0x7f540007be10 2023-07-14T18:07:54.727 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.912+0000 7f53fffff700 1 --2- 172.21.15.133:0/2311748301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f540007be10 0x7f540019b3a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f53f0003e60 tx=0x7f53f0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:54.727 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.912+0000 7f53fd7fa700 1 -- 172.21.15.133:0/2311748301 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f53f0003b20 con 0x7f540007be10 2023-07-14T18:07:54.727 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.912+0000 7f53fd7fa700 1 -- 172.21.15.133:0/2311748301 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53f0003c80 con 0x7f540007be10 2023-07-14T18:07:54.727 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.912+0000 7f5406716700 1 -- 172.21.15.133:0/2311748301 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f540019bae0 con 0x7f540007be10 2023-07-14T18:07:54.727 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.912+0000 7f53fd7fa700 1 -- 172.21.15.133:0/2311748301 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f53f0024760 con 0x7f540007be10 2023-07-14T18:07:54.727 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.913+0000 7f5406716700 1 -- 172.21.15.133:0/2311748301 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f540019bf00 con 0x7f540007be10 2023-07-14T18:07:54.727 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.914+0000 7f53fd7fa700 1 -- 172.21.15.133:0/2311748301 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f53f001a430 con 0x7f540007be10 2023-07-14T18:07:54.727 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.914+0000 7f5406716700 1 -- 172.21.15.133:0/2311748301 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5400061c70 con 0x7f540007be10 2023-07-14T18:07:54.728 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.914+0000 7f53fd7fa700 1 --2- 172.21.15.133:0/2311748301 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f53e8038800 0x7f53e803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:54.728 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.914+0000 7f53fd7fa700 1 -- 172.21.15.133:0/2311748301 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f53f004cb60 con 0x7f540007be10 2023-07-14T18:07:54.728 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.914+0000 7f53ff7fe700 1 --2- 172.21.15.133:0/2311748301 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f53e8038800 0x7f53e803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:54.728 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.915+0000 7f53ff7fe700 1 --2- 172.21.15.133:0/2311748301 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f53e8038800 0x7f53e803acb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f53f4006fd0 tx=0x7f53f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:54.729 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:53.919+0000 7f53fd7fa700 1 -- 172.21.15.133:0/2311748301 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f53f0010380 con 0x7f540007be10 2023-07-14T18:07:54.729 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:54.058+0000 7f53fd7fa700 1 -- 172.21.15.133:0/2311748301 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f53f001abc0 con 0x7f540007be10 2023-07-14T18:07:54.729 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:54.113+0000 7f5406716700 1 -- 172.21.15.133:0/2311748301 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f540019ed60 con 0x7f540007be10 2023-07-14T18:07:54.729 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:54.113+0000 7f53fd7fa700 1 -- 172.21.15.133:0/2311748301 <== mon.0 v2:172.21.15.133:3300/0 8 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f53f0018d20 con 0x7f540007be10 2023-07-14T18:07:54.729 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:54.116+0000 7f5406716700 1 -- 172.21.15.133:0/2311748301 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f53e8038800 msgr2=0x7f53e803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:54.729 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:54.116+0000 7f5406716700 1 --2- 172.21.15.133:0/2311748301 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f53e8038800 0x7f53e803acb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f53f4006fd0 tx=0x7f53f4006e40 comp rx=0 tx=0).stop 2023-07-14T18:07:54.729 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:54.116+0000 7f5406716700 1 -- 172.21.15.133:0/2311748301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f540007be10 msgr2=0x7f540019b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:54.730 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:54.116+0000 7f5406716700 1 --2- 172.21.15.133:0/2311748301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f540007be10 0x7f540019b3a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f53f0003e60 tx=0x7f53f0011a30 comp rx=0 tx=0).stop 2023-07-14T18:07:54.730 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:54.116+0000 7f5406716700 1 -- 172.21.15.133:0/2311748301 shutdown_connections 2023-07-14T18:07:54.730 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:54.116+0000 7f5406716700 1 --2- 172.21.15.133:0/2311748301 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f53e8038800 0x7f53e803acb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:54.730 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:54.116+0000 7f5406716700 1 --2- 172.21.15.133:0/2311748301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f540007be10 0x7f540019b3a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:54.730 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:54.116+0000 7f5406716700 1 -- 172.21.15.133:0/2311748301 >> 172.21.15.133:0/2311748301 conn(0x7f5400100ab0 msgr2=0x7f5400101840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:54.730 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:54.116+0000 7f5406716700 1 -- 172.21.15.133:0/2311748301 shutdown_connections 2023-07-14T18:07:54.730 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:54.116+0000 7f5406716700 1 -- 172.21.15.133:0/2311748301 wait complete. 2023-07-14T18:07:55.396 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:55 smithi133 ceph-mon[126384]: mgrmap e13: x(active, since 6s) 2023-07-14T18:07:56.100 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.256+0000 7fa9b9b6a700 1 Processor -- start 2023-07-14T18:07:56.100 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.256+0000 7fa9b9b6a700 1 -- start start 2023-07-14T18:07:56.101 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.256+0000 7fa9b9b6a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9b407bdf0 0x7fa9b407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:56.101 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.256+0000 7fa9b9b6a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b407a830 con 0x7fa9b407bdf0 2023-07-14T18:07:56.101 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.256+0000 7fa9b37fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9b407bdf0 0x7fa9b407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:56.101 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.256+0000 7fa9b37fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9b407bdf0 0x7fa9b407a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59498/0 (socket says 172.21.15.133:59498) 2023-07-14T18:07:56.101 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.256+0000 7fa9b37fe700 1 -- 172.21.15.133:0/671891803 learned_addr learned my addr 172.21.15.133:0/671891803 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:56.101 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.257+0000 7fa9b37fe700 1 -- 172.21.15.133:0/671891803 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b407a970 con 0x7fa9b407bdf0 2023-07-14T18:07:56.102 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.257+0000 7fa9b37fe700 1 --2- 172.21.15.133:0/671891803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9b407bdf0 0x7fa9b407a2f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40098a0 tx=0x7fa9a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2386963823ca531 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:56.102 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.257+0000 7fa9b27fc700 1 -- 172.21.15.133:0/671891803 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9a4004160 con 0x7fa9b407bdf0 2023-07-14T18:07:56.102 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.257+0000 7fa9b27fc700 1 -- 172.21.15.133:0/671891803 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9a40042c0 con 0x7fa9b407bdf0 2023-07-14T18:07:56.102 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.257+0000 7fa9b27fc700 1 -- 172.21.15.133:0/671891803 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9a40119b0 con 0x7fa9b407bdf0 2023-07-14T18:07:56.102 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.258+0000 7fa9b9b6a700 1 -- 172.21.15.133:0/671891803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9b407bdf0 msgr2=0x7fa9b407a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:56.102 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.258+0000 7fa9b9b6a700 1 --2- 172.21.15.133:0/671891803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9b407bdf0 0x7fa9b407a2f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40098a0 tx=0x7fa9a4009bb0 comp rx=0 tx=0).stop 2023-07-14T18:07:56.102 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.258+0000 7fa9b9b6a700 1 -- 172.21.15.133:0/671891803 shutdown_connections 2023-07-14T18:07:56.102 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.258+0000 7fa9b9b6a700 1 --2- 172.21.15.133:0/671891803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9b407bdf0 0x7fa9b407a2f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:56.103 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.258+0000 7fa9b9b6a700 1 -- 172.21.15.133:0/671891803 >> 172.21.15.133:0/671891803 conn(0x7fa9b4100ab0 msgr2=0x7fa9b4102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:56.103 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.258+0000 7fa9b9b6a700 1 -- 172.21.15.133:0/671891803 shutdown_connections 2023-07-14T18:07:56.103 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.258+0000 7fa9b9b6a700 1 -- 172.21.15.133:0/671891803 wait complete. 2023-07-14T18:07:56.103 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.259+0000 7fa9b9b6a700 1 Processor -- start 2023-07-14T18:07:56.103 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.259+0000 7fa9b9b6a700 1 -- start start 2023-07-14T18:07:56.103 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.259+0000 7fa9b9b6a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9b407bdf0 0x7fa9b419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:56.103 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.259+0000 7fa9b9b6a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b419b8a0 con 0x7fa9b407bdf0 2023-07-14T18:07:56.103 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.260+0000 7fa9b37fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9b407bdf0 0x7fa9b419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:56.104 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.260+0000 7fa9b37fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9b407bdf0 0x7fa9b419b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59506/0 (socket says 172.21.15.133:59506) 2023-07-14T18:07:56.104 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.260+0000 7fa9b37fe700 1 -- 172.21.15.133:0/2717533419 learned_addr learned my addr 172.21.15.133:0/2717533419 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:56.104 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.260+0000 7fa9b37fe700 1 -- 172.21.15.133:0/2717533419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a4009550 con 0x7fa9b407bdf0 2023-07-14T18:07:56.104 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.260+0000 7fa9b37fe700 1 --2- 172.21.15.133:0/2717533419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9b407bdf0 0x7fa9b419b360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4000c00 tx=0x7fa9a4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:56.104 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.260+0000 7fa9b0ff9700 1 -- 172.21.15.133:0/2717533419 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9a40039f0 con 0x7fa9b407bdf0 2023-07-14T18:07:56.104 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.261+0000 7fa9b0ff9700 1 -- 172.21.15.133:0/2717533419 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9a4003b50 con 0x7fa9b407bdf0 2023-07-14T18:07:56.104 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.261+0000 7fa9b9b6a700 1 -- 172.21.15.133:0/2717533419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9b419baa0 con 0x7fa9b407bdf0 2023-07-14T18:07:56.105 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.261+0000 7fa9b0ff9700 1 -- 172.21.15.133:0/2717533419 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9a401a780 con 0x7fa9b407bdf0 2023-07-14T18:07:56.105 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.261+0000 7fa9b9b6a700 1 -- 172.21.15.133:0/2717533419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b419bec0 con 0x7fa9b407bdf0 2023-07-14T18:07:56.105 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.262+0000 7fa9b0ff9700 1 -- 172.21.15.133:0/2717533419 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7fa9a4024440 con 0x7fa9b407bdf0 2023-07-14T18:07:56.105 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.262+0000 7fa9b9b6a700 1 -- 172.21.15.133:0/2717533419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9b4061c70 con 0x7fa9b407bdf0 2023-07-14T18:07:56.105 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.262+0000 7fa9b0ff9700 1 --2- 172.21.15.133:0/2717533419 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa99c038850 0x7fa99c03ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:56.105 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.262+0000 7fa9b0ff9700 1 -- 172.21.15.133:0/2717533419 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa9a404c990 con 0x7fa9b407bdf0 2023-07-14T18:07:56.105 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.262+0000 7fa9b2ffd700 1 --2- 172.21.15.133:0/2717533419 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa99c038850 0x7fa99c03ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:56.105 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.263+0000 7fa9b2ffd700 1 --2- 172.21.15.133:0/2717533419 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa99c038850 0x7fa99c03ad00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8006fd0 tx=0x7fa9a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:56.106 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.267+0000 7fa9b0ff9700 1 -- 172.21.15.133:0/2717533419 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fa9a4018db0 con 0x7fa9b407bdf0 2023-07-14T18:07:56.106 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.569+0000 7fa9b9b6a700 1 -- 172.21.15.133:0/2717533419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fa9b419ed40 con 0x7fa9b407bdf0 2023-07-14T18:07:56.106 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.572+0000 7fa9b0ff9700 1 -- 172.21.15.133:0/2717533419 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7fa9a4018350 con 0x7fa9b407bdf0 2023-07-14T18:07:56.106 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.574+0000 7fa9b9b6a700 1 -- 172.21.15.133:0/2717533419 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa99c038850 msgr2=0x7fa99c03ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:56.106 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.575+0000 7fa9b9b6a700 1 --2- 172.21.15.133:0/2717533419 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa99c038850 0x7fa99c03ad00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8006fd0 tx=0x7fa9a8006e40 comp rx=0 tx=0).stop 2023-07-14T18:07:56.106 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.575+0000 7fa9b9b6a700 1 -- 172.21.15.133:0/2717533419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9b407bdf0 msgr2=0x7fa9b419b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:56.106 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.575+0000 7fa9b9b6a700 1 --2- 172.21.15.133:0/2717533419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9b407bdf0 0x7fa9b419b360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4000c00 tx=0x7fa9a4011a30 comp rx=0 tx=0).stop 2023-07-14T18:07:56.107 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.575+0000 7fa9b9b6a700 1 -- 172.21.15.133:0/2717533419 shutdown_connections 2023-07-14T18:07:56.107 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.575+0000 7fa9b9b6a700 1 --2- 172.21.15.133:0/2717533419 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa99c038850 0x7fa99c03ad00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:56.107 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.575+0000 7fa9b9b6a700 1 --2- 172.21.15.133:0/2717533419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9b407bdf0 0x7fa9b419b360 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:56.107 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.575+0000 7fa9b9b6a700 1 -- 172.21.15.133:0/2717533419 >> 172.21.15.133:0/2717533419 conn(0x7fa9b4100ab0 msgr2=0x7fa9b4102130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:56.107 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.575+0000 7fa9b9b6a700 1 -- 172.21.15.133:0/2717533419 shutdown_connections 2023-07-14T18:07:56.107 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-07-14T18:07:55.575+0000 7fa9b9b6a700 1 -- 172.21.15.133:0/2717533419 wait complete. 2023-07-14T18:07:56.107 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-07-14T18:07:56.107 INFO:teuthology.orchestra.run.smithi133.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-07-14T18:07:56.108 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:07:56.108 INFO:teuthology.orchestra.run.smithi133.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-07-14T18:07:56.108 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:07:56.108 INFO:teuthology.orchestra.run.smithi133.stdout:Or, if you are only running a single cluster on this host: 2023-07-14T18:07:56.108 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:07:56.108 INFO:teuthology.orchestra.run.smithi133.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-07-14T18:07:56.108 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:07:56.108 INFO:teuthology.orchestra.run.smithi133.stdout:Please consider enabling telemetry to help improve Ceph: 2023-07-14T18:07:56.109 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:07:56.109 INFO:teuthology.orchestra.run.smithi133.stdout: ceph telemetry on 2023-07-14T18:07:56.109 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:07:56.109 INFO:teuthology.orchestra.run.smithi133.stdout:For more information see: 2023-07-14T18:07:56.109 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:07:56.109 INFO:teuthology.orchestra.run.smithi133.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-07-14T18:07:56.109 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:07:56.110 INFO:teuthology.orchestra.run.smithi133.stdout:Bootstrap complete. 2023-07-14T18:07:56.157 INFO:tasks.cephadm:Fetching config... 2023-07-14T18:07:56.158 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T18:07:56.158 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-07-14T18:07:56.200 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-07-14T18:07:56.200 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T18:07:56.201 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-07-14T18:07:56.284 INFO:tasks.cephadm:Fetching mon keyring... 2023-07-14T18:07:56.284 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T18:07:56.285 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/keyring of=/dev/stdout 2023-07-14T18:07:56.394 INFO:tasks.cephadm:Fetching pub ssh key... 2023-07-14T18:07:56.395 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T18:07:56.395 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-07-14T18:07:56.477 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-07-14T18:07:56.478 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrFlTG0nyEvUyDLAOrZfbfgyWYxB55ugggPTkX1pNAXRJIxVvjXLaa/ghiJ/h4lmU069hgLyCYatnO8RtEsgSYdTPlDI/ecVmJgprQV5qwYx0nR8vGQSQJIAQDbUQ0LIW9k1zVQp4ZvNCYx3W60+vD2pVA+/9Qe0IevJqiyjchDJGrqJMfYk4vCVNLkl0v5HNuJqZa/ygE4a/tH8YyyF8s67sbA/dxNQC8dRjkc0L3L/UpX42g/+jX92b4vgqikvyXEOR0GMiA89Yyry8ysTBn2VfIPAi53EJ2B6/4txeX06xM7DQ3NzFrk7cH7+3ir4d2QHhn4QyP9yE+BU7eFRhrLz9udTHw/5/0CDKfMUqARb89N+L5lwtC6YIvdiq3be5QlbuvBKIRnZlhq4+Gk7kchQvafaLOyC2Kafo30YqHP/B+TiDmw4qIpcfofGDqW6xoQ5dYJ+ziQCNnqemZp1ROuawGoOj9hRlZ8SUF9aOP/3dEblsyfYrRXHPBSQJQGfk= ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-14T18:07:56.618 INFO:teuthology.orchestra.run.smithi133.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrFlTG0nyEvUyDLAOrZfbfgyWYxB55ugggPTkX1pNAXRJIxVvjXLaa/ghiJ/h4lmU069hgLyCYatnO8RtEsgSYdTPlDI/ecVmJgprQV5qwYx0nR8vGQSQJIAQDbUQ0LIW9k1zVQp4ZvNCYx3W60+vD2pVA+/9Qe0IevJqiyjchDJGrqJMfYk4vCVNLkl0v5HNuJqZa/ygE4a/tH8YyyF8s67sbA/dxNQC8dRjkc0L3L/UpX42g/+jX92b4vgqikvyXEOR0GMiA89Yyry8ysTBn2VfIPAi53EJ2B6/4txeX06xM7DQ3NzFrk7cH7+3ir4d2QHhn4QyP9yE+BU7eFRhrLz9udTHw/5/0CDKfMUqARb89N+L5lwtC6YIvdiq3be5QlbuvBKIRnZlhq4+Gk7kchQvafaLOyC2Kafo30YqHP/B+TiDmw4qIpcfofGDqW6xoQ5dYJ+ziQCNnqemZp1ROuawGoOj9hRlZ8SUF9aOP/3dEblsyfYrRXHPBSQJQGfk= ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c 2023-07-14T18:07:56.639 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:56 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2717533419' entity='client.admin' 2023-07-14T18:07:56.648 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrFlTG0nyEvUyDLAOrZfbfgyWYxB55ugggPTkX1pNAXRJIxVvjXLaa/ghiJ/h4lmU069hgLyCYatnO8RtEsgSYdTPlDI/ecVmJgprQV5qwYx0nR8vGQSQJIAQDbUQ0LIW9k1zVQp4ZvNCYx3W60+vD2pVA+/9Qe0IevJqiyjchDJGrqJMfYk4vCVNLkl0v5HNuJqZa/ygE4a/tH8YyyF8s67sbA/dxNQC8dRjkc0L3L/UpX42g/+jX92b4vgqikvyXEOR0GMiA89Yyry8ysTBn2VfIPAi53EJ2B6/4txeX06xM7DQ3NzFrk7cH7+3ir4d2QHhn4QyP9yE+BU7eFRhrLz9udTHw/5/0CDKfMUqARb89N+L5lwtC6YIvdiq3be5QlbuvBKIRnZlhq4+Gk7kchQvafaLOyC2Kafo30YqHP/B+TiDmw4qIpcfofGDqW6xoQ5dYJ+ziQCNnqemZp1ROuawGoOj9hRlZ8SUF9aOP/3dEblsyfYrRXHPBSQJQGfk= ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-14T18:07:56.693 INFO:teuthology.orchestra.run.smithi161.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrFlTG0nyEvUyDLAOrZfbfgyWYxB55ugggPTkX1pNAXRJIxVvjXLaa/ghiJ/h4lmU069hgLyCYatnO8RtEsgSYdTPlDI/ecVmJgprQV5qwYx0nR8vGQSQJIAQDbUQ0LIW9k1zVQp4ZvNCYx3W60+vD2pVA+/9Qe0IevJqiyjchDJGrqJMfYk4vCVNLkl0v5HNuJqZa/ygE4a/tH8YyyF8s67sbA/dxNQC8dRjkc0L3L/UpX42g/+jX92b4vgqikvyXEOR0GMiA89Yyry8ysTBn2VfIPAi53EJ2B6/4txeX06xM7DQ3NzFrk7cH7+3ir4d2QHhn4QyP9yE+BU7eFRhrLz9udTHw/5/0CDKfMUqARb89N+L5lwtC6YIvdiq3be5QlbuvBKIRnZlhq4+Gk7kchQvafaLOyC2Kafo30YqHP/B+TiDmw4qIpcfofGDqW6xoQ5dYJ+ziQCNnqemZp1ROuawGoOj9hRlZ8SUF9aOP/3dEblsyfYrRXHPBSQJQGfk= ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c 2023-07-14T18:07:56.705 DEBUG:teuthology.orchestra.run.smithi171:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrFlTG0nyEvUyDLAOrZfbfgyWYxB55ugggPTkX1pNAXRJIxVvjXLaa/ghiJ/h4lmU069hgLyCYatnO8RtEsgSYdTPlDI/ecVmJgprQV5qwYx0nR8vGQSQJIAQDbUQ0LIW9k1zVQp4ZvNCYx3W60+vD2pVA+/9Qe0IevJqiyjchDJGrqJMfYk4vCVNLkl0v5HNuJqZa/ygE4a/tH8YyyF8s67sbA/dxNQC8dRjkc0L3L/UpX42g/+jX92b4vgqikvyXEOR0GMiA89Yyry8ysTBn2VfIPAi53EJ2B6/4txeX06xM7DQ3NzFrk7cH7+3ir4d2QHhn4QyP9yE+BU7eFRhrLz9udTHw/5/0CDKfMUqARb89N+L5lwtC6YIvdiq3be5QlbuvBKIRnZlhq4+Gk7kchQvafaLOyC2Kafo30YqHP/B+TiDmw4qIpcfofGDqW6xoQ5dYJ+ziQCNnqemZp1ROuawGoOj9hRlZ8SUF9aOP/3dEblsyfYrRXHPBSQJQGfk= ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-14T18:07:56.750 INFO:teuthology.orchestra.run.smithi171.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrFlTG0nyEvUyDLAOrZfbfgyWYxB55ugggPTkX1pNAXRJIxVvjXLaa/ghiJ/h4lmU069hgLyCYatnO8RtEsgSYdTPlDI/ecVmJgprQV5qwYx0nR8vGQSQJIAQDbUQ0LIW9k1zVQp4ZvNCYx3W60+vD2pVA+/9Qe0IevJqiyjchDJGrqJMfYk4vCVNLkl0v5HNuJqZa/ygE4a/tH8YyyF8s67sbA/dxNQC8dRjkc0L3L/UpX42g/+jX92b4vgqikvyXEOR0GMiA89Yyry8ysTBn2VfIPAi53EJ2B6/4txeX06xM7DQ3NzFrk7cH7+3ir4d2QHhn4QyP9yE+BU7eFRhrLz9udTHw/5/0CDKfMUqARb89N+L5lwtC6YIvdiq3be5QlbuvBKIRnZlhq4+Gk7kchQvafaLOyC2Kafo30YqHP/B+TiDmw4qIpcfofGDqW6xoQ5dYJ+ziQCNnqemZp1ROuawGoOj9hRlZ8SUF9aOP/3dEblsyfYrRXHPBSQJQGfk= ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c 2023-07-14T18:07:56.763 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-07-14T18:07:57.691 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:07:59.088 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:58 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:07:59.088 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:58 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:07:59.089 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:58 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi133", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:07:59.089 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:58 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:07:59.089 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:58 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:07:59.089 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:07:58 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:07:59.381 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.346+0000 7f96c55d2700 1 -- 172.21.15.133:0/2910388986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96c00ff730 msgr2=0x7f96c00ffb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.346+0000 7f96c55d2700 1 --2- 172.21.15.133:0/2910388986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96c00ff730 0x7f96c00ffb40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009910 tx=0x7f96b0009c20 comp rx=0 tx=0).stop 2023-07-14T18:07:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.347+0000 7f96c55d2700 1 -- 172.21.15.133:0/2910388986 shutdown_connections 2023-07-14T18:07:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.347+0000 7f96c55d2700 1 --2- 172.21.15.133:0/2910388986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96c00ff730 0x7f96c00ffb40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.347+0000 7f96c55d2700 1 -- 172.21.15.133:0/2910388986 >> 172.21.15.133:0/2910388986 conn(0x7f96c00facc0 msgr2=0x7f96c00fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.347+0000 7f96c55d2700 1 -- 172.21.15.133:0/2910388986 shutdown_connections 2023-07-14T18:07:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.347+0000 7f96c55d2700 1 -- 172.21.15.133:0/2910388986 wait complete. 2023-07-14T18:07:59.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.348+0000 7f96c55d2700 1 Processor -- start 2023-07-14T18:07:59.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.348+0000 7f96c55d2700 1 -- start start 2023-07-14T18:07:59.384 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.349+0000 7f96c55d2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96c00ff730 0x7f96c019d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:59.384 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.349+0000 7f96c55d2700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c019d5d0 con 0x7f96c00ff730 2023-07-14T18:07:59.384 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.349+0000 7f96beffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96c00ff730 0x7f96c019d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:59.385 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.350+0000 7f96beffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96c00ff730 0x7f96c019d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42222/0 (socket says 172.21.15.133:42222) 2023-07-14T18:07:59.385 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.350+0000 7f96beffd700 1 -- 172.21.15.133:0/360235032 learned_addr learned my addr 172.21.15.133:0/360235032 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:07:59.385 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.351+0000 7f96beffd700 1 -- 172.21.15.133:0/360235032 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b00095f0 con 0x7f96c00ff730 2023-07-14T18:07:59.386 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.351+0000 7f96beffd700 1 --2- 172.21.15.133:0/360235032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96c00ff730 0x7f96c019d090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f96b0004f40 tx=0x7f96b00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:59.386 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.351+0000 7f96a7fff700 1 -- 172.21.15.133:0/360235032 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96b001d070 con 0x7f96c00ff730 2023-07-14T18:07:59.386 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.351+0000 7f96c55d2700 1 -- 172.21.15.133:0/360235032 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c019d7d0 con 0x7f96c00ff730 2023-07-14T18:07:59.387 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.351+0000 7f96a7fff700 1 -- 172.21.15.133:0/360235032 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96b000bc10 con 0x7f96c00ff730 2023-07-14T18:07:59.387 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.352+0000 7f96a7fff700 1 -- 172.21.15.133:0/360235032 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96b0003a70 con 0x7f96c00ff730 2023-07-14T18:07:59.387 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.352+0000 7f96c55d2700 1 -- 172.21.15.133:0/360235032 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c019dc70 con 0x7f96c00ff730 2023-07-14T18:07:59.387 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.353+0000 7f96c55d2700 1 -- 172.21.15.133:0/360235032 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c006d810 con 0x7f96c00ff730 2023-07-14T18:07:59.387 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.353+0000 7f96a7fff700 1 -- 172.21.15.133:0/360235032 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f96b0003ce0 con 0x7f96c00ff730 2023-07-14T18:07:59.388 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.354+0000 7f96a7fff700 1 --2- 172.21.15.133:0/360235032 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f96a8038800 0x7f96a803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:07:59.388 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.354+0000 7f96a7fff700 1 -- 172.21.15.133:0/360235032 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f96b004c550 con 0x7f96c00ff730 2023-07-14T18:07:59.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.354+0000 7f96be7fc700 1 --2- 172.21.15.133:0/360235032 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f96a8038800 0x7f96a803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:07:59.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.355+0000 7f96be7fc700 1 --2- 172.21.15.133:0/360235032 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f96a8038800 0x7f96a803acb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f96b4006fd0 tx=0x7f96b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:07:59.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.359+0000 7f96a7fff700 1 -- 172.21.15.133:0/360235032 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f96b002abf0 con 0x7f96c00ff730 2023-07-14T18:07:59.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.560+0000 7f96c55d2700 1 -- 172.21.15.133:0/360235032 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f96c002c8e0 con 0x7f96c00ff730 2023-07-14T18:07:59.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.564+0000 7f96a7fff700 1 -- 172.21.15.133:0/360235032 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f96b0026070 con 0x7f96c00ff730 2023-07-14T18:07:59.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.566+0000 7f96c55d2700 1 -- 172.21.15.133:0/360235032 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f96a8038800 msgr2=0x7f96a803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:59.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.566+0000 7f96c55d2700 1 --2- 172.21.15.133:0/360235032 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f96a8038800 0x7f96a803acb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f96b4006fd0 tx=0x7f96b4006e40 comp rx=0 tx=0).stop 2023-07-14T18:07:59.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.566+0000 7f96c55d2700 1 -- 172.21.15.133:0/360235032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96c00ff730 msgr2=0x7f96c019d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:07:59.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.566+0000 7f96c55d2700 1 --2- 172.21.15.133:0/360235032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96c00ff730 0x7f96c019d090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f96b0004f40 tx=0x7f96b00057c0 comp rx=0 tx=0).stop 2023-07-14T18:07:59.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.567+0000 7f96c55d2700 1 -- 172.21.15.133:0/360235032 shutdown_connections 2023-07-14T18:07:59.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.567+0000 7f96c55d2700 1 --2- 172.21.15.133:0/360235032 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f96a8038800 0x7f96a803acb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:59.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.567+0000 7f96c55d2700 1 --2- 172.21.15.133:0/360235032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96c00ff730 0x7f96c019d090 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:07:59.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.567+0000 7f96c55d2700 1 -- 172.21.15.133:0/360235032 >> 172.21.15.133:0/360235032 conn(0x7f96c00facc0 msgr2=0x7f96c00fb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:07:59.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.567+0000 7f96c55d2700 1 -- 172.21.15.133:0/360235032 shutdown_connections 2023-07-14T18:07:59.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:07:59.567+0000 7f96c55d2700 1 -- 172.21.15.133:0/360235032 wait complete. 2023-07-14T18:08:00.100 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-07-14T18:08:00.101 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-07-14T18:08:00.816 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:08:00.896 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:00 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/360235032' entity='client.admin' 2023-07-14T18:08:00.896 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:00 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:08:00.897 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:00 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:02.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.279+0000 7f417c017700 1 -- 172.21.15.133:0/2671588545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41740ff730 msgr2=0x7f41740ffb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:02.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.279+0000 7f417c017700 1 --2- 172.21.15.133:0/2671588545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41740ff730 0x7f41740ffb40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4170009910 tx=0x7f4170009c20 comp rx=0 tx=0).stop 2023-07-14T18:08:02.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.279+0000 7f417c017700 1 -- 172.21.15.133:0/2671588545 shutdown_connections 2023-07-14T18:08:02.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.279+0000 7f417c017700 1 --2- 172.21.15.133:0/2671588545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41740ff730 0x7f41740ffb40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:02.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.279+0000 7f417c017700 1 -- 172.21.15.133:0/2671588545 >> 172.21.15.133:0/2671588545 conn(0x7f41740facc0 msgr2=0x7f41740fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:08:02.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.279+0000 7f417c017700 1 -- 172.21.15.133:0/2671588545 shutdown_connections 2023-07-14T18:08:02.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.279+0000 7f417c017700 1 -- 172.21.15.133:0/2671588545 wait complete. 2023-07-14T18:08:02.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.280+0000 7f417c017700 1 Processor -- start 2023-07-14T18:08:02.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.280+0000 7f417c017700 1 -- start start 2023-07-14T18:08:02.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.280+0000 7f417c017700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41740ff730 0x7f4174194880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:02.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.280+0000 7f417c017700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4174194dc0 con 0x7f41740ff730 2023-07-14T18:08:02.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.281+0000 7f4179db3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41740ff730 0x7f4174194880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:02.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.281+0000 7f4179db3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41740ff730 0x7f4174194880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42232/0 (socket says 172.21.15.133:42232) 2023-07-14T18:08:02.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.281+0000 7f4179db3700 1 -- 172.21.15.133:0/3878515206 learned_addr learned my addr 172.21.15.133:0/3878515206 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:08:02.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.281+0000 7f4179db3700 1 -- 172.21.15.133:0/3878515206 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41700095f0 con 0x7f41740ff730 2023-07-14T18:08:02.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.281+0000 7f4179db3700 1 --2- 172.21.15.133:0/3878515206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41740ff730 0x7f4174194880 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4170004f40 tx=0x7f41700057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:08:02.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.281+0000 7f416affd700 1 -- 172.21.15.133:0/3878515206 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f417001d070 con 0x7f41740ff730 2023-07-14T18:08:02.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.281+0000 7f417c017700 1 -- 172.21.15.133:0/3878515206 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4174194fc0 con 0x7f41740ff730 2023-07-14T18:08:02.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.281+0000 7f416affd700 1 -- 172.21.15.133:0/3878515206 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f417000bc10 con 0x7f41740ff730 2023-07-14T18:08:02.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.281+0000 7f416affd700 1 -- 172.21.15.133:0/3878515206 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4170003a70 con 0x7f41740ff730 2023-07-14T18:08:02.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.282+0000 7f417c017700 1 -- 172.21.15.133:0/3878515206 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4174195460 con 0x7f41740ff730 2023-07-14T18:08:02.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.282+0000 7f416affd700 1 -- 172.21.15.133:0/3878515206 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f4170003d30 con 0x7f41740ff730 2023-07-14T18:08:02.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.283+0000 7f417c017700 1 -- 172.21.15.133:0/3878515206 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f417418e300 con 0x7f41740ff730 2023-07-14T18:08:02.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.283+0000 7f416affd700 1 --2- 172.21.15.133:0/3878515206 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f41600387b0 0x7f416003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:02.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.283+0000 7f416affd700 1 -- 172.21.15.133:0/3878515206 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f417004c750 con 0x7f41740ff730 2023-07-14T18:08:02.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.283+0000 7f41795b2700 1 --2- 172.21.15.133:0/3878515206 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f41600387b0 0x7f416003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:02.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.284+0000 7f41795b2700 1 --2- 172.21.15.133:0/3878515206 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f41600387b0 0x7f416003ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4164006fd0 tx=0x7f4164006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:08:02.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.288+0000 7f416affd700 1 -- 172.21.15.133:0/3878515206 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f4170029950 con 0x7f41740ff730 2023-07-14T18:08:02.521 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.487+0000 7f417c017700 1 -- 172.21.15.133:0/3878515206 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f4174060b10 con 0x7f41600387b0 2023-07-14T18:08:02.525 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.490+0000 7f416affd700 1 -- 172.21.15.133:0/3878515206 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4174060b10 con 0x7f41600387b0 2023-07-14T18:08:02.527 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.492+0000 7f417c017700 1 -- 172.21.15.133:0/3878515206 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f41600387b0 msgr2=0x7f416003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:02.527 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.492+0000 7f417c017700 1 --2- 172.21.15.133:0/3878515206 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f41600387b0 0x7f416003ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4164006fd0 tx=0x7f4164006e40 comp rx=0 tx=0).stop 2023-07-14T18:08:02.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.492+0000 7f417c017700 1 -- 172.21.15.133:0/3878515206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41740ff730 msgr2=0x7f4174194880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:02.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.492+0000 7f417c017700 1 --2- 172.21.15.133:0/3878515206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41740ff730 0x7f4174194880 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4170004f40 tx=0x7f41700057c0 comp rx=0 tx=0).stop 2023-07-14T18:08:02.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.493+0000 7f417c017700 1 -- 172.21.15.133:0/3878515206 shutdown_connections 2023-07-14T18:08:02.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.493+0000 7f417c017700 1 --2- 172.21.15.133:0/3878515206 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f41600387b0 0x7f416003ac60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:02.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.493+0000 7f417c017700 1 --2- 172.21.15.133:0/3878515206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41740ff730 0x7f4174194880 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:02.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.493+0000 7f417c017700 1 -- 172.21.15.133:0/3878515206 >> 172.21.15.133:0/3878515206 conn(0x7f41740facc0 msgr2=0x7f41740fb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:08:02.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.493+0000 7f417c017700 1 -- 172.21.15.133:0/3878515206 shutdown_connections 2023-07-14T18:08:02.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:02.493+0000 7f417c017700 1 -- 172.21.15.133:0/3878515206 wait complete. 2023-07-14T18:08:02.988 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi161 2023-07-14T18:08:02.988 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T18:08:02.989 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.conf 2023-07-14T18:08:03.009 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T18:08:03.009 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-14T18:08:03.068 INFO:tasks.cephadm:Adding host smithi161 to orchestrator... 2023-07-14T18:08:03.069 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph orch host add smithi161 2023-07-14T18:08:03.813 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:08:03.897 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:03 smithi133 ceph-mon[126384]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:08:03.897 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:03 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:03.897 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:03 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:08:03.897 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:03 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:03.897 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:03 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:08:03.897 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:03 smithi133 ceph-mon[126384]: Updating smithi133:/etc/ceph/ceph.conf 2023-07-14T18:08:04.746 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:04 smithi133 ceph-mon[126384]: Updating smithi133:/var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T18:08:04.746 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:04 smithi133 ceph-mon[126384]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2023-07-14T18:08:05.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.306+0000 7f4da9c2a700 1 -- 172.21.15.133:0/679641894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4da4105740 msgr2=0x7f4da4105b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:05.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.306+0000 7f4da9c2a700 1 --2- 172.21.15.133:0/679641894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4da4105740 0x7f4da4105b10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4d94009910 tx=0x7f4d94009c20 comp rx=0 tx=0).stop 2023-07-14T18:08:05.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.306+0000 7f4da9c2a700 1 -- 172.21.15.133:0/679641894 shutdown_connections 2023-07-14T18:08:05.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.306+0000 7f4da9c2a700 1 --2- 172.21.15.133:0/679641894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4da4105740 0x7f4da4105b10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:05.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.306+0000 7f4da9c2a700 1 -- 172.21.15.133:0/679641894 >> 172.21.15.133:0/679641894 conn(0x7f4da40fb280 msgr2=0x7f4da40fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:08:05.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.306+0000 7f4da9c2a700 1 -- 172.21.15.133:0/679641894 shutdown_connections 2023-07-14T18:08:05.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.306+0000 7f4da9c2a700 1 -- 172.21.15.133:0/679641894 wait complete. 2023-07-14T18:08:05.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.307+0000 7f4da9c2a700 1 Processor -- start 2023-07-14T18:08:05.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.308+0000 7f4da9c2a700 1 -- start start 2023-07-14T18:08:05.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.308+0000 7f4da9c2a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4da4105740 0x7f4da4072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:05.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.308+0000 7f4da9c2a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da40748c0 con 0x7f4da4105740 2023-07-14T18:08:05.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.308+0000 7f4da37fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4da4105740 0x7f4da4072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:05.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.308+0000 7f4da37fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4da4105740 0x7f4da4072940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42262/0 (socket says 172.21.15.133:42262) 2023-07-14T18:08:05.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.308+0000 7f4da37fe700 1 -- 172.21.15.133:0/3157323730 learned_addr learned my addr 172.21.15.133:0/3157323730 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:08:05.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.308+0000 7f4da37fe700 1 -- 172.21.15.133:0/3157323730 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d940095f0 con 0x7f4da4105740 2023-07-14T18:08:05.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.309+0000 7f4da37fe700 1 --2- 172.21.15.133:0/3157323730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4da4105740 0x7f4da4072940 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4d940092e0 tx=0x7f4d94004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:08:05.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.309+0000 7f4da0ff9700 1 -- 172.21.15.133:0/3157323730 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d9401d070 con 0x7f4da4105740 2023-07-14T18:08:05.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.309+0000 7f4da0ff9700 1 -- 172.21.15.133:0/3157323730 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d9400bc10 con 0x7f4da4105740 2023-07-14T18:08:05.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.309+0000 7f4da9c2a700 1 -- 172.21.15.133:0/3157323730 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da4072e80 con 0x7f4da4105740 2023-07-14T18:08:05.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.309+0000 7f4da0ff9700 1 -- 172.21.15.133:0/3157323730 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d94003930 con 0x7f4da4105740 2023-07-14T18:08:05.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.309+0000 7f4da9c2a700 1 -- 172.21.15.133:0/3157323730 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da4073380 con 0x7f4da4105740 2023-07-14T18:08:05.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.310+0000 7f4da0ff9700 1 -- 172.21.15.133:0/3157323730 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f4d94003bb0 con 0x7f4da4105740 2023-07-14T18:08:05.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.310+0000 7f4da9c2a700 1 -- 172.21.15.133:0/3157323730 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da4104a30 con 0x7f4da4105740 2023-07-14T18:08:05.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.310+0000 7f4da0ff9700 1 --2- 172.21.15.133:0/3157323730 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4d8c0387b0 0x7f4d8c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:05.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.310+0000 7f4da0ff9700 1 -- 172.21.15.133:0/3157323730 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4d9404c520 con 0x7f4da4105740 2023-07-14T18:08:05.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.311+0000 7f4da2ffd700 1 --2- 172.21.15.133:0/3157323730 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4d8c0387b0 0x7f4d8c03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:05.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.311+0000 7f4da2ffd700 1 --2- 172.21.15.133:0/3157323730 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4d8c0387b0 0x7f4d8c03ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4d98006fd0 tx=0x7f4d98006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:08:05.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.316+0000 7f4da0ff9700 1 -- 172.21.15.133:0/3157323730 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f4d9400ba70 con 0x7f4da4105740 2023-07-14T18:08:05.549 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:05.515+0000 7f4da9c2a700 1 -- 172.21.15.133:0/3157323730 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}) v1 -- 0x7f4da4073610 con 0x7f4d8c0387b0 2023-07-14T18:08:06.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:05 smithi133 ceph-mon[126384]: Updating smithi133:/var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T18:08:06.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:05 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:06.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:05 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:06.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:05 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:07.146 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:06 smithi133 ceph-mon[126384]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:08:07.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:07.834+0000 7f4da0ff9700 1 -- 172.21.15.133:0/3157323730 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f4da4073610 con 0x7f4d8c0387b0 2023-07-14T18:08:07.870 INFO:teuthology.orchestra.run.smithi133.stdout:Added host 'smithi161' with addr '172.21.15.161' 2023-07-14T18:08:07.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:07.837+0000 7f4da9c2a700 1 -- 172.21.15.133:0/3157323730 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4d8c0387b0 msgr2=0x7f4d8c03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:07.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:07.837+0000 7f4da9c2a700 1 --2- 172.21.15.133:0/3157323730 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4d8c0387b0 0x7f4d8c03ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4d98006fd0 tx=0x7f4d98006e40 comp rx=0 tx=0).stop 2023-07-14T18:08:07.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:07.837+0000 7f4da9c2a700 1 -- 172.21.15.133:0/3157323730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4da4105740 msgr2=0x7f4da4072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:07.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:07.837+0000 7f4da9c2a700 1 --2- 172.21.15.133:0/3157323730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4da4105740 0x7f4da4072940 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4d940092e0 tx=0x7f4d94004ff0 comp rx=0 tx=0).stop 2023-07-14T18:08:07.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:07.838+0000 7f4da9c2a700 1 -- 172.21.15.133:0/3157323730 shutdown_connections 2023-07-14T18:08:07.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:07.838+0000 7f4da9c2a700 1 --2- 172.21.15.133:0/3157323730 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4d8c0387b0 0x7f4d8c03ac60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:07.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:07.838+0000 7f4da9c2a700 1 --2- 172.21.15.133:0/3157323730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4da4105740 0x7f4da4072940 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:07.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:07.838+0000 7f4da9c2a700 1 -- 172.21.15.133:0/3157323730 >> 172.21.15.133:0/3157323730 conn(0x7f4da40fb280 msgr2=0x7f4da40fbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:08:07.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:07.838+0000 7f4da9c2a700 1 -- 172.21.15.133:0/3157323730 shutdown_connections 2023-07-14T18:08:07.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:07.838+0000 7f4da9c2a700 1 -- 172.21.15.133:0/3157323730 wait complete. 2023-07-14T18:08:07.998 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:07 smithi133 ceph-mon[126384]: Deploying cephadm binary to smithi161 2023-07-14T18:08:08.306 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph orch host ls --format=json 2023-07-14T18:08:09.023 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:08:09.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:08 smithi133 ceph-mon[126384]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:09.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:08 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:09.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:08 smithi133 ceph-mon[126384]: Added host smithi161 2023-07-14T18:08:09.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:08 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:08:09.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:08 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:10.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:09 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:10.415 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.380+0000 7f0e9604c700 1 -- 172.21.15.133:0/1313767532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e90105740 msgr2=0x7f0e90105b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:10.415 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.380+0000 7f0e9604c700 1 --2- 172.21.15.133:0/1313767532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e90105740 0x7f0e90105b10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0e80009910 tx=0x7f0e80009c20 comp rx=0 tx=0).stop 2023-07-14T18:08:10.416 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.380+0000 7f0e9604c700 1 -- 172.21.15.133:0/1313767532 shutdown_connections 2023-07-14T18:08:10.416 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.380+0000 7f0e9604c700 1 --2- 172.21.15.133:0/1313767532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e90105740 0x7f0e90105b10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:10.416 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.380+0000 7f0e9604c700 1 -- 172.21.15.133:0/1313767532 >> 172.21.15.133:0/1313767532 conn(0x7f0e900fb280 msgr2=0x7f0e900fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:08:10.416 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.380+0000 7f0e9604c700 1 -- 172.21.15.133:0/1313767532 shutdown_connections 2023-07-14T18:08:10.416 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.380+0000 7f0e9604c700 1 -- 172.21.15.133:0/1313767532 wait complete. 2023-07-14T18:08:10.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.381+0000 7f0e9604c700 1 Processor -- start 2023-07-14T18:08:10.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.381+0000 7f0e9604c700 1 -- start start 2023-07-14T18:08:10.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.382+0000 7f0e9604c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e90105740 0x7f0e90072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:10.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.382+0000 7f0e9604c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e900748c0 con 0x7f0e90105740 2023-07-14T18:08:10.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.382+0000 7f0e8f7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e90105740 0x7f0e90072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:10.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.382+0000 7f0e8f7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e90105740 0x7f0e90072940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43786/0 (socket says 172.21.15.133:43786) 2023-07-14T18:08:10.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.382+0000 7f0e8f7fe700 1 -- 172.21.15.133:0/2505474113 learned_addr learned my addr 172.21.15.133:0/2505474113 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:08:10.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.382+0000 7f0e8f7fe700 1 -- 172.21.15.133:0/2505474113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e800095f0 con 0x7f0e90105740 2023-07-14T18:08:10.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.382+0000 7f0e8f7fe700 1 --2- 172.21.15.133:0/2505474113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e90105740 0x7f0e90072940 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0e80004f40 tx=0x7f0e80005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:08:10.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.383+0000 7f0e8cff9700 1 -- 172.21.15.133:0/2505474113 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e8001d070 con 0x7f0e90105740 2023-07-14T18:08:10.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.383+0000 7f0e8cff9700 1 -- 172.21.15.133:0/2505474113 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e8000be20 con 0x7f0e90105740 2023-07-14T18:08:10.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.383+0000 7f0e9604c700 1 -- 172.21.15.133:0/2505474113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e90072e80 con 0x7f0e90105740 2023-07-14T18:08:10.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.383+0000 7f0e8cff9700 1 -- 172.21.15.133:0/2505474113 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e80022d00 con 0x7f0e90105740 2023-07-14T18:08:10.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.383+0000 7f0e9604c700 1 -- 172.21.15.133:0/2505474113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e90073380 con 0x7f0e90105740 2023-07-14T18:08:10.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.384+0000 7f0e8cff9700 1 -- 172.21.15.133:0/2505474113 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f0e80022500 con 0x7f0e90105740 2023-07-14T18:08:10.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.384+0000 7f0e9604c700 1 -- 172.21.15.133:0/2505474113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e90104a30 con 0x7f0e90105740 2023-07-14T18:08:10.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.384+0000 7f0e8cff9700 1 --2- 172.21.15.133:0/2505474113 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0e78038800 0x7f0e7803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:10.421 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.384+0000 7f0e8cff9700 1 -- 172.21.15.133:0/2505474113 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0e8004ca30 con 0x7f0e90105740 2023-07-14T18:08:10.421 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.385+0000 7f0e8effd700 1 --2- 172.21.15.133:0/2505474113 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0e78038800 0x7f0e7803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:10.421 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.385+0000 7f0e8effd700 1 --2- 172.21.15.133:0/2505474113 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0e78038800 0x7f0e7803acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0e84006fd0 tx=0x7f0e84006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:08:10.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.390+0000 7f0e8cff9700 1 -- 172.21.15.133:0/2505474113 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f0e8002bba0 con 0x7f0e90105740 2023-07-14T18:08:10.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.588+0000 7f0e9604c700 1 -- 172.21.15.133:0/2505474113 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0e90073610 con 0x7f0e78038800 2023-07-14T18:08:10.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.590+0000 7f0e8cff9700 1 -- 172.21.15.133:0/2505474113 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f0e90073610 con 0x7f0e78038800 2023-07-14T18:08:10.625 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:08:10.625 INFO:teuthology.orchestra.run.smithi133.stdout:[{"addr": "172.21.15.133", "hostname": "smithi133", "labels": [], "status": ""}, {"addr": "172.21.15.161", "hostname": "smithi161", "labels": [], "status": ""}] 2023-07-14T18:08:10.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.592+0000 7f0e9604c700 1 -- 172.21.15.133:0/2505474113 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0e78038800 msgr2=0x7f0e7803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:10.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.592+0000 7f0e9604c700 1 --2- 172.21.15.133:0/2505474113 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0e78038800 0x7f0e7803acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0e84006fd0 tx=0x7f0e84006e40 comp rx=0 tx=0).stop 2023-07-14T18:08:10.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.592+0000 7f0e9604c700 1 -- 172.21.15.133:0/2505474113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e90105740 msgr2=0x7f0e90072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:10.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.592+0000 7f0e9604c700 1 --2- 172.21.15.133:0/2505474113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e90105740 0x7f0e90072940 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0e80004f40 tx=0x7f0e80005020 comp rx=0 tx=0).stop 2023-07-14T18:08:10.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.592+0000 7f0e9604c700 1 -- 172.21.15.133:0/2505474113 shutdown_connections 2023-07-14T18:08:10.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.592+0000 7f0e9604c700 1 --2- 172.21.15.133:0/2505474113 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0e78038800 0x7f0e7803acb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:10.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.593+0000 7f0e9604c700 1 --2- 172.21.15.133:0/2505474113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e90105740 0x7f0e90072940 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:10.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.593+0000 7f0e9604c700 1 -- 172.21.15.133:0/2505474113 >> 172.21.15.133:0/2505474113 conn(0x7f0e900fb280 msgr2=0x7f0e900fbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:08:10.629 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.593+0000 7f0e9604c700 1 -- 172.21.15.133:0/2505474113 shutdown_connections 2023-07-14T18:08:10.629 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:10.593+0000 7f0e9604c700 1 -- 172.21.15.133:0/2505474113 wait complete. 2023-07-14T18:08:11.214 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:10 smithi133 ceph-mon[126384]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:11.214 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:10 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:11.216 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi171 2023-07-14T18:08:11.216 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2023-07-14T18:08:11.216 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/etc/ceph/ceph.conf 2023-07-14T18:08:11.237 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2023-07-14T18:08:11.237 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-14T18:08:11.296 INFO:tasks.cephadm:Adding host smithi171 to orchestrator... 2023-07-14T18:08:11.296 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph orch host add smithi171 2023-07-14T18:08:12.013 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:08:12.276 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:11 smithi133 ceph-mon[126384]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T18:08:13.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:12 smithi133 ceph-mon[126384]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:13.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:12 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:13.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:12 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:13.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:12 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:13.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:12 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:13.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:12 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:08:13.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:12 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:13.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:12 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:08:13.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:12 smithi133 ceph-mon[126384]: Updating smithi161:/etc/ceph/ceph.conf 2023-07-14T18:08:13.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:12 smithi133 ceph-mon[126384]: Updating smithi161:/var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T18:08:13.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.475+0000 7f8dc6bb3700 1 -- 172.21.15.133:0/2493695935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dc0105740 msgr2=0x7f8dc0105b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:13.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.475+0000 7f8dc6bb3700 1 --2- 172.21.15.133:0/2493695935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dc0105740 0x7f8dc0105b10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc009910 tx=0x7f8dbc009c20 comp rx=0 tx=0).stop 2023-07-14T18:08:13.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.475+0000 7f8dc6bb3700 1 -- 172.21.15.133:0/2493695935 shutdown_connections 2023-07-14T18:08:13.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.475+0000 7f8dc6bb3700 1 --2- 172.21.15.133:0/2493695935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dc0105740 0x7f8dc0105b10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:13.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.475+0000 7f8dc6bb3700 1 -- 172.21.15.133:0/2493695935 >> 172.21.15.133:0/2493695935 conn(0x7f8dc00fb280 msgr2=0x7f8dc00fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:08:13.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.475+0000 7f8dc6bb3700 1 -- 172.21.15.133:0/2493695935 shutdown_connections 2023-07-14T18:08:13.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.475+0000 7f8dc6bb3700 1 -- 172.21.15.133:0/2493695935 wait complete. 2023-07-14T18:08:13.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.476+0000 7f8dc6bb3700 1 Processor -- start 2023-07-14T18:08:13.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.476+0000 7f8dc6bb3700 1 -- start start 2023-07-14T18:08:13.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.477+0000 7f8dc6bb3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dc0105740 0x7f8dc019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:13.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.477+0000 7f8dc6bb3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc019aed0 con 0x7f8dc0105740 2023-07-14T18:08:13.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.477+0000 7f8dc494f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dc0105740 0x7f8dc019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:13.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.477+0000 7f8dc494f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dc0105740 0x7f8dc019a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43802/0 (socket says 172.21.15.133:43802) 2023-07-14T18:08:13.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.477+0000 7f8dc494f700 1 -- 172.21.15.133:0/285362744 learned_addr learned my addr 172.21.15.133:0/285362744 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:08:13.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.477+0000 7f8dc494f700 1 -- 172.21.15.133:0/285362744 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dbc0095f0 con 0x7f8dc0105740 2023-07-14T18:08:13.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.477+0000 7f8dc494f700 1 --2- 172.21.15.133:0/285362744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dc0105740 0x7f8dc019a990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc004f40 tx=0x7f8dbc005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:08:13.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.478+0000 7f8db5ffb700 1 -- 172.21.15.133:0/285362744 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8dbc01d070 con 0x7f8dc0105740 2023-07-14T18:08:13.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.478+0000 7f8db5ffb700 1 -- 172.21.15.133:0/285362744 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8dbc00be20 con 0x7f8dc0105740 2023-07-14T18:08:13.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.478+0000 7f8dc6bb3700 1 -- 172.21.15.133:0/285362744 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc019b0d0 con 0x7f8dc0105740 2023-07-14T18:08:13.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.478+0000 7f8db5ffb700 1 -- 172.21.15.133:0/285362744 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8dbc022d00 con 0x7f8dc0105740 2023-07-14T18:08:13.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.478+0000 7f8dc6bb3700 1 -- 172.21.15.133:0/285362744 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc0194a10 con 0x7f8dc0105740 2023-07-14T18:08:13.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.479+0000 7f8db5ffb700 1 -- 172.21.15.133:0/285362744 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f8dbc00b960 con 0x7f8dc0105740 2023-07-14T18:08:13.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.479+0000 7f8dc6bb3700 1 -- 172.21.15.133:0/285362744 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc0061c70 con 0x7f8dc0105740 2023-07-14T18:08:13.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.480+0000 7f8db5ffb700 1 --2- 172.21.15.133:0/285362744 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8da8038800 0x7f8da803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:13.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.480+0000 7f8db5ffb700 1 -- 172.21.15.133:0/285362744 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8dbc04ce30 con 0x7f8dc0105740 2023-07-14T18:08:13.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.480+0000 7f8db7fff700 1 --2- 172.21.15.133:0/285362744 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8da8038800 0x7f8da803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:13.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.480+0000 7f8db7fff700 1 --2- 172.21.15.133:0/285362744 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8da8038800 0x7f8da803acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8db0006fd0 tx=0x7f8db0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:08:13.519 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.484+0000 7f8db5ffb700 1 -- 172.21.15.133:0/285362744 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f8dbc051070 con 0x7f8dc0105740 2023-07-14T18:08:13.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:13.702+0000 7f8dc6bb3700 1 -- 172.21.15.133:0/285362744 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}) v1 -- 0x7f8dc0194cc0 con 0x7f8da8038800 2023-07-14T18:08:14.897 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:14 smithi133 ceph-mon[126384]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2023-07-14T18:08:14.897 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:14 smithi133 ceph-mon[126384]: Updating smithi161:/var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T18:08:14.897 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:14 smithi133 ceph-mon[126384]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:14.897 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:14 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:14.897 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:14 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:14.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:14 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:14.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:14 smithi133 ceph-mon[126384]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:08:16.143 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:16.108+0000 7f8db5ffb700 1 -- 172.21.15.133:0/285362744 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f8dc0194cc0 con 0x7f8da8038800 2023-07-14T18:08:16.144 INFO:teuthology.orchestra.run.smithi133.stdout:Added host 'smithi171' with addr '172.21.15.171' 2023-07-14T18:08:16.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:16.111+0000 7f8dc6bb3700 1 -- 172.21.15.133:0/285362744 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8da8038800 msgr2=0x7f8da803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:16.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:16.111+0000 7f8dc6bb3700 1 --2- 172.21.15.133:0/285362744 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8da8038800 0x7f8da803acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8db0006fd0 tx=0x7f8db0006e40 comp rx=0 tx=0).stop 2023-07-14T18:08:16.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:16.111+0000 7f8dc6bb3700 1 -- 172.21.15.133:0/285362744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dc0105740 msgr2=0x7f8dc019a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:16.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:16.111+0000 7f8dc6bb3700 1 --2- 172.21.15.133:0/285362744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dc0105740 0x7f8dc019a990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc004f40 tx=0x7f8dbc005020 comp rx=0 tx=0).stop 2023-07-14T18:08:16.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:16.111+0000 7f8dc6bb3700 1 -- 172.21.15.133:0/285362744 shutdown_connections 2023-07-14T18:08:16.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:16.111+0000 7f8dc6bb3700 1 --2- 172.21.15.133:0/285362744 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8da8038800 0x7f8da803acb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:16.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:16.111+0000 7f8dc6bb3700 1 --2- 172.21.15.133:0/285362744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dc0105740 0x7f8dc019a990 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:16.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:16.111+0000 7f8dc6bb3700 1 -- 172.21.15.133:0/285362744 >> 172.21.15.133:0/285362744 conn(0x7f8dc00fb280 msgr2=0x7f8dc00fbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:08:16.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:16.111+0000 7f8dc6bb3700 1 -- 172.21.15.133:0/285362744 shutdown_connections 2023-07-14T18:08:16.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:16.111+0000 7f8dc6bb3700 1 -- 172.21.15.133:0/285362744 wait complete. 2023-07-14T18:08:16.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:15 smithi133 ceph-mon[126384]: Deploying cephadm binary to smithi171 2023-07-14T18:08:16.596 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph orch host ls --format=json 2023-07-14T18:08:16.897 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:16 smithi133 ceph-mon[126384]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:16.897 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:16 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:16.897 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:16 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:08:16.897 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:16 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:17.312 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:08:17.864 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:17 smithi133 ceph-mon[126384]: Added host smithi171 2023-07-14T18:08:17.864 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:17 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:18.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.696+0000 7f43e213c700 1 -- 172.21.15.133:0/3662976597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43dc105740 msgr2=0x7f43dc105b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:18.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.696+0000 7f43e213c700 1 --2- 172.21.15.133:0/3662976597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43dc105740 0x7f43dc105b10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009910 tx=0x7f43cc009c20 comp rx=0 tx=0).stop 2023-07-14T18:08:18.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.696+0000 7f43e213c700 1 -- 172.21.15.133:0/3662976597 shutdown_connections 2023-07-14T18:08:18.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.696+0000 7f43e213c700 1 --2- 172.21.15.133:0/3662976597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43dc105740 0x7f43dc105b10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:18.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.696+0000 7f43e213c700 1 -- 172.21.15.133:0/3662976597 >> 172.21.15.133:0/3662976597 conn(0x7f43dc0fb280 msgr2=0x7f43dc0fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:08:18.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.696+0000 7f43e213c700 1 -- 172.21.15.133:0/3662976597 shutdown_connections 2023-07-14T18:08:18.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.696+0000 7f43e213c700 1 -- 172.21.15.133:0/3662976597 wait complete. 2023-07-14T18:08:18.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.697+0000 7f43e213c700 1 Processor -- start 2023-07-14T18:08:18.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.698+0000 7f43e213c700 1 -- start start 2023-07-14T18:08:18.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.698+0000 7f43e213c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43dc105740 0x7f43dc19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:18.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.698+0000 7f43e213c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc19aed0 con 0x7f43dc105740 2023-07-14T18:08:18.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.698+0000 7f43db7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43dc105740 0x7f43dc19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:18.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.698+0000 7f43db7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43dc105740 0x7f43dc19a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46532/0 (socket says 172.21.15.133:46532) 2023-07-14T18:08:18.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.698+0000 7f43db7fe700 1 -- 172.21.15.133:0/2205424477 learned_addr learned my addr 172.21.15.133:0/2205424477 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:08:18.735 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.699+0000 7f43db7fe700 1 -- 172.21.15.133:0/2205424477 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc0095f0 con 0x7f43dc105740 2023-07-14T18:08:18.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.699+0000 7f43db7fe700 1 --2- 172.21.15.133:0/2205424477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43dc105740 0x7f43dc19a990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0092e0 tx=0x7f43cc004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:08:18.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.699+0000 7f43d8ff9700 1 -- 172.21.15.133:0/2205424477 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43cc01d070 con 0x7f43dc105740 2023-07-14T18:08:18.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.699+0000 7f43e213c700 1 -- 172.21.15.133:0/2205424477 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43dc19b0d0 con 0x7f43dc105740 2023-07-14T18:08:18.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.699+0000 7f43d8ff9700 1 -- 172.21.15.133:0/2205424477 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43cc00bc10 con 0x7f43dc105740 2023-07-14T18:08:18.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.699+0000 7f43d8ff9700 1 -- 172.21.15.133:0/2205424477 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43cc003930 con 0x7f43dc105740 2023-07-14T18:08:18.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.699+0000 7f43e213c700 1 -- 172.21.15.133:0/2205424477 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43dc194a10 con 0x7f43dc105740 2023-07-14T18:08:18.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.701+0000 7f43d8ff9700 1 -- 172.21.15.133:0/2205424477 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f43cc003bb0 con 0x7f43dc105740 2023-07-14T18:08:18.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.701+0000 7f43e213c700 1 -- 172.21.15.133:0/2205424477 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43dc061c70 con 0x7f43dc105740 2023-07-14T18:08:18.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.701+0000 7f43d8ff9700 1 --2- 172.21.15.133:0/2205424477 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f43c40387b0 0x7f43c403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:18.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.701+0000 7f43d8ff9700 1 -- 172.21.15.133:0/2205424477 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f43cc04c520 con 0x7f43dc105740 2023-07-14T18:08:18.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.701+0000 7f43daffd700 1 --2- 172.21.15.133:0/2205424477 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f43c40387b0 0x7f43c403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:18.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.702+0000 7f43daffd700 1 --2- 172.21.15.133:0/2205424477 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f43c40387b0 0x7f43c403ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f43d0006fd0 tx=0x7f43d0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:08:18.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.706+0000 7f43d8ff9700 1 -- 172.21.15.133:0/2205424477 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f43cc00ba70 con 0x7f43dc105740 2023-07-14T18:08:18.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.907+0000 7f43e213c700 1 -- 172.21.15.133:0/2205424477 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f43dc195440 con 0x7f43c40387b0 2023-07-14T18:08:18.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.908+0000 7f43d8ff9700 1 -- 172.21.15.133:0/2205424477 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+241 (secure 0 0 0) 0x7f43dc195440 con 0x7f43c40387b0 2023-07-14T18:08:18.944 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:08:18.944 INFO:teuthology.orchestra.run.smithi133.stdout:[{"addr": "172.21.15.133", "hostname": "smithi133", "labels": [], "status": ""}, {"addr": "172.21.15.161", "hostname": "smithi161", "labels": [], "status": ""}, {"addr": "172.21.15.171", "hostname": "smithi171", "labels": [], "status": ""}] 2023-07-14T18:08:18.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.910+0000 7f43e213c700 1 -- 172.21.15.133:0/2205424477 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f43c40387b0 msgr2=0x7f43c403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:18.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.910+0000 7f43e213c700 1 --2- 172.21.15.133:0/2205424477 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f43c40387b0 0x7f43c403ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f43d0006fd0 tx=0x7f43d0006e40 comp rx=0 tx=0).stop 2023-07-14T18:08:18.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.910+0000 7f43e213c700 1 -- 172.21.15.133:0/2205424477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43dc105740 msgr2=0x7f43dc19a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:18.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.910+0000 7f43e213c700 1 --2- 172.21.15.133:0/2205424477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43dc105740 0x7f43dc19a990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0092e0 tx=0x7f43cc004ff0 comp rx=0 tx=0).stop 2023-07-14T18:08:18.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.910+0000 7f43e213c700 1 -- 172.21.15.133:0/2205424477 shutdown_connections 2023-07-14T18:08:18.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.910+0000 7f43e213c700 1 --2- 172.21.15.133:0/2205424477 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f43c40387b0 0x7f43c403ac60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:18.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.910+0000 7f43e213c700 1 --2- 172.21.15.133:0/2205424477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43dc105740 0x7f43dc19a990 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:18.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.910+0000 7f43e213c700 1 -- 172.21.15.133:0/2205424477 >> 172.21.15.133:0/2205424477 conn(0x7f43dc0fb280 msgr2=0x7f43dc0fbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:08:18.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.911+0000 7f43e213c700 1 -- 172.21.15.133:0/2205424477 shutdown_connections 2023-07-14T18:08:18.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:18.911+0000 7f43e213c700 1 -- 172.21.15.133:0/2205424477 wait complete. 2023-07-14T18:08:19.079 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:18 smithi133 ceph-mon[126384]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:19.080 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:18 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:19.387 INFO:tasks.cephadm:Setting crush tunables to default 2023-07-14T18:08:19.387 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph osd crush tunables default 2023-07-14T18:08:20.102 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:08:20.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:19 smithi133 ceph-mon[126384]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T18:08:20.897 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:20 smithi133 ceph-mon[126384]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:21.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.451+0000 7ff61edc3700 1 -- 172.21.15.133:0/3366397928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff618105740 msgr2=0x7ff618105b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:21.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.451+0000 7ff61edc3700 1 --2- 172.21.15.133:0/3366397928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff618105740 0x7ff618105b10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009910 tx=0x7ff60c009c20 comp rx=0 tx=0).stop 2023-07-14T18:08:21.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.452+0000 7ff61edc3700 1 -- 172.21.15.133:0/3366397928 shutdown_connections 2023-07-14T18:08:21.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.452+0000 7ff61edc3700 1 --2- 172.21.15.133:0/3366397928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff618105740 0x7ff618105b10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:21.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.452+0000 7ff61edc3700 1 -- 172.21.15.133:0/3366397928 >> 172.21.15.133:0/3366397928 conn(0x7ff6180fb280 msgr2=0x7ff6180fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:08:21.489 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.452+0000 7ff61edc3700 1 -- 172.21.15.133:0/3366397928 shutdown_connections 2023-07-14T18:08:21.489 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.452+0000 7ff61edc3700 1 -- 172.21.15.133:0/3366397928 wait complete. 2023-07-14T18:08:21.490 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.453+0000 7ff61edc3700 1 Processor -- start 2023-07-14T18:08:21.490 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.453+0000 7ff61edc3700 1 -- start start 2023-07-14T18:08:21.490 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.453+0000 7ff61edc3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff618105740 0x7ff618196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:21.490 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.453+0000 7ff61edc3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6181972d0 con 0x7ff618105740 2023-07-14T18:08:21.490 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.454+0000 7ff61cb5f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff618105740 0x7ff618196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:21.490 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.454+0000 7ff61cb5f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff618105740 0x7ff618196d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46562/0 (socket says 172.21.15.133:46562) 2023-07-14T18:08:21.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.454+0000 7ff61cb5f700 1 -- 172.21.15.133:0/1576027545 learned_addr learned my addr 172.21.15.133:0/1576027545 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:08:21.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.454+0000 7ff61cb5f700 1 -- 172.21.15.133:0/1576027545 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60c0095f0 con 0x7ff618105740 2023-07-14T18:08:21.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.454+0000 7ff61cb5f700 1 --2- 172.21.15.133:0/1576027545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff618105740 0x7ff618196d90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0092e0 tx=0x7ff60c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:08:21.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.454+0000 7ff615ffb700 1 -- 172.21.15.133:0/1576027545 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff60c01d070 con 0x7ff618105740 2023-07-14T18:08:21.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.454+0000 7ff615ffb700 1 -- 172.21.15.133:0/1576027545 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff60c00bc10 con 0x7ff618105740 2023-07-14T18:08:21.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.454+0000 7ff61edc3700 1 -- 172.21.15.133:0/1576027545 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6181974d0 con 0x7ff618105740 2023-07-14T18:08:21.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.454+0000 7ff615ffb700 1 -- 172.21.15.133:0/1576027545 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff60c003930 con 0x7ff618105740 2023-07-14T18:08:21.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.455+0000 7ff61edc3700 1 -- 172.21.15.133:0/1576027545 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff618197970 con 0x7ff618105740 2023-07-14T18:08:21.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.456+0000 7ff615ffb700 1 -- 172.21.15.133:0/1576027545 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7ff60c003b60 con 0x7ff618105740 2023-07-14T18:08:21.493 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.456+0000 7ff61edc3700 1 -- 172.21.15.133:0/1576027545 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff618061c70 con 0x7ff618105740 2023-07-14T18:08:21.493 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.456+0000 7ff615ffb700 1 --2- 172.21.15.133:0/1576027545 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff608038800 0x7ff60803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:21.493 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.456+0000 7ff615ffb700 1 -- 172.21.15.133:0/1576027545 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff60c04c3c0 con 0x7ff618105740 2023-07-14T18:08:21.493 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.456+0000 7ff617fff700 1 --2- 172.21.15.133:0/1576027545 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff608038800 0x7ff60803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:21.493 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.457+0000 7ff617fff700 1 --2- 172.21.15.133:0/1576027545 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff608038800 0x7ff60803acb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff604006fd0 tx=0x7ff604006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:08:21.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.461+0000 7ff615ffb700 1 -- 172.21.15.133:0/1576027545 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7ff60c00ba70 con 0x7ff618105740 2023-07-14T18:08:21.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.669+0000 7ff61edc3700 1 -- 172.21.15.133:0/1576027545 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7ff618102540 con 0x7ff618105740 2023-07-14T18:08:21.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.952+0000 7ff615ffb700 1 -- 172.21.15.133:0/1576027545 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7ff60c026020 con 0x7ff618105740 2023-07-14T18:08:21.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.954+0000 7ff61edc3700 1 -- 172.21.15.133:0/1576027545 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff608038800 msgr2=0x7ff60803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:21.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.954+0000 7ff61edc3700 1 --2- 172.21.15.133:0/1576027545 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff608038800 0x7ff60803acb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff604006fd0 tx=0x7ff604006e40 comp rx=0 tx=0).stop 2023-07-14T18:08:21.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.954+0000 7ff61edc3700 1 -- 172.21.15.133:0/1576027545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff618105740 msgr2=0x7ff618196d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:21.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.954+0000 7ff61edc3700 1 --2- 172.21.15.133:0/1576027545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff618105740 0x7ff618196d90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0092e0 tx=0x7ff60c004ff0 comp rx=0 tx=0).stop 2023-07-14T18:08:21.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.954+0000 7ff61edc3700 1 -- 172.21.15.133:0/1576027545 shutdown_connections 2023-07-14T18:08:21.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.954+0000 7ff61edc3700 1 --2- 172.21.15.133:0/1576027545 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff608038800 0x7ff60803acb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:21.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.954+0000 7ff61edc3700 1 --2- 172.21.15.133:0/1576027545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff618105740 0x7ff618196d90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:21.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.954+0000 7ff61edc3700 1 -- 172.21.15.133:0/1576027545 >> 172.21.15.133:0/1576027545 conn(0x7ff6180fb280 msgr2=0x7ff618074340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:08:21.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.955+0000 7ff61edc3700 1 -- 172.21.15.133:0/1576027545 shutdown_connections 2023-07-14T18:08:21.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:21.955+0000 7ff61edc3700 1 -- 172.21.15.133:0/1576027545 wait complete. 2023-07-14T18:08:21.991 INFO:teuthology.orchestra.run.smithi133.stderr:adjusted tunables profile to default 2023-07-14T18:08:22.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:21 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:22.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:21 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:22.398 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:21 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:22.398 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:21 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:22.398 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:21 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:08:22.398 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:21 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:22.398 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:21 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:08:22.398 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:21 smithi133 ceph-mon[126384]: Updating smithi171:/etc/ceph/ceph.conf 2023-07-14T18:08:22.398 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:21 smithi133 ceph-mon[126384]: Updating smithi171:/var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T18:08:22.398 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:21 smithi133 ceph-mon[126384]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:22.399 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:21 smithi133 ceph-mon[126384]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2023-07-14T18:08:22.399 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:21 smithi133 ceph-mon[126384]: Updating smithi171:/var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T18:08:22.399 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:21 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1576027545' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-14T18:08:22.399 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:21 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:22.399 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:21 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:22.399 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:21 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:22.475 INFO:tasks.cephadm:Adding mon.a on smithi133 2023-07-14T18:08:22.476 INFO:tasks.cephadm:Adding mon.b on smithi161 2023-07-14T18:08:22.476 INFO:tasks.cephadm:Adding mon.c on smithi171 2023-07-14T18:08:22.476 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph orch apply mon '3;smithi133:172.21.15.133=a;smithi161:172.21.15.161=b;smithi171:172.21.15.171=c' 2023-07-14T18:08:23.055 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T18:08:23.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:22 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1576027545' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-14T18:08:23.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:22 smithi133 ceph-mon[126384]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T18:08:24.345 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.310+0000 7f92c0e1a700 1 -- 172.21.15.171:0/2030097746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92bc0ff690 msgr2=0x7f92bc0ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:24.346 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.310+0000 7f92c0e1a700 1 --2- 172.21.15.171:0/2030097746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92bc0ff690 0x7f92bc0ffa60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f92a4009910 tx=0x7f92a4009c20 comp rx=0 tx=0).stop 2023-07-14T18:08:24.346 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.310+0000 7f92c0e1a700 1 -- 172.21.15.171:0/2030097746 shutdown_connections 2023-07-14T18:08:24.346 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.310+0000 7f92c0e1a700 1 --2- 172.21.15.171:0/2030097746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92bc0ff690 0x7f92bc0ffa60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:24.346 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.310+0000 7f92c0e1a700 1 -- 172.21.15.171:0/2030097746 >> 172.21.15.171:0/2030097746 conn(0x7f92bc075070 msgr2=0x7f92bc075470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:08:24.346 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.310+0000 7f92c0e1a700 1 -- 172.21.15.171:0/2030097746 shutdown_connections 2023-07-14T18:08:24.346 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.310+0000 7f92c0e1a700 1 -- 172.21.15.171:0/2030097746 wait complete. 2023-07-14T18:08:24.347 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.310+0000 7f92c0e1a700 1 Processor -- start 2023-07-14T18:08:24.348 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.311+0000 7f92c0e1a700 1 -- start start 2023-07-14T18:08:24.348 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.311+0000 7f92c0e1a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92bc0ff690 0x7f92bc195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:24.348 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.311+0000 7f92c0e1a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc196000 con 0x7f92bc0ff690 2023-07-14T18:08:24.348 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.311+0000 7f92ba59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92bc0ff690 0x7f92bc195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:24.348 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.311+0000 7f92ba59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92bc0ff690 0x7f92bc195ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.171:57498/0 (socket says 172.21.15.171:57498) 2023-07-14T18:08:24.348 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.311+0000 7f92ba59c700 1 -- 172.21.15.171:0/1605366733 learned_addr learned my addr 172.21.15.171:0/1605366733 (peer_addr_for_me v2:172.21.15.171:0/0) 2023-07-14T18:08:24.349 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.311+0000 7f92ba59c700 1 -- 172.21.15.171:0/1605366733 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a40095f0 con 0x7f92bc0ff690 2023-07-14T18:08:24.349 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.312+0000 7f92ba59c700 1 --2- 172.21.15.171:0/1605366733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92bc0ff690 0x7f92bc195ac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f92a4004f40 tx=0x7f92a4005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:08:24.349 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.312+0000 7f92b37fe700 1 -- 172.21.15.171:0/1605366733 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f92a401d070 con 0x7f92bc0ff690 2023-07-14T18:08:24.349 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.312+0000 7f92c0e1a700 1 -- 172.21.15.171:0/1605366733 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92bc196200 con 0x7f92bc0ff690 2023-07-14T18:08:24.349 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.312+0000 7f92b37fe700 1 -- 172.21.15.171:0/1605366733 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92a400be20 con 0x7f92bc0ff690 2023-07-14T18:08:24.349 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.312+0000 7f92b37fe700 1 -- 172.21.15.171:0/1605366733 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f92a4022d00 con 0x7f92bc0ff690 2023-07-14T18:08:24.349 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.312+0000 7f92c0e1a700 1 -- 172.21.15.171:0/1605366733 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92bc196620 con 0x7f92bc0ff690 2023-07-14T18:08:24.350 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.313+0000 7f92b37fe700 1 -- 172.21.15.171:0/1605366733 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f92a4022500 con 0x7f92bc0ff690 2023-07-14T18:08:24.350 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.313+0000 7f92c0e1a700 1 -- 172.21.15.171:0/1605366733 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92bc103bf0 con 0x7f92bc0ff690 2023-07-14T18:08:24.350 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.313+0000 7f92b37fe700 1 --2- 172.21.15.171:0/1605366733 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f92a80387b0 0x7f92a803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:24.351 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.313+0000 7f92b37fe700 1 -- 172.21.15.171:0/1605366733 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f92a404c7d0 con 0x7f92bc0ff690 2023-07-14T18:08:24.351 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.313+0000 7f92b9d9b700 1 --2- 172.21.15.171:0/1605366733 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f92a80387b0 0x7f92a803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:24.351 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.314+0000 7f92b9d9b700 1 --2- 172.21.15.171:0/1605366733 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f92a80387b0 0x7f92a803ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f92ac006fd0 tx=0x7f92ac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:08:24.353 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.319+0000 7f92b37fe700 1 -- 172.21.15.171:0/1605366733 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f92a402a540 con 0x7f92bc0ff690 2023-07-14T18:08:24.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:23 smithi133 ceph-mon[126384]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:24.546 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.510+0000 7f92c0e1a700 1 -- 172.21.15.171:0/1605366733 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi133:172.21.15.133=a;smithi161:172.21.15.161=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f92bc0fe910 con 0x7f92a80387b0 2023-07-14T18:08:24.553 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.518+0000 7f92b37fe700 1 -- 172.21.15.171:0/1605366733 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f92bc0fe910 con 0x7f92a80387b0 2023-07-14T18:08:24.553 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled mon update... 2023-07-14T18:08:24.555 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.520+0000 7f92c0e1a700 1 -- 172.21.15.171:0/1605366733 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f92a80387b0 msgr2=0x7f92a803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:24.555 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.520+0000 7f92c0e1a700 1 --2- 172.21.15.171:0/1605366733 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f92a80387b0 0x7f92a803ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f92ac006fd0 tx=0x7f92ac006e40 comp rx=0 tx=0).stop 2023-07-14T18:08:24.555 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.520+0000 7f92c0e1a700 1 -- 172.21.15.171:0/1605366733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92bc0ff690 msgr2=0x7f92bc195ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:24.556 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.520+0000 7f92c0e1a700 1 --2- 172.21.15.171:0/1605366733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92bc0ff690 0x7f92bc195ac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f92a4004f40 tx=0x7f92a4005020 comp rx=0 tx=0).stop 2023-07-14T18:08:24.556 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.520+0000 7f92c0e1a700 1 -- 172.21.15.171:0/1605366733 shutdown_connections 2023-07-14T18:08:24.556 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.520+0000 7f92c0e1a700 1 --2- 172.21.15.171:0/1605366733 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f92a80387b0 0x7f92a803ac60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:24.556 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.520+0000 7f92c0e1a700 1 --2- 172.21.15.171:0/1605366733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92bc0ff690 0x7f92bc195ac0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:24.556 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.520+0000 7f92c0e1a700 1 -- 172.21.15.171:0/1605366733 >> 172.21.15.171:0/1605366733 conn(0x7f92bc075070 msgr2=0x7f92bc0fc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:08:24.556 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.520+0000 7f92c0e1a700 1 -- 172.21.15.171:0/1605366733 shutdown_connections 2023-07-14T18:08:24.556 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:24.520+0000 7f92c0e1a700 1 -- 172.21.15.171:0/1605366733 wait complete. 2023-07-14T18:08:24.913 DEBUG:teuthology.orchestra.run.smithi161:mon.b> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mon.b.service 2023-07-14T18:08:24.916 DEBUG:teuthology.orchestra.run.smithi171:mon.c> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mon.c.service 2023-07-14T18:08:24.918 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-14T18:08:24.918 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph mon dump -f json 2023-07-14T18:08:24.947 INFO:journalctl@ceph.mon.c.smithi171.stdout:-- Logs begin at Fri 2023-07-14 17:41:48 UTC. -- 2023-07-14T18:08:24.951 INFO:journalctl@ceph.mon.b.smithi161.stdout:-- Logs begin at Fri 2023-07-14 17:41:43 UTC. -- 2023-07-14T18:08:25.557 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T18:08:25.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:25 smithi133 ceph-mon[126384]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi133:172.21.15.133=a;smithi161:172.21.15.161=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:08:25.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:25 smithi133 ceph-mon[126384]: Saving service mon spec with placement smithi133:172.21.15.133=a;smithi161:172.21.15.161=b;smithi171:172.21.15.171=c;count:3 2023-07-14T18:08:25.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:25 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:25.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:25 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:08:25.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:25 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:25.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:25 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:08:25.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:25 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:25.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:25 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T18:08:25.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:25 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:25.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:25 smithi133 ceph-mon[126384]: Deploying daemon mon.c on smithi171 2023-07-14T18:08:27.013 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.978+0000 7fe750dd0700 1 -- 172.21.15.171:0/3246799194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe74c104650 msgr2=0x7fe74c104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:27.014 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.978+0000 7fe750dd0700 1 --2- 172.21.15.171:0/3246799194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe74c104650 0x7fe74c104a20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe734009910 tx=0x7fe734009c20 comp rx=0 tx=0).stop 2023-07-14T18:08:27.014 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.978+0000 7fe750dd0700 1 -- 172.21.15.171:0/3246799194 shutdown_connections 2023-07-14T18:08:27.014 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.978+0000 7fe750dd0700 1 --2- 172.21.15.171:0/3246799194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe74c104650 0x7fe74c104a20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:27.014 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.978+0000 7fe750dd0700 1 -- 172.21.15.171:0/3246799194 >> 172.21.15.171:0/3246799194 conn(0x7fe74c0743e0 msgr2=0x7fe74c0747e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:08:27.014 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.978+0000 7fe750dd0700 1 -- 172.21.15.171:0/3246799194 shutdown_connections 2023-07-14T18:08:27.014 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.978+0000 7fe750dd0700 1 -- 172.21.15.171:0/3246799194 wait complete. 2023-07-14T18:08:27.015 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.979+0000 7fe750dd0700 1 Processor -- start 2023-07-14T18:08:27.015 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.979+0000 7fe750dd0700 1 -- start start 2023-07-14T18:08:27.015 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.979+0000 7fe750dd0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe74c104650 0x7fe74c195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:27.015 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.979+0000 7fe750dd0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c195f80 con 0x7fe74c104650 2023-07-14T18:08:27.016 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.980+0000 7fe74a59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe74c104650 0x7fe74c195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:27.016 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.980+0000 7fe74a59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe74c104650 0x7fe74c195a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.171:57504/0 (socket says 172.21.15.171:57504) 2023-07-14T18:08:27.017 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.980+0000 7fe74a59c700 1 -- 172.21.15.171:0/3904400650 learned_addr learned my addr 172.21.15.171:0/3904400650 (peer_addr_for_me v2:172.21.15.171:0/0) 2023-07-14T18:08:27.017 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.980+0000 7fe74a59c700 1 -- 172.21.15.171:0/3904400650 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7340095f0 con 0x7fe74c104650 2023-07-14T18:08:27.017 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.980+0000 7fe74a59c700 1 --2- 172.21.15.171:0/3904400650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe74c104650 0x7fe74c195a40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe7340092e0 tx=0x7fe734004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:08:27.017 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.980+0000 7fe7437fe700 1 -- 172.21.15.171:0/3904400650 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe73401d070 con 0x7fe74c104650 2023-07-14T18:08:27.017 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.980+0000 7fe750dd0700 1 -- 172.21.15.171:0/3904400650 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe74c196180 con 0x7fe74c104650 2023-07-14T18:08:27.017 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.980+0000 7fe7437fe700 1 -- 172.21.15.171:0/3904400650 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe73400bc10 con 0x7fe74c104650 2023-07-14T18:08:27.017 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.980+0000 7fe7437fe700 1 -- 172.21.15.171:0/3904400650 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe734003930 con 0x7fe74c104650 2023-07-14T18:08:27.017 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.980+0000 7fe750dd0700 1 -- 172.21.15.171:0/3904400650 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe74c196620 con 0x7fe74c104650 2023-07-14T18:08:27.018 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.981+0000 7fe7437fe700 1 -- 172.21.15.171:0/3904400650 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7fe734003bb0 con 0x7fe74c104650 2023-07-14T18:08:27.018 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.981+0000 7fe750dd0700 1 -- 172.21.15.171:0/3904400650 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe74c103940 con 0x7fe74c104650 2023-07-14T18:08:27.018 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.982+0000 7fe7437fe700 1 --2- 172.21.15.171:0/3904400650 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe7380387b0 0x7fe73803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:27.018 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.982+0000 7fe7437fe700 1 -- 172.21.15.171:0/3904400650 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe73404c520 con 0x7fe74c104650 2023-07-14T18:08:27.018 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.982+0000 7fe749d9b700 1 --2- 172.21.15.171:0/3904400650 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe7380387b0 0x7fe73803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:27.019 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.982+0000 7fe749d9b700 1 --2- 172.21.15.171:0/3904400650 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe7380387b0 0x7fe73803ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe73c006fd0 tx=0x7fe73c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:08:27.022 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:26.987+0000 7fe7437fe700 1 -- 172.21.15.171:0/3904400650 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fe73400ba70 con 0x7fe74c104650 2023-07-14T18:08:27.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:26 smithi133 ceph-mon[126384]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:27.330 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:27.295+0000 7fe750dd0700 1 -- 172.21.15.171:0/3904400650 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe74c067ee0 con 0x7fe74c104650 2023-07-14T18:08:27.331 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:27.296+0000 7fe7437fe700 1 -- 172.21.15.171:0/3904400650 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7fe734026070 con 0x7fe74c104650 2023-07-14T18:08:27.331 INFO:teuthology.orchestra.run.smithi171.stdout: 2023-07-14T18:08:27.331 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":1,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","modified":"2023-07-14T18:06:46.356659Z","created":"2023-07-14T18:06:46.356659Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:3300","nonce":0},{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-07-14T18:08:27.333 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:27.298+0000 7fe750dd0700 1 -- 172.21.15.171:0/3904400650 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe7380387b0 msgr2=0x7fe73803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:27.333 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:27.298+0000 7fe750dd0700 1 --2- 172.21.15.171:0/3904400650 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe7380387b0 0x7fe73803ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe73c006fd0 tx=0x7fe73c006e40 comp rx=0 tx=0).stop 2023-07-14T18:08:27.333 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:27.298+0000 7fe750dd0700 1 -- 172.21.15.171:0/3904400650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe74c104650 msgr2=0x7fe74c195a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:27.333 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:27.298+0000 7fe750dd0700 1 --2- 172.21.15.171:0/3904400650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe74c104650 0x7fe74c195a40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe7340092e0 tx=0x7fe734004ff0 comp rx=0 tx=0).stop 2023-07-14T18:08:27.333 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:27.298+0000 7fe750dd0700 1 -- 172.21.15.171:0/3904400650 shutdown_connections 2023-07-14T18:08:27.333 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:27.298+0000 7fe750dd0700 1 --2- 172.21.15.171:0/3904400650 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe7380387b0 0x7fe73803ac60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:27.334 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:27.298+0000 7fe750dd0700 1 --2- 172.21.15.171:0/3904400650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe74c104650 0x7fe74c195a40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:27.334 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:27.298+0000 7fe750dd0700 1 -- 172.21.15.171:0/3904400650 >> 172.21.15.171:0/3904400650 conn(0x7fe74c0743e0 msgr2=0x7fe74c105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:08:27.334 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:27.298+0000 7fe750dd0700 1 -- 172.21.15.171:0/3904400650 shutdown_connections 2023-07-14T18:08:27.334 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:27.298+0000 7fe750dd0700 1 -- 172.21.15.171:0/3904400650 wait complete. 2023-07-14T18:08:27.334 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 1 2023-07-14T18:08:28.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:27 smithi133 ceph-mon[126384]: from='client.? 172.21.15.171:0/3904400650' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T18:08:28.293 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: mon.c@-1(???) e0 preinit fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c 2023-07-14T18:08:28.293 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: mon.c@-1(synchronizing).mds e1 new map 2023-07-14T18:08:28.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: mon.c@-1(synchronizing).mds e1 print_map 2023-07-14T18:08:28.607 INFO:journalctl@ceph.mon.c.smithi171.stdout: e1 2023-07-14T18:08:28.607 INFO:journalctl@ceph.mon.c.smithi171.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-14T18:08:28.608 INFO:journalctl@ceph.mon.c.smithi171.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-14T18:08:28.608 INFO:journalctl@ceph.mon.c.smithi171.stdout: legacy client fscid: -1 2023-07-14T18:08:28.608 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2023-07-14T18:08:28.608 INFO:journalctl@ceph.mon.c.smithi171.stdout: No filesystems configured 2023-07-14T18:08:28.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-07-14T18:08:28.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-07-14T18:08:28.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-07-14T18:08:28.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-07-14T18:08:28.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-07-14T18:08:28.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-07-14T18:08:28.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-14T18:08:28.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T18:08:28.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T18:08:28.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T18:08:28.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T18:08:28.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:28.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:28.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:28.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:28.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:28.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:08:28.611 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:28.611 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:08:28.611 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: Updating smithi161:/etc/ceph/ceph.conf 2023-07-14T18:08:28.611 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: Updating smithi161:/var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T18:08:28.611 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2023-07-14T18:08:28.611 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: Updating smithi161:/var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T18:08:28.611 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:28.611 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:28.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:28.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:28.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:08:28.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: Deploying cephadm binary to smithi171 2023-07-14T18:08:28.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:28.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:28.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:08:28.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:28.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: Added host smithi171 2023-07-14T18:08:28.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:28.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:28.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:28.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T18:08:28.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:28.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:28.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:28.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:28.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:28.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:08:28.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:28.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:08:28.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: Updating smithi171:/etc/ceph/ceph.conf 2023-07-14T18:08:28.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: Updating smithi171:/var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T18:08:28.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:28.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2023-07-14T18:08:28.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: Updating smithi171:/var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T18:08:28.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1576027545' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-14T18:08:28.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:28.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:28.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:28.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1576027545' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-14T18:08:28.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T18:08:28.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:28.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi133:172.21.15.133=a;smithi161:172.21.15.161=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:08:28.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: Saving service mon spec with placement smithi133:172.21.15.133=a;smithi161:172.21.15.161=b;smithi171:172.21.15.171=c;count:3 2023-07-14T18:08:28.617 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:28.617 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:08:28.617 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:28.617 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:08:28.617 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:28.617 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T18:08:28.617 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:28.618 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: Deploying daemon mon.c on smithi171 2023-07-14T18:08:28.618 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:28.618 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: from='client.? 172.21.15.171:0/3904400650' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T18:08:28.618 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-14T18:08:28.618 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-07-14T18:08:28.618 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: expand_channel_meta expand map: {default=false} 2023-07-14T18:08:28.618 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: expand_channel_meta from 'false' to 'false' 2023-07-14T18:08:28.618 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: expand_channel_meta expanded map: {default=false} 2023-07-14T18:08:28.619 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: expand_channel_meta expand map: {default=info} 2023-07-14T18:08:28.619 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: expand_channel_meta from 'info' to 'info' 2023-07-14T18:08:28.619 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: expand_channel_meta expanded map: {default=info} 2023-07-14T18:08:28.619 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: expand_channel_meta expand map: {default=daemon} 2023-07-14T18:08:28.619 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-14T18:08:28.619 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: expand_channel_meta expanded map: {default=daemon} 2023-07-14T18:08:28.619 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: expand_channel_meta expand map: {default=debug} 2023-07-14T18:08:28.619 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: expand_channel_meta from 'debug' to 'debug' 2023-07-14T18:08:28.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 ceph-mon[129241]: expand_channel_meta expanded map: {default=debug} 2023-07-14T18:08:28.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:28 smithi171 systemd[1]: Started Ceph mon.c for 06d0f5fa-2271-11ee-9b33-001a4aab830c. 2023-07-14T18:08:28.809 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-14T18:08:28.810 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph mon dump -f json 2023-07-14T18:08:29.371 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T18:08:33.589 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.552+0000 7f6de889f700 1 -- 172.21.15.171:0/2364571181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6dcc0035c0 msgr2=0x7f6dcc005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:33.589 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.552+0000 7f6de889f700 1 --2- 172.21.15.171:0/2364571181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6dcc0035c0 0x7f6dcc005a40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6de40ffc70 tx=0x7f6dd800fe60 comp rx=0 tx=0).stop 2023-07-14T18:08:33.589 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.552+0000 7f6de889f700 1 -- 172.21.15.171:0/2364571181 shutdown_connections 2023-07-14T18:08:33.589 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.552+0000 7f6de889f700 1 --2- 172.21.15.171:0/2364571181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6dcc0035c0 0x7f6dcc005a40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:33.590 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.552+0000 7f6de889f700 1 -- 172.21.15.171:0/2364571181 >> 172.21.15.171:0/2364571181 conn(0x7f6de4074500 msgr2=0x7f6de4074900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:08:33.590 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.552+0000 7f6de889f700 1 -- 172.21.15.171:0/2364571181 shutdown_connections 2023-07-14T18:08:33.590 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.553+0000 7f6de889f700 1 -- 172.21.15.171:0/2364571181 wait complete. 2023-07-14T18:08:33.592 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.554+0000 7f6de889f700 1 Processor -- start 2023-07-14T18:08:33.592 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.554+0000 7f6de889f700 1 -- start start 2023-07-14T18:08:33.593 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.554+0000 7f6de889f700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6dcc0035c0 0x7f6de4195b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:33.593 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.554+0000 7f6de889f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6de4196070 0x7f6de41964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:33.593 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.554+0000 7f6de889f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de419eba0 con 0x7f6de4196070 2023-07-14T18:08:33.593 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.554+0000 7f6de889f700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de419ece0 con 0x7f6dcc0035c0 2023-07-14T18:08:33.593 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.554+0000 7f6de259c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6dcc0035c0 0x7f6de4195b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:33.593 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.554+0000 7f6de259c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6dcc0035c0 0x7f6de4195b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:42692/0 (socket says 172.21.15.171:42692) 2023-07-14T18:08:33.593 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.554+0000 7f6de259c700 1 -- 172.21.15.171:0/3877881443 learned_addr learned my addr 172.21.15.171:0/3877881443 (peer_addr_for_me v2:172.21.15.171:0/0) 2023-07-14T18:08:33.594 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.554+0000 7f6de259c700 1 -- 172.21.15.171:0/3877881443 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6dcc0035c0 msgr2=0x7f6de4195b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-07-14T18:08:33.594 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.554+0000 7f6de259c700 1 -- 172.21.15.171:0/3877881443 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6dcc0035c0 msgr2=0x7f6de4195b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-14T18:08:33.594 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.554+0000 7f6de259c700 1 --2- 172.21.15.171:0/3877881443 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6dcc0035c0 0x7f6de4195b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-14T18:08:33.594 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.554+0000 7f6de259c700 1 --2- 172.21.15.171:0/3877881443 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6dcc0035c0 0x7f6de4195b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T18:08:33.594 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.554+0000 7f6de1d9b700 1 --2- 172.21.15.171:0/3877881443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6de4196070 0x7f6de41964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:33.594 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.555+0000 7f6de1d9b700 1 -- 172.21.15.171:0/3877881443 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6dcc0035c0 msgr2=0x7f6de4195b30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T18:08:33.594 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.555+0000 7f6de1d9b700 1 --2- 172.21.15.171:0/3877881443 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6dcc0035c0 0x7f6de4195b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:33.594 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.555+0000 7f6de1d9b700 1 -- 172.21.15.171:0/3877881443 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd800f8b0 con 0x7f6de4196070 2023-07-14T18:08:33.595 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.555+0000 7f6de1d9b700 1 --2- 172.21.15.171:0/3877881443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6de4196070 0x7f6de41964e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6dd400b4d0 tx=0x7f6dd400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:08:33.595 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.555+0000 7f6dd37fe700 1 -- 172.21.15.171:0/3877881443 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f6dd4011630 con 0x7f6de4196070 2023-07-14T18:08:33.595 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.555+0000 7f6de889f700 1 -- 172.21.15.171:0/3877881443 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de419eee0 con 0x7f6de4196070 2023-07-14T18:08:33.595 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.555+0000 7f6dd37fe700 1 -- 172.21.15.171:0/3877881443 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dd4011c50 con 0x7f6de4196070 2023-07-14T18:08:33.595 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.555+0000 7f6dd37fe700 1 -- 172.21.15.171:0/3877881443 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f6dd4010d90 con 0x7f6de4196070 2023-07-14T18:08:33.595 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.555+0000 7f6de889f700 1 -- 172.21.15.171:0/3877881443 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de419f480 con 0x7f6de4196070 2023-07-14T18:08:33.595 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.556+0000 7f6dd37fe700 1 -- 172.21.15.171:0/3877881443 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f6dd401a400 con 0x7f6de4196070 2023-07-14T18:08:33.596 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.557+0000 7f6dd37fe700 1 --2- 172.21.15.171:0/3877881443 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6dc40385f0 0x7f6dc403aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:33.596 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.557+0000 7f6dd37fe700 1 -- 172.21.15.171:0/3877881443 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6dd404ae00 con 0x7f6de4196070 2023-07-14T18:08:33.596 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.557+0000 7f6de889f700 1 -- 172.21.15.171:0/3877881443 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de4061c70 con 0x7f6de4196070 2023-07-14T18:08:33.596 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.557+0000 7f6de259c700 1 --2- 172.21.15.171:0/3877881443 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6dc40385f0 0x7f6dc403aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:33.597 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.558+0000 7f6de259c700 1 --2- 172.21.15.171:0/3877881443 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6dc40385f0 0x7f6dc403aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8005ed0 tx=0x7f6dd8005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:08:33.597 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.562+0000 7f6dd37fe700 1 -- 172.21.15.171:0/3877881443 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f6dd40178b0 con 0x7f6de4196070 2023-07-14T18:08:33.647 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:33 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T18:08:33.648 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:33 smithi133 ceph-mon[126384]: mon.a calling monitor election 2023-07-14T18:08:33.648 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:33 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:08:33.648 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:33 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:08:33.648 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:33 smithi133 ceph-mon[126384]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:33.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:33 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:08:33.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:33 smithi133 ceph-mon[126384]: mon.c calling monitor election 2023-07-14T18:08:33.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:33 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:08:33.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:33 smithi133 ceph-mon[126384]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:33.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:33 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:08:33.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:33 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:08:33.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:33 smithi133 ceph-mon[126384]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-14T18:08:33.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:33 smithi133 ceph-mon[126384]: monmap e2: 2 mons at {a=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0],c=[v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0]} removed_ranks: {} 2023-07-14T18:08:33.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:33 smithi133 ceph-mon[126384]: fsmap 2023-07-14T18:08:33.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:33 smithi133 ceph-mon[126384]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T18:08:33.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:33 smithi133 ceph-mon[126384]: mgrmap e13: x(active, since 45s) 2023-07-14T18:08:33.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:33 smithi133 ceph-mon[126384]: overall HEALTH_OK 2023-07-14T18:08:33.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:33 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:33.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:33 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:33.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:33 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T18:08:33.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:33 smithi171 ceph-mon[129241]: mon.a calling monitor election 2023-07-14T18:08:33.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:33 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:08:33.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:33 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:08:33.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:33 smithi171 ceph-mon[129241]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:33.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:33 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:08:33.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:33 smithi171 ceph-mon[129241]: mon.c calling monitor election 2023-07-14T18:08:33.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:33 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:08:33.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:33 smithi171 ceph-mon[129241]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:33.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:33 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:08:33.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:33 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:08:33.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:33 smithi171 ceph-mon[129241]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-14T18:08:33.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:33 smithi171 ceph-mon[129241]: monmap e2: 2 mons at {a=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0],c=[v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0]} removed_ranks: {} 2023-07-14T18:08:33.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:33 smithi171 ceph-mon[129241]: fsmap 2023-07-14T18:08:33.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:33 smithi171 ceph-mon[129241]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T18:08:33.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:33 smithi171 ceph-mon[129241]: mgrmap e13: x(active, since 45s) 2023-07-14T18:08:33.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:33 smithi171 ceph-mon[129241]: overall HEALTH_OK 2023-07-14T18:08:33.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:33 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:33.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:33 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:33.863 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.824+0000 7f6de889f700 1 -- 172.21.15.171:0/3877881443 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6de40683b0 con 0x7f6de4196070 2023-07-14T18:08:33.864 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.825+0000 7f6dd37fe700 1 -- 172.21.15.171:0/3877881443 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1010 (secure 0 0 0) 0x7f6dd4048020 con 0x7f6de4196070 2023-07-14T18:08:33.864 INFO:teuthology.orchestra.run.smithi171.stdout: 2023-07-14T18:08:33.865 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":2,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","modified":"2023-07-14T18:08:28.266424Z","created":"2023-07-14T18:06:46.356659Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:3300","nonce":0},{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:3300","nonce":0},{"type":"v1","addr":"172.21.15.171:6789","nonce":0}]},"addr":"172.21.15.171:6789/0","public_addr":"172.21.15.171:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-07-14T18:08:33.866 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.827+0000 7f6de889f700 1 -- 172.21.15.171:0/3877881443 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6dc40385f0 msgr2=0x7f6dc403aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:33.866 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.827+0000 7f6de889f700 1 --2- 172.21.15.171:0/3877881443 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6dc40385f0 0x7f6dc403aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8005ed0 tx=0x7f6dd8005ab0 comp rx=0 tx=0).stop 2023-07-14T18:08:33.866 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.827+0000 7f6de889f700 1 -- 172.21.15.171:0/3877881443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6de4196070 msgr2=0x7f6de41964e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:33.867 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.827+0000 7f6de889f700 1 --2- 172.21.15.171:0/3877881443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6de4196070 0x7f6de41964e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6dd400b4d0 tx=0x7f6dd400b890 comp rx=0 tx=0).stop 2023-07-14T18:08:33.867 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.827+0000 7f6de889f700 1 -- 172.21.15.171:0/3877881443 shutdown_connections 2023-07-14T18:08:33.867 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.827+0000 7f6de889f700 1 --2- 172.21.15.171:0/3877881443 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6dc40385f0 0x7f6dc403aaa0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:33.867 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.827+0000 7f6de889f700 1 --2- 172.21.15.171:0/3877881443 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6dcc0035c0 0x7f6de4195b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:33.868 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.827+0000 7f6de889f700 1 --2- 172.21.15.171:0/3877881443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6de4196070 0x7f6de41964e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:33.868 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.827+0000 7f6de889f700 1 -- 172.21.15.171:0/3877881443 >> 172.21.15.171:0/3877881443 conn(0x7f6de4074500 msgr2=0x7f6de4102330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:08:33.868 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.827+0000 7f6de889f700 1 -- 172.21.15.171:0/3877881443 shutdown_connections 2023-07-14T18:08:33.869 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:33.827+0000 7f6de889f700 1 -- 172.21.15.171:0/3877881443 wait complete. 2023-07-14T18:08:33.869 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 2 2023-07-14T18:08:34.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:34 smithi171 ceph-mon[129241]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:34.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:34 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:34.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:34 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T18:08:34.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:34 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:34.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:34 smithi171 ceph-mon[129241]: Deploying daemon mon.b on smithi161 2023-07-14T18:08:34.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:34 smithi171 ceph-mon[129241]: from='client.? 172.21.15.171:0/3877881443' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T18:08:34.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:34 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:08:34.897 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:34 smithi133 ceph-mon[126384]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:34.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:34 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:34.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:34 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T18:08:34.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:34 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:34.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:34 smithi133 ceph-mon[126384]: Deploying daemon mon.b on smithi161 2023-07-14T18:08:34.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:34 smithi133 ceph-mon[126384]: from='client.? 172.21.15.171:0/3877881443' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T18:08:34.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:34 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:08:35.335 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-14T18:08:35.336 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph mon dump -f json 2023-07-14T18:08:35.647 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:08:35 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:08:35.266+0000 7fa68072d700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-07-14T18:08:35.899 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T18:08:36.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:35 smithi161 systemd[1]: Starting Ceph mon.b for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:08:36.592 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 podman[128953]: 2023-07-14T18:08:36.593 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 bash[128953]: 553187cae4bee4b56ddf1f27c12a480acd3d53a23ad7aa4e623454e0d60b0631 2023-07-14T18:08:36.593 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: set uid:gid to 167:167 (ceph:ceph) 2023-07-14T18:08:36.593 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: ceph version 18.0.0-4940-g81c9387d (81c9387d2e7576da065747ec9c16d1c7d69016ec) reef (dev), process ceph-mon, pid 7 2023-07-14T18:08:36.593 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: pidfile_write: ignore empty --pid-file 2023-07-14T18:08:36.593 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: load: jerasure load: lrc 2023-07-14T18:08:36.593 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: RocksDB version: 7.9.2 2023-07-14T18:08:36.593 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Git sha 0 2023-07-14T18:08:36.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Compile date 2023-07-14 14:45:42 2023-07-14T18:08:36.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: DB SUMMARY 2023-07-14T18:08:36.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: DB Session ID: 1PIOI5STUWWTJJWPQ4DO 2023-07-14T18:08:36.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: CURRENT file: CURRENT 2023-07-14T18:08:36.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: IDENTITY file: IDENTITY 2023-07-14T18:08:36.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-07-14T18:08:36.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-07-14T18:08:36.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-07-14T18:08:36.595 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.error_if_exists: 0 2023-07-14T18:08:36.595 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.create_if_missing: 0 2023-07-14T18:08:36.595 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.paranoid_checks: 1 2023-07-14T18:08:36.595 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-14T18:08:36.595 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-14T18:08:36.595 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-14T18:08:36.595 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.env: 0x56368f1b6c60 2023-07-14T18:08:36.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.fs: PosixFileSystem 2023-07-14T18:08:36.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.info_log: 0x5636911852a0 2023-07-14T18:08:36.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.max_file_opening_threads: 16 2023-07-14T18:08:36.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.statistics: (nil) 2023-07-14T18:08:36.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.use_fsync: 0 2023-07-14T18:08:36.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.max_log_file_size: 0 2023-07-14T18:08:36.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-14T18:08:36.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-14T18:08:36.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.keep_log_file_num: 1000 2023-07-14T18:08:36.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.recycle_log_file_num: 0 2023-07-14T18:08:36.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.allow_fallocate: 1 2023-07-14T18:08:36.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.allow_mmap_reads: 0 2023-07-14T18:08:36.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.allow_mmap_writes: 0 2023-07-14T18:08:36.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.use_direct_reads: 0 2023-07-14T18:08:36.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-14T18:08:36.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.create_missing_column_families: 0 2023-07-14T18:08:36.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.db_log_dir: 2023-07-14T18:08:36.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.wal_dir: 2023-07-14T18:08:36.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-14T18:08:36.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-14T18:08:36.600 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-14T18:08:36.600 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-14T18:08:36.600 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-14T18:08:36.600 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-14T18:08:36.600 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.advise_random_on_open: 1 2023-07-14T18:08:36.600 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.db_write_buffer_size: 0 2023-07-14T18:08:36.600 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.write_buffer_manager: 0x5636910a63c0 2023-07-14T18:08:36.600 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-14T18:08:36.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-14T18:08:36.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-14T18:08:36.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.rate_limiter: (nil) 2023-07-14T18:08:36.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-14T18:08:36.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.wal_recovery_mode: 2 2023-07-14T18:08:36.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.enable_thread_tracking: 0 2023-07-14T18:08:36.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.enable_pipelined_write: 0 2023-07-14T18:08:36.602 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.unordered_write: 0 2023-07-14T18:08:36.602 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-14T18:08:36.602 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-14T18:08:36.602 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-14T18:08:36.602 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-14T18:08:36.602 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.row_cache: None 2023-07-14T18:08:36.602 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.wal_filter: None 2023-07-14T18:08:36.602 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-14T18:08:36.603 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.allow_ingest_behind: 0 2023-07-14T18:08:36.603 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.two_write_queues: 0 2023-07-14T18:08:36.603 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.manual_wal_flush: 0 2023-07-14T18:08:36.603 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.wal_compression: 0 2023-07-14T18:08:36.603 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.atomic_flush: 0 2023-07-14T18:08:36.603 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-14T18:08:36.603 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-14T18:08:36.603 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-14T18:08:36.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.log_readahead_size: 0 2023-07-14T18:08:36.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-14T18:08:36.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.best_efforts_recovery: 0 2023-07-14T18:08:36.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-14T18:08:36.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-14T18:08:36.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.allow_data_in_errors: 0 2023-07-14T18:08:36.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.db_host_id: __hostname__ 2023-07-14T18:08:36.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-14T18:08:36.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.max_background_jobs: 2 2023-07-14T18:08:36.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.max_background_compactions: -1 2023-07-14T18:08:36.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.max_subcompactions: 1 2023-07-14T18:08:36.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-14T18:08:36.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-14T18:08:36.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-14T18:08:36.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.max_total_wal_size: 0 2023-07-14T18:08:36.606 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-14T18:08:36.606 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-14T18:08:36.606 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-14T18:08:36.606 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-14T18:08:36.606 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.max_open_files: -1 2023-07-14T18:08:36.606 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.bytes_per_sync: 0 2023-07-14T18:08:36.606 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-14T18:08:36.606 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-14T18:08:36.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.compaction_readahead_size: 0 2023-07-14T18:08:36.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.max_background_flushes: -1 2023-07-14T18:08:36.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Compression algorithms supported: 2023-07-14T18:08:36.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-14T18:08:36.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: kZSTD supported: 0 2023-07-14T18:08:36.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: kXpressCompression supported: 0 2023-07-14T18:08:36.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: kLZ4HCCompression supported: 1 2023-07-14T18:08:36.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: kZlibCompression supported: 1 2023-07-14T18:08:36.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: kSnappyCompression supported: 1 2023-07-14T18:08:36.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: kLZ4Compression supported: 1 2023-07-14T18:08:36.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: kBZip2Compression supported: 0 2023-07-14T18:08:36.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-14T18:08:36.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-14T18:08:36.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-07-14T18:08:36.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-14T18:08:36.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-14T18:08:36.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.merge_operator: 2023-07-14T18:08:36.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.compaction_filter: None 2023-07-14T18:08:36.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.compaction_filter_factory: None 2023-07-14T18:08:36.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.sst_partitioner_factory: None 2023-07-14T18:08:36.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-14T18:08:36.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-14T18:08:36.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5636911853a0) 2023-07-14T18:08:36.609 INFO:journalctl@ceph.mon.b.smithi161.stdout: cache_index_and_filter_blocks: 1 2023-07-14T18:08:36.610 INFO:journalctl@ceph.mon.b.smithi161.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-14T18:08:36.610 INFO:journalctl@ceph.mon.b.smithi161.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-14T18:08:36.610 INFO:journalctl@ceph.mon.b.smithi161.stdout: pin_top_level_index_and_filter: 1 2023-07-14T18:08:36.610 INFO:journalctl@ceph.mon.b.smithi161.stdout: index_type: 0 2023-07-14T18:08:36.610 INFO:journalctl@ceph.mon.b.smithi161.stdout: data_block_index_type: 0 2023-07-14T18:08:36.610 INFO:journalctl@ceph.mon.b.smithi161.stdout: index_shortening: 1 2023-07-14T18:08:36.610 INFO:journalctl@ceph.mon.b.smithi161.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-14T18:08:36.611 INFO:journalctl@ceph.mon.b.smithi161.stdout: checksum: 4 2023-07-14T18:08:36.611 INFO:journalctl@ceph.mon.b.smithi161.stdout: no_block_cache: 0 2023-07-14T18:08:36.611 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_cache: 0x5636911451f0 2023-07-14T18:08:36.611 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_cache_name: BinnedLRUCache 2023-07-14T18:08:36.611 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_cache_options: 2023-07-14T18:08:36.611 INFO:journalctl@ceph.mon.b.smithi161.stdout: capacity : 536870912 2023-07-14T18:08:36.611 INFO:journalctl@ceph.mon.b.smithi161.stdout: num_shard_bits : 4 2023-07-14T18:08:36.611 INFO:journalctl@ceph.mon.b.smithi161.stdout: strict_capacity_limit : 0 2023-07-14T18:08:36.611 INFO:journalctl@ceph.mon.b.smithi161.stdout: high_pri_pool_ratio: 0.000 2023-07-14T18:08:36.612 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_cache_compressed: (nil) 2023-07-14T18:08:36.612 INFO:journalctl@ceph.mon.b.smithi161.stdout: persistent_cache: (nil) 2023-07-14T18:08:36.612 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_size: 4096 2023-07-14T18:08:36.612 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_size_deviation: 10 2023-07-14T18:08:36.612 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_restart_interval: 16 2023-07-14T18:08:36.612 INFO:journalctl@ceph.mon.b.smithi161.stdout: index_block_restart_interval: 1 2023-07-14T18:08:36.612 INFO:journalctl@ceph.mon.b.smithi161.stdout: metadata_block_size: 4096 2023-07-14T18:08:36.613 INFO:journalctl@ceph.mon.b.smithi161.stdout: partition_filters: 0 2023-07-14T18:08:36.613 INFO:journalctl@ceph.mon.b.smithi161.stdout: use_delta_encoding: 1 2023-07-14T18:08:36.613 INFO:journalctl@ceph.mon.b.smithi161.stdout: filter_policy: bloomfilter 2023-07-14T18:08:36.613 INFO:journalctl@ceph.mon.b.smithi161.stdout: whole_key_filtering: 1 2023-07-14T18:08:36.613 INFO:journalctl@ceph.mon.b.smithi161.stdout: verify_compression: 0 2023-07-14T18:08:36.613 INFO:journalctl@ceph.mon.b.smithi161.stdout: read_amp_bytes_per_bit: 0 2023-07-14T18:08:36.613 INFO:journalctl@ceph.mon.b.smithi161.stdout: format_version: 5 2023-07-14T18:08:36.613 INFO:journalctl@ceph.mon.b.smithi161.stdout: enable_index_compression: 1 2023-07-14T18:08:36.614 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_align: 0 2023-07-14T18:08:36.614 INFO:journalctl@ceph.mon.b.smithi161.stdout: max_auto_readahead_size: 262144 2023-07-14T18:08:36.614 INFO:journalctl@ceph.mon.b.smithi161.stdout: prepopulate_block_cache: 0 2023-07-14T18:08:36.614 INFO:journalctl@ceph.mon.b.smithi161.stdout: initial_auto_readahead_size: 8192 2023-07-14T18:08:36.614 INFO:journalctl@ceph.mon.b.smithi161.stdout: num_file_reads_for_auto_readahead: 2 2023-07-14T18:08:36.614 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.write_buffer_size: 33554432 2023-07-14T18:08:36.614 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.max_write_buffer_number: 2 2023-07-14T18:08:36.614 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.compression: NoCompression 2023-07-14T18:08:36.615 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.bottommost_compression: Disabled 2023-07-14T18:08:36.615 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.prefix_extractor: nullptr 2023-07-14T18:08:36.615 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-14T18:08:36.615 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.num_levels: 7 2023-07-14T18:08:36.615 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-14T18:08:36.615 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-14T18:08:36.615 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-14T18:08:36.615 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-14T18:08:36.616 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-14T18:08:36.616 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-14T18:08:36.616 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-14T18:08:36.616 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-14T18:08:36.616 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-14T18:08:36.616 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-14T18:08:36.616 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-14T18:08:36.616 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-14T18:08:36.617 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-14T18:08:36.617 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.compression_opts.level: 32767 2023-07-14T18:08:36.617 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.compression_opts.strategy: 0 2023-07-14T18:08:36.617 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-14T18:08:36.617 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-14T18:08:36.617 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-14T18:08:36.617 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-14T18:08:36.617 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.compression_opts.enabled: false 2023-07-14T18:08:36.618 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-14T18:08:36.618 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-14T18:08:36.618 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-14T18:08:36.618 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-14T18:08:36.618 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.target_file_size_base: 67108864 2023-07-14T18:08:36.618 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-14T18:08:36.618 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-14T18:08:36.618 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-14T18:08:36.619 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-14T18:08:36.619 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-14T18:08:36.619 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-14T18:08:36.619 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-14T18:08:36.619 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-14T18:08:36.619 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-14T18:08:36.619 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-14T18:08:36.619 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-14T18:08:36.620 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-14T18:08:36.620 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-14T18:08:36.620 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-14T18:08:36.620 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.arena_block_size: 1048576 2023-07-14T18:08:36.620 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-14T18:08:36.620 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-14T18:08:36.620 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.disable_auto_compactions: 0 2023-07-14T18:08:36.621 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-14T18:08:36.621 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-14T18:08:36.621 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-14T18:08:36.621 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-14T18:08:36.621 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-14T18:08:36.621 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-14T18:08:36.621 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-14T18:08:36.621 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-14T18:08:36.622 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-14T18:08:36.622 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-14T18:08:36.622 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.table_properties_collectors: 2023-07-14T18:08:36.622 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.inplace_update_support: 0 2023-07-14T18:08:36.622 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-14T18:08:36.622 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-14T18:08:36.622 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-14T18:08:36.622 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-14T18:08:36.623 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.bloom_locality: 0 2023-07-14T18:08:36.623 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.max_successive_merges: 0 2023-07-14T18:08:36.623 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-14T18:08:36.623 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.paranoid_file_checks: 0 2023-07-14T18:08:36.623 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.force_consistency_checks: 1 2023-07-14T18:08:36.623 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.report_bg_io_stats: 0 2023-07-14T18:08:36.623 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.ttl: 2592000 2023-07-14T18:08:36.623 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-14T18:08:36.624 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-14T18:08:36.624 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-14T18:08:36.624 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.enable_blob_files: false 2023-07-14T18:08:36.624 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.min_blob_size: 0 2023-07-14T18:08:36.624 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.blob_file_size: 268435456 2023-07-14T18:08:36.624 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-14T18:08:36.624 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-14T18:08:36.624 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-14T18:08:36.625 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-14T18:08:36.625 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-14T18:08:36.625 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.blob_file_starting_level: 0 2023-07-14T18:08:36.625 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-14T18:08:36.625 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-07-14T18:08:36.625 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-07-14T18:08:36.625 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 09201a42-9be4-4776-9045-c9d40a22bcc8 2023-07-14T18:08:36.626 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689358116287318, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-07-14T18:08:36.626 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-07-14T18:08:36.626 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689358116287774, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689358116, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "09201a42-9be4-4776-9045-c9d40a22bcc8", "db_session_id": "1PIOI5STUWWTJJWPQ4DO", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-07-14T18:08:36.626 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689358116287842, "job": 1, "event": "recovery_finished"} 2023-07-14T18:08:36.626 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-07-14T18:08:36.626 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-14T18:08:36.626 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5636911de000 2023-07-14T18:08:36.626 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: DB pointer 0x5636911ca000 2023-07-14T18:08:36.627 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-14T18:08:36.627 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-14T18:08:36.627 INFO:journalctl@ceph.mon.b.smithi161.stdout: ** DB Stats ** 2023-07-14T18:08:36.627 INFO:journalctl@ceph.mon.b.smithi161.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-14T18:08:36.627 INFO:journalctl@ceph.mon.b.smithi161.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-14T18:08:36.627 INFO:journalctl@ceph.mon.b.smithi161.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-14T18:08:36.627 INFO:journalctl@ceph.mon.b.smithi161.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-14T18:08:36.627 INFO:journalctl@ceph.mon.b.smithi161.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-14T18:08:36.628 INFO:journalctl@ceph.mon.b.smithi161.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-14T18:08:36.628 INFO:journalctl@ceph.mon.b.smithi161.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-14T18:08:36.628 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2023-07-14T18:08:36.628 INFO:journalctl@ceph.mon.b.smithi161.stdout: ** Compaction Stats [default] ** 2023-07-14T18:08:36.628 INFO:journalctl@ceph.mon.b.smithi161.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-14T18:08:36.628 INFO:journalctl@ceph.mon.b.smithi161.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-14T18:08:36.628 INFO:journalctl@ceph.mon.b.smithi161.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-14T18:08:36.628 INFO:journalctl@ceph.mon.b.smithi161.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-14T18:08:36.629 INFO:journalctl@ceph.mon.b.smithi161.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-14T18:08:36.629 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2023-07-14T18:08:36.629 INFO:journalctl@ceph.mon.b.smithi161.stdout: ** Compaction Stats [default] ** 2023-07-14T18:08:36.629 INFO:journalctl@ceph.mon.b.smithi161.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-14T18:08:36.629 INFO:journalctl@ceph.mon.b.smithi161.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-14T18:08:36.629 INFO:journalctl@ceph.mon.b.smithi161.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-14T18:08:36.629 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2023-07-14T18:08:36.630 INFO:journalctl@ceph.mon.b.smithi161.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-14T18:08:36.630 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2023-07-14T18:08:36.630 INFO:journalctl@ceph.mon.b.smithi161.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-14T18:08:36.630 INFO:journalctl@ceph.mon.b.smithi161.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-14T18:08:36.630 INFO:journalctl@ceph.mon.b.smithi161.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-14T18:08:36.630 INFO:journalctl@ceph.mon.b.smithi161.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-14T18:08:36.630 INFO:journalctl@ceph.mon.b.smithi161.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-14T18:08:36.630 INFO:journalctl@ceph.mon.b.smithi161.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-14T18:08:36.631 INFO:journalctl@ceph.mon.b.smithi161.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-14T18:08:36.631 INFO:journalctl@ceph.mon.b.smithi161.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-14T18:08:36.631 INFO:journalctl@ceph.mon.b.smithi161.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-14T18:08:36.631 INFO:journalctl@ceph.mon.b.smithi161.stdout: Block cache BinnedLRUCache@0x5636911451f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-07-14T18:08:36.631 INFO:journalctl@ceph.mon.b.smithi161.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-07-14T18:08:36.631 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2023-07-14T18:08:36.631 INFO:journalctl@ceph.mon.b.smithi161.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-14T18:08:36.631 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-07-14T18:08:36.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: using public_addr v2:172.21.15.161:0/0 -> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] 2023-07-14T18:08:36.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 systemd[1]: Started Ceph mon.b for 06d0f5fa-2271-11ee-9b33-001a4aab830c. 2023-07-14T18:08:36.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: starting mon.b rank -1 at public addrs [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] at bind addrs [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c 2023-07-14T18:08:36.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: mon.b@-1(???) e0 preinit fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c 2023-07-14T18:08:36.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: mon.b@-1(synchronizing).mds e1 new map 2023-07-14T18:08:36.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: mon.b@-1(synchronizing).mds e1 print_map 2023-07-14T18:08:36.632 INFO:journalctl@ceph.mon.b.smithi161.stdout: e1 2023-07-14T18:08:36.632 INFO:journalctl@ceph.mon.b.smithi161.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-14T18:08:36.633 INFO:journalctl@ceph.mon.b.smithi161.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-14T18:08:36.633 INFO:journalctl@ceph.mon.b.smithi161.stdout: legacy client fscid: -1 2023-07-14T18:08:36.633 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2023-07-14T18:08:36.633 INFO:journalctl@ceph.mon.b.smithi161.stdout: No filesystems configured 2023-07-14T18:08:36.633 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-07-14T18:08:36.633 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-07-14T18:08:36.633 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-07-14T18:08:36.633 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-07-14T18:08:36.634 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-07-14T18:08:36.634 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-07-14T18:08:36.634 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-14T18:08:36.634 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T18:08:36.634 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T18:08:36.634 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T18:08:36.634 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T18:08:36.634 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:36.635 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:36.635 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:36.635 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:36.635 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:36.635 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:08:36.635 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:36.635 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:08:36.635 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: Updating smithi161:/etc/ceph/ceph.conf 2023-07-14T18:08:36.636 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: Updating smithi161:/var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T18:08:36.636 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2023-07-14T18:08:36.636 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: Updating smithi161:/var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T18:08:36.636 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:36.636 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:36.636 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:36.636 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:36.637 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:08:36.637 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: Deploying cephadm binary to smithi171 2023-07-14T18:08:36.637 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:36.637 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:36.637 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:08:36.637 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:36.637 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: Added host smithi171 2023-07-14T18:08:36.637 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:36.638 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:36.638 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:36.638 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T18:08:36.638 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:36.638 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:36.638 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:36.638 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:36.638 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:36.639 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:08:36.639 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:36.639 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:08:36.639 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: Updating smithi171:/etc/ceph/ceph.conf 2023-07-14T18:08:36.639 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: Updating smithi171:/var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T18:08:36.639 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:36.639 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2023-07-14T18:08:36.639 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: Updating smithi171:/var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T18:08:36.640 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1576027545' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-14T18:08:36.640 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:36.640 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:36.640 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:36.640 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1576027545' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-14T18:08:36.640 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T18:08:36.640 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:36.641 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi133:172.21.15.133=a;smithi161:172.21.15.161=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:08:36.641 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: Saving service mon spec with placement smithi133:172.21.15.133=a;smithi161:172.21.15.161=b;smithi171:172.21.15.171=c;count:3 2023-07-14T18:08:36.641 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:36.641 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:08:36.641 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:36.641 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:08:36.641 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:36.641 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T18:08:36.642 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:36.642 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: Deploying daemon mon.c on smithi171 2023-07-14T18:08:36.642 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:36.642 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='client.? 172.21.15.171:0/3904400650' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T18:08:36.642 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T18:08:36.642 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: mon.a calling monitor election 2023-07-14T18:08:36.642 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:08:36.642 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:08:36.643 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:36.643 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:08:36.643 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: mon.c calling monitor election 2023-07-14T18:08:36.643 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:08:36.643 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:36.643 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:08:36.643 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:08:36.643 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-14T18:08:36.644 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: monmap e2: 2 mons at {a=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0],c=[v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0]} removed_ranks: {} 2023-07-14T18:08:36.644 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: fsmap 2023-07-14T18:08:36.644 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T18:08:36.644 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: mgrmap e13: x(active, since 45s) 2023-07-14T18:08:36.644 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: overall HEALTH_OK 2023-07-14T18:08:36.644 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:36.644 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:36.644 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:36.645 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:36.645 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T18:08:36.645 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:36.645 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: Deploying daemon mon.b on smithi161 2023-07-14T18:08:36.645 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='client.? 172.21.15.171:0/3877881443' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T18:08:36.645 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:08:36.645 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-14T18:08:36.646 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:36 smithi161 ceph-mon[128993]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-07-14T18:08:41.547 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.510+0000 7f1ccb033700 1 -- 172.21.15.171:0/2592512131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cb40035c0 msgr2=0x7f1cb4005a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:41.547 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.510+0000 7f1ccb033700 1 --2- 172.21.15.171:0/2592512131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cb40035c0 0x7f1cb4005a20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40fe760 tx=0x7f1cb800fe10 comp rx=0 tx=0).stop 2023-07-14T18:08:41.547 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.510+0000 7f1ccb033700 1 -- 172.21.15.171:0/2592512131 shutdown_connections 2023-07-14T18:08:41.548 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.510+0000 7f1ccb033700 1 --2- 172.21.15.171:0/2592512131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cb40035c0 0x7f1cb4005a20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:41.548 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.510+0000 7f1ccb033700 1 -- 172.21.15.171:0/2592512131 >> 172.21.15.171:0/2592512131 conn(0x7f1cc4074500 msgr2=0x7f1cc4074900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:08:41.548 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.510+0000 7f1ccb033700 1 -- 172.21.15.171:0/2592512131 shutdown_connections 2023-07-14T18:08:41.548 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.510+0000 7f1ccb033700 1 -- 172.21.15.171:0/2592512131 wait complete. 2023-07-14T18:08:41.548 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.511+0000 7f1ccb033700 1 Processor -- start 2023-07-14T18:08:41.548 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.511+0000 7f1ccb033700 1 -- start start 2023-07-14T18:08:41.550 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.511+0000 7f1ccb033700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cb40035c0 0x7f1cc41003f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:41.550 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.511+0000 7f1ccb033700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cc4100930 0x7f1cc4100da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:41.550 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.511+0000 7f1ccb033700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cc41a0fb0 0x7f1cc41a3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:41.551 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.512+0000 7f1ccb033700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc41a3ac0 con 0x7f1cb40035c0 2023-07-14T18:08:41.551 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.512+0000 7f1ccb033700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc41a3c30 con 0x7f1cc4100930 2023-07-14T18:08:41.551 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.512+0000 7f1ccb033700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc41a3df0 con 0x7f1cc41a0fb0 2023-07-14T18:08:41.551 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.512+0000 7f1cc95d0700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cc41a0fb0 0x7f1cc41a3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:41.551 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.512+0000 7f1cc95d0700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cc41a0fb0 0x7f1cc41a3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:42724/0 (socket says 172.21.15.171:42724) 2023-07-14T18:08:41.551 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.512+0000 7f1cc95d0700 1 -- 172.21.15.171:0/3787835640 learned_addr learned my addr 172.21.15.171:0/3787835640 (peer_addr_for_me v2:172.21.15.171:0/0) 2023-07-14T18:08:41.551 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.512+0000 7f1cc3fff700 1 --2- 172.21.15.171:0/3787835640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cc4100930 0x7f1cc4100da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:41.552 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.512+0000 7f1cc95d0700 1 -- 172.21.15.171:0/3787835640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cc41a0fb0 msgr2=0x7f1cc41a3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-07-14T18:08:41.552 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.512+0000 7f1cc8dcf700 1 --2- 172.21.15.171:0/3787835640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cb40035c0 0x7f1cc41003f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:41.552 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.512+0000 7f1cc95d0700 1 -- 172.21.15.171:0/3787835640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cc41a0fb0 msgr2=0x7f1cc41a3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-14T18:08:41.552 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.512+0000 7f1cc95d0700 1 --2- 172.21.15.171:0/3787835640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cc41a0fb0 0x7f1cc41a3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-14T18:08:41.552 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.512+0000 7f1cc3fff700 1 -- 172.21.15.171:0/3787835640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cc4100930 msgr2=0x7f1cc4100da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-07-14T18:08:41.552 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.512+0000 7f1cc3fff700 1 -- 172.21.15.171:0/3787835640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cc4100930 msgr2=0x7f1cc4100da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-14T18:08:41.553 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.512+0000 7f1cc3fff700 1 --2- 172.21.15.171:0/3787835640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cc4100930 0x7f1cc4100da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-14T18:08:41.553 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.512+0000 7f1cc95d0700 1 --2- 172.21.15.171:0/3787835640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cc41a0fb0 0x7f1cc41a3390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T18:08:41.553 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.512+0000 7f1cc3fff700 1 --2- 172.21.15.171:0/3787835640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cc4100930 0x7f1cc4100da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T18:08:41.553 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.512+0000 7f1cc8dcf700 1 -- 172.21.15.171:0/3787835640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cc41a0fb0 msgr2=0x7f1cc41a3390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T18:08:41.553 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.512+0000 7f1cc8dcf700 1 --2- 172.21.15.171:0/3787835640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cc41a0fb0 0x7f1cc41a3390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:41.553 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.512+0000 7f1cc8dcf700 1 -- 172.21.15.171:0/3787835640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cc4100930 msgr2=0x7f1cc4100da0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T18:08:41.553 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.512+0000 7f1cc8dcf700 1 --2- 172.21.15.171:0/3787835640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cc4100930 0x7f1cc4100da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:41.554 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.512+0000 7f1cc8dcf700 1 -- 172.21.15.171:0/3787835640 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb800f860 con 0x7f1cb40035c0 2023-07-14T18:08:41.554 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.513+0000 7f1cc8dcf700 1 --2- 172.21.15.171:0/3787835640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cb40035c0 0x7f1cc41003f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40fe760 tx=0x7f1cb8012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:08:41.554 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.513+0000 7f1cc1ffb700 1 -- 172.21.15.171:0/3787835640 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8003d10 con 0x7f1cb40035c0 2023-07-14T18:08:41.554 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.513+0000 7f1cc1ffb700 1 -- 172.21.15.171:0/3787835640 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cb8003e70 con 0x7f1cb40035c0 2023-07-14T18:08:41.554 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.513+0000 7f1ccb033700 1 -- 172.21.15.171:0/3787835640 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc41a4010 con 0x7f1cb40035c0 2023-07-14T18:08:41.554 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.513+0000 7f1cc1ffb700 1 -- 172.21.15.171:0/3787835640 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb80289a0 con 0x7f1cb40035c0 2023-07-14T18:08:41.554 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.513+0000 7f1ccb033700 1 -- 172.21.15.171:0/3787835640 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc41a4560 con 0x7f1cb40035c0 2023-07-14T18:08:41.555 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.514+0000 7f1cc1ffb700 1 -- 172.21.15.171:0/3787835640 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f1cb8024040 con 0x7f1cb40035c0 2023-07-14T18:08:41.555 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.514+0000 7f1cc1ffb700 1 --2- 172.21.15.171:0/3787835640 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f1ca8038c80 0x7f1ca803b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:41.555 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.514+0000 7f1ccb033700 1 -- 172.21.15.171:0/3787835640 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc4061c70 con 0x7f1cb40035c0 2023-07-14T18:08:41.555 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.514+0000 7f1cc1ffb700 1 -- 172.21.15.171:0/3787835640 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1cb804e2e0 con 0x7f1cb40035c0 2023-07-14T18:08:41.556 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.515+0000 7f1cc3fff700 1 --2- 172.21.15.171:0/3787835640 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f1ca8038c80 0x7f1ca803b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:41.556 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.515+0000 7f1cc3fff700 1 --2- 172.21.15.171:0/3787835640 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f1ca8038c80 0x7f1ca803b130 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009850 tx=0x7f1cb0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:08:41.556 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.520+0000 7f1cc1ffb700 1 -- 172.21.15.171:0/3787835640 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f1cb801c760 con 0x7f1cb40035c0 2023-07-14T18:08:41.838 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.802+0000 7f1ccb033700 1 -- 172.21.15.171:0/3787835640 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1cc41a4840 con 0x7f1cb40035c0 2023-07-14T18:08:41.838 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.802+0000 7f1cc1ffb700 1 -- 172.21.15.171:0/3787835640 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1280 (secure 0 0 0) 0x7f1cb802c090 con 0x7f1cb40035c0 2023-07-14T18:08:41.839 INFO:teuthology.orchestra.run.smithi171.stdout: 2023-07-14T18:08:41.839 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":3,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","modified":"2023-07-14T18:08:36.318667Z","created":"2023-07-14T18:06:46.356659Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:3300","nonce":0},{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:3300","nonce":0},{"type":"v1","addr":"172.21.15.171:6789","nonce":0}]},"addr":"172.21.15.171:6789/0","public_addr":"172.21.15.171:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2023-07-14T18:08:41.840 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.805+0000 7f1ccb033700 1 -- 172.21.15.171:0/3787835640 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f1ca8038c80 msgr2=0x7f1ca803b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:41.841 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.805+0000 7f1ccb033700 1 --2- 172.21.15.171:0/3787835640 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f1ca8038c80 0x7f1ca803b130 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009850 tx=0x7f1cb0008040 comp rx=0 tx=0).stop 2023-07-14T18:08:41.841 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.805+0000 7f1ccb033700 1 -- 172.21.15.171:0/3787835640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cb40035c0 msgr2=0x7f1cc41003f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:41.841 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.805+0000 7f1ccb033700 1 --2- 172.21.15.171:0/3787835640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cb40035c0 0x7f1cc41003f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40fe760 tx=0x7f1cb8012c30 comp rx=0 tx=0).stop 2023-07-14T18:08:41.841 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.805+0000 7f1ccb033700 1 -- 172.21.15.171:0/3787835640 shutdown_connections 2023-07-14T18:08:41.842 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.805+0000 7f1ccb033700 1 --2- 172.21.15.171:0/3787835640 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f1ca8038c80 0x7f1ca803b130 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:41.842 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.805+0000 7f1ccb033700 1 --2- 172.21.15.171:0/3787835640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cc41a0fb0 0x7f1cc41a3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:41.842 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.805+0000 7f1ccb033700 1 --2- 172.21.15.171:0/3787835640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cb40035c0 0x7f1cc41003f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:41.842 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.805+0000 7f1ccb033700 1 --2- 172.21.15.171:0/3787835640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cc4100930 0x7f1cc4100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:41.842 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.805+0000 7f1ccb033700 1 -- 172.21.15.171:0/3787835640 >> 172.21.15.171:0/3787835640 conn(0x7f1cc4074500 msgr2=0x7f1cc4102590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:08:41.843 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.805+0000 7f1ccb033700 1 -- 172.21.15.171:0/3787835640 shutdown_connections 2023-07-14T18:08:41.843 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:41.805+0000 7f1ccb033700 1 -- 172.21.15.171:0/3787835640 wait complete. 2023-07-14T18:08:41.843 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 3 2023-07-14T18:08:41.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:41 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T18:08:41.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:41 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:08:41.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:41 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:08:41.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:41 smithi161 ceph-mon[128993]: mon.a calling monitor election 2023-07-14T18:08:41.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:41 smithi161 ceph-mon[128993]: mon.c calling monitor election 2023-07-14T18:08:41.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:41 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:08:41.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:41 smithi161 ceph-mon[128993]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:41.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:41 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:08:41.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:41 smithi161 ceph-mon[128993]: mon.b calling monitor election 2023-07-14T18:08:41.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:41 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:08:41.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:41 smithi161 ceph-mon[128993]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:41.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:41 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:08:41.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:41 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:08:41.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:41 smithi161 ceph-mon[128993]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-14T18:08:41.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:41 smithi161 ceph-mon[128993]: monmap e3: 3 mons at {a=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0],b=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0],c=[v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0]} removed_ranks: {} 2023-07-14T18:08:41.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:41 smithi161 ceph-mon[128993]: fsmap 2023-07-14T18:08:41.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:41 smithi161 ceph-mon[128993]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T18:08:41.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:41 smithi161 ceph-mon[128993]: mgrmap e13: x(active, since 53s) 2023-07-14T18:08:41.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:41 smithi161 ceph-mon[128993]: overall HEALTH_OK 2023-07-14T18:08:41.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:41 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:41.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:41 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:41.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:41 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T18:08:41.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:41 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:08:41.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:41 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:08:41.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:41 smithi171 ceph-mon[129241]: mon.a calling monitor election 2023-07-14T18:08:41.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:41 smithi171 ceph-mon[129241]: mon.c calling monitor election 2023-07-14T18:08:41.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:41 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:08:41.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:41 smithi171 ceph-mon[129241]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:41.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:41 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:08:41.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:41 smithi171 ceph-mon[129241]: mon.b calling monitor election 2023-07-14T18:08:41.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:41 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:08:41.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:41 smithi171 ceph-mon[129241]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:41.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:41 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:08:41.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:41 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:08:41.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:41 smithi171 ceph-mon[129241]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-14T18:08:41.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:41 smithi171 ceph-mon[129241]: monmap e3: 3 mons at {a=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0],b=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0],c=[v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0]} removed_ranks: {} 2023-07-14T18:08:41.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:41 smithi171 ceph-mon[129241]: fsmap 2023-07-14T18:08:41.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:41 smithi171 ceph-mon[129241]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T18:08:41.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:41 smithi171 ceph-mon[129241]: mgrmap e13: x(active, since 53s) 2023-07-14T18:08:41.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:41 smithi171 ceph-mon[129241]: overall HEALTH_OK 2023-07-14T18:08:41.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:41 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:41.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:41 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:41.897 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:41 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T18:08:41.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:41 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:08:41.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:41 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:08:41.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:41 smithi133 ceph-mon[126384]: mon.a calling monitor election 2023-07-14T18:08:41.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:41 smithi133 ceph-mon[126384]: mon.c calling monitor election 2023-07-14T18:08:41.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:41 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:08:41.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:41 smithi133 ceph-mon[126384]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:41.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:41 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:08:41.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:41 smithi133 ceph-mon[126384]: mon.b calling monitor election 2023-07-14T18:08:41.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:41 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:08:41.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:41 smithi133 ceph-mon[126384]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:41.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:41 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:08:41.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:41 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:08:41.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:41 smithi133 ceph-mon[126384]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-14T18:08:41.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:41 smithi133 ceph-mon[126384]: monmap e3: 3 mons at {a=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0],b=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0],c=[v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0]} removed_ranks: {} 2023-07-14T18:08:41.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:41 smithi133 ceph-mon[126384]: fsmap 2023-07-14T18:08:41.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:41 smithi133 ceph-mon[126384]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T18:08:41.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:41 smithi133 ceph-mon[126384]: mgrmap e13: x(active, since 53s) 2023-07-14T18:08:41.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:41 smithi133 ceph-mon[126384]: overall HEALTH_OK 2023-07-14T18:08:41.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:41 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:41.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:41 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:42.252 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-07-14T18:08:42.253 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph config generate-minimal-conf 2023-07-14T18:08:42.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:42 smithi171 ceph-mon[129241]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:42.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:42 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:42.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:42 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:42.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:42 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:08:42.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:42 smithi171 ceph-mon[129241]: from='client.? 172.21.15.171:0/3787835640' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T18:08:42.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:42 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:08:42.648 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:42 smithi133 ceph-mon[126384]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:42.648 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:42 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:42.648 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:42 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:42.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:42 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:08:42.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:42 smithi133 ceph-mon[126384]: from='client.? 172.21.15.171:0/3787835640' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T18:08:42.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:42 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:08:42.842 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:42 smithi161 ceph-mon[128993]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:42.842 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:42 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:42.842 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:42 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:42.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:42 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:08:42.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:42 smithi161 ceph-mon[128993]: from='client.? 172.21.15.171:0/3787835640' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T18:08:42.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:42 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:08:42.990 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:08:43.593 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:08:43 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:08:43.317+0000 7fa68072d700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-07-14T18:08:43.594 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:43 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:43.594 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:43 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:43.594 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:43 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:43.594 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:43 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:43.594 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:43 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:43.594 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:43 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:43.594 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:43 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:08:43.842 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:43 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:43.842 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:43 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:43.842 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:43 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:43.842 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:43 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:43.842 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:43 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:43.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:43 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:43.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:43 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:08:43.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:43 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:43.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:43 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:43.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:43 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:43.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:43 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:43.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:43 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:43.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:43 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:43.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:43 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:08:44.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.460+0000 7f35acad3700 1 -- 172.21.15.133:0/175192962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35a8105740 msgr2=0x7f35a8105b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:44.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.460+0000 7f35acad3700 1 --2- 172.21.15.133:0/175192962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35a8105740 0x7f35a8105b10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3598009910 tx=0x7f3598009c20 comp rx=0 tx=0).stop 2023-07-14T18:08:44.497 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.460+0000 7f35acad3700 1 -- 172.21.15.133:0/175192962 shutdown_connections 2023-07-14T18:08:44.497 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.460+0000 7f35acad3700 1 --2- 172.21.15.133:0/175192962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35a8105740 0x7f35a8105b10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:44.497 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.460+0000 7f35acad3700 1 -- 172.21.15.133:0/175192962 >> 172.21.15.133:0/175192962 conn(0x7f35a80fb280 msgr2=0x7f35a80fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:08:44.497 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.460+0000 7f35acad3700 1 -- 172.21.15.133:0/175192962 shutdown_connections 2023-07-14T18:08:44.497 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.460+0000 7f35acad3700 1 -- 172.21.15.133:0/175192962 wait complete. 2023-07-14T18:08:44.498 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.462+0000 7f35acad3700 1 Processor -- start 2023-07-14T18:08:44.498 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.462+0000 7f35acad3700 1 -- start start 2023-07-14T18:08:44.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.463+0000 7f35acad3700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35a8105740 0x7f35a8072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:44.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.463+0000 7f35acad3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35a8072e80 0x7f35a80732f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:44.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.463+0000 7f35acad3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35a81a0f70 0x7f35a81a3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:44.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.463+0000 7f35acad3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a81a39a0 con 0x7f35a8072e80 2023-07-14T18:08:44.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.463+0000 7f35acad3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a81a3b10 con 0x7f35a81a0f70 2023-07-14T18:08:44.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.463+0000 7f35acad3700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a81a3cd0 con 0x7f35a8105740 2023-07-14T18:08:44.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.463+0000 7f35a659c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35a8105740 0x7f35a8072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:44.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.463+0000 7f35a659c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35a8105740 0x7f35a8072940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:34144/0 (socket says 172.21.15.133:34144) 2023-07-14T18:08:44.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.463+0000 7f35a659c700 1 -- 172.21.15.133:0/2257268963 learned_addr learned my addr 172.21.15.133:0/2257268963 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:08:44.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.463+0000 7f35a5d9b700 1 --2- 172.21.15.133:0/2257268963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35a8072e80 0x7f35a80732f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:44.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.463+0000 7f35a6d9d700 1 --2- 172.21.15.133:0/2257268963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35a81a0f70 0x7f35a81a3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:44.501 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.463+0000 7f35a659c700 1 -- 172.21.15.133:0/2257268963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35a81a0f70 msgr2=0x7f35a81a3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:44.501 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.463+0000 7f35a659c700 1 --2- 172.21.15.133:0/2257268963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35a81a0f70 0x7f35a81a3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:44.501 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.463+0000 7f35a659c700 1 -- 172.21.15.133:0/2257268963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35a8072e80 msgr2=0x7f35a80732f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:44.501 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.463+0000 7f35a659c700 1 --2- 172.21.15.133:0/2257268963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35a8072e80 0x7f35a80732f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:44.501 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.463+0000 7f35a659c700 1 -- 172.21.15.133:0/2257268963 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35980095f0 con 0x7f35a8105740 2023-07-14T18:08:44.501 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.463+0000 7f35a5d9b700 1 --2- 172.21.15.133:0/2257268963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35a8072e80 0x7f35a80732f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:08:44.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.464+0000 7f35a659c700 1 --2- 172.21.15.133:0/2257268963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35a8105740 0x7f35a8072940 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3598004cb0 tx=0x7f3598004ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:08:44.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.464+0000 7f35977fe700 1 -- 172.21.15.133:0/2257268963 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359801d070 con 0x7f35a8105740 2023-07-14T18:08:44.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.464+0000 7f35acad3700 1 -- 172.21.15.133:0/2257268963 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a81a3f50 con 0x7f35a8105740 2023-07-14T18:08:44.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.464+0000 7f35977fe700 1 -- 172.21.15.133:0/2257268963 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f359800be20 con 0x7f35a8105740 2023-07-14T18:08:44.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.464+0000 7f35977fe700 1 -- 172.21.15.133:0/2257268963 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3598022e40 con 0x7f35a8105740 2023-07-14T18:08:44.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.464+0000 7f35acad3700 1 -- 172.21.15.133:0/2257268963 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a81a4360 con 0x7f35a8105740 2023-07-14T18:08:44.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.465+0000 7f35977fe700 1 -- 172.21.15.133:0/2257268963 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f359800b960 con 0x7f35a8105740 2023-07-14T18:08:44.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.465+0000 7f35acad3700 1 -- 172.21.15.133:0/2257268963 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a8061c70 con 0x7f35a8105740 2023-07-14T18:08:44.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.465+0000 7f35977fe700 1 --2- 172.21.15.133:0/2257268963 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3590038c30 0x7f359003b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:44.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.465+0000 7f35977fe700 1 -- 172.21.15.133:0/2257268963 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f359804c850 con 0x7f35a8105740 2023-07-14T18:08:44.504 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.466+0000 7f35a5d9b700 1 --2- 172.21.15.133:0/2257268963 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3590038c30 0x7f359003b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:44.504 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.466+0000 7f35a5d9b700 1 --2- 172.21.15.133:0/2257268963 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3590038c30 0x7f359003b0e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f359c009ec0 tx=0x7f359c009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:08:44.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.472+0000 7f35977fe700 1 -- 172.21.15.133:0/2257268963 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f359800e850 con 0x7f35a8105740 2023-07-14T18:08:44.721 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.685+0000 7f35acad3700 1 -- 172.21.15.133:0/2257268963 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f35a8073e70 con 0x7f35a8105740 2023-07-14T18:08:44.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.686+0000 7f35977fe700 1 -- 172.21.15.133:0/2257268963 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+277 (secure 0 0 0) 0x7f3598027070 con 0x7f35a8105740 2023-07-14T18:08:44.722 INFO:teuthology.orchestra.run.smithi133.stdout:# minimal ceph.conf for 06d0f5fa-2271-11ee-9b33-001a4aab830c 2023-07-14T18:08:44.722 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2023-07-14T18:08:44.722 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = 06d0f5fa-2271-11ee-9b33-001a4aab830c 2023-07-14T18:08:44.722 INFO:teuthology.orchestra.run.smithi133.stdout: mon_host = [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] 2023-07-14T18:08:44.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.688+0000 7f35acad3700 1 -- 172.21.15.133:0/2257268963 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3590038c30 msgr2=0x7f359003b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:44.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.688+0000 7f35acad3700 1 --2- 172.21.15.133:0/2257268963 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3590038c30 0x7f359003b0e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f359c009ec0 tx=0x7f359c009210 comp rx=0 tx=0).stop 2023-07-14T18:08:44.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.688+0000 7f35acad3700 1 -- 172.21.15.133:0/2257268963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35a8105740 msgr2=0x7f35a8072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:44.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.688+0000 7f35acad3700 1 --2- 172.21.15.133:0/2257268963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35a8105740 0x7f35a8072940 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3598004cb0 tx=0x7f3598004ce0 comp rx=0 tx=0).stop 2023-07-14T18:08:44.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.689+0000 7f35acad3700 1 -- 172.21.15.133:0/2257268963 shutdown_connections 2023-07-14T18:08:44.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.689+0000 7f35acad3700 1 --2- 172.21.15.133:0/2257268963 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3590038c30 0x7f359003b0e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:44.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.689+0000 7f35acad3700 1 --2- 172.21.15.133:0/2257268963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35a81a0f70 0x7f35a81a3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:44.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.689+0000 7f35acad3700 1 --2- 172.21.15.133:0/2257268963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35a8105740 0x7f35a8072940 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:44.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.689+0000 7f35acad3700 1 --2- 172.21.15.133:0/2257268963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35a8072e80 0x7f35a80732f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:44.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.689+0000 7f35acad3700 1 -- 172.21.15.133:0/2257268963 >> 172.21.15.133:0/2257268963 conn(0x7f35a80fb280 msgr2=0x7f35a80fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:08:44.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.689+0000 7f35acad3700 1 -- 172.21.15.133:0/2257268963 shutdown_connections 2023-07-14T18:08:44.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:44.689+0000 7f35acad3700 1 -- 172.21.15.133:0/2257268963 wait complete. 2023-07-14T18:08:45.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:44 smithi161 ceph-mon[128993]: Updating smithi133:/etc/ceph/ceph.conf 2023-07-14T18:08:45.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:44 smithi161 ceph-mon[128993]: Updating smithi161:/etc/ceph/ceph.conf 2023-07-14T18:08:45.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:44 smithi161 ceph-mon[128993]: Updating smithi171:/etc/ceph/ceph.conf 2023-07-14T18:08:45.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:44 smithi161 ceph-mon[128993]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:45.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:44 smithi161 ceph-mon[128993]: Updating smithi161:/var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T18:08:45.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:44 smithi161 ceph-mon[128993]: Updating smithi171:/var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T18:08:45.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:44 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:45.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:44 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:45.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:44 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:45.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:44 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:45.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:44 smithi161 ceph-mon[128993]: Updating smithi133:/var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T18:08:45.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:44 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:45.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:44 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:45.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:44 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:45.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:44 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T18:08:45.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:44 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T18:08:45.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:44 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:45.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:44 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2257268963' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:45.096 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:44 smithi133 ceph-mon[126384]: Updating smithi133:/etc/ceph/ceph.conf 2023-07-14T18:08:45.097 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:44 smithi133 ceph-mon[126384]: Updating smithi161:/etc/ceph/ceph.conf 2023-07-14T18:08:45.097 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:44 smithi133 ceph-mon[126384]: Updating smithi171:/etc/ceph/ceph.conf 2023-07-14T18:08:45.097 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:44 smithi133 ceph-mon[126384]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:45.097 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:44 smithi133 ceph-mon[126384]: Updating smithi161:/var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T18:08:45.098 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:44 smithi133 ceph-mon[126384]: Updating smithi171:/var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T18:08:45.098 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:44 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:45.098 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:44 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:45.098 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:44 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:45.098 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:44 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:45.099 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:44 smithi133 ceph-mon[126384]: Updating smithi133:/var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T18:08:45.099 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:44 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:45.099 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:44 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:45.099 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:44 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:45.100 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:44 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T18:08:45.100 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:44 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T18:08:45.100 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:44 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:45.100 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:44 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2257268963' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:45.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:44 smithi171 ceph-mon[129241]: Updating smithi133:/etc/ceph/ceph.conf 2023-07-14T18:08:45.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:44 smithi171 ceph-mon[129241]: Updating smithi161:/etc/ceph/ceph.conf 2023-07-14T18:08:45.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:44 smithi171 ceph-mon[129241]: Updating smithi171:/etc/ceph/ceph.conf 2023-07-14T18:08:45.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:44 smithi171 ceph-mon[129241]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:45.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:44 smithi171 ceph-mon[129241]: Updating smithi161:/var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T18:08:45.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:44 smithi171 ceph-mon[129241]: Updating smithi171:/var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T18:08:45.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:44 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:45.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:44 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:45.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:44 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:45.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:44 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:45.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:44 smithi171 ceph-mon[129241]: Updating smithi133:/var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T18:08:45.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:44 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:45.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:44 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:45.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:44 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:45.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:44 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T18:08:45.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:44 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T18:08:45.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:44 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:45.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:44 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2257268963' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:45.177 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-07-14T18:08:45.177 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T18:08:45.178 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.conf 2023-07-14T18:08:45.254 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T18:08:45.254 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-14T18:08:45.363 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T18:08:45.363 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.conf 2023-07-14T18:08:45.400 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T18:08:45.400 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-14T18:08:45.471 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2023-07-14T18:08:45.471 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.conf 2023-07-14T18:08:45.507 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2023-07-14T18:08:45.507 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-14T18:08:45.577 INFO:tasks.cephadm:Adding mgr.x on smithi133 2023-07-14T18:08:45.578 INFO:tasks.cephadm:Adding mgr.y on smithi161 2023-07-14T18:08:45.578 INFO:tasks.cephadm:Adding mgr.z on smithi171 2023-07-14T18:08:45.578 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph orch apply mgr '3;smithi133=x;smithi161=y;smithi171=z' 2023-07-14T18:08:45.897 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:45 smithi133 ceph-mon[126384]: Reconfiguring mon.a (unknown last config time)... 2023-07-14T18:08:45.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:45 smithi133 ceph-mon[126384]: Reconfiguring daemon mon.a on smithi133 2023-07-14T18:08:46.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:45 smithi161 ceph-mon[128993]: Reconfiguring mon.a (unknown last config time)... 2023-07-14T18:08:46.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:45 smithi161 ceph-mon[128993]: Reconfiguring daemon mon.a on smithi133 2023-07-14T18:08:46.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:45 smithi171 ceph-mon[129241]: Reconfiguring mon.a (unknown last config time)... 2023-07-14T18:08:46.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:45 smithi171 ceph-mon[129241]: Reconfiguring daemon mon.a on smithi133 2023-07-14T18:08:46.171 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T18:08:47.024 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:46 smithi161 ceph-mon[128993]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:47.024 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:46 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:47.025 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:46 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:47.025 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:46 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T18:08:47.025 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:46 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T18:08:47.025 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:46 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:47.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:46 smithi171 ceph-mon[129241]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:47.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:46 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:47.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:46 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:47.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:46 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T18:08:47.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:46 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T18:08:47.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:46 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:47.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:46 smithi133 ceph-mon[126384]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:47.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:46 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:47.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:46 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:47.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:46 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T18:08:47.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:46 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T18:08:47.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:46 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:47.380 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.344+0000 7fa74b8f2700 1 -- 172.21.15.171:0/4281058196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7440fe730 msgr2=0x7fa7440feb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:47.381 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.344+0000 7fa74b8f2700 1 --2- 172.21.15.171:0/4281058196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7440fe730 0x7fa7440feb40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa740009910 tx=0x7fa740009c20 comp rx=0 tx=0).stop 2023-07-14T18:08:47.381 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.344+0000 7fa74b8f2700 1 -- 172.21.15.171:0/4281058196 shutdown_connections 2023-07-14T18:08:47.381 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.344+0000 7fa74b8f2700 1 --2- 172.21.15.171:0/4281058196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7440fe730 0x7fa7440feb40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:47.381 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.344+0000 7fa74b8f2700 1 -- 172.21.15.171:0/4281058196 >> 172.21.15.171:0/4281058196 conn(0x7fa7440f9ce0 msgr2=0x7fa7440fc110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:08:47.381 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.344+0000 7fa74b8f2700 1 -- 172.21.15.171:0/4281058196 shutdown_connections 2023-07-14T18:08:47.381 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.344+0000 7fa74b8f2700 1 -- 172.21.15.171:0/4281058196 wait complete. 2023-07-14T18:08:47.383 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.345+0000 7fa74b8f2700 1 Processor -- start 2023-07-14T18:08:47.383 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.345+0000 7fa74b8f2700 1 -- start start 2023-07-14T18:08:47.384 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.345+0000 7fa74b8f2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7440fe730 0x7fa7441939f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:47.384 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.345+0000 7fa74b8f2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa744193f30 0x7fa744198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:47.384 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.345+0000 7fa74b8f2700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa7441943a0 0x7fa74419b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:47.384 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.345+0000 7fa74b8f2700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74419bcb0 con 0x7fa7440fe730 2023-07-14T18:08:47.384 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.345+0000 7fa74b8f2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74419be20 con 0x7fa744193f30 2023-07-14T18:08:47.384 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.345+0000 7fa74b8f2700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74419bfe0 con 0x7fa7441943a0 2023-07-14T18:08:47.384 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.345+0000 7fa749e8f700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa7441943a0 0x7fa74419b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:47.385 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.345+0000 7fa749e8f700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa7441943a0 0x7fa74419b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:42734/0 (socket says 172.21.15.171:42734) 2023-07-14T18:08:47.385 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.345+0000 7fa749e8f700 1 -- 172.21.15.171:0/2700357062 learned_addr learned my addr 172.21.15.171:0/2700357062 (peer_addr_for_me v2:172.21.15.171:0/0) 2023-07-14T18:08:47.385 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.345+0000 7fa74968e700 1 --2- 172.21.15.171:0/2700357062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7440fe730 0x7fa7441939f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:47.385 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.345+0000 7fa748e8d700 1 --2- 172.21.15.171:0/2700357062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa744193f30 0x7fa744198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:47.385 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.346+0000 7fa749e8f700 1 -- 172.21.15.171:0/2700357062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa744193f30 msgr2=0x7fa744198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:47.385 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.346+0000 7fa749e8f700 1 --2- 172.21.15.171:0/2700357062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa744193f30 0x7fa744198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:47.385 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.346+0000 7fa749e8f700 1 -- 172.21.15.171:0/2700357062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7440fe730 msgr2=0x7fa7441939f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:47.386 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.346+0000 7fa749e8f700 1 --2- 172.21.15.171:0/2700357062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7440fe730 0x7fa7441939f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:47.386 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.346+0000 7fa749e8f700 1 -- 172.21.15.171:0/2700357062 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7400095f0 con 0x7fa7441943a0 2023-07-14T18:08:47.386 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.346+0000 7fa749e8f700 1 --2- 172.21.15.171:0/2700357062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa7441943a0 0x7fa74419b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00df40 tx=0x7fa73c00df70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:08:47.386 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.346+0000 7fa73a7fc700 1 -- 172.21.15.171:0/2700357062 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c00e720 con 0x7fa7441943a0 2023-07-14T18:08:47.386 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.346+0000 7fa73a7fc700 1 -- 172.21.15.171:0/2700357062 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa73c00bb40 con 0x7fa7441943a0 2023-07-14T18:08:47.386 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.346+0000 7fa73a7fc700 1 -- 172.21.15.171:0/2700357062 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c0163f0 con 0x7fa7441943a0 2023-07-14T18:08:47.386 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.346+0000 7fa74b8f2700 1 -- 172.21.15.171:0/2700357062 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa744067da0 con 0x7fa7441943a0 2023-07-14T18:08:47.386 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.346+0000 7fa74b8f2700 1 -- 172.21.15.171:0/2700357062 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7440682d0 con 0x7fa7441943a0 2023-07-14T18:08:47.387 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.347+0000 7fa73a7fc700 1 -- 172.21.15.171:0/2700357062 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7fa73c016550 con 0x7fa7441943a0 2023-07-14T18:08:47.387 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.347+0000 7fa73a7fc700 1 --2- 172.21.15.171:0/2700357062 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa730038c80 0x7fa73003b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:47.387 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.347+0000 7fa73a7fc700 1 -- 172.21.15.171:0/2700357062 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa73c04c7c0 con 0x7fa7441943a0 2023-07-14T18:08:47.387 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.347+0000 7fa74b8f2700 1 -- 172.21.15.171:0/2700357062 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa744061c70 con 0x7fa7441943a0 2023-07-14T18:08:47.387 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.347+0000 7fa74968e700 1 --2- 172.21.15.171:0/2700357062 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa730038c80 0x7fa73003b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:47.388 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.348+0000 7fa74968e700 1 --2- 172.21.15.171:0/2700357062 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa730038c80 0x7fa73003b130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa740006010 tx=0x7fa7400058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:08:47.388 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.352+0000 7fa73a7fc700 1 -- 172.21.15.171:0/2700357062 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fa73c014710 con 0x7fa7441943a0 2023-07-14T18:08:47.593 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.557+0000 7fa74b8f2700 1 -- 172.21.15.171:0/2700357062 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi133=x;smithi161=y;smithi171=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7440684f0 con 0x7fa730038c80 2023-07-14T18:08:47.600 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.564+0000 7fa73a7fc700 1 -- 172.21.15.171:0/2700357062 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa7440684f0 con 0x7fa730038c80 2023-07-14T18:08:47.600 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled mgr update... 2023-07-14T18:08:47.602 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.566+0000 7fa74b8f2700 1 -- 172.21.15.171:0/2700357062 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa730038c80 msgr2=0x7fa73003b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:47.602 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.566+0000 7fa74b8f2700 1 --2- 172.21.15.171:0/2700357062 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa730038c80 0x7fa73003b130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa740006010 tx=0x7fa7400058e0 comp rx=0 tx=0).stop 2023-07-14T18:08:47.603 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.566+0000 7fa74b8f2700 1 -- 172.21.15.171:0/2700357062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa7441943a0 msgr2=0x7fa74419b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:47.603 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.566+0000 7fa74b8f2700 1 --2- 172.21.15.171:0/2700357062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa7441943a0 0x7fa74419b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00df40 tx=0x7fa73c00df70 comp rx=0 tx=0).stop 2023-07-14T18:08:47.603 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.566+0000 7fa74b8f2700 1 -- 172.21.15.171:0/2700357062 shutdown_connections 2023-07-14T18:08:47.603 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.566+0000 7fa74b8f2700 1 --2- 172.21.15.171:0/2700357062 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa730038c80 0x7fa73003b130 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:47.603 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.566+0000 7fa74b8f2700 1 --2- 172.21.15.171:0/2700357062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa7441943a0 0x7fa74419b510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:47.603 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.566+0000 7fa74b8f2700 1 --2- 172.21.15.171:0/2700357062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7440fe730 0x7fa7441939f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:47.603 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.566+0000 7fa74b8f2700 1 --2- 172.21.15.171:0/2700357062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa744193f30 0x7fa744198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:47.603 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.566+0000 7fa74b8f2700 1 -- 172.21.15.171:0/2700357062 >> 172.21.15.171:0/2700357062 conn(0x7fa7440f9ce0 msgr2=0x7fa7440fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:08:47.604 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.566+0000 7fa74b8f2700 1 -- 172.21.15.171:0/2700357062 shutdown_connections 2023-07-14T18:08:47.604 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:08:47.566+0000 7fa74b8f2700 1 -- 172.21.15.171:0/2700357062 wait complete. 2023-07-14T18:08:47.985 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:47 smithi171 ceph-mon[129241]: Reconfiguring mon.b (monmap changed)... 2023-07-14T18:08:47.985 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:47 smithi171 ceph-mon[129241]: Reconfiguring daemon mon.b on smithi161 2023-07-14T18:08:47.986 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:47 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:47.986 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:47 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:47.986 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:47 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T18:08:47.986 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:47 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T18:08:47.986 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:47 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:47.986 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:47 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:48.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:47 smithi161 ceph-mon[128993]: Reconfiguring mon.b (monmap changed)... 2023-07-14T18:08:48.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:47 smithi161 ceph-mon[128993]: Reconfiguring daemon mon.b on smithi161 2023-07-14T18:08:48.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:47 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:48.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:47 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:48.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:47 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T18:08:48.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:47 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T18:08:48.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:47 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:48.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:47 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:48.095 DEBUG:teuthology.orchestra.run.smithi161:mgr.y> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mgr.y.service 2023-07-14T18:08:48.097 DEBUG:teuthology.orchestra.run.smithi171:mgr.z> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mgr.z.service 2023-07-14T18:08:48.099 INFO:tasks.cephadm:Deploying OSDs... 2023-07-14T18:08:48.100 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T18:08:48.100 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/scratch_devs of=/dev/stdout 2023-07-14T18:08:48.128 INFO:journalctl@ceph.mgr.y.smithi161.stdout:-- Logs begin at Fri 2023-07-14 17:41:43 UTC. -- 2023-07-14T18:08:48.129 INFO:journalctl@ceph.mgr.z.smithi171.stdout:-- Logs begin at Fri 2023-07-14 17:41:48 UTC. -- 2023-07-14T18:08:48.148 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-14T18:08:48.149 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_1 2023-07-14T18:08:48.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:47 smithi133 ceph-mon[126384]: Reconfiguring mon.b (monmap changed)... 2023-07-14T18:08:48.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:47 smithi133 ceph-mon[126384]: Reconfiguring daemon mon.b on smithi161 2023-07-14T18:08:48.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:47 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:48.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:47 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:48.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:47 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T18:08:48.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:47 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T18:08:48.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:47 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:48.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:47 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:48.235 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-14T18:08:48.235 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T18:08:48.236 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-07-14T18:08:48.236 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T18:08:48.236 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T18:08:48.236 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2023-07-14 18:07:59.059406715 +0000 2023-07-14T18:08:48.236 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2023-07-14 18:00:06.915038482 +0000 2023-07-14T18:08:48.236 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2023-07-14 18:00:06.915038482 +0000 2023-07-14T18:08:48.236 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2023-07-14 18:00:06.886038978 +0000 2023-07-14T18:08:48.237 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-14T18:08:48.304 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2023-07-14T18:08:48.304 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2023-07-14T18:08:48.304 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000270344 s, 1.9 MB/s 2023-07-14T18:08:48.306 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-14T18:08:48.351 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_2 2023-07-14T18:08:48.434 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-14T18:08:48.435 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T18:08:48.435 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-07-14T18:08:48.435 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T18:08:48.435 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T18:08:48.435 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2023-07-14 18:07:59.059406715 +0000 2023-07-14T18:08:48.435 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2023-07-14 18:00:07.758024029 +0000 2023-07-14T18:08:48.435 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2023-07-14 18:00:07.758024029 +0000 2023-07-14T18:08:48.436 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2023-07-14 18:00:07.730024509 +0000 2023-07-14T18:08:48.436 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-14T18:08:48.538 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2023-07-14T18:08:48.539 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2023-07-14T18:08:48.539 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000324632 s, 1.6 MB/s 2023-07-14T18:08:48.541 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-14T18:08:48.586 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_3 2023-07-14T18:08:48.669 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-14T18:08:48.669 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T18:08:48.669 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-07-14T18:08:48.669 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T18:08:48.670 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T18:08:48.670 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2023-07-14 18:07:59.059406715 +0000 2023-07-14T18:08:48.670 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2023-07-14 18:00:08.606009492 +0000 2023-07-14T18:08:48.670 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2023-07-14 18:00:08.606009492 +0000 2023-07-14T18:08:48.670 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2023-07-14 18:00:08.579009955 +0000 2023-07-14T18:08:48.671 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-14T18:08:48.774 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2023-07-14T18:08:48.774 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2023-07-14T18:08:48.774 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000365227 s, 1.4 MB/s 2023-07-14T18:08:48.776 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-14T18:08:48.824 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_4 2023-07-14T18:08:48.870 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-14T18:08:48.870 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T18:08:48.870 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-07-14T18:08:48.870 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T18:08:48.870 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T18:08:48.870 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2023-07-14 18:07:59.059406715 +0000 2023-07-14T18:08:48.871 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2023-07-14 18:00:09.451994990 +0000 2023-07-14T18:08:48.871 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2023-07-14 18:00:09.451994990 +0000 2023-07-14T18:08:48.871 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2023-07-14 18:00:09.424995453 +0000 2023-07-14T18:08:48.871 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-14T18:08:48.974 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2023-07-14T18:08:48.974 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2023-07-14T18:08:48.974 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000331521 s, 1.5 MB/s 2023-07-14T18:08:48.976 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-14T18:08:49.021 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T18:08:49.022 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/scratch_devs of=/dev/stdout 2023-07-14T18:08:49.042 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-14T18:08:49.043 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_1 2023-07-14T18:08:49.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:48 smithi161 ceph-mon[128993]: Reconfiguring mon.c (monmap changed)... 2023-07-14T18:08:49.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:48 smithi161 ceph-mon[128993]: Reconfiguring daemon mon.c on smithi171 2023-07-14T18:08:49.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:48 smithi161 ceph-mon[128993]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:49.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:48 smithi161 ceph-mon[128993]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi133=x;smithi161=y;smithi171=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:08:49.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:48 smithi161 ceph-mon[128993]: Saving service mgr spec with placement smithi133=x;smithi161=y;smithi171=z;count:3 2023-07-14T18:08:49.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:49.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:49.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:08:49.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:49.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:08:49.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:49.103 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-14T18:08:49.103 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T18:08:49.103 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 377317 Links: 1 2023-07-14T18:08:49.103 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T18:08:49.104 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T18:08:49.104 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2023-07-14 18:08:35.754134324 +0000 2023-07-14T18:08:49.104 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2023-07-14 17:57:26.857843695 +0000 2023-07-14T18:08:49.104 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2023-07-14 17:57:26.857843695 +0000 2023-07-14T18:08:49.104 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2023-07-14T18:08:49.104 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-14T18:08:49.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:48 smithi171 ceph-mon[129241]: Reconfiguring mon.c (monmap changed)... 2023-07-14T18:08:49.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:48 smithi171 ceph-mon[129241]: Reconfiguring daemon mon.c on smithi171 2023-07-14T18:08:49.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:48 smithi171 ceph-mon[129241]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:49.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:48 smithi171 ceph-mon[129241]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi133=x;smithi161=y;smithi171=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:08:49.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:48 smithi171 ceph-mon[129241]: Saving service mgr spec with placement smithi133=x;smithi161=y;smithi171=z;count:3 2023-07-14T18:08:49.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:49.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:49.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:08:49.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:49.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:08:49.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:49.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:48 smithi133 ceph-mon[126384]: Reconfiguring mon.c (monmap changed)... 2023-07-14T18:08:49.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:48 smithi133 ceph-mon[126384]: Reconfiguring daemon mon.c on smithi171 2023-07-14T18:08:49.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:48 smithi133 ceph-mon[126384]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:49.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:48 smithi133 ceph-mon[126384]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi133=x;smithi161=y;smithi171=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:08:49.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:48 smithi133 ceph-mon[126384]: Saving service mgr spec with placement smithi133=x;smithi161=y;smithi171=z;count:3 2023-07-14T18:08:49.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:49.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:49.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:08:49.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:49.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:08:49.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:49.173 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2023-07-14T18:08:49.173 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2023-07-14T18:08:49.173 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000273547 s, 1.9 MB/s 2023-07-14T18:08:49.174 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-14T18:08:49.235 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_2 2023-07-14T18:08:49.294 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-14T18:08:49.295 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T18:08:49.295 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 381636 Links: 1 2023-07-14T18:08:49.295 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T18:08:49.295 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T18:08:49.295 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2023-07-14 18:08:35.755134305 +0000 2023-07-14T18:08:49.295 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2023-07-14 17:57:27.365833807 +0000 2023-07-14T18:08:49.295 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2023-07-14 17:57:27.365833807 +0000 2023-07-14T18:08:49.296 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2023-07-14T18:08:49.296 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-14T18:08:49.362 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2023-07-14T18:08:49.362 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2023-07-14T18:08:49.362 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000251947 s, 2.0 MB/s 2023-07-14T18:08:49.363 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-14T18:08:49.422 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_3 2023-07-14T18:08:49.480 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-14T18:08:49.480 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T18:08:49.480 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 377473 Links: 1 2023-07-14T18:08:49.481 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T18:08:49.481 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T18:08:49.481 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2023-07-14 18:08:35.755134305 +0000 2023-07-14T18:08:49.481 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2023-07-14 17:57:27.881823764 +0000 2023-07-14T18:08:49.481 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2023-07-14 17:57:27.881823764 +0000 2023-07-14T18:08:49.481 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2023-07-14T18:08:49.482 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-14T18:08:49.547 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2023-07-14T18:08:49.547 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2023-07-14T18:08:49.547 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000220789 s, 2.3 MB/s 2023-07-14T18:08:49.548 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-14T18:08:49.606 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_4 2023-07-14T18:08:49.664 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-14T18:08:49.664 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T18:08:49.664 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 374738 Links: 1 2023-07-14T18:08:49.664 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T18:08:49.665 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T18:08:49.665 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2023-07-14 18:08:35.755134305 +0000 2023-07-14T18:08:49.665 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2023-07-14 17:57:28.398813701 +0000 2023-07-14T18:08:49.665 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2023-07-14 17:57:28.398813701 +0000 2023-07-14T18:08:49.665 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2023-07-14T18:08:49.665 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-14T18:08:49.729 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2023-07-14T18:08:49.729 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2023-07-14T18:08:49.729 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000194207 s, 2.6 MB/s 2023-07-14T18:08:49.730 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-14T18:08:49.788 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2023-07-14T18:08:49.789 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/scratch_devs of=/dev/stdout 2023-07-14T18:08:49.806 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-14T18:08:49.806 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_1 2023-07-14T18:08:49.863 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-14T18:08:49.864 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T18:08:49.864 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 6h/6d Inode: 376446 Links: 1 2023-07-14T18:08:49.864 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T18:08:49.864 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T18:08:49.865 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2023-07-14 18:08:27.663996265 +0000 2023-07-14T18:08:49.865 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2023-07-14 17:57:33.315274046 +0000 2023-07-14T18:08:49.865 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2023-07-14 17:57:33.315274046 +0000 2023-07-14T18:08:49.865 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: - 2023-07-14T18:08:49.865 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-14T18:08:49.931 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2023-07-14T18:08:49.931 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2023-07-14T18:08:49.931 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000246935 s, 2.1 MB/s 2023-07-14T18:08:49.932 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-14T18:08:49.969 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:49 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T18:08:49.969 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:49 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T18:08:49.969 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:49 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T18:08:49.969 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:49 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:49.969 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:49 smithi171 ceph-mon[129241]: Deploying daemon mgr.z on smithi171 2023-07-14T18:08:49.986 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_2 2023-07-14T18:08:50.043 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-14T18:08:50.043 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T18:08:50.043 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 6h/6d Inode: 376516 Links: 1 2023-07-14T18:08:50.044 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T18:08:50.044 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T18:08:50.044 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2023-07-14 18:08:27.663996265 +0000 2023-07-14T18:08:50.044 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2023-07-14 17:57:33.849264529 +0000 2023-07-14T18:08:50.044 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2023-07-14 17:57:33.849264529 +0000 2023-07-14T18:08:50.044 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: - 2023-07-14T18:08:50.045 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-14T18:08:50.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:49 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T18:08:50.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:49 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T18:08:50.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:49 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T18:08:50.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:49 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:50.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:49 smithi161 ceph-mon[128993]: Deploying daemon mgr.z on smithi171 2023-07-14T18:08:50.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:49 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T18:08:50.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:49 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T18:08:50.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:49 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T18:08:50.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:49 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:50.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:49 smithi133 ceph-mon[126384]: Deploying daemon mgr.z on smithi171 2023-07-14T18:08:50.149 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2023-07-14T18:08:50.150 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2023-07-14T18:08:50.150 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.00023262 s, 2.2 MB/s 2023-07-14T18:08:50.150 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-14T18:08:50.168 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_3 2023-07-14T18:08:50.226 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-14T18:08:50.227 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T18:08:50.227 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 6h/6d Inode: 381436 Links: 1 2023-07-14T18:08:50.227 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T18:08:50.227 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T18:08:50.228 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2023-07-14 18:08:50.063578962 +0000 2023-07-14T18:08:50.228 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2023-07-14 17:57:34.378255101 +0000 2023-07-14T18:08:50.228 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2023-07-14 17:57:34.378255101 +0000 2023-07-14T18:08:50.228 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: - 2023-07-14T18:08:50.229 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-14T18:08:50.298 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2023-07-14T18:08:50.298 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2023-07-14T18:08:50.298 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000213599 s, 2.4 MB/s 2023-07-14T18:08:50.300 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-14T18:08:50.359 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_4 2023-07-14T18:08:50.418 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-14T18:08:50.418 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T18:08:50.418 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 6h/6d Inode: 381650 Links: 1 2023-07-14T18:08:50.418 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T18:08:50.418 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T18:08:50.418 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2023-07-14 18:08:50.191576577 +0000 2023-07-14T18:08:50.419 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2023-07-14 17:57:34.899245817 +0000 2023-07-14T18:08:50.419 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2023-07-14 17:57:34.899245817 +0000 2023-07-14T18:08:50.419 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: - 2023-07-14T18:08:50.419 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-14T18:08:50.484 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2023-07-14T18:08:50.484 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2023-07-14T18:08:50.484 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000168403 s, 3.0 MB/s 2023-07-14T18:08:50.485 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-14T18:08:50.542 INFO:tasks.cephadm:Deploying osd.0 on smithi133 with /dev/vg_nvme/lv_4... 2023-07-14T18:08:50.543 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-14T18:08:50.568 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:50 smithi171 systemd[1]: Starting Ceph mgr.z for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:08:50.835 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:50 smithi171 podman[131293]: 2023-07-14T18:08:50.836 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:50 smithi171 bash[131293]: b1154041911895b8bb8ec0381bd76a80e7e6dec75c7e92448674ae4888fb10ac 2023-07-14T18:08:50.836 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:50 smithi171 ceph-mgr[131376]: -- 172.21.15.171:0/788848620 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555f24fbf1e0 con 0x555f24f94c00 2023-07-14T18:08:50.897 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:50 smithi133 ceph-mon[126384]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:51.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:50 smithi161 ceph-mon[128993]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:51.103 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:50 smithi171 ceph-mon[129241]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:51.104 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:50 smithi171 systemd[1]: Started Ceph mgr.z for 06d0f5fa-2271-11ee-9b33-001a4aab830c. 2023-07-14T18:08:51.104 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:50 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-z[131352]: 2023-07-14T18:08:50.899+0000 7fc6daafc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T18:08:51.104 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:50 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-z[131352]: 2023-07-14T18:08:50.979+0000 7fc6daafc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T18:08:51.268 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:08:51.357 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:51 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-z[131352]: 2023-07-14T18:08:51.067+0000 7fc6daafc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T18:08:51.357 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:51 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-z[131352]: 2023-07-14T18:08:51.159+0000 7fc6daafc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T18:08:51.669 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:51 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-z[131352]: 2023-07-14T18:08:51.326+0000 7fc6daafc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T18:08:51.669 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:51 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-z[131352]: 2023-07-14T18:08:51.633+0000 7fc6daafc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T18:08:51.925 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:51 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-z[131352]: 2023-07-14T18:08:51.721+0000 7fc6daafc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T18:08:51.925 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:51 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-z[131352]: 2023-07-14T18:08:51.855+0000 7fc6daafc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T18:08:52.124 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:51 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:52.124 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:51 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:52.124 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:51 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:52.124 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:51 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T18:08:52.125 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:51 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T18:08:52.125 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:51 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T18:08:52.125 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:51 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:52.137 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:51 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:52.137 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:51 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:52.137 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:51 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:52.137 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:51 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T18:08:52.137 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:51 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T18:08:52.137 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:51 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T18:08:52.137 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:51 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:52.357 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:51 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-z[131352]: 2023-07-14T18:08:51.951+0000 7fc6daafc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T18:08:52.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:51 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:52.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:51 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:52.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:51 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:52.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:51 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T18:08:52.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:51 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T18:08:52.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:51 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T18:08:52.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:51 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:52.692 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:52 smithi161 systemd[1]: Starting Ceph mgr.y for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:08:52.857 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:52 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-z[131352]: 2023-07-14T18:08:52.368+0000 7fc6daafc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T18:08:52.951 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:52 smithi161 podman[130431]: 2023-07-14T18:08:52.951 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:52 smithi161 ceph-mgr[130471]: -- 172.21.15.161:0/2853187742 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560aa55f31e0 con 0x560aa55c8800 2023-07-14T18:08:53.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:52 smithi161 ceph-mon[128993]: Deploying daemon mgr.y on smithi161 2023-07-14T18:08:53.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:52 smithi161 ceph-mon[128993]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:53.223 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:52 smithi161 bash[130431]: fa7f8318e07201b8acb152f8170f34f5202c294b2464517d1e5f73622d4397f4 2023-07-14T18:08:53.224 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:52 smithi161 systemd[1]: Started Ceph mgr.y for 06d0f5fa-2271-11ee-9b33-001a4aab830c. 2023-07-14T18:08:53.224 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:53 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-y[130447]: 2023-07-14T18:08:53.014+0000 7f0d6546d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T18:08:53.224 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:53 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-y[130447]: 2023-07-14T18:08:53.096+0000 7f0d6546d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T18:08:53.224 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:53 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-y[130447]: 2023-07-14T18:08:53.187+0000 7f0d6546d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T18:08:53.224 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-14T18:08:53.225 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-14T18:08:53.225 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2023-07-14T18:08:53.225 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2023-07-14T18:08:53.225 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.047232 s, 222 MB/s 2023-07-14T18:08:53.225 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2023-07-14T18:08:53.261 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:52 smithi133 ceph-mon[126384]: Deploying daemon mgr.y on smithi161 2023-07-14T18:08:53.261 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:52 smithi133 ceph-mon[126384]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:53.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:52 smithi171 ceph-mon[129241]: Deploying daemon mgr.y on smithi161 2023-07-14T18:08:53.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:52 smithi171 ceph-mon[129241]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:53.358 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:52 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-z[131352]: 2023-07-14T18:08:52.940+0000 7fc6daafc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T18:08:53.358 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:53 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-z[131352]: 2023-07-14T18:08:53.111+0000 7fc6daafc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T18:08:53.501 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:53 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-y[130447]: 2023-07-14T18:08:53.285+0000 7f0d6546d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T18:08:53.501 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:53 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-y[130447]: 2023-07-14T18:08:53.464+0000 7f0d6546d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T18:08:53.742 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi133:vg_nvme/lv_4 2023-07-14T18:08:54.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:54 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:54.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:54 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:54.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:54 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:54.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:54 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:54.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:54 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:08:54.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:54 smithi161 ceph-mon[128993]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:54.093 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:53 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-y[130447]: 2023-07-14T18:08:53.790+0000 7f0d6546d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T18:08:54.093 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:53 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-y[130447]: 2023-07-14T18:08:53.893+0000 7f0d6546d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T18:08:54.093 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:54 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-y[130447]: 2023-07-14T18:08:54.035+0000 7f0d6546d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T18:08:54.222 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:53 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-z[131352]: 2023-07-14T18:08:53.913+0000 7fc6daafc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T18:08:54.223 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:54 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-z[131352]: 2023-07-14T18:08:54.006+0000 7fc6daafc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T18:08:54.223 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:54 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:54.223 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:54 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:54.223 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:54 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:54.223 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:54 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:54.224 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:54 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:08:54.224 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:54 smithi171 ceph-mon[129241]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:54.398 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:54 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:54.398 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:54 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:54.398 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:54 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:54.398 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:54 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:54.398 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:54 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:08:54.398 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:54 smithi133 ceph-mon[126384]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:54.466 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:08:54.516 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:54 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-z[131352]: 2023-07-14T18:08:54.185+0000 7fc6daafc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T18:08:54.592 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:54 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-y[130447]: 2023-07-14T18:08:54.133+0000 7f0d6546d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T18:08:54.857 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:54 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-z[131352]: 2023-07-14T18:08:54.479+0000 7fc6daafc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T18:08:54.857 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:54 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-z[131352]: 2023-07-14T18:08:54.718+0000 7fc6daafc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T18:08:55.092 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:54 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-y[130447]: 2023-07-14T18:08:54.593+0000 7f0d6546d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T18:08:55.168 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:54 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-z[131352]: 2023-07-14T18:08:54.868+0000 7fc6daafc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T18:08:55.592 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:55 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-y[130447]: 2023-07-14T18:08:55.177+0000 7f0d6546d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T18:08:55.593 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:55 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-y[130447]: 2023-07-14T18:08:55.346+0000 7f0d6546d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T18:08:55.607 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:55 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-z[131352]: 2023-07-14T18:08:55.131+0000 7fc6daafc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T18:08:55.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.952+0000 7fbb52f3c700 1 -- 172.21.15.133:0/2990894201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb4c101850 msgr2=0x7fbb4c108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:55.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.952+0000 7fbb52f3c700 1 --2- 172.21.15.133:0/2990894201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb4c101850 0x7fbb4c108110 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400b0a0 tx=0x7fbb4400b3b0 comp rx=0 tx=0).stop 2023-07-14T18:08:55.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.952+0000 7fbb52f3c700 1 -- 172.21.15.133:0/2990894201 shutdown_connections 2023-07-14T18:08:55.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.952+0000 7fbb52f3c700 1 --2- 172.21.15.133:0/2990894201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb4c0ffcc0 0x7fbb4c1000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:55.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.952+0000 7fbb52f3c700 1 --2- 172.21.15.133:0/2990894201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb4c101850 0x7fbb4c108110 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:55.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.952+0000 7fbb52f3c700 1 --2- 172.21.15.133:0/2990894201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb4c100ec0 0x7fbb4c101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:55.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.952+0000 7fbb52f3c700 1 -- 172.21.15.133:0/2990894201 >> 172.21.15.133:0/2990894201 conn(0x7fbb4c0fb2d0 msgr2=0x7fbb4c0fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:08:55.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.952+0000 7fbb52f3c700 1 -- 172.21.15.133:0/2990894201 shutdown_connections 2023-07-14T18:08:55.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.952+0000 7fbb52f3c700 1 -- 172.21.15.133:0/2990894201 wait complete. 2023-07-14T18:08:55.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.953+0000 7fbb52f3c700 1 Processor -- start 2023-07-14T18:08:55.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.953+0000 7fbb52f3c700 1 -- start start 2023-07-14T18:08:55.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.953+0000 7fbb52f3c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb4c0ffcc0 0x7fbb4c197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:55.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.953+0000 7fbb52f3c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb4c100ec0 0x7fbb4c198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:55.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.953+0000 7fbb52f3c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb4c101850 0x7fbb4c19f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:55.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.954+0000 7fbb52f3c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c19f8e0 con 0x7fbb4c0ffcc0 2023-07-14T18:08:55.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.954+0000 7fbb52f3c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c19fa20 con 0x7fbb4c101850 2023-07-14T18:08:55.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.954+0000 7fbb52f3c700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c19fbb0 con 0x7fbb4c100ec0 2023-07-14T18:08:55.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.954+0000 7fbb4bfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb4c100ec0 0x7fbb4c198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:55.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.954+0000 7fbb4bfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb4c100ec0 0x7fbb4c198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:51754/0 (socket says 172.21.15.133:51754) 2023-07-14T18:08:55.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.954+0000 7fbb4bfff700 1 -- 172.21.15.133:0/2707000769 learned_addr learned my addr 172.21.15.133:0/2707000769 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:08:55.994 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.954+0000 7fbb50cd8700 1 --2- 172.21.15.133:0/2707000769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb4c0ffcc0 0x7fbb4c197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:55.994 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.954+0000 7fbb514d9700 1 --2- 172.21.15.133:0/2707000769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb4c101850 0x7fbb4c19f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:55.994 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.954+0000 7fbb4bfff700 1 -- 172.21.15.133:0/2707000769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb4c101850 msgr2=0x7fbb4c19f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:55.994 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.954+0000 7fbb4bfff700 1 --2- 172.21.15.133:0/2707000769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb4c101850 0x7fbb4c19f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:55.994 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.954+0000 7fbb4bfff700 1 -- 172.21.15.133:0/2707000769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb4c0ffcc0 msgr2=0x7fbb4c197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:08:55.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.954+0000 7fbb4bfff700 1 --2- 172.21.15.133:0/2707000769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb4c0ffcc0 0x7fbb4c197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:08:55.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.954+0000 7fbb4bfff700 1 -- 172.21.15.133:0/2707000769 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb44009d00 con 0x7fbb4c100ec0 2023-07-14T18:08:55.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.954+0000 7fbb514d9700 1 --2- 172.21.15.133:0/2707000769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb4c101850 0x7fbb4c19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:08:55.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.954+0000 7fbb4bfff700 1 --2- 172.21.15.133:0/2707000769 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb4c100ec0 0x7fbb4c198290 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800e9d0 tx=0x7fbb3800ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:08:55.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.955+0000 7fbb49ffb700 1 -- 172.21.15.133:0/2707000769 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb38009790 con 0x7fbb4c100ec0 2023-07-14T18:08:55.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.955+0000 7fbb49ffb700 1 -- 172.21.15.133:0/2707000769 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb3800ee70 con 0x7fbb4c100ec0 2023-07-14T18:08:55.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.955+0000 7fbb52f3c700 1 -- 172.21.15.133:0/2707000769 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb4c19fe90 con 0x7fbb4c100ec0 2023-07-14T18:08:55.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.955+0000 7fbb49ffb700 1 -- 172.21.15.133:0/2707000769 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb38010ea0 con 0x7fbb4c100ec0 2023-07-14T18:08:55.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.955+0000 7fbb52f3c700 1 -- 172.21.15.133:0/2707000769 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb4c1a0350 con 0x7fbb4c100ec0 2023-07-14T18:08:55.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.956+0000 7fbb52f3c700 1 -- 172.21.15.133:0/2707000769 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb4c10b4e0 con 0x7fbb4c100ec0 2023-07-14T18:08:55.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.956+0000 7fbb49ffb700 1 -- 172.21.15.133:0/2707000769 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7fbb38010430 con 0x7fbb4c100ec0 2023-07-14T18:08:55.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.957+0000 7fbb49ffb700 1 --2- 172.21.15.133:0/2707000769 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fbb3c0389e0 0x7fbb3c03ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:08:55.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.957+0000 7fbb49ffb700 1 -- 172.21.15.133:0/2707000769 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbb38014070 con 0x7fbb4c100ec0 2023-07-14T18:08:55.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.957+0000 7fbb50cd8700 1 --2- 172.21.15.133:0/2707000769 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fbb3c0389e0 0x7fbb3c03ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:08:56.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.958+0000 7fbb50cd8700 1 --2- 172.21.15.133:0/2707000769 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fbb3c0389e0 0x7fbb3c03ae90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c100d20 tx=0x7fbb40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:08:56.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:55.962+0000 7fbb49ffb700 1 -- 172.21.15.133:0/2707000769 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fbb3801f590 con 0x7fbb4c100ec0 2023-07-14T18:08:56.001 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:55 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-z[131352]: 2023-07-14T18:08:55.831+0000 7fc6daafc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T18:08:56.001 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:55 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:56.001 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:55 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:56.001 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:55 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:56.002 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:55 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:56.002 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:55 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:56.002 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:55 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:08:56.003 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:55 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:56.003 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:55 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T18:08:56.003 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:55 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T18:08:56.003 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:55 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:56.020 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:55 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:56.020 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:55 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:56.020 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:55 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:56.020 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:55 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:56.020 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:55 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:56.021 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:55 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:08:56.021 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:55 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:56.021 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:55 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T18:08:56.021 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:55 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T18:08:56.021 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:55 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:56.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:55 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:56.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:55 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:56.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:55 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:56.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:55 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:56.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:55 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:56.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:55 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:08:56.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:55 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:56.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:55 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T18:08:56.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:55 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T18:08:56.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:55 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:56.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:56.188+0000 7fbb52f3c700 1 -- 172.21.15.133:0/2707000769 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb4c107e90 con 0x7fbb3c0389e0 2023-07-14T18:08:56.302 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:55 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-z[131352]: 2023-07-14T18:08:55.963+0000 7fc6daafc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T18:08:56.302 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:56 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-z[131352]: 2023-07-14T18:08:56.153+0000 7fc6daafc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T18:08:56.302 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:56 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-z[131352]: 2023-07-14T18:08:56.265+0000 7fc6daafc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T18:08:56.401 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:56 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-y[130447]: 2023-07-14T18:08:56.106+0000 7f0d6546d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T18:08:56.401 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:56 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-y[130447]: 2023-07-14T18:08:56.196+0000 7f0d6546d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T18:08:56.607 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:56 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-z[131352]: 2023-07-14T18:08:56.482+0000 7fc6daafc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T18:08:57.050 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:56 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-y[130447]: 2023-07-14T18:08:56.364+0000 7f0d6546d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T18:08:57.051 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:56 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-y[130447]: 2023-07-14T18:08:56.637+0000 7f0d6546d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T18:08:57.051 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:56 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-y[130447]: 2023-07-14T18:08:56.867+0000 7f0d6546d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T18:08:57.051 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:56 smithi161 ceph-mon[128993]: Reconfiguring mgr.x (unknown last config time)... 2023-07-14T18:08:57.051 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:56 smithi161 ceph-mon[128993]: Reconfiguring daemon mgr.x on smithi133 2023-07-14T18:08:57.051 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:56 smithi161 ceph-mon[128993]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:57.051 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:56 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:08:57.052 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:56 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:08:57.052 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:56 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:57.052 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:56 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:57.052 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:56 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:57.052 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:56 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:08:57.052 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:56 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:57.052 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:56 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:08:57.053 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:56 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:57.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:56 smithi171 ceph-mon[129241]: Reconfiguring mgr.x (unknown last config time)... 2023-07-14T18:08:57.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:56 smithi171 ceph-mon[129241]: Reconfiguring daemon mgr.x on smithi133 2023-07-14T18:08:57.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:56 smithi171 ceph-mon[129241]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:57.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:56 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:08:57.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:56 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:08:57.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:56 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:57.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:56 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:57.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:56 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:57.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:56 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:08:57.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:56 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:57.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:56 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:08:57.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:56 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:57.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:56 smithi133 ceph-mon[126384]: Reconfiguring mgr.x (unknown last config time)... 2023-07-14T18:08:57.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:56 smithi133 ceph-mon[126384]: Reconfiguring daemon mgr.x on smithi133 2023-07-14T18:08:57.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:56 smithi133 ceph-mon[126384]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:57.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:56 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:08:57.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:56 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:08:57.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:56 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:57.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:56 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:57.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:56 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:57.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:56 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:08:57.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:56 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:08:57.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:56 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:08:57.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:56 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:57.592 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:57 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-y[130447]: 2023-07-14T18:08:57.014+0000 7f0d6546d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T18:08:57.592 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:57 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-y[130447]: 2023-07-14T18:08:57.273+0000 7f0d6546d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T18:08:57.822 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:08:57 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-z[131352]: 2023-07-14T18:08:57.346+0000 7fc6daafc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T18:08:57.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:08:57.789+0000 7fbb49ffb700 1 -- 172.21.15.133:0/2707000769 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mgrmap(e 14) v1 ==== 90600+0+0 (secure 0 0 0) 0x7fbb380186d0 con 0x7fbb4c100ec0 2023-07-14T18:08:58.092 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:57 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-y[130447]: 2023-07-14T18:08:57.972+0000 7f0d6546d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T18:08:58.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:57 smithi161 ceph-mon[128993]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:08:58.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:57 smithi161 ceph-mon[128993]: Standby manager daemon z started 2023-07-14T18:08:58.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:57 smithi161 ceph-mon[128993]: from='mgr.? 172.21.15.171:0/1884974042' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-14T18:08:58.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:57 smithi161 ceph-mon[128993]: from='mgr.? 172.21.15.171:0/1884974042' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T18:08:58.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:57 smithi161 ceph-mon[128993]: from='mgr.? 172.21.15.171:0/1884974042' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-14T18:08:58.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:57 smithi161 ceph-mon[128993]: from='mgr.? 172.21.15.171:0/1884974042' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T18:08:58.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:57 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:58.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:57 smithi171 ceph-mon[129241]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:08:58.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:57 smithi171 ceph-mon[129241]: Standby manager daemon z started 2023-07-14T18:08:58.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:57 smithi171 ceph-mon[129241]: from='mgr.? 172.21.15.171:0/1884974042' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-14T18:08:58.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:57 smithi171 ceph-mon[129241]: from='mgr.? 172.21.15.171:0/1884974042' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T18:08:58.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:57 smithi171 ceph-mon[129241]: from='mgr.? 172.21.15.171:0/1884974042' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-14T18:08:58.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:57 smithi171 ceph-mon[129241]: from='mgr.? 172.21.15.171:0/1884974042' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T18:08:58.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:57 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:58.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:57 smithi133 ceph-mon[126384]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:08:58.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:57 smithi133 ceph-mon[126384]: Standby manager daemon z started 2023-07-14T18:08:58.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:57 smithi133 ceph-mon[126384]: from='mgr.? 172.21.15.171:0/1884974042' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-14T18:08:58.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:57 smithi133 ceph-mon[126384]: from='mgr.? 172.21.15.171:0/1884974042' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T18:08:58.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:57 smithi133 ceph-mon[126384]: from='mgr.? 172.21.15.171:0/1884974042' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-14T18:08:58.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:57 smithi133 ceph-mon[126384]: from='mgr.? 172.21.15.171:0/1884974042' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T18:08:58.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:57 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:08:58.435 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:58 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-y[130447]: 2023-07-14T18:08:58.101+0000 7f0d6546d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T18:08:58.436 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:58 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-y[130447]: 2023-07-14T18:08:58.288+0000 7f0d6546d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T18:08:58.823 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:58 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-y[130447]: 2023-07-14T18:08:58.399+0000 7f0d6546d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T18:08:58.824 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:58 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-y[130447]: 2023-07-14T18:08:58.612+0000 7f0d6546d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T18:08:59.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:58 smithi161 ceph-mon[128993]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:59.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:58 smithi161 ceph-mon[128993]: mgrmap e14: x(active, since 70s), standbys: z 2023-07-14T18:08:59.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:58 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-14T18:08:59.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:58 smithi171 ceph-mon[129241]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:59.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:58 smithi171 ceph-mon[129241]: mgrmap e14: x(active, since 70s), standbys: z 2023-07-14T18:08:59.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:58 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-14T18:08:59.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:58 smithi133 ceph-mon[126384]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:08:59.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:58 smithi133 ceph-mon[126384]: mgrmap e14: x(active, since 70s), standbys: z 2023-07-14T18:08:59.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:58 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-14T18:08:59.824 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:08:59 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-y[130447]: 2023-07-14T18:08:59.463+0000 7f0d6546d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T18:09:00.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:59 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/573325178' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85a29de7-d712-422a-b647-d9e90091aaf1"}]: dispatch 2023-07-14T18:09:00.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:59 smithi161 ceph-mon[128993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85a29de7-d712-422a-b647-d9e90091aaf1"}]: dispatch 2023-07-14T18:09:00.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:59 smithi161 ceph-mon[128993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85a29de7-d712-422a-b647-d9e90091aaf1"}]': finished 2023-07-14T18:09:00.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:59 smithi161 ceph-mon[128993]: osdmap e5: 1 total, 0 up, 1 in 2023-07-14T18:09:00.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:59 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T18:09:00.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:59 smithi161 ceph-mon[128993]: Standby manager daemon y started 2023-07-14T18:09:00.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:59 smithi161 ceph-mon[128993]: from='mgr.? 172.21.15.161:0/1739490477' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-14T18:09:00.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:59 smithi161 ceph-mon[128993]: from='mgr.? 172.21.15.161:0/1739490477' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T18:09:00.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:59 smithi161 ceph-mon[128993]: from='mgr.? 172.21.15.161:0/1739490477' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-14T18:09:00.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:08:59 smithi161 ceph-mon[128993]: from='mgr.? 172.21.15.161:0/1739490477' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T18:09:00.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:59 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/573325178' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85a29de7-d712-422a-b647-d9e90091aaf1"}]: dispatch 2023-07-14T18:09:00.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:59 smithi171 ceph-mon[129241]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85a29de7-d712-422a-b647-d9e90091aaf1"}]: dispatch 2023-07-14T18:09:00.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:59 smithi171 ceph-mon[129241]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85a29de7-d712-422a-b647-d9e90091aaf1"}]': finished 2023-07-14T18:09:00.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:59 smithi171 ceph-mon[129241]: osdmap e5: 1 total, 0 up, 1 in 2023-07-14T18:09:00.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:59 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T18:09:00.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:59 smithi171 ceph-mon[129241]: Standby manager daemon y started 2023-07-14T18:09:00.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:59 smithi171 ceph-mon[129241]: from='mgr.? 172.21.15.161:0/1739490477' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-14T18:09:00.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:59 smithi171 ceph-mon[129241]: from='mgr.? 172.21.15.161:0/1739490477' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T18:09:00.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:59 smithi171 ceph-mon[129241]: from='mgr.? 172.21.15.161:0/1739490477' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-14T18:09:00.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:08:59 smithi171 ceph-mon[129241]: from='mgr.? 172.21.15.161:0/1739490477' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T18:09:00.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:59 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/573325178' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85a29de7-d712-422a-b647-d9e90091aaf1"}]: dispatch 2023-07-14T18:09:00.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:59 smithi133 ceph-mon[126384]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85a29de7-d712-422a-b647-d9e90091aaf1"}]: dispatch 2023-07-14T18:09:00.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:59 smithi133 ceph-mon[126384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85a29de7-d712-422a-b647-d9e90091aaf1"}]': finished 2023-07-14T18:09:00.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:59 smithi133 ceph-mon[126384]: osdmap e5: 1 total, 0 up, 1 in 2023-07-14T18:09:00.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:59 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T18:09:00.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:59 smithi133 ceph-mon[126384]: Standby manager daemon y started 2023-07-14T18:09:00.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:59 smithi133 ceph-mon[126384]: from='mgr.? 172.21.15.161:0/1739490477' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-14T18:09:00.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:59 smithi133 ceph-mon[126384]: from='mgr.? 172.21.15.161:0/1739490477' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T18:09:00.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:59 smithi133 ceph-mon[126384]: from='mgr.? 172.21.15.161:0/1739490477' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-14T18:09:00.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:08:59 smithi133 ceph-mon[126384]: from='mgr.? 172.21.15.161:0/1739490477' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T18:09:00.188 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:09:00.152+0000 7fbb49ffb700 1 -- 172.21.15.133:0/2707000769 <== mon.1 v2:172.21.15.171:3300/0 8 ==== mgrmap(e 15) v1 ==== 135660+0+0 (secure 0 0 0) 0x7fbb380106e0 con 0x7fbb4c100ec0 2023-07-14T18:09:01.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:00 smithi161 ceph-mon[128993]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:01.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:00 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2984814760' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:09:01.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:00 smithi161 ceph-mon[128993]: mgrmap e15: x(active, since 72s), standbys: z, y 2023-07-14T18:09:01.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:00 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-14T18:09:01.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:00 smithi171 ceph-mon[129241]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:01.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:00 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2984814760' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:09:01.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:00 smithi171 ceph-mon[129241]: mgrmap e15: x(active, since 72s), standbys: z, y 2023-07-14T18:09:01.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:00 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-14T18:09:01.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:00 smithi133 ceph-mon[126384]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:01.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:00 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2984814760' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:09:01.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:00 smithi133 ceph-mon[126384]: mgrmap e15: x(active, since 72s), standbys: z, y 2023-07-14T18:09:01.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:00 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-14T18:09:03.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:02 smithi161 ceph-mon[128993]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:03.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:02 smithi171 ceph-mon[129241]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:03.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:02 smithi133 ceph-mon[126384]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:04.953 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:04 smithi133 ceph-mon[126384]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:05.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:04 smithi161 ceph-mon[128993]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:05.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:04 smithi171 ceph-mon[129241]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:06.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:06 smithi133 ceph-mon[126384]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:06.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:06 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-14T18:09:06.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:06 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:07.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:06 smithi161 ceph-mon[128993]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:07.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:06 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-14T18:09:07.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:06 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:07.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:06 smithi171 ceph-mon[129241]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:07.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:06 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-14T18:09:07.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:06 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:08.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:07 smithi161 ceph-mon[128993]: Deploying daemon osd.0 on smithi133 2023-07-14T18:09:08.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:07 smithi171 ceph-mon[129241]: Deploying daemon osd.0 on smithi133 2023-07-14T18:09:08.108 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:07 smithi133 ceph-mon[126384]: Deploying daemon osd.0 on smithi133 2023-07-14T18:09:09.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:08 smithi171 ceph-mon[129241]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:09.131 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:08 smithi133 ceph-mon[126384]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:09.343 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:08 smithi161 ceph-mon[128993]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:11.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:10 smithi171 ceph-mon[129241]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:11.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:10 smithi133 ceph-mon[126384]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:11.342 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:10 smithi161 ceph-mon[128993]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:13.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:12 smithi161 ceph-mon[128993]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:13.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:12 smithi133 ceph-mon[126384]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:13.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:12 smithi171 ceph-mon[129241]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:14.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:13 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:14.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:13 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:14.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:13 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:09:14.342 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:13 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:14.343 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:13 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:14.343 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:13 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:09:14.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:13 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:14.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:13 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:14.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:13 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:09:14.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:14 smithi133 ceph-mon[126384]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:15.342 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:14 smithi161 ceph-mon[128993]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:15.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:14 smithi171 ceph-mon[129241]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:16.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:16 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:16.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:16 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:16.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:16 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:16.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:16 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:09:16.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:16 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:16.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:16 smithi171 ceph-mon[129241]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:16.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:16 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:16.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:16 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:16.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:16 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:16.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:16 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:09:16.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:16 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:16.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:16 smithi133 ceph-mon[126384]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:16.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:09:16.754+0000 7fbb49ffb700 1 -- 172.21.15.133:0/2707000769 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbb4c107e90 con 0x7fbb3c0389e0 2023-07-14T18:09:16.791 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 0 on host 'smithi133' 2023-07-14T18:09:16.793 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:09:16.756+0000 7fbb52f3c700 1 -- 172.21.15.133:0/2707000769 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fbb3c0389e0 msgr2=0x7fbb3c03ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:09:16.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:09:16.756+0000 7fbb52f3c700 1 --2- 172.21.15.133:0/2707000769 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fbb3c0389e0 0x7fbb3c03ae90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c100d20 tx=0x7fbb40009260 comp rx=0 tx=0).stop 2023-07-14T18:09:16.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:09:16.756+0000 7fbb52f3c700 1 -- 172.21.15.133:0/2707000769 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb4c100ec0 msgr2=0x7fbb4c198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:09:16.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:09:16.756+0000 7fbb52f3c700 1 --2- 172.21.15.133:0/2707000769 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb4c100ec0 0x7fbb4c198290 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800e9d0 tx=0x7fbb3800ece0 comp rx=0 tx=0).stop 2023-07-14T18:09:16.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:09:16.757+0000 7fbb52f3c700 1 -- 172.21.15.133:0/2707000769 shutdown_connections 2023-07-14T18:09:16.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:09:16.757+0000 7fbb52f3c700 1 --2- 172.21.15.133:0/2707000769 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fbb3c0389e0 0x7fbb3c03ae90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:09:16.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:09:16.757+0000 7fbb52f3c700 1 --2- 172.21.15.133:0/2707000769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb4c101850 0x7fbb4c19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:09:16.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:09:16.757+0000 7fbb52f3c700 1 --2- 172.21.15.133:0/2707000769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb4c0ffcc0 0x7fbb4c197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:09:16.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:09:16.757+0000 7fbb52f3c700 1 --2- 172.21.15.133:0/2707000769 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb4c100ec0 0x7fbb4c198290 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:09:16.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:09:16.757+0000 7fbb52f3c700 1 -- 172.21.15.133:0/2707000769 >> 172.21.15.133:0/2707000769 conn(0x7fbb4c0fb2d0 msgr2=0x7fbb4c106770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:09:16.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:09:16.757+0000 7fbb52f3c700 1 -- 172.21.15.133:0/2707000769 shutdown_connections 2023-07-14T18:09:16.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:09:16.757+0000 7fbb52f3c700 1 -- 172.21.15.133:0/2707000769 wait complete. 2023-07-14T18:09:16.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:16 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:16.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:16 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:16.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:16 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:16.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:16 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:09:16.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:16 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:16.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:16 smithi161 ceph-mon[128993]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:17.196 DEBUG:teuthology.orchestra.run.smithi133:osd.0> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.0.service 2023-07-14T18:09:17.200 INFO:tasks.cephadm:Deploying osd.1 on smithi161 with /dev/vg_nvme/lv_4... 2023-07-14T18:09:17.201 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-14T18:09:17.270 INFO:journalctl@ceph.osd.0.smithi133.stdout:-- Logs begin at Fri 2023-07-14 17:46:17 UTC. -- 2023-07-14T18:09:17.762 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T18:09:17.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:17 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:17.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:17 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:17.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:17 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:09:17.993 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:17 smithi161 ceph-mon[128993]: from='osd.0 [v2:172.21.15.133:6802/3076510801,v1:172.21.15.133:6803/3076510801]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-14T18:09:18.053 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:17 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:18.054 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:17 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:18.054 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:17 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:09:18.055 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:17 smithi133 ceph-mon[126384]: from='osd.0 [v2:172.21.15.133:6802/3076510801,v1:172.21.15.133:6803/3076510801]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-14T18:09:18.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:17 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:18.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:17 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:18.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:17 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:09:18.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:17 smithi171 ceph-mon[129241]: from='osd.0 [v2:172.21.15.133:6802/3076510801,v1:172.21.15.133:6803/3076510801]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-14T18:09:18.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:18 smithi161 ceph-mon[128993]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:18.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:18 smithi161 ceph-mon[128993]: from='osd.0 [v2:172.21.15.133:6802/3076510801,v1:172.21.15.133:6803/3076510801]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-14T18:09:18.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:18 smithi161 ceph-mon[128993]: osdmap e6: 1 total, 0 up, 1 in 2023-07-14T18:09:18.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:18 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T18:09:18.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:18 smithi161 ceph-mon[128993]: from='osd.0 [v2:172.21.15.133:6802/3076510801,v1:172.21.15.133:6803/3076510801]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T18:09:18.885 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:18 smithi133 ceph-mon[126384]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:18.885 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:18 smithi133 ceph-mon[126384]: from='osd.0 [v2:172.21.15.133:6802/3076510801,v1:172.21.15.133:6803/3076510801]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-14T18:09:18.885 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:18 smithi133 ceph-mon[126384]: osdmap e6: 1 total, 0 up, 1 in 2023-07-14T18:09:18.886 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:18 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T18:09:18.886 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:18 smithi133 ceph-mon[126384]: from='osd.0 [v2:172.21.15.133:6802/3076510801,v1:172.21.15.133:6803/3076510801]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T18:09:18.886 INFO:journalctl@ceph.osd.0.smithi133.stdout:Jul 14 18:09:18 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-0[139322]: 2023-07-14T18:09:18.725+0000 7f652fe12700 -1 osd.0 0 waiting for initial osdmap 2023-07-14T18:09:18.886 INFO:journalctl@ceph.osd.0.smithi133.stdout:Jul 14 18:09:18 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-0[139322]: 2023-07-14T18:09:18.732+0000 7f6529403700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T18:09:19.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:18 smithi171 ceph-mon[129241]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:19.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:18 smithi171 ceph-mon[129241]: from='osd.0 [v2:172.21.15.133:6802/3076510801,v1:172.21.15.133:6803/3076510801]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-14T18:09:19.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:18 smithi171 ceph-mon[129241]: osdmap e6: 1 total, 0 up, 1 in 2023-07-14T18:09:19.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:18 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T18:09:19.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:18 smithi171 ceph-mon[129241]: from='osd.0 [v2:172.21.15.133:6802/3076510801,v1:172.21.15.133:6803/3076510801]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T18:09:19.373 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-14T18:09:19.373 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-14T18:09:19.373 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2023-07-14T18:09:19.374 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2023-07-14T18:09:19.374 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110401 s, 950 MB/s 2023-07-14T18:09:19.374 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2023-07-14T18:09:19.719 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi161:vg_nvme/lv_4 2023-07-14T18:09:19.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:19 smithi133 ceph-mon[126384]: from='osd.0 [v2:172.21.15.133:6802/3076510801,v1:172.21.15.133:6803/3076510801]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-07-14T18:09:19.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:19 smithi133 ceph-mon[126384]: osdmap e7: 1 total, 0 up, 1 in 2023-07-14T18:09:19.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:19 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T18:09:19.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:19 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T18:09:20.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:19 smithi161 ceph-mon[128993]: from='osd.0 [v2:172.21.15.133:6802/3076510801,v1:172.21.15.133:6803/3076510801]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-07-14T18:09:20.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:19 smithi161 ceph-mon[128993]: osdmap e7: 1 total, 0 up, 1 in 2023-07-14T18:09:20.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:19 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T18:09:20.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:19 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T18:09:20.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:19 smithi171 ceph-mon[129241]: from='osd.0 [v2:172.21.15.133:6802/3076510801,v1:172.21.15.133:6803/3076510801]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-07-14T18:09:20.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:19 smithi171 ceph-mon[129241]: osdmap e7: 1 total, 0 up, 1 in 2023-07-14T18:09:20.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:19 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T18:09:20.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:19 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T18:09:20.274 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T18:09:21.033 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:20 smithi161 ceph-mon[128993]: purged_snaps scrub starts 2023-07-14T18:09:21.033 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:20 smithi161 ceph-mon[128993]: purged_snaps scrub ok 2023-07-14T18:09:21.034 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:20 smithi161 ceph-mon[128993]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:21.034 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:20 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T18:09:21.034 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:20 smithi161 ceph-mon[128993]: osd.0 [v2:172.21.15.133:6802/3076510801,v1:172.21.15.133:6803/3076510801] boot 2023-07-14T18:09:21.034 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:20 smithi161 ceph-mon[128993]: osdmap e8: 1 total, 1 up, 1 in 2023-07-14T18:09:21.034 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:20 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T18:09:21.034 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:20 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:21.034 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:20 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:21.035 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:20 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:09:21.035 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:20 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:21.035 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:20 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:21.035 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:20 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:09:21.035 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:20 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:21.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:20 smithi171 ceph-mon[129241]: purged_snaps scrub starts 2023-07-14T18:09:21.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:20 smithi171 ceph-mon[129241]: purged_snaps scrub ok 2023-07-14T18:09:21.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:20 smithi171 ceph-mon[129241]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:21.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:20 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T18:09:21.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:20 smithi171 ceph-mon[129241]: osd.0 [v2:172.21.15.133:6802/3076510801,v1:172.21.15.133:6803/3076510801] boot 2023-07-14T18:09:21.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:20 smithi171 ceph-mon[129241]: osdmap e8: 1 total, 1 up, 1 in 2023-07-14T18:09:21.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:20 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T18:09:21.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:20 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:21.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:20 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:21.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:20 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:09:21.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:20 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:21.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:20 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:21.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:20 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:09:21.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:20 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:21.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:20 smithi133 ceph-mon[126384]: purged_snaps scrub starts 2023-07-14T18:09:21.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:20 smithi133 ceph-mon[126384]: purged_snaps scrub ok 2023-07-14T18:09:21.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:20 smithi133 ceph-mon[126384]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T18:09:21.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:20 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T18:09:21.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:20 smithi133 ceph-mon[126384]: osd.0 [v2:172.21.15.133:6802/3076510801,v1:172.21.15.133:6803/3076510801] boot 2023-07-14T18:09:21.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:20 smithi133 ceph-mon[126384]: osdmap e8: 1 total, 1 up, 1 in 2023-07-14T18:09:21.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:20 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T18:09:21.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:20 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:21.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:20 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:21.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:20 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:09:21.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:20 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:21.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:20 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:21.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:20 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:09:21.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:20 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:21.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.424+0000 7f035d648700 1 -- 172.21.15.161:0/2396110179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03580fee60 msgr2=0x7f03580ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:09:21.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.424+0000 7f035d648700 1 --2- 172.21.15.161:0/2396110179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03580fee60 0x7f03580ff2d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f034c009990 tx=0x7f034c009ca0 comp rx=0 tx=0).stop 2023-07-14T18:09:21.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.424+0000 7f035d648700 1 -- 172.21.15.161:0/2396110179 shutdown_connections 2023-07-14T18:09:21.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.424+0000 7f035d648700 1 --2- 172.21.15.161:0/2396110179 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03580ff810 0x7f03581070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:09:21.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.424+0000 7f035d648700 1 --2- 172.21.15.161:0/2396110179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0358100290 0x7f03581006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:09:21.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.424+0000 7f035d648700 1 --2- 172.21.15.161:0/2396110179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03580fee60 0x7f03580ff2d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:09:21.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.424+0000 7f035d648700 1 -- 172.21.15.161:0/2396110179 >> 172.21.15.161:0/2396110179 conn(0x7f03580f99c0 msgr2=0x7f03580fbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:09:21.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.424+0000 7f035d648700 1 -- 172.21.15.161:0/2396110179 shutdown_connections 2023-07-14T18:09:21.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.424+0000 7f035d648700 1 -- 172.21.15.161:0/2396110179 wait complete. 2023-07-14T18:09:21.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.424+0000 7f035d648700 1 Processor -- start 2023-07-14T18:09:21.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.424+0000 7f035d648700 1 -- start start 2023-07-14T18:09:21.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.425+0000 7f035d648700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03580fee60 0x7f0358197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:09:21.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.425+0000 7f035d648700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03580ff810 0x7f0358198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:09:21.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.425+0000 7f035d648700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0358100290 0x7f035819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:09:21.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.425+0000 7f035d648700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035819f680 con 0x7f03580ff810 2023-07-14T18:09:21.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.425+0000 7f035d648700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035819f7c0 con 0x7f0358100290 2023-07-14T18:09:21.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.425+0000 7f035d648700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035819f950 con 0x7f03580fee60 2023-07-14T18:09:21.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.425+0000 7f03577fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0358100290 0x7f035819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:09:21.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.425+0000 7f03577fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0358100290 0x7f035819f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54656/0 (socket says 172.21.15.161:54656) 2023-07-14T18:09:21.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.425+0000 7f03577fe700 1 -- 172.21.15.161:0/3331343662 learned_addr learned my addr 172.21.15.161:0/3331343662 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T18:09:21.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.425+0000 7f03567fc700 1 --2- 172.21.15.161:0/3331343662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03580ff810 0x7f0358198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:09:21.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.425+0000 7f0356ffd700 1 --2- 172.21.15.161:0/3331343662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03580fee60 0x7f0358197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:09:21.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.425+0000 7f03577fe700 1 -- 172.21.15.161:0/3331343662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03580fee60 msgr2=0x7f0358197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:09:21.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.425+0000 7f03577fe700 1 --2- 172.21.15.161:0/3331343662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03580fee60 0x7f0358197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:09:21.465 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.425+0000 7f03577fe700 1 -- 172.21.15.161:0/3331343662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03580ff810 msgr2=0x7f0358198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:09:21.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.425+0000 7f03577fe700 1 --2- 172.21.15.161:0/3331343662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03580ff810 0x7f0358198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:09:21.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.425+0000 7f03577fe700 1 -- 172.21.15.161:0/3331343662 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034c0095f0 con 0x7f0358100290 2023-07-14T18:09:21.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.425+0000 7f03577fe700 1 --2- 172.21.15.161:0/3331343662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0358100290 0x7f035819f0b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f035000ada0 tx=0x7f035000c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:09:21.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.425+0000 7f033ffff700 1 -- 172.21.15.161:0/3331343662 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035000f070 con 0x7f0358100290 2023-07-14T18:09:21.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.425+0000 7f035d648700 1 -- 172.21.15.161:0/3331343662 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f035819fc30 con 0x7f0358100290 2023-07-14T18:09:21.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.425+0000 7f033ffff700 1 -- 172.21.15.161:0/3331343662 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0350007b30 con 0x7f0358100290 2023-07-14T18:09:21.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.425+0000 7f033ffff700 1 -- 172.21.15.161:0/3331343662 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0350005280 con 0x7f0358100290 2023-07-14T18:09:21.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.425+0000 7f035d648700 1 -- 172.21.15.161:0/3331343662 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03581a0130 con 0x7f0358100290 2023-07-14T18:09:21.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.426+0000 7f035d648700 1 -- 172.21.15.161:0/3331343662 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0358106130 con 0x7f0358100290 2023-07-14T18:09:21.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.427+0000 7f033ffff700 1 -- 172.21.15.161:0/3331343662 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 15) v1 ==== 135660+0+0 (secure 0 0 0) 0x7f035000c8d0 con 0x7f0358100290 2023-07-14T18:09:21.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.428+0000 7f033ffff700 1 --2- 172.21.15.161:0/3331343662 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f03400a10a0 0x7f03400a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:09:21.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.428+0000 7f033ffff700 1 -- 172.21.15.161:0/3331343662 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f03500ca360 con 0x7f0358100290 2023-07-14T18:09:21.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.428+0000 7f0356ffd700 1 --2- 172.21.15.161:0/3331343662 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f03400a10a0 0x7f03400a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:09:21.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.429+0000 7f0356ffd700 1 --2- 172.21.15.161:0/3331343662 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f03400a10a0 0x7f03400a3550 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f034800ac50 tx=0x7f034800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:09:21.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.432+0000 7f033ffff700 1 -- 172.21.15.161:0/3331343662 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f03500955b0 con 0x7f0358100290 2023-07-14T18:09:21.662 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:21.625+0000 7f035d648700 1 -- 172.21.15.161:0/3331343662 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f0358060a80 con 0x7f03400a10a0 2023-07-14T18:09:22.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:21 smithi161 ceph-mon[128993]: Detected new or changed devices on smithi133 2023-07-14T18:09:22.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:21 smithi161 ceph-mon[128993]: Adjusting osd_memory_target on smithi133 to 17267M 2023-07-14T18:09:22.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:21 smithi161 ceph-mon[128993]: osdmap e9: 1 total, 1 up, 1 in 2023-07-14T18:09:22.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:21 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:09:22.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:21 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:09:22.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:21 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:22.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:21 smithi171 ceph-mon[129241]: Detected new or changed devices on smithi133 2023-07-14T18:09:22.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:21 smithi171 ceph-mon[129241]: Adjusting osd_memory_target on smithi133 to 17267M 2023-07-14T18:09:22.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:21 smithi171 ceph-mon[129241]: osdmap e9: 1 total, 1 up, 1 in 2023-07-14T18:09:22.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:21 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:09:22.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:21 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:09:22.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:21 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:22.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:21 smithi133 ceph-mon[126384]: Detected new or changed devices on smithi133 2023-07-14T18:09:22.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:21 smithi133 ceph-mon[126384]: Adjusting osd_memory_target on smithi133 to 17267M 2023-07-14T18:09:22.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:21 smithi133 ceph-mon[126384]: osdmap e9: 1 total, 1 up, 1 in 2023-07-14T18:09:22.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:21 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:09:22.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:21 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:09:22.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:21 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:23.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:22 smithi161 ceph-mon[128993]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:23.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:22 smithi161 ceph-mon[128993]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:09:23.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:22 smithi171 ceph-mon[129241]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:23.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:22 smithi171 ceph-mon[129241]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:09:23.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:22 smithi133 ceph-mon[126384]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:23.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:22 smithi133 ceph-mon[126384]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:09:25.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:24 smithi161 ceph-mon[128993]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:25.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:24 smithi161 ceph-mon[128993]: from='client.? 172.21.15.161:0/3482666634' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80ad4e78-7917-495e-b119-99f7c7c36243"}]: dispatch 2023-07-14T18:09:25.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:24 smithi161 ceph-mon[128993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80ad4e78-7917-495e-b119-99f7c7c36243"}]: dispatch 2023-07-14T18:09:25.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:24 smithi161 ceph-mon[128993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80ad4e78-7917-495e-b119-99f7c7c36243"}]': finished 2023-07-14T18:09:25.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:24 smithi161 ceph-mon[128993]: osdmap e10: 2 total, 1 up, 2 in 2023-07-14T18:09:25.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:24 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T18:09:25.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:24 smithi161 ceph-mon[128993]: from='client.? 172.21.15.161:0/3056705773' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:09:25.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:24 smithi171 ceph-mon[129241]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:25.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:24 smithi171 ceph-mon[129241]: from='client.? 172.21.15.161:0/3482666634' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80ad4e78-7917-495e-b119-99f7c7c36243"}]: dispatch 2023-07-14T18:09:25.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:24 smithi171 ceph-mon[129241]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80ad4e78-7917-495e-b119-99f7c7c36243"}]: dispatch 2023-07-14T18:09:25.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:24 smithi171 ceph-mon[129241]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80ad4e78-7917-495e-b119-99f7c7c36243"}]': finished 2023-07-14T18:09:25.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:24 smithi171 ceph-mon[129241]: osdmap e10: 2 total, 1 up, 2 in 2023-07-14T18:09:25.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:24 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T18:09:25.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:24 smithi171 ceph-mon[129241]: from='client.? 172.21.15.161:0/3056705773' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:09:25.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:24 smithi133 ceph-mon[126384]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:25.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:24 smithi133 ceph-mon[126384]: from='client.? 172.21.15.161:0/3482666634' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80ad4e78-7917-495e-b119-99f7c7c36243"}]: dispatch 2023-07-14T18:09:25.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:24 smithi133 ceph-mon[126384]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80ad4e78-7917-495e-b119-99f7c7c36243"}]: dispatch 2023-07-14T18:09:25.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:24 smithi133 ceph-mon[126384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80ad4e78-7917-495e-b119-99f7c7c36243"}]': finished 2023-07-14T18:09:25.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:24 smithi133 ceph-mon[126384]: osdmap e10: 2 total, 1 up, 2 in 2023-07-14T18:09:25.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:24 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T18:09:25.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:24 smithi133 ceph-mon[126384]: from='client.? 172.21.15.161:0/3056705773' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:09:27.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:26 smithi161 ceph-mon[128993]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:27.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:26 smithi171 ceph-mon[129241]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:27.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:26 smithi133 ceph-mon[126384]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:29.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:28 smithi161 ceph-mon[128993]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:29.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:28 smithi171 ceph-mon[129241]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:29.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:28 smithi133 ceph-mon[126384]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:31.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:30 smithi171 ceph-mon[129241]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:31.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:30 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-14T18:09:31.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:30 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:31.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:30 smithi133 ceph-mon[126384]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:31.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:30 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-14T18:09:31.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:30 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:31.151 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:30 smithi161 ceph-mon[128993]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:31.151 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:30 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-14T18:09:31.151 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:30 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:32.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:31 smithi161 ceph-mon[128993]: Deploying daemon osd.1 on smithi161 2023-07-14T18:09:32.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:31 smithi171 ceph-mon[129241]: Deploying daemon osd.1 on smithi161 2023-07-14T18:09:32.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:31 smithi133 ceph-mon[126384]: Deploying daemon osd.1 on smithi161 2023-07-14T18:09:33.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:32 smithi161 ceph-mon[128993]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:33.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:32 smithi133 ceph-mon[126384]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:33.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:32 smithi171 ceph-mon[129241]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:35.115 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:34 smithi161 ceph-mon[128993]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:35.115 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:34 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:35.115 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:34 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:09:35.116 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:34 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:35.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:34 smithi133 ceph-mon[126384]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:35.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:34 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:35.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:34 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:09:35.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:34 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:35.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:34 smithi171 ceph-mon[129241]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:35.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:34 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:35.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:34 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:09:35.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:34 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:37.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:36 smithi133 ceph-mon[126384]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:37.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:36 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:37.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:36 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:37.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:36 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:37.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:36 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:09:37.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:36 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:37.236 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:36 smithi161 ceph-mon[128993]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:37.237 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:37.237 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:37.237 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:37.237 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:09:37.237 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:37.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:37.240+0000 7f033ffff700 1 -- 172.21.15.161:0/3331343662 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0358060a80 con 0x7f03400a10a0 2023-07-14T18:09:37.278 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 1 on host 'smithi161' 2023-07-14T18:09:37.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:37.242+0000 7f035d648700 1 -- 172.21.15.161:0/3331343662 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f03400a10a0 msgr2=0x7f03400a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:09:37.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:37.242+0000 7f035d648700 1 --2- 172.21.15.161:0/3331343662 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f03400a10a0 0x7f03400a3550 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f034800ac50 tx=0x7f034800a380 comp rx=0 tx=0).stop 2023-07-14T18:09:37.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:37.242+0000 7f035d648700 1 -- 172.21.15.161:0/3331343662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0358100290 msgr2=0x7f035819f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:09:37.280 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:37.242+0000 7f035d648700 1 --2- 172.21.15.161:0/3331343662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0358100290 0x7f035819f0b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f035000ada0 tx=0x7f035000c330 comp rx=0 tx=0).stop 2023-07-14T18:09:37.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:37.242+0000 7f035d648700 1 -- 172.21.15.161:0/3331343662 shutdown_connections 2023-07-14T18:09:37.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:37.242+0000 7f035d648700 1 --2- 172.21.15.161:0/3331343662 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f03400a10a0 0x7f03400a3550 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:09:37.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:37.242+0000 7f035d648700 1 --2- 172.21.15.161:0/3331343662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0358100290 0x7f035819f0b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:09:37.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:37.242+0000 7f035d648700 1 --2- 172.21.15.161:0/3331343662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03580fee60 0x7f0358197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:09:37.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:37.242+0000 7f035d648700 1 --2- 172.21.15.161:0/3331343662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03580ff810 0x7f0358198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:09:37.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:37.242+0000 7f035d648700 1 -- 172.21.15.161:0/3331343662 >> 172.21.15.161:0/3331343662 conn(0x7f03580f99c0 msgr2=0x7f0358103270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:09:37.281 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:37.242+0000 7f035d648700 1 -- 172.21.15.161:0/3331343662 shutdown_connections 2023-07-14T18:09:37.282 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:09:37.242+0000 7f035d648700 1 -- 172.21.15.161:0/3331343662 wait complete. 2023-07-14T18:09:37.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:36 smithi171 ceph-mon[129241]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:37.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:36 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:37.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:36 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:37.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:36 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:37.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:36 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:09:37.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:36 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:37.759 DEBUG:teuthology.orchestra.run.smithi161:osd.1> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.1.service 2023-07-14T18:09:37.763 INFO:tasks.cephadm:Deploying osd.2 on smithi171 with /dev/vg_nvme/lv_4... 2023-07-14T18:09:37.764 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-14T18:09:37.791 INFO:journalctl@ceph.osd.1.smithi161.stdout:-- Logs begin at Fri 2023-07-14 17:41:43 UTC. -- 2023-07-14T18:09:38.326 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T18:09:38.542 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:38 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:38.542 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:38 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:38.542 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:38 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:09:38.543 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:38 smithi161 ceph-mon[128993]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:38.573 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:38 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:38.573 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:38 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:38.573 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:38 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:09:38.573 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:38 smithi171 ceph-mon[129241]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:38.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:38 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:38.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:38 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:38.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:38 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:09:38.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:38 smithi133 ceph-mon[126384]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:38.843 INFO:journalctl@ceph.osd.1.smithi161.stdout:Jul 14 18:09:38 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-1[134087]: 2023-07-14T18:09:38.620+0000 7fcd3e25c640 -1 osd.1 0 log_to_monitors true 2023-07-14T18:09:39.343 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:39 smithi161 ceph-mon[128993]: from='osd.1 [v2:172.21.15.161:6800/1841901852,v1:172.21.15.161:6801/1841901852]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T18:09:39.343 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:39 smithi161 ceph-mon[128993]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T18:09:39.387 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:39 smithi171 ceph-mon[129241]: from='osd.1 [v2:172.21.15.161:6800/1841901852,v1:172.21.15.161:6801/1841901852]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T18:09:39.387 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:39 smithi171 ceph-mon[129241]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T18:09:39.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:39 smithi133 ceph-mon[126384]: from='osd.1 [v2:172.21.15.161:6800/1841901852,v1:172.21.15.161:6801/1841901852]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T18:09:39.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:39 smithi133 ceph-mon[126384]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T18:09:39.834 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-14T18:09:39.834 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-14T18:09:39.835 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2023-07-14T18:09:39.835 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2023-07-14T18:09:39.835 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111898 s, 937 MB/s 2023-07-14T18:09:39.835 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2023-07-14T18:09:40.178 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi171:vg_nvme/lv_4 2023-07-14T18:09:40.593 INFO:journalctl@ceph.osd.1.smithi161.stdout:Jul 14 18:09:40 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-1[134087]: 2023-07-14T18:09:40.221+0000 7fcd338d4700 -1 osd.1 0 waiting for initial osdmap 2023-07-14T18:09:40.593 INFO:journalctl@ceph.osd.1.smithi161.stdout:Jul 14 18:09:40 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-1[134087]: 2023-07-14T18:09:40.224+0000 7fcd2b6c2700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T18:09:40.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:40 smithi161 ceph-mon[128993]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-14T18:09:40.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:40 smithi161 ceph-mon[128993]: osdmap e11: 2 total, 1 up, 2 in 2023-07-14T18:09:40.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:40 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T18:09:40.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:40 smithi161 ceph-mon[128993]: from='osd.1 [v2:172.21.15.161:6800/1841901852,v1:172.21.15.161:6801/1841901852]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T18:09:40.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:40 smithi161 ceph-mon[128993]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T18:09:40.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:40 smithi161 ceph-mon[128993]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:40.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:40 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:40.595 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:40 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:40.595 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:40 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:09:40.595 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:40 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:40.595 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:40 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:40.595 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:40 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:09:40.595 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:40 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:40.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:40 smithi171 ceph-mon[129241]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-14T18:09:40.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:40 smithi171 ceph-mon[129241]: osdmap e11: 2 total, 1 up, 2 in 2023-07-14T18:09:40.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:40 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T18:09:40.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:40 smithi171 ceph-mon[129241]: from='osd.1 [v2:172.21.15.161:6800/1841901852,v1:172.21.15.161:6801/1841901852]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T18:09:40.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:40 smithi171 ceph-mon[129241]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T18:09:40.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:40 smithi171 ceph-mon[129241]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:40.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:40 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:40.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:40 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:40.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:40 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:09:40.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:40 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:40.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:40 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:40.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:40 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:09:40.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:40 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:40.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:40 smithi133 ceph-mon[126384]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-14T18:09:40.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:40 smithi133 ceph-mon[126384]: osdmap e11: 2 total, 1 up, 2 in 2023-07-14T18:09:40.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:40 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T18:09:40.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:40 smithi133 ceph-mon[126384]: from='osd.1 [v2:172.21.15.161:6800/1841901852,v1:172.21.15.161:6801/1841901852]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T18:09:40.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:40 smithi133 ceph-mon[126384]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T18:09:40.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:40 smithi133 ceph-mon[126384]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T18:09:40.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:40 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:40.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:40 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:40.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:40 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:09:40.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:40 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:40.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:40 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:40.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:40 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:09:40.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:40 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:40.732 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T18:09:41.593 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:41 smithi161 ceph-mon[128993]: Detected new or changed devices on smithi161 2023-07-14T18:09:41.593 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:41 smithi161 ceph-mon[128993]: Adjusting osd_memory_target on smithi161 to 17168M 2023-07-14T18:09:41.593 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:41 smithi161 ceph-mon[128993]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T18:09:41.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:41 smithi161 ceph-mon[128993]: osdmap e12: 2 total, 1 up, 2 in 2023-07-14T18:09:41.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:41 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T18:09:41.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:41 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T18:09:41.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:41 smithi171 ceph-mon[129241]: Detected new or changed devices on smithi161 2023-07-14T18:09:41.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:41 smithi171 ceph-mon[129241]: Adjusting osd_memory_target on smithi161 to 17168M 2023-07-14T18:09:41.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:41 smithi171 ceph-mon[129241]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T18:09:41.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:41 smithi171 ceph-mon[129241]: osdmap e12: 2 total, 1 up, 2 in 2023-07-14T18:09:41.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:41 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T18:09:41.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:41 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T18:09:41.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:41 smithi133 ceph-mon[126384]: Detected new or changed devices on smithi161 2023-07-14T18:09:41.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:41 smithi133 ceph-mon[126384]: Adjusting osd_memory_target on smithi161 to 17168M 2023-07-14T18:09:41.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:41 smithi133 ceph-mon[126384]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T18:09:41.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:41 smithi133 ceph-mon[126384]: osdmap e12: 2 total, 1 up, 2 in 2023-07-14T18:09:41.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:41 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T18:09:41.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:41 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T18:09:41.906 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.863+0000 7f25fc54d700 1 -- 172.21.15.171:0/2552321723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25f41001c0 msgr2=0x7f25f41005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:09:41.907 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.863+0000 7f25fc54d700 1 --2- 172.21.15.171:0/2552321723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25f41001c0 0x7f25f41005d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009990 tx=0x7f25e4009ca0 comp rx=0 tx=0).stop 2023-07-14T18:09:41.907 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.863+0000 7f25fc54d700 1 -- 172.21.15.171:0/2552321723 shutdown_connections 2023-07-14T18:09:41.907 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.863+0000 7f25fc54d700 1 --2- 172.21.15.171:0/2552321723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25f40ff820 0x7f25f41070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:09:41.907 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.863+0000 7f25fc54d700 1 --2- 172.21.15.171:0/2552321723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25f41001c0 0x7f25f41005d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:09:41.908 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.863+0000 7f25fc54d700 1 --2- 172.21.15.171:0/2552321723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25f40fee70 0x7f25f40ff2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:09:41.908 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.863+0000 7f25fc54d700 1 -- 172.21.15.171:0/2552321723 >> 172.21.15.171:0/2552321723 conn(0x7f25f40f9990 msgr2=0x7f25f40fbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:09:41.908 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.863+0000 7f25fc54d700 1 -- 172.21.15.171:0/2552321723 shutdown_connections 2023-07-14T18:09:41.908 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.864+0000 7f25fc54d700 1 -- 172.21.15.171:0/2552321723 wait complete. 2023-07-14T18:09:41.909 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.865+0000 7f25fc54d700 1 Processor -- start 2023-07-14T18:09:41.909 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.865+0000 7f25fc54d700 1 -- start start 2023-07-14T18:09:41.909 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.865+0000 7f25fc54d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25f40fee70 0x7f25f4194950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:09:41.910 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.865+0000 7f25fc54d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25f40ff820 0x7f25f4194e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:09:41.910 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.865+0000 7f25fc54d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25f41001c0 0x7f25f419bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:09:41.910 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.865+0000 7f25fc54d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f419c4e0 con 0x7f25f40ff820 2023-07-14T18:09:41.910 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.865+0000 7f25fc54d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f419c620 con 0x7f25f41001c0 2023-07-14T18:09:41.911 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.865+0000 7f25fc54d700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f419c7b0 con 0x7f25f40fee70 2023-07-14T18:09:41.911 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.865+0000 7f25fa2e9700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25f40fee70 0x7f25f4194950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:09:41.911 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.865+0000 7f25fa2e9700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25f40fee70 0x7f25f4194950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:42764/0 (socket says 172.21.15.171:42764) 2023-07-14T18:09:41.912 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.865+0000 7f25fa2e9700 1 -- 172.21.15.171:0/2784736812 learned_addr learned my addr 172.21.15.171:0/2784736812 (peer_addr_for_me v2:172.21.15.171:0/0) 2023-07-14T18:09:41.912 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.865+0000 7f25faaea700 1 --2- 172.21.15.171:0/2784736812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25f41001c0 0x7f25f419bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:09:41.912 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.865+0000 7f25f9ae8700 1 --2- 172.21.15.171:0/2784736812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25f40ff820 0x7f25f4194e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:09:41.913 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.866+0000 7f25fa2e9700 1 -- 172.21.15.171:0/2784736812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25f41001c0 msgr2=0x7f25f419bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:09:41.913 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.866+0000 7f25fa2e9700 1 --2- 172.21.15.171:0/2784736812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25f41001c0 0x7f25f419bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:09:41.913 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.866+0000 7f25fa2e9700 1 -- 172.21.15.171:0/2784736812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25f40ff820 msgr2=0x7f25f4194e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:09:41.914 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.866+0000 7f25fa2e9700 1 --2- 172.21.15.171:0/2784736812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25f40ff820 0x7f25f4194e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:09:41.914 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.866+0000 7f25fa2e9700 1 -- 172.21.15.171:0/2784736812 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e40095f0 con 0x7f25f40fee70 2023-07-14T18:09:41.914 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.866+0000 7f25fa2e9700 1 --2- 172.21.15.171:0/2784736812 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25f40fee70 0x7f25f4194950 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f25e400c010 tx=0x7f25e400f700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:09:41.914 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.866+0000 7f25eb7fe700 1 -- 172.21.15.171:0/2784736812 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e401d070 con 0x7f25f40fee70 2023-07-14T18:09:41.915 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.866+0000 7f25eb7fe700 1 -- 172.21.15.171:0/2784736812 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25e4022470 con 0x7f25f40fee70 2023-07-14T18:09:41.915 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.866+0000 7f25fc54d700 1 -- 172.21.15.171:0/2784736812 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f419ca30 con 0x7f25f40fee70 2023-07-14T18:09:41.915 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.866+0000 7f25eb7fe700 1 -- 172.21.15.171:0/2784736812 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e4017630 con 0x7f25f40fee70 2023-07-14T18:09:41.916 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.866+0000 7f25fc54d700 1 -- 172.21.15.171:0/2784736812 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f419cea0 con 0x7f25f40fee70 2023-07-14T18:09:41.916 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.867+0000 7f25fc54d700 1 -- 172.21.15.171:0/2784736812 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25f410a420 con 0x7f25f40fee70 2023-07-14T18:09:41.916 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.868+0000 7f25eb7fe700 1 -- 172.21.15.171:0/2784736812 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 15) v1 ==== 135660+0+0 (secure 0 0 0) 0x7f25e40228e0 con 0x7f25f40fee70 2023-07-14T18:09:41.917 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.868+0000 7f25eb7fe700 1 --2- 172.21.15.171:0/2784736812 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f25e00a1140 0x7f25e00a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:09:41.917 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.869+0000 7f25eb7fe700 1 -- 172.21.15.171:0/2784736812 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f25e40cc810 con 0x7f25f40fee70 2023-07-14T18:09:41.917 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.869+0000 7f25f9ae8700 1 --2- 172.21.15.171:0/2784736812 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f25e00a1140 0x7f25e00a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:09:41.918 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.869+0000 7f25f9ae8700 1 --2- 172.21.15.171:0/2784736812 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f25e00a1140 0x7f25e00a35f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f25f40ffef0 tx=0x7f25f000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:09:41.918 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:41.872+0000 7f25eb7fe700 1 -- 172.21.15.171:0/2784736812 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f25e4097840 con 0x7f25f40fee70 2023-07-14T18:09:42.103 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:42.065+0000 7f25fc54d700 1 -- 172.21.15.171:0/2784736812 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f25f40fe6f0 con 0x7f25e00a1140 2023-07-14T18:09:42.593 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:42 smithi161 ceph-mon[128993]: purged_snaps scrub starts 2023-07-14T18:09:42.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:42 smithi161 ceph-mon[128993]: purged_snaps scrub ok 2023-07-14T18:09:42.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:42 smithi161 ceph-mon[128993]: osd.1 [v2:172.21.15.161:6800/1841901852,v1:172.21.15.161:6801/1841901852] boot 2023-07-14T18:09:42.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:42 smithi161 ceph-mon[128993]: osdmap e13: 2 total, 2 up, 2 in 2023-07-14T18:09:42.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:42 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T18:09:42.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:42 smithi161 ceph-mon[128993]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:09:42.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:42 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:09:42.595 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:42 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:09:42.595 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:42 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:42.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:42 smithi171 ceph-mon[129241]: purged_snaps scrub starts 2023-07-14T18:09:42.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:42 smithi171 ceph-mon[129241]: purged_snaps scrub ok 2023-07-14T18:09:42.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:42 smithi171 ceph-mon[129241]: osd.1 [v2:172.21.15.161:6800/1841901852,v1:172.21.15.161:6801/1841901852] boot 2023-07-14T18:09:42.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:42 smithi171 ceph-mon[129241]: osdmap e13: 2 total, 2 up, 2 in 2023-07-14T18:09:42.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:42 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T18:09:42.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:42 smithi171 ceph-mon[129241]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:09:42.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:42 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:09:42.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:42 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:09:42.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:42 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:42.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:42 smithi133 ceph-mon[126384]: purged_snaps scrub starts 2023-07-14T18:09:42.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:42 smithi133 ceph-mon[126384]: purged_snaps scrub ok 2023-07-14T18:09:42.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:42 smithi133 ceph-mon[126384]: osd.1 [v2:172.21.15.161:6800/1841901852,v1:172.21.15.161:6801/1841901852] boot 2023-07-14T18:09:42.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:42 smithi133 ceph-mon[126384]: osdmap e13: 2 total, 2 up, 2 in 2023-07-14T18:09:42.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:42 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T18:09:42.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:42 smithi133 ceph-mon[126384]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:09:42.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:42 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:09:42.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:42 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:09:42.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:42 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:43.593 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:43 smithi161 ceph-mon[128993]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:09:43.593 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:43 smithi161 ceph-mon[128993]: osdmap e14: 2 total, 2 up, 2 in 2023-07-14T18:09:43.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:43 smithi171 ceph-mon[129241]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:09:43.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:43 smithi171 ceph-mon[129241]: osdmap e14: 2 total, 2 up, 2 in 2023-07-14T18:09:43.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:43 smithi133 ceph-mon[126384]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:09:43.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:43 smithi133 ceph-mon[126384]: osdmap e14: 2 total, 2 up, 2 in 2023-07-14T18:09:44.593 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:44 smithi161 ceph-mon[128993]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:09:44.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:44 smithi171 ceph-mon[129241]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:09:44.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:44 smithi133 ceph-mon[126384]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:09:45.593 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:45 smithi161 ceph-mon[128993]: from='client.? 172.21.15.171:0/2674609128' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc4bfd37-1fdd-47c6-a48e-6d910daf9e78"}]: dispatch 2023-07-14T18:09:45.593 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:45 smithi161 ceph-mon[128993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc4bfd37-1fdd-47c6-a48e-6d910daf9e78"}]: dispatch 2023-07-14T18:09:45.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:45 smithi161 ceph-mon[128993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc4bfd37-1fdd-47c6-a48e-6d910daf9e78"}]': finished 2023-07-14T18:09:45.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:45 smithi161 ceph-mon[128993]: osdmap e15: 3 total, 2 up, 3 in 2023-07-14T18:09:45.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:45 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T18:09:45.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:45 smithi161 ceph-mon[128993]: from='client.? 172.21.15.171:0/1963654270' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:09:45.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:45 smithi171 ceph-mon[129241]: from='client.? 172.21.15.171:0/2674609128' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc4bfd37-1fdd-47c6-a48e-6d910daf9e78"}]: dispatch 2023-07-14T18:09:45.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:45 smithi171 ceph-mon[129241]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc4bfd37-1fdd-47c6-a48e-6d910daf9e78"}]: dispatch 2023-07-14T18:09:45.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:45 smithi171 ceph-mon[129241]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc4bfd37-1fdd-47c6-a48e-6d910daf9e78"}]': finished 2023-07-14T18:09:45.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:45 smithi171 ceph-mon[129241]: osdmap e15: 3 total, 2 up, 3 in 2023-07-14T18:09:45.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:45 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T18:09:45.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:45 smithi171 ceph-mon[129241]: from='client.? 172.21.15.171:0/1963654270' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:09:45.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:45 smithi133 ceph-mon[126384]: from='client.? 172.21.15.171:0/2674609128' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc4bfd37-1fdd-47c6-a48e-6d910daf9e78"}]: dispatch 2023-07-14T18:09:45.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:45 smithi133 ceph-mon[126384]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc4bfd37-1fdd-47c6-a48e-6d910daf9e78"}]: dispatch 2023-07-14T18:09:45.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:45 smithi133 ceph-mon[126384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc4bfd37-1fdd-47c6-a48e-6d910daf9e78"}]': finished 2023-07-14T18:09:45.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:45 smithi133 ceph-mon[126384]: osdmap e15: 3 total, 2 up, 3 in 2023-07-14T18:09:45.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:45 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T18:09:45.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:45 smithi133 ceph-mon[126384]: from='client.? 172.21.15.171:0/1963654270' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:09:46.593 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:46 smithi161 ceph-mon[128993]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:09:46.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:46 smithi171 ceph-mon[129241]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:09:46.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:46 smithi133 ceph-mon[126384]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:09:48.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:48 smithi161 ceph-mon[128993]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:09:48.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:48 smithi171 ceph-mon[129241]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:09:48.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:48 smithi133 ceph-mon[126384]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:09:50.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:50 smithi171 ceph-mon[129241]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:09:51.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:50 smithi161 ceph-mon[128993]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:09:51.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:50 smithi133 ceph-mon[126384]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:09:52.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:51 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-14T18:09:52.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:51 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:52.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:51 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-14T18:09:52.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:51 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:52.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:51 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-14T18:09:52.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:51 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:53.092 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:52 smithi171 ceph-mon[129241]: Deploying daemon osd.2 on smithi171 2023-07-14T18:09:53.092 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:52 smithi171 ceph-mon[129241]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:09:53.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:52 smithi161 ceph-mon[128993]: Deploying daemon osd.2 on smithi171 2023-07-14T18:09:53.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:52 smithi161 ceph-mon[128993]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:09:53.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:52 smithi133 ceph-mon[126384]: Deploying daemon osd.2 on smithi171 2023-07-14T18:09:53.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:52 smithi133 ceph-mon[126384]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:09:55.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:54 smithi161 ceph-mon[128993]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:09:55.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:54 smithi171 ceph-mon[129241]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:09:55.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:54 smithi133 ceph-mon[126384]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:09:56.688 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:56 smithi171 ceph-mon[129241]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:09:56.688 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:56 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:56.688 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:56 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:56.688 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:56 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:09:56.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:56 smithi161 ceph-mon[128993]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:09:56.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:56 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:56.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:56 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:56.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:56 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:09:56.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:56 smithi133 ceph-mon[126384]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:09:56.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:56 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:56.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:56 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:56.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:56 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:09:58.621 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 2 on host 'smithi171' 2023-07-14T18:09:58.621 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:58.582+0000 7f25eb7fe700 1 -- 172.21.15.171:0/2784736812 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f25f40fe6f0 con 0x7f25e00a1140 2023-07-14T18:09:58.623 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:58.585+0000 7f25fc54d700 1 -- 172.21.15.171:0/2784736812 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f25e00a1140 msgr2=0x7f25e00a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:09:58.623 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:58.585+0000 7f25fc54d700 1 --2- 172.21.15.171:0/2784736812 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f25e00a1140 0x7f25e00a35f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f25f40ffef0 tx=0x7f25f000a380 comp rx=0 tx=0).stop 2023-07-14T18:09:58.623 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:58.585+0000 7f25fc54d700 1 -- 172.21.15.171:0/2784736812 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25f40fee70 msgr2=0x7f25f4194950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:09:58.623 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:58.585+0000 7f25fc54d700 1 --2- 172.21.15.171:0/2784736812 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25f40fee70 0x7f25f4194950 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f25e400c010 tx=0x7f25e400f700 comp rx=0 tx=0).stop 2023-07-14T18:09:58.623 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:58.585+0000 7f25fc54d700 1 -- 172.21.15.171:0/2784736812 shutdown_connections 2023-07-14T18:09:58.624 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:58.585+0000 7f25fc54d700 1 --2- 172.21.15.171:0/2784736812 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f25e00a1140 0x7f25e00a35f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:09:58.624 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:58.585+0000 7f25fc54d700 1 --2- 172.21.15.171:0/2784736812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25f41001c0 0x7f25f419bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:09:58.624 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:58.585+0000 7f25fc54d700 1 --2- 172.21.15.171:0/2784736812 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25f40fee70 0x7f25f4194950 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:09:58.624 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:58.585+0000 7f25fc54d700 1 --2- 172.21.15.171:0/2784736812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25f40ff820 0x7f25f4194e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:09:58.624 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:58.585+0000 7f25fc54d700 1 -- 172.21.15.171:0/2784736812 >> 172.21.15.171:0/2784736812 conn(0x7f25f40f9990 msgr2=0x7f25f40fbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:09:58.624 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:58.585+0000 7f25fc54d700 1 -- 172.21.15.171:0/2784736812 shutdown_connections 2023-07-14T18:09:58.624 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:09:58.585+0000 7f25fc54d700 1 -- 172.21.15.171:0/2784736812 wait complete. 2023-07-14T18:09:58.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:58 smithi161 ceph-mon[128993]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:09:58.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:58 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:58.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:58 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:58.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:58 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:58.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:58 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:09:58.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:58 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:58.845 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:58 smithi171 ceph-mon[129241]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:09:58.846 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:58 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:58.846 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:58 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:58.846 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:58 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:58.846 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:58 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:09:58.846 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:58 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:58.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:58 smithi133 ceph-mon[126384]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:09:58.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:58 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:58.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:58 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:58.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:58 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:09:58.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:58 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:09:58.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:58 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:59.214 DEBUG:teuthology.orchestra.run.smithi171:osd.2> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.2.service 2023-07-14T18:09:59.216 INFO:tasks.cephadm:Deploying osd.3 on smithi133 with /dev/vg_nvme/lv_3... 2023-07-14T18:09:59.217 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-14T18:09:59.248 INFO:journalctl@ceph.osd.2.smithi171.stdout:-- Logs begin at Fri 2023-07-14 17:41:48 UTC. -- 2023-07-14T18:09:59.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:59 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:59.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:59 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:59.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:59 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:09:59.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:59 smithi161 ceph-mon[128993]: from='osd.2 [v2:172.21.15.171:6800/346103099,v1:172.21.15.171:6801/346103099]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T18:09:59.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:09:59 smithi161 ceph-mon[128993]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T18:09:59.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:59 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:59.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:59 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:59.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:59 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:09:59.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:59 smithi171 ceph-mon[129241]: from='osd.2 [v2:172.21.15.171:6800/346103099,v1:172.21.15.171:6801/346103099]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T18:09:59.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:09:59 smithi171 ceph-mon[129241]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T18:09:59.860 INFO:journalctl@ceph.osd.2.smithi171.stdout:Jul 14 18:09:59 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-2[135063]: 2023-07-14T18:09:59.494+0000 7f621169e640 -1 osd.2 0 log_to_monitors true 2023-07-14T18:09:59.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:59 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:59.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:59 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:09:59.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:59 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:09:59.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:59 smithi133 ceph-mon[126384]: from='osd.2 [v2:172.21.15.171:6800/346103099,v1:172.21.15.171:6801/346103099]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T18:09:59.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:09:59 smithi133 ceph-mon[126384]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T18:09:59.943 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:10:00.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:00 smithi171 ceph-mon[129241]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:10:00.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:00 smithi171 ceph-mon[129241]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-14T18:10:00.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:00 smithi171 ceph-mon[129241]: osdmap e16: 3 total, 2 up, 3 in 2023-07-14T18:10:00.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:00 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T18:10:00.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:00 smithi171 ceph-mon[129241]: from='osd.2 [v2:172.21.15.171:6800/346103099,v1:172.21.15.171:6801/346103099]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2023-07-14T18:10:00.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:00 smithi171 ceph-mon[129241]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2023-07-14T18:10:00.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:00 smithi171 ceph-mon[129241]: overall HEALTH_OK 2023-07-14T18:10:00.861 INFO:journalctl@ceph.osd.2.smithi171.stdout:Jul 14 18:10:00 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-2[135063]: 2023-07-14T18:10:00.562+0000 7f6206d16700 -1 osd.2 0 waiting for initial osdmap 2023-07-14T18:10:00.861 INFO:journalctl@ceph.osd.2.smithi171.stdout:Jul 14 18:10:00 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-2[135063]: 2023-07-14T18:10:00.566+0000 7f6200b08700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T18:10:00.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:00 smithi133 ceph-mon[126384]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:10:00.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:00 smithi133 ceph-mon[126384]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-14T18:10:00.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:00 smithi133 ceph-mon[126384]: osdmap e16: 3 total, 2 up, 3 in 2023-07-14T18:10:00.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:00 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T18:10:00.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:00 smithi133 ceph-mon[126384]: from='osd.2 [v2:172.21.15.171:6800/346103099,v1:172.21.15.171:6801/346103099]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2023-07-14T18:10:00.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:00 smithi133 ceph-mon[126384]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2023-07-14T18:10:00.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:00 smithi133 ceph-mon[126384]: overall HEALTH_OK 2023-07-14T18:10:01.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:00 smithi161 ceph-mon[128993]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:10:01.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:00 smithi161 ceph-mon[128993]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-14T18:10:01.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:00 smithi161 ceph-mon[128993]: osdmap e16: 3 total, 2 up, 3 in 2023-07-14T18:10:01.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:00 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T18:10:01.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:00 smithi161 ceph-mon[128993]: from='osd.2 [v2:172.21.15.171:6800/346103099,v1:172.21.15.171:6801/346103099]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2023-07-14T18:10:01.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:00 smithi161 ceph-mon[128993]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2023-07-14T18:10:01.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:00 smithi161 ceph-mon[128993]: overall HEALTH_OK 2023-07-14T18:10:01.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:01 smithi171 ceph-mon[129241]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2023-07-14T18:10:01.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:01 smithi171 ceph-mon[129241]: osdmap e17: 3 total, 2 up, 3 in 2023-07-14T18:10:01.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:01 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T18:10:01.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:01 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T18:10:01.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:01 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:01.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:01 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:01.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:01 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:10:01.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:01 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:01.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:01 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:01.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:01 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:10:01.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:01 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:01.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:01 smithi133 ceph-mon[126384]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2023-07-14T18:10:01.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:01 smithi133 ceph-mon[126384]: osdmap e17: 3 total, 2 up, 3 in 2023-07-14T18:10:01.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:01 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T18:10:01.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:01 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T18:10:01.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:01 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:01.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:01 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:01.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:01 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:10:01.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:01 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:01.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:01 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:01.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:01 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:10:01.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:01 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:02.066 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-14T18:10:02.066 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-14T18:10:02.066 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2023-07-14T18:10:02.066 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2023-07-14T18:10:02.066 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0299468 s, 350 MB/s 2023-07-14T18:10:02.067 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2023-07-14T18:10:02.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:01 smithi161 ceph-mon[128993]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2023-07-14T18:10:02.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:01 smithi161 ceph-mon[128993]: osdmap e17: 3 total, 2 up, 3 in 2023-07-14T18:10:02.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:01 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T18:10:02.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:01 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T18:10:02.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:01 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:02.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:01 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:02.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:01 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:10:02.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:01 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:02.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:01 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:02.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:01 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:10:02.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:01 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:02.522 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi133:vg_nvme/lv_3 2023-07-14T18:10:02.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:02 smithi133 ceph-mon[126384]: purged_snaps scrub starts 2023-07-14T18:10:02.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:02 smithi133 ceph-mon[126384]: purged_snaps scrub ok 2023-07-14T18:10:02.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:02 smithi133 ceph-mon[126384]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:10:02.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:02 smithi133 ceph-mon[126384]: Detected new or changed devices on smithi171 2023-07-14T18:10:02.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:02 smithi133 ceph-mon[126384]: Adjusting osd_memory_target on smithi171 to 17168M 2023-07-14T18:10:02.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:02 smithi133 ceph-mon[126384]: osd.2 [v2:172.21.15.171:6800/346103099,v1:172.21.15.171:6801/346103099] boot 2023-07-14T18:10:02.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:02 smithi133 ceph-mon[126384]: osdmap e18: 3 total, 3 up, 3 in 2023-07-14T18:10:02.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:02 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T18:10:03.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:02 smithi161 ceph-mon[128993]: purged_snaps scrub starts 2023-07-14T18:10:03.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:02 smithi161 ceph-mon[128993]: purged_snaps scrub ok 2023-07-14T18:10:03.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:02 smithi161 ceph-mon[128993]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:10:03.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:02 smithi161 ceph-mon[128993]: Detected new or changed devices on smithi171 2023-07-14T18:10:03.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:02 smithi161 ceph-mon[128993]: Adjusting osd_memory_target on smithi171 to 17168M 2023-07-14T18:10:03.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:02 smithi161 ceph-mon[128993]: osd.2 [v2:172.21.15.171:6800/346103099,v1:172.21.15.171:6801/346103099] boot 2023-07-14T18:10:03.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:02 smithi161 ceph-mon[128993]: osdmap e18: 3 total, 3 up, 3 in 2023-07-14T18:10:03.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:02 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T18:10:03.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:02 smithi171 ceph-mon[129241]: purged_snaps scrub starts 2023-07-14T18:10:03.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:02 smithi171 ceph-mon[129241]: purged_snaps scrub ok 2023-07-14T18:10:03.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:02 smithi171 ceph-mon[129241]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T18:10:03.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:02 smithi171 ceph-mon[129241]: Detected new or changed devices on smithi171 2023-07-14T18:10:03.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:02 smithi171 ceph-mon[129241]: Adjusting osd_memory_target on smithi171 to 17168M 2023-07-14T18:10:03.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:02 smithi171 ceph-mon[129241]: osd.2 [v2:172.21.15.171:6800/346103099,v1:172.21.15.171:6801/346103099] boot 2023-07-14T18:10:03.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:02 smithi171 ceph-mon[129241]: osdmap e18: 3 total, 3 up, 3 in 2023-07-14T18:10:03.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:02 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T18:10:03.242 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:10:04.131 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:03 smithi133 ceph-mon[126384]: osdmap e19: 3 total, 3 up, 3 in 2023-07-14T18:10:04.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:03 smithi161 ceph-mon[128993]: osdmap e19: 3 total, 3 up, 3 in 2023-07-14T18:10:04.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:03 smithi171 ceph-mon[129241]: osdmap e19: 3 total, 3 up, 3 in 2023-07-14T18:10:04.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.688+0000 7f7038038700 1 -- 172.21.15.133:0/4154952147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7030103060 msgr2=0x7f7030105440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:10:04.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.688+0000 7f7038038700 1 --2- 172.21.15.133:0/4154952147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7030103060 0x7f7030105440 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7020009990 tx=0x7f7020009ca0 comp rx=0 tx=0).stop 2023-07-14T18:10:04.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.688+0000 7f7038038700 1 -- 172.21.15.133:0/4154952147 shutdown_connections 2023-07-14T18:10:04.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.688+0000 7f7038038700 1 --2- 172.21.15.133:0/4154952147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7030105980 0x7f7030107de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:10:04.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.688+0000 7f7038038700 1 --2- 172.21.15.133:0/4154952147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7030100740 0x7f7030102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:10:04.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.688+0000 7f7038038700 1 --2- 172.21.15.133:0/4154952147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7030103060 0x7f7030105440 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:10:04.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.688+0000 7f7038038700 1 -- 172.21.15.133:0/4154952147 >> 172.21.15.133:0/4154952147 conn(0x7f70300fa1e0 msgr2=0x7f70300fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:10:04.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.688+0000 7f7038038700 1 -- 172.21.15.133:0/4154952147 shutdown_connections 2023-07-14T18:10:04.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.688+0000 7f7038038700 1 -- 172.21.15.133:0/4154952147 wait complete. 2023-07-14T18:10:04.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.689+0000 7f7038038700 1 Processor -- start 2023-07-14T18:10:04.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.689+0000 7f7038038700 1 -- start start 2023-07-14T18:10:04.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.690+0000 7f7038038700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7030100740 0x7f7030195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:10:04.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.690+0000 7f7038038700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7030103060 0x7f7030195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:10:04.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.690+0000 7f7038038700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7030105980 0x7f703019cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:10:04.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.690+0000 7f7038038700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703019d590 con 0x7f7030100740 2023-07-14T18:10:04.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.690+0000 7f7038038700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703019d6d0 con 0x7f7030105980 2023-07-14T18:10:04.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.690+0000 7f7038038700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703019d860 con 0x7f7030103060 2023-07-14T18:10:04.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.690+0000 7f7035dd4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7030100740 0x7f7030195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:10:04.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.690+0000 7f70355d3700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7030103060 0x7f7030195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:10:04.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.690+0000 7f7035dd4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7030100740 0x7f7030195a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37488/0 (socket says 172.21.15.133:37488) 2023-07-14T18:10:04.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.690+0000 7f7035dd4700 1 -- 172.21.15.133:0/3637917794 learned_addr learned my addr 172.21.15.133:0/3637917794 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:10:04.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.690+0000 7f70365d5700 1 --2- 172.21.15.133:0/3637917794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7030105980 0x7f703019cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:10:04.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.690+0000 7f7035dd4700 1 -- 172.21.15.133:0/3637917794 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7030103060 msgr2=0x7f7030195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:10:04.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.690+0000 7f7035dd4700 1 --2- 172.21.15.133:0/3637917794 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7030103060 0x7f7030195f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:10:04.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.690+0000 7f7035dd4700 1 -- 172.21.15.133:0/3637917794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7030105980 msgr2=0x7f703019cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:10:04.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.690+0000 7f7035dd4700 1 --2- 172.21.15.133:0/3637917794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7030105980 0x7f703019cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:10:04.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.690+0000 7f7035dd4700 1 -- 172.21.15.133:0/3637917794 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f702c009520 con 0x7f7030100740 2023-07-14T18:10:04.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.690+0000 7f70355d3700 1 --2- 172.21.15.133:0/3637917794 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7030103060 0x7f7030195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:10:04.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.691+0000 7f7035dd4700 1 --2- 172.21.15.133:0/3637917794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7030100740 0x7f7030195a00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f702c00eb60 tx=0x7f702c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:10:04.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.691+0000 7f7026ffd700 1 -- 172.21.15.133:0/3637917794 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c00cbd0 con 0x7f7030100740 2023-07-14T18:10:04.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.691+0000 7f7026ffd700 1 -- 172.21.15.133:0/3637917794 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f702c0044e0 con 0x7f7030100740 2023-07-14T18:10:04.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.691+0000 7f7026ffd700 1 -- 172.21.15.133:0/3637917794 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c003df0 con 0x7f7030100740 2023-07-14T18:10:04.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.691+0000 7f7038038700 1 -- 172.21.15.133:0/3637917794 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70200095f0 con 0x7f7030100740 2023-07-14T18:10:04.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.691+0000 7f7038038700 1 -- 172.21.15.133:0/3637917794 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f703019de60 con 0x7f7030100740 2023-07-14T18:10:04.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.692+0000 7f7038038700 1 -- 172.21.15.133:0/3637917794 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70300fbd50 con 0x7f7030100740 2023-07-14T18:10:04.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.694+0000 7f7026ffd700 1 -- 172.21.15.133:0/3637917794 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 15) v1 ==== 135660+0+0 (secure 0 0 0) 0x7f702c003f50 con 0x7f7030100740 2023-07-14T18:10:04.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.695+0000 7f7026ffd700 1 --2- 172.21.15.133:0/3637917794 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f701c0a1140 0x7f701c0a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:10:04.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.696+0000 7f7026ffd700 1 -- 172.21.15.133:0/3637917794 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3226+0+0 (secure 0 0 0) 0x7f702c014070 con 0x7f7030100740 2023-07-14T18:10:04.735 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.696+0000 7f70355d3700 1 --2- 172.21.15.133:0/3637917794 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f701c0a1140 0x7f701c0a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:10:04.735 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.696+0000 7f70355d3700 1 --2- 172.21.15.133:0/3637917794 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f701c0a1140 0x7f701c0a35f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f702000b2e0 tx=0x7f7020005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:10:04.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.698+0000 7f7026ffd700 1 -- 172.21.15.133:0/3637917794 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f702c096550 con 0x7f7030100740 2023-07-14T18:10:04.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:04.904+0000 7f7038038700 1 -- 172.21.15.133:0/3637917794 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f7030060a80 con 0x7f701c0a1140 2023-07-14T18:10:05.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:04 smithi161 ceph-mon[128993]: pgmap v77: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T18:10:05.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:04 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-14T18:10:05.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:04 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-14T18:10:05.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:04 smithi161 ceph-mon[128993]: osdmap e20: 3 total, 3 up, 3 in 2023-07-14T18:10:05.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:04 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T18:10:05.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:04 smithi161 ceph-mon[128993]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:10:05.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:04 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:10:05.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:04 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:10:05.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:04 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:05.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:04 smithi171 ceph-mon[129241]: pgmap v77: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T18:10:05.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:04 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-14T18:10:05.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:04 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-14T18:10:05.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:04 smithi171 ceph-mon[129241]: osdmap e20: 3 total, 3 up, 3 in 2023-07-14T18:10:05.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:04 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T18:10:05.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:04 smithi171 ceph-mon[129241]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:10:05.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:04 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:10:05.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:04 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:10:05.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:04 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:05.399 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:04 smithi133 ceph-mon[126384]: pgmap v77: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T18:10:05.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:04 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-14T18:10:05.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:04 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-14T18:10:05.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:04 smithi133 ceph-mon[126384]: osdmap e20: 3 total, 3 up, 3 in 2023-07-14T18:10:05.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:04 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T18:10:05.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:04 smithi133 ceph-mon[126384]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:10:05.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:04 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:10:05.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:04 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:10:05.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:04 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:06.047 INFO:journalctl@ceph.osd.0.smithi133.stdout:Jul 14 18:10:05 smithi133 sudo[142339]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T18:10:06.047 INFO:journalctl@ceph.osd.0.smithi133.stdout:Jul 14 18:10:05 smithi133 sudo[142339]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T18:10:06.048 INFO:journalctl@ceph.osd.0.smithi133.stdout:Jul 14 18:10:05 smithi133 sudo[142339]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T18:10:06.048 INFO:journalctl@ceph.osd.0.smithi133.stdout:Jul 14 18:10:05 smithi133 sudo[142339]: pam_unix(sudo:session): session closed for user root 2023-07-14T18:10:06.048 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:06 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-14T18:10:06.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:06.018+0000 7f7026ffd700 1 -- 172.21.15.133:0/3637917794 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f702c004200 con 0x7f7030100740 2023-07-14T18:10:06.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:06 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-14T18:10:06.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:06 smithi161 ceph-mon[128993]: osdmap e21: 3 total, 3 up, 3 in 2023-07-14T18:10:06.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:06 smithi161 ceph-mon[128993]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T18:10:06.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:06 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-14T18:10:06.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:06 smithi171 ceph-mon[129241]: osdmap e21: 3 total, 3 up, 3 in 2023-07-14T18:10:06.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:06 smithi171 ceph-mon[129241]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T18:10:06.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:06 smithi133 ceph-mon[126384]: osdmap e21: 3 total, 3 up, 3 in 2023-07-14T18:10:06.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:06 smithi133 ceph-mon[126384]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T18:10:06.398 INFO:journalctl@ceph.osd.0.smithi133.stdout:Jul 14 18:10:06 smithi133 sudo[142385]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T18:10:06.398 INFO:journalctl@ceph.osd.0.smithi133.stdout:Jul 14 18:10:06 smithi133 sudo[142385]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T18:10:06.398 INFO:journalctl@ceph.osd.0.smithi133.stdout:Jul 14 18:10:06 smithi133 sudo[142385]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T18:10:06.398 INFO:journalctl@ceph.osd.0.smithi133.stdout:Jul 14 18:10:06 smithi133 sudo[142385]: pam_unix(sudo:session): session closed for user root 2023-07-14T18:10:06.773 INFO:journalctl@ceph.osd.1.smithi161.stdout:Jul 14 18:10:06 smithi161 sudo[136293]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T18:10:06.773 INFO:journalctl@ceph.osd.1.smithi161.stdout:Jul 14 18:10:06 smithi161 sudo[136293]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T18:10:06.773 INFO:journalctl@ceph.osd.1.smithi161.stdout:Jul 14 18:10:06 smithi161 sudo[136293]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T18:10:06.773 INFO:journalctl@ceph.osd.1.smithi161.stdout:Jul 14 18:10:06 smithi161 sudo[136293]: pam_unix(sudo:session): session closed for user root 2023-07-14T18:10:07.050 INFO:journalctl@ceph.osd.1.smithi161.stdout:Jul 14 18:10:06 smithi161 sudo[136296]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T18:10:07.050 INFO:journalctl@ceph.osd.1.smithi161.stdout:Jul 14 18:10:06 smithi161 sudo[136296]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T18:10:07.050 INFO:journalctl@ceph.osd.1.smithi161.stdout:Jul 14 18:10:06 smithi161 sudo[136296]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T18:10:07.050 INFO:journalctl@ceph.osd.1.smithi161.stdout:Jul 14 18:10:06 smithi161 sudo[136296]: pam_unix(sudo:session): session closed for user root 2023-07-14T18:10:07.291 INFO:journalctl@ceph.osd.2.smithi171.stdout:Jul 14 18:10:07 smithi171 sudo[137301]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T18:10:07.291 INFO:journalctl@ceph.osd.2.smithi171.stdout:Jul 14 18:10:07 smithi171 sudo[137301]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T18:10:07.291 INFO:journalctl@ceph.osd.2.smithi171.stdout:Jul 14 18:10:07 smithi171 sudo[137301]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T18:10:07.292 INFO:journalctl@ceph.osd.2.smithi171.stdout:Jul 14 18:10:07 smithi171 sudo[137301]: pam_unix(sudo:session): session closed for user root 2023-07-14T18:10:07.292 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:07 smithi171 ceph-mon[129241]: osdmap e22: 3 total, 3 up, 3 in 2023-07-14T18:10:07.292 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:07 smithi171 ceph-mon[129241]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-14T18:10:07.292 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:07 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:07.293 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:07 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:07.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:07 smithi161 ceph-mon[128993]: osdmap e22: 3 total, 3 up, 3 in 2023-07-14T18:10:07.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:07 smithi161 ceph-mon[128993]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-14T18:10:07.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:07 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:07.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:07 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:07.399 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:07 smithi133 ceph-mon[126384]: osdmap e22: 3 total, 3 up, 3 in 2023-07-14T18:10:07.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:07 smithi133 ceph-mon[126384]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-14T18:10:07.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:07 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:07.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:07 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:07.609 INFO:journalctl@ceph.osd.2.smithi171.stdout:Jul 14 18:10:07 smithi171 sudo[137304]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T18:10:07.609 INFO:journalctl@ceph.osd.2.smithi171.stdout:Jul 14 18:10:07 smithi171 sudo[137304]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T18:10:07.610 INFO:journalctl@ceph.osd.2.smithi171.stdout:Jul 14 18:10:07 smithi171 sudo[137304]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T18:10:07.610 INFO:journalctl@ceph.osd.2.smithi171.stdout:Jul 14 18:10:07 smithi171 sudo[137304]: pam_unix(sudo:session): session closed for user root 2023-07-14T18:10:08.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:07 smithi133 sudo[142495]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T18:10:08.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:07 smithi133 sudo[142495]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T18:10:08.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:07 smithi133 sudo[142495]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T18:10:08.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:07 smithi133 sudo[142495]: pam_unix(sudo:session): session closed for user root 2023-07-14T18:10:08.626 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:08 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:08.626 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:08 smithi171 ceph-mon[129241]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T18:10:08.626 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:08 smithi171 sudo[137307]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T18:10:08.626 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:08 smithi171 sudo[137307]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T18:10:08.626 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:08 smithi171 sudo[137307]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T18:10:08.627 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:08 smithi171 sudo[137307]: pam_unix(sudo:session): session closed for user root 2023-07-14T18:10:08.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:08 smithi133 sudo[142499]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T18:10:08.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:08 smithi133 sudo[142499]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T18:10:08.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:08 smithi133 sudo[142499]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T18:10:08.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:08 smithi133 sudo[142499]: pam_unix(sudo:session): session closed for user root 2023-07-14T18:10:08.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:08 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:08.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:08 smithi133 ceph-mon[126384]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T18:10:08.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:08 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:08.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:08 smithi161 ceph-mon[128993]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T18:10:09.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:08 smithi171 sudo[137310]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T18:10:09.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:08 smithi171 sudo[137310]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T18:10:09.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:08 smithi171 sudo[137310]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T18:10:09.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:08 smithi171 sudo[137310]: pam_unix(sudo:session): session closed for user root 2023-07-14T18:10:09.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 sudo[136299]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T18:10:09.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 sudo[136299]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T18:10:09.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 sudo[136299]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T18:10:09.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 sudo[136299]: pam_unix(sudo:session): session closed for user root 2023-07-14T18:10:09.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:09 smithi133 ceph-mon[126384]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T18:10:09.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:09 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3138496326' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb"}]: dispatch 2023-07-14T18:10:09.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:09 smithi133 ceph-mon[126384]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T18:10:09.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:09 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T18:10:09.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:09 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:10:09.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:09 smithi133 ceph-mon[126384]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb"}]: dispatch 2023-07-14T18:10:09.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:09 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:10:09.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:09 smithi133 ceph-mon[126384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb"}]': finished 2023-07-14T18:10:09.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:09 smithi133 ceph-mon[126384]: osdmap e23: 4 total, 3 up, 4 in 2023-07-14T18:10:09.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:09 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T18:10:09.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:09 smithi133 ceph-mon[126384]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T18:10:09.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:09 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T18:10:09.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:09 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:10:09.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:09 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:10:09.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:09 smithi133 ceph-mon[126384]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T18:10:09.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:09 smithi133 ceph-mon[126384]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T18:10:09.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:09 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T18:10:09.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:09 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:10:09.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:09 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:10:09.653 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:09 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2491497677' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:10:09.653 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:09 smithi133 ceph-mon[126384]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T18:10:09.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 sudo[136302]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T18:10:09.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 sudo[136302]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T18:10:09.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 sudo[136302]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T18:10:09.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 sudo[136302]: pam_unix(sudo:session): session closed for user root 2023-07-14T18:10:09.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 ceph-mon[128993]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T18:10:09.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3138496326' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb"}]: dispatch 2023-07-14T18:10:09.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 ceph-mon[128993]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T18:10:09.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T18:10:09.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:10:09.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 ceph-mon[128993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb"}]: dispatch 2023-07-14T18:10:09.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:10:09.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 ceph-mon[128993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb"}]': finished 2023-07-14T18:10:09.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 ceph-mon[128993]: osdmap e23: 4 total, 3 up, 4 in 2023-07-14T18:10:09.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T18:10:09.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 ceph-mon[128993]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T18:10:09.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T18:10:09.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:10:09.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:10:09.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 ceph-mon[128993]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T18:10:09.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 ceph-mon[128993]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T18:10:09.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T18:10:09.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:10:09.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:10:09.848 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2491497677' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:10:09.848 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:09 smithi161 ceph-mon[128993]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T18:10:09.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:09 smithi171 ceph-mon[129241]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T18:10:09.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:09 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3138496326' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb"}]: dispatch 2023-07-14T18:10:09.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:09 smithi171 ceph-mon[129241]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T18:10:09.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:09 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T18:10:09.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:09 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:10:09.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:09 smithi171 ceph-mon[129241]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb"}]: dispatch 2023-07-14T18:10:09.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:09 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:10:09.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:09 smithi171 ceph-mon[129241]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb"}]': finished 2023-07-14T18:10:09.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:09 smithi171 ceph-mon[129241]: osdmap e23: 4 total, 3 up, 4 in 2023-07-14T18:10:09.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:09 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T18:10:09.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:09 smithi171 ceph-mon[129241]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T18:10:09.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:09 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T18:10:09.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:09 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:10:09.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:09 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:10:09.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:09 smithi171 ceph-mon[129241]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T18:10:09.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:09 smithi171 ceph-mon[129241]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T18:10:09.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:09 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T18:10:09.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:09 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T18:10:09.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:09 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T18:10:09.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:09 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2491497677' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:10:09.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:09 smithi171 ceph-mon[129241]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T18:10:11.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:10 smithi161 ceph-mon[128993]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T18:10:11.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:10 smithi171 ceph-mon[129241]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T18:10:11.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:10 smithi133 ceph-mon[126384]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T18:10:13.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:12 smithi161 ceph-mon[128993]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 7.2 KiB/s rd, 326 KiB/s wr, 22 op/s 2023-07-14T18:10:13.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:12 smithi171 ceph-mon[129241]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 7.2 KiB/s rd, 326 KiB/s wr, 22 op/s 2023-07-14T18:10:13.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:12 smithi133 ceph-mon[126384]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 7.2 KiB/s rd, 326 KiB/s wr, 22 op/s 2023-07-14T18:10:15.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:14 smithi161 ceph-mon[128993]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 4.8 KiB/s rd, 218 KiB/s wr, 15 op/s 2023-07-14T18:10:15.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:14 smithi171 ceph-mon[129241]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 4.8 KiB/s rd, 218 KiB/s wr, 15 op/s 2023-07-14T18:10:15.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:14 smithi133 ceph-mon[126384]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 4.8 KiB/s rd, 218 KiB/s wr, 15 op/s 2023-07-14T18:10:16.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:15 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-14T18:10:16.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:15 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:16.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:15 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-14T18:10:16.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:15 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:16.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:15 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-14T18:10:16.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:15 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:17.021 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:16 smithi133 ceph-mon[126384]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-07-14T18:10:17.021 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:16 smithi133 ceph-mon[126384]: Deploying daemon osd.3 on smithi133 2023-07-14T18:10:17.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:16 smithi171 ceph-mon[129241]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-07-14T18:10:17.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:16 smithi171 ceph-mon[129241]: Deploying daemon osd.3 on smithi133 2023-07-14T18:10:17.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:16 smithi161 ceph-mon[128993]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-07-14T18:10:17.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:16 smithi161 ceph-mon[128993]: Deploying daemon osd.3 on smithi133 2023-07-14T18:10:19.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:18 smithi171 ceph-mon[129241]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.9 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-07-14T18:10:19.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:18 smithi133 ceph-mon[126384]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.9 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-07-14T18:10:19.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:18 smithi161 ceph-mon[128993]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.9 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-07-14T18:10:21.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:20 smithi171 ceph-mon[129241]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.6 KiB/s rd, 118 KiB/s wr, 8 op/s 2023-07-14T18:10:21.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:20 smithi133 ceph-mon[126384]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.6 KiB/s rd, 118 KiB/s wr, 8 op/s 2023-07-14T18:10:21.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:20 smithi161 ceph-mon[128993]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.6 KiB/s rd, 118 KiB/s wr, 8 op/s 2023-07-14T18:10:23.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:22 smithi133 ceph-mon[126384]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.4 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-07-14T18:10:23.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:22 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:23.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:22 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:23.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:22 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:10:23.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:22 smithi161 ceph-mon[128993]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.4 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-07-14T18:10:23.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:22 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:23.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:22 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:23.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:22 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:10:23.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:22 smithi171 ceph-mon[129241]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.4 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-07-14T18:10:23.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:22 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:23.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:22 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:23.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:22 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:10:25.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:24 smithi133 ceph-mon[126384]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T18:10:25.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:24 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:25.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:24 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:25.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:24 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:25.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:24 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:10:25.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:24 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:25.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:24 smithi161 ceph-mon[128993]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T18:10:25.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:24 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:25.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:24 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:25.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:24 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:25.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:24 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:10:25.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:24 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:25.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:24 smithi171 ceph-mon[129241]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T18:10:25.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:24 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:25.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:24 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:25.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:24 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:25.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:24 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:10:25.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:24 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:25.653 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:25.614+0000 7f7026ffd700 1 -- 172.21.15.133:0/3637917794 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7030060a80 con 0x7f701c0a1140 2023-07-14T18:10:25.653 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 3 on host 'smithi133' 2023-07-14T18:10:25.655 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:25.617+0000 7f7038038700 1 -- 172.21.15.133:0/3637917794 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f701c0a1140 msgr2=0x7f701c0a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:10:25.655 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:25.617+0000 7f7038038700 1 --2- 172.21.15.133:0/3637917794 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f701c0a1140 0x7f701c0a35f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f702000b2e0 tx=0x7f7020005fb0 comp rx=0 tx=0).stop 2023-07-14T18:10:25.655 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:25.617+0000 7f7038038700 1 -- 172.21.15.133:0/3637917794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7030100740 msgr2=0x7f7030195a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:10:25.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:25.617+0000 7f7038038700 1 --2- 172.21.15.133:0/3637917794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7030100740 0x7f7030195a00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f702c00eb60 tx=0x7f702c00ee70 comp rx=0 tx=0).stop 2023-07-14T18:10:25.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:25.617+0000 7f7038038700 1 -- 172.21.15.133:0/3637917794 shutdown_connections 2023-07-14T18:10:25.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:25.617+0000 7f7038038700 1 --2- 172.21.15.133:0/3637917794 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f701c0a1140 0x7f701c0a35f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:10:25.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:25.617+0000 7f7038038700 1 --2- 172.21.15.133:0/3637917794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7030105980 0x7f703019cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:10:25.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:25.617+0000 7f7038038700 1 --2- 172.21.15.133:0/3637917794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7030100740 0x7f7030195a00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:10:25.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:25.617+0000 7f7038038700 1 --2- 172.21.15.133:0/3637917794 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7030103060 0x7f7030195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:10:25.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:25.617+0000 7f7038038700 1 -- 172.21.15.133:0/3637917794 >> 172.21.15.133:0/3637917794 conn(0x7f70300fa1e0 msgr2=0x7f70301013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:10:25.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:25.617+0000 7f7038038700 1 -- 172.21.15.133:0/3637917794 shutdown_connections 2023-07-14T18:10:25.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:10:25.617+0000 7f7038038700 1 -- 172.21.15.133:0/3637917794 wait complete. 2023-07-14T18:10:26.060 DEBUG:teuthology.orchestra.run.smithi133:osd.3> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.3.service 2023-07-14T18:10:26.064 INFO:tasks.cephadm:Deploying osd.4 on smithi161 with /dev/vg_nvme/lv_3... 2023-07-14T18:10:26.065 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-14T18:10:26.137 INFO:journalctl@ceph.osd.3.smithi133.stdout:-- Logs begin at Fri 2023-07-14 17:46:17 UTC. -- 2023-07-14T18:10:26.627 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T18:10:26.876 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:26 smithi161 ceph-mon[128993]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T18:10:26.876 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:26 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:26.876 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:26 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:26.876 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:26 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:10:26.876 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:26 smithi161 ceph-mon[128993]: from='osd.3 [v2:172.21.15.133:6810/2152994143,v1:172.21.15.133:6811/2152994143]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-14T18:10:26.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:26 smithi133 ceph-mon[126384]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T18:10:26.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:26 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:26.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:26 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:26.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:26 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:10:26.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:26 smithi133 ceph-mon[126384]: from='osd.3 [v2:172.21.15.133:6810/2152994143,v1:172.21.15.133:6811/2152994143]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-14T18:10:27.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:26 smithi171 ceph-mon[129241]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T18:10:27.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:26 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:27.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:26 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:27.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:26 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:10:27.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:26 smithi171 ceph-mon[129241]: from='osd.3 [v2:172.21.15.133:6810/2152994143,v1:172.21.15.133:6811/2152994143]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-14T18:10:28.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:27 smithi161 ceph-mon[128993]: from='osd.3 [v2:172.21.15.133:6810/2152994143,v1:172.21.15.133:6811/2152994143]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-14T18:10:28.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:27 smithi161 ceph-mon[128993]: osdmap e24: 4 total, 3 up, 4 in 2023-07-14T18:10:28.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:27 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T18:10:28.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:27 smithi161 ceph-mon[128993]: from='osd.3 [v2:172.21.15.133:6810/2152994143,v1:172.21.15.133:6811/2152994143]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T18:10:28.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:27 smithi133 ceph-mon[126384]: from='osd.3 [v2:172.21.15.133:6810/2152994143,v1:172.21.15.133:6811/2152994143]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-14T18:10:28.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:27 smithi133 ceph-mon[126384]: osdmap e24: 4 total, 3 up, 4 in 2023-07-14T18:10:28.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:27 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T18:10:28.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:27 smithi133 ceph-mon[126384]: from='osd.3 [v2:172.21.15.133:6810/2152994143,v1:172.21.15.133:6811/2152994143]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T18:10:28.151 INFO:journalctl@ceph.osd.3.smithi133.stdout:Jul 14 18:10:27 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-3[144787]: 2023-07-14T18:10:27.845+0000 7f415d28b700 -1 osd.3 0 waiting for initial osdmap 2023-07-14T18:10:28.152 INFO:journalctl@ceph.osd.3.smithi133.stdout:Jul 14 18:10:27 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-3[144787]: 2023-07-14T18:10:27.852+0000 7f415707d700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T18:10:28.239 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-14T18:10:28.239 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-14T18:10:28.239 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2023-07-14T18:10:28.239 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2023-07-14T18:10:28.239 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110039 s, 953 MB/s 2023-07-14T18:10:28.240 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2023-07-14T18:10:28.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:27 smithi171 ceph-mon[129241]: from='osd.3 [v2:172.21.15.133:6810/2152994143,v1:172.21.15.133:6811/2152994143]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-14T18:10:28.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:27 smithi171 ceph-mon[129241]: osdmap e24: 4 total, 3 up, 4 in 2023-07-14T18:10:28.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:27 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T18:10:28.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:27 smithi171 ceph-mon[129241]: from='osd.3 [v2:172.21.15.133:6810/2152994143,v1:172.21.15.133:6811/2152994143]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T18:10:28.582 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi161:vg_nvme/lv_3 2023-07-14T18:10:29.139 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T18:10:29.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:28 smithi133 ceph-mon[126384]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T18:10:29.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:28 smithi133 ceph-mon[126384]: from='osd.3 [v2:172.21.15.133:6810/2152994143,v1:172.21.15.133:6811/2152994143]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-07-14T18:10:29.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:28 smithi133 ceph-mon[126384]: osdmap e25: 4 total, 3 up, 4 in 2023-07-14T18:10:29.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:28 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T18:10:29.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:28 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T18:10:29.324 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:28 smithi161 ceph-mon[128993]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T18:10:29.324 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:28 smithi161 ceph-mon[128993]: from='osd.3 [v2:172.21.15.133:6810/2152994143,v1:172.21.15.133:6811/2152994143]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-07-14T18:10:29.325 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:28 smithi161 ceph-mon[128993]: osdmap e25: 4 total, 3 up, 4 in 2023-07-14T18:10:29.325 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:28 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T18:10:29.325 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:28 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T18:10:29.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:28 smithi171 ceph-mon[129241]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T18:10:29.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:28 smithi171 ceph-mon[129241]: from='osd.3 [v2:172.21.15.133:6810/2152994143,v1:172.21.15.133:6811/2152994143]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-07-14T18:10:29.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:28 smithi171 ceph-mon[129241]: osdmap e25: 4 total, 3 up, 4 in 2023-07-14T18:10:29.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T18:10:29.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T18:10:30.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:29 smithi161 ceph-mon[128993]: purged_snaps scrub starts 2023-07-14T18:10:30.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:29 smithi161 ceph-mon[128993]: purged_snaps scrub ok 2023-07-14T18:10:30.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:29 smithi161 ceph-mon[128993]: osd.3 [v2:172.21.15.133:6810/2152994143,v1:172.21.15.133:6811/2152994143] boot 2023-07-14T18:10:30.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:29 smithi161 ceph-mon[128993]: osdmap e26: 4 total, 4 up, 4 in 2023-07-14T18:10:30.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:29 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T18:10:30.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:29 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:30.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:29 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:30.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:29 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:10:30.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:29 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:10:30.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:29 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:30.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:29 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:30.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:29 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:10:30.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:29 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:30.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:29 smithi161 ceph-mon[128993]: osdmap e27: 4 total, 4 up, 4 in 2023-07-14T18:10:30.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:29 smithi133 ceph-mon[126384]: purged_snaps scrub starts 2023-07-14T18:10:30.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:29 smithi133 ceph-mon[126384]: purged_snaps scrub ok 2023-07-14T18:10:30.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:29 smithi133 ceph-mon[126384]: osd.3 [v2:172.21.15.133:6810/2152994143,v1:172.21.15.133:6811/2152994143] boot 2023-07-14T18:10:30.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:29 smithi133 ceph-mon[126384]: osdmap e26: 4 total, 4 up, 4 in 2023-07-14T18:10:30.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:29 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T18:10:30.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:29 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:30.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:29 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:30.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:29 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:10:30.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:29 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:10:30.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:29 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:30.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:29 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:30.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:29 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:10:30.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:29 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:30.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:29 smithi133 ceph-mon[126384]: osdmap e27: 4 total, 4 up, 4 in 2023-07-14T18:10:30.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.277+0000 7f6b939ca700 1 -- 172.21.15.161:0/397324211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b8c0ff820 msgr2=0x7f6b8c1070f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:10:30.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.277+0000 7f6b939ca700 1 --2- 172.21.15.161:0/397324211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b8c0ff820 0x7f6b8c1070f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400b0a0 tx=0x7f6b8400b3b0 comp rx=0 tx=0).stop 2023-07-14T18:10:30.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.277+0000 7f6b939ca700 1 -- 172.21.15.161:0/397324211 shutdown_connections 2023-07-14T18:10:30.315 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.277+0000 7f6b939ca700 1 --2- 172.21.15.161:0/397324211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b8c0ff820 0x7f6b8c1070f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:10:30.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.277+0000 7f6b939ca700 1 --2- 172.21.15.161:0/397324211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b8c100150 0x7f6b8c100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:10:30.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.277+0000 7f6b939ca700 1 --2- 172.21.15.161:0/397324211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b8c0fee90 0x7f6b8c0ff2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:10:30.316 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.277+0000 7f6b939ca700 1 -- 172.21.15.161:0/397324211 >> 172.21.15.161:0/397324211 conn(0x7f6b8c0f9990 msgr2=0x7f6b8c0fbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:10:30.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.277+0000 7f6b939ca700 1 -- 172.21.15.161:0/397324211 shutdown_connections 2023-07-14T18:10:30.317 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.277+0000 7f6b939ca700 1 -- 172.21.15.161:0/397324211 wait complete. 2023-07-14T18:10:30.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.277+0000 7f6b939ca700 1 Processor -- start 2023-07-14T18:10:30.318 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.277+0000 7f6b939ca700 1 -- start start 2023-07-14T18:10:30.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.277+0000 7f6b939ca700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b8c0fee90 0x7f6b8c0fe900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:10:30.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.277+0000 7f6b939ca700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b8c0ff820 0x7f6b8c19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:10:30.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.277+0000 7f6b939ca700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b8c100150 0x7f6b8c19cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:10:30.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.277+0000 7f6b939ca700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c19d760 con 0x7f6b8c0ff820 2023-07-14T18:10:30.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.278+0000 7f6b939ca700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c19d8d0 con 0x7f6b8c0fee90 2023-07-14T18:10:30.319 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.278+0000 7f6b939ca700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c19da90 con 0x7f6b8c100150 2023-07-14T18:10:30.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.278+0000 7f6b91766700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b8c0fee90 0x7f6b8c0fe900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:10:30.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.278+0000 7f6b91766700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b8c0fee90 0x7f6b8c0fe900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54724/0 (socket says 172.21.15.161:54724) 2023-07-14T18:10:30.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.278+0000 7f6b91766700 1 -- 172.21.15.161:0/4155188523 learned_addr learned my addr 172.21.15.161:0/4155188523 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T18:10:30.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.278+0000 7f6b91766700 1 -- 172.21.15.161:0/4155188523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b8c100150 msgr2=0x7f6b8c19cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:10:30.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.278+0000 7f6b91766700 1 --2- 172.21.15.161:0/4155188523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b8c100150 0x7f6b8c19cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:10:30.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.278+0000 7f6b91766700 1 -- 172.21.15.161:0/4155188523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b8c0ff820 msgr2=0x7f6b8c19a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:10:30.320 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.278+0000 7f6b91766700 1 --2- 172.21.15.161:0/4155188523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b8c0ff820 0x7f6b8c19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:10:30.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.278+0000 7f6b91766700 1 -- 172.21.15.161:0/4155188523 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b84009d00 con 0x7f6b8c0fee90 2023-07-14T18:10:30.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.278+0000 7f6b91766700 1 --2- 172.21.15.161:0/4155188523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b8c0fee90 0x7f6b8c0fe900 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800b500 tx=0x7f6b8800b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:10:30.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.278+0000 7f6b827fc700 1 -- 172.21.15.161:0/4155188523 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b88011830 con 0x7f6b8c0fee90 2023-07-14T18:10:30.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.278+0000 7f6b827fc700 1 -- 172.21.15.161:0/4155188523 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b88011e50 con 0x7f6b8c0fee90 2023-07-14T18:10:30.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.278+0000 7f6b939ca700 1 -- 172.21.15.161:0/4155188523 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b8c19dd70 con 0x7f6b8c0fee90 2023-07-14T18:10:30.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.278+0000 7f6b827fc700 1 -- 172.21.15.161:0/4155188523 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8800f2e0 con 0x7f6b8c0fee90 2023-07-14T18:10:30.321 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.278+0000 7f6b939ca700 1 -- 172.21.15.161:0/4155188523 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b8c19e2e0 con 0x7f6b8c0fee90 2023-07-14T18:10:30.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.279+0000 7f6b939ca700 1 -- 172.21.15.161:0/4155188523 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b8c10a520 con 0x7f6b8c0fee90 2023-07-14T18:10:30.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.280+0000 7f6b827fc700 1 -- 172.21.15.161:0/4155188523 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6b88010580 con 0x7f6b8c0fee90 2023-07-14T18:10:30.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.281+0000 7f6b827fc700 1 --2- 172.21.15.161:0/4155188523 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6b780a1250 0x7f6b780a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:10:30.322 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.281+0000 7f6b827fc700 1 -- 172.21.15.161:0/4155188523 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f6b880cb1f0 con 0x7f6b8c0fee90 2023-07-14T18:10:30.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.281+0000 7f6b90f65700 1 --2- 172.21.15.161:0/4155188523 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6b780a1250 0x7f6b780a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:10:30.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.282+0000 7f6b90f65700 1 --2- 172.21.15.161:0/4155188523 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6b780a1250 0x7f6b780a3700 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0fee60 tx=0x7f6b7c006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:10:30.323 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.286+0000 7f6b827fc700 1 -- 172.21.15.161:0/4155188523 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f6b88095aa0 con 0x7f6b8c0fee90 2023-07-14T18:10:30.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:29 smithi171 ceph-mon[129241]: purged_snaps scrub starts 2023-07-14T18:10:30.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:29 smithi171 ceph-mon[129241]: purged_snaps scrub ok 2023-07-14T18:10:30.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:29 smithi171 ceph-mon[129241]: osd.3 [v2:172.21.15.133:6810/2152994143,v1:172.21.15.133:6811/2152994143] boot 2023-07-14T18:10:30.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:29 smithi171 ceph-mon[129241]: osdmap e26: 4 total, 4 up, 4 in 2023-07-14T18:10:30.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:29 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T18:10:30.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:29 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:30.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:29 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:30.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:29 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:10:30.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:29 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:10:30.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:29 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:30.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:29 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:30.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:29 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:10:30.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:29 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:30.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:29 smithi171 ceph-mon[129241]: osdmap e27: 4 total, 4 up, 4 in 2023-07-14T18:10:30.517 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:30.479+0000 7f6b939ca700 1 -- 172.21.15.161:0/4155188523 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b8c060a80 con 0x7f6b780a1250 2023-07-14T18:10:31.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:30 smithi133 ceph-mon[126384]: Detected new or changed devices on smithi133 2023-07-14T18:10:31.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:30 smithi133 ceph-mon[126384]: Adjusting osd_memory_target on smithi133 to 8633M 2023-07-14T18:10:31.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:30 smithi133 ceph-mon[126384]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T18:10:31.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:30 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:10:31.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:30 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:10:31.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:30 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:31.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:30 smithi133 ceph-mon[126384]: osdmap e28: 4 total, 4 up, 4 in 2023-07-14T18:10:31.299 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:30 smithi161 ceph-mon[128993]: Detected new or changed devices on smithi133 2023-07-14T18:10:31.299 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:30 smithi161 ceph-mon[128993]: Adjusting osd_memory_target on smithi133 to 8633M 2023-07-14T18:10:31.300 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:30 smithi161 ceph-mon[128993]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T18:10:31.300 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:30 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:10:31.300 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:30 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:10:31.300 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:30 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:31.300 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:30 smithi161 ceph-mon[128993]: osdmap e28: 4 total, 4 up, 4 in 2023-07-14T18:10:31.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:30 smithi171 ceph-mon[129241]: Detected new or changed devices on smithi133 2023-07-14T18:10:31.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:30 smithi171 ceph-mon[129241]: Adjusting osd_memory_target on smithi133 to 8633M 2023-07-14T18:10:31.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:30 smithi171 ceph-mon[129241]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T18:10:31.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:30 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:10:31.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:30 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:10:31.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:30 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:31.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:30 smithi171 ceph-mon[129241]: osdmap e28: 4 total, 4 up, 4 in 2023-07-14T18:10:32.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:31 smithi133 ceph-mon[126384]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:10:32.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:31 smithi133 ceph-mon[126384]: osdmap e29: 4 total, 4 up, 4 in 2023-07-14T18:10:32.154 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:31 smithi161 ceph-mon[128993]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:10:32.154 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:31 smithi161 ceph-mon[128993]: osdmap e29: 4 total, 4 up, 4 in 2023-07-14T18:10:32.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:31 smithi171 ceph-mon[129241]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:10:32.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:31 smithi171 ceph-mon[129241]: osdmap e29: 4 total, 4 up, 4 in 2023-07-14T18:10:33.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:32 smithi133 ceph-mon[126384]: pgmap v100: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T18:10:33.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:32 smithi133 ceph-mon[126384]: from='client.? 172.21.15.161:0/1984440653' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4601a1c2-254b-4c4d-9b15-8fc952b3d2f8"}]: dispatch 2023-07-14T18:10:33.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:32 smithi133 ceph-mon[126384]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4601a1c2-254b-4c4d-9b15-8fc952b3d2f8"}]: dispatch 2023-07-14T18:10:33.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:32 smithi133 ceph-mon[126384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4601a1c2-254b-4c4d-9b15-8fc952b3d2f8"}]': finished 2023-07-14T18:10:33.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:32 smithi133 ceph-mon[126384]: osdmap e30: 5 total, 4 up, 5 in 2023-07-14T18:10:33.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:32 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T18:10:33.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:32 smithi161 ceph-mon[128993]: pgmap v100: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T18:10:33.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:32 smithi161 ceph-mon[128993]: from='client.? 172.21.15.161:0/1984440653' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4601a1c2-254b-4c4d-9b15-8fc952b3d2f8"}]: dispatch 2023-07-14T18:10:33.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:32 smithi161 ceph-mon[128993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4601a1c2-254b-4c4d-9b15-8fc952b3d2f8"}]: dispatch 2023-07-14T18:10:33.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:32 smithi161 ceph-mon[128993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4601a1c2-254b-4c4d-9b15-8fc952b3d2f8"}]': finished 2023-07-14T18:10:33.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:32 smithi161 ceph-mon[128993]: osdmap e30: 5 total, 4 up, 5 in 2023-07-14T18:10:33.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:32 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T18:10:33.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:32 smithi171 ceph-mon[129241]: pgmap v100: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T18:10:33.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:32 smithi171 ceph-mon[129241]: from='client.? 172.21.15.161:0/1984440653' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4601a1c2-254b-4c4d-9b15-8fc952b3d2f8"}]: dispatch 2023-07-14T18:10:33.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:32 smithi171 ceph-mon[129241]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4601a1c2-254b-4c4d-9b15-8fc952b3d2f8"}]: dispatch 2023-07-14T18:10:33.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:32 smithi171 ceph-mon[129241]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4601a1c2-254b-4c4d-9b15-8fc952b3d2f8"}]': finished 2023-07-14T18:10:33.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:32 smithi171 ceph-mon[129241]: osdmap e30: 5 total, 4 up, 5 in 2023-07-14T18:10:33.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:32 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T18:10:34.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:33 smithi133 ceph-mon[126384]: from='client.? 172.21.15.161:0/3963670635' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:10:34.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:33 smithi161 ceph-mon[128993]: from='client.? 172.21.15.161:0/3963670635' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:10:34.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:33 smithi171 ceph-mon[129241]: from='client.? 172.21.15.161:0/3963670635' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:10:35.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:34 smithi133 ceph-mon[126384]: pgmap v103: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T18:10:35.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:34 smithi161 ceph-mon[128993]: pgmap v103: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T18:10:35.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:34 smithi171 ceph-mon[129241]: pgmap v103: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T18:10:37.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:36 smithi161 ceph-mon[128993]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2023-07-14T18:10:37.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:36 smithi171 ceph-mon[129241]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2023-07-14T18:10:37.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:36 smithi133 ceph-mon[126384]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2023-07-14T18:10:39.338 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:38 smithi161 ceph-mon[128993]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2023-07-14T18:10:39.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:38 smithi171 ceph-mon[129241]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2023-07-14T18:10:39.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:38 smithi133 ceph-mon[126384]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2023-07-14T18:10:40.217 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:39 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-14T18:10:40.217 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:39 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:40.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:39 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-14T18:10:40.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:39 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:40.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:39 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-14T18:10:40.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:39 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:41.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:40 smithi161 ceph-mon[128993]: Deploying daemon osd.4 on smithi161 2023-07-14T18:10:41.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:40 smithi161 ceph-mon[128993]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T18:10:41.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:40 smithi171 ceph-mon[129241]: Deploying daemon osd.4 on smithi161 2023-07-14T18:10:41.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:40 smithi171 ceph-mon[129241]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T18:10:41.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:40 smithi133 ceph-mon[126384]: Deploying daemon osd.4 on smithi161 2023-07-14T18:10:41.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:40 smithi133 ceph-mon[126384]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T18:10:43.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:42 smithi161 ceph-mon[128993]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T18:10:43.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:42 smithi171 ceph-mon[129241]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T18:10:43.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:42 smithi133 ceph-mon[126384]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T18:10:44.816 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:44 smithi161 ceph-mon[128993]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T18:10:44.816 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:44 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:44.816 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:44 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:44.817 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:44 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:10:45.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:44 smithi171 ceph-mon[129241]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T18:10:45.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:44 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:45.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:44 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:45.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:44 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:10:45.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:44 smithi133 ceph-mon[126384]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T18:10:45.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:44 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:45.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:44 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:45.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:44 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:10:46.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:46.646+0000 7f6b827fc700 1 -- 172.21.15.161:0/4155188523 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6b8c060a80 con 0x7f6b780a1250 2023-07-14T18:10:46.685 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 4 on host 'smithi161' 2023-07-14T18:10:46.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:46.648+0000 7f6b939ca700 1 -- 172.21.15.161:0/4155188523 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6b780a1250 msgr2=0x7f6b780a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:10:46.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:46.648+0000 7f6b939ca700 1 --2- 172.21.15.161:0/4155188523 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6b780a1250 0x7f6b780a3700 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0fee60 tx=0x7f6b7c006d20 comp rx=0 tx=0).stop 2023-07-14T18:10:46.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:46.648+0000 7f6b939ca700 1 -- 172.21.15.161:0/4155188523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b8c0fee90 msgr2=0x7f6b8c0fe900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:10:46.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:46.648+0000 7f6b939ca700 1 --2- 172.21.15.161:0/4155188523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b8c0fee90 0x7f6b8c0fe900 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800b500 tx=0x7f6b8800b8c0 comp rx=0 tx=0).stop 2023-07-14T18:10:46.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:46.648+0000 7f6b939ca700 1 -- 172.21.15.161:0/4155188523 shutdown_connections 2023-07-14T18:10:46.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:46.648+0000 7f6b939ca700 1 --2- 172.21.15.161:0/4155188523 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6b780a1250 0x7f6b780a3700 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:10:46.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:46.648+0000 7f6b939ca700 1 --2- 172.21.15.161:0/4155188523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b8c0fee90 0x7f6b8c0fe900 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:10:46.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:46.648+0000 7f6b939ca700 1 --2- 172.21.15.161:0/4155188523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b8c100150 0x7f6b8c19cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:10:46.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:46.648+0000 7f6b939ca700 1 --2- 172.21.15.161:0/4155188523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b8c0ff820 0x7f6b8c19a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:10:46.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:46.648+0000 7f6b939ca700 1 -- 172.21.15.161:0/4155188523 >> 172.21.15.161:0/4155188523 conn(0x7f6b8c0f9990 msgr2=0x7f6b8c0fbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:10:46.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:46.649+0000 7f6b939ca700 1 -- 172.21.15.161:0/4155188523 shutdown_connections 2023-07-14T18:10:46.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:10:46.649+0000 7f6b939ca700 1 -- 172.21.15.161:0/4155188523 wait complete. 2023-07-14T18:10:47.083 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:46 smithi161 ceph-mon[128993]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T18:10:47.083 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:46 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:47.083 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:46 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:47.084 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:46 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:47.084 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:46 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:10:47.084 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:46 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:47.084 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:46 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:47.084 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:46 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:10:47.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:46 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:47.118 DEBUG:teuthology.orchestra.run.smithi161:osd.4> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.4.service 2023-07-14T18:10:47.120 INFO:tasks.cephadm:Deploying osd.5 on smithi171 with /dev/vg_nvme/lv_3... 2023-07-14T18:10:47.120 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-14T18:10:47.162 INFO:journalctl@ceph.osd.4.smithi161.stdout:-- Logs begin at Fri 2023-07-14 17:41:43 UTC. -- 2023-07-14T18:10:47.165 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:46 smithi171 ceph-mon[129241]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T18:10:47.165 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:46 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:47.165 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:46 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:47.166 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:46 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:47.166 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:46 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:10:47.166 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:46 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:47.166 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:46 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:47.166 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:46 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:10:47.166 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:46 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:47.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:46 smithi133 ceph-mon[126384]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T18:10:47.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:46 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:47.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:46 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:47.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:46 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:47.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:46 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:10:47.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:46 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:47.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:46 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:47.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:46 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:10:47.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:46 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:47.692 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T18:10:47.899 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jul 14 18:10:47 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-4[139499]: 2023-07-14T18:10:47.601+0000 7f4f1e681640 -1 osd.4 0 log_to_monitors true 2023-07-14T18:10:48.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:47 smithi171 ceph-mon[129241]: from='osd.4 [v2:172.21.15.161:6808/47213291,v1:172.21.15.161:6809/47213291]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T18:10:48.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:47 smithi171 ceph-mon[129241]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T18:10:48.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:47 smithi161 ceph-mon[128993]: from='osd.4 [v2:172.21.15.161:6808/47213291,v1:172.21.15.161:6809/47213291]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T18:10:48.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:47 smithi161 ceph-mon[128993]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T18:10:48.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:47 smithi133 ceph-mon[126384]: from='osd.4 [v2:172.21.15.161:6808/47213291,v1:172.21.15.161:6809/47213291]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T18:10:48.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:47 smithi133 ceph-mon[126384]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T18:10:49.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:48 smithi171 ceph-mon[129241]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T18:10:49.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:48 smithi171 ceph-mon[129241]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-14T18:10:49.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:48 smithi171 ceph-mon[129241]: osdmap e31: 5 total, 4 up, 5 in 2023-07-14T18:10:49.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T18:10:49.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:48 smithi171 ceph-mon[129241]: from='osd.4 [v2:172.21.15.161:6808/47213291,v1:172.21.15.161:6809/47213291]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T18:10:49.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:48 smithi171 ceph-mon[129241]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T18:10:49.289 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-14T18:10:49.290 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-14T18:10:49.290 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2023-07-14T18:10:49.290 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2023-07-14T18:10:49.290 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111316 s, 942 MB/s 2023-07-14T18:10:49.290 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2023-07-14T18:10:49.303 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jul 14 18:10:48 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-4[139499]: 2023-07-14T18:10:48.896+0000 7f4f13cf9700 -1 osd.4 0 waiting for initial osdmap 2023-07-14T18:10:49.303 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jul 14 18:10:48 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-4[139499]: 2023-07-14T18:10:48.902+0000 7f4f0faef700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T18:10:49.304 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:48 smithi161 ceph-mon[128993]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T18:10:49.304 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:48 smithi161 ceph-mon[128993]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-14T18:10:49.304 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:48 smithi161 ceph-mon[128993]: osdmap e31: 5 total, 4 up, 5 in 2023-07-14T18:10:49.304 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T18:10:49.304 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:48 smithi161 ceph-mon[128993]: from='osd.4 [v2:172.21.15.161:6808/47213291,v1:172.21.15.161:6809/47213291]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T18:10:49.305 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:48 smithi161 ceph-mon[128993]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T18:10:49.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:48 smithi133 ceph-mon[126384]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T18:10:49.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:48 smithi133 ceph-mon[126384]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-14T18:10:49.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:48 smithi133 ceph-mon[126384]: osdmap e31: 5 total, 4 up, 5 in 2023-07-14T18:10:49.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T18:10:49.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:48 smithi133 ceph-mon[126384]: from='osd.4 [v2:172.21.15.161:6808/47213291,v1:172.21.15.161:6809/47213291]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T18:10:49.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:48 smithi133 ceph-mon[126384]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T18:10:49.745 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi171:vg_nvme/lv_3 2023-07-14T18:10:50.298 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T18:10:50.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:49 smithi161 ceph-mon[128993]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T18:10:50.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:49 smithi161 ceph-mon[128993]: osdmap e32: 5 total, 4 up, 5 in 2023-07-14T18:10:50.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:49 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T18:10:50.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:49 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T18:10:50.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:49 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:50.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:49 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:50.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:49 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:10:50.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:49 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:10:50.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:49 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:50.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:49 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:50.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:49 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:10:50.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:49 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:50.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:49 smithi171 ceph-mon[129241]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T18:10:50.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:49 smithi171 ceph-mon[129241]: osdmap e32: 5 total, 4 up, 5 in 2023-07-14T18:10:50.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:49 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T18:10:50.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:49 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T18:10:50.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:49 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:50.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:49 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:50.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:49 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:10:50.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:49 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:10:50.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:49 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:50.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:49 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:50.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:49 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:10:50.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:49 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:50.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:49 smithi133 ceph-mon[126384]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T18:10:50.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:49 smithi133 ceph-mon[126384]: osdmap e32: 5 total, 4 up, 5 in 2023-07-14T18:10:50.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:49 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T18:10:50.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:49 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T18:10:50.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:49 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:50.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:49 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:50.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:49 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:10:50.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:49 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:10:50.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:49 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:50.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:49 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:50.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:49 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:10:50.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:49 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:10:51.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:50 smithi161 ceph-mon[128993]: purged_snaps scrub starts 2023-07-14T18:10:51.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:50 smithi161 ceph-mon[128993]: purged_snaps scrub ok 2023-07-14T18:10:51.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:50 smithi161 ceph-mon[128993]: Detected new or changed devices on smithi161 2023-07-14T18:10:51.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:50 smithi161 ceph-mon[128993]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T18:10:51.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:50 smithi161 ceph-mon[128993]: Adjusting osd_memory_target on smithi161 to 8584M 2023-07-14T18:10:51.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:50 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T18:10:51.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:50 smithi161 ceph-mon[128993]: osd.4 [v2:172.21.15.161:6808/47213291,v1:172.21.15.161:6809/47213291] boot 2023-07-14T18:10:51.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:50 smithi161 ceph-mon[128993]: osdmap e33: 5 total, 5 up, 5 in 2023-07-14T18:10:51.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:50 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T18:10:51.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:50 smithi171 ceph-mon[129241]: purged_snaps scrub starts 2023-07-14T18:10:51.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:50 smithi171 ceph-mon[129241]: purged_snaps scrub ok 2023-07-14T18:10:51.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:50 smithi171 ceph-mon[129241]: Detected new or changed devices on smithi161 2023-07-14T18:10:51.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:50 smithi171 ceph-mon[129241]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T18:10:51.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:50 smithi171 ceph-mon[129241]: Adjusting osd_memory_target on smithi161 to 8584M 2023-07-14T18:10:51.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:50 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T18:10:51.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:50 smithi171 ceph-mon[129241]: osd.4 [v2:172.21.15.161:6808/47213291,v1:172.21.15.161:6809/47213291] boot 2023-07-14T18:10:51.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:50 smithi171 ceph-mon[129241]: osdmap e33: 5 total, 5 up, 5 in 2023-07-14T18:10:51.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:50 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T18:10:51.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:50 smithi133 ceph-mon[126384]: purged_snaps scrub starts 2023-07-14T18:10:51.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:50 smithi133 ceph-mon[126384]: purged_snaps scrub ok 2023-07-14T18:10:51.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:50 smithi133 ceph-mon[126384]: Detected new or changed devices on smithi161 2023-07-14T18:10:51.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:50 smithi133 ceph-mon[126384]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T18:10:51.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:50 smithi133 ceph-mon[126384]: Adjusting osd_memory_target on smithi161 to 8584M 2023-07-14T18:10:51.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:50 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T18:10:51.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:50 smithi133 ceph-mon[126384]: osd.4 [v2:172.21.15.161:6808/47213291,v1:172.21.15.161:6809/47213291] boot 2023-07-14T18:10:51.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:50 smithi133 ceph-mon[126384]: osdmap e33: 5 total, 5 up, 5 in 2023-07-14T18:10:51.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:50 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T18:10:51.595 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.556+0000 7f84bdd50700 1 -- 172.21.15.171:0/3712850693 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84b80febb0 msgr2=0x7f84b80fefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:10:51.595 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.556+0000 7f84bdd50700 1 --2- 172.21.15.171:0/3712850693 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84b80febb0 0x7f84b80fefc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009990 tx=0x7f84a8009ca0 comp rx=0 tx=0).stop 2023-07-14T18:10:51.596 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.556+0000 7f84bdd50700 1 -- 172.21.15.171:0/3712850693 shutdown_connections 2023-07-14T18:10:51.596 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.556+0000 7f84bdd50700 1 --2- 172.21.15.171:0/3712850693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84b8100740 0x7f84b8107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:10:51.596 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.556+0000 7f84bdd50700 1 --2- 172.21.15.171:0/3712850693 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84b80febb0 0x7f84b80fefc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:10:51.596 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.556+0000 7f84bdd50700 1 --2- 172.21.15.171:0/3712850693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84b80ffdb0 0x7f84b8100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:10:51.596 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.556+0000 7f84bdd50700 1 -- 172.21.15.171:0/3712850693 >> 172.21.15.171:0/3712850693 conn(0x7f84b80fa1e0 msgr2=0x7f84b80fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:10:51.596 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.556+0000 7f84bdd50700 1 -- 172.21.15.171:0/3712850693 shutdown_connections 2023-07-14T18:10:51.597 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.556+0000 7f84bdd50700 1 -- 172.21.15.171:0/3712850693 wait complete. 2023-07-14T18:10:51.598 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.557+0000 7f84bdd50700 1 Processor -- start 2023-07-14T18:10:51.598 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.557+0000 7f84bdd50700 1 -- start start 2023-07-14T18:10:51.599 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.557+0000 7f84bdd50700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84b80febb0 0x7f84b8068c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:10:51.599 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.557+0000 7f84bdd50700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84b80ffdb0 0x7f84b819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:10:51.599 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.557+0000 7f84bdd50700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84b8100740 0x7f84b819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:10:51.599 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.557+0000 7f84bdd50700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b819f760 con 0x7f84b80febb0 2023-07-14T18:10:51.599 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.557+0000 7f84bdd50700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b819f8d0 con 0x7f84b8100740 2023-07-14T18:10:51.599 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.557+0000 7f84bdd50700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b819fa90 con 0x7f84b80ffdb0 2023-07-14T18:10:51.599 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.557+0000 7f84b6ffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84b80ffdb0 0x7f84b819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:10:51.600 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.557+0000 7f84b6ffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84b80ffdb0 0x7f84b819cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:42850/0 (socket says 172.21.15.171:42850) 2023-07-14T18:10:51.600 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.557+0000 7f84b6ffd700 1 -- 172.21.15.171:0/121616832 learned_addr learned my addr 172.21.15.171:0/121616832 (peer_addr_for_me v2:172.21.15.171:0/0) 2023-07-14T18:10:51.600 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.557+0000 7f84b7fff700 1 --2- 172.21.15.171:0/121616832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84b8100740 0x7f84b819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:10:51.600 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.557+0000 7f84b6ffd700 1 -- 172.21.15.171:0/121616832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84b8100740 msgr2=0x7f84b819f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:10:51.600 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.557+0000 7f84b6ffd700 1 --2- 172.21.15.171:0/121616832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84b8100740 0x7f84b819f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:10:51.600 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.557+0000 7f84b6ffd700 1 -- 172.21.15.171:0/121616832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84b80febb0 msgr2=0x7f84b8068c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:10:51.600 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.557+0000 7f84b6ffd700 1 --2- 172.21.15.171:0/121616832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84b80febb0 0x7f84b8068c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:10:51.601 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.557+0000 7f84b6ffd700 1 -- 172.21.15.171:0/121616832 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a80095f0 con 0x7f84b80ffdb0 2023-07-14T18:10:51.601 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.558+0000 7f84b7fff700 1 --2- 172.21.15.171:0/121616832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84b8100740 0x7f84b819f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:10:51.601 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.558+0000 7f84b6ffd700 1 --2- 172.21.15.171:0/121616832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84b80ffdb0 0x7f84b819cb60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f84a000d980 tx=0x7f84a000dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:10:51.601 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.558+0000 7f84b4ff9700 1 -- 172.21.15.171:0/121616832 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a000bb60 con 0x7f84b80ffdb0 2023-07-14T18:10:51.601 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.558+0000 7f84bdd50700 1 -- 172.21.15.171:0/121616832 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b819fd70 con 0x7f84b80ffdb0 2023-07-14T18:10:51.601 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.558+0000 7f84b4ff9700 1 -- 172.21.15.171:0/121616832 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84a0007640 con 0x7f84b80ffdb0 2023-07-14T18:10:51.601 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.558+0000 7f84b4ff9700 1 -- 172.21.15.171:0/121616832 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a0004e90 con 0x7f84b80ffdb0 2023-07-14T18:10:51.601 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.558+0000 7f84bdd50700 1 -- 172.21.15.171:0/121616832 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b81a0260 con 0x7f84b80ffdb0 2023-07-14T18:10:51.602 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.559+0000 7f84bdd50700 1 -- 172.21.15.171:0/121616832 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b810a370 con 0x7f84b80ffdb0 2023-07-14T18:10:51.602 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.560+0000 7f84b4ff9700 1 -- 172.21.15.171:0/121616832 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f84a000bcc0 con 0x7f84b80ffdb0 2023-07-14T18:10:51.603 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.560+0000 7f84b4ff9700 1 --2- 172.21.15.171:0/121616832 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f84a40a1150 0x7f84a40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:10:51.603 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.561+0000 7f84b4ff9700 1 -- 172.21.15.171:0/121616832 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f84a00cb850 con 0x7f84b80ffdb0 2023-07-14T18:10:51.603 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.561+0000 7f84b77fe700 1 --2- 172.21.15.171:0/121616832 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f84a40a1150 0x7f84a40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:10:51.603 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.561+0000 7f84b77fe700 1 --2- 172.21.15.171:0/121616832 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f84a40a1150 0x7f84a40a3600 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f84a80098e0 tx=0x7f84a8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:10:51.604 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.565+0000 7f84b4ff9700 1 -- 172.21.15.171:0/121616832 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f84a0099a00 con 0x7f84b80ffdb0 2023-07-14T18:10:51.794 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:10:51.755+0000 7f84bdd50700 1 -- 172.21.15.171:0/121616832 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f84b8060a80 con 0x7f84a40a1150 2023-07-14T18:10:52.678 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:52 smithi171 ceph-mon[129241]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:10:52.678 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:52 smithi171 ceph-mon[129241]: osdmap e34: 5 total, 5 up, 5 in 2023-07-14T18:10:52.678 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:52 smithi171 ceph-mon[129241]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:10:52.678 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:52 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:10:52.678 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:52 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:10:52.679 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:52 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:52.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:52 smithi161 ceph-mon[128993]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:10:52.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:52 smithi161 ceph-mon[128993]: osdmap e34: 5 total, 5 up, 5 in 2023-07-14T18:10:52.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:52 smithi161 ceph-mon[128993]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:10:52.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:52 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:10:52.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:52 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:10:52.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:52 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:52.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:52 smithi133 ceph-mon[126384]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:10:52.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:52 smithi133 ceph-mon[126384]: osdmap e34: 5 total, 5 up, 5 in 2023-07-14T18:10:52.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:52 smithi133 ceph-mon[126384]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:10:52.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:52 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:10:52.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:52 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:10:52.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:52 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:10:54.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:54 smithi161 ceph-mon[128993]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:10:54.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:54 smithi161 ceph-mon[128993]: from='client.? 172.21.15.171:0/1900933264' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cca180a-9140-4e49-835b-86842dae0574"}]: dispatch 2023-07-14T18:10:54.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:54 smithi161 ceph-mon[128993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cca180a-9140-4e49-835b-86842dae0574"}]: dispatch 2023-07-14T18:10:54.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:54 smithi161 ceph-mon[128993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6cca180a-9140-4e49-835b-86842dae0574"}]': finished 2023-07-14T18:10:54.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:54 smithi161 ceph-mon[128993]: osdmap e35: 6 total, 5 up, 6 in 2023-07-14T18:10:54.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:54 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T18:10:54.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:54 smithi171 ceph-mon[129241]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:10:54.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:54 smithi171 ceph-mon[129241]: from='client.? 172.21.15.171:0/1900933264' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cca180a-9140-4e49-835b-86842dae0574"}]: dispatch 2023-07-14T18:10:54.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:54 smithi171 ceph-mon[129241]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cca180a-9140-4e49-835b-86842dae0574"}]: dispatch 2023-07-14T18:10:54.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:54 smithi171 ceph-mon[129241]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6cca180a-9140-4e49-835b-86842dae0574"}]': finished 2023-07-14T18:10:54.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:54 smithi171 ceph-mon[129241]: osdmap e35: 6 total, 5 up, 6 in 2023-07-14T18:10:54.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:54 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T18:10:54.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:54 smithi133 ceph-mon[126384]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:10:54.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:54 smithi133 ceph-mon[126384]: from='client.? 172.21.15.171:0/1900933264' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cca180a-9140-4e49-835b-86842dae0574"}]: dispatch 2023-07-14T18:10:54.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:54 smithi133 ceph-mon[126384]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cca180a-9140-4e49-835b-86842dae0574"}]: dispatch 2023-07-14T18:10:54.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:54 smithi133 ceph-mon[126384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6cca180a-9140-4e49-835b-86842dae0574"}]': finished 2023-07-14T18:10:54.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:54 smithi133 ceph-mon[126384]: osdmap e35: 6 total, 5 up, 6 in 2023-07-14T18:10:54.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:54 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T18:10:55.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:55 smithi161 ceph-mon[128993]: from='client.? 172.21.15.171:0/2199851275' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:10:55.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:55 smithi171 ceph-mon[129241]: from='client.? 172.21.15.171:0/2199851275' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:10:55.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:55 smithi133 ceph-mon[126384]: from='client.? 172.21.15.171:0/2199851275' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:10:57.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:56 smithi171 ceph-mon[129241]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:10:57.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:56 smithi133 ceph-mon[126384]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:10:57.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:56 smithi161 ceph-mon[128993]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:10:59.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:10:58 smithi171 ceph-mon[129241]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:10:59.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:10:58 smithi133 ceph-mon[126384]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:10:59.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:10:58 smithi161 ceph-mon[128993]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:11:01.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:00 smithi171 ceph-mon[129241]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:11:01.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:00 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-14T18:11:01.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:00 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:01.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:00 smithi133 ceph-mon[126384]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:11:01.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:00 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-14T18:11:01.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:00 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:01.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:00 smithi161 ceph-mon[128993]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:11:01.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:00 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-14T18:11:01.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:00 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:02.118 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:01 smithi171 ceph-mon[129241]: Deploying daemon osd.5 on smithi171 2023-07-14T18:11:02.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:01 smithi133 ceph-mon[126384]: Deploying daemon osd.5 on smithi171 2023-07-14T18:11:02.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:01 smithi161 ceph-mon[128993]: Deploying daemon osd.5 on smithi171 2023-07-14T18:11:03.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:02 smithi171 ceph-mon[129241]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:11:03.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:02 smithi133 ceph-mon[126384]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:11:03.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:02 smithi161 ceph-mon[128993]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:11:05.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:04 smithi171 ceph-mon[129241]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:11:05.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:04 smithi133 ceph-mon[126384]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:11:05.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:04 smithi161 ceph-mon[128993]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:11:06.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:06 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:06.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:06 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:06.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:06 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:11:06.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:06 smithi171 ceph-mon[129241]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:11:06.595 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:06 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:06.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:06 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:06.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:06 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:11:06.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:06 smithi161 ceph-mon[128993]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:11:06.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:06 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:06.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:06 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:06.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:06 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:11:06.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:06 smithi133 ceph-mon[126384]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:11:08.207 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:11:08.168+0000 7f84b4ff9700 1 -- 172.21.15.171:0/121616832 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f84b8060a80 con 0x7f84a40a1150 2023-07-14T18:11:08.208 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 5 on host 'smithi171' 2023-07-14T18:11:08.209 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:11:08.170+0000 7f84bdd50700 1 -- 172.21.15.171:0/121616832 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f84a40a1150 msgr2=0x7f84a40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:11:08.210 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:11:08.170+0000 7f84bdd50700 1 --2- 172.21.15.171:0/121616832 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f84a40a1150 0x7f84a40a3600 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f84a80098e0 tx=0x7f84a8005fb0 comp rx=0 tx=0).stop 2023-07-14T18:11:08.210 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:11:08.170+0000 7f84bdd50700 1 -- 172.21.15.171:0/121616832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84b80ffdb0 msgr2=0x7f84b819cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:11:08.210 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:11:08.170+0000 7f84bdd50700 1 --2- 172.21.15.171:0/121616832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84b80ffdb0 0x7f84b819cb60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f84a000d980 tx=0x7f84a000dd40 comp rx=0 tx=0).stop 2023-07-14T18:11:08.210 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:11:08.170+0000 7f84bdd50700 1 -- 172.21.15.171:0/121616832 shutdown_connections 2023-07-14T18:11:08.210 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:11:08.170+0000 7f84bdd50700 1 --2- 172.21.15.171:0/121616832 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f84a40a1150 0x7f84a40a3600 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:11:08.211 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:11:08.170+0000 7f84bdd50700 1 --2- 172.21.15.171:0/121616832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84b8100740 0x7f84b819f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:11:08.211 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:11:08.170+0000 7f84bdd50700 1 --2- 172.21.15.171:0/121616832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84b80febb0 0x7f84b8068c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:11:08.211 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:11:08.170+0000 7f84bdd50700 1 --2- 172.21.15.171:0/121616832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84b80ffdb0 0x7f84b819cb60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:11:08.211 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:11:08.170+0000 7f84bdd50700 1 -- 172.21.15.171:0/121616832 >> 172.21.15.171:0/121616832 conn(0x7f84b80fa1e0 msgr2=0x7f84b80fc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:11:08.212 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:11:08.171+0000 7f84bdd50700 1 -- 172.21.15.171:0/121616832 shutdown_connections 2023-07-14T18:11:08.212 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:11:08.171+0000 7f84bdd50700 1 -- 172.21.15.171:0/121616832 wait complete. 2023-07-14T18:11:08.578 DEBUG:teuthology.orchestra.run.smithi171:osd.5> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.5.service 2023-07-14T18:11:08.580 INFO:tasks.cephadm:Deploying osd.6 on smithi133 with /dev/vg_nvme/lv_2... 2023-07-14T18:11:08.581 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-14T18:11:08.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:08 smithi171 ceph-mon[129241]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:11:08.604 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:08 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:08.604 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:08 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:08.604 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:08 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:08.604 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:08 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:11:08.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:08 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:08.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:08 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:08.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:08 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:08.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:08 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:11:08.609 INFO:journalctl@ceph.osd.5.smithi171.stdout:-- Logs begin at Fri 2023-07-14 17:41:48 UTC. -- 2023-07-14T18:11:08.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:08 smithi161 ceph-mon[128993]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:11:08.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:08 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:08.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:08 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:08.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:08 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:08.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:08 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:11:08.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:08 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:08.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:08 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:08.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:08 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:08.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:08 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:11:08.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:08 smithi133 ceph-mon[126384]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:11:08.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:08 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:08.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:08 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:08.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:08 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:08.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:08 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:11:08.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:08 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:08.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:08 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:08.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:08 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:08.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:08 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:11:09.325 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:11:09.360 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:11:09 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:11:09.038+0000 7f76bdeda640 -1 osd.5 0 log_to_monitors true 2023-07-14T18:11:09.681 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:09 smithi171 ceph-mon[129241]: from='osd.5 [v2:172.21.15.171:6808/777556751,v1:172.21.15.171:6809/777556751]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-14T18:11:09.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:09 smithi171 ceph-mon[129241]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-14T18:11:09.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:09 smithi161 ceph-mon[128993]: from='osd.5 [v2:172.21.15.171:6808/777556751,v1:172.21.15.171:6809/777556751]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-14T18:11:09.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:09 smithi161 ceph-mon[128993]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-14T18:11:09.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:09 smithi133 ceph-mon[126384]: from='osd.5 [v2:172.21.15.171:6808/777556751,v1:172.21.15.171:6809/777556751]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-14T18:11:09.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:09 smithi133 ceph-mon[126384]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-14T18:11:10.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:10 smithi133 ceph-mon[126384]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:11:10.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:10 smithi133 ceph-mon[126384]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-14T18:11:10.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:10 smithi133 ceph-mon[126384]: osdmap e36: 6 total, 5 up, 6 in 2023-07-14T18:11:10.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:10 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T18:11:10.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:10 smithi133 ceph-mon[126384]: from='osd.5 [v2:172.21.15.171:6808/777556751,v1:172.21.15.171:6809/777556751]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2023-07-14T18:11:10.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:10 smithi133 ceph-mon[126384]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2023-07-14T18:11:10.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:10 smithi161 ceph-mon[128993]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:11:10.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:10 smithi161 ceph-mon[128993]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-14T18:11:10.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:10 smithi161 ceph-mon[128993]: osdmap e36: 6 total, 5 up, 6 in 2023-07-14T18:11:10.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:10 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T18:11:10.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:10 smithi161 ceph-mon[128993]: from='osd.5 [v2:172.21.15.171:6808/777556751,v1:172.21.15.171:6809/777556751]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2023-07-14T18:11:10.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:10 smithi161 ceph-mon[128993]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2023-07-14T18:11:10.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:10 smithi171 ceph-mon[129241]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:11:10.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:10 smithi171 ceph-mon[129241]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-14T18:11:10.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:10 smithi171 ceph-mon[129241]: osdmap e36: 6 total, 5 up, 6 in 2023-07-14T18:11:10.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:10 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T18:11:10.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:10 smithi171 ceph-mon[129241]: from='osd.5 [v2:172.21.15.171:6808/777556751,v1:172.21.15.171:6809/777556751]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2023-07-14T18:11:10.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:10 smithi171 ceph-mon[129241]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2023-07-14T18:11:10.862 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:11:10 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:11:10.496+0000 7f76b3552700 -1 osd.5 0 waiting for initial osdmap 2023-07-14T18:11:10.862 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:11:10 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:11:10.502+0000 7f76af348700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T18:11:11.450 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-14T18:11:11.451 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-14T18:11:11.451 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2023-07-14T18:11:11.451 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2023-07-14T18:11:11.451 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.031703 s, 331 MB/s 2023-07-14T18:11:11.451 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2023-07-14T18:11:11.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:11 smithi161 ceph-mon[128993]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2023-07-14T18:11:11.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:11 smithi161 ceph-mon[128993]: osdmap e37: 6 total, 5 up, 6 in 2023-07-14T18:11:11.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:11 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T18:11:11.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:11 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T18:11:11.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:11 smithi161 ceph-mon[128993]: Detected new or changed devices on smithi171 2023-07-14T18:11:11.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:11 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:11.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:11 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:11.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:11 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:11:11.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:11 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:11:11.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:11 smithi161 ceph-mon[128993]: Adjusting osd_memory_target on smithi171 to 8584M 2023-07-14T18:11:11.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:11 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:11.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:11 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:11.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:11 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:11:11.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:11 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:11.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:11 smithi161 ceph-mon[128993]: osd.5 [v2:172.21.15.171:6808/777556751,v1:172.21.15.171:6809/777556751] boot 2023-07-14T18:11:11.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:11 smithi161 ceph-mon[128993]: osdmap e38: 6 total, 6 up, 6 in 2023-07-14T18:11:11.848 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:11 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T18:11:11.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:11 smithi171 ceph-mon[129241]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2023-07-14T18:11:11.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:11 smithi171 ceph-mon[129241]: osdmap e37: 6 total, 5 up, 6 in 2023-07-14T18:11:11.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:11 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T18:11:11.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:11 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T18:11:11.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:11 smithi171 ceph-mon[129241]: Detected new or changed devices on smithi171 2023-07-14T18:11:11.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:11 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:11.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:11 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:11.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:11 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:11:11.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:11 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:11:11.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:11 smithi171 ceph-mon[129241]: Adjusting osd_memory_target on smithi171 to 8584M 2023-07-14T18:11:11.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:11 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:11.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:11 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:11.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:11 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:11:11.864 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:11 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:11.864 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:11 smithi171 ceph-mon[129241]: osd.5 [v2:172.21.15.171:6808/777556751,v1:172.21.15.171:6809/777556751] boot 2023-07-14T18:11:11.864 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:11 smithi171 ceph-mon[129241]: osdmap e38: 6 total, 6 up, 6 in 2023-07-14T18:11:11.864 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:11 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T18:11:11.897 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:11 smithi133 ceph-mon[126384]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2023-07-14T18:11:11.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:11 smithi133 ceph-mon[126384]: osdmap e37: 6 total, 5 up, 6 in 2023-07-14T18:11:11.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:11 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T18:11:11.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:11 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T18:11:11.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:11 smithi133 ceph-mon[126384]: Detected new or changed devices on smithi171 2023-07-14T18:11:11.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:11 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:11.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:11 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:11.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:11 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:11:11.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:11 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:11:11.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:11 smithi133 ceph-mon[126384]: Adjusting osd_memory_target on smithi171 to 8584M 2023-07-14T18:11:11.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:11 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:11.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:11 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:11.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:11 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:11:11.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:11 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:11.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:11 smithi133 ceph-mon[126384]: osd.5 [v2:172.21.15.171:6808/777556751,v1:172.21.15.171:6809/777556751] boot 2023-07-14T18:11:11.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:11 smithi133 ceph-mon[126384]: osdmap e38: 6 total, 6 up, 6 in 2023-07-14T18:11:11.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:11 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T18:11:11.901 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi133:vg_nvme/lv_2 2023-07-14T18:11:12.618 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:11:13.229 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:12 smithi133 ceph-mon[126384]: purged_snaps scrub starts 2023-07-14T18:11:13.230 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:12 smithi133 ceph-mon[126384]: purged_snaps scrub ok 2023-07-14T18:11:13.230 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:12 smithi133 ceph-mon[126384]: pgmap v129: 1 pgs: 1 peering; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:11:13.230 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:12 smithi133 ceph-mon[126384]: osdmap e39: 6 total, 6 up, 6 in 2023-07-14T18:11:13.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:12 smithi161 ceph-mon[128993]: purged_snaps scrub starts 2023-07-14T18:11:13.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:12 smithi161 ceph-mon[128993]: purged_snaps scrub ok 2023-07-14T18:11:13.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:12 smithi161 ceph-mon[128993]: pgmap v129: 1 pgs: 1 peering; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:11:13.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:12 smithi161 ceph-mon[128993]: osdmap e39: 6 total, 6 up, 6 in 2023-07-14T18:11:13.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:12 smithi171 ceph-mon[129241]: purged_snaps scrub starts 2023-07-14T18:11:13.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:12 smithi171 ceph-mon[129241]: purged_snaps scrub ok 2023-07-14T18:11:13.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:12 smithi171 ceph-mon[129241]: pgmap v129: 1 pgs: 1 peering; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T18:11:13.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:12 smithi171 ceph-mon[129241]: osdmap e39: 6 total, 6 up, 6 in 2023-07-14T18:11:14.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.053+0000 7f59bbba3700 1 -- 172.21.15.133:0/2695111099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b4101850 msgr2=0x7f59b4108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:11:14.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.053+0000 7f59bbba3700 1 --2- 172.21.15.133:0/2695111099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b4101850 0x7f59b4108110 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f59ac00b0a0 tx=0x7f59ac00b3b0 comp rx=0 tx=0).stop 2023-07-14T18:11:14.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.053+0000 7f59bbba3700 1 -- 172.21.15.133:0/2695111099 shutdown_connections 2023-07-14T18:11:14.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.053+0000 7f59bbba3700 1 --2- 172.21.15.133:0/2695111099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59b40ffcc0 0x7f59b41000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:11:14.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.053+0000 7f59bbba3700 1 --2- 172.21.15.133:0/2695111099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b4101850 0x7f59b4108110 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:11:14.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.053+0000 7f59bbba3700 1 --2- 172.21.15.133:0/2695111099 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f59b4100ec0 0x7f59b4101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:11:14.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.053+0000 7f59bbba3700 1 -- 172.21.15.133:0/2695111099 >> 172.21.15.133:0/2695111099 conn(0x7f59b40fb2d0 msgr2=0x7f59b40fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:11:14.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.053+0000 7f59bbba3700 1 -- 172.21.15.133:0/2695111099 shutdown_connections 2023-07-14T18:11:14.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.053+0000 7f59bbba3700 1 -- 172.21.15.133:0/2695111099 wait complete. 2023-07-14T18:11:14.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.054+0000 7f59bbba3700 1 Processor -- start 2023-07-14T18:11:14.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.054+0000 7f59bbba3700 1 -- start start 2023-07-14T18:11:14.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.055+0000 7f59bbba3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59b40ffcc0 0x7f59b4195b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:11:14.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.055+0000 7f59bbba3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b4100ec0 0x7f59b4196050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:11:14.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.055+0000 7f59bbba3700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f59b4101850 0x7f59b419d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:11:14.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.055+0000 7f59bbba3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b419d6a0 con 0x7f59b4100ec0 2023-07-14T18:11:14.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.055+0000 7f59bbba3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b419d7e0 con 0x7f59b40ffcc0 2023-07-14T18:11:14.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.055+0000 7f59bbba3700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b419d970 con 0x7f59b4101850 2023-07-14T18:11:14.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.055+0000 7f59b913e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b4100ec0 0x7f59b4196050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:11:14.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.055+0000 7f59b913e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b4100ec0 0x7f59b4196050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48508/0 (socket says 172.21.15.133:48508) 2023-07-14T18:11:14.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.055+0000 7f59b913e700 1 -- 172.21.15.133:0/2194873971 learned_addr learned my addr 172.21.15.133:0/2194873971 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:11:14.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.055+0000 7f59ba140700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f59b4101850 0x7f59b419d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:11:14.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.055+0000 7f59b993f700 1 --2- 172.21.15.133:0/2194873971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59b40ffcc0 0x7f59b4195b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:11:14.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.055+0000 7f59ba140700 1 -- 172.21.15.133:0/2194873971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59b40ffcc0 msgr2=0x7f59b4195b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:11:14.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.055+0000 7f59ba140700 1 --2- 172.21.15.133:0/2194873971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59b40ffcc0 0x7f59b4195b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:11:14.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.055+0000 7f59ba140700 1 -- 172.21.15.133:0/2194873971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b4100ec0 msgr2=0x7f59b4196050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:11:14.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.055+0000 7f59ba140700 1 --2- 172.21.15.133:0/2194873971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b4100ec0 0x7f59b4196050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:11:14.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.055+0000 7f59ba140700 1 -- 172.21.15.133:0/2194873971 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ac009d00 con 0x7f59b4101850 2023-07-14T18:11:14.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.055+0000 7f59b913e700 1 --2- 172.21.15.133:0/2194873971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b4100ec0 0x7f59b4196050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:11:14.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.056+0000 7f59ba140700 1 --2- 172.21.15.133:0/2194873971 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f59b4101850 0x7f59b419d0d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f59ac00b070 tx=0x7f59ac007930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:11:14.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.056+0000 7f59aaffd700 1 -- 172.21.15.133:0/2194873971 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac00e040 con 0x7f59b4101850 2023-07-14T18:11:14.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.056+0000 7f59bbba3700 1 -- 172.21.15.133:0/2194873971 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b419dbf0 con 0x7f59b4101850 2023-07-14T18:11:14.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.056+0000 7f59aaffd700 1 -- 172.21.15.133:0/2194873971 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59ac004c50 con 0x7f59b4101850 2023-07-14T18:11:14.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.056+0000 7f59aaffd700 1 -- 172.21.15.133:0/2194873971 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac0052a0 con 0x7f59b4101850 2023-07-14T18:11:14.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.056+0000 7f59bbba3700 1 -- 172.21.15.133:0/2194873971 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b419e060 con 0x7f59b4101850 2023-07-14T18:11:14.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.057+0000 7f59bbba3700 1 -- 172.21.15.133:0/2194873971 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b4107200 con 0x7f59b4101850 2023-07-14T18:11:14.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.059+0000 7f59aaffd700 1 -- 172.21.15.133:0/2194873971 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f59ac01d470 con 0x7f59b4101850 2023-07-14T18:11:14.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.060+0000 7f59aaffd700 1 --2- 172.21.15.133:0/2194873971 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f59a00a1150 0x7f59a00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:11:14.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.060+0000 7f59aaffd700 1 -- 172.21.15.133:0/2194873971 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f59ac0cd130 con 0x7f59b4101850 2023-07-14T18:11:14.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.061+0000 7f59b993f700 1 --2- 172.21.15.133:0/2194873971 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f59a00a1150 0x7f59a00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:11:14.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.061+0000 7f59b993f700 1 --2- 172.21.15.133:0/2194873971 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f59a00a1150 0x7f59a00a3600 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f59b4100d20 tx=0x7f59b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:11:14.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.062+0000 7f59aaffd700 1 -- 172.21.15.133:0/2194873971 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f59ac097710 con 0x7f59b4101850 2023-07-14T18:11:14.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:14.269+0000 7f59bbba3700 1 -- 172.21.15.133:0/2194873971 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f59b4060a80 con 0x7f59a00a1150 2023-07-14T18:11:14.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:14 smithi161 ceph-mon[128993]: pgmap v132: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T18:11:14.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:14 smithi161 ceph-mon[128993]: osdmap e40: 6 total, 6 up, 6 in 2023-07-14T18:11:14.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:14 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:11:14.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:14 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:11:14.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:14 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:14.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:14 smithi171 ceph-mon[129241]: pgmap v132: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T18:11:14.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:14 smithi171 ceph-mon[129241]: osdmap e40: 6 total, 6 up, 6 in 2023-07-14T18:11:14.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:14 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:11:14.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:14 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:11:14.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:14 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:14.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:14 smithi133 ceph-mon[126384]: pgmap v132: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T18:11:14.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:14 smithi133 ceph-mon[126384]: osdmap e40: 6 total, 6 up, 6 in 2023-07-14T18:11:14.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:14 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:11:14.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:14 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:11:14.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:14 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:15.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:15 smithi133 ceph-mon[126384]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:11:15.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:15 smithi133 ceph-mon[126384]: osdmap e41: 6 total, 6 up, 6 in 2023-07-14T18:11:15.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:15 smithi161 ceph-mon[128993]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:11:15.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:15 smithi161 ceph-mon[128993]: osdmap e41: 6 total, 6 up, 6 in 2023-07-14T18:11:15.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:15 smithi171 ceph-mon[129241]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:11:15.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:15 smithi171 ceph-mon[129241]: osdmap e41: 6 total, 6 up, 6 in 2023-07-14T18:11:16.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:16 smithi133 ceph-mon[126384]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T18:11:16.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:16 smithi133 ceph-mon[126384]: osdmap e42: 6 total, 6 up, 6 in 2023-07-14T18:11:16.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:16 smithi161 ceph-mon[128993]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T18:11:16.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:16 smithi161 ceph-mon[128993]: osdmap e42: 6 total, 6 up, 6 in 2023-07-14T18:11:16.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:16 smithi171 ceph-mon[129241]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T18:11:16.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:16 smithi171 ceph-mon[129241]: osdmap e42: 6 total, 6 up, 6 in 2023-07-14T18:11:17.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:17 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2456616719' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ea0299ff-7740-4494-a3a9-1503b3fc0ecd"}]: dispatch 2023-07-14T18:11:17.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:17 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2456616719' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ea0299ff-7740-4494-a3a9-1503b3fc0ecd"}]': finished 2023-07-14T18:11:17.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:17 smithi161 ceph-mon[128993]: osdmap e43: 7 total, 6 up, 7 in 2023-07-14T18:11:17.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:17 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T18:11:17.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:17 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2456616719' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ea0299ff-7740-4494-a3a9-1503b3fc0ecd"}]: dispatch 2023-07-14T18:11:17.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:17 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2456616719' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ea0299ff-7740-4494-a3a9-1503b3fc0ecd"}]': finished 2023-07-14T18:11:17.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:17 smithi171 ceph-mon[129241]: osdmap e43: 7 total, 6 up, 7 in 2023-07-14T18:11:17.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:17 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T18:11:17.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:17 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2456616719' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ea0299ff-7740-4494-a3a9-1503b3fc0ecd"}]: dispatch 2023-07-14T18:11:17.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:17 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2456616719' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ea0299ff-7740-4494-a3a9-1503b3fc0ecd"}]': finished 2023-07-14T18:11:17.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:17 smithi133 ceph-mon[126384]: osdmap e43: 7 total, 6 up, 7 in 2023-07-14T18:11:17.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:17 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T18:11:18.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:18 smithi161 ceph-mon[128993]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T18:11:18.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:18 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3159951322' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:11:18.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:18 smithi171 ceph-mon[129241]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T18:11:18.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:18 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3159951322' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:11:18.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:18 smithi133 ceph-mon[126384]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T18:11:18.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:18 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3159951322' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:11:21.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:20 smithi171 ceph-mon[129241]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 98 KiB/s, 0 objects/s recovering 2023-07-14T18:11:21.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:20 smithi133 ceph-mon[126384]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 98 KiB/s, 0 objects/s recovering 2023-07-14T18:11:21.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:20 smithi161 ceph-mon[128993]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 98 KiB/s, 0 objects/s recovering 2023-07-14T18:11:23.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:22 smithi133 ceph-mon[126384]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2023-07-14T18:11:23.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:22 smithi161 ceph-mon[128993]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2023-07-14T18:11:23.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:22 smithi171 ceph-mon[129241]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2023-07-14T18:11:25.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:24 smithi133 ceph-mon[126384]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-14T18:11:25.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:24 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-14T18:11:25.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:24 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:25.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:24 smithi161 ceph-mon[128993]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-14T18:11:25.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:24 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-14T18:11:25.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:24 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:25.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:24 smithi171 ceph-mon[129241]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-14T18:11:25.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:24 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-14T18:11:25.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:24 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:26.043 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:25 smithi133 ceph-mon[126384]: Deploying daemon osd.6 on smithi133 2023-07-14T18:11:26.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:25 smithi161 ceph-mon[128993]: Deploying daemon osd.6 on smithi133 2023-07-14T18:11:26.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:25 smithi171 ceph-mon[129241]: Deploying daemon osd.6 on smithi133 2023-07-14T18:11:27.111 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:26 smithi133 ceph-mon[126384]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-14T18:11:27.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:26 smithi161 ceph-mon[128993]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-14T18:11:27.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:26 smithi171 ceph-mon[129241]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-14T18:11:29.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:28 smithi133 ceph-mon[126384]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-14T18:11:29.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:28 smithi161 ceph-mon[128993]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-14T18:11:29.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:28 smithi171 ceph-mon[129241]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-14T18:11:30.889 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:30 smithi133 ceph-mon[126384]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-14T18:11:31.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:30 smithi161 ceph-mon[128993]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-14T18:11:31.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:30 smithi171 ceph-mon[129241]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-14T18:11:32.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:31 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:32.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:31 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:32.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:31 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:11:32.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:31 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:32.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:31 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:32.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:31 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:11:32.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:31 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:32.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:31 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:32.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:31 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:11:33.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:32 smithi161 ceph-mon[128993]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-14T18:11:33.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:32 smithi171 ceph-mon[129241]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-14T18:11:33.385 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:32 smithi133 ceph-mon[126384]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-14T18:11:34.671 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:34.632+0000 7f59aaffd700 1 -- 172.21.15.133:0/2194873971 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f59b4060a80 con 0x7f59a00a1150 2023-07-14T18:11:34.672 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 6 on host 'smithi133' 2023-07-14T18:11:34.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:34.635+0000 7f59bbba3700 1 -- 172.21.15.133:0/2194873971 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f59a00a1150 msgr2=0x7f59a00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:11:34.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:34.635+0000 7f59bbba3700 1 --2- 172.21.15.133:0/2194873971 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f59a00a1150 0x7f59a00a3600 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f59b4100d20 tx=0x7f59b0009260 comp rx=0 tx=0).stop 2023-07-14T18:11:34.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:34.635+0000 7f59bbba3700 1 -- 172.21.15.133:0/2194873971 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f59b4101850 msgr2=0x7f59b419d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:11:34.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:34.635+0000 7f59bbba3700 1 --2- 172.21.15.133:0/2194873971 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f59b4101850 0x7f59b419d0d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f59ac00b070 tx=0x7f59ac007930 comp rx=0 tx=0).stop 2023-07-14T18:11:34.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:34.636+0000 7f59bbba3700 1 -- 172.21.15.133:0/2194873971 shutdown_connections 2023-07-14T18:11:34.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:34.636+0000 7f59bbba3700 1 --2- 172.21.15.133:0/2194873971 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f59a00a1150 0x7f59a00a3600 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:11:34.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:34.636+0000 7f59bbba3700 1 --2- 172.21.15.133:0/2194873971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59b40ffcc0 0x7f59b4195b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:11:34.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:34.636+0000 7f59bbba3700 1 --2- 172.21.15.133:0/2194873971 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f59b4101850 0x7f59b419d0d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:11:34.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:34.636+0000 7f59bbba3700 1 --2- 172.21.15.133:0/2194873971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b4100ec0 0x7f59b4196050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:11:34.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:34.636+0000 7f59bbba3700 1 -- 172.21.15.133:0/2194873971 >> 172.21.15.133:0/2194873971 conn(0x7f59b40fb2d0 msgr2=0x7f59b4104340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:11:34.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:34.636+0000 7f59bbba3700 1 -- 172.21.15.133:0/2194873971 shutdown_connections 2023-07-14T18:11:34.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:11:34.636+0000 7f59bbba3700 1 -- 172.21.15.133:0/2194873971 wait complete. 2023-07-14T18:11:35.138 DEBUG:teuthology.orchestra.run.smithi133:osd.6> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.6.service 2023-07-14T18:11:35.143 INFO:tasks.cephadm:Deploying osd.7 on smithi161 with /dev/vg_nvme/lv_2... 2023-07-14T18:11:35.143 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-14T18:11:35.214 INFO:journalctl@ceph.osd.6.smithi133.stdout:-- Logs begin at Fri 2023-07-14 17:46:17 UTC. -- 2023-07-14T18:11:35.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:34 smithi161 ceph-mon[128993]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-14T18:11:35.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:34 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:35.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:34 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:35.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:34 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:35.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:34 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:11:35.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:34 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:35.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:34 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:35.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:34 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:35.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:34 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:11:35.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:34 smithi171 ceph-mon[129241]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-14T18:11:35.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:34 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:35.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:34 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:35.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:34 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:35.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:34 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:11:35.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:34 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:35.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:34 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:35.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:34 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:35.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:34 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:11:35.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:34 smithi133 ceph-mon[126384]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-14T18:11:35.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:34 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:35.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:34 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:35.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:34 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:35.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:34 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:11:35.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:34 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:35.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:34 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:35.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:34 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:35.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:34 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:11:35.707 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T18:11:36.117 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:35 smithi133 ceph-mon[126384]: from='osd.6 [v2:172.21.15.133:6818/2884152358,v1:172.21.15.133:6819/2884152358]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-14T18:11:36.276 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:35 smithi161 ceph-mon[128993]: from='osd.6 [v2:172.21.15.133:6818/2884152358,v1:172.21.15.133:6819/2884152358]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-14T18:11:36.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:35 smithi171 ceph-mon[129241]: from='osd.6 [v2:172.21.15.133:6818/2884152358,v1:172.21.15.133:6819/2884152358]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-14T18:11:37.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:36 smithi133 ceph-mon[126384]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-14T18:11:37.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:36 smithi133 ceph-mon[126384]: from='osd.6 [v2:172.21.15.133:6818/2884152358,v1:172.21.15.133:6819/2884152358]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-14T18:11:37.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:36 smithi133 ceph-mon[126384]: osdmap e44: 7 total, 6 up, 7 in 2023-07-14T18:11:37.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:36 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T18:11:37.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:36 smithi133 ceph-mon[126384]: from='osd.6 [v2:172.21.15.133:6818/2884152358,v1:172.21.15.133:6819/2884152358]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T18:11:37.154 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:11:36 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:11:36.977+0000 7f405b6ba700 -1 osd.6 0 waiting for initial osdmap 2023-07-14T18:11:37.154 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:11:36 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:11:36.987+0000 7f40554ac700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T18:11:37.176 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-14T18:11:37.176 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-14T18:11:37.176 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2023-07-14T18:11:37.176 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2023-07-14T18:11:37.176 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0148385 s, 707 MB/s 2023-07-14T18:11:37.177 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2023-07-14T18:11:37.301 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:36 smithi161 ceph-mon[128993]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-14T18:11:37.301 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:36 smithi161 ceph-mon[128993]: from='osd.6 [v2:172.21.15.133:6818/2884152358,v1:172.21.15.133:6819/2884152358]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-14T18:11:37.301 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:36 smithi161 ceph-mon[128993]: osdmap e44: 7 total, 6 up, 7 in 2023-07-14T18:11:37.301 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T18:11:37.302 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:36 smithi161 ceph-mon[128993]: from='osd.6 [v2:172.21.15.133:6818/2884152358,v1:172.21.15.133:6819/2884152358]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T18:11:37.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:36 smithi171 ceph-mon[129241]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-14T18:11:37.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:36 smithi171 ceph-mon[129241]: from='osd.6 [v2:172.21.15.133:6818/2884152358,v1:172.21.15.133:6819/2884152358]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-14T18:11:37.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:36 smithi171 ceph-mon[129241]: osdmap e44: 7 total, 6 up, 7 in 2023-07-14T18:11:37.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:36 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T18:11:37.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:36 smithi171 ceph-mon[129241]: from='osd.6 [v2:172.21.15.133:6818/2884152358,v1:172.21.15.133:6819/2884152358]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T18:11:37.561 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi161:vg_nvme/lv_2 2023-07-14T18:11:38.110 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T18:11:38.307 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:37 smithi161 ceph-mon[128993]: purged_snaps scrub starts 2023-07-14T18:11:38.308 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:37 smithi161 ceph-mon[128993]: purged_snaps scrub ok 2023-07-14T18:11:38.308 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:37 smithi161 ceph-mon[128993]: from='osd.6 [v2:172.21.15.133:6818/2884152358,v1:172.21.15.133:6819/2884152358]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-07-14T18:11:38.308 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:37 smithi161 ceph-mon[128993]: osdmap e45: 7 total, 6 up, 7 in 2023-07-14T18:11:38.308 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:37 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T18:11:38.308 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:37 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T18:11:38.308 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:37 smithi161 ceph-mon[128993]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-14T18:11:38.308 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:37 smithi161 ceph-mon[128993]: Detected new or changed devices on smithi133 2023-07-14T18:11:38.309 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:37 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:38.309 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:37 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:38.309 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:37 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:11:38.309 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:37 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:11:38.309 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:37 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:11:38.309 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:37 smithi161 ceph-mon[128993]: Adjusting osd_memory_target on smithi133 to 5755M 2023-07-14T18:11:38.309 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:37 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:38.310 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:37 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:38.310 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:37 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:11:38.310 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:37 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:38.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:37 smithi171 ceph-mon[129241]: purged_snaps scrub starts 2023-07-14T18:11:38.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:37 smithi171 ceph-mon[129241]: purged_snaps scrub ok 2023-07-14T18:11:38.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:37 smithi171 ceph-mon[129241]: from='osd.6 [v2:172.21.15.133:6818/2884152358,v1:172.21.15.133:6819/2884152358]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-07-14T18:11:38.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:37 smithi171 ceph-mon[129241]: osdmap e45: 7 total, 6 up, 7 in 2023-07-14T18:11:38.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:37 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T18:11:38.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:37 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T18:11:38.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:37 smithi171 ceph-mon[129241]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-14T18:11:38.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:37 smithi171 ceph-mon[129241]: Detected new or changed devices on smithi133 2023-07-14T18:11:38.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:37 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:38.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:37 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:38.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:37 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:11:38.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:37 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:11:38.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:37 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:11:38.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:37 smithi171 ceph-mon[129241]: Adjusting osd_memory_target on smithi133 to 5755M 2023-07-14T18:11:38.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:37 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:38.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:37 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:38.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:37 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:11:38.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:37 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:38.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:37 smithi133 ceph-mon[126384]: purged_snaps scrub starts 2023-07-14T18:11:38.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:37 smithi133 ceph-mon[126384]: purged_snaps scrub ok 2023-07-14T18:11:38.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:37 smithi133 ceph-mon[126384]: from='osd.6 [v2:172.21.15.133:6818/2884152358,v1:172.21.15.133:6819/2884152358]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-07-14T18:11:38.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:37 smithi133 ceph-mon[126384]: osdmap e45: 7 total, 6 up, 7 in 2023-07-14T18:11:38.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:37 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T18:11:38.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:37 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T18:11:38.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:37 smithi133 ceph-mon[126384]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-14T18:11:38.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:37 smithi133 ceph-mon[126384]: Detected new or changed devices on smithi133 2023-07-14T18:11:38.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:37 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:38.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:37 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:38.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:37 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:11:38.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:37 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:11:38.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:37 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:11:38.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:37 smithi133 ceph-mon[126384]: Adjusting osd_memory_target on smithi133 to 5755M 2023-07-14T18:11:38.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:37 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:38.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:37 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:38.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:37 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:11:38.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:37 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:39.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.293+0000 7f9034bf6700 1 -- 172.21.15.161:0/414352691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9030100740 msgr2=0x7f9030107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:11:39.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.293+0000 7f9034bf6700 1 --2- 172.21.15.161:0/414352691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9030100740 0x7f9030107000 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f902800b0a0 tx=0x7f902800b3b0 comp rx=0 tx=0).stop 2023-07-14T18:11:39.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.293+0000 7f9034bf6700 1 -- 172.21.15.161:0/414352691 shutdown_connections 2023-07-14T18:11:39.335 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.293+0000 7f9034bf6700 1 --2- 172.21.15.161:0/414352691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9030100740 0x7f9030107000 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:11:39.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.293+0000 7f9034bf6700 1 --2- 172.21.15.161:0/414352691 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f90300febb0 0x7f90300fefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:11:39.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.293+0000 7f9034bf6700 1 --2- 172.21.15.161:0/414352691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90300ffdb0 0x7f9030100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:11:39.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.293+0000 7f9034bf6700 1 -- 172.21.15.161:0/414352691 >> 172.21.15.161:0/414352691 conn(0x7f90300fa1e0 msgr2=0x7f90300fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:11:39.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.294+0000 7f9034bf6700 1 -- 172.21.15.161:0/414352691 shutdown_connections 2023-07-14T18:11:39.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.294+0000 7f9034bf6700 1 -- 172.21.15.161:0/414352691 wait complete. 2023-07-14T18:11:39.336 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.294+0000 7f9034bf6700 1 Processor -- start 2023-07-14T18:11:39.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.294+0000 7f9034bf6700 1 -- start start 2023-07-14T18:11:39.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.294+0000 7f9034bf6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90300febb0 0x7f9030194a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:11:39.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.294+0000 7f9034bf6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90300ffdb0 0x7f9030194f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:11:39.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.294+0000 7f9034bf6700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9030100740 0x7f903019bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:11:39.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.295+0000 7f9034bf6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903019c5a0 con 0x7f90300ffdb0 2023-07-14T18:11:39.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.295+0000 7f9034bf6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903019c6e0 con 0x7f90300febb0 2023-07-14T18:11:39.337 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.295+0000 7f9034bf6700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903019c870 con 0x7f9030100740 2023-07-14T18:11:39.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.295+0000 7f902e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90300febb0 0x7f9030194a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:11:39.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.295+0000 7f902e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90300febb0 0x7f9030194a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54838/0 (socket says 172.21.15.161:54838) 2023-07-14T18:11:39.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.295+0000 7f902e59c700 1 -- 172.21.15.161:0/1229206985 learned_addr learned my addr 172.21.15.161:0/1229206985 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T18:11:39.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.295+0000 7f902ed9d700 1 --2- 172.21.15.161:0/1229206985 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9030100740 0x7f903019bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:11:39.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.295+0000 7f902dd9b700 1 --2- 172.21.15.161:0/1229206985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90300ffdb0 0x7f9030194f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:11:39.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.295+0000 7f902e59c700 1 -- 172.21.15.161:0/1229206985 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9030100740 msgr2=0x7f903019bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:11:39.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.295+0000 7f902e59c700 1 --2- 172.21.15.161:0/1229206985 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9030100740 0x7f903019bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:11:39.338 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.295+0000 7f902e59c700 1 -- 172.21.15.161:0/1229206985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90300ffdb0 msgr2=0x7f9030194f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:11:39.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.295+0000 7f902e59c700 1 --2- 172.21.15.161:0/1229206985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90300ffdb0 0x7f9030194f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:11:39.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.295+0000 7f902e59c700 1 -- 172.21.15.161:0/1229206985 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9028009d00 con 0x7f90300febb0 2023-07-14T18:11:39.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.295+0000 7f902ed9d700 1 --2- 172.21.15.161:0/1229206985 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9030100740 0x7f903019bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:11:39.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.295+0000 7f902e59c700 1 --2- 172.21.15.161:0/1229206985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90300febb0 0x7f9030194a10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f902000d980 tx=0x7f902000dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:11:39.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.295+0000 7f90277fe700 1 -- 172.21.15.161:0/1229206985 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902000de60 con 0x7f90300febb0 2023-07-14T18:11:39.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.295+0000 7f9034bf6700 1 -- 172.21.15.161:0/1229206985 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f903019cb50 con 0x7f90300febb0 2023-07-14T18:11:39.339 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.295+0000 7f90277fe700 1 -- 172.21.15.161:0/1229206985 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90200044e0 con 0x7f90300febb0 2023-07-14T18:11:39.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.295+0000 7f90277fe700 1 -- 172.21.15.161:0/1229206985 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9020003bc0 con 0x7f90300febb0 2023-07-14T18:11:39.340 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.295+0000 7f9034bf6700 1 -- 172.21.15.161:0/1229206985 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f903019d0a0 con 0x7f90300febb0 2023-07-14T18:11:39.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.296+0000 7f9034bf6700 1 -- 172.21.15.161:0/1229206985 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f903010a3f0 con 0x7f90300febb0 2023-07-14T18:11:39.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.298+0000 7f90277fe700 1 -- 172.21.15.161:0/1229206985 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f9020009bb0 con 0x7f90300febb0 2023-07-14T18:11:39.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.298+0000 7f90277fe700 1 --2- 172.21.15.161:0/1229206985 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f901c0a1150 0x7f901c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:11:39.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.299+0000 7f90277fe700 1 -- 172.21.15.161:0/1229206985 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f90200cbd50 con 0x7f90300febb0 2023-07-14T18:11:39.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.299+0000 7f902dd9b700 1 --2- 172.21.15.161:0/1229206985 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f901c0a1150 0x7f901c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:11:39.341 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.299+0000 7f902dd9b700 1 --2- 172.21.15.161:0/1229206985 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f901c0a1150 0x7f901c0a3600 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9018006fd0 tx=0x7f9018008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:11:39.342 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.302+0000 7f90277fe700 1 -- 172.21.15.161:0/1229206985 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f9020096110 con 0x7f90300febb0 2023-07-14T18:11:39.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:38 smithi161 ceph-mon[128993]: osd.6 [v2:172.21.15.133:6818/2884152358,v1:172.21.15.133:6819/2884152358] boot 2023-07-14T18:11:39.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:38 smithi161 ceph-mon[128993]: osdmap e46: 7 total, 7 up, 7 in 2023-07-14T18:11:39.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:38 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T18:11:39.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:38 smithi171 ceph-mon[129241]: osd.6 [v2:172.21.15.133:6818/2884152358,v1:172.21.15.133:6819/2884152358] boot 2023-07-14T18:11:39.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:38 smithi171 ceph-mon[129241]: osdmap e46: 7 total, 7 up, 7 in 2023-07-14T18:11:39.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:38 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T18:11:39.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:38 smithi133 ceph-mon[126384]: osd.6 [v2:172.21.15.133:6818/2884152358,v1:172.21.15.133:6819/2884152358] boot 2023-07-14T18:11:39.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:38 smithi133 ceph-mon[126384]: osdmap e46: 7 total, 7 up, 7 in 2023-07-14T18:11:39.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:38 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T18:11:39.535 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:39.495+0000 7f9034bf6700 1 -- 172.21.15.161:0/1229206985 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9030060a80 con 0x7f901c0a1150 2023-07-14T18:11:40.314 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:39 smithi161 ceph-mon[128993]: osdmap e47: 7 total, 7 up, 7 in 2023-07-14T18:11:40.314 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:39 smithi161 ceph-mon[128993]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:40.314 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:39 smithi161 ceph-mon[128993]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:11:40.314 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:39 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:11:40.315 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:39 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:11:40.315 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:39 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:40.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:39 smithi171 ceph-mon[129241]: osdmap e47: 7 total, 7 up, 7 in 2023-07-14T18:11:40.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:39 smithi171 ceph-mon[129241]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:40.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:39 smithi171 ceph-mon[129241]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:11:40.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:39 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:11:40.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:39 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:11:40.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:39 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:40.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:39 smithi133 ceph-mon[126384]: osdmap e47: 7 total, 7 up, 7 in 2023-07-14T18:11:40.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:39 smithi133 ceph-mon[126384]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:40.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:39 smithi133 ceph-mon[126384]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:11:40.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:39 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:11:40.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:39 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:11:40.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:39 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:42.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:42 smithi161 ceph-mon[128993]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:42.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:42 smithi161 ceph-mon[128993]: from='client.? 172.21.15.161:0/2466923957' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd"}]: dispatch 2023-07-14T18:11:42.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:42 smithi161 ceph-mon[128993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd"}]: dispatch 2023-07-14T18:11:42.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:42 smithi161 ceph-mon[128993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd"}]': finished 2023-07-14T18:11:42.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:42 smithi161 ceph-mon[128993]: osdmap e48: 8 total, 7 up, 8 in 2023-07-14T18:11:42.848 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:42 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T18:11:42.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:42 smithi171 ceph-mon[129241]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:42.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:42 smithi171 ceph-mon[129241]: from='client.? 172.21.15.161:0/2466923957' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd"}]: dispatch 2023-07-14T18:11:42.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:42 smithi171 ceph-mon[129241]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd"}]: dispatch 2023-07-14T18:11:42.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:42 smithi171 ceph-mon[129241]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd"}]': finished 2023-07-14T18:11:42.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:42 smithi171 ceph-mon[129241]: osdmap e48: 8 total, 7 up, 8 in 2023-07-14T18:11:42.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:42 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T18:11:42.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:42 smithi133 ceph-mon[126384]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:42.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:42 smithi133 ceph-mon[126384]: from='client.? 172.21.15.161:0/2466923957' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd"}]: dispatch 2023-07-14T18:11:42.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:42 smithi133 ceph-mon[126384]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd"}]: dispatch 2023-07-14T18:11:42.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:42 smithi133 ceph-mon[126384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd"}]': finished 2023-07-14T18:11:42.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:42 smithi133 ceph-mon[126384]: osdmap e48: 8 total, 7 up, 8 in 2023-07-14T18:11:42.903 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:42 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T18:11:43.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:43 smithi161 ceph-mon[128993]: from='client.? 172.21.15.161:0/2976464457' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:11:43.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:43 smithi171 ceph-mon[129241]: from='client.? 172.21.15.161:0/2976464457' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:11:43.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:43 smithi133 ceph-mon[126384]: from='client.? 172.21.15.161:0/2976464457' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:11:44.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:44 smithi161 ceph-mon[128993]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:44.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:44 smithi171 ceph-mon[129241]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:44.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:44 smithi133 ceph-mon[126384]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:47.024 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:46 smithi161 ceph-mon[128993]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:47.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:46 smithi133 ceph-mon[126384]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:47.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:46 smithi171 ceph-mon[129241]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:49.138 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:48 smithi161 ceph-mon[128993]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:49.139 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-14T18:11:49.139 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:49.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:48 smithi133 ceph-mon[126384]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:49.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-14T18:11:49.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:49.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:48 smithi171 ceph-mon[129241]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:49.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-14T18:11:49.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:50.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:49 smithi133 ceph-mon[126384]: Deploying daemon osd.7 on smithi161 2023-07-14T18:11:50.302 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:49 smithi161 ceph-mon[128993]: Deploying daemon osd.7 on smithi161 2023-07-14T18:11:50.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:49 smithi171 ceph-mon[129241]: Deploying daemon osd.7 on smithi161 2023-07-14T18:11:50.884 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:50 smithi161 ceph-mon[128993]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:51.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:50 smithi133 ceph-mon[126384]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:51.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:50 smithi171 ceph-mon[129241]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:53.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:52 smithi133 ceph-mon[126384]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:53.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:52 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:53.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:52 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:53.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:52 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:11:53.255 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:52 smithi161 ceph-mon[128993]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:53.256 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:52 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:53.256 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:52 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:53.256 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:52 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:11:53.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:52 smithi171 ceph-mon[129241]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:53.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:52 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:53.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:52 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:53.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:52 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:11:55.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:54 smithi161 ceph-mon[128993]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:55.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:54 smithi133 ceph-mon[126384]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:55.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:54 smithi171 ceph-mon[129241]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:55.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:55.558+0000 7f90277fe700 1 -- 172.21.15.161:0/1229206985 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9030060a80 con 0x7f901c0a1150 2023-07-14T18:11:55.598 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 7 on host 'smithi161' 2023-07-14T18:11:55.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:55.561+0000 7f9034bf6700 1 -- 172.21.15.161:0/1229206985 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f901c0a1150 msgr2=0x7f901c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:11:55.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:55.561+0000 7f9034bf6700 1 --2- 172.21.15.161:0/1229206985 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f901c0a1150 0x7f901c0a3600 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9018006fd0 tx=0x7f9018008040 comp rx=0 tx=0).stop 2023-07-14T18:11:55.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:55.561+0000 7f9034bf6700 1 -- 172.21.15.161:0/1229206985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90300febb0 msgr2=0x7f9030194a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:11:55.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:55.561+0000 7f9034bf6700 1 --2- 172.21.15.161:0/1229206985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90300febb0 0x7f9030194a10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f902000d980 tx=0x7f902000dd40 comp rx=0 tx=0).stop 2023-07-14T18:11:55.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:55.561+0000 7f9034bf6700 1 -- 172.21.15.161:0/1229206985 shutdown_connections 2023-07-14T18:11:55.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:55.561+0000 7f9034bf6700 1 --2- 172.21.15.161:0/1229206985 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f901c0a1150 0x7f901c0a3600 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:11:55.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:55.561+0000 7f9034bf6700 1 --2- 172.21.15.161:0/1229206985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90300febb0 0x7f9030194a10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:11:55.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:55.561+0000 7f9034bf6700 1 --2- 172.21.15.161:0/1229206985 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9030100740 0x7f903019bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:11:55.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:55.561+0000 7f9034bf6700 1 --2- 172.21.15.161:0/1229206985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90300ffdb0 0x7f9030194f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:11:55.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:55.561+0000 7f9034bf6700 1 -- 172.21.15.161:0/1229206985 >> 172.21.15.161:0/1229206985 conn(0x7f90300fa1e0 msgr2=0x7f90300fc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:11:55.602 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:55.561+0000 7f9034bf6700 1 -- 172.21.15.161:0/1229206985 shutdown_connections 2023-07-14T18:11:55.603 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:11:55.561+0000 7f9034bf6700 1 -- 172.21.15.161:0/1229206985 wait complete. 2023-07-14T18:11:56.013 DEBUG:teuthology.orchestra.run.smithi161:osd.7> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.7.service 2023-07-14T18:11:56.015 INFO:tasks.cephadm:Deploying osd.8 on smithi171 with /dev/vg_nvme/lv_2... 2023-07-14T18:11:56.016 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-14T18:11:56.052 INFO:journalctl@ceph.osd.7.smithi161.stdout:-- Logs begin at Fri 2023-07-14 17:41:43 UTC. -- 2023-07-14T18:11:56.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:56 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:56.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:56 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:56.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:56 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:56.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:56 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:11:56.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:56 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:56.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:56 smithi161 ceph-mon[128993]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:56.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:56 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:56.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:56 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:56.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:56 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:11:56.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:56 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:56.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:56 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:56.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:56 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:56.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:56 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:11:56.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:56 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:56.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:56 smithi171 ceph-mon[129241]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:56.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:56 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:56.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:56 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:56.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:56 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:11:56.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:56 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:56.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:56 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:56.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:56 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:56.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:56 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:11:56.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:56 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:56.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:56 smithi133 ceph-mon[126384]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:56.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:56 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:56.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:56 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:56.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:56 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:11:56.579 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T18:11:57.048 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:11:56 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:11:56.735+0000 7faad7246640 -1 osd.7 0 log_to_monitors true 2023-07-14T18:11:57.305 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:57 smithi161 ceph-mon[128993]: from='osd.7 [v2:172.21.15.161:6816/4183705749,v1:172.21.15.161:6817/4183705749]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T18:11:57.306 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:57 smithi161 ceph-mon[128993]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T18:11:57.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:57 smithi133 ceph-mon[126384]: from='osd.7 [v2:172.21.15.161:6816/4183705749,v1:172.21.15.161:6817/4183705749]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T18:11:57.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:57 smithi133 ceph-mon[126384]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T18:11:57.585 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:57 smithi171 ceph-mon[129241]: from='osd.7 [v2:172.21.15.161:6816/4183705749,v1:172.21.15.161:6817/4183705749]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T18:11:57.585 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:57 smithi171 ceph-mon[129241]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T18:11:58.079 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-14T18:11:58.080 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-14T18:11:58.080 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2023-07-14T18:11:58.080 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2023-07-14T18:11:58.080 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0150062 s, 699 MB/s 2023-07-14T18:11:58.080 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2023-07-14T18:11:58.150 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:58 smithi161 ceph-mon[128993]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-14T18:11:58.150 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:58 smithi161 ceph-mon[128993]: osdmap e49: 8 total, 7 up, 8 in 2023-07-14T18:11:58.151 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:58 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T18:11:58.151 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:58 smithi161 ceph-mon[128993]: from='osd.7 [v2:172.21.15.161:6816/4183705749,v1:172.21.15.161:6817/4183705749]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T18:11:58.151 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:58 smithi161 ceph-mon[128993]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T18:11:58.151 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:58 smithi161 ceph-mon[128993]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:58.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:58 smithi133 ceph-mon[126384]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-14T18:11:58.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:58 smithi133 ceph-mon[126384]: osdmap e49: 8 total, 7 up, 8 in 2023-07-14T18:11:58.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:58 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T18:11:58.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:58 smithi133 ceph-mon[126384]: from='osd.7 [v2:172.21.15.161:6816/4183705749,v1:172.21.15.161:6817/4183705749]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T18:11:58.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:58 smithi133 ceph-mon[126384]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T18:11:58.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:58 smithi133 ceph-mon[126384]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:58.488 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:58 smithi171 ceph-mon[129241]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-14T18:11:58.489 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:58 smithi171 ceph-mon[129241]: osdmap e49: 8 total, 7 up, 8 in 2023-07-14T18:11:58.489 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:58 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T18:11:58.489 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:58 smithi171 ceph-mon[129241]: from='osd.7 [v2:172.21.15.161:6816/4183705749,v1:172.21.15.161:6817/4183705749]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T18:11:58.489 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:58 smithi171 ceph-mon[129241]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T18:11:58.489 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:58 smithi171 ceph-mon[129241]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:11:58.535 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi171:vg_nvme/lv_2 2023-07-14T18:11:58.596 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:11:58 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:11:58.114+0000 7faacc8be700 -1 osd.7 0 waiting for initial osdmap 2023-07-14T18:11:58.597 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:11:58 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:11:58.123+0000 7faac5eaf700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T18:11:59.085 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T18:11:59.478 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:59 smithi171 ceph-mon[129241]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T18:11:59.478 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:59 smithi171 ceph-mon[129241]: osdmap e50: 8 total, 7 up, 8 in 2023-07-14T18:11:59.478 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:59 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T18:11:59.478 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:59 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T18:11:59.478 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:59 smithi171 ceph-mon[129241]: Detected new or changed devices on smithi161 2023-07-14T18:11:59.478 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:59 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:59.479 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:59 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:59.479 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:59 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:11:59.479 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:59 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:11:59.479 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:59 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:11:59.479 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:59 smithi171 ceph-mon[129241]: Adjusting osd_memory_target on smithi161 to 5722M 2023-07-14T18:11:59.479 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:59 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:59.479 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:59 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:59.479 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:59 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:11:59.480 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:11:59 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:59.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:59 smithi161 ceph-mon[128993]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T18:11:59.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:59 smithi161 ceph-mon[128993]: osdmap e50: 8 total, 7 up, 8 in 2023-07-14T18:11:59.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:59 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T18:11:59.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:59 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T18:11:59.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:59 smithi161 ceph-mon[128993]: Detected new or changed devices on smithi161 2023-07-14T18:11:59.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:59 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:59.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:59 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:59.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:59 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:11:59.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:59 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:11:59.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:59 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:11:59.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:59 smithi161 ceph-mon[128993]: Adjusting osd_memory_target on smithi161 to 5722M 2023-07-14T18:11:59.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:59 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:59.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:59 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:59.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:59 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:11:59.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:11:59 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:59.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:59 smithi133 ceph-mon[126384]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T18:11:59.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:59 smithi133 ceph-mon[126384]: osdmap e50: 8 total, 7 up, 8 in 2023-07-14T18:11:59.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:59 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T18:11:59.653 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:59 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T18:11:59.653 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:59 smithi133 ceph-mon[126384]: Detected new or changed devices on smithi161 2023-07-14T18:11:59.653 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:59 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:59.653 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:59 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:59.654 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:59 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:11:59.654 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:59 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:11:59.654 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:59 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:11:59.654 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:59 smithi133 ceph-mon[126384]: Adjusting osd_memory_target on smithi161 to 5722M 2023-07-14T18:11:59.655 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:59 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:11:59.655 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:59 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:11:59.655 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:59 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:11:59.656 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:11:59 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:00.299 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.259+0000 7f5566c22700 1 -- 172.21.15.171:0/3240903644 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5560100740 msgr2=0x7f5560107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:12:00.300 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.259+0000 7f5566c22700 1 --2- 172.21.15.171:0/3240903644 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5560100740 0x7f5560107000 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f555800b0a0 tx=0x7f555800b3b0 comp rx=0 tx=0).stop 2023-07-14T18:12:00.300 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.259+0000 7f5566c22700 1 -- 172.21.15.171:0/3240903644 shutdown_connections 2023-07-14T18:12:00.300 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.259+0000 7f5566c22700 1 --2- 172.21.15.171:0/3240903644 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5560100740 0x7f5560107000 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:12:00.300 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.259+0000 7f5566c22700 1 --2- 172.21.15.171:0/3240903644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55600febb0 0x7f55600fefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:12:00.300 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.259+0000 7f5566c22700 1 --2- 172.21.15.171:0/3240903644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55600ffdb0 0x7f5560100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:12:00.301 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.259+0000 7f5566c22700 1 -- 172.21.15.171:0/3240903644 >> 172.21.15.171:0/3240903644 conn(0x7f55600fa1e0 msgr2=0x7f55600fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:12:00.301 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.259+0000 7f5566c22700 1 -- 172.21.15.171:0/3240903644 shutdown_connections 2023-07-14T18:12:00.301 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.259+0000 7f5566c22700 1 -- 172.21.15.171:0/3240903644 wait complete. 2023-07-14T18:12:00.302 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.260+0000 7f5566c22700 1 Processor -- start 2023-07-14T18:12:00.303 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.260+0000 7f5566c22700 1 -- start start 2023-07-14T18:12:00.303 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.260+0000 7f5566c22700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55600febb0 0x7f55601949b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:12:00.303 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.260+0000 7f5566c22700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55600ffdb0 0x7f5560194ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:12:00.303 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.260+0000 7f5566c22700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5560100740 0x7f556019bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:12:00.303 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.260+0000 7f5566c22700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556019c540 con 0x7f5560100740 2023-07-14T18:12:00.303 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.260+0000 7f5566c22700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556019c680 con 0x7f55600febb0 2023-07-14T18:12:00.303 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.260+0000 7f5566c22700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5560068180 con 0x7f55600ffdb0 2023-07-14T18:12:00.304 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.260+0000 7f555ffff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55600ffdb0 0x7f5560194ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:12:00.304 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.260+0000 7f555ffff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55600ffdb0 0x7f5560194ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:42964/0 (socket says 172.21.15.171:42964) 2023-07-14T18:12:00.304 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.260+0000 7f555ffff700 1 -- 172.21.15.171:0/81587439 learned_addr learned my addr 172.21.15.171:0/81587439 (peer_addr_for_me v2:172.21.15.171:0/0) 2023-07-14T18:12:00.304 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.260+0000 7f55649be700 1 --2- 172.21.15.171:0/81587439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55600febb0 0x7f55601949b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:12:00.304 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.260+0000 7f55651bf700 1 --2- 172.21.15.171:0/81587439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5560100740 0x7f556019bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:12:00.304 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.260+0000 7f555ffff700 1 -- 172.21.15.171:0/81587439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55600febb0 msgr2=0x7f55601949b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:12:00.304 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.260+0000 7f555ffff700 1 --2- 172.21.15.171:0/81587439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55600febb0 0x7f55601949b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:12:00.305 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.260+0000 7f555ffff700 1 -- 172.21.15.171:0/81587439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5560100740 msgr2=0x7f556019bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:12:00.305 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.260+0000 7f555ffff700 1 --2- 172.21.15.171:0/81587439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5560100740 0x7f556019bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:12:00.305 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.260+0000 7f555ffff700 1 -- 172.21.15.171:0/81587439 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5558009d00 con 0x7f55600ffdb0 2023-07-14T18:12:00.305 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.260+0000 7f555ffff700 1 --2- 172.21.15.171:0/81587439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55600ffdb0 0x7f5560194ef0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f554c00d950 tx=0x7f554c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:12:00.305 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.260+0000 7f555dffb700 1 -- 172.21.15.171:0/81587439 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c009940 con 0x7f55600ffdb0 2023-07-14T18:12:00.305 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.261+0000 7f555dffb700 1 -- 172.21.15.171:0/81587439 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f554c010460 con 0x7f55600ffdb0 2023-07-14T18:12:00.305 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.261+0000 7f5566c22700 1 -- 172.21.15.171:0/81587439 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5560068460 con 0x7f55600ffdb0 2023-07-14T18:12:00.306 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.261+0000 7f555dffb700 1 -- 172.21.15.171:0/81587439 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c00f620 con 0x7f55600ffdb0 2023-07-14T18:12:00.306 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.261+0000 7f5566c22700 1 -- 172.21.15.171:0/81587439 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5560068960 con 0x7f55600ffdb0 2023-07-14T18:12:00.306 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.262+0000 7f5566c22700 1 -- 172.21.15.171:0/81587439 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5560106150 con 0x7f55600ffdb0 2023-07-14T18:12:00.307 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.263+0000 7f555dffb700 1 -- 172.21.15.171:0/81587439 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f554c0108d0 con 0x7f55600ffdb0 2023-07-14T18:12:00.307 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.263+0000 7f555dffb700 1 --2- 172.21.15.171:0/81587439 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f55500a1150 0x7f55500a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:12:00.307 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.263+0000 7f555dffb700 1 -- 172.21.15.171:0/81587439 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f554c0cbc40 con 0x7f55600ffdb0 2023-07-14T18:12:00.307 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.263+0000 7f55649be700 1 --2- 172.21.15.171:0/81587439 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f55500a1150 0x7f55500a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:12:00.307 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.264+0000 7f55649be700 1 --2- 172.21.15.171:0/81587439 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f55500a1150 0x7f55500a3600 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f55600ffc10 tx=0x7f555400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:12:00.308 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.267+0000 7f555dffb700 1 -- 172.21.15.171:0/81587439 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f554c095ed0 con 0x7f55600ffdb0 2023-07-14T18:12:00.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:00 smithi171 ceph-mon[129241]: purged_snaps scrub starts 2023-07-14T18:12:00.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:00 smithi171 ceph-mon[129241]: purged_snaps scrub ok 2023-07-14T18:12:00.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:00 smithi171 ceph-mon[129241]: osd.7 [v2:172.21.15.161:6816/4183705749,v1:172.21.15.161:6817/4183705749] boot 2023-07-14T18:12:00.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:00 smithi171 ceph-mon[129241]: osdmap e51: 8 total, 8 up, 8 in 2023-07-14T18:12:00.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:00 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T18:12:00.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:00 smithi171 ceph-mon[129241]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:12:00.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:00 smithi171 ceph-mon[129241]: osdmap e52: 8 total, 8 up, 8 in 2023-07-14T18:12:00.503 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:00.462+0000 7f5566c22700 1 -- 172.21.15.171:0/81587439 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f5560068bf0 con 0x7f55500a1150 2023-07-14T18:12:00.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:00 smithi161 ceph-mon[128993]: purged_snaps scrub starts 2023-07-14T18:12:00.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:00 smithi161 ceph-mon[128993]: purged_snaps scrub ok 2023-07-14T18:12:00.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:00 smithi161 ceph-mon[128993]: osd.7 [v2:172.21.15.161:6816/4183705749,v1:172.21.15.161:6817/4183705749] boot 2023-07-14T18:12:00.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:00 smithi161 ceph-mon[128993]: osdmap e51: 8 total, 8 up, 8 in 2023-07-14T18:12:00.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:00 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T18:12:00.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:00 smithi161 ceph-mon[128993]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:12:00.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:00 smithi161 ceph-mon[128993]: osdmap e52: 8 total, 8 up, 8 in 2023-07-14T18:12:00.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:00 smithi133 ceph-mon[126384]: purged_snaps scrub starts 2023-07-14T18:12:00.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:00 smithi133 ceph-mon[126384]: purged_snaps scrub ok 2023-07-14T18:12:00.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:00 smithi133 ceph-mon[126384]: osd.7 [v2:172.21.15.161:6816/4183705749,v1:172.21.15.161:6817/4183705749] boot 2023-07-14T18:12:00.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:00 smithi133 ceph-mon[126384]: osdmap e51: 8 total, 8 up, 8 in 2023-07-14T18:12:00.653 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:00 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T18:12:00.653 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:00 smithi133 ceph-mon[126384]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T18:12:00.653 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:00 smithi133 ceph-mon[126384]: osdmap e52: 8 total, 8 up, 8 in 2023-07-14T18:12:01.526 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:01 smithi171 ceph-mon[129241]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:12:01.526 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:01 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:12:01.526 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:01 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:12:01.527 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:01 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:01.527 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:01 smithi171 ceph-mon[129241]: osdmap e53: 8 total, 8 up, 8 in 2023-07-14T18:12:01.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:01 smithi161 ceph-mon[128993]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:12:01.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:01 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:12:01.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:01 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:12:01.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:01 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:01.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:01 smithi161 ceph-mon[128993]: osdmap e53: 8 total, 8 up, 8 in 2023-07-14T18:12:01.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:01 smithi133 ceph-mon[126384]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:12:01.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:01 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:12:01.653 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:01 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:12:01.653 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:01 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:01.653 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:01 smithi133 ceph-mon[126384]: osdmap e53: 8 total, 8 up, 8 in 2023-07-14T18:12:03.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:02 smithi133 ceph-mon[126384]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:03.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:02 smithi133 ceph-mon[126384]: osdmap e54: 8 total, 8 up, 8 in 2023-07-14T18:12:03.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:02 smithi161 ceph-mon[128993]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:03.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:02 smithi161 ceph-mon[128993]: osdmap e54: 8 total, 8 up, 8 in 2023-07-14T18:12:03.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:02 smithi171 ceph-mon[129241]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:03.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:02 smithi171 ceph-mon[129241]: osdmap e54: 8 total, 8 up, 8 in 2023-07-14T18:12:04.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:03 smithi133 ceph-mon[126384]: osdmap e55: 8 total, 8 up, 8 in 2023-07-14T18:12:04.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:03 smithi133 ceph-mon[126384]: from='client.? 172.21.15.171:0/2564103206' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "87dffc07-bf48-48c8-b418-5c4b509686d7"}]: dispatch 2023-07-14T18:12:04.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:03 smithi133 ceph-mon[126384]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "87dffc07-bf48-48c8-b418-5c4b509686d7"}]: dispatch 2023-07-14T18:12:04.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:03 smithi133 ceph-mon[126384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "87dffc07-bf48-48c8-b418-5c4b509686d7"}]': finished 2023-07-14T18:12:04.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:03 smithi133 ceph-mon[126384]: osdmap e56: 9 total, 8 up, 9 in 2023-07-14T18:12:04.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:03 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T18:12:04.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:03 smithi133 ceph-mon[126384]: from='client.? 172.21.15.171:0/1606875057' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:12:04.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:03 smithi161 ceph-mon[128993]: osdmap e55: 8 total, 8 up, 8 in 2023-07-14T18:12:04.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:03 smithi161 ceph-mon[128993]: from='client.? 172.21.15.171:0/2564103206' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "87dffc07-bf48-48c8-b418-5c4b509686d7"}]: dispatch 2023-07-14T18:12:04.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:03 smithi161 ceph-mon[128993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "87dffc07-bf48-48c8-b418-5c4b509686d7"}]: dispatch 2023-07-14T18:12:04.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:03 smithi161 ceph-mon[128993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "87dffc07-bf48-48c8-b418-5c4b509686d7"}]': finished 2023-07-14T18:12:04.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:03 smithi161 ceph-mon[128993]: osdmap e56: 9 total, 8 up, 9 in 2023-07-14T18:12:04.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:03 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T18:12:04.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:03 smithi161 ceph-mon[128993]: from='client.? 172.21.15.171:0/1606875057' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:12:04.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:03 smithi171 ceph-mon[129241]: osdmap e55: 8 total, 8 up, 8 in 2023-07-14T18:12:04.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:03 smithi171 ceph-mon[129241]: from='client.? 172.21.15.171:0/2564103206' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "87dffc07-bf48-48c8-b418-5c4b509686d7"}]: dispatch 2023-07-14T18:12:04.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:03 smithi171 ceph-mon[129241]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "87dffc07-bf48-48c8-b418-5c4b509686d7"}]: dispatch 2023-07-14T18:12:04.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:03 smithi171 ceph-mon[129241]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "87dffc07-bf48-48c8-b418-5c4b509686d7"}]': finished 2023-07-14T18:12:04.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:03 smithi171 ceph-mon[129241]: osdmap e56: 9 total, 8 up, 9 in 2023-07-14T18:12:04.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:03 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T18:12:04.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:03 smithi171 ceph-mon[129241]: from='client.? 172.21.15.171:0/1606875057' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:12:05.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:04 smithi161 ceph-mon[128993]: pgmap v174: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:05.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:04 smithi171 ceph-mon[129241]: pgmap v174: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:05.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:04 smithi133 ceph-mon[126384]: pgmap v174: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:07.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:06 smithi161 ceph-mon[128993]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:07.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:06 smithi171 ceph-mon[129241]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:07.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:06 smithi133 ceph-mon[126384]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:09.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:08 smithi171 ceph-mon[129241]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:09.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:08 smithi161 ceph-mon[128993]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:09.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:08 smithi133 ceph-mon[126384]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:10.321 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:09 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-14T18:12:10.321 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:09 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:10.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:09 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-14T18:12:10.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:09 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:10.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:09 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-14T18:12:10.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:09 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:11.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:10 smithi171 ceph-mon[129241]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:11.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:10 smithi171 ceph-mon[129241]: Deploying daemon osd.8 on smithi171 2023-07-14T18:12:11.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:10 smithi161 ceph-mon[128993]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:11.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:10 smithi161 ceph-mon[128993]: Deploying daemon osd.8 on smithi171 2023-07-14T18:12:11.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:10 smithi133 ceph-mon[126384]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:11.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:10 smithi133 ceph-mon[126384]: Deploying daemon osd.8 on smithi171 2023-07-14T18:12:13.210 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:12 smithi171 ceph-mon[129241]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:13.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:12 smithi161 ceph-mon[128993]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:13.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:12 smithi133 ceph-mon[126384]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:15.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:14 smithi171 ceph-mon[129241]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:15.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:14 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:15.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:14 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:15.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:14 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:12:15.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:14 smithi161 ceph-mon[128993]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:15.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:14 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:15.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:14 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:15.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:14 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:12:15.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:14 smithi133 ceph-mon[126384]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:15.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:14 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:15.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:14 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:15.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:14 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:12:16.864 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:16.823+0000 7f555dffb700 1 -- 172.21.15.171:0/81587439 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5560068bf0 con 0x7f55500a1150 2023-07-14T18:12:16.864 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 8 on host 'smithi171' 2023-07-14T18:12:16.866 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:16.825+0000 7f5566c22700 1 -- 172.21.15.171:0/81587439 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f55500a1150 msgr2=0x7f55500a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:12:16.866 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:16.825+0000 7f5566c22700 1 --2- 172.21.15.171:0/81587439 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f55500a1150 0x7f55500a3600 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f55600ffc10 tx=0x7f555400b410 comp rx=0 tx=0).stop 2023-07-14T18:12:16.866 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:16.825+0000 7f5566c22700 1 -- 172.21.15.171:0/81587439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55600ffdb0 msgr2=0x7f5560194ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:12:16.866 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:16.825+0000 7f5566c22700 1 --2- 172.21.15.171:0/81587439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55600ffdb0 0x7f5560194ef0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f554c00d950 tx=0x7f554c00dc60 comp rx=0 tx=0).stop 2023-07-14T18:12:16.867 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:16.825+0000 7f5566c22700 1 -- 172.21.15.171:0/81587439 shutdown_connections 2023-07-14T18:12:16.867 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:16.825+0000 7f5566c22700 1 --2- 172.21.15.171:0/81587439 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f55500a1150 0x7f55500a3600 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:12:16.867 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:16.825+0000 7f5566c22700 1 --2- 172.21.15.171:0/81587439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55600febb0 0x7f55601949b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:12:16.867 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:16.825+0000 7f5566c22700 1 --2- 172.21.15.171:0/81587439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5560100740 0x7f556019bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:12:16.867 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:16.825+0000 7f5566c22700 1 --2- 172.21.15.171:0/81587439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55600ffdb0 0x7f5560194ef0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:12:16.867 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:16.825+0000 7f5566c22700 1 -- 172.21.15.171:0/81587439 >> 172.21.15.171:0/81587439 conn(0x7f55600fa1e0 msgr2=0x7f5560103290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:12:16.867 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:16.825+0000 7f5566c22700 1 -- 172.21.15.171:0/81587439 shutdown_connections 2023-07-14T18:12:16.867 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:12:16.825+0000 7f5566c22700 1 -- 172.21.15.171:0/81587439 wait complete. 2023-07-14T18:12:17.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:16 smithi171 ceph-mon[129241]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:17.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:16 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:17.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:16 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:17.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:16 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:17.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:16 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:12:17.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:16 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:17.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:16 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:17.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:16 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:17.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:16 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:12:17.297 DEBUG:teuthology.orchestra.run.smithi171:osd.8> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.8.service 2023-07-14T18:12:17.299 INFO:tasks.cephadm:Deploying osd.9 on smithi133 with /dev/vg_nvme/lv_1... 2023-07-14T18:12:17.299 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-14T18:12:17.327 INFO:journalctl@ceph.osd.8.smithi171.stdout:-- Logs begin at Fri 2023-07-14 17:41:48 UTC. -- 2023-07-14T18:12:17.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:16 smithi161 ceph-mon[128993]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:17.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:16 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:17.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:16 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:17.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:16 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:17.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:16 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:12:17.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:16 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:17.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:16 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:17.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:16 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:17.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:16 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:12:17.368 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:16 smithi133 ceph-mon[126384]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:17.368 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:16 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:17.369 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:16 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:17.369 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:16 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:17.369 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:16 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:12:17.369 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:16 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:17.369 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:16 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:17.369 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:16 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:17.369 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:16 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:12:18.031 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:12:18.112 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:12:17 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:12:17.987+0000 7f6d60b5c640 -1 osd.8 0 log_to_monitors true 2023-07-14T18:12:19.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:18 smithi171 ceph-mon[129241]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:19.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:18 smithi171 ceph-mon[129241]: from='osd.8 [v2:172.21.15.171:6816/2771494483,v1:172.21.15.171:6817/2771494483]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-14T18:12:19.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:18 smithi171 ceph-mon[129241]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-14T18:12:19.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:18 smithi133 ceph-mon[126384]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:19.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:18 smithi133 ceph-mon[126384]: from='osd.8 [v2:172.21.15.171:6816/2771494483,v1:172.21.15.171:6817/2771494483]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-14T18:12:19.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:18 smithi133 ceph-mon[126384]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-14T18:12:19.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:18 smithi161 ceph-mon[128993]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:19.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:18 smithi161 ceph-mon[128993]: from='osd.8 [v2:172.21.15.171:6816/2771494483,v1:172.21.15.171:6817/2771494483]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-14T18:12:19.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:18 smithi161 ceph-mon[128993]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-14T18:12:19.973 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:19 smithi171 ceph-mon[129241]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-14T18:12:19.974 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:19 smithi171 ceph-mon[129241]: osdmap e57: 9 total, 8 up, 9 in 2023-07-14T18:12:19.974 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:19 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T18:12:19.974 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:19 smithi171 ceph-mon[129241]: from='osd.8 [v2:172.21.15.171:6816/2771494483,v1:172.21.15.171:6817/2771494483]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2023-07-14T18:12:19.975 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:19 smithi171 ceph-mon[129241]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2023-07-14T18:12:19.975 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:19 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:19.975 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:19 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:19.976 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:19 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:12:19.976 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:19 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:12:19.976 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:19 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:12:19.976 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:19 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:19.976 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:19 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:19.977 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:19 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:12:19.977 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:19 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:20.044 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-14T18:12:20.044 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-14T18:12:20.044 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2023-07-14T18:12:20.045 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2023-07-14T18:12:20.045 INFO:teuthology.orchestra.run.smithi133.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0557348 s, 188 MB/s 2023-07-14T18:12:20.045 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2023-07-14T18:12:20.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:19 smithi161 ceph-mon[128993]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-14T18:12:20.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:19 smithi161 ceph-mon[128993]: osdmap e57: 9 total, 8 up, 9 in 2023-07-14T18:12:20.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:19 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T18:12:20.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:19 smithi161 ceph-mon[128993]: from='osd.8 [v2:172.21.15.171:6816/2771494483,v1:172.21.15.171:6817/2771494483]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2023-07-14T18:12:20.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:19 smithi161 ceph-mon[128993]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2023-07-14T18:12:20.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:19 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:20.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:19 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:20.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:19 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:12:20.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:19 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:12:20.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:19 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:12:20.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:19 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:20.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:19 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:20.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:19 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:12:20.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:19 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:20.362 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:12:19 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:12:19.936+0000 7f6d561d4700 -1 osd.8 0 waiting for initial osdmap 2023-07-14T18:12:20.362 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:12:19 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:12:19.945+0000 7f6d4dfc2700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T18:12:20.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:19 smithi133 ceph-mon[126384]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-14T18:12:20.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:19 smithi133 ceph-mon[126384]: osdmap e57: 9 total, 8 up, 9 in 2023-07-14T18:12:20.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:19 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T18:12:20.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:19 smithi133 ceph-mon[126384]: from='osd.8 [v2:172.21.15.171:6816/2771494483,v1:172.21.15.171:6817/2771494483]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2023-07-14T18:12:20.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:19 smithi133 ceph-mon[126384]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2023-07-14T18:12:20.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:19 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:20.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:19 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:20.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:19 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:12:20.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:19 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:12:20.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:19 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:12:20.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:19 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:20.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:19 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:20.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:19 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:12:20.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:19 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:20.504 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi133:vg_nvme/lv_1 2023-07-14T18:12:21.226 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:12:21.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:20 smithi161 ceph-mon[128993]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:21.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:20 smithi161 ceph-mon[128993]: Detected new or changed devices on smithi171 2023-07-14T18:12:21.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:20 smithi161 ceph-mon[128993]: Adjusting osd_memory_target on smithi171 to 5722M 2023-07-14T18:12:21.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:20 smithi161 ceph-mon[128993]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2023-07-14T18:12:21.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:20 smithi161 ceph-mon[128993]: osdmap e58: 9 total, 8 up, 9 in 2023-07-14T18:12:21.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:20 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T18:12:21.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:20 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T18:12:21.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:20 smithi171 ceph-mon[129241]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:21.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:20 smithi171 ceph-mon[129241]: Detected new or changed devices on smithi171 2023-07-14T18:12:21.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:20 smithi171 ceph-mon[129241]: Adjusting osd_memory_target on smithi171 to 5722M 2023-07-14T18:12:21.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:20 smithi171 ceph-mon[129241]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2023-07-14T18:12:21.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:20 smithi171 ceph-mon[129241]: osdmap e58: 9 total, 8 up, 9 in 2023-07-14T18:12:21.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:20 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T18:12:21.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:20 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T18:12:21.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:20 smithi133 ceph-mon[126384]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:21.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:20 smithi133 ceph-mon[126384]: Detected new or changed devices on smithi171 2023-07-14T18:12:21.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:20 smithi133 ceph-mon[126384]: Adjusting osd_memory_target on smithi171 to 5722M 2023-07-14T18:12:21.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:20 smithi133 ceph-mon[126384]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2023-07-14T18:12:21.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:20 smithi133 ceph-mon[126384]: osdmap e58: 9 total, 8 up, 9 in 2023-07-14T18:12:21.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:20 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T18:12:21.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:20 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T18:12:22.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:21 smithi133 ceph-mon[126384]: purged_snaps scrub starts 2023-07-14T18:12:22.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:21 smithi133 ceph-mon[126384]: purged_snaps scrub ok 2023-07-14T18:12:22.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:21 smithi133 ceph-mon[126384]: osd.8 [v2:172.21.15.171:6816/2771494483,v1:172.21.15.171:6817/2771494483] boot 2023-07-14T18:12:22.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:21 smithi133 ceph-mon[126384]: osdmap e59: 9 total, 9 up, 9 in 2023-07-14T18:12:22.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:21 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T18:12:22.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:21 smithi161 ceph-mon[128993]: purged_snaps scrub starts 2023-07-14T18:12:22.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:21 smithi161 ceph-mon[128993]: purged_snaps scrub ok 2023-07-14T18:12:22.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:21 smithi161 ceph-mon[128993]: osd.8 [v2:172.21.15.171:6816/2771494483,v1:172.21.15.171:6817/2771494483] boot 2023-07-14T18:12:22.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:21 smithi161 ceph-mon[128993]: osdmap e59: 9 total, 9 up, 9 in 2023-07-14T18:12:22.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:21 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T18:12:22.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:21 smithi171 ceph-mon[129241]: purged_snaps scrub starts 2023-07-14T18:12:22.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:21 smithi171 ceph-mon[129241]: purged_snaps scrub ok 2023-07-14T18:12:22.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:21 smithi171 ceph-mon[129241]: osd.8 [v2:172.21.15.171:6816/2771494483,v1:172.21.15.171:6817/2771494483] boot 2023-07-14T18:12:22.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:21 smithi171 ceph-mon[129241]: osdmap e59: 9 total, 9 up, 9 in 2023-07-14T18:12:22.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:21 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T18:12:22.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.665+0000 7f9e7a0c8700 1 -- 172.21.15.133:0/1285189102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e74100ec0 msgr2=0x7f9e74101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:12:22.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.665+0000 7f9e7a0c8700 1 --2- 172.21.15.133:0/1285189102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e74100ec0 0x7f9e74101310 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9e68009990 tx=0x7f9e68009ca0 comp rx=0 tx=0).stop 2023-07-14T18:12:22.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.665+0000 7f9e7a0c8700 1 -- 172.21.15.133:0/1285189102 shutdown_connections 2023-07-14T18:12:22.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.665+0000 7f9e7a0c8700 1 --2- 172.21.15.133:0/1285189102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e74101850 0x7f9e74108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:12:22.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.665+0000 7f9e7a0c8700 1 --2- 172.21.15.133:0/1285189102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e740ffcc0 0x7f9e741000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:12:22.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.665+0000 7f9e7a0c8700 1 --2- 172.21.15.133:0/1285189102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e74100ec0 0x7f9e74101310 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:12:22.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.665+0000 7f9e7a0c8700 1 -- 172.21.15.133:0/1285189102 >> 172.21.15.133:0/1285189102 conn(0x7f9e740fb2d0 msgr2=0x7f9e740fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:12:22.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.665+0000 7f9e7a0c8700 1 -- 172.21.15.133:0/1285189102 shutdown_connections 2023-07-14T18:12:22.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.665+0000 7f9e7a0c8700 1 -- 172.21.15.133:0/1285189102 wait complete. 2023-07-14T18:12:22.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.666+0000 7f9e7a0c8700 1 Processor -- start 2023-07-14T18:12:22.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.666+0000 7f9e7a0c8700 1 -- start start 2023-07-14T18:12:22.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.667+0000 7f9e7a0c8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e740ffcc0 0x7f9e74197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:12:22.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.667+0000 7f9e7a0c8700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e74100ec0 0x7f9e74198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:12:22.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.667+0000 7f9e7a0c8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e74101850 0x7f9e7419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:12:22.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.667+0000 7f9e7a0c8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7419f760 con 0x7f9e74101850 2023-07-14T18:12:22.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.667+0000 7f9e7a0c8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7419f8a0 con 0x7f9e740ffcc0 2023-07-14T18:12:22.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.667+0000 7f9e7a0c8700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7419fa60 con 0x7f9e74100ec0 2023-07-14T18:12:22.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.667+0000 7f9e73fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e74101850 0x7f9e7419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:12:22.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.667+0000 7f9e737fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e740ffcc0 0x7f9e74197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:12:22.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.667+0000 7f9e73fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e74101850 0x7f9e7419f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58824/0 (socket says 172.21.15.133:58824) 2023-07-14T18:12:22.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.667+0000 7f9e73fff700 1 -- 172.21.15.133:0/2023363832 learned_addr learned my addr 172.21.15.133:0/2023363832 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:12:22.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.667+0000 7f9e72ffd700 1 --2- 172.21.15.133:0/2023363832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e74100ec0 0x7f9e74198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:12:22.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.667+0000 7f9e73fff700 1 -- 172.21.15.133:0/2023363832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e74100ec0 msgr2=0x7f9e74198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:12:22.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.667+0000 7f9e73fff700 1 --2- 172.21.15.133:0/2023363832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e74100ec0 0x7f9e74198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:12:22.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.667+0000 7f9e73fff700 1 -- 172.21.15.133:0/2023363832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e740ffcc0 msgr2=0x7f9e74197c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:12:22.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.667+0000 7f9e73fff700 1 --2- 172.21.15.133:0/2023363832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e740ffcc0 0x7f9e74197c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:12:22.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.667+0000 7f9e73fff700 1 -- 172.21.15.133:0/2023363832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e680095f0 con 0x7f9e74101850 2023-07-14T18:12:22.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.667+0000 7f9e737fe700 1 --2- 172.21.15.133:0/2023363832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e740ffcc0 0x7f9e74197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:12:22.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.668+0000 7f9e73fff700 1 --2- 172.21.15.133:0/2023363832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e74101850 0x7f9e7419f190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00ef30 tx=0x7f9e6c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:12:22.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.668+0000 7f9e70ff9700 1 -- 172.21.15.133:0/2023363832 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c018070 con 0x7f9e74101850 2023-07-14T18:12:22.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.668+0000 7f9e70ff9700 1 -- 172.21.15.133:0/2023363832 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e6c00f040 con 0x7f9e74101850 2023-07-14T18:12:22.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.668+0000 7f9e7a0c8700 1 -- 172.21.15.133:0/2023363832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e7419fce0 con 0x7f9e74101850 2023-07-14T18:12:22.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.668+0000 7f9e70ff9700 1 -- 172.21.15.133:0/2023363832 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c00cd00 con 0x7f9e74101850 2023-07-14T18:12:22.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.668+0000 7f9e7a0c8700 1 -- 172.21.15.133:0/2023363832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e741a0180 con 0x7f9e74101850 2023-07-14T18:12:22.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.669+0000 7f9e7a0c8700 1 -- 172.21.15.133:0/2023363832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e7410b540 con 0x7f9e74101850 2023-07-14T18:12:22.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.671+0000 7f9e70ff9700 1 -- 172.21.15.133:0/2023363832 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f9e6c003a00 con 0x7f9e74101850 2023-07-14T18:12:22.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.672+0000 7f9e70ff9700 1 --2- 172.21.15.133:0/2023363832 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f9e5c0a11d0 0x7f9e5c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:12:22.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.672+0000 7f9e70ff9700 1 -- 172.21.15.133:0/2023363832 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f9e6c0cbac0 con 0x7f9e74101850 2023-07-14T18:12:22.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.672+0000 7f9e737fe700 1 --2- 172.21.15.133:0/2023363832 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f9e5c0a11d0 0x7f9e5c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:12:22.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.673+0000 7f9e737fe700 1 --2- 172.21.15.133:0/2023363832 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f9e5c0a11d0 0x7f9e5c0a3680 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9e741990f0 tx=0x7f9e64009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:12:22.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.676+0000 7f9e70ff9700 1 -- 172.21.15.133:0/2023363832 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f9e6c095b90 con 0x7f9e74101850 2023-07-14T18:12:22.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:22.881+0000 7f9e7a0c8700 1 -- 172.21.15.133:0/2023363832 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e74107800 con 0x7f9e5c0a11d0 2023-07-14T18:12:23.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:22 smithi161 ceph-mon[128993]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:23.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:22 smithi161 ceph-mon[128993]: osdmap e60: 9 total, 9 up, 9 in 2023-07-14T18:12:23.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:22 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:12:23.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:22 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:12:23.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:22 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:23.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:22 smithi171 ceph-mon[129241]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:23.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:22 smithi171 ceph-mon[129241]: osdmap e60: 9 total, 9 up, 9 in 2023-07-14T18:12:23.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:22 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:12:23.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:22 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:12:23.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:22 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:23.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:22 smithi133 ceph-mon[126384]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-14T18:12:23.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:22 smithi133 ceph-mon[126384]: osdmap e60: 9 total, 9 up, 9 in 2023-07-14T18:12:23.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:22 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:12:23.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:22 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:12:23.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:22 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:24.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:23 smithi161 ceph-mon[128993]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:12:24.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:23 smithi161 ceph-mon[128993]: osdmap e61: 9 total, 9 up, 9 in 2023-07-14T18:12:24.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:23 smithi171 ceph-mon[129241]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:12:24.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:23 smithi171 ceph-mon[129241]: osdmap e61: 9 total, 9 up, 9 in 2023-07-14T18:12:24.376 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:23 smithi133 ceph-mon[126384]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:12:24.377 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:23 smithi133 ceph-mon[126384]: osdmap e61: 9 total, 9 up, 9 in 2023-07-14T18:12:25.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:24 smithi161 ceph-mon[128993]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T18:12:25.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:24 smithi161 ceph-mon[128993]: osdmap e62: 9 total, 9 up, 9 in 2023-07-14T18:12:25.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:24 smithi171 ceph-mon[129241]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T18:12:25.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:24 smithi171 ceph-mon[129241]: osdmap e62: 9 total, 9 up, 9 in 2023-07-14T18:12:25.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:24 smithi133 ceph-mon[126384]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T18:12:25.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:24 smithi133 ceph-mon[126384]: osdmap e62: 9 total, 9 up, 9 in 2023-07-14T18:12:26.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:25 smithi161 ceph-mon[128993]: osdmap e63: 9 total, 9 up, 9 in 2023-07-14T18:12:26.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:25 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/89828175' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0832870-79ce-4bce-8045-58b7a7f6dc69"}]: dispatch 2023-07-14T18:12:26.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:25 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/89828175' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0832870-79ce-4bce-8045-58b7a7f6dc69"}]': finished 2023-07-14T18:12:26.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:25 smithi161 ceph-mon[128993]: osdmap e64: 10 total, 9 up, 10 in 2023-07-14T18:12:26.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:25 smithi171 ceph-mon[129241]: osdmap e63: 9 total, 9 up, 9 in 2023-07-14T18:12:26.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:25 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/89828175' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0832870-79ce-4bce-8045-58b7a7f6dc69"}]: dispatch 2023-07-14T18:12:26.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:25 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/89828175' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0832870-79ce-4bce-8045-58b7a7f6dc69"}]': finished 2023-07-14T18:12:26.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:25 smithi171 ceph-mon[129241]: osdmap e64: 10 total, 9 up, 10 in 2023-07-14T18:12:26.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:25 smithi133 ceph-mon[126384]: osdmap e63: 9 total, 9 up, 9 in 2023-07-14T18:12:26.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:25 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/89828175' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0832870-79ce-4bce-8045-58b7a7f6dc69"}]: dispatch 2023-07-14T18:12:26.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:25 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/89828175' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0832870-79ce-4bce-8045-58b7a7f6dc69"}]': finished 2023-07-14T18:12:26.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:25 smithi133 ceph-mon[126384]: osdmap e64: 10 total, 9 up, 10 in 2023-07-14T18:12:27.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:26 smithi161 ceph-mon[128993]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T18:12:27.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:26 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T18:12:27.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:26 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/4270128494' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:12:27.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:26 smithi171 ceph-mon[129241]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T18:12:27.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:26 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T18:12:27.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:26 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/4270128494' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:12:27.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:26 smithi133 ceph-mon[126384]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T18:12:27.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:26 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T18:12:27.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:26 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/4270128494' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:12:28.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:27 smithi161 ceph-mon[128993]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 129 KiB/s, 0 objects/s recovering 2023-07-14T18:12:28.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:27 smithi171 ceph-mon[129241]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 129 KiB/s, 0 objects/s recovering 2023-07-14T18:12:28.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:27 smithi133 ceph-mon[126384]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 129 KiB/s, 0 objects/s recovering 2023-07-14T18:12:31.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:30 smithi133 ceph-mon[126384]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T18:12:31.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:30 smithi161 ceph-mon[128993]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T18:12:31.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:30 smithi171 ceph-mon[129241]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T18:12:33.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:32 smithi133 ceph-mon[126384]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-07-14T18:12:33.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:32 smithi161 ceph-mon[128993]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-07-14T18:12:33.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:32 smithi171 ceph-mon[129241]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-07-14T18:12:34.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:33 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-14T18:12:34.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:33 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:34.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:33 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-14T18:12:34.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:33 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:34.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:33 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-14T18:12:34.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:33 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:35.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:34 smithi133 ceph-mon[126384]: Deploying daemon osd.9 on smithi133 2023-07-14T18:12:35.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:34 smithi133 ceph-mon[126384]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-07-14T18:12:35.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:34 smithi161 ceph-mon[128993]: Deploying daemon osd.9 on smithi133 2023-07-14T18:12:35.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:34 smithi161 ceph-mon[128993]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-07-14T18:12:35.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:34 smithi171 ceph-mon[129241]: Deploying daemon osd.9 on smithi133 2023-07-14T18:12:35.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:34 smithi171 ceph-mon[129241]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-07-14T18:12:37.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:36 smithi133 ceph-mon[126384]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T18:12:37.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:36 smithi161 ceph-mon[128993]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T18:12:37.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:36 smithi171 ceph-mon[129241]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T18:12:39.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:38 smithi133 ceph-mon[126384]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T18:12:39.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:38 smithi161 ceph-mon[128993]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T18:12:39.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:38 smithi171 ceph-mon[129241]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T18:12:40.765 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:40 smithi133 ceph-mon[126384]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T18:12:40.766 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:40 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:40.766 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:40 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:40.766 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:40 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:12:40.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:40 smithi161 ceph-mon[128993]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T18:12:40.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:40 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:40.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:40 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:40.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:40 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:12:40.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:40 smithi171 ceph-mon[129241]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T18:12:40.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:40 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:40.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:40 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:40.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:40 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:12:42.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:42 smithi161 ceph-mon[128993]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T18:12:42.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:42 smithi171 ceph-mon[129241]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T18:12:42.903 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:42 smithi133 ceph-mon[126384]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T18:12:43.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:42.997+0000 7f9e70ff9700 1 -- 172.21.15.133:0/2023363832 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9e74107800 con 0x7f9e5c0a11d0 2023-07-14T18:12:43.039 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 9 on host 'smithi133' 2023-07-14T18:12:43.040 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:43.000+0000 7f9e7a0c8700 1 -- 172.21.15.133:0/2023363832 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f9e5c0a11d0 msgr2=0x7f9e5c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:12:43.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:43.000+0000 7f9e7a0c8700 1 --2- 172.21.15.133:0/2023363832 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f9e5c0a11d0 0x7f9e5c0a3680 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9e741990f0 tx=0x7f9e64009260 comp rx=0 tx=0).stop 2023-07-14T18:12:43.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:43.000+0000 7f9e7a0c8700 1 -- 172.21.15.133:0/2023363832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e74101850 msgr2=0x7f9e7419f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:12:43.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:43.000+0000 7f9e7a0c8700 1 --2- 172.21.15.133:0/2023363832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e74101850 0x7f9e7419f190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00ef30 tx=0x7f9e6c00c3a0 comp rx=0 tx=0).stop 2023-07-14T18:12:43.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:43.000+0000 7f9e7a0c8700 1 -- 172.21.15.133:0/2023363832 shutdown_connections 2023-07-14T18:12:43.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:43.000+0000 7f9e7a0c8700 1 --2- 172.21.15.133:0/2023363832 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f9e5c0a11d0 0x7f9e5c0a3680 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:12:43.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:43.000+0000 7f9e7a0c8700 1 --2- 172.21.15.133:0/2023363832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e740ffcc0 0x7f9e74197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:12:43.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:43.000+0000 7f9e7a0c8700 1 --2- 172.21.15.133:0/2023363832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e74101850 0x7f9e7419f190 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:12:43.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:43.000+0000 7f9e7a0c8700 1 --2- 172.21.15.133:0/2023363832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e74100ec0 0x7f9e74198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:12:43.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:43.000+0000 7f9e7a0c8700 1 -- 172.21.15.133:0/2023363832 >> 172.21.15.133:0/2023363832 conn(0x7f9e740fb2d0 msgr2=0x7f9e740fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:12:43.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:43.000+0000 7f9e7a0c8700 1 -- 172.21.15.133:0/2023363832 shutdown_connections 2023-07-14T18:12:43.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:12:43.000+0000 7f9e7a0c8700 1 -- 172.21.15.133:0/2023363832 wait complete. 2023-07-14T18:12:43.552 DEBUG:teuthology.orchestra.run.smithi133:osd.9> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.9.service 2023-07-14T18:12:43.556 INFO:tasks.cephadm:Deploying osd.10 on smithi161 with /dev/vg_nvme/lv_1... 2023-07-14T18:12:43.557 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-14T18:12:43.632 INFO:journalctl@ceph.osd.9.smithi133.stdout:-- Logs begin at Fri 2023-07-14 17:46:17 UTC. -- 2023-07-14T18:12:43.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:43 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:43.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:43 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:43.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:43 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:43.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:43 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:12:43.848 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:43 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:43.848 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:43 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:43.848 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:43 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:12:43.848 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:43 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:43.848 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:43 smithi161 ceph-mon[128993]: from='osd.9 [v2:172.21.15.133:6826/2066164149,v1:172.21.15.133:6827/2066164149]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-14T18:12:43.848 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:43 smithi161 ceph-mon[128993]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-14T18:12:43.903 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:43 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:43.903 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:43 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:43.904 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:43 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:43.904 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:43 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:12:43.904 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:43 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:43.904 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:43 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:43.904 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:43 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:12:43.905 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:43 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:43.905 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:43 smithi133 ceph-mon[126384]: from='osd.9 [v2:172.21.15.133:6826/2066164149,v1:172.21.15.133:6827/2066164149]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-14T18:12:43.905 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:43 smithi133 ceph-mon[126384]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-14T18:12:44.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:43 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:44.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:43 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:44.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:43 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:44.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:43 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:12:44.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:43 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:44.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:43 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:44.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:43 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:12:44.115 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:43 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:44.115 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:43 smithi171 ceph-mon[129241]: from='osd.9 [v2:172.21.15.133:6826/2066164149,v1:172.21.15.133:6827/2066164149]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-14T18:12:44.115 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:43 smithi171 ceph-mon[129241]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-14T18:12:44.123 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T18:12:45.252 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:44 smithi161 ceph-mon[128993]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T18:12:45.252 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:44 smithi161 ceph-mon[128993]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-14T18:12:45.252 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:44 smithi161 ceph-mon[128993]: osdmap e65: 10 total, 9 up, 10 in 2023-07-14T18:12:45.252 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:44 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T18:12:45.253 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:44 smithi161 ceph-mon[128993]: from='osd.9 [v2:172.21.15.133:6826/2066164149,v1:172.21.15.133:6827/2066164149]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T18:12:45.253 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:44 smithi161 ceph-mon[128993]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T18:12:45.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:44 smithi171 ceph-mon[129241]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T18:12:45.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:44 smithi171 ceph-mon[129241]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-14T18:12:45.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:44 smithi171 ceph-mon[129241]: osdmap e65: 10 total, 9 up, 10 in 2023-07-14T18:12:45.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:44 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T18:12:45.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:44 smithi171 ceph-mon[129241]: from='osd.9 [v2:172.21.15.133:6826/2066164149,v1:172.21.15.133:6827/2066164149]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T18:12:45.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:44 smithi171 ceph-mon[129241]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T18:12:45.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:44 smithi133 ceph-mon[126384]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T18:12:45.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:44 smithi133 ceph-mon[126384]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-14T18:12:45.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:44 smithi133 ceph-mon[126384]: osdmap e65: 10 total, 9 up, 10 in 2023-07-14T18:12:45.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:44 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T18:12:45.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:44 smithi133 ceph-mon[126384]: from='osd.9 [v2:172.21.15.133:6826/2066164149,v1:172.21.15.133:6827/2066164149]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T18:12:45.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:44 smithi133 ceph-mon[126384]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T18:12:45.404 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:12:44 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:12:44.959+0000 7f7ef7fba700 -1 osd.9 0 waiting for initial osdmap 2023-07-14T18:12:45.404 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:12:44 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:12:44.974+0000 7f7ef25ad700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T18:12:45.704 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-14T18:12:45.704 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-14T18:12:45.705 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2023-07-14T18:12:45.705 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2023-07-14T18:12:45.705 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111033 s, 944 MB/s 2023-07-14T18:12:45.705 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2023-07-14T18:12:46.176 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi161:vg_nvme/lv_1 2023-07-14T18:12:46.302 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:45 smithi133 ceph-mon[126384]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-07-14T18:12:46.302 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:45 smithi133 ceph-mon[126384]: osdmap e66: 10 total, 9 up, 10 in 2023-07-14T18:12:46.302 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:45 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T18:12:46.302 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:45 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T18:12:46.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:45 smithi161 ceph-mon[128993]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-07-14T18:12:46.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:45 smithi161 ceph-mon[128993]: osdmap e66: 10 total, 9 up, 10 in 2023-07-14T18:12:46.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:45 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T18:12:46.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:45 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T18:12:46.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:45 smithi171 ceph-mon[129241]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-07-14T18:12:46.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:45 smithi171 ceph-mon[129241]: osdmap e66: 10 total, 9 up, 10 in 2023-07-14T18:12:46.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:45 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T18:12:46.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:45 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T18:12:46.725 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T18:12:47.216 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:46 smithi161 ceph-mon[128993]: purged_snaps scrub starts 2023-07-14T18:12:47.217 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:46 smithi161 ceph-mon[128993]: purged_snaps scrub ok 2023-07-14T18:12:47.217 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:46 smithi161 ceph-mon[128993]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T18:12:47.218 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:46 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T18:12:47.218 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:46 smithi161 ceph-mon[128993]: osd.9 [v2:172.21.15.133:6826/2066164149,v1:172.21.15.133:6827/2066164149] boot 2023-07-14T18:12:47.218 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:46 smithi161 ceph-mon[128993]: osdmap e67: 10 total, 10 up, 10 in 2023-07-14T18:12:47.218 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:46 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T18:12:47.218 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:46 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:47.218 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:46 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:47.219 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:46 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:12:47.219 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:46 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:12:47.219 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:46 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:12:47.219 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:46 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:12:47.219 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:46 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:47.219 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:46 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:47.219 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:46 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:12:47.220 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:46 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:47.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:46 smithi171 ceph-mon[129241]: purged_snaps scrub starts 2023-07-14T18:12:47.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:46 smithi171 ceph-mon[129241]: purged_snaps scrub ok 2023-07-14T18:12:47.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:46 smithi171 ceph-mon[129241]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T18:12:47.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:46 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T18:12:47.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:46 smithi171 ceph-mon[129241]: osd.9 [v2:172.21.15.133:6826/2066164149,v1:172.21.15.133:6827/2066164149] boot 2023-07-14T18:12:47.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:46 smithi171 ceph-mon[129241]: osdmap e67: 10 total, 10 up, 10 in 2023-07-14T18:12:47.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:46 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T18:12:47.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:46 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:47.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:46 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:47.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:46 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:12:47.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:46 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:12:47.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:46 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:12:47.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:46 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:12:47.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:46 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:47.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:46 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:47.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:46 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:12:47.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:46 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:47.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:46 smithi133 ceph-mon[126384]: purged_snaps scrub starts 2023-07-14T18:12:47.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:46 smithi133 ceph-mon[126384]: purged_snaps scrub ok 2023-07-14T18:12:47.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:46 smithi133 ceph-mon[126384]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T18:12:47.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:46 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T18:12:47.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:46 smithi133 ceph-mon[126384]: osd.9 [v2:172.21.15.133:6826/2066164149,v1:172.21.15.133:6827/2066164149] boot 2023-07-14T18:12:47.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:46 smithi133 ceph-mon[126384]: osdmap e67: 10 total, 10 up, 10 in 2023-07-14T18:12:47.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:46 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T18:12:47.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:46 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:47.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:46 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:47.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:46 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:12:47.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:46 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:12:47.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:46 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:12:47.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:46 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:12:47.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:46 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:47.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:46 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:47.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:46 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:12:47.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:46 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:12:47.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.932+0000 7efe72b2f700 1 -- 172.21.15.161:0/2898683585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe6c100ee0 msgr2=0x7efe6c101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:12:47.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.932+0000 7efe72b2f700 1 --2- 172.21.15.161:0/2898683585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe6c100ee0 0x7efe6c101330 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efe58009990 tx=0x7efe58009ca0 comp rx=0 tx=0).stop 2023-07-14T18:12:47.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.932+0000 7efe72b2f700 1 -- 172.21.15.161:0/2898683585 shutdown_connections 2023-07-14T18:12:47.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.932+0000 7efe72b2f700 1 --2- 172.21.15.161:0/2898683585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe6c101870 0x7efe6c108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:12:47.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.932+0000 7efe72b2f700 1 --2- 172.21.15.161:0/2898683585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe6c0ffce0 0x7efe6c1000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:12:47.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.932+0000 7efe72b2f700 1 --2- 172.21.15.161:0/2898683585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe6c100ee0 0x7efe6c101330 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:12:47.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.932+0000 7efe72b2f700 1 -- 172.21.15.161:0/2898683585 >> 172.21.15.161:0/2898683585 conn(0x7efe6c0fb2f0 msgr2=0x7efe6c0fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:12:47.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.932+0000 7efe72b2f700 1 -- 172.21.15.161:0/2898683585 shutdown_connections 2023-07-14T18:12:47.975 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.932+0000 7efe72b2f700 1 -- 172.21.15.161:0/2898683585 wait complete. 2023-07-14T18:12:47.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.933+0000 7efe72b2f700 1 Processor -- start 2023-07-14T18:12:47.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.933+0000 7efe72b2f700 1 -- start start 2023-07-14T18:12:47.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.933+0000 7efe72b2f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe6c0ffce0 0x7efe6c195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:12:47.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.933+0000 7efe72b2f700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe6c100ee0 0x7efe6c195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:12:47.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.933+0000 7efe72b2f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe6c101870 0x7efe6c19d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:12:47.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.933+0000 7efe72b2f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c19d630 con 0x7efe6c0ffce0 2023-07-14T18:12:47.976 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.933+0000 7efe72b2f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c19d770 con 0x7efe6c101870 2023-07-14T18:12:47.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.933+0000 7efe72b2f700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c19d900 con 0x7efe6c100ee0 2023-07-14T18:12:47.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.933+0000 7efe710cc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe6c101870 0x7efe6c19d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:12:47.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.933+0000 7efe710cc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe6c101870 0x7efe6c19d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54982/0 (socket says 172.21.15.161:54982) 2023-07-14T18:12:47.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.933+0000 7efe710cc700 1 -- 172.21.15.161:0/349948992 learned_addr learned my addr 172.21.15.161:0/349948992 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T18:12:47.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.933+0000 7efe6bfff700 1 --2- 172.21.15.161:0/349948992 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe6c100ee0 0x7efe6c195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:12:47.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.934+0000 7efe710cc700 1 -- 172.21.15.161:0/349948992 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe6c100ee0 msgr2=0x7efe6c195fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:12:47.977 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.934+0000 7efe710cc700 1 --2- 172.21.15.161:0/349948992 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe6c100ee0 0x7efe6c195fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:12:47.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.934+0000 7efe710cc700 1 -- 172.21.15.161:0/349948992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe6c0ffce0 msgr2=0x7efe6c195aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:12:47.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.934+0000 7efe710cc700 1 --2- 172.21.15.161:0/349948992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe6c0ffce0 0x7efe6c195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:12:47.978 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.934+0000 7efe710cc700 1 -- 172.21.15.161:0/349948992 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe580095f0 con 0x7efe6c101870 2023-07-14T18:12:47.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.934+0000 7efe710cc700 1 --2- 172.21.15.161:0/349948992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe6c101870 0x7efe6c19d060 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7efe6400ada0 tx=0x7efe6400c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:12:47.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.934+0000 7efe69ffb700 1 -- 172.21.15.161:0/349948992 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6400f070 con 0x7efe6c101870 2023-07-14T18:12:47.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.934+0000 7efe69ffb700 1 -- 172.21.15.161:0/349948992 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe64007b30 con 0x7efe6c101870 2023-07-14T18:12:47.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.934+0000 7efe72b2f700 1 -- 172.21.15.161:0/349948992 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe6c19dbe0 con 0x7efe6c101870 2023-07-14T18:12:47.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.934+0000 7efe69ffb700 1 -- 172.21.15.161:0/349948992 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe640053e0 con 0x7efe6c101870 2023-07-14T18:12:47.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.934+0000 7efe72b2f700 1 -- 172.21.15.161:0/349948992 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe6c19e090 con 0x7efe6c101870 2023-07-14T18:12:47.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.935+0000 7efe72b2f700 1 -- 172.21.15.161:0/349948992 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe6c1072d0 con 0x7efe6c101870 2023-07-14T18:12:47.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.936+0000 7efe69ffb700 1 -- 172.21.15.161:0/349948992 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7efe6400c980 con 0x7efe6c101870 2023-07-14T18:12:47.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.936+0000 7efe69ffb700 1 --2- 172.21.15.161:0/349948992 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efe5c0a1150 0x7efe5c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:12:47.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.936+0000 7efe69ffb700 1 -- 172.21.15.161:0/349948992 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7efe640cb6f0 con 0x7efe6c101870 2023-07-14T18:12:47.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.937+0000 7efe708cb700 1 --2- 172.21.15.161:0/349948992 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efe5c0a1150 0x7efe5c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:12:47.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.937+0000 7efe708cb700 1 --2- 172.21.15.161:0/349948992 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efe5c0a1150 0x7efe5c0a3600 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efe6c100d40 tx=0x7efe60006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:12:47.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:47.941+0000 7efe69ffb700 1 -- 172.21.15.161:0/349948992 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7efe640956d0 con 0x7efe6c101870 2023-07-14T18:12:48.176 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:12:48.134+0000 7efe72b2f700 1 -- 172.21.15.161:0/349948992 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7efe6c060a80 con 0x7efe5c0a1150 2023-07-14T18:12:48.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:47 smithi161 ceph-mon[128993]: Detected new or changed devices on smithi133 2023-07-14T18:12:48.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:47 smithi161 ceph-mon[128993]: Adjusting osd_memory_target on smithi133 to 4316M 2023-07-14T18:12:48.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:47 smithi161 ceph-mon[128993]: osdmap e68: 10 total, 10 up, 10 in 2023-07-14T18:12:48.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:47 smithi171 ceph-mon[129241]: Detected new or changed devices on smithi133 2023-07-14T18:12:48.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:47 smithi171 ceph-mon[129241]: Adjusting osd_memory_target on smithi133 to 4316M 2023-07-14T18:12:48.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:47 smithi171 ceph-mon[129241]: osdmap e68: 10 total, 10 up, 10 in 2023-07-14T18:12:48.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:47 smithi133 ceph-mon[126384]: Detected new or changed devices on smithi133 2023-07-14T18:12:48.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:47 smithi133 ceph-mon[126384]: Adjusting osd_memory_target on smithi133 to 4316M 2023-07-14T18:12:48.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:47 smithi133 ceph-mon[126384]: osdmap e68: 10 total, 10 up, 10 in 2023-07-14T18:12:49.233 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:48 smithi161 ceph-mon[128993]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:12:49.233 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:12:49.233 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:12:49.234 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:49.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:48 smithi171 ceph-mon[129241]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:12:49.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:12:49.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:12:49.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:49.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:48 smithi133 ceph-mon[126384]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:12:49.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:12:49.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:12:49.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:50.097 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:49 smithi161 ceph-mon[128993]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:12:50.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:49 smithi171 ceph-mon[129241]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:12:50.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:49 smithi133 ceph-mon[126384]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:12:51.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:50 smithi161 ceph-mon[128993]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:12:51.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:50 smithi161 ceph-mon[128993]: from='client.? 172.21.15.161:0/1445690371' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "081f0484-d134-4b2e-9c48-ef8e6a646422"}]: dispatch 2023-07-14T18:12:51.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:50 smithi161 ceph-mon[128993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "081f0484-d134-4b2e-9c48-ef8e6a646422"}]: dispatch 2023-07-14T18:12:51.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:50 smithi161 ceph-mon[128993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "081f0484-d134-4b2e-9c48-ef8e6a646422"}]': finished 2023-07-14T18:12:51.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:50 smithi161 ceph-mon[128993]: osdmap e69: 11 total, 10 up, 11 in 2023-07-14T18:12:51.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:50 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T18:12:51.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:50 smithi171 ceph-mon[129241]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:12:51.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:50 smithi171 ceph-mon[129241]: from='client.? 172.21.15.161:0/1445690371' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "081f0484-d134-4b2e-9c48-ef8e6a646422"}]: dispatch 2023-07-14T18:12:51.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:50 smithi171 ceph-mon[129241]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "081f0484-d134-4b2e-9c48-ef8e6a646422"}]: dispatch 2023-07-14T18:12:51.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:50 smithi171 ceph-mon[129241]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "081f0484-d134-4b2e-9c48-ef8e6a646422"}]': finished 2023-07-14T18:12:51.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:50 smithi171 ceph-mon[129241]: osdmap e69: 11 total, 10 up, 11 in 2023-07-14T18:12:51.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:50 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T18:12:51.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:50 smithi133 ceph-mon[126384]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:12:51.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:50 smithi133 ceph-mon[126384]: from='client.? 172.21.15.161:0/1445690371' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "081f0484-d134-4b2e-9c48-ef8e6a646422"}]: dispatch 2023-07-14T18:12:51.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:50 smithi133 ceph-mon[126384]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "081f0484-d134-4b2e-9c48-ef8e6a646422"}]: dispatch 2023-07-14T18:12:51.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:50 smithi133 ceph-mon[126384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "081f0484-d134-4b2e-9c48-ef8e6a646422"}]': finished 2023-07-14T18:12:51.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:50 smithi133 ceph-mon[126384]: osdmap e69: 11 total, 10 up, 11 in 2023-07-14T18:12:51.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:50 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T18:12:52.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:51 smithi161 ceph-mon[128993]: from='client.? 172.21.15.161:0/2937740908' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:12:52.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:51 smithi171 ceph-mon[129241]: from='client.? 172.21.15.161:0/2937740908' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:12:52.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:51 smithi133 ceph-mon[126384]: from='client.? 172.21.15.161:0/2937740908' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:12:53.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:52 smithi161 ceph-mon[128993]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:12:53.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:52 smithi171 ceph-mon[129241]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:12:53.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:52 smithi133 ceph-mon[126384]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:12:54.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:53 smithi161 ceph-mon[128993]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:12:54.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:53 smithi171 ceph-mon[129241]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:12:54.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:53 smithi133 ceph-mon[126384]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:12:57.097 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:56 smithi161 ceph-mon[128993]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:12:57.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:56 smithi133 ceph-mon[126384]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:12:57.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:56 smithi171 ceph-mon[129241]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:12:57.981 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:57 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-14T18:12:57.982 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:57 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:58.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:57 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-14T18:12:58.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:57 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:58.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:57 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-14T18:12:58.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:57 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:12:59.098 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:58 smithi161 ceph-mon[128993]: Deploying daemon osd.10 on smithi161 2023-07-14T18:12:59.098 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:12:58 smithi161 ceph-mon[128993]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:12:59.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:58 smithi133 ceph-mon[126384]: Deploying daemon osd.10 on smithi161 2023-07-14T18:12:59.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:12:58 smithi133 ceph-mon[126384]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:12:59.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:58 smithi171 ceph-mon[129241]: Deploying daemon osd.10 on smithi161 2023-07-14T18:12:59.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:12:58 smithi171 ceph-mon[129241]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:13:01.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:00 smithi133 ceph-mon[126384]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:13:01.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:00 smithi161 ceph-mon[128993]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:13:01.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:00 smithi171 ceph-mon[129241]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:13:02.573 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:02 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:02.574 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:02 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:02.574 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:02 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:13:02.574 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:02 smithi161 ceph-mon[128993]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:13:02.654 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:02 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:02.655 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:02 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:02.655 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:02 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:13:02.655 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:02 smithi133 ceph-mon[126384]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:13:02.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:02 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:02.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:02 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:02.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:02 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:13:02.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:02 smithi171 ceph-mon[129241]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:13:04.238 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 10 on host 'smithi161' 2023-07-14T18:13:04.238 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:13:04.196+0000 7efe69ffb700 1 -- 172.21.15.161:0/349948992 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7efe6c060a80 con 0x7efe5c0a1150 2023-07-14T18:13:04.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:13:04.199+0000 7efe72b2f700 1 -- 172.21.15.161:0/349948992 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efe5c0a1150 msgr2=0x7efe5c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:04.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:13:04.199+0000 7efe72b2f700 1 --2- 172.21.15.161:0/349948992 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efe5c0a1150 0x7efe5c0a3600 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efe6c100d40 tx=0x7efe60006d20 comp rx=0 tx=0).stop 2023-07-14T18:13:04.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:13:04.199+0000 7efe72b2f700 1 -- 172.21.15.161:0/349948992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe6c101870 msgr2=0x7efe6c19d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:04.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:13:04.199+0000 7efe72b2f700 1 --2- 172.21.15.161:0/349948992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe6c101870 0x7efe6c19d060 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7efe6400ada0 tx=0x7efe6400c330 comp rx=0 tx=0).stop 2023-07-14T18:13:04.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:13:04.199+0000 7efe72b2f700 1 -- 172.21.15.161:0/349948992 shutdown_connections 2023-07-14T18:13:04.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:13:04.199+0000 7efe72b2f700 1 --2- 172.21.15.161:0/349948992 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efe5c0a1150 0x7efe5c0a3600 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:04.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:13:04.199+0000 7efe72b2f700 1 --2- 172.21.15.161:0/349948992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe6c101870 0x7efe6c19d060 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:04.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:13:04.199+0000 7efe72b2f700 1 --2- 172.21.15.161:0/349948992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe6c0ffce0 0x7efe6c195aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:04.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:13:04.199+0000 7efe72b2f700 1 --2- 172.21.15.161:0/349948992 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe6c100ee0 0x7efe6c195fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:04.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:13:04.199+0000 7efe72b2f700 1 -- 172.21.15.161:0/349948992 >> 172.21.15.161:0/349948992 conn(0x7efe6c0fb2f0 msgr2=0x7efe6c104410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:04.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:13:04.199+0000 7efe72b2f700 1 -- 172.21.15.161:0/349948992 shutdown_connections 2023-07-14T18:13:04.242 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T18:13:04.199+0000 7efe72b2f700 1 -- 172.21.15.161:0/349948992 wait complete. 2023-07-14T18:13:04.636 DEBUG:teuthology.orchestra.run.smithi161:osd.10> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.10.service 2023-07-14T18:13:04.639 INFO:tasks.cephadm:Deploying osd.11 on smithi171 with /dev/vg_nvme/lv_1... 2023-07-14T18:13:04.639 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-14T18:13:04.670 INFO:journalctl@ceph.osd.10.smithi161.stdout:-- Logs begin at Fri 2023-07-14 17:41:43 UTC. -- 2023-07-14T18:13:05.097 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:04 smithi161 ceph-mon[128993]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:13:05.098 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:04 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:05.098 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:04 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:05.098 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:04 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:05.098 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:04 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:13:05.098 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:04 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:05.098 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:04 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:05.099 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:04 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:05.099 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:04 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:13:05.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:04 smithi171 ceph-mon[129241]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:13:05.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:04 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:05.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:04 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:05.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:04 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:05.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:04 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:13:05.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:04 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:05.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:04 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:05.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:04 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:05.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:04 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:13:05.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:04 smithi133 ceph-mon[126384]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:13:05.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:04 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:05.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:04 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:05.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:04 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:05.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:04 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:13:05.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:04 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:05.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:04 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:05.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:04 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:05.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:04 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:13:05.203 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T18:13:05.464 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:13:05 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:13:05.283+0000 7ff6340a6640 -1 osd.10 0 log_to_monitors true 2023-07-14T18:13:06.012 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:05 smithi171 ceph-mon[129241]: from='osd.10 [v2:172.21.15.161:6824/2162333626,v1:172.21.15.161:6825/2162333626]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T18:13:06.012 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:05 smithi171 ceph-mon[129241]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T18:13:06.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:05 smithi133 ceph-mon[126384]: from='osd.10 [v2:172.21.15.161:6824/2162333626,v1:172.21.15.161:6825/2162333626]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T18:13:06.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:05 smithi133 ceph-mon[126384]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T18:13:06.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:05 smithi161 ceph-mon[128993]: from='osd.10 [v2:172.21.15.161:6824/2162333626,v1:172.21.15.161:6825/2162333626]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T18:13:06.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:05 smithi161 ceph-mon[128993]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T18:13:06.808 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-14T18:13:06.808 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-14T18:13:06.808 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2023-07-14T18:13:06.808 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2023-07-14T18:13:06.808 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113217 s, 926 MB/s 2023-07-14T18:13:06.809 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2023-07-14T18:13:07.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:06 smithi171 ceph-mon[129241]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:13:07.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:06 smithi171 ceph-mon[129241]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-14T18:13:07.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:06 smithi171 ceph-mon[129241]: osdmap e70: 11 total, 10 up, 11 in 2023-07-14T18:13:07.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:06 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T18:13:07.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:06 smithi171 ceph-mon[129241]: from='osd.10 [v2:172.21.15.161:6824/2162333626,v1:172.21.15.161:6825/2162333626]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T18:13:07.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:06 smithi171 ceph-mon[129241]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T18:13:07.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:06 smithi133 ceph-mon[126384]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:13:07.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:06 smithi133 ceph-mon[126384]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-14T18:13:07.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:06 smithi133 ceph-mon[126384]: osdmap e70: 11 total, 10 up, 11 in 2023-07-14T18:13:07.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:06 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T18:13:07.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:06 smithi133 ceph-mon[126384]: from='osd.10 [v2:172.21.15.161:6824/2162333626,v1:172.21.15.161:6825/2162333626]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T18:13:07.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:06 smithi133 ceph-mon[126384]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T18:13:07.257 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi171:vg_nvme/lv_1 2023-07-14T18:13:07.347 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:13:06 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:13:06.861+0000 7ff62971e700 -1 osd.10 0 waiting for initial osdmap 2023-07-14T18:13:07.348 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:13:06 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:13:06.872+0000 7ff622d0f700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T18:13:07.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:06 smithi161 ceph-mon[128993]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:13:07.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:06 smithi161 ceph-mon[128993]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-14T18:13:07.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:06 smithi161 ceph-mon[128993]: osdmap e70: 11 total, 10 up, 11 in 2023-07-14T18:13:07.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:06 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T18:13:07.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:06 smithi161 ceph-mon[128993]: from='osd.10 [v2:172.21.15.161:6824/2162333626,v1:172.21.15.161:6825/2162333626]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T18:13:07.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:06 smithi161 ceph-mon[128993]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T18:13:07.811 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T18:13:08.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:07 smithi133 ceph-mon[126384]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T18:13:08.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:07 smithi133 ceph-mon[126384]: osdmap e71: 11 total, 10 up, 11 in 2023-07-14T18:13:08.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:07 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T18:13:08.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:07 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T18:13:08.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:07 smithi133 ceph-mon[126384]: Detected new or changed devices on smithi161 2023-07-14T18:13:08.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:07 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:08.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:07 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:08.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:07 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:13:08.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:07 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:13:08.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:07 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:13:08.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:07 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:13:08.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:07 smithi133 ceph-mon[126384]: Adjusting osd_memory_target on smithi161 to 4292M 2023-07-14T18:13:08.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:07 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:08.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:07 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:08.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:07 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:13:08.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:07 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:08.218 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:07 smithi171 ceph-mon[129241]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T18:13:08.218 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:07 smithi171 ceph-mon[129241]: osdmap e71: 11 total, 10 up, 11 in 2023-07-14T18:13:08.218 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:07 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T18:13:08.218 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:07 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T18:13:08.218 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:07 smithi171 ceph-mon[129241]: Detected new or changed devices on smithi161 2023-07-14T18:13:08.219 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:07 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:08.219 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:07 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:08.219 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:07 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:13:08.219 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:07 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:13:08.219 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:07 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:13:08.219 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:07 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:13:08.220 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:07 smithi171 ceph-mon[129241]: Adjusting osd_memory_target on smithi161 to 4292M 2023-07-14T18:13:08.220 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:07 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:08.220 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:07 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:08.220 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:07 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:13:08.220 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:07 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:08.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:07 smithi161 ceph-mon[128993]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T18:13:08.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:07 smithi161 ceph-mon[128993]: osdmap e71: 11 total, 10 up, 11 in 2023-07-14T18:13:08.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:07 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T18:13:08.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:07 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T18:13:08.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:07 smithi161 ceph-mon[128993]: Detected new or changed devices on smithi161 2023-07-14T18:13:08.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:07 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:08.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:07 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:08.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:07 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:13:08.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:07 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:13:08.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:07 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:13:08.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:07 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:13:08.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:07 smithi161 ceph-mon[128993]: Adjusting osd_memory_target on smithi161 to 4292M 2023-07-14T18:13:08.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:07 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:08.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:07 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:08.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:07 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:13:08.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:07 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:09.107 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.064+0000 7f1822592700 1 -- 172.21.15.171:0/1966717968 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f181c0febb0 msgr2=0x7f181c0fefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:09.107 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.065+0000 7f1822592700 1 --2- 172.21.15.171:0/1966717968 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f181c0febb0 0x7f181c0fefc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f180c009990 tx=0x7f180c009ca0 comp rx=0 tx=0).stop 2023-07-14T18:13:09.107 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.065+0000 7f1822592700 1 -- 172.21.15.171:0/1966717968 shutdown_connections 2023-07-14T18:13:09.108 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.065+0000 7f1822592700 1 --2- 172.21.15.171:0/1966717968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f181c100740 0x7f181c107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:09.108 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.065+0000 7f1822592700 1 --2- 172.21.15.171:0/1966717968 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f181c0febb0 0x7f181c0fefc0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:09.108 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.065+0000 7f1822592700 1 --2- 172.21.15.171:0/1966717968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f181c0ffdb0 0x7f181c100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:09.108 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.065+0000 7f1822592700 1 -- 172.21.15.171:0/1966717968 >> 172.21.15.171:0/1966717968 conn(0x7f181c0fa1e0 msgr2=0x7f181c0fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:09.108 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.065+0000 7f1822592700 1 -- 172.21.15.171:0/1966717968 shutdown_connections 2023-07-14T18:13:09.108 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.065+0000 7f1822592700 1 -- 172.21.15.171:0/1966717968 wait complete. 2023-07-14T18:13:09.108 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.065+0000 7f1822592700 1 Processor -- start 2023-07-14T18:13:09.109 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.065+0000 7f1822592700 1 -- start start 2023-07-14T18:13:09.111 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.066+0000 7f1822592700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f181c0febb0 0x7f181c195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:09.112 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.066+0000 7f1822592700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f181c0ffdb0 0x7f181c195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:09.112 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.066+0000 7f1822592700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f181c100740 0x7f181c19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:09.112 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.066+0000 7f1822592700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181c19d4e0 con 0x7f181c0ffdb0 2023-07-14T18:13:09.113 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.066+0000 7f1822592700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181c19d620 con 0x7f181c0febb0 2023-07-14T18:13:09.113 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.066+0000 7f1822592700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181c19d7b0 con 0x7f181c100740 2023-07-14T18:13:09.113 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.066+0000 7f1820b2f700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f181c100740 0x7f181c19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:09.114 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.066+0000 7f1820b2f700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f181c100740 0x7f181c19cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:43122/0 (socket says 172.21.15.171:43122) 2023-07-14T18:13:09.114 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.066+0000 7f1820b2f700 1 -- 172.21.15.171:0/738238555 learned_addr learned my addr 172.21.15.171:0/738238555 (peer_addr_for_me v2:172.21.15.171:0/0) 2023-07-14T18:13:09.114 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.066+0000 7f181b7fe700 1 --2- 172.21.15.171:0/738238555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f181c0ffdb0 0x7f181c195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:09.114 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.066+0000 7f181bfff700 1 --2- 172.21.15.171:0/738238555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f181c0febb0 0x7f181c195950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:09.115 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.066+0000 7f1820b2f700 1 -- 172.21.15.171:0/738238555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f181c0febb0 msgr2=0x7f181c195950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:09.115 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.066+0000 7f1820b2f700 1 --2- 172.21.15.171:0/738238555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f181c0febb0 0x7f181c195950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:09.115 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.066+0000 7f1820b2f700 1 -- 172.21.15.171:0/738238555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f181c0ffdb0 msgr2=0x7f181c195e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:09.115 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.066+0000 7f1820b2f700 1 --2- 172.21.15.171:0/738238555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f181c0ffdb0 0x7f181c195e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:09.115 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.066+0000 7f1820b2f700 1 -- 172.21.15.171:0/738238555 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180c0095f0 con 0x7f181c100740 2023-07-14T18:13:09.116 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.066+0000 7f1820b2f700 1 --2- 172.21.15.171:0/738238555 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f181c100740 0x7f181c19cf10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f181400add0 tx=0x7f181400c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:09.116 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.066+0000 7f18197fa700 1 -- 172.21.15.171:0/738238555 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181400f070 con 0x7f181c100740 2023-07-14T18:13:09.116 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.066+0000 7f18197fa700 1 -- 172.21.15.171:0/738238555 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1814007b30 con 0x7f181c100740 2023-07-14T18:13:09.116 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.066+0000 7f1822592700 1 -- 172.21.15.171:0/738238555 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f181c19da90 con 0x7f181c100740 2023-07-14T18:13:09.116 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.066+0000 7f18197fa700 1 -- 172.21.15.171:0/738238555 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18140053e0 con 0x7f181c100740 2023-07-14T18:13:09.116 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.066+0000 7f1822592700 1 -- 172.21.15.171:0/738238555 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f181c19df40 con 0x7f181c100740 2023-07-14T18:13:09.116 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.068+0000 7f1822592700 1 -- 172.21.15.171:0/738238555 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f181c10a370 con 0x7f181c100740 2023-07-14T18:13:09.117 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.068+0000 7f18197fa700 1 -- 172.21.15.171:0/738238555 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f181400c940 con 0x7f181c100740 2023-07-14T18:13:09.117 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.069+0000 7f18197fa700 1 --2- 172.21.15.171:0/738238555 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f18040a1150 0x7f18040a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:09.117 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.069+0000 7f18197fa700 1 -- 172.21.15.171:0/738238555 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f18140cb9d0 con 0x7f181c100740 2023-07-14T18:13:09.117 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.069+0000 7f181bfff700 1 --2- 172.21.15.171:0/738238555 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f18040a1150 0x7f18040a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:09.118 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.070+0000 7f181bfff700 1 --2- 172.21.15.171:0/738238555 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f18040a1150 0x7f18040a3600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f180c0095c0 tx=0x7f180c009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:09.118 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.073+0000 7f18197fa700 1 -- 172.21.15.171:0/738238555 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f18140957e0 con 0x7f181c100740 2023-07-14T18:13:09.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:08 smithi133 ceph-mon[126384]: purged_snaps scrub starts 2023-07-14T18:13:09.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:08 smithi133 ceph-mon[126384]: purged_snaps scrub ok 2023-07-14T18:13:09.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:08 smithi133 ceph-mon[126384]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:13:09.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:08 smithi133 ceph-mon[126384]: osd.10 [v2:172.21.15.161:6824/2162333626,v1:172.21.15.161:6825/2162333626] boot 2023-07-14T18:13:09.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:08 smithi133 ceph-mon[126384]: osdmap e72: 11 total, 11 up, 11 in 2023-07-14T18:13:09.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:08 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T18:13:09.307 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:09.265+0000 7f1822592700 1 -- 172.21.15.171:0/738238555 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f181c0fb110 con 0x7f18040a1150 2023-07-14T18:13:09.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:08 smithi161 ceph-mon[128993]: purged_snaps scrub starts 2023-07-14T18:13:09.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:08 smithi161 ceph-mon[128993]: purged_snaps scrub ok 2023-07-14T18:13:09.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:08 smithi161 ceph-mon[128993]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:13:09.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:08 smithi161 ceph-mon[128993]: osd.10 [v2:172.21.15.161:6824/2162333626,v1:172.21.15.161:6825/2162333626] boot 2023-07-14T18:13:09.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:08 smithi161 ceph-mon[128993]: osdmap e72: 11 total, 11 up, 11 in 2023-07-14T18:13:09.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:08 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T18:13:09.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:08 smithi171 ceph-mon[129241]: purged_snaps scrub starts 2023-07-14T18:13:09.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:08 smithi171 ceph-mon[129241]: purged_snaps scrub ok 2023-07-14T18:13:09.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:08 smithi171 ceph-mon[129241]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-14T18:13:09.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:08 smithi171 ceph-mon[129241]: osd.10 [v2:172.21.15.161:6824/2162333626,v1:172.21.15.161:6825/2162333626] boot 2023-07-14T18:13:09.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:08 smithi171 ceph-mon[129241]: osdmap e72: 11 total, 11 up, 11 in 2023-07-14T18:13:09.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:08 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T18:13:10.169 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:09 smithi171 ceph-mon[129241]: osdmap e73: 11 total, 11 up, 11 in 2023-07-14T18:13:10.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:09 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:13:10.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:09 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:13:10.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:09 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:10.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:09 smithi161 ceph-mon[128993]: osdmap e73: 11 total, 11 up, 11 in 2023-07-14T18:13:10.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:09 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:13:10.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:09 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:13:10.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:09 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:10.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:09 smithi133 ceph-mon[126384]: osdmap e73: 11 total, 11 up, 11 in 2023-07-14T18:13:10.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:09 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T18:13:10.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:09 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T18:13:10.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:09 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:10.977 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:10 smithi171 ceph-mon[129241]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:13:10.977 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:10 smithi171 ceph-mon[129241]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T18:13:10.978 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:10 smithi171 ceph-mon[129241]: osdmap e74: 11 total, 11 up, 11 in 2023-07-14T18:13:11.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:10 smithi161 ceph-mon[128993]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:13:11.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:10 smithi161 ceph-mon[128993]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T18:13:11.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:10 smithi161 ceph-mon[128993]: osdmap e74: 11 total, 11 up, 11 in 2023-07-14T18:13:11.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:10 smithi133 ceph-mon[126384]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:13:11.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:10 smithi133 ceph-mon[126384]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T18:13:11.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:10 smithi133 ceph-mon[126384]: osdmap e74: 11 total, 11 up, 11 in 2023-07-14T18:13:12.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:11 smithi161 ceph-mon[128993]: osdmap e75: 11 total, 11 up, 11 in 2023-07-14T18:13:12.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:11 smithi161 ceph-mon[128993]: from='client.? 172.21.15.171:0/3502745570' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa"}]: dispatch 2023-07-14T18:13:12.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:11 smithi161 ceph-mon[128993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa"}]: dispatch 2023-07-14T18:13:12.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:11 smithi161 ceph-mon[128993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa"}]': finished 2023-07-14T18:13:12.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:11 smithi161 ceph-mon[128993]: osdmap e76: 12 total, 11 up, 12 in 2023-07-14T18:13:12.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:11 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T18:13:12.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:11 smithi171 ceph-mon[129241]: osdmap e75: 11 total, 11 up, 11 in 2023-07-14T18:13:12.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:11 smithi171 ceph-mon[129241]: from='client.? 172.21.15.171:0/3502745570' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa"}]: dispatch 2023-07-14T18:13:12.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:11 smithi171 ceph-mon[129241]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa"}]: dispatch 2023-07-14T18:13:12.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:11 smithi171 ceph-mon[129241]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa"}]': finished 2023-07-14T18:13:12.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:11 smithi171 ceph-mon[129241]: osdmap e76: 12 total, 11 up, 12 in 2023-07-14T18:13:12.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:11 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T18:13:12.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:11 smithi133 ceph-mon[126384]: osdmap e75: 11 total, 11 up, 11 in 2023-07-14T18:13:12.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:11 smithi133 ceph-mon[126384]: from='client.? 172.21.15.171:0/3502745570' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa"}]: dispatch 2023-07-14T18:13:12.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:11 smithi133 ceph-mon[126384]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa"}]: dispatch 2023-07-14T18:13:12.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:11 smithi133 ceph-mon[126384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa"}]': finished 2023-07-14T18:13:12.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:11 smithi133 ceph-mon[126384]: osdmap e76: 12 total, 11 up, 12 in 2023-07-14T18:13:12.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:11 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T18:13:13.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:12 smithi161 ceph-mon[128993]: pgmap v227: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-07-14T18:13:13.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:12 smithi161 ceph-mon[128993]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-07-14T18:13:13.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:12 smithi161 ceph-mon[128993]: from='client.? 172.21.15.171:0/671243440' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:13:13.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:12 smithi171 ceph-mon[129241]: pgmap v227: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-07-14T18:13:13.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:12 smithi171 ceph-mon[129241]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-07-14T18:13:13.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:12 smithi171 ceph-mon[129241]: from='client.? 172.21.15.171:0/671243440' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:13:13.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:12 smithi133 ceph-mon[126384]: pgmap v227: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-07-14T18:13:13.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:12 smithi133 ceph-mon[126384]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-07-14T18:13:13.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:12 smithi133 ceph-mon[126384]: from='client.? 172.21.15.171:0/671243440' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T18:13:15.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:14 smithi161 ceph-mon[128993]: pgmap v229: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-07-14T18:13:15.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:14 smithi171 ceph-mon[129241]: pgmap v229: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-07-14T18:13:15.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:14 smithi133 ceph-mon[126384]: pgmap v229: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-07-14T18:13:16.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:15 smithi171 ceph-mon[129241]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-07-14T18:13:16.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:15 smithi171 ceph-mon[129241]: Cluster is now healthy 2023-07-14T18:13:16.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:15 smithi161 ceph-mon[128993]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-07-14T18:13:16.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:15 smithi161 ceph-mon[128993]: Cluster is now healthy 2023-07-14T18:13:16.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:15 smithi133 ceph-mon[126384]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-07-14T18:13:16.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:15 smithi133 ceph-mon[126384]: Cluster is now healthy 2023-07-14T18:13:17.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:16 smithi161 ceph-mon[128993]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T18:13:17.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:16 smithi171 ceph-mon[129241]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T18:13:17.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:16 smithi133 ceph-mon[126384]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T18:13:19.334 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:18 smithi171 ceph-mon[129241]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2023-07-14T18:13:19.335 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:18 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-14T18:13:19.335 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:18 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:19.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:18 smithi161 ceph-mon[128993]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2023-07-14T18:13:19.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:18 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-14T18:13:19.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:18 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:19.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:18 smithi133 ceph-mon[126384]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2023-07-14T18:13:19.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:18 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-14T18:13:19.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:18 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:20.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:19 smithi161 ceph-mon[128993]: Deploying daemon osd.11 on smithi171 2023-07-14T18:13:20.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:19 smithi171 ceph-mon[129241]: Deploying daemon osd.11 on smithi171 2023-07-14T18:13:20.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:19 smithi133 ceph-mon[126384]: Deploying daemon osd.11 on smithi171 2023-07-14T18:13:21.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:20 smithi171 ceph-mon[129241]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-07-14T18:13:21.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:20 smithi161 ceph-mon[128993]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-07-14T18:13:21.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:20 smithi133 ceph-mon[126384]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-07-14T18:13:23.071 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:22 smithi171 ceph-mon[129241]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-14T18:13:23.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:22 smithi161 ceph-mon[128993]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-14T18:13:23.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:22 smithi133 ceph-mon[126384]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-14T18:13:24.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:24 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:24.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:24 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:24.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:24 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:13:24.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:24 smithi171 ceph-mon[129241]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-07-14T18:13:24.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:24 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:24.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:24 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:24.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:24 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:13:24.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:24 smithi133 ceph-mon[126384]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-07-14T18:13:24.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:24 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:24.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:24 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:24.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:24 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:13:24.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:24 smithi161 ceph-mon[128993]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-07-14T18:13:25.923 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 11 on host 'smithi171' 2023-07-14T18:13:25.923 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:25.880+0000 7f18197fa700 1 -- 172.21.15.171:0/738238555 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f181c0fb110 con 0x7f18040a1150 2023-07-14T18:13:25.925 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:25.883+0000 7f1822592700 1 -- 172.21.15.171:0/738238555 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f18040a1150 msgr2=0x7f18040a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:25.925 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:25.883+0000 7f1822592700 1 --2- 172.21.15.171:0/738238555 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f18040a1150 0x7f18040a3600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f180c0095c0 tx=0x7f180c009530 comp rx=0 tx=0).stop 2023-07-14T18:13:25.925 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:25.883+0000 7f1822592700 1 -- 172.21.15.171:0/738238555 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f181c100740 msgr2=0x7f181c19cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:25.925 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:25.883+0000 7f1822592700 1 --2- 172.21.15.171:0/738238555 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f181c100740 0x7f181c19cf10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f181400add0 tx=0x7f181400c330 comp rx=0 tx=0).stop 2023-07-14T18:13:25.926 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:25.883+0000 7f1822592700 1 -- 172.21.15.171:0/738238555 shutdown_connections 2023-07-14T18:13:25.926 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:25.883+0000 7f1822592700 1 --2- 172.21.15.171:0/738238555 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f18040a1150 0x7f18040a3600 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:25.926 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:25.883+0000 7f1822592700 1 --2- 172.21.15.171:0/738238555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f181c0febb0 0x7f181c195950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:25.926 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:25.883+0000 7f1822592700 1 --2- 172.21.15.171:0/738238555 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f181c100740 0x7f181c19cf10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:25.926 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:25.883+0000 7f1822592700 1 --2- 172.21.15.171:0/738238555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f181c0ffdb0 0x7f181c195e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:25.926 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:25.883+0000 7f1822592700 1 -- 172.21.15.171:0/738238555 >> 172.21.15.171:0/738238555 conn(0x7f181c0fa1e0 msgr2=0x7f181c0faa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:25.927 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:25.883+0000 7f1822592700 1 -- 172.21.15.171:0/738238555 shutdown_connections 2023-07-14T18:13:25.927 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:25.883+0000 7f1822592700 1 -- 172.21.15.171:0/738238555 wait complete. 2023-07-14T18:13:26.398 DEBUG:teuthology.orchestra.run.smithi171:osd.11> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.11.service 2023-07-14T18:13:26.400 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-07-14T18:13:26.401 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph osd stat -f json 2023-07-14T18:13:26.429 INFO:journalctl@ceph.osd.11.smithi171.stdout:-- Logs begin at Fri 2023-07-14 17:41:48 UTC. -- 2023-07-14T18:13:27.113 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:13:26 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:13:26.996+0000 7fc9353bd640 -1 osd.11 0 log_to_monitors true 2023-07-14T18:13:27.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:26 smithi171 ceph-mon[129241]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-14T18:13:27.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:26 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:27.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:26 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:27.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:26 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:27.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:26 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:27.126 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:13:27.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:26 smithi133 ceph-mon[126384]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-14T18:13:27.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:26 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:27.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:26 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:27.158 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:26 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:27.158 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:26 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:27.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:26 smithi161 ceph-mon[128993]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-14T18:13:27.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:26 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:27.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:26 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:27.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:26 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:27.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:26 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:27.904 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:27 smithi133 ceph-mon[126384]: from='osd.11 [v2:172.21.15.171:6824/3403894313,v1:172.21.15.171:6825/3403894313]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T18:13:27.904 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:27 smithi133 ceph-mon[126384]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T18:13:28.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:27 smithi171 ceph-mon[129241]: from='osd.11 [v2:172.21.15.171:6824/3403894313,v1:172.21.15.171:6825/3403894313]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T18:13:28.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:27 smithi171 ceph-mon[129241]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T18:13:28.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:27 smithi161 ceph-mon[128993]: from='osd.11 [v2:172.21.15.171:6824/3403894313,v1:172.21.15.171:6825/3403894313]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T18:13:28.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:27 smithi161 ceph-mon[128993]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T18:13:28.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.501+0000 7f043485c700 1 -- 172.21.15.133:0/1727281586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04300ffcc0 msgr2=0x7f04301000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:28.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.501+0000 7f043485c700 1 --2- 172.21.15.133:0/1727281586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04300ffcc0 0x7f04301000d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0420009990 tx=0x7f0420009ca0 comp rx=0 tx=0).stop 2023-07-14T18:13:28.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.501+0000 7f043485c700 1 -- 172.21.15.133:0/1727281586 shutdown_connections 2023-07-14T18:13:28.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.501+0000 7f043485c700 1 --2- 172.21.15.133:0/1727281586 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0430101850 0x7f0430108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:28.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.501+0000 7f043485c700 1 --2- 172.21.15.133:0/1727281586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04300ffcc0 0x7f04301000d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:28.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.501+0000 7f043485c700 1 --2- 172.21.15.133:0/1727281586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0430100ec0 0x7f0430101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:28.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.501+0000 7f043485c700 1 -- 172.21.15.133:0/1727281586 >> 172.21.15.133:0/1727281586 conn(0x7f04300fb2d0 msgr2=0x7f04300fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:28.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.501+0000 7f043485c700 1 -- 172.21.15.133:0/1727281586 shutdown_connections 2023-07-14T18:13:28.545 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.501+0000 7f043485c700 1 -- 172.21.15.133:0/1727281586 wait complete. 2023-07-14T18:13:28.546 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.502+0000 7f043485c700 1 Processor -- start 2023-07-14T18:13:28.546 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.502+0000 7f043485c700 1 -- start start 2023-07-14T18:13:28.546 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.502+0000 7f043485c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04300ffcc0 0x7f0430197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:28.546 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.502+0000 7f043485c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0430100ec0 0x7f0430198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:28.547 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.503+0000 7f043485c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0430101850 0x7f043019f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:28.547 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.503+0000 7f043485c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043019f790 con 0x7f0430100ec0 2023-07-14T18:13:28.547 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.503+0000 7f043485c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043019f8d0 con 0x7f0430101850 2023-07-14T18:13:28.547 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.503+0000 7f043485c700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043019fa90 con 0x7f04300ffcc0 2023-07-14T18:13:28.547 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.503+0000 7f042dd9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0430100ec0 0x7f0430198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:28.547 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.503+0000 7f042dd9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0430100ec0 0x7f0430198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50110/0 (socket says 172.21.15.133:50110) 2023-07-14T18:13:28.548 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.503+0000 7f042dd9b700 1 -- 172.21.15.133:0/95051990 learned_addr learned my addr 172.21.15.133:0/95051990 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:13:28.548 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.503+0000 7f042ed9d700 1 --2- 172.21.15.133:0/95051990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0430101850 0x7f043019f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:28.548 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.503+0000 7f042e59c700 1 --2- 172.21.15.133:0/95051990 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04300ffcc0 0x7f0430197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:28.548 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.503+0000 7f042dd9b700 1 -- 172.21.15.133:0/95051990 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04300ffcc0 msgr2=0x7f0430197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:28.548 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.503+0000 7f042dd9b700 1 --2- 172.21.15.133:0/95051990 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04300ffcc0 0x7f0430197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:28.548 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.503+0000 7f042dd9b700 1 -- 172.21.15.133:0/95051990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0430101850 msgr2=0x7f043019f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:28.548 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.503+0000 7f042dd9b700 1 --2- 172.21.15.133:0/95051990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0430101850 0x7f043019f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:28.548 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.503+0000 7f042dd9b700 1 -- 172.21.15.133:0/95051990 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04200095f0 con 0x7f0430100ec0 2023-07-14T18:13:28.549 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.503+0000 7f042dd9b700 1 --2- 172.21.15.133:0/95051990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0430100ec0 0x7f0430198180 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f042400d980 tx=0x7f042400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:28.549 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.504+0000 7f041f7fe700 1 -- 172.21.15.133:0/95051990 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0424004220 con 0x7f0430100ec0 2023-07-14T18:13:28.549 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.504+0000 7f041f7fe700 1 -- 172.21.15.133:0/95051990 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0424007640 con 0x7f0430100ec0 2023-07-14T18:13:28.549 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.504+0000 7f041f7fe700 1 -- 172.21.15.133:0/95051990 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0424005690 con 0x7f0430100ec0 2023-07-14T18:13:28.549 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.504+0000 7f043485c700 1 -- 172.21.15.133:0/95051990 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f043019fd70 con 0x7f0430100ec0 2023-07-14T18:13:28.549 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.504+0000 7f043485c700 1 -- 172.21.15.133:0/95051990 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04301a01c0 con 0x7f0430100ec0 2023-07-14T18:13:28.550 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.505+0000 7f043485c700 1 -- 172.21.15.133:0/95051990 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0430192390 con 0x7f0430100ec0 2023-07-14T18:13:28.550 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.507+0000 7f041f7fe700 1 -- 172.21.15.133:0/95051990 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f0424004750 con 0x7f0430100ec0 2023-07-14T18:13:28.550 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.508+0000 7f041f7fe700 1 --2- 172.21.15.133:0/95051990 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f04180a11d0 0x7f04180a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:28.551 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.508+0000 7f041f7fe700 1 -- 172.21.15.133:0/95051990 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f04240cc180 con 0x7f0430100ec0 2023-07-14T18:13:28.551 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.508+0000 7f042e59c700 1 --2- 172.21.15.133:0/95051990 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f04180a11d0 0x7f04180a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:28.551 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.509+0000 7f042e59c700 1 --2- 172.21.15.133:0/95051990 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f04180a11d0 0x7f04180a3680 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0420000c00 tx=0x7f0420005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:28.554 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.512+0000 7f041f7fe700 1 -- 172.21.15.133:0/95051990 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f0424095e00 con 0x7f0430100ec0 2023-07-14T18:13:28.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.708+0000 7f043485c700 1 -- 172.21.15.133:0/95051990 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f043002cd00 con 0x7f0430100ec0 2023-07-14T18:13:28.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.708+0000 7f041f7fe700 1 -- 172.21.15.133:0/95051990 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) v1 ==== 74+0+133 (secure 0 0 0) 0x7f0424099420 con 0x7f0430100ec0 2023-07-14T18:13:28.751 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:13:28.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.711+0000 7f043485c700 1 -- 172.21.15.133:0/95051990 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f04180a11d0 msgr2=0x7f04180a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:28.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.711+0000 7f043485c700 1 --2- 172.21.15.133:0/95051990 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f04180a11d0 0x7f04180a3680 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0420000c00 tx=0x7f0420005fb0 comp rx=0 tx=0).stop 2023-07-14T18:13:28.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.711+0000 7f043485c700 1 -- 172.21.15.133:0/95051990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0430100ec0 msgr2=0x7f0430198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:28.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.711+0000 7f043485c700 1 --2- 172.21.15.133:0/95051990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0430100ec0 0x7f0430198180 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f042400d980 tx=0x7f042400dc90 comp rx=0 tx=0).stop 2023-07-14T18:13:28.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.711+0000 7f043485c700 1 -- 172.21.15.133:0/95051990 shutdown_connections 2023-07-14T18:13:28.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.711+0000 7f043485c700 1 --2- 172.21.15.133:0/95051990 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f04180a11d0 0x7f04180a3680 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:28.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.711+0000 7f043485c700 1 --2- 172.21.15.133:0/95051990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0430101850 0x7f043019f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:28.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.711+0000 7f043485c700 1 --2- 172.21.15.133:0/95051990 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04300ffcc0 0x7f0430197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:28.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.711+0000 7f043485c700 1 --2- 172.21.15.133:0/95051990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0430100ec0 0x7f0430198180 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:28.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.711+0000 7f043485c700 1 -- 172.21.15.133:0/95051990 >> 172.21.15.133:0/95051990 conn(0x7f04300fb2d0 msgr2=0x7f0430106800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:28.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.711+0000 7f043485c700 1 -- 172.21.15.133:0/95051990 shutdown_connections 2023-07-14T18:13:28.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:28.711+0000 7f043485c700 1 -- 172.21.15.133:0/95051990 wait complete. 2023-07-14T18:13:29.113 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:13:28 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:13:28.841+0000 7fc92aa35700 -1 osd.11 0 waiting for initial osdmap 2023-07-14T18:13:29.114 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:13:28 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:13:28.854+0000 7fc924827700 -1 osd.11 78 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T18:13:29.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:28 smithi171 ceph-mon[129241]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-14T18:13:29.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:28 smithi171 ceph-mon[129241]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-14T18:13:29.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:28 smithi171 ceph-mon[129241]: osdmap e77: 12 total, 11 up, 12 in 2023-07-14T18:13:29.115 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T18:13:29.115 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:28 smithi171 ceph-mon[129241]: from='osd.11 [v2:172.21.15.171:6824/3403894313,v1:172.21.15.171:6825/3403894313]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2023-07-14T18:13:29.115 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:28 smithi171 ceph-mon[129241]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2023-07-14T18:13:29.115 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:29.115 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:29.115 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:28 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/95051990' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T18:13:29.115 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:13:29.116 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:13:29.116 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:13:29.116 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:13:29.116 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:29.116 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:29.116 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:13:29.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:29.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:28 smithi133 ceph-mon[126384]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-14T18:13:29.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:28 smithi133 ceph-mon[126384]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-14T18:13:29.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:28 smithi133 ceph-mon[126384]: osdmap e77: 12 total, 11 up, 12 in 2023-07-14T18:13:29.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:28 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T18:13:29.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:28 smithi133 ceph-mon[126384]: from='osd.11 [v2:172.21.15.171:6824/3403894313,v1:172.21.15.171:6825/3403894313]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2023-07-14T18:13:29.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:28 smithi133 ceph-mon[126384]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2023-07-14T18:13:29.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:28 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:29.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:28 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:29.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:28 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/95051990' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T18:13:29.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:28 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:13:29.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:28 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:13:29.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:28 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:13:29.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:28 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:13:29.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:28 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:29.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:28 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:29.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:28 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:13:29.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:28 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:29.298 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":77,"num_osds":12,"num_up_osds":11,"osd_up_since":1689358387,"num_in_osds":12,"osd_in_since":1689358391,"num_remapped_pgs":0} 2023-07-14T18:13:29.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:28 smithi161 ceph-mon[128993]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-14T18:13:29.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:28 smithi161 ceph-mon[128993]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-14T18:13:29.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:28 smithi161 ceph-mon[128993]: osdmap e77: 12 total, 11 up, 12 in 2023-07-14T18:13:29.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:28 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T18:13:29.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:28 smithi161 ceph-mon[128993]: from='osd.11 [v2:172.21.15.171:6824/3403894313,v1:172.21.15.171:6825/3403894313]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2023-07-14T18:13:29.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:28 smithi161 ceph-mon[128993]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2023-07-14T18:13:29.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:28 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:29.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:28 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:29.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:28 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/95051990' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T18:13:29.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:28 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:13:29.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:28 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:13:29.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:28 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:13:29.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:28 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:13:29.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:28 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:29.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:28 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:29.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:28 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:13:29.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:28 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:30.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:29 smithi133 ceph-mon[126384]: Detected new or changed devices on smithi171 2023-07-14T18:13:30.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:29 smithi133 ceph-mon[126384]: Adjusting osd_memory_target on smithi171 to 4292M 2023-07-14T18:13:30.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:29 smithi133 ceph-mon[126384]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2023-07-14T18:13:30.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:29 smithi133 ceph-mon[126384]: osdmap e78: 12 total, 11 up, 12 in 2023-07-14T18:13:30.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:29 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T18:13:30.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:29 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T18:13:30.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:29 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:13:30.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:29 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:30.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:29 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:13:30.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:29 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:30.299 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph osd stat -f json 2023-07-14T18:13:30.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:29 smithi161 ceph-mon[128993]: Detected new or changed devices on smithi171 2023-07-14T18:13:30.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:29 smithi161 ceph-mon[128993]: Adjusting osd_memory_target on smithi171 to 4292M 2023-07-14T18:13:30.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:29 smithi161 ceph-mon[128993]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2023-07-14T18:13:30.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:29 smithi161 ceph-mon[128993]: osdmap e78: 12 total, 11 up, 12 in 2023-07-14T18:13:30.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:29 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T18:13:30.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:29 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T18:13:30.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:29 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:13:30.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:29 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:30.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:29 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:13:30.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:29 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:30.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:29 smithi171 ceph-mon[129241]: Detected new or changed devices on smithi171 2023-07-14T18:13:30.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:29 smithi171 ceph-mon[129241]: Adjusting osd_memory_target on smithi171 to 4292M 2023-07-14T18:13:30.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:29 smithi171 ceph-mon[129241]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2023-07-14T18:13:30.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:29 smithi171 ceph-mon[129241]: osdmap e78: 12 total, 11 up, 12 in 2023-07-14T18:13:30.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:29 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T18:13:30.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:29 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T18:13:30.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:29 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:13:30.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:29 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:30.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:29 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:13:30.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:29 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:31.029 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:13:31.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:30 smithi133 ceph-mon[126384]: purged_snaps scrub starts 2023-07-14T18:13:31.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:30 smithi133 ceph-mon[126384]: purged_snaps scrub ok 2023-07-14T18:13:31.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:30 smithi133 ceph-mon[126384]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-14T18:13:31.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:30 smithi133 ceph-mon[126384]: osd.11 [v2:172.21.15.171:6824/3403894313,v1:172.21.15.171:6825/3403894313] boot 2023-07-14T18:13:31.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:30 smithi133 ceph-mon[126384]: osdmap e79: 12 total, 12 up, 12 in 2023-07-14T18:13:31.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:30 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T18:13:31.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:30 smithi161 ceph-mon[128993]: purged_snaps scrub starts 2023-07-14T18:13:31.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:30 smithi161 ceph-mon[128993]: purged_snaps scrub ok 2023-07-14T18:13:31.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:30 smithi161 ceph-mon[128993]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-14T18:13:31.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:30 smithi161 ceph-mon[128993]: osd.11 [v2:172.21.15.171:6824/3403894313,v1:172.21.15.171:6825/3403894313] boot 2023-07-14T18:13:31.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:30 smithi161 ceph-mon[128993]: osdmap e79: 12 total, 12 up, 12 in 2023-07-14T18:13:31.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:30 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T18:13:31.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:30 smithi171 ceph-mon[129241]: purged_snaps scrub starts 2023-07-14T18:13:31.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:30 smithi171 ceph-mon[129241]: purged_snaps scrub ok 2023-07-14T18:13:31.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:30 smithi171 ceph-mon[129241]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-14T18:13:31.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:30 smithi171 ceph-mon[129241]: osd.11 [v2:172.21.15.171:6824/3403894313,v1:172.21.15.171:6825/3403894313] boot 2023-07-14T18:13:31.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:30 smithi171 ceph-mon[129241]: osdmap e79: 12 total, 12 up, 12 in 2023-07-14T18:13:31.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:30 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T18:13:32.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:31 smithi133 ceph-mon[126384]: osdmap e80: 12 total, 12 up, 12 in 2023-07-14T18:13:32.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:31 smithi161 ceph-mon[128993]: osdmap e80: 12 total, 12 up, 12 in 2023-07-14T18:13:32.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:31 smithi171 ceph-mon[129241]: osdmap e80: 12 total, 12 up, 12 in 2023-07-14T18:13:32.520 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.477+0000 7f7a2759e700 1 -- 172.21.15.133:0/1766324523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a28100d20 msgr2=0x7f7a28101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:32.520 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.477+0000 7f7a2759e700 1 --2- 172.21.15.133:0/1766324523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a28100d20 0x7f7a28101170 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c009990 tx=0x7f7a1c009ca0 comp rx=0 tx=0).stop 2023-07-14T18:13:32.520 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.477+0000 7f7a2759e700 1 -- 172.21.15.133:0/1766324523 shutdown_connections 2023-07-14T18:13:32.521 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.477+0000 7f7a2759e700 1 --2- 172.21.15.133:0/1766324523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a281016b0 0x7f7a28107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:32.521 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.477+0000 7f7a2759e700 1 --2- 172.21.15.133:0/1766324523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a280ffb20 0x7f7a280fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:32.521 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.477+0000 7f7a2759e700 1 --2- 172.21.15.133:0/1766324523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a28100d20 0x7f7a28101170 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:32.522 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.477+0000 7f7a2759e700 1 -- 172.21.15.133:0/1766324523 >> 172.21.15.133:0/1766324523 conn(0x7f7a280fb170 msgr2=0x7f7a280fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:32.522 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.477+0000 7f7a2759e700 1 -- 172.21.15.133:0/1766324523 shutdown_connections 2023-07-14T18:13:32.522 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.477+0000 7f7a2759e700 1 -- 172.21.15.133:0/1766324523 wait complete. 2023-07-14T18:13:32.525 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.478+0000 7f7a2759e700 1 Processor -- start 2023-07-14T18:13:32.525 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.479+0000 7f7a2759e700 1 -- start start 2023-07-14T18:13:32.526 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.479+0000 7f7a2759e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a280ffb20 0x7f7a281979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:32.526 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.479+0000 7f7a2759e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a28100d20 0x7f7a28197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:32.526 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.479+0000 7f7a2759e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a281016b0 0x7f7a2819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:32.526 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.479+0000 7f7a2759e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2819f540 con 0x7f7a28100d20 2023-07-14T18:13:32.527 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.479+0000 7f7a2759e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2819f680 con 0x7f7a280ffb20 2023-07-14T18:13:32.527 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.479+0000 7f7a2759e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2819f810 con 0x7f7a281016b0 2023-07-14T18:13:32.527 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.479+0000 7f7a25d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a28100d20 0x7f7a28197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:32.527 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.479+0000 7f7a25d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a28100d20 0x7f7a28197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50128/0 (socket says 172.21.15.133:50128) 2023-07-14T18:13:32.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.479+0000 7f7a2659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a280ffb20 0x7f7a281979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:32.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.479+0000 7f7a25d9b700 1 -- 172.21.15.133:0/3846557116 learned_addr learned my addr 172.21.15.133:0/3846557116 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:13:32.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.479+0000 7f7a26d9d700 1 --2- 172.21.15.133:0/3846557116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a281016b0 0x7f7a2819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:32.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.480+0000 7f7a25d9b700 1 -- 172.21.15.133:0/3846557116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a281016b0 msgr2=0x7f7a2819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:32.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.480+0000 7f7a25d9b700 1 --2- 172.21.15.133:0/3846557116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a281016b0 0x7f7a2819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:32.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.480+0000 7f7a25d9b700 1 -- 172.21.15.133:0/3846557116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a280ffb20 msgr2=0x7f7a281979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:32.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.480+0000 7f7a25d9b700 1 --2- 172.21.15.133:0/3846557116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a280ffb20 0x7f7a281979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:32.530 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.480+0000 7f7a25d9b700 1 -- 172.21.15.133:0/3846557116 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1c0095f0 con 0x7f7a28100d20 2023-07-14T18:13:32.530 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.480+0000 7f7a2659c700 1 --2- 172.21.15.133:0/3846557116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a280ffb20 0x7f7a281979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:13:32.530 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.480+0000 7f7a25d9b700 1 --2- 172.21.15.133:0/3846557116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a28100d20 0x7f7a28197ef0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c004770 tx=0x7f7a1c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:32.530 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.480+0000 7f7a177fe700 1 -- 172.21.15.133:0/3846557116 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c01d070 con 0x7f7a28100d20 2023-07-14T18:13:32.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.480+0000 7f7a177fe700 1 -- 172.21.15.133:0/3846557116 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a1c005030 con 0x7f7a28100d20 2023-07-14T18:13:32.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.480+0000 7f7a2759e700 1 -- 172.21.15.133:0/3846557116 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a2819fa90 con 0x7f7a28100d20 2023-07-14T18:13:32.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.480+0000 7f7a177fe700 1 -- 172.21.15.133:0/3846557116 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c004b50 con 0x7f7a28100d20 2023-07-14T18:13:32.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.480+0000 7f7a2759e700 1 -- 172.21.15.133:0/3846557116 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a2819fe40 con 0x7f7a28100d20 2023-07-14T18:13:32.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.481+0000 7f7a2759e700 1 -- 172.21.15.133:0/3846557116 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a2810b3a0 con 0x7f7a28100d20 2023-07-14T18:13:32.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.483+0000 7f7a177fe700 1 -- 172.21.15.133:0/3846557116 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f7a1c004cf0 con 0x7f7a28100d20 2023-07-14T18:13:32.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.485+0000 7f7a177fe700 1 --2- 172.21.15.133:0/3846557116 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f7a100a1150 0x7f7a100a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:32.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.486+0000 7f7a177fe700 1 -- 172.21.15.133:0/3846557116 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7a1c0cd6e0 con 0x7f7a28100d20 2023-07-14T18:13:32.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.486+0000 7f7a2659c700 1 --2- 172.21.15.133:0/3846557116 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f7a100a1150 0x7f7a100a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:32.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.486+0000 7f7a2659c700 1 --2- 172.21.15.133:0/3846557116 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f7a100a1150 0x7f7a100a3600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7a28100b80 tx=0x7f7a18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:32.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.490+0000 7f7a177fe700 1 -- 172.21.15.133:0/3846557116 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f7a1c05a780 con 0x7f7a28100d20 2023-07-14T18:13:32.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.684+0000 7f7a2759e700 1 -- 172.21.15.133:0/3846557116 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f7a281a0450 con 0x7f7a28100d20 2023-07-14T18:13:32.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.684+0000 7f7a177fe700 1 -- 172.21.15.133:0/3846557116 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7f7a1c0972c0 con 0x7f7a28100d20 2023-07-14T18:13:32.727 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:13:32.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.687+0000 7f7a2759e700 1 -- 172.21.15.133:0/3846557116 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f7a100a1150 msgr2=0x7f7a100a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:32.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.687+0000 7f7a2759e700 1 --2- 172.21.15.133:0/3846557116 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f7a100a1150 0x7f7a100a3600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7a28100b80 tx=0x7f7a18009260 comp rx=0 tx=0).stop 2023-07-14T18:13:32.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.687+0000 7f7a2759e700 1 -- 172.21.15.133:0/3846557116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a28100d20 msgr2=0x7f7a28197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:32.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.687+0000 7f7a2759e700 1 --2- 172.21.15.133:0/3846557116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a28100d20 0x7f7a28197ef0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c004770 tx=0x7f7a1c009580 comp rx=0 tx=0).stop 2023-07-14T18:13:32.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.687+0000 7f7a2759e700 1 -- 172.21.15.133:0/3846557116 shutdown_connections 2023-07-14T18:13:32.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.687+0000 7f7a2759e700 1 --2- 172.21.15.133:0/3846557116 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f7a100a1150 0x7f7a100a3600 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:32.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.687+0000 7f7a2759e700 1 --2- 172.21.15.133:0/3846557116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a280ffb20 0x7f7a281979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:32.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.687+0000 7f7a2759e700 1 --2- 172.21.15.133:0/3846557116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a281016b0 0x7f7a2819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:32.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.687+0000 7f7a2759e700 1 --2- 172.21.15.133:0/3846557116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a28100d20 0x7f7a28197ef0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:32.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.687+0000 7f7a2759e700 1 -- 172.21.15.133:0/3846557116 >> 172.21.15.133:0/3846557116 conn(0x7f7a280fb170 msgr2=0x7f7a281065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:32.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.687+0000 7f7a2759e700 1 -- 172.21.15.133:0/3846557116 shutdown_connections 2023-07-14T18:13:32.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:32.687+0000 7f7a2759e700 1 -- 172.21.15.133:0/3846557116 wait complete. 2023-07-14T18:13:33.142 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:32 smithi133 ceph-mon[126384]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:33.142 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:32 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3846557116' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T18:13:33.206 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1689358409,"num_in_osds":12,"osd_in_since":1689358391,"num_remapped_pgs":0} 2023-07-14T18:13:33.207 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph osd dump --format=json 2023-07-14T18:13:33.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:32 smithi161 ceph-mon[128993]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:33.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:32 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3846557116' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T18:13:33.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:32 smithi171 ceph-mon[129241]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:33.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:32 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3846557116' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T18:13:33.930 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:13:35.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.275+0000 7feed6380700 1 -- 172.21.15.133:0/1409095075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feed010c510 msgr2=0x7feed010e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:35.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.275+0000 7feed6380700 1 --2- 172.21.15.133:0/1409095075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feed010c510 0x7feed010e8f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7feec800b0a0 tx=0x7feec800b3b0 comp rx=0 tx=0).stop 2023-07-14T18:13:35.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.275+0000 7feed6380700 1 -- 172.21.15.133:0/1409095075 shutdown_connections 2023-07-14T18:13:35.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.275+0000 7feed6380700 1 --2- 172.21.15.133:0/1409095075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feed010c510 0x7feed010e8f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:35.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.275+0000 7feed6380700 1 --2- 172.21.15.133:0/1409095075 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feed00729e0 0x7feed0072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:35.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.275+0000 7feed6380700 1 --2- 172.21.15.133:0/1409095075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feed00732f0 0x7feed010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:35.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.275+0000 7feed6380700 1 -- 172.21.15.133:0/1409095075 >> 172.21.15.133:0/1409095075 conn(0x7feed00fb790 msgr2=0x7feed00fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:35.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.275+0000 7feed6380700 1 -- 172.21.15.133:0/1409095075 shutdown_connections 2023-07-14T18:13:35.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.275+0000 7feed6380700 1 -- 172.21.15.133:0/1409095075 wait complete. 2023-07-14T18:13:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.276+0000 7feed6380700 1 Processor -- start 2023-07-14T18:13:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.276+0000 7feed6380700 1 -- start start 2023-07-14T18:13:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.277+0000 7feed6380700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feed00729e0 0x7feed0198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.277+0000 7feed6380700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feed00732f0 0x7feed0198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:35.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.277+0000 7feed6380700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feed010c510 0x7feed019c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:35.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.277+0000 7feed6380700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed019d080 con 0x7feed010c510 2023-07-14T18:13:35.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.277+0000 7feed6380700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed019d1f0 con 0x7feed00729e0 2023-07-14T18:13:35.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.277+0000 7feed6380700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed019d3b0 con 0x7feed00732f0 2023-07-14T18:13:35.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.277+0000 7feecf7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feed00732f0 0x7feed0198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:35.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.277+0000 7feed491d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feed010c510 0x7feed019c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:35.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.277+0000 7feecf7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feed00732f0 0x7feed0198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:50448/0 (socket says 172.21.15.133:50448) 2023-07-14T18:13:35.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.277+0000 7feecf7fe700 1 -- 172.21.15.133:0/2943733863 learned_addr learned my addr 172.21.15.133:0/2943733863 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:13:35.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.277+0000 7feecffff700 1 --2- 172.21.15.133:0/2943733863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feed00729e0 0x7feed0198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:35.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.277+0000 7feed491d700 1 -- 172.21.15.133:0/2943733863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feed00732f0 msgr2=0x7feed0198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:35.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.277+0000 7feed491d700 1 --2- 172.21.15.133:0/2943733863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feed00732f0 0x7feed0198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:35.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.277+0000 7feed491d700 1 -- 172.21.15.133:0/2943733863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feed00729e0 msgr2=0x7feed0198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:35.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.277+0000 7feed491d700 1 --2- 172.21.15.133:0/2943733863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feed00729e0 0x7feed0198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:35.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.277+0000 7feed491d700 1 -- 172.21.15.133:0/2943733863 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec8009d00 con 0x7feed010c510 2023-07-14T18:13:35.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.278+0000 7feed491d700 1 --2- 172.21.15.133:0/2943733863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feed010c510 0x7feed019c9e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7feec8009fd0 tx=0x7feec800bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:35.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.278+0000 7feecd7fa700 1 -- 172.21.15.133:0/2943733863 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec800e040 con 0x7feed010c510 2023-07-14T18:13:35.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.278+0000 7feed6380700 1 -- 172.21.15.133:0/2943733863 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed019d630 con 0x7feed010c510 2023-07-14T18:13:35.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.278+0000 7feecd7fa700 1 -- 172.21.15.133:0/2943733863 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feec80094c0 con 0x7feed010c510 2023-07-14T18:13:35.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.278+0000 7feecd7fa700 1 -- 172.21.15.133:0/2943733863 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8005040 con 0x7feed010c510 2023-07-14T18:13:35.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.278+0000 7feed6380700 1 -- 172.21.15.133:0/2943733863 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed01a4e60 con 0x7feed010c510 2023-07-14T18:13:35.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.279+0000 7feed6380700 1 -- 172.21.15.133:0/2943733863 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed0106e30 con 0x7feed010c510 2023-07-14T18:13:35.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.281+0000 7feecd7fa700 1 -- 172.21.15.133:0/2943733863 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7feec801d470 con 0x7feed010c510 2023-07-14T18:13:35.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.283+0000 7feecd7fa700 1 --2- 172.21.15.133:0/2943733863 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7feebc0a1150 0x7feebc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:35.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.283+0000 7feecd7fa700 1 -- 172.21.15.133:0/2943733863 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7feec80cda20 con 0x7feed010c510 2023-07-14T18:13:35.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.283+0000 7feecffff700 1 --2- 172.21.15.133:0/2943733863 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7feebc0a1150 0x7feebc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:35.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.284+0000 7feecffff700 1 --2- 172.21.15.133:0/2943733863 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7feebc0a1150 0x7feebc0a3600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7feec0009c60 tx=0x7feec0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:35.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.288+0000 7feecd7fa700 1 -- 172.21.15.133:0/2943733863 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7feec8097600 con 0x7feed010c510 2023-07-14T18:13:35.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:34 smithi161 ceph-mon[128993]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:35.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:34 smithi171 ceph-mon[129241]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:35.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:34 smithi133 ceph-mon[126384]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:35.525 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.483+0000 7feed6380700 1 -- 172.21.15.133:0/2943733863 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feed004e340 con 0x7feed010c510 2023-07-14T18:13:35.525 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.484+0000 7feecd7fa700 1 -- 172.21.15.133:0/2943733863 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18465 (secure 0 0 0) 0x7feec802d2f0 con 0x7feed010c510 2023-07-14T18:13:35.526 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:13:35.527 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":80,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","created":"2023-07-14T18:06:49.561582+0000","modified":"2023-07-14T18:13:30.945504+0000","last_up_change":"2023-07-14T18:13:29.836099+0000","last_in_change":"2023-07-14T18:13:11.606710+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T18:10:03.945039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"85a29de7-d712-422a-b647-d9e90091aaf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6803","nonce":3076510801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6805","nonce":3076510801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6809","nonce":3076510801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6807","nonce":3076510801}]},"public_addr":"172.21.15.133:6803/3076510801","cluster_addr":"172.21.15.133:6805/3076510801","heartbeat_back_addr":"172.21.15.133:6809/3076510801","heartbeat_front_addr":"172.21.15.133:6807/3076510801","state":["exists","up"]},{"osd":1,"uuid":"80ad4e78-7917-495e-b119-99f7c7c36243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6801","nonce":1841901852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6803","nonce":1841901852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6807","nonce":1841901852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6805","nonce":1841901852}]},"public_addr":"172.21.15.161:6801/1841901852","cluster_addr":"172.21.15.161:6803/1841901852","heartbeat_back_addr":"172.21.15.161:6807/1841901852","heartbeat_front_addr":"172.21.15.161:6805/1841901852","state":["exists","up"]},{"osd":2,"uuid":"fc4bfd37-1fdd-47c6-a48e-6d910daf9e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6801","nonce":346103099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6803","nonce":346103099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6807","nonce":346103099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6805","nonce":346103099}]},"public_addr":"172.21.15.171:6801/346103099","cluster_addr":"172.21.15.171:6803/346103099","heartbeat_back_addr":"172.21.15.171:6807/346103099","heartbeat_front_addr":"172.21.15.171:6805/346103099","state":["exists","up"]},{"osd":3,"uuid":"4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6811","nonce":2152994143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6813","nonce":2152994143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6817","nonce":2152994143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6815","nonce":2152994143}]},"public_addr":"172.21.15.133:6811/2152994143","cluster_addr":"172.21.15.133:6813/2152994143","heartbeat_back_addr":"172.21.15.133:6817/2152994143","heartbeat_front_addr":"172.21.15.133:6815/2152994143","state":["exists","up"]},{"osd":4,"uuid":"4601a1c2-254b-4c4d-9b15-8fc952b3d2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6809","nonce":47213291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6811","nonce":47213291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6815","nonce":47213291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6813","nonce":47213291}]},"public_addr":"172.21.15.161:6809/47213291","cluster_addr":"172.21.15.161:6811/47213291","heartbeat_back_addr":"172.21.15.161:6815/47213291","heartbeat_front_addr":"172.21.15.161:6813/47213291","state":["exists","up"]},{"osd":5,"uuid":"6cca180a-9140-4e49-835b-86842dae0574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6809","nonce":777556751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6811","nonce":777556751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6815","nonce":777556751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6813","nonce":777556751}]},"public_addr":"172.21.15.171:6809/777556751","cluster_addr":"172.21.15.171:6811/777556751","heartbeat_back_addr":"172.21.15.171:6815/777556751","heartbeat_front_addr":"172.21.15.171:6813/777556751","state":["exists","up"]},{"osd":6,"uuid":"ea0299ff-7740-4494-a3a9-1503b3fc0ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6819","nonce":2884152358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6821","nonce":2884152358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6825","nonce":2884152358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6823","nonce":2884152358}]},"public_addr":"172.21.15.133:6819/2884152358","cluster_addr":"172.21.15.133:6821/2884152358","heartbeat_back_addr":"172.21.15.133:6825/2884152358","heartbeat_front_addr":"172.21.15.133:6823/2884152358","state":["exists","up"]},{"osd":7,"uuid":"b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6817","nonce":4183705749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6819","nonce":4183705749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6823","nonce":4183705749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6821","nonce":4183705749}]},"public_addr":"172.21.15.161:6817/4183705749","cluster_addr":"172.21.15.161:6819/4183705749","heartbeat_back_addr":"172.21.15.161:6823/4183705749","heartbeat_front_addr":"172.21.15.161:6821/4183705749","state":["exists","up"]},{"osd":8,"uuid":"87dffc07-bf48-48c8-b418-5c4b509686d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6817","nonce":2771494483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6819","nonce":2771494483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6823","nonce":2771494483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6821","nonce":2771494483}]},"public_addr":"172.21.15.171:6817/2771494483","cluster_addr":"172.21.15.171:6819/2771494483","heartbeat_back_addr":"172.21.15.171:6823/2771494483","heartbeat_front_addr":"172.21.15.171:6821/2771494483","state":["exists","up"]},{"osd":9,"uuid":"e0832870-79ce-4bce-8045-58b7a7f6dc69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6827","nonce":2066164149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6829","nonce":2066164149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6833","nonce":2066164149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6831","nonce":2066164149}]},"public_addr":"172.21.15.133:6827/2066164149","cluster_addr":"172.21.15.133:6829/2066164149","heartbeat_back_addr":"172.21.15.133:6833/2066164149","heartbeat_front_addr":"172.21.15.133:6831/2066164149","state":["exists","up"]},{"osd":10,"uuid":"081f0484-d134-4b2e-9c48-ef8e6a646422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6825","nonce":2162333626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6827","nonce":2162333626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6831","nonce":2162333626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6829","nonce":2162333626}]},"public_addr":"172.21.15.161:6825/2162333626","cluster_addr":"172.21.15.161:6827/2162333626","heartbeat_back_addr":"172.21.15.161:6831/2162333626","heartbeat_front_addr":"172.21.15.161:6829/2162333626","state":["exists","up"]},{"osd":11,"uuid":"4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6825","nonce":3403894313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6827","nonce":3403894313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6831","nonce":3403894313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6829","nonce":3403894313}]},"public_addr":"172.21.15.171:6825/3403894313","cluster_addr":"172.21.15.171:6827/3403894313","heartbeat_back_addr":"172.21.15.171:6831/3403894313","heartbeat_front_addr":"172.21.15.171:6829/3403894313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:17.816229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:39.638543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:00.543317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:26.988921+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:48.644694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:10.049860+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:35.986508+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:57.753846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:19.018245+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:44.460450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:06.279960+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:27.955006+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/1832569241":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6801/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:0/4219815304":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:6801/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/432253615":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1840700480":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1807775439":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1089030734":"2023-07-15T18:07:47.346704+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T18:13:35.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.486+0000 7feed6380700 1 -- 172.21.15.133:0/2943733863 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7feebc0a1150 msgr2=0x7feebc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:35.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.486+0000 7feed6380700 1 --2- 172.21.15.133:0/2943733863 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7feebc0a1150 0x7feebc0a3600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7feec0009c60 tx=0x7feec0009260 comp rx=0 tx=0).stop 2023-07-14T18:13:35.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.486+0000 7feed6380700 1 -- 172.21.15.133:0/2943733863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feed010c510 msgr2=0x7feed019c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:35.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.486+0000 7feed6380700 1 --2- 172.21.15.133:0/2943733863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feed010c510 0x7feed019c9e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7feec8009fd0 tx=0x7feec800bce0 comp rx=0 tx=0).stop 2023-07-14T18:13:35.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.486+0000 7feed6380700 1 -- 172.21.15.133:0/2943733863 shutdown_connections 2023-07-14T18:13:35.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.486+0000 7feed6380700 1 --2- 172.21.15.133:0/2943733863 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7feebc0a1150 0x7feebc0a3600 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:35.530 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.486+0000 7feed6380700 1 --2- 172.21.15.133:0/2943733863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feed00729e0 0x7feed0198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:35.530 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.486+0000 7feed6380700 1 --2- 172.21.15.133:0/2943733863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feed010c510 0x7feed019c9e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:35.530 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.486+0000 7feed6380700 1 --2- 172.21.15.133:0/2943733863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feed00732f0 0x7feed0198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:35.530 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.486+0000 7feed6380700 1 -- 172.21.15.133:0/2943733863 >> 172.21.15.133:0/2943733863 conn(0x7feed00fb790 msgr2=0x7feed0103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:35.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.486+0000 7feed6380700 1 -- 172.21.15.133:0/2943733863 shutdown_connections 2023-07-14T18:13:35.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:35.486+0000 7feed6380700 1 -- 172.21.15.133:0/2943733863 wait complete. 2023-07-14T18:13:36.008 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T18:10:03.945039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T18:13:36.009 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph osd pool get .mgr pg_num 2023-07-14T18:13:36.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:35 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2943733863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:13:36.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:35 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2943733863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:13:36.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:35 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2943733863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:13:36.731 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:13:37.231 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:36 smithi133 ceph-mon[126384]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:37.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:36 smithi161 ceph-mon[128993]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:37.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:36 smithi171 ceph-mon[129241]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:38.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.044+0000 7f07fbba9700 1 -- 172.21.15.133:0/4270700271 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07f40729e0 msgr2=0x7f07f4072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:38.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.044+0000 7f07fbba9700 1 --2- 172.21.15.133:0/4270700271 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07f40729e0 0x7f07f4072db0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f07f0009990 tx=0x7f07f0009ca0 comp rx=0 tx=0).stop 2023-07-14T18:13:38.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.044+0000 7f07fbba9700 1 -- 172.21.15.133:0/4270700271 shutdown_connections 2023-07-14T18:13:38.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.044+0000 7f07fbba9700 1 --2- 172.21.15.133:0/4270700271 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07f410c510 0x7f07f410e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:38.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.044+0000 7f07fbba9700 1 --2- 172.21.15.133:0/4270700271 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07f40729e0 0x7f07f4072db0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:38.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.044+0000 7f07fbba9700 1 --2- 172.21.15.133:0/4270700271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07f40732f0 0x7f07f410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:38.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.044+0000 7f07fbba9700 1 -- 172.21.15.133:0/4270700271 >> 172.21.15.133:0/4270700271 conn(0x7f07f40fb790 msgr2=0x7f07f40fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:38.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.045+0000 7f07fbba9700 1 -- 172.21.15.133:0/4270700271 shutdown_connections 2023-07-14T18:13:38.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.045+0000 7f07fbba9700 1 -- 172.21.15.133:0/4270700271 wait complete. 2023-07-14T18:13:38.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.045+0000 7f07fbba9700 1 Processor -- start 2023-07-14T18:13:38.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.046+0000 7f07fbba9700 1 -- start start 2023-07-14T18:13:38.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.046+0000 7f07fbba9700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07f40729e0 0x7f07f4197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:38.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.046+0000 7f07fbba9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07f40732f0 0x7f07f4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:38.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.046+0000 7f07fbba9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07f410c510 0x7f07f419c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:38.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.046+0000 7f07fbba9700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f419cf80 con 0x7f07f410c510 2023-07-14T18:13:38.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.046+0000 7f07fbba9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f419d0f0 con 0x7f07f40732f0 2023-07-14T18:13:38.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.046+0000 7f07fbba9700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f419d2b0 con 0x7f07f40729e0 2023-07-14T18:13:38.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.046+0000 7f07fa146700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07f410c510 0x7f07f419c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:38.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.046+0000 7f07fa146700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07f410c510 0x7f07f419c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49204/0 (socket says 172.21.15.133:49204) 2023-07-14T18:13:38.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.046+0000 7f07fa146700 1 -- 172.21.15.133:0/2004226920 learned_addr learned my addr 172.21.15.133:0/2004226920 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:13:38.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.046+0000 7f07f9945700 1 --2- 172.21.15.133:0/2004226920 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07f40729e0 0x7f07f4197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:38.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.046+0000 7f07f9144700 1 --2- 172.21.15.133:0/2004226920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07f40732f0 0x7f07f4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:38.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.047+0000 7f07fa146700 1 -- 172.21.15.133:0/2004226920 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07f40729e0 msgr2=0x7f07f4197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:38.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.047+0000 7f07fa146700 1 --2- 172.21.15.133:0/2004226920 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07f40729e0 0x7f07f4197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:38.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.047+0000 7f07fa146700 1 -- 172.21.15.133:0/2004226920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07f40732f0 msgr2=0x7f07f4198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:38.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.047+0000 7f07fa146700 1 --2- 172.21.15.133:0/2004226920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07f40732f0 0x7f07f4198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:38.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.047+0000 7f07fa146700 1 -- 172.21.15.133:0/2004226920 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f00095f0 con 0x7f07f410c510 2023-07-14T18:13:38.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.047+0000 7f07fa146700 1 --2- 172.21.15.133:0/2004226920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07f410c510 0x7f07f419c8e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f07ec007ee0 tx=0x7f07ec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:38.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.047+0000 7f07eaffd700 1 -- 172.21.15.133:0/2004226920 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ec017070 con 0x7f07f410c510 2023-07-14T18:13:38.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.047+0000 7f07eaffd700 1 -- 172.21.15.133:0/2004226920 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07ec00f040 con 0x7f07f410c510 2023-07-14T18:13:38.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.047+0000 7f07fbba9700 1 -- 172.21.15.133:0/2004226920 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f419d590 con 0x7f07f410c510 2023-07-14T18:13:38.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.047+0000 7f07eaffd700 1 -- 172.21.15.133:0/2004226920 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ec005580 con 0x7f07f410c510 2023-07-14T18:13:38.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.047+0000 7f07fbba9700 1 -- 172.21.15.133:0/2004226920 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f41a4db0 con 0x7f07f410c510 2023-07-14T18:13:38.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.049+0000 7f07fbba9700 1 -- 172.21.15.133:0/2004226920 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07f404e340 con 0x7f07f410c510 2023-07-14T18:13:38.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.050+0000 7f07eaffd700 1 -- 172.21.15.133:0/2004226920 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f07ec00db20 con 0x7f07f410c510 2023-07-14T18:13:38.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.052+0000 7f07eaffd700 1 --2- 172.21.15.133:0/2004226920 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f07e00a11d0 0x7f07e00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:38.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.052+0000 7f07eaffd700 1 -- 172.21.15.133:0/2004226920 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f07ec0cbb50 con 0x7f07f410c510 2023-07-14T18:13:38.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.052+0000 7f07f9945700 1 --2- 172.21.15.133:0/2004226920 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f07e00a11d0 0x7f07e00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:38.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.052+0000 7f07f9945700 1 --2- 172.21.15.133:0/2004226920 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f07e00a11d0 0x7f07e00a3680 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f07f0000c00 tx=0x7f07f000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:38.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.056+0000 7f07eaffd700 1 -- 172.21.15.133:0/2004226920 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f07ec0956a0 con 0x7f07f410c510 2023-07-14T18:13:38.294 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.252+0000 7f07fbba9700 1 -- 172.21.15.133:0/2004226920 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f07f41a5130 con 0x7f07f410c510 2023-07-14T18:13:38.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.252+0000 7f07eaffd700 1 -- 172.21.15.133:0/2004226920 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f07ec098cc0 con 0x7f07f410c510 2023-07-14T18:13:38.295 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-07-14T18:13:38.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.255+0000 7f07fbba9700 1 -- 172.21.15.133:0/2004226920 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f07e00a11d0 msgr2=0x7f07e00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:38.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.255+0000 7f07fbba9700 1 --2- 172.21.15.133:0/2004226920 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f07e00a11d0 0x7f07e00a3680 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f07f0000c00 tx=0x7f07f000b260 comp rx=0 tx=0).stop 2023-07-14T18:13:38.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.255+0000 7f07fbba9700 1 -- 172.21.15.133:0/2004226920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07f410c510 msgr2=0x7f07f419c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:38.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.255+0000 7f07fbba9700 1 --2- 172.21.15.133:0/2004226920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07f410c510 0x7f07f419c8e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f07ec007ee0 tx=0x7f07ec00d3b0 comp rx=0 tx=0).stop 2023-07-14T18:13:38.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.255+0000 7f07fbba9700 1 -- 172.21.15.133:0/2004226920 shutdown_connections 2023-07-14T18:13:38.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.255+0000 7f07fbba9700 1 --2- 172.21.15.133:0/2004226920 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f07e00a11d0 0x7f07e00a3680 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:38.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.255+0000 7f07fbba9700 1 --2- 172.21.15.133:0/2004226920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07f410c510 0x7f07f419c8e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:38.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.255+0000 7f07fbba9700 1 --2- 172.21.15.133:0/2004226920 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07f40729e0 0x7f07f4197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:38.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.255+0000 7f07fbba9700 1 --2- 172.21.15.133:0/2004226920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07f40732f0 0x7f07f4198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:38.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.255+0000 7f07fbba9700 1 -- 172.21.15.133:0/2004226920 >> 172.21.15.133:0/2004226920 conn(0x7f07f40fb790 msgr2=0x7f07f410e600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:38.299 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.255+0000 7f07fbba9700 1 -- 172.21.15.133:0/2004226920 shutdown_connections 2023-07-14T18:13:38.299 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:38.255+0000 7f07fbba9700 1 -- 172.21.15.133:0/2004226920 wait complete. 2023-07-14T18:13:38.803 INFO:tasks.cephadm:Adding mds.a on smithi133 2023-07-14T18:13:38.804 INFO:tasks.cephadm:Adding mds.d on smithi133 2023-07-14T18:13:38.804 INFO:tasks.cephadm:Adding mds.g on smithi133 2023-07-14T18:13:38.804 INFO:tasks.cephadm:Adding mds.j on smithi133 2023-07-14T18:13:38.804 INFO:tasks.cephadm:Adding mds.b on smithi161 2023-07-14T18:13:38.805 INFO:tasks.cephadm:Adding mds.e on smithi161 2023-07-14T18:13:38.805 INFO:tasks.cephadm:Adding mds.h on smithi161 2023-07-14T18:13:38.805 INFO:tasks.cephadm:Adding mds.k on smithi161 2023-07-14T18:13:38.805 INFO:tasks.cephadm:Adding mds.c on smithi171 2023-07-14T18:13:38.805 INFO:tasks.cephadm:Adding mds.f on smithi171 2023-07-14T18:13:38.806 INFO:tasks.cephadm:Adding mds.i on smithi171 2023-07-14T18:13:38.806 INFO:tasks.cephadm:Adding mds.l on smithi171 2023-07-14T18:13:38.806 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph orch apply mds all '12;smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l' 2023-07-14T18:13:39.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:38 smithi171 ceph-mon[129241]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:39.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:38 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2004226920' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:13:39.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:38 smithi161 ceph-mon[128993]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:39.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:38 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2004226920' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:13:39.373 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T18:13:39.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:38 smithi133 ceph-mon[126384]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:39.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:38 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2004226920' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:13:40.611 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.567+0000 7f7e2bfad700 1 -- 172.21.15.171:0/1771787886 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e240febb0 msgr2=0x7f7e240fefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:40.611 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.567+0000 7f7e2bfad700 1 --2- 172.21.15.171:0/1771787886 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e240febb0 0x7f7e240fefc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7e14009990 tx=0x7f7e14009ca0 comp rx=0 tx=0).stop 2023-07-14T18:13:40.612 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.567+0000 7f7e2bfad700 1 -- 172.21.15.171:0/1771787886 shutdown_connections 2023-07-14T18:13:40.612 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.567+0000 7f7e2bfad700 1 --2- 172.21.15.171:0/1771787886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e24100740 0x7f7e24107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:40.612 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.567+0000 7f7e2bfad700 1 --2- 172.21.15.171:0/1771787886 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e240febb0 0x7f7e240fefc0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:40.612 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.567+0000 7f7e2bfad700 1 --2- 172.21.15.171:0/1771787886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e240ffdb0 0x7f7e24100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:40.612 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.567+0000 7f7e2bfad700 1 -- 172.21.15.171:0/1771787886 >> 172.21.15.171:0/1771787886 conn(0x7f7e240fa1e0 msgr2=0x7f7e240fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:40.612 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.567+0000 7f7e2bfad700 1 -- 172.21.15.171:0/1771787886 shutdown_connections 2023-07-14T18:13:40.613 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.567+0000 7f7e2bfad700 1 -- 172.21.15.171:0/1771787886 wait complete. 2023-07-14T18:13:40.613 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.568+0000 7f7e2bfad700 1 Processor -- start 2023-07-14T18:13:40.613 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.568+0000 7f7e2bfad700 1 -- start start 2023-07-14T18:13:40.613 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.568+0000 7f7e2bfad700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e240febb0 0x7f7e241949a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:40.614 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.568+0000 7f7e2bfad700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e240ffdb0 0x7f7e24194ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:40.614 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.568+0000 7f7e2bfad700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e24100740 0x7f7e2419bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:40.614 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.568+0000 7f7e2bfad700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2419c530 con 0x7f7e240febb0 2023-07-14T18:13:40.614 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.568+0000 7f7e2bfad700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2419c670 con 0x7f7e24100740 2023-07-14T18:13:40.614 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.568+0000 7f7e2bfad700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2419c800 con 0x7f7e240ffdb0 2023-07-14T18:13:40.614 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.568+0000 7f7e29548700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e240ffdb0 0x7f7e24194ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:40.615 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.568+0000 7f7e29548700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e240ffdb0 0x7f7e24194ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:43246/0 (socket says 172.21.15.171:43246) 2023-07-14T18:13:40.615 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.568+0000 7f7e29548700 1 -- 172.21.15.171:0/2042420497 learned_addr learned my addr 172.21.15.171:0/2042420497 (peer_addr_for_me v2:172.21.15.171:0/0) 2023-07-14T18:13:40.615 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.568+0000 7f7e2a54a700 1 --2- 172.21.15.171:0/2042420497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e24100740 0x7f7e2419bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:40.615 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.568+0000 7f7e29548700 1 -- 172.21.15.171:0/2042420497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e24100740 msgr2=0x7f7e2419bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:40.615 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.568+0000 7f7e29548700 1 --2- 172.21.15.171:0/2042420497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e24100740 0x7f7e2419bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:40.615 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.568+0000 7f7e29548700 1 -- 172.21.15.171:0/2042420497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e240febb0 msgr2=0x7f7e241949a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:40.615 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.568+0000 7f7e29548700 1 --2- 172.21.15.171:0/2042420497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e240febb0 0x7f7e241949a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:40.616 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.568+0000 7f7e29548700 1 -- 172.21.15.171:0/2042420497 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e140095f0 con 0x7f7e240ffdb0 2023-07-14T18:13:40.616 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.568+0000 7f7e2a54a700 1 --2- 172.21.15.171:0/2042420497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e24100740 0x7f7e2419bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:13:40.616 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.569+0000 7f7e29548700 1 --2- 172.21.15.171:0/2042420497 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e240ffdb0 0x7f7e24194ee0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7e2000b560 tx=0x7f7e2000b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:40.616 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.569+0000 7f7e1affd700 1 -- 172.21.15.171:0/2042420497 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e20011830 con 0x7f7e240ffdb0 2023-07-14T18:13:40.616 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.569+0000 7f7e1affd700 1 -- 172.21.15.171:0/2042420497 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e20011e50 con 0x7f7e240ffdb0 2023-07-14T18:13:40.616 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.569+0000 7f7e2bfad700 1 -- 172.21.15.171:0/2042420497 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2419cae0 con 0x7f7e240ffdb0 2023-07-14T18:13:40.617 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.569+0000 7f7e1affd700 1 -- 172.21.15.171:0/2042420497 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2000f2e0 con 0x7f7e240ffdb0 2023-07-14T18:13:40.618 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.569+0000 7f7e2bfad700 1 -- 172.21.15.171:0/2042420497 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2419cfe0 con 0x7f7e240ffdb0 2023-07-14T18:13:40.618 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.570+0000 7f7e2bfad700 1 -- 172.21.15.171:0/2042420497 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e241061a0 con 0x7f7e240ffdb0 2023-07-14T18:13:40.618 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.571+0000 7f7e1affd700 1 -- 172.21.15.171:0/2042420497 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f7e20011990 con 0x7f7e240ffdb0 2023-07-14T18:13:40.618 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.571+0000 7f7e1affd700 1 --2- 172.21.15.171:0/2042420497 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f7e100a1150 0x7f7e100a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:40.618 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.571+0000 7f7e1affd700 1 -- 172.21.15.171:0/2042420497 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7e200cc4f0 con 0x7f7e240ffdb0 2023-07-14T18:13:40.619 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.572+0000 7f7e29d49700 1 --2- 172.21.15.171:0/2042420497 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f7e100a1150 0x7f7e100a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:40.619 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.572+0000 7f7e29d49700 1 --2- 172.21.15.171:0/2042420497 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f7e100a1150 0x7f7e100a3600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7e140095c0 tx=0x7f7e14009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:40.619 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.575+0000 7f7e1affd700 1 -- 172.21.15.171:0/2042420497 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f7e200960e0 con 0x7f7e240ffdb0 2023-07-14T18:13:40.811 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.769+0000 7f7e2bfad700 1 -- 172.21.15.171:0/2042420497 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e24060c80 con 0x7f7e100a1150 2023-07-14T18:13:40.820 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.778+0000 7f7e1affd700 1 -- 172.21.15.171:0/2042420497 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f7e24060c80 con 0x7f7e100a1150 2023-07-14T18:13:40.821 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled mds.all update... 2023-07-14T18:13:40.823 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.780+0000 7f7e2bfad700 1 -- 172.21.15.171:0/2042420497 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f7e100a1150 msgr2=0x7f7e100a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:40.823 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.780+0000 7f7e2bfad700 1 --2- 172.21.15.171:0/2042420497 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f7e100a1150 0x7f7e100a3600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7e140095c0 tx=0x7f7e14009530 comp rx=0 tx=0).stop 2023-07-14T18:13:40.823 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.780+0000 7f7e2bfad700 1 -- 172.21.15.171:0/2042420497 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e240ffdb0 msgr2=0x7f7e24194ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:40.824 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.780+0000 7f7e2bfad700 1 --2- 172.21.15.171:0/2042420497 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e240ffdb0 0x7f7e24194ee0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7e2000b560 tx=0x7f7e2000b920 comp rx=0 tx=0).stop 2023-07-14T18:13:40.824 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.780+0000 7f7e2bfad700 1 -- 172.21.15.171:0/2042420497 shutdown_connections 2023-07-14T18:13:40.824 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.780+0000 7f7e2bfad700 1 --2- 172.21.15.171:0/2042420497 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f7e100a1150 0x7f7e100a3600 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:40.825 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.780+0000 7f7e2bfad700 1 --2- 172.21.15.171:0/2042420497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e24100740 0x7f7e2419bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:40.825 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.780+0000 7f7e2bfad700 1 --2- 172.21.15.171:0/2042420497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e240febb0 0x7f7e241949a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:40.825 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.780+0000 7f7e2bfad700 1 --2- 172.21.15.171:0/2042420497 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e240ffdb0 0x7f7e24194ee0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:40.825 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.780+0000 7f7e2bfad700 1 -- 172.21.15.171:0/2042420497 >> 172.21.15.171:0/2042420497 conn(0x7f7e240fa1e0 msgr2=0x7f7e241032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:40.826 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.780+0000 7f7e2bfad700 1 -- 172.21.15.171:0/2042420497 shutdown_connections 2023-07-14T18:13:40.826 INFO:teuthology.orchestra.run.smithi171.stderr:2023-07-14T18:13:40.780+0000 7f7e2bfad700 1 -- 172.21.15.171:0/2042420497 wait complete. 2023-07-14T18:13:41.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:40 smithi171 ceph-mon[129241]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:41.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:40 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:41.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:40 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:13:41.229 DEBUG:teuthology.orchestra.run.smithi133:mds.a> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.a.service 2023-07-14T18:13:41.236 DEBUG:teuthology.orchestra.run.smithi133:mds.d> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.d.service 2023-07-14T18:13:41.282 DEBUG:teuthology.orchestra.run.smithi133:mds.g> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.g.service 2023-07-14T18:13:41.300 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:40 smithi133 ceph-mon[126384]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:41.301 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:40 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:41.301 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:40 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:13:41.304 DEBUG:teuthology.orchestra.run.smithi133:mds.j> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.j.service 2023-07-14T18:13:41.318 INFO:journalctl@ceph.mds.a.smithi133.stdout:-- Logs begin at Fri 2023-07-14 17:46:17 UTC. -- 2023-07-14T18:13:41.321 DEBUG:teuthology.orchestra.run.smithi161:mds.b> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.b.service 2023-07-14T18:13:41.324 DEBUG:teuthology.orchestra.run.smithi161:mds.e> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.e.service 2023-07-14T18:13:41.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:40 smithi161 ceph-mon[128993]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:41.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:40 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:41.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:40 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:13:41.349 DEBUG:teuthology.orchestra.run.smithi161:mds.h> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.h.service 2023-07-14T18:13:41.353 INFO:journalctl@ceph.mds.b.smithi161.stdout:-- Logs begin at Fri 2023-07-14 17:41:43 UTC. -- 2023-07-14T18:13:41.354 DEBUG:teuthology.orchestra.run.smithi161:mds.k> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.k.service 2023-07-14T18:13:41.361 INFO:journalctl@ceph.mds.d.smithi133.stdout:-- Logs begin at Fri 2023-07-14 17:46:17 UTC. -- 2023-07-14T18:13:41.376 INFO:journalctl@ceph.mds.e.smithi161.stdout:-- Logs begin at Fri 2023-07-14 17:41:43 UTC. -- 2023-07-14T18:13:41.377 DEBUG:teuthology.orchestra.run.smithi171:mds.c> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.c.service 2023-07-14T18:13:41.379 DEBUG:teuthology.orchestra.run.smithi171:mds.f> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.f.service 2023-07-14T18:13:41.380 INFO:journalctl@ceph.mds.h.smithi161.stdout:-- Logs begin at Fri 2023-07-14 17:41:43 UTC. -- 2023-07-14T18:13:41.382 INFO:journalctl@ceph.mds.g.smithi133.stdout:-- Logs begin at Fri 2023-07-14 17:46:17 UTC. -- 2023-07-14T18:13:41.395 INFO:journalctl@ceph.mds.j.smithi133.stdout:-- Logs begin at Fri 2023-07-14 17:46:17 UTC. -- 2023-07-14T18:13:41.402 INFO:journalctl@ceph.mds.k.smithi161.stdout:-- Logs begin at Fri 2023-07-14 17:41:43 UTC. -- 2023-07-14T18:13:41.409 INFO:journalctl@ceph.mds.c.smithi171.stdout:-- Logs begin at Fri 2023-07-14 17:41:48 UTC. -- 2023-07-14T18:13:41.410 DEBUG:teuthology.orchestra.run.smithi171:mds.i> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.i.service 2023-07-14T18:13:41.435 INFO:journalctl@ceph.mds.f.smithi171.stdout:-- Logs begin at Fri 2023-07-14 17:41:48 UTC. -- 2023-07-14T18:13:41.436 DEBUG:teuthology.orchestra.run.smithi171:mds.l> sudo journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.l.service 2023-07-14T18:13:41.461 INFO:journalctl@ceph.mds.i.smithi171.stdout:-- Logs begin at Fri 2023-07-14 17:41:48 UTC. -- 2023-07-14T18:13:41.462 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-07-14T18:13:41.463 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T18:13:41.488 INFO:journalctl@ceph.mds.l.smithi171.stdout:-- Logs begin at Fri 2023-07-14 17:41:48 UTC. -- 2023-07-14T18:13:41.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.632+0000 7f5477bff700 1 -- 172.21.15.133:0/2763662600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5470102f30 msgr2=0x7f5470105310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:41.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.632+0000 7f5477bff700 1 --2- 172.21.15.133:0/2763662600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5470102f30 0x7f5470105310 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5460009990 tx=0x7f5460009ca0 comp rx=0 tx=0).stop 2023-07-14T18:13:41.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.632+0000 7f5477bff700 1 -- 172.21.15.133:0/2763662600 shutdown_connections 2023-07-14T18:13:41.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.632+0000 7f5477bff700 1 --2- 172.21.15.133:0/2763662600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5470100610 0x7f54701029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:41.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.632+0000 7f5477bff700 1 --2- 172.21.15.133:0/2763662600 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5470105850 0x7f5470107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:41.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.632+0000 7f5477bff700 1 --2- 172.21.15.133:0/2763662600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5470102f30 0x7f5470105310 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:41.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.632+0000 7f5477bff700 1 -- 172.21.15.133:0/2763662600 >> 172.21.15.133:0/2763662600 conn(0x7f54700fa090 msgr2=0x7f54700fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:41.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.633+0000 7f5477bff700 1 -- 172.21.15.133:0/2763662600 shutdown_connections 2023-07-14T18:13:41.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.633+0000 7f5477bff700 1 -- 172.21.15.133:0/2763662600 wait complete. 2023-07-14T18:13:41.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.634+0000 7f5477bff700 1 Processor -- start 2023-07-14T18:13:41.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.634+0000 7f5477bff700 1 -- start start 2023-07-14T18:13:41.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.635+0000 7f5477bff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5470100610 0x7f54701958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:41.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.635+0000 7f5477bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5470102f30 0x7f5470195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:41.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.635+0000 7f5477bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5470105850 0x7f547019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:41.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.635+0000 7f5477bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547019d430 con 0x7f5470102f30 2023-07-14T18:13:41.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.635+0000 7f5477bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547019d570 con 0x7f5470105850 2023-07-14T18:13:41.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.635+0000 7f5477bff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547019d700 con 0x7f5470100610 2023-07-14T18:13:41.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.635+0000 7f547519a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5470102f30 0x7f5470195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:41.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.635+0000 7f547519a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5470102f30 0x7f5470195df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49226/0 (socket says 172.21.15.133:49226) 2023-07-14T18:13:41.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.635+0000 7f547519a700 1 -- 172.21.15.133:0/3865798258 learned_addr learned my addr 172.21.15.133:0/3865798258 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:13:41.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.635+0000 7f547599b700 1 --2- 172.21.15.133:0/3865798258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5470100610 0x7f54701958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:41.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.635+0000 7f547619c700 1 --2- 172.21.15.133:0/3865798258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5470105850 0x7f547019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:41.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.635+0000 7f547519a700 1 -- 172.21.15.133:0/3865798258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5470100610 msgr2=0x7f54701958b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:41.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.635+0000 7f547519a700 1 --2- 172.21.15.133:0/3865798258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5470100610 0x7f54701958b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:41.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.635+0000 7f547519a700 1 -- 172.21.15.133:0/3865798258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5470105850 msgr2=0x7f547019ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:41.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.635+0000 7f547519a700 1 --2- 172.21.15.133:0/3865798258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5470105850 0x7f547019ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:41.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.635+0000 7f547519a700 1 -- 172.21.15.133:0/3865798258 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54600095f0 con 0x7f5470102f30 2023-07-14T18:13:41.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.635+0000 7f547519a700 1 --2- 172.21.15.133:0/3865798258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5470102f30 0x7f5470195df0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f546000b680 tx=0x7f54600057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:41.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.636+0000 7f5466ffd700 1 -- 172.21.15.133:0/3865798258 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546001d070 con 0x7f5470102f30 2023-07-14T18:13:41.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.636+0000 7f5466ffd700 1 -- 172.21.15.133:0/3865798258 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f546000fbc0 con 0x7f5470102f30 2023-07-14T18:13:41.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.636+0000 7f5477bff700 1 -- 172.21.15.133:0/3865798258 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f547019d980 con 0x7f5470102f30 2023-07-14T18:13:41.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.636+0000 7f5466ffd700 1 -- 172.21.15.133:0/3865798258 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5460017650 con 0x7f5470102f30 2023-07-14T18:13:41.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.636+0000 7f5477bff700 1 -- 172.21.15.133:0/3865798258 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f547019de00 con 0x7f5470102f30 2023-07-14T18:13:41.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.637+0000 7f5477bff700 1 -- 172.21.15.133:0/3865798258 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54700fbc00 con 0x7f5470102f30 2023-07-14T18:13:41.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.639+0000 7f5466ffd700 1 -- 172.21.15.133:0/3865798258 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f5460022470 con 0x7f5470102f30 2023-07-14T18:13:41.683 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.641+0000 7f5466ffd700 1 --2- 172.21.15.133:0/3865798258 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f545c0a1220 0x7f545c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:41.683 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.641+0000 7f5466ffd700 1 -- 172.21.15.133:0/3865798258 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f54600cdbf0 con 0x7f5470102f30 2023-07-14T18:13:41.683 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.641+0000 7f547599b700 1 --2- 172.21.15.133:0/3865798258 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f545c0a1220 0x7f545c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:41.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.642+0000 7f547599b700 1 --2- 172.21.15.133:0/3865798258 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f545c0a1220 0x7f545c0a36d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f546c009c60 tx=0x7f546c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:41.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.645+0000 7f5466ffd700 1 -- 172.21.15.133:0/3865798258 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f54600977d0 con 0x7f5470102f30 2023-07-14T18:13:41.889 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.847+0000 7f5477bff700 1 -- 172.21.15.133:0/3865798258 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f547002c8e0 con 0x7f5470102f30 2023-07-14T18:13:41.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.848+0000 7f5466ffd700 1 -- 172.21.15.133:0/3865798258 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18465 (secure 0 0 0) 0x7f546009adf0 con 0x7f5470102f30 2023-07-14T18:13:41.919 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:13:41.919 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":80,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","created":"2023-07-14T18:06:49.561582+0000","modified":"2023-07-14T18:13:30.945504+0000","last_up_change":"2023-07-14T18:13:29.836099+0000","last_in_change":"2023-07-14T18:13:11.606710+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T18:10:03.945039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"85a29de7-d712-422a-b647-d9e90091aaf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6803","nonce":3076510801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6805","nonce":3076510801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6809","nonce":3076510801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6807","nonce":3076510801}]},"public_addr":"172.21.15.133:6803/3076510801","cluster_addr":"172.21.15.133:6805/3076510801","heartbeat_back_addr":"172.21.15.133:6809/3076510801","heartbeat_front_addr":"172.21.15.133:6807/3076510801","state":["exists","up"]},{"osd":1,"uuid":"80ad4e78-7917-495e-b119-99f7c7c36243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6801","nonce":1841901852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6803","nonce":1841901852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6807","nonce":1841901852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6805","nonce":1841901852}]},"public_addr":"172.21.15.161:6801/1841901852","cluster_addr":"172.21.15.161:6803/1841901852","heartbeat_back_addr":"172.21.15.161:6807/1841901852","heartbeat_front_addr":"172.21.15.161:6805/1841901852","state":["exists","up"]},{"osd":2,"uuid":"fc4bfd37-1fdd-47c6-a48e-6d910daf9e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6801","nonce":346103099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6803","nonce":346103099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6807","nonce":346103099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6805","nonce":346103099}]},"public_addr":"172.21.15.171:6801/346103099","cluster_addr":"172.21.15.171:6803/346103099","heartbeat_back_addr":"172.21.15.171:6807/346103099","heartbeat_front_addr":"172.21.15.171:6805/346103099","state":["exists","up"]},{"osd":3,"uuid":"4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6811","nonce":2152994143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6813","nonce":2152994143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6817","nonce":2152994143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6815","nonce":2152994143}]},"public_addr":"172.21.15.133:6811/2152994143","cluster_addr":"172.21.15.133:6813/2152994143","heartbeat_back_addr":"172.21.15.133:6817/2152994143","heartbeat_front_addr":"172.21.15.133:6815/2152994143","state":["exists","up"]},{"osd":4,"uuid":"4601a1c2-254b-4c4d-9b15-8fc952b3d2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6809","nonce":47213291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6811","nonce":47213291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6815","nonce":47213291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6813","nonce":47213291}]},"public_addr":"172.21.15.161:6809/47213291","cluster_addr":"172.21.15.161:6811/47213291","heartbeat_back_addr":"172.21.15.161:6815/47213291","heartbeat_front_addr":"172.21.15.161:6813/47213291","state":["exists","up"]},{"osd":5,"uuid":"6cca180a-9140-4e49-835b-86842dae0574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6809","nonce":777556751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6811","nonce":777556751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6815","nonce":777556751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6813","nonce":777556751}]},"public_addr":"172.21.15.171:6809/777556751","cluster_addr":"172.21.15.171:6811/777556751","heartbeat_back_addr":"172.21.15.171:6815/777556751","heartbeat_front_addr":"172.21.15.171:6813/777556751","state":["exists","up"]},{"osd":6,"uuid":"ea0299ff-7740-4494-a3a9-1503b3fc0ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6819","nonce":2884152358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6821","nonce":2884152358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6825","nonce":2884152358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6823","nonce":2884152358}]},"public_addr":"172.21.15.133:6819/2884152358","cluster_addr":"172.21.15.133:6821/2884152358","heartbeat_back_addr":"172.21.15.133:6825/2884152358","heartbeat_front_addr":"172.21.15.133:6823/2884152358","state":["exists","up"]},{"osd":7,"uuid":"b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6817","nonce":4183705749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6819","nonce":4183705749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6823","nonce":4183705749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6821","nonce":4183705749}]},"public_addr":"172.21.15.161:6817/4183705749","cluster_addr":"172.21.15.161:6819/4183705749","heartbeat_back_addr":"172.21.15.161:6823/4183705749","heartbeat_front_addr":"172.21.15.161:6821/4183705749","state":["exists","up"]},{"osd":8,"uuid":"87dffc07-bf48-48c8-b418-5c4b509686d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6817","nonce":2771494483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6819","nonce":2771494483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6823","nonce":2771494483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6821","nonce":2771494483}]},"public_addr":"172.21.15.171:6817/2771494483","cluster_addr":"172.21.15.171:6819/2771494483","heartbeat_back_addr":"172.21.15.171:6823/2771494483","heartbeat_front_addr":"172.21.15.171:6821/2771494483","state":["exists","up"]},{"osd":9,"uuid":"e0832870-79ce-4bce-8045-58b7a7f6dc69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6827","nonce":2066164149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6829","nonce":2066164149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6833","nonce":2066164149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6831","nonce":2066164149}]},"public_addr":"172.21.15.133:6827/2066164149","cluster_addr":"172.21.15.133:6829/2066164149","heartbeat_back_addr":"172.21.15.133:6833/2066164149","heartbeat_front_addr":"172.21.15.133:6831/2066164149","state":["exists","up"]},{"osd":10,"uuid":"081f0484-d134-4b2e-9c48-ef8e6a646422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6825","nonce":2162333626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6827","nonce":2162333626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6831","nonce":2162333626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6829","nonce":2162333626}]},"public_addr":"172.21.15.161:6825/2162333626","cluster_addr":"172.21.15.161:6827/2162333626","heartbeat_back_addr":"172.21.15.161:6831/2162333626","heartbeat_front_addr":"172.21.15.161:6829/2162333626","state":["exists","up"]},{"osd":11,"uuid":"4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6825","nonce":3403894313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6827","nonce":3403894313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6831","nonce":3403894313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6829","nonce":3403894313}]},"public_addr":"172.21.15.171:6825/3403894313","cluster_addr":"172.21.15.171:6827/3403894313","heartbeat_back_addr":"172.21.15.171:6831/3403894313","heartbeat_front_addr":"172.21.15.171:6829/3403894313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:17.816229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:39.638543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:00.543317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:26.988921+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:48.644694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:10.049860+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:35.986508+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:57.753846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:19.018245+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:44.460450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:06.279960+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:27.955006+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/1832569241":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6801/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:0/4219815304":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:6801/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/432253615":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1840700480":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1807775439":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1089030734":"2023-07-15T18:07:47.346704+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T18:13:41.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.851+0000 7f5477bff700 1 -- 172.21.15.133:0/3865798258 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f545c0a1220 msgr2=0x7f545c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:41.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.851+0000 7f5477bff700 1 --2- 172.21.15.133:0/3865798258 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f545c0a1220 0x7f545c0a36d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f546c009c60 tx=0x7f546c009260 comp rx=0 tx=0).stop 2023-07-14T18:13:41.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.851+0000 7f5477bff700 1 -- 172.21.15.133:0/3865798258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5470102f30 msgr2=0x7f5470195df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:41.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.851+0000 7f5477bff700 1 --2- 172.21.15.133:0/3865798258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5470102f30 0x7f5470195df0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f546000b680 tx=0x7f54600057b0 comp rx=0 tx=0).stop 2023-07-14T18:13:41.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.851+0000 7f5477bff700 1 -- 172.21.15.133:0/3865798258 shutdown_connections 2023-07-14T18:13:41.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.851+0000 7f5477bff700 1 --2- 172.21.15.133:0/3865798258 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f545c0a1220 0x7f545c0a36d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:41.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.851+0000 7f5477bff700 1 --2- 172.21.15.133:0/3865798258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5470105850 0x7f547019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:41.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.851+0000 7f5477bff700 1 --2- 172.21.15.133:0/3865798258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5470100610 0x7f54701958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:41.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.851+0000 7f5477bff700 1 --2- 172.21.15.133:0/3865798258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5470102f30 0x7f5470195df0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:41.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.851+0000 7f5477bff700 1 -- 172.21.15.133:0/3865798258 >> 172.21.15.133:0/3865798258 conn(0x7f54700fa090 msgr2=0x7f5470101270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:41.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.851+0000 7f5477bff700 1 -- 172.21.15.133:0/3865798258 shutdown_connections 2023-07-14T18:13:41.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:41.851+0000 7f5477bff700 1 -- 172.21.15.133:0/3865798258 wait complete. 2023-07-14T18:13:41.923 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T18:10:03.945039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T18:13:41.924 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T18:13:42.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.083+0000 7f5d9e5ff700 1 -- 172.21.15.133:0/2710591183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d981017a0 msgr2=0x7f5d98108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:42.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.083+0000 7f5d9e5ff700 1 --2- 172.21.15.133:0/2710591183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d981017a0 0x7f5d98108060 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000b0a0 tx=0x7f5d9000b3b0 comp rx=0 tx=0).stop 2023-07-14T18:13:42.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.083+0000 7f5d9e5ff700 1 -- 172.21.15.133:0/2710591183 shutdown_connections 2023-07-14T18:13:42.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.083+0000 7f5d9e5ff700 1 --2- 172.21.15.133:0/2710591183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d981017a0 0x7f5d98108060 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:42.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.083+0000 7f5d9e5ff700 1 --2- 172.21.15.133:0/2710591183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d980ffc20 0x7f5d98100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:42.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.083+0000 7f5d9e5ff700 1 --2- 172.21.15.133:0/2710591183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d98100e10 0x7f5d98101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:42.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.083+0000 7f5d9e5ff700 1 -- 172.21.15.133:0/2710591183 >> 172.21.15.133:0/2710591183 conn(0x7f5d980fb260 msgr2=0x7f5d980fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:42.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.083+0000 7f5d9e5ff700 1 -- 172.21.15.133:0/2710591183 shutdown_connections 2023-07-14T18:13:42.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.083+0000 7f5d9e5ff700 1 -- 172.21.15.133:0/2710591183 wait complete. 2023-07-14T18:13:42.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.084+0000 7f5d9e5ff700 1 Processor -- start 2023-07-14T18:13:42.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.084+0000 7f5d9e5ff700 1 -- start start 2023-07-14T18:13:42.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.085+0000 7f5d9e5ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d980ffc20 0x7f5d98197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:42.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.085+0000 7f5d9e5ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d98100e10 0x7f5d981981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:42.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.085+0000 7f5d9e5ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d981017a0 0x7f5d9819f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:42.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.085+0000 7f5d9e5ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9819f7f0 con 0x7f5d981017a0 2023-07-14T18:13:42.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.085+0000 7f5d9e5ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9819f960 con 0x7f5d980ffc20 2023-07-14T18:13:42.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.085+0000 7f5d9e5ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9819fb20 con 0x7f5d98100e10 2023-07-14T18:13:42.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.085+0000 7f5d9cb9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d981017a0 0x7f5d9819f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:42.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.085+0000 7f5d9cb9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d981017a0 0x7f5d9819f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49244/0 (socket says 172.21.15.133:49244) 2023-07-14T18:13:42.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.085+0000 7f5d9cb9c700 1 -- 172.21.15.133:0/2463829434 learned_addr learned my addr 172.21.15.133:0/2463829434 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:13:42.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.085+0000 7f5d977fe700 1 --2- 172.21.15.133:0/2463829434 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d98100e10 0x7f5d981981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:42.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.085+0000 7f5d97fff700 1 --2- 172.21.15.133:0/2463829434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d980ffc20 0x7f5d98197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:42.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.085+0000 7f5d9cb9c700 1 -- 172.21.15.133:0/2463829434 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d98100e10 msgr2=0x7f5d981981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:42.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.085+0000 7f5d9cb9c700 1 --2- 172.21.15.133:0/2463829434 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d98100e10 0x7f5d981981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:42.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.085+0000 7f5d9cb9c700 1 -- 172.21.15.133:0/2463829434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d980ffc20 msgr2=0x7f5d98197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:42.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.085+0000 7f5d9cb9c700 1 --2- 172.21.15.133:0/2463829434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d980ffc20 0x7f5d98197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:42.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.085+0000 7f5d9cb9c700 1 -- 172.21.15.133:0/2463829434 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d90009d00 con 0x7f5d981017a0 2023-07-14T18:13:42.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.085+0000 7f5d977fe700 1 --2- 172.21.15.133:0/2463829434 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d98100e10 0x7f5d981981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:13:42.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.086+0000 7f5d9cb9c700 1 --2- 172.21.15.133:0/2463829434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d981017a0 0x7f5d9819f220 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000d010 tx=0x7f5d9000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:42.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.086+0000 7f5d957fa700 1 -- 172.21.15.133:0/2463829434 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9000e040 con 0x7f5d981017a0 2023-07-14T18:13:42.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.086+0000 7f5d957fa700 1 -- 172.21.15.133:0/2463829434 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d90003e10 con 0x7f5d981017a0 2023-07-14T18:13:42.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.086+0000 7f5d9e5ff700 1 -- 172.21.15.133:0/2463829434 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9819fda0 con 0x7f5d981017a0 2023-07-14T18:13:42.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.086+0000 7f5d957fa700 1 -- 172.21.15.133:0/2463829434 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d900052a0 con 0x7f5d981017a0 2023-07-14T18:13:42.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.086+0000 7f5d9e5ff700 1 -- 172.21.15.133:0/2463829434 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d981a01c0 con 0x7f5d981017a0 2023-07-14T18:13:42.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.087+0000 7f5d9e5ff700 1 -- 172.21.15.133:0/2463829434 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d9810b450 con 0x7f5d981017a0 2023-07-14T18:13:42.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.089+0000 7f5d957fa700 1 -- 172.21.15.133:0/2463829434 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f5d90005400 con 0x7f5d981017a0 2023-07-14T18:13:42.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.090+0000 7f5d957fa700 1 --2- 172.21.15.133:0/2463829434 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5d800a1150 0x7f5d800a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:42.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.091+0000 7f5d957fa700 1 -- 172.21.15.133:0/2463829434 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5d900cd980 con 0x7f5d981017a0 2023-07-14T18:13:42.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.091+0000 7f5d97fff700 1 --2- 172.21.15.133:0/2463829434 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5d800a1150 0x7f5d800a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:42.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.091+0000 7f5d97fff700 1 --2- 172.21.15.133:0/2463829434 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5d800a1150 0x7f5d800a3600 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5d98198b50 tx=0x7f5d88009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:42.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.092+0000 7f5d957fa700 1 -- 172.21.15.133:0/2463829434 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f5d90097560 con 0x7f5d981017a0 2023-07-14T18:13:42.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.289+0000 7f5d9e5ff700 1 -- 172.21.15.133:0/2463829434 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5d981a0770 con 0x7f5d981017a0 2023-07-14T18:13:42.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.290+0000 7f5d957fa700 1 -- 172.21.15.133:0/2463829434 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5d9009ab80 con 0x7f5d981017a0 2023-07-14T18:13:42.332 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-07-14T18:13:42.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.292+0000 7f5d9e5ff700 1 -- 172.21.15.133:0/2463829434 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5d800a1150 msgr2=0x7f5d800a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.292+0000 7f5d9e5ff700 1 --2- 172.21.15.133:0/2463829434 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5d800a1150 0x7f5d800a3600 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5d98198b50 tx=0x7f5d88009260 comp rx=0 tx=0).stop 2023-07-14T18:13:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.292+0000 7f5d9e5ff700 1 -- 172.21.15.133:0/2463829434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d981017a0 msgr2=0x7f5d9819f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.292+0000 7f5d9e5ff700 1 --2- 172.21.15.133:0/2463829434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d981017a0 0x7f5d9819f220 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000d010 tx=0x7f5d9000bc80 comp rx=0 tx=0).stop 2023-07-14T18:13:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.292+0000 7f5d9e5ff700 1 -- 172.21.15.133:0/2463829434 shutdown_connections 2023-07-14T18:13:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.292+0000 7f5d9e5ff700 1 --2- 172.21.15.133:0/2463829434 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5d800a1150 0x7f5d800a3600 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.292+0000 7f5d9e5ff700 1 --2- 172.21.15.133:0/2463829434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d980ffc20 0x7f5d98197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.292+0000 7f5d9e5ff700 1 --2- 172.21.15.133:0/2463829434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d981017a0 0x7f5d9819f220 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:42.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.292+0000 7f5d9e5ff700 1 --2- 172.21.15.133:0/2463829434 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d98100e10 0x7f5d981981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:42.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.292+0000 7f5d9e5ff700 1 -- 172.21.15.133:0/2463829434 >> 172.21.15.133:0/2463829434 conn(0x7f5d980fb260 msgr2=0x7f5d981066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:42.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.292+0000 7f5d9e5ff700 1 -- 172.21.15.133:0/2463829434 shutdown_connections 2023-07-14T18:13:42.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.293+0000 7f5d9e5ff700 1 -- 172.21.15.133:0/2463829434 wait complete. 2023-07-14T18:13:42.346 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:41 smithi171 ceph-mon[129241]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:13:42.347 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:41 smithi171 ceph-mon[129241]: Saving service mds.all spec with placement smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2023-07-14T18:13:42.347 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:41 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:42.347 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:41 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:13:42.347 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:41 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:42.347 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:41 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:13:42.347 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:41 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:13:42.347 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:41 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:42.348 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:41 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3865798258' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:13:42.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:41 smithi161 ceph-mon[128993]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:13:42.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:41 smithi161 ceph-mon[128993]: Saving service mds.all spec with placement smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2023-07-14T18:13:42.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:41 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:42.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:41 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:13:42.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:41 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:42.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:41 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:13:42.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:41 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:13:42.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:41 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:42.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:41 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3865798258' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:13:42.351 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:41 smithi133 ceph-mon[126384]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:13:42.351 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:41 smithi133 ceph-mon[126384]: Saving service mds.all spec with placement smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2023-07-14T18:13:42.352 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:41 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:42.352 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:41 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:13:42.352 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:41 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:42.352 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:41 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:13:42.352 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:41 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:13:42.352 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:41 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:42.352 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:41 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3865798258' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:13:42.353 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T18:13:42.553 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.510+0000 7fee065ff700 1 -- 172.21.15.133:0/3786972373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee00100e20 msgr2=0x7fee00101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:42.553 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.510+0000 7fee065ff700 1 --2- 172.21.15.133:0/3786972373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee00100e20 0x7fee00101270 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fedf4009990 tx=0x7fedf4009ca0 comp rx=0 tx=0).stop 2023-07-14T18:13:42.553 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.510+0000 7fee065ff700 1 -- 172.21.15.133:0/3786972373 shutdown_connections 2023-07-14T18:13:42.553 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.510+0000 7fee065ff700 1 --2- 172.21.15.133:0/3786972373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee001017b0 0x7fee00108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:42.554 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.510+0000 7fee065ff700 1 --2- 172.21.15.133:0/3786972373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee000ffc30 0x7fee00100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:42.554 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.510+0000 7fee065ff700 1 --2- 172.21.15.133:0/3786972373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee00100e20 0x7fee00101270 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:42.554 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.510+0000 7fee065ff700 1 -- 172.21.15.133:0/3786972373 >> 172.21.15.133:0/3786972373 conn(0x7fee000fb210 msgr2=0x7fee000fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:42.554 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.510+0000 7fee065ff700 1 -- 172.21.15.133:0/3786972373 shutdown_connections 2023-07-14T18:13:42.554 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.510+0000 7fee065ff700 1 -- 172.21.15.133:0/3786972373 wait complete. 2023-07-14T18:13:42.556 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.511+0000 7fee065ff700 1 Processor -- start 2023-07-14T18:13:42.556 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.512+0000 7fee065ff700 1 -- start start 2023-07-14T18:13:42.556 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.512+0000 7fee065ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee000ffc30 0x7fee00195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:42.556 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.512+0000 7fee065ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee00100e20 0x7fee00195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:42.556 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.512+0000 7fee065ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee001017b0 0x7fee0019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:42.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.512+0000 7fee065ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0019d4c0 con 0x7fee00100e20 2023-07-14T18:13:42.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.512+0000 7fee065ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0019d600 con 0x7fee000ffc30 2023-07-14T18:13:42.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.512+0000 7fee065ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0019d790 con 0x7fee001017b0 2023-07-14T18:13:42.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.512+0000 7fedff7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee00100e20 0x7fee00195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:42.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.512+0000 7fedff7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee00100e20 0x7fee00195e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49264/0 (socket says 172.21.15.133:49264) 2023-07-14T18:13:42.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.512+0000 7fedff7fe700 1 -- 172.21.15.133:0/3855036067 learned_addr learned my addr 172.21.15.133:0/3855036067 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:13:42.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.512+0000 7fee04b9c700 1 --2- 172.21.15.133:0/3855036067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee001017b0 0x7fee0019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:42.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.512+0000 7fedfffff700 1 --2- 172.21.15.133:0/3855036067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee000ffc30 0x7fee00195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:42.558 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.512+0000 7fedff7fe700 1 -- 172.21.15.133:0/3855036067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee001017b0 msgr2=0x7fee0019cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:42.558 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.512+0000 7fedff7fe700 1 --2- 172.21.15.133:0/3855036067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee001017b0 0x7fee0019cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:42.558 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.512+0000 7fedff7fe700 1 -- 172.21.15.133:0/3855036067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee000ffc30 msgr2=0x7fee00195940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:42.558 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.512+0000 7fedff7fe700 1 --2- 172.21.15.133:0/3855036067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee000ffc30 0x7fee00195940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:42.558 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.512+0000 7fedff7fe700 1 -- 172.21.15.133:0/3855036067 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf40095f0 con 0x7fee00100e20 2023-07-14T18:13:42.558 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.513+0000 7fee04b9c700 1 --2- 172.21.15.133:0/3855036067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee001017b0 0x7fee0019cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:13:42.558 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.513+0000 7fedff7fe700 1 --2- 172.21.15.133:0/3855036067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee00100e20 0x7fee00195e80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fedf4004760 tx=0x7fedf4004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:42.558 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.513+0000 7fedfd7fa700 1 -- 172.21.15.133:0/3855036067 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf401d070 con 0x7fee00100e20 2023-07-14T18:13:42.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.513+0000 7fedfd7fa700 1 -- 172.21.15.133:0/3855036067 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedf4004830 con 0x7fee00100e20 2023-07-14T18:13:42.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.513+0000 7fedfd7fa700 1 -- 172.21.15.133:0/3855036067 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf4005350 con 0x7fee00100e20 2023-07-14T18:13:42.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.513+0000 7fee065ff700 1 -- 172.21.15.133:0/3855036067 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee0019da10 con 0x7fee00100e20 2023-07-14T18:13:42.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.513+0000 7fee065ff700 1 -- 172.21.15.133:0/3855036067 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee0019dd80 con 0x7fee00100e20 2023-07-14T18:13:42.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.514+0000 7fee065ff700 1 -- 172.21.15.133:0/3855036067 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee00107180 con 0x7fee00100e20 2023-07-14T18:13:42.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.516+0000 7fedfd7fa700 1 -- 172.21.15.133:0/3855036067 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fedf4005530 con 0x7fee00100e20 2023-07-14T18:13:42.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.518+0000 7fedfd7fa700 1 --2- 172.21.15.133:0/3855036067 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fede80a1220 0x7fede80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:42.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.518+0000 7fedfd7fa700 1 -- 172.21.15.133:0/3855036067 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fedf40cd660 con 0x7fee00100e20 2023-07-14T18:13:42.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.518+0000 7fedfffff700 1 --2- 172.21.15.133:0/3855036067 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fede80a1220 0x7fede80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:42.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.518+0000 7fedfffff700 1 --2- 172.21.15.133:0/3855036067 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fede80a1220 0x7fede80a36d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fee00100c80 tx=0x7fedf0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:42.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.519+0000 7fedfd7fa700 1 -- 172.21.15.133:0/3855036067 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fedf4097240 con 0x7fee00100e20 2023-07-14T18:13:42.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.767+0000 7fee065ff700 1 -- 172.21.15.133:0/3855036067 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee0002c8e0 con 0x7fee00100e20 2023-07-14T18:13:42.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.768+0000 7fedfd7fa700 1 -- 172.21.15.133:0/3855036067 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7fedf409a860 con 0x7fee00100e20 2023-07-14T18:13:42.810 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:13:42.811 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-07-14T18:13:42.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.770+0000 7fee065ff700 1 -- 172.21.15.133:0/3855036067 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fede80a1220 msgr2=0x7fede80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:42.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.770+0000 7fee065ff700 1 --2- 172.21.15.133:0/3855036067 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fede80a1220 0x7fede80a36d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fee00100c80 tx=0x7fedf0009260 comp rx=0 tx=0).stop 2023-07-14T18:13:42.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.770+0000 7fee065ff700 1 -- 172.21.15.133:0/3855036067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee00100e20 msgr2=0x7fee00195e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:42.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.770+0000 7fee065ff700 1 --2- 172.21.15.133:0/3855036067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee00100e20 0x7fee00195e80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fedf4004760 tx=0x7fedf4004790 comp rx=0 tx=0).stop 2023-07-14T18:13:42.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.770+0000 7fee065ff700 1 -- 172.21.15.133:0/3855036067 shutdown_connections 2023-07-14T18:13:42.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.770+0000 7fee065ff700 1 --2- 172.21.15.133:0/3855036067 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fede80a1220 0x7fede80a36d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:42.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.770+0000 7fee065ff700 1 --2- 172.21.15.133:0/3855036067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee000ffc30 0x7fee00195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:42.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.770+0000 7fee065ff700 1 --2- 172.21.15.133:0/3855036067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee001017b0 0x7fee0019cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:42.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.770+0000 7fee065ff700 1 --2- 172.21.15.133:0/3855036067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee00100e20 0x7fee00195e80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:42.814 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.770+0000 7fee065ff700 1 -- 172.21.15.133:0/3855036067 >> 172.21.15.133:0/3855036067 conn(0x7fee000fb210 msgr2=0x7fee001042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:42.814 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.770+0000 7fee065ff700 1 -- 172.21.15.133:0/3855036067 shutdown_connections 2023-07-14T18:13:42.814 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:42.770+0000 7fee065ff700 1 -- 172.21.15.133:0/3855036067 wait complete. 2023-07-14T18:13:42.814 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 1 2023-07-14T18:13:43.015 INFO:journalctl@ceph.mds.l.smithi171.stdout:Jul 14 18:13:42 smithi171 systemd[1]: Starting Ceph mds.l for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:13:43.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:42 smithi133 ceph-mon[126384]: Deploying daemon mds.l on smithi171 2023-07-14T18:13:43.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:42 smithi133 ceph-mon[126384]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:43.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:42 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2463829434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:13:43.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:42 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3855036067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:13:43.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:42 smithi161 ceph-mon[128993]: Deploying daemon mds.l on smithi171 2023-07-14T18:13:43.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:42 smithi161 ceph-mon[128993]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:43.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:42 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2463829434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:13:43.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:42 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3855036067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:13:43.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:42 smithi171 ceph-mon[129241]: Deploying daemon mds.l on smithi171 2023-07-14T18:13:43.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:42 smithi171 ceph-mon[129241]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:43.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:42 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2463829434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:13:43.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:42 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3855036067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:13:43.365 INFO:journalctl@ceph.mds.l.smithi171.stdout:Jul 14 18:13:43 smithi171 podman[154575]: 2023-07-14T18:13:43.863 INFO:journalctl@ceph.mds.l.smithi171.stdout:Jul 14 18:13:43 smithi171 ceph-mds[154615]: -- 172.21.15.171:0/571940066 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5632e70551e0 con 0x5632e702a800 2023-07-14T18:13:43.864 INFO:journalctl@ceph.mds.l.smithi171.stdout:Jul 14 18:13:43 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-l[154591]: starting mds.l at 2023-07-14T18:13:43.864 INFO:journalctl@ceph.mds.l.smithi171.stdout:Jul 14 18:13:43 smithi171 bash[154575]: c15c523df221054f5137005852f2532038f2df8246b33108e7abcbe63c528601 2023-07-14T18:13:43.864 INFO:journalctl@ceph.mds.l.smithi171.stdout:Jul 14 18:13:43 smithi171 systemd[1]: Started Ceph mds.l for 06d0f5fa-2271-11ee-9b33-001a4aab830c. 2023-07-14T18:13:44.767 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:44 smithi161 ceph-mon[128993]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:44.768 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:44 smithi161 ceph-mon[128993]: mds.? [v2:172.21.15.171:6832/2722373740,v1:172.21.15.171:6833/2722373740] up:boot 2023-07-14T18:13:44.768 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:44 smithi161 ceph-mon[128993]: fsmap 1 up:standby 2023-07-14T18:13:44.768 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:44 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-14T18:13:44.768 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:44 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:44.768 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:44 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:44.768 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:44 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:44.769 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:44 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:13:44.769 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:44 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:13:44.769 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:44 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:44.769 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:44 smithi161 ceph-mon[128993]: Deploying daemon mds.k on smithi161 2023-07-14T18:13:44.832 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T18:13:44.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:44 smithi171 ceph-mon[129241]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:44.864 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:44 smithi171 ceph-mon[129241]: mds.? [v2:172.21.15.171:6832/2722373740,v1:172.21.15.171:6833/2722373740] up:boot 2023-07-14T18:13:44.864 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:44 smithi171 ceph-mon[129241]: fsmap 1 up:standby 2023-07-14T18:13:44.864 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:44 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-14T18:13:44.864 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:44 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:44.864 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:44 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:44.865 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:44 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:44.865 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:44 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:13:44.865 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:44 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:13:44.865 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:44 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:44.865 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:44 smithi171 ceph-mon[129241]: Deploying daemon mds.k on smithi161 2023-07-14T18:13:44.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:44 smithi133 ceph-mon[126384]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:44.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:44 smithi133 ceph-mon[126384]: mds.? [v2:172.21.15.171:6832/2722373740,v1:172.21.15.171:6833/2722373740] up:boot 2023-07-14T18:13:44.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:44 smithi133 ceph-mon[126384]: fsmap 1 up:standby 2023-07-14T18:13:44.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:44 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-14T18:13:44.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:44 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:44.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:44 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:44.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:44 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:44.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:44 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:13:44.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:44 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:13:44.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:44 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:44.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:44 smithi133 ceph-mon[126384]: Deploying daemon mds.k on smithi161 2023-07-14T18:13:45.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:44.998+0000 7f7b331ff700 1 -- 172.21.15.133:0/1108955062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b2c100730 msgr2=0x7f7b2c102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:45.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:44.998+0000 7f7b331ff700 1 --2- 172.21.15.133:0/1108955062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b2c100730 0x7f7b2c102b10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009990 tx=0x7f7b20009ca0 comp rx=0 tx=0).stop 2023-07-14T18:13:45.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:44.999+0000 7f7b331ff700 1 -- 172.21.15.133:0/1108955062 shutdown_connections 2023-07-14T18:13:45.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:44.999+0000 7f7b331ff700 1 --2- 172.21.15.133:0/1108955062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b2c100730 0x7f7b2c102b10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:45.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:44.999+0000 7f7b331ff700 1 --2- 172.21.15.133:0/1108955062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b2c105970 0x7f7b2c107d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:45.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:44.999+0000 7f7b331ff700 1 --2- 172.21.15.133:0/1108955062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7b2c103050 0x7f7b2c105430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:45.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:44.999+0000 7f7b331ff700 1 -- 172.21.15.133:0/1108955062 >> 172.21.15.133:0/1108955062 conn(0x7f7b2c0fa130 msgr2=0x7f7b2c0fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:45.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:44.999+0000 7f7b331ff700 1 -- 172.21.15.133:0/1108955062 shutdown_connections 2023-07-14T18:13:45.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:44.999+0000 7f7b331ff700 1 -- 172.21.15.133:0/1108955062 wait complete. 2023-07-14T18:13:45.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.000+0000 7f7b331ff700 1 Processor -- start 2023-07-14T18:13:45.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.000+0000 7f7b331ff700 1 -- start start 2023-07-14T18:13:45.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.000+0000 7f7b331ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b2c100730 0x7f7b2c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:45.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.000+0000 7f7b331ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7b2c103050 0x7f7b2c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:45.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.000+0000 7f7b331ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b2c105970 0x7f7b2c19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:45.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.000+0000 7f7b331ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c19f650 con 0x7f7b2c100730 2023-07-14T18:13:45.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.000+0000 7f7b331ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c19f790 con 0x7f7b2c105970 2023-07-14T18:13:45.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.000+0000 7f7b331ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c19f920 con 0x7f7b2c103050 2023-07-14T18:13:45.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.000+0000 7f7b30f9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b2c100730 0x7f7b2c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:45.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.000+0000 7f7b3179c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b2c105970 0x7f7b2c19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:45.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.000+0000 7f7b30f9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b2c100730 0x7f7b2c197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49296/0 (socket says 172.21.15.133:49296) 2023-07-14T18:13:45.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.000+0000 7f7b30f9b700 1 -- 172.21.15.133:0/786558465 learned_addr learned my addr 172.21.15.133:0/786558465 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:13:45.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.000+0000 7f7b2bfff700 1 --2- 172.21.15.133:0/786558465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7b2c103050 0x7f7b2c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:45.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.001+0000 7f7b3179c700 1 -- 172.21.15.133:0/786558465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7b2c103050 msgr2=0x7f7b2c198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:45.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.001+0000 7f7b3179c700 1 --2- 172.21.15.133:0/786558465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7b2c103050 0x7f7b2c198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:45.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.001+0000 7f7b3179c700 1 -- 172.21.15.133:0/786558465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b2c100730 msgr2=0x7f7b2c197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:45.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.001+0000 7f7b3179c700 1 --2- 172.21.15.133:0/786558465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b2c100730 0x7f7b2c197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:45.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.001+0000 7f7b3179c700 1 -- 172.21.15.133:0/786558465 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b200095f0 con 0x7f7b2c105970 2023-07-14T18:13:45.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.001+0000 7f7b3179c700 1 --2- 172.21.15.133:0/786558465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b2c105970 0x7f7b2c19f080 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400efc0 tx=0x7f7b2400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:45.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.001+0000 7f7b29ffb700 1 -- 172.21.15.133:0/786558465 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b24018070 con 0x7f7b2c105970 2023-07-14T18:13:45.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.001+0000 7f7b29ffb700 1 -- 172.21.15.133:0/786558465 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b2400f040 con 0x7f7b2c105970 2023-07-14T18:13:45.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.001+0000 7f7b331ff700 1 -- 172.21.15.133:0/786558465 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b2c19fba0 con 0x7f7b2c105970 2023-07-14T18:13:45.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.001+0000 7f7b29ffb700 1 -- 172.21.15.133:0/786558465 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2400cd40 con 0x7f7b2c105970 2023-07-14T18:13:45.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.001+0000 7f7b331ff700 1 -- 172.21.15.133:0/786558465 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b2c1a0060 con 0x7f7b2c105970 2023-07-14T18:13:45.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.002+0000 7f7b331ff700 1 -- 172.21.15.133:0/786558465 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b2c1074e0 con 0x7f7b2c105970 2023-07-14T18:13:45.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.004+0000 7f7b29ffb700 1 -- 172.21.15.133:0/786558465 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f7b24003a00 con 0x7f7b2c105970 2023-07-14T18:13:45.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.006+0000 7f7b29ffb700 1 --2- 172.21.15.133:0/786558465 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f7b1c0a11a0 0x7f7b1c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:45.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.006+0000 7f7b29ffb700 1 -- 172.21.15.133:0/786558465 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7b240cca80 con 0x7f7b2c105970 2023-07-14T18:13:45.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.006+0000 7f7b30f9b700 1 --2- 172.21.15.133:0/786558465 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f7b1c0a11a0 0x7f7b1c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:45.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.006+0000 7f7b30f9b700 1 --2- 172.21.15.133:0/786558465 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f7b1c0a11a0 0x7f7b1c0a3650 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7b20006010 tx=0x7f7b2000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:45.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.009+0000 7f7b29ffb700 1 -- 172.21.15.133:0/786558465 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f7b24096660 con 0x7f7b2c105970 2023-07-14T18:13:45.098 INFO:journalctl@ceph.mds.k.smithi161.stdout:Jul 14 18:13:44 smithi161 systemd[1]: Starting Ceph mds.k for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:13:45.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.251+0000 7f7b331ff700 1 -- 172.21.15.133:0/786558465 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b2c02cb30 con 0x7f7b2c105970 2023-07-14T18:13:45.294 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.252+0000 7f7b29ffb700 1 -- 172.21.15.133:0/786558465 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7f7b24099c80 con 0x7f7b2c105970 2023-07-14T18:13:45.294 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:13:45.295 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2722373740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2722373740},{"type":"v1","addr":"172.21.15.171:6833","nonce":2722373740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-14T18:13:45.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.254+0000 7f7b331ff700 1 -- 172.21.15.133:0/786558465 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f7b1c0a11a0 msgr2=0x7f7b1c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:45.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.254+0000 7f7b331ff700 1 --2- 172.21.15.133:0/786558465 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f7b1c0a11a0 0x7f7b1c0a3650 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7b20006010 tx=0x7f7b2000b260 comp rx=0 tx=0).stop 2023-07-14T18:13:45.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.254+0000 7f7b331ff700 1 -- 172.21.15.133:0/786558465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b2c105970 msgr2=0x7f7b2c19f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:45.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.254+0000 7f7b331ff700 1 --2- 172.21.15.133:0/786558465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b2c105970 0x7f7b2c19f080 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400efc0 tx=0x7f7b2400c3a0 comp rx=0 tx=0).stop 2023-07-14T18:13:45.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.254+0000 7f7b331ff700 1 -- 172.21.15.133:0/786558465 shutdown_connections 2023-07-14T18:13:45.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.254+0000 7f7b331ff700 1 --2- 172.21.15.133:0/786558465 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f7b1c0a11a0 0x7f7b1c0a3650 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:45.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.254+0000 7f7b331ff700 1 --2- 172.21.15.133:0/786558465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b2c105970 0x7f7b2c19f080 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:45.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.254+0000 7f7b331ff700 1 --2- 172.21.15.133:0/786558465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b2c100730 0x7f7b2c197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:45.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.254+0000 7f7b331ff700 1 --2- 172.21.15.133:0/786558465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7b2c103050 0x7f7b2c198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:45.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.254+0000 7f7b331ff700 1 -- 172.21.15.133:0/786558465 >> 172.21.15.133:0/786558465 conn(0x7f7b2c0fa130 msgr2=0x7f7b2c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:45.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.254+0000 7f7b331ff700 1 -- 172.21.15.133:0/786558465 shutdown_connections 2023-07-14T18:13:45.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:45.255+0000 7f7b331ff700 1 -- 172.21.15.133:0/786558465 wait complete. 2023-07-14T18:13:45.299 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 2 2023-07-14T18:13:45.593 INFO:journalctl@ceph.mds.k.smithi161.stdout:Jul 14 18:13:45 smithi161 podman[153327]: 2023-07-14T18:13:45.593 INFO:journalctl@ceph.mds.k.smithi161.stdout:Jul 14 18:13:45 smithi161 ceph-mds[153367]: -- 172.21.15.161:0/3295973820 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5621505f11e0 con 0x5621505c6c00 2023-07-14T18:13:45.593 INFO:journalctl@ceph.mds.k.smithi161.stdout:Jul 14 18:13:45 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-k[153343]: starting mds.k at 2023-07-14T18:13:45.593 INFO:journalctl@ceph.mds.k.smithi161.stdout:Jul 14 18:13:45 smithi161 bash[153327]: c6cf83c92bd5eeeaa29d85e0b7a0dc45d70988d0b63fc1da84026bfb48495c77 2023-07-14T18:13:45.848 INFO:journalctl@ceph.mds.k.smithi161.stdout:Jul 14 18:13:45 smithi161 systemd[1]: Started Ceph mds.k for 06d0f5fa-2271-11ee-9b33-001a4aab830c. 2023-07-14T18:13:46.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:45 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/786558465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:13:46.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:45 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:46.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:45 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:46.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:45 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:46.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:45 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:13:46.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:45 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:13:46.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:45 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:46.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:45 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/786558465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:13:46.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:45 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:46.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:45 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:46.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:45 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:46.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:45 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:13:46.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:45 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:13:46.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:45 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:46.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:45 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/786558465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:13:46.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:45 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:46.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:45 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:46.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:45 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:46.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:45 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:13:46.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:45 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:13:46.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:45 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:47.271 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:46 smithi133 ceph-mon[126384]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:47.272 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:46 smithi133 ceph-mon[126384]: Deploying daemon mds.j on smithi133 2023-07-14T18:13:47.272 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:46 smithi133 ceph-mon[126384]: mds.? [v2:172.21.15.161:6832/4023127259,v1:172.21.15.161:6833/4023127259] up:boot 2023-07-14T18:13:47.272 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:46 smithi133 ceph-mon[126384]: fsmap 2 up:standby 2023-07-14T18:13:47.272 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:46 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-14T18:13:47.318 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T18:13:47.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:46 smithi161 ceph-mon[128993]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:47.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:46 smithi161 ceph-mon[128993]: Deploying daemon mds.j on smithi133 2023-07-14T18:13:47.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:46 smithi161 ceph-mon[128993]: mds.? [v2:172.21.15.161:6832/4023127259,v1:172.21.15.161:6833/4023127259] up:boot 2023-07-14T18:13:47.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:46 smithi161 ceph-mon[128993]: fsmap 2 up:standby 2023-07-14T18:13:47.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:46 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-14T18:13:47.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:46 smithi171 ceph-mon[129241]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:47.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:46 smithi171 ceph-mon[129241]: Deploying daemon mds.j on smithi133 2023-07-14T18:13:47.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:46 smithi171 ceph-mon[129241]: mds.? [v2:172.21.15.161:6832/4023127259,v1:172.21.15.161:6833/4023127259] up:boot 2023-07-14T18:13:47.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:46 smithi171 ceph-mon[129241]: fsmap 2 up:standby 2023-07-14T18:13:47.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:46 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-14T18:13:47.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.664+0000 7f8fc4d9d700 1 -- 172.21.15.133:0/1034927017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fc0074680 msgr2=0x7f8fc0072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:47.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.664+0000 7f8fc4d9d700 1 --2- 172.21.15.133:0/1034927017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fc0074680 0x7f8fc0072ad0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0009990 tx=0x7f8fb0009ca0 comp rx=0 tx=0).stop 2023-07-14T18:13:47.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.664+0000 7f8fc4d9d700 1 -- 172.21.15.133:0/1034927017 shutdown_connections 2023-07-14T18:13:47.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.664+0000 7f8fc4d9d700 1 --2- 172.21.15.133:0/1034927017 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fc01079d0 0x7f8fc0109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:47.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.664+0000 7f8fc4d9d700 1 --2- 172.21.15.133:0/1034927017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fc0074680 0x7f8fc0072ad0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:47.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.664+0000 7f8fc4d9d700 1 --2- 172.21.15.133:0/1034927017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fc00730a0 0x7f8fc0073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:47.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.664+0000 7f8fc4d9d700 1 -- 172.21.15.133:0/1034927017 >> 172.21.15.133:0/1034927017 conn(0x7f8fc00fb110 msgr2=0x7f8fc00fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:47.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.664+0000 7f8fc4d9d700 1 -- 172.21.15.133:0/1034927017 shutdown_connections 2023-07-14T18:13:47.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.664+0000 7f8fc4d9d700 1 -- 172.21.15.133:0/1034927017 wait complete. 2023-07-14T18:13:47.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.665+0000 7f8fc4d9d700 1 Processor -- start 2023-07-14T18:13:47.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.666+0000 7f8fc4d9d700 1 -- start start 2023-07-14T18:13:47.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.666+0000 7f8fc4d9d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fc00730a0 0x7f8fc01978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:47.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.666+0000 7f8fc4d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fc0074680 0x7f8fc0197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:47.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.666+0000 7f8fc4d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fc01079d0 0x7f8fc019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:47.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.666+0000 7f8fc4d9d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc019f430 con 0x7f8fc0074680 2023-07-14T18:13:47.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.666+0000 7f8fc4d9d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc019f570 con 0x7f8fc01079d0 2023-07-14T18:13:47.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.666+0000 7f8fc4d9d700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc019f700 con 0x7f8fc00730a0 2023-07-14T18:13:47.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.666+0000 7f8fbeffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fc0074680 0x7f8fc0197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:47.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.666+0000 7f8fbf7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fc00730a0 0x7f8fc01978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:47.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.666+0000 7f8fbeffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fc0074680 0x7f8fc0197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50164/0 (socket says 172.21.15.133:50164) 2023-07-14T18:13:47.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.666+0000 7f8fbeffd700 1 -- 172.21.15.133:0/3901915710 learned_addr learned my addr 172.21.15.133:0/3901915710 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:13:47.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.666+0000 7f8fbf7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fc00730a0 0x7f8fc01978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:40008/0 (socket says 172.21.15.133:40008) 2023-07-14T18:13:47.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.667+0000 7f8fbffff700 1 --2- 172.21.15.133:0/3901915710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fc01079d0 0x7f8fc019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:47.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.667+0000 7f8fbf7fe700 1 -- 172.21.15.133:0/3901915710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fc01079d0 msgr2=0x7f8fc019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:47.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.667+0000 7f8fbf7fe700 1 --2- 172.21.15.133:0/3901915710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fc01079d0 0x7f8fc019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:47.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.667+0000 7f8fbf7fe700 1 -- 172.21.15.133:0/3901915710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fc0074680 msgr2=0x7f8fc0197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:47.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.667+0000 7f8fbf7fe700 1 --2- 172.21.15.133:0/3901915710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fc0074680 0x7f8fc0197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:47.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.667+0000 7f8fbf7fe700 1 -- 172.21.15.133:0/3901915710 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb00095f0 con 0x7f8fc00730a0 2023-07-14T18:13:47.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.667+0000 7f8fbeffd700 1 --2- 172.21.15.133:0/3901915710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fc0074680 0x7f8fc0197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:13:47.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.667+0000 7f8fbf7fe700 1 --2- 172.21.15.133:0/3901915710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fc00730a0 0x7f8fc01978b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000b680 tx=0x7f8fb0009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:47.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.668+0000 7f8fbcff9700 1 -- 172.21.15.133:0/3901915710 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb001d070 con 0x7f8fc00730a0 2023-07-14T18:13:47.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.668+0000 7f8fc4d9d700 1 -- 172.21.15.133:0/3901915710 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc019f980 con 0x7f8fc00730a0 2023-07-14T18:13:47.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.668+0000 7f8fc4d9d700 1 -- 172.21.15.133:0/3901915710 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc019fe00 con 0x7f8fc00730a0 2023-07-14T18:13:47.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.668+0000 7f8fbcff9700 1 -- 172.21.15.133:0/3901915710 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fb0021410 con 0x7f8fc00730a0 2023-07-14T18:13:47.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.668+0000 7f8fbcff9700 1 -- 172.21.15.133:0/3901915710 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0021ea0 con 0x7f8fc00730a0 2023-07-14T18:13:47.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.669+0000 7f8fc4d9d700 1 -- 172.21.15.133:0/3901915710 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc0106080 con 0x7f8fc00730a0 2023-07-14T18:13:47.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.671+0000 7f8fbcff9700 1 -- 172.21.15.133:0/3901915710 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f8fb0021830 con 0x7f8fc00730a0 2023-07-14T18:13:47.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.672+0000 7f8fbcff9700 1 --2- 172.21.15.133:0/3901915710 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8fa80a12c0 0x7f8fa80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:47.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.672+0000 7f8fbcff9700 1 -- 172.21.15.133:0/3901915710 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8fb00cda50 con 0x7f8fc00730a0 2023-07-14T18:13:47.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.672+0000 7f8fbeffd700 1 --2- 172.21.15.133:0/3901915710 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8fa80a12c0 0x7f8fa80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:47.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.673+0000 7f8fbeffd700 1 --2- 172.21.15.133:0/3901915710 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8fa80a12c0 0x7f8fa80a3770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4005950 tx=0x7f8fb4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:47.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.674+0000 7f8fbcff9700 1 -- 172.21.15.133:0/3901915710 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f8fb0097630 con 0x7f8fc00730a0 2023-07-14T18:13:47.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.941+0000 7f8fc4d9d700 1 -- 172.21.15.133:0/3901915710 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fc002c8e0 con 0x7f8fc00730a0 2023-07-14T18:13:47.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.942+0000 7f8fbcff9700 1 -- 172.21.15.133:0/3901915710 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2020 (secure 0 0 0) 0x7f8fb009ac50 con 0x7f8fc00730a0 2023-07-14T18:13:47.985 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:13:47.985 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/4023127259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4023127259},{"type":"v1","addr":"172.21.15.161:6833","nonce":4023127259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2722373740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2722373740},{"type":"v1","addr":"172.21.15.171:6833","nonce":2722373740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-14T18:13:47.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.945+0000 7f8fc4d9d700 1 -- 172.21.15.133:0/3901915710 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8fa80a12c0 msgr2=0x7f8fa80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:47.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.945+0000 7f8fc4d9d700 1 --2- 172.21.15.133:0/3901915710 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8fa80a12c0 0x7f8fa80a3770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4005950 tx=0x7f8fb4009260 comp rx=0 tx=0).stop 2023-07-14T18:13:47.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.945+0000 7f8fc4d9d700 1 -- 172.21.15.133:0/3901915710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fc00730a0 msgr2=0x7f8fc01978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:47.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.945+0000 7f8fc4d9d700 1 --2- 172.21.15.133:0/3901915710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fc00730a0 0x7f8fc01978b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000b680 tx=0x7f8fb0009580 comp rx=0 tx=0).stop 2023-07-14T18:13:47.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.945+0000 7f8fc4d9d700 1 -- 172.21.15.133:0/3901915710 shutdown_connections 2023-07-14T18:13:47.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.945+0000 7f8fc4d9d700 1 --2- 172.21.15.133:0/3901915710 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8fa80a12c0 0x7f8fa80a3770 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:47.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.945+0000 7f8fc4d9d700 1 --2- 172.21.15.133:0/3901915710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fc01079d0 0x7f8fc019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:47.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.945+0000 7f8fc4d9d700 1 --2- 172.21.15.133:0/3901915710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fc00730a0 0x7f8fc01978b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:47.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.945+0000 7f8fc4d9d700 1 --2- 172.21.15.133:0/3901915710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fc0074680 0x7f8fc0197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:47.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.945+0000 7f8fc4d9d700 1 -- 172.21.15.133:0/3901915710 >> 172.21.15.133:0/3901915710 conn(0x7f8fc00fb110 msgr2=0x7f8fc0109d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:47.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.945+0000 7f8fc4d9d700 1 -- 172.21.15.133:0/3901915710 shutdown_connections 2023-07-14T18:13:47.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:47.945+0000 7f8fc4d9d700 1 -- 172.21.15.133:0/3901915710 wait complete. 2023-07-14T18:13:47.989 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 3 2023-07-14T18:13:48.154 INFO:journalctl@ceph.mds.j.smithi133.stdout:Jul 14 18:13:47 smithi133 systemd[1]: Starting Ceph mds.j for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:13:48.640 INFO:journalctl@ceph.mds.j.smithi133.stdout:Jul 14 18:13:48 smithi133 podman[159733]: 2023-07-14T18:13:48.641 INFO:journalctl@ceph.mds.j.smithi133.stdout:Jul 14 18:13:48 smithi133 bash[159733]: 04633dea74e4706f11d7baf0a393710c217694d0a996508f601d444eb4067f1b 2023-07-14T18:13:48.641 INFO:journalctl@ceph.mds.j.smithi133.stdout:Jul 14 18:13:48 smithi133 ceph-mds[159773]: -- 172.21.15.133:0/1380051789 <== mon.0 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5608952451e0 con 0x56089521b000 2023-07-14T18:13:48.641 INFO:journalctl@ceph.mds.j.smithi133.stdout:Jul 14 18:13:48 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-j[159749]: starting mds.j at 2023-07-14T18:13:48.904 INFO:journalctl@ceph.mds.j.smithi133.stdout:Jul 14 18:13:48 smithi133 systemd[1]: Started Ceph mds.j for 06d0f5fa-2271-11ee-9b33-001a4aab830c. 2023-07-14T18:13:49.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:48 smithi161 ceph-mon[128993]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:49.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:48 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3901915710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:13:49.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:49.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:49.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:49.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:13:49.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:13:49.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:49.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:48 smithi171 ceph-mon[129241]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:49.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:48 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3901915710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:13:49.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:49.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:49.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:49.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:13:49.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:13:49.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:49.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:48 smithi133 ceph-mon[126384]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:49.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:48 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3901915710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:13:49.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:49.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:49.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:49.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:13:49.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:13:49.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:49.997 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:49 smithi133 ceph-mon[126384]: Deploying daemon mds.d on smithi133 2023-07-14T18:13:49.997 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:49 smithi133 ceph-mon[126384]: mds.? [v2:172.21.15.133:6834/2399005074,v1:172.21.15.133:6835/2399005074] up:boot 2023-07-14T18:13:49.997 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:49 smithi133 ceph-mon[126384]: fsmap 3 up:standby 2023-07-14T18:13:49.997 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:49 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-14T18:13:50.008 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T18:13:50.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.171+0000 7f0f5d7ff700 1 -- 172.21.15.133:0/492602294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f58100e20 msgr2=0x7f0f58101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:50.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.171+0000 7f0f5d7ff700 1 --2- 172.21.15.133:0/492602294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f58100e20 0x7f0f58101270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c009990 tx=0x7f0f4c009ca0 comp rx=0 tx=0).stop 2023-07-14T18:13:50.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.171+0000 7f0f5d7ff700 1 -- 172.21.15.133:0/492602294 shutdown_connections 2023-07-14T18:13:50.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.171+0000 7f0f5d7ff700 1 --2- 172.21.15.133:0/492602294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f581017b0 0x7f0f58108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:50.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.171+0000 7f0f5d7ff700 1 --2- 172.21.15.133:0/492602294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f580ffc30 0x7f0f58100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:50.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.171+0000 7f0f5d7ff700 1 --2- 172.21.15.133:0/492602294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f58100e20 0x7f0f58101270 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:50.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.171+0000 7f0f5d7ff700 1 -- 172.21.15.133:0/492602294 >> 172.21.15.133:0/492602294 conn(0x7f0f580fb210 msgr2=0x7f0f580fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:50.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.171+0000 7f0f5d7ff700 1 -- 172.21.15.133:0/492602294 shutdown_connections 2023-07-14T18:13:50.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.171+0000 7f0f5d7ff700 1 -- 172.21.15.133:0/492602294 wait complete. 2023-07-14T18:13:50.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.172+0000 7f0f5d7ff700 1 Processor -- start 2023-07-14T18:13:50.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.172+0000 7f0f5d7ff700 1 -- start start 2023-07-14T18:13:50.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.172+0000 7f0f5d7ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f580ffc30 0x7f0f580743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:50.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.172+0000 7f0f5d7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f58100e20 0x7f0f58072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:50.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.173+0000 7f0f5d7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f581017b0 0x7f0f58072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:50.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.173+0000 7f0f5d7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f58073710 con 0x7f0f58100e20 2023-07-14T18:13:50.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.173+0000 7f0f5d7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f58073880 con 0x7f0f581017b0 2023-07-14T18:13:50.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.173+0000 7f0f5d7ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5819f990 con 0x7f0f580ffc30 2023-07-14T18:13:50.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.173+0000 7f0f567fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f58100e20 0x7f0f58072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:50.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.173+0000 7f0f567fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f58100e20 0x7f0f58072a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50208/0 (socket says 172.21.15.133:50208) 2023-07-14T18:13:50.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.173+0000 7f0f567fc700 1 -- 172.21.15.133:0/1843691716 learned_addr learned my addr 172.21.15.133:0/1843691716 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:13:50.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.173+0000 7f0f577fe700 1 --2- 172.21.15.133:0/1843691716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f581017b0 0x7f0f58072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:50.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.173+0000 7f0f56ffd700 1 --2- 172.21.15.133:0/1843691716 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f580ffc30 0x7f0f580743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:50.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.173+0000 7f0f567fc700 1 -- 172.21.15.133:0/1843691716 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f580ffc30 msgr2=0x7f0f580743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:50.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.173+0000 7f0f567fc700 1 --2- 172.21.15.133:0/1843691716 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f580ffc30 0x7f0f580743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:50.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.173+0000 7f0f567fc700 1 -- 172.21.15.133:0/1843691716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f581017b0 msgr2=0x7f0f58072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:50.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.173+0000 7f0f567fc700 1 --2- 172.21.15.133:0/1843691716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f581017b0 0x7f0f58072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:50.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.173+0000 7f0f567fc700 1 -- 172.21.15.133:0/1843691716 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4c0095f0 con 0x7f0f58100e20 2023-07-14T18:13:50.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.173+0000 7f0f577fe700 1 --2- 172.21.15.133:0/1843691716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f581017b0 0x7f0f58072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:13:50.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.173+0000 7f0f567fc700 1 --2- 172.21.15.133:0/1843691716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f58100e20 0x7f0f58072a30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c005910 tx=0x7f0f4c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:50.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.174+0000 7f0f3ffff700 1 -- 172.21.15.133:0/1843691716 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c01d070 con 0x7f0f58100e20 2023-07-14T18:13:50.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.174+0000 7f0f3ffff700 1 -- 172.21.15.133:0/1843691716 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f4c003f60 con 0x7f0f58100e20 2023-07-14T18:13:50.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.174+0000 7f0f5d7ff700 1 -- 172.21.15.133:0/1843691716 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f5819fc10 con 0x7f0f58100e20 2023-07-14T18:13:50.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.174+0000 7f0f3ffff700 1 -- 172.21.15.133:0/1843691716 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c022c30 con 0x7f0f58100e20 2023-07-14T18:13:50.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.174+0000 7f0f5d7ff700 1 -- 172.21.15.133:0/1843691716 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f581a0030 con 0x7f0f58100e20 2023-07-14T18:13:50.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.175+0000 7f0f5d7ff700 1 -- 172.21.15.133:0/1843691716 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f58107120 con 0x7f0f58100e20 2023-07-14T18:13:50.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.177+0000 7f0f3ffff700 1 -- 172.21.15.133:0/1843691716 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f0f4c0040d0 con 0x7f0f58100e20 2023-07-14T18:13:50.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.178+0000 7f0f3ffff700 1 --2- 172.21.15.133:0/1843691716 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0f400a11a0 0x7f0f400a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:50.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.178+0000 7f0f3ffff700 1 -- 172.21.15.133:0/1843691716 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0f4c0ce0f0 con 0x7f0f58100e20 2023-07-14T18:13:50.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.178+0000 7f0f56ffd700 1 --2- 172.21.15.133:0/1843691716 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0f400a11a0 0x7f0f400a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:50.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.179+0000 7f0f56ffd700 1 --2- 172.21.15.133:0/1843691716 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0f400a11a0 0x7f0f400a3650 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0f58100c80 tx=0x7f0f4800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:50.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.180+0000 7f0f3ffff700 1 -- 172.21.15.133:0/1843691716 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f0f4c097cd0 con 0x7f0f58100e20 2023-07-14T18:13:50.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:49 smithi161 ceph-mon[128993]: Deploying daemon mds.d on smithi133 2023-07-14T18:13:50.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:49 smithi161 ceph-mon[128993]: mds.? [v2:172.21.15.133:6834/2399005074,v1:172.21.15.133:6835/2399005074] up:boot 2023-07-14T18:13:50.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:49 smithi161 ceph-mon[128993]: fsmap 3 up:standby 2023-07-14T18:13:50.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:49 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-14T18:13:50.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:49 smithi171 ceph-mon[129241]: Deploying daemon mds.d on smithi133 2023-07-14T18:13:50.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:49 smithi171 ceph-mon[129241]: mds.? [v2:172.21.15.133:6834/2399005074,v1:172.21.15.133:6835/2399005074] up:boot 2023-07-14T18:13:50.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:49 smithi171 ceph-mon[129241]: fsmap 3 up:standby 2023-07-14T18:13:50.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:49 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-14T18:13:50.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.460+0000 7f0f5d7ff700 1 -- 172.21.15.133:0/1843691716 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f5802cd00 con 0x7f0f58100e20 2023-07-14T18:13:50.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.461+0000 7f0f3ffff700 1 -- 172.21.15.133:0/1843691716 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2780 (secure 0 0 0) 0x7f0f4c09b2f0 con 0x7f0f58100e20 2023-07-14T18:13:50.504 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:13:50.504 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/4023127259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4023127259},{"type":"v1","addr":"172.21.15.161:6833","nonce":4023127259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2722373740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2722373740},{"type":"v1","addr":"172.21.15.171:6833","nonce":2722373740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2399005074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2399005074},{"type":"v1","addr":"172.21.15.133:6835","nonce":2399005074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[]} 2023-07-14T18:13:50.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.463+0000 7f0f5d7ff700 1 -- 172.21.15.133:0/1843691716 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0f400a11a0 msgr2=0x7f0f400a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:50.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.463+0000 7f0f5d7ff700 1 --2- 172.21.15.133:0/1843691716 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0f400a11a0 0x7f0f400a3650 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0f58100c80 tx=0x7f0f4800b410 comp rx=0 tx=0).stop 2023-07-14T18:13:50.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.463+0000 7f0f5d7ff700 1 -- 172.21.15.133:0/1843691716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f58100e20 msgr2=0x7f0f58072a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:50.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.463+0000 7f0f5d7ff700 1 --2- 172.21.15.133:0/1843691716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f58100e20 0x7f0f58072a30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c005910 tx=0x7f0f4c005740 comp rx=0 tx=0).stop 2023-07-14T18:13:50.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.463+0000 7f0f5d7ff700 1 -- 172.21.15.133:0/1843691716 shutdown_connections 2023-07-14T18:13:50.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.463+0000 7f0f5d7ff700 1 --2- 172.21.15.133:0/1843691716 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0f400a11a0 0x7f0f400a3650 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:50.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.463+0000 7f0f5d7ff700 1 --2- 172.21.15.133:0/1843691716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f581017b0 0x7f0f58072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:50.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.463+0000 7f0f5d7ff700 1 --2- 172.21.15.133:0/1843691716 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f580ffc30 0x7f0f580743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:50.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.463+0000 7f0f5d7ff700 1 --2- 172.21.15.133:0/1843691716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f58100e20 0x7f0f58072a30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:50.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.463+0000 7f0f5d7ff700 1 -- 172.21.15.133:0/1843691716 >> 172.21.15.133:0/1843691716 conn(0x7f0f580fb210 msgr2=0x7f0f58104260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:50.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.463+0000 7f0f5d7ff700 1 -- 172.21.15.133:0/1843691716 shutdown_connections 2023-07-14T18:13:50.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:50.463+0000 7f0f5d7ff700 1 -- 172.21.15.133:0/1843691716 wait complete. 2023-07-14T18:13:50.508 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 4 2023-07-14T18:13:51.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:50 smithi133 ceph-mon[126384]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:51.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:50 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1843691716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:13:51.155 INFO:journalctl@ceph.mds.d.smithi133.stdout:Jul 14 18:13:50 smithi133 systemd[1]: Starting Ceph mds.d for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:13:51.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:50 smithi161 ceph-mon[128993]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:51.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:50 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1843691716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:13:51.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:50 smithi171 ceph-mon[129241]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:51.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:50 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1843691716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:13:51.732 INFO:journalctl@ceph.mds.d.smithi133.stdout:Jul 14 18:13:51 smithi133 podman[160118]: 2023-07-14T18:13:51.733 INFO:journalctl@ceph.mds.d.smithi133.stdout:Jul 14 18:13:51 smithi133 ceph-mds[160158]: -- 172.21.15.133:0/512639846 <== mon.0 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55df313451e0 con 0x55df3131ac00 2023-07-14T18:13:51.733 INFO:journalctl@ceph.mds.d.smithi133.stdout:Jul 14 18:13:51 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-d[160134]: starting mds.d at 2023-07-14T18:13:51.733 INFO:journalctl@ceph.mds.d.smithi133.stdout:Jul 14 18:13:51 smithi133 bash[160118]: 5e4aae41fd129e2f3fbc571f87c9d70fdbdcddec5523eb6f9f0f97d52bee8d6f 2023-07-14T18:13:52.154 INFO:journalctl@ceph.mds.d.smithi133.stdout:Jul 14 18:13:51 smithi133 systemd[1]: Started Ceph mds.d for 06d0f5fa-2271-11ee-9b33-001a4aab830c. 2023-07-14T18:13:52.527 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T18:13:52.735 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.692+0000 7fe2f9bff700 1 -- 172.21.15.133:0/850361694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2f41017b0 msgr2=0x7fe2f4108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:52.735 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.692+0000 7fe2f9bff700 1 --2- 172.21.15.133:0/850361694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2f41017b0 0x7fe2f4108070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec00b0a0 tx=0x7fe2ec00b3b0 comp rx=0 tx=0).stop 2023-07-14T18:13:52.735 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.692+0000 7fe2f9bff700 1 -- 172.21.15.133:0/850361694 shutdown_connections 2023-07-14T18:13:52.735 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.692+0000 7fe2f9bff700 1 --2- 172.21.15.133:0/850361694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2f41017b0 0x7fe2f4108070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:52.735 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.692+0000 7fe2f9bff700 1 --2- 172.21.15.133:0/850361694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2f40ffc30 0x7fe2f4100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:52.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.692+0000 7fe2f9bff700 1 --2- 172.21.15.133:0/850361694 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2f4100e20 0x7fe2f4101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:52.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.692+0000 7fe2f9bff700 1 -- 172.21.15.133:0/850361694 >> 172.21.15.133:0/850361694 conn(0x7fe2f40fb210 msgr2=0x7fe2f40fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:52.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.692+0000 7fe2f9bff700 1 -- 172.21.15.133:0/850361694 shutdown_connections 2023-07-14T18:13:52.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.692+0000 7fe2f9bff700 1 -- 172.21.15.133:0/850361694 wait complete. 2023-07-14T18:13:52.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.693+0000 7fe2f9bff700 1 Processor -- start 2023-07-14T18:13:52.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.694+0000 7fe2f9bff700 1 -- start start 2023-07-14T18:13:52.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.694+0000 7fe2f9bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2f40ffc30 0x7fe2f41959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:52.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.694+0000 7fe2f9bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2f4100e20 0x7fe2f4195f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:52.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.694+0000 7fe2f9bff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2f41017b0 0x7fe2f419cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:52.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.694+0000 7fe2f9bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f419d540 con 0x7fe2f40ffc30 2023-07-14T18:13:52.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.694+0000 7fe2f9bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f419d680 con 0x7fe2f4100e20 2023-07-14T18:13:52.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.694+0000 7fe2f9bff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f419d810 con 0x7fe2f41017b0 2023-07-14T18:13:52.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.694+0000 7fe2f37fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2f40ffc30 0x7fe2f41959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:52.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.694+0000 7fe2f37fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2f40ffc30 0x7fe2f41959c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50236/0 (socket says 172.21.15.133:50236) 2023-07-14T18:13:52.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.694+0000 7fe2f37fe700 1 -- 172.21.15.133:0/951559507 learned_addr learned my addr 172.21.15.133:0/951559507 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:13:52.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.694+0000 7fe2f2ffd700 1 --2- 172.21.15.133:0/951559507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2f4100e20 0x7fe2f4195f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:52.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.694+0000 7fe2f3fff700 1 --2- 172.21.15.133:0/951559507 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2f41017b0 0x7fe2f419cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:52.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.695+0000 7fe2f37fe700 1 -- 172.21.15.133:0/951559507 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2f41017b0 msgr2=0x7fe2f419cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:52.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.695+0000 7fe2f37fe700 1 --2- 172.21.15.133:0/951559507 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2f41017b0 0x7fe2f419cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:52.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.695+0000 7fe2f37fe700 1 -- 172.21.15.133:0/951559507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2f4100e20 msgr2=0x7fe2f4195f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:52.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.695+0000 7fe2f37fe700 1 --2- 172.21.15.133:0/951559507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2f4100e20 0x7fe2f4195f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:52.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.695+0000 7fe2f37fe700 1 -- 172.21.15.133:0/951559507 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e4009520 con 0x7fe2f40ffc30 2023-07-14T18:13:52.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.695+0000 7fe2f37fe700 1 --2- 172.21.15.133:0/951559507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2f40ffc30 0x7fe2f41959c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400ea50 tx=0x7fe2e400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:52.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.695+0000 7fe2f0ff9700 1 -- 172.21.15.133:0/951559507 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e400cbd0 con 0x7fe2f40ffc30 2023-07-14T18:13:52.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.695+0000 7fe2f9bff700 1 -- 172.21.15.133:0/951559507 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2ec009d00 con 0x7fe2f40ffc30 2023-07-14T18:13:52.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.695+0000 7fe2f0ff9700 1 -- 172.21.15.133:0/951559507 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2e400cd30 con 0x7fe2f40ffc30 2023-07-14T18:13:52.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.695+0000 7fe2f0ff9700 1 -- 172.21.15.133:0/951559507 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4018680 con 0x7fe2f40ffc30 2023-07-14T18:13:52.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.695+0000 7fe2f9bff700 1 -- 172.21.15.133:0/951559507 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f419dec0 con 0x7fe2f40ffc30 2023-07-14T18:13:52.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.696+0000 7fe2f9bff700 1 -- 172.21.15.133:0/951559507 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f410b410 con 0x7fe2f40ffc30 2023-07-14T18:13:52.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.699+0000 7fe2f0ff9700 1 -- 172.21.15.133:0/951559507 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe2e4010430 con 0x7fe2f40ffc30 2023-07-14T18:13:52.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.700+0000 7fe2f0ff9700 1 --2- 172.21.15.133:0/951559507 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe2dc0a11a0 0x7fe2dc0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:52.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.700+0000 7fe2f0ff9700 1 -- 172.21.15.133:0/951559507 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe2e4014070 con 0x7fe2f40ffc30 2023-07-14T18:13:52.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.700+0000 7fe2f2ffd700 1 --2- 172.21.15.133:0/951559507 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe2dc0a11a0 0x7fe2dc0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:52.743 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.700+0000 7fe2f2ffd700 1 --2- 172.21.15.133:0/951559507 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe2dc0a11a0 0x7fe2dc0a3650 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8009c60 tx=0x7fe2e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:52.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.701+0000 7fe2f0ff9700 1 -- 172.21.15.133:0/951559507 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fe2e4096b00 con 0x7fe2f40ffc30 2023-07-14T18:13:52.904 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:52 smithi133 ceph-mon[126384]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:52.905 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:52 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:52.905 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:52 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:52.905 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:52 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:52.905 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:52 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:13:52.905 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:52 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:13:52.906 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:52 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:52.906 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:52 smithi133 ceph-mon[126384]: Deploying daemon mds.e on smithi161 2023-07-14T18:13:52.906 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:52 smithi133 ceph-mon[126384]: mds.? [v2:172.21.15.133:6836/3168151271,v1:172.21.15.133:6837/3168151271] up:boot 2023-07-14T18:13:52.906 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:52 smithi133 ceph-mon[126384]: fsmap 4 up:standby 2023-07-14T18:13:52.906 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:52 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-14T18:13:52.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.946+0000 7fe2f9bff700 1 -- 172.21.15.133:0/951559507 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2f41069f0 con 0x7fe2f40ffc30 2023-07-14T18:13:52.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.946+0000 7fe2f0ff9700 1 -- 172.21.15.133:0/951559507 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3540 (secure 0 0 0) 0x7fe2e409a120 con 0x7fe2f40ffc30 2023-07-14T18:13:52.989 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:13:52.989 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3168151271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3168151271},{"type":"v1","addr":"172.21.15.133:6837","nonce":3168151271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/4023127259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4023127259},{"type":"v1","addr":"172.21.15.161:6833","nonce":4023127259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2722373740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2722373740},{"type":"v1","addr":"172.21.15.171:6833","nonce":2722373740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2399005074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2399005074},{"type":"v1","addr":"172.21.15.133:6835","nonce":2399005074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[]} 2023-07-14T18:13:52.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.949+0000 7fe2f9bff700 1 -- 172.21.15.133:0/951559507 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe2dc0a11a0 msgr2=0x7fe2dc0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:52.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.949+0000 7fe2f9bff700 1 --2- 172.21.15.133:0/951559507 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe2dc0a11a0 0x7fe2dc0a3650 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8009c60 tx=0x7fe2e8009260 comp rx=0 tx=0).stop 2023-07-14T18:13:52.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.949+0000 7fe2f9bff700 1 -- 172.21.15.133:0/951559507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2f40ffc30 msgr2=0x7fe2f41959c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.949+0000 7fe2f9bff700 1 --2- 172.21.15.133:0/951559507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2f40ffc30 0x7fe2f41959c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400ea50 tx=0x7fe2e400ed60 comp rx=0 tx=0).stop 2023-07-14T18:13:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.949+0000 7fe2f9bff700 1 -- 172.21.15.133:0/951559507 shutdown_connections 2023-07-14T18:13:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.949+0000 7fe2f9bff700 1 --2- 172.21.15.133:0/951559507 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe2dc0a11a0 0x7fe2dc0a3650 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.949+0000 7fe2f9bff700 1 --2- 172.21.15.133:0/951559507 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2f41017b0 0x7fe2f419cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.949+0000 7fe2f9bff700 1 --2- 172.21.15.133:0/951559507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2f40ffc30 0x7fe2f41959c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.949+0000 7fe2f9bff700 1 --2- 172.21.15.133:0/951559507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2f4100e20 0x7fe2f4195f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.949+0000 7fe2f9bff700 1 -- 172.21.15.133:0/951559507 >> 172.21.15.133:0/951559507 conn(0x7fe2f40fb210 msgr2=0x7fe2f40fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.949+0000 7fe2f9bff700 1 -- 172.21.15.133:0/951559507 shutdown_connections 2023-07-14T18:13:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:52.949+0000 7fe2f9bff700 1 -- 172.21.15.133:0/951559507 wait complete. 2023-07-14T18:13:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 5 2023-07-14T18:13:53.065 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:52 smithi161 ceph-mon[128993]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:53.065 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:52 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:53.065 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:52 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:53.066 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:52 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:53.066 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:52 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:13:53.066 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:52 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:13:53.066 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:52 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:53.066 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:52 smithi161 ceph-mon[128993]: Deploying daemon mds.e on smithi161 2023-07-14T18:13:53.066 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:52 smithi161 ceph-mon[128993]: mds.? [v2:172.21.15.133:6836/3168151271,v1:172.21.15.133:6837/3168151271] up:boot 2023-07-14T18:13:53.066 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:52 smithi161 ceph-mon[128993]: fsmap 4 up:standby 2023-07-14T18:13:53.067 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:52 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-14T18:13:53.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:52 smithi171 ceph-mon[129241]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:53.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:52 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:53.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:52 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:53.115 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:52 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:53.115 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:52 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:13:53.115 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:52 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:13:53.115 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:52 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:53.115 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:52 smithi171 ceph-mon[129241]: Deploying daemon mds.e on smithi161 2023-07-14T18:13:53.116 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:52 smithi171 ceph-mon[129241]: mds.? [v2:172.21.15.133:6836/3168151271,v1:172.21.15.133:6837/3168151271] up:boot 2023-07-14T18:13:53.116 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:52 smithi171 ceph-mon[129241]: fsmap 4 up:standby 2023-07-14T18:13:53.116 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:52 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-14T18:13:53.348 INFO:journalctl@ceph.mds.e.smithi161.stdout:Jul 14 18:13:53 smithi161 systemd[1]: Starting Ceph mds.e for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:13:53.848 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:53 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/951559507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:13:53.848 INFO:journalctl@ceph.mds.e.smithi161.stdout:Jul 14 18:13:53 smithi161 podman[153675]: 2023-07-14T18:13:54.098 INFO:journalctl@ceph.mds.e.smithi161.stdout:Jul 14 18:13:53 smithi161 bash[153675]: d2d3f6e9dcbfb14699c02acc992cc5beb411158b013974f8560ab42ec1edbf65 2023-07-14T18:13:54.099 INFO:journalctl@ceph.mds.e.smithi161.stdout:Jul 14 18:13:53 smithi161 ceph-mds[153715]: -- 172.21.15.161:0/396724971 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b5638731e0 con 0x55b563849000 2023-07-14T18:13:54.099 INFO:journalctl@ceph.mds.e.smithi161.stdout:Jul 14 18:13:53 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-e[153691]: starting mds.e at 2023-07-14T18:13:54.099 INFO:journalctl@ceph.mds.e.smithi161.stdout:Jul 14 18:13:53 smithi161 systemd[1]: Started Ceph mds.e for 06d0f5fa-2271-11ee-9b33-001a4aab830c. 2023-07-14T18:13:54.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:53 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/951559507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:13:54.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:53 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/951559507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:13:55.012 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T18:13:55.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.175+0000 7f948d7ff700 1 -- 172.21.15.133:0/655985408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9488100da0 msgr2=0x7f94881011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:55.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.175+0000 7f948d7ff700 1 --2- 172.21.15.133:0/655985408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9488100da0 0x7f94881011f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f947c009990 tx=0x7f947c009ca0 comp rx=0 tx=0).stop 2023-07-14T18:13:55.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.175+0000 7f948d7ff700 1 -- 172.21.15.133:0/655985408 shutdown_connections 2023-07-14T18:13:55.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.175+0000 7f948d7ff700 1 --2- 172.21.15.133:0/655985408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9488101730 0x7f9488107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:55.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.175+0000 7f948d7ff700 1 --2- 172.21.15.133:0/655985408 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94880ffbb0 0x7f94880fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:55.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.175+0000 7f948d7ff700 1 --2- 172.21.15.133:0/655985408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9488100da0 0x7f94881011f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:55.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.175+0000 7f948d7ff700 1 -- 172.21.15.133:0/655985408 >> 172.21.15.133:0/655985408 conn(0x7f94880fb1b0 msgr2=0x7f94880fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:55.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.175+0000 7f948d7ff700 1 -- 172.21.15.133:0/655985408 shutdown_connections 2023-07-14T18:13:55.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.175+0000 7f948d7ff700 1 -- 172.21.15.133:0/655985408 wait complete. 2023-07-14T18:13:55.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.176+0000 7f948d7ff700 1 Processor -- start 2023-07-14T18:13:55.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.176+0000 7f948d7ff700 1 -- start start 2023-07-14T18:13:55.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.177+0000 7f948d7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94880ffbb0 0x7f9488197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:55.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.177+0000 7f948d7ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9488100da0 0x7f9488197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:55.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.177+0000 7f948d7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9488101730 0x7f948819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:55.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.177+0000 7f948d7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948819f620 con 0x7f94880ffbb0 2023-07-14T18:13:55.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.177+0000 7f948d7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948819f760 con 0x7f9488101730 2023-07-14T18:13:55.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.177+0000 7f948d7ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948819f8f0 con 0x7f9488100da0 2023-07-14T18:13:55.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.177+0000 7f9486ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94880ffbb0 0x7f9488197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:55.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.177+0000 7f94867fc700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9488100da0 0x7f9488197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:55.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.177+0000 7f94867fc700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9488100da0 0x7f9488197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:40076/0 (socket says 172.21.15.133:40076) 2023-07-14T18:13:55.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.177+0000 7f94867fc700 1 -- 172.21.15.133:0/644812809 learned_addr learned my addr 172.21.15.133:0/644812809 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:13:55.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.177+0000 7f94877fe700 1 --2- 172.21.15.133:0/644812809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9488101730 0x7f948819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:55.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.177+0000 7f94867fc700 1 -- 172.21.15.133:0/644812809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9488101730 msgr2=0x7f948819f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:55.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.177+0000 7f94867fc700 1 --2- 172.21.15.133:0/644812809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9488101730 0x7f948819f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:55.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.177+0000 7f94867fc700 1 -- 172.21.15.133:0/644812809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94880ffbb0 msgr2=0x7f9488197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:55.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.177+0000 7f94867fc700 1 --2- 172.21.15.133:0/644812809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94880ffbb0 0x7f9488197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:55.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.177+0000 7f94867fc700 1 -- 172.21.15.133:0/644812809 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947c0095f0 con 0x7f9488100da0 2023-07-14T18:13:55.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.177+0000 7f9486ffd700 1 --2- 172.21.15.133:0/644812809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94880ffbb0 0x7f9488197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:13:55.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.178+0000 7f94867fc700 1 --2- 172.21.15.133:0/644812809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9488100da0 0x7f9488197fe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f947c004760 tx=0x7f947c004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:55.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.178+0000 7f946ffff700 1 -- 172.21.15.133:0/644812809 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c01d070 con 0x7f9488100da0 2023-07-14T18:13:55.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.178+0000 7f946ffff700 1 -- 172.21.15.133:0/644812809 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f947c004830 con 0x7f9488100da0 2023-07-14T18:13:55.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.178+0000 7f948d7ff700 1 -- 172.21.15.133:0/644812809 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948819fb70 con 0x7f9488100da0 2023-07-14T18:13:55.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.178+0000 7f946ffff700 1 -- 172.21.15.133:0/644812809 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c005350 con 0x7f9488100da0 2023-07-14T18:13:55.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.178+0000 7f948d7ff700 1 -- 172.21.15.133:0/644812809 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948819ff30 con 0x7f9488100da0 2023-07-14T18:13:55.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.179+0000 7f948d7ff700 1 -- 172.21.15.133:0/644812809 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948810b440 con 0x7f9488100da0 2023-07-14T18:13:55.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.181+0000 7f946ffff700 1 -- 172.21.15.133:0/644812809 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f947c005510 con 0x7f9488100da0 2023-07-14T18:13:55.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.182+0000 7f946ffff700 1 --2- 172.21.15.133:0/644812809 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f94700a11a0 0x7f94700a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:55.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.182+0000 7f946ffff700 1 -- 172.21.15.133:0/644812809 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f947c0cd6b0 con 0x7f9488100da0 2023-07-14T18:13:55.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.182+0000 7f9486ffd700 1 --2- 172.21.15.133:0/644812809 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f94700a11a0 0x7f94700a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:55.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.183+0000 7f9486ffd700 1 --2- 172.21.15.133:0/644812809 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f94700a11a0 0x7f94700a3650 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9488100c00 tx=0x7f9478009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:55.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.186+0000 7f946ffff700 1 -- 172.21.15.133:0/644812809 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f947c05a780 con 0x7f9488100da0 2023-07-14T18:13:55.305 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:54 smithi171 ceph-mon[129241]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:55.305 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:54 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:55.305 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:54 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:55.306 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:54 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:55.306 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:54 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:13:55.306 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:54 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:13:55.306 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:54 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:55.306 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:54 smithi171 ceph-mon[129241]: Deploying daemon mds.f on smithi171 2023-07-14T18:13:55.306 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:54 smithi171 ceph-mon[129241]: mds.? [v2:172.21.15.161:6834/3725502564,v1:172.21.15.161:6835/3725502564] up:boot 2023-07-14T18:13:55.306 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:54 smithi171 ceph-mon[129241]: fsmap 5 up:standby 2023-07-14T18:13:55.307 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:54 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T18:13:55.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:54 smithi161 ceph-mon[128993]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:55.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:54 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:55.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:54 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:55.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:54 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:55.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:54 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:13:55.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:54 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:13:55.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:54 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:55.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:54 smithi161 ceph-mon[128993]: Deploying daemon mds.f on smithi171 2023-07-14T18:13:55.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:54 smithi161 ceph-mon[128993]: mds.? [v2:172.21.15.161:6834/3725502564,v1:172.21.15.161:6835/3725502564] up:boot 2023-07-14T18:13:55.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:54 smithi161 ceph-mon[128993]: fsmap 5 up:standby 2023-07-14T18:13:55.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:54 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T18:13:55.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:54 smithi133 ceph-mon[126384]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:55.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:54 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:55.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:54 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:55.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:54 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:55.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:54 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:13:55.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:54 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:13:55.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:54 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:55.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:54 smithi133 ceph-mon[126384]: Deploying daemon mds.f on smithi171 2023-07-14T18:13:55.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:54 smithi133 ceph-mon[126384]: mds.? [v2:172.21.15.161:6834/3725502564,v1:172.21.15.161:6835/3725502564] up:boot 2023-07-14T18:13:55.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:54 smithi133 ceph-mon[126384]: fsmap 5 up:standby 2023-07-14T18:13:55.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:54 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T18:13:55.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.431+0000 7f948d7ff700 1 -- 172.21.15.133:0/644812809 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94881a0540 con 0x7f9488100da0 2023-07-14T18:13:55.475 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:13:55.475 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3168151271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3168151271},{"type":"v1","addr":"172.21.15.133:6837","nonce":3168151271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/4023127259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4023127259},{"type":"v1","addr":"172.21.15.161:6833","nonce":4023127259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2722373740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2722373740},{"type":"v1","addr":"172.21.15.171:6833","nonce":2722373740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2399005074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2399005074},{"type":"v1","addr":"172.21.15.133:6835","nonce":2399005074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3725502564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3725502564},{"type":"v1","addr":"172.21.15.161:6835","nonce":3725502564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-07-14T18:13:55.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.431+0000 7f946ffff700 1 -- 172.21.15.133:0/644812809 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4300 (secure 0 0 0) 0x7f947c097290 con 0x7f9488100da0 2023-07-14T18:13:55.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.434+0000 7f948d7ff700 1 -- 172.21.15.133:0/644812809 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f94700a11a0 msgr2=0x7f94700a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:55.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.434+0000 7f948d7ff700 1 --2- 172.21.15.133:0/644812809 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f94700a11a0 0x7f94700a3650 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9488100c00 tx=0x7f9478009260 comp rx=0 tx=0).stop 2023-07-14T18:13:55.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.434+0000 7f948d7ff700 1 -- 172.21.15.133:0/644812809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9488100da0 msgr2=0x7f9488197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:55.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.434+0000 7f948d7ff700 1 --2- 172.21.15.133:0/644812809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9488100da0 0x7f9488197fe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f947c004760 tx=0x7f947c004790 comp rx=0 tx=0).stop 2023-07-14T18:13:55.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.434+0000 7f948d7ff700 1 -- 172.21.15.133:0/644812809 shutdown_connections 2023-07-14T18:13:55.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.434+0000 7f948d7ff700 1 --2- 172.21.15.133:0/644812809 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f94700a11a0 0x7f94700a3650 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:55.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.434+0000 7f948d7ff700 1 --2- 172.21.15.133:0/644812809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9488101730 0x7f948819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:55.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.434+0000 7f948d7ff700 1 --2- 172.21.15.133:0/644812809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94880ffbb0 0x7f9488197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:55.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.434+0000 7f948d7ff700 1 --2- 172.21.15.133:0/644812809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9488100da0 0x7f9488197fe0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:55.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.434+0000 7f948d7ff700 1 -- 172.21.15.133:0/644812809 >> 172.21.15.133:0/644812809 conn(0x7f94880fb1b0 msgr2=0x7f94880fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:55.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.434+0000 7f948d7ff700 1 -- 172.21.15.133:0/644812809 shutdown_connections 2023-07-14T18:13:55.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:55.434+0000 7f948d7ff700 1 -- 172.21.15.133:0/644812809 wait complete. 2023-07-14T18:13:55.480 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 6 2023-07-14T18:13:55.614 INFO:journalctl@ceph.mds.f.smithi171.stdout:Jul 14 18:13:55 smithi171 systemd[1]: Starting Ceph mds.f for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:13:56.100 INFO:journalctl@ceph.mds.f.smithi171.stdout:Jul 14 18:13:55 smithi171 podman[154916]: 2023-07-14T18:13:56.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:55 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/644812809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:13:56.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:55 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/644812809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:13:56.364 INFO:journalctl@ceph.mds.f.smithi171.stdout:Jul 14 18:13:56 smithi171 ceph-mds[154956]: -- 172.21.15.171:0/1743092422 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5640ccfb51e0 con 0x5640ccf8a800 2023-07-14T18:13:56.364 INFO:journalctl@ceph.mds.f.smithi171.stdout:Jul 14 18:13:56 smithi171 bash[154916]: 68f88c720a0eddf3668415c824d48d781b5e47bf611ee75caa6c6bfab122717a 2023-07-14T18:13:56.364 INFO:journalctl@ceph.mds.f.smithi171.stdout:Jul 14 18:13:56 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-f[154932]: starting mds.f at 2023-07-14T18:13:56.364 INFO:journalctl@ceph.mds.f.smithi171.stdout:Jul 14 18:13:56 smithi171 systemd[1]: Started Ceph mds.f for 06d0f5fa-2271-11ee-9b33-001a4aab830c. 2023-07-14T18:13:56.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:55 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/644812809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:13:57.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:56 smithi171 ceph-mon[129241]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:57.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:56 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:57.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:56 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:57.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:56 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:57.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:56 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:13:57.115 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:56 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:13:57.115 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:56 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:57.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:56 smithi161 ceph-mon[128993]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:57.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:56 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:57.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:56 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:57.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:56 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:57.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:56 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:13:57.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:56 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:13:57.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:56 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:57.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:56 smithi133 ceph-mon[126384]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:57.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:56 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:57.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:56 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:57.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:56 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:57.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:56 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:13:57.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:56 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:13:57.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:56 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:57.498 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T18:13:57.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.663+0000 7f007a1ff700 1 -- 172.21.15.133:0/3948228671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00741017b0 msgr2=0x7f0074108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:57.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.663+0000 7f007a1ff700 1 --2- 172.21.15.133:0/3948228671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00741017b0 0x7f0074108070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f006c00b0a0 tx=0x7f006c00b3b0 comp rx=0 tx=0).stop 2023-07-14T18:13:57.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.663+0000 7f007a1ff700 1 -- 172.21.15.133:0/3948228671 shutdown_connections 2023-07-14T18:13:57.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.663+0000 7f007a1ff700 1 --2- 172.21.15.133:0/3948228671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00740ffc30 0x7f0074100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:57.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.663+0000 7f007a1ff700 1 --2- 172.21.15.133:0/3948228671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00741017b0 0x7f0074108070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:57.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.663+0000 7f007a1ff700 1 --2- 172.21.15.133:0/3948228671 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0074100e20 0x7f0074101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:57.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.663+0000 7f007a1ff700 1 -- 172.21.15.133:0/3948228671 >> 172.21.15.133:0/3948228671 conn(0x7f00740fb210 msgr2=0x7f00740fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:57.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.664+0000 7f007a1ff700 1 -- 172.21.15.133:0/3948228671 shutdown_connections 2023-07-14T18:13:57.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.664+0000 7f007a1ff700 1 -- 172.21.15.133:0/3948228671 wait complete. 2023-07-14T18:13:57.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.664+0000 7f007a1ff700 1 Processor -- start 2023-07-14T18:13:57.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.665+0000 7f007a1ff700 1 -- start start 2023-07-14T18:13:57.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.665+0000 7f007a1ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00740ffc30 0x7f00740743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:57.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.665+0000 7f007a1ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0074100e20 0x7f0074072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:57.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.665+0000 7f007a1ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f00741017b0 0x7f0074072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:57.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.665+0000 7f007a1ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0074073710 con 0x7f0074100e20 2023-07-14T18:13:57.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.665+0000 7f007a1ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0074073880 con 0x7f00740ffc30 2023-07-14T18:13:57.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.665+0000 7f007a1ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007419fa90 con 0x7f00741017b0 2023-07-14T18:13:57.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.665+0000 7f0072ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0074100e20 0x7f0074072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:57.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.665+0000 7f0073fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f00741017b0 0x7f0074072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:57.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.665+0000 7f0073fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f00741017b0 0x7f0074072f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:50116/0 (socket says 172.21.15.133:50116) 2023-07-14T18:13:57.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.665+0000 7f0072ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0074100e20 0x7f0074072a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44900/0 (socket says 172.21.15.133:44900) 2023-07-14T18:13:57.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.665+0000 7f0073fff700 1 -- 172.21.15.133:0/1538764398 learned_addr learned my addr 172.21.15.133:0/1538764398 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:13:57.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.665+0000 7f00737fe700 1 --2- 172.21.15.133:0/1538764398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00740ffc30 0x7f00740743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:57.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.666+0000 7f0073fff700 1 -- 172.21.15.133:0/1538764398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00740ffc30 msgr2=0x7f00740743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:57.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.666+0000 7f0073fff700 1 --2- 172.21.15.133:0/1538764398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00740ffc30 0x7f00740743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:57.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.666+0000 7f0073fff700 1 -- 172.21.15.133:0/1538764398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0074100e20 msgr2=0x7f0074072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:57.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.666+0000 7f0073fff700 1 --2- 172.21.15.133:0/1538764398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0074100e20 0x7f0074072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:57.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.666+0000 7f0073fff700 1 -- 172.21.15.133:0/1538764398 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006c009d00 con 0x7f00741017b0 2023-07-14T18:13:57.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.666+0000 7f0072ffd700 1 --2- 172.21.15.133:0/1538764398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0074100e20 0x7f0074072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:13:57.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.666+0000 7f0073fff700 1 --2- 172.21.15.133:0/1538764398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f00741017b0 0x7f0074072f70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f006c00d010 tx=0x7f006c00bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:57.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.666+0000 7f0070ff9700 1 -- 172.21.15.133:0/1538764398 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c00e040 con 0x7f00741017b0 2023-07-14T18:13:57.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.666+0000 7f007a1ff700 1 -- 172.21.15.133:0/1538764398 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f007419fd10 con 0x7f00741017b0 2023-07-14T18:13:57.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.666+0000 7f0070ff9700 1 -- 172.21.15.133:0/1538764398 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f006c003e10 con 0x7f00741017b0 2023-07-14T18:13:57.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.666+0000 7f0070ff9700 1 -- 172.21.15.133:0/1538764398 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c0052a0 con 0x7f00741017b0 2023-07-14T18:13:57.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.666+0000 7f007a1ff700 1 -- 172.21.15.133:0/1538764398 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00741a0130 con 0x7f00741017b0 2023-07-14T18:13:57.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.667+0000 7f007a1ff700 1 -- 172.21.15.133:0/1538764398 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f007410b4d0 con 0x7f00741017b0 2023-07-14T18:13:57.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.669+0000 7f0070ff9700 1 -- 172.21.15.133:0/1538764398 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f006c01d480 con 0x7f00741017b0 2023-07-14T18:13:57.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.670+0000 7f0070ff9700 1 --2- 172.21.15.133:0/1538764398 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f005c0a11a0 0x7f005c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:13:57.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.670+0000 7f0070ff9700 1 -- 172.21.15.133:0/1538764398 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f006c0cdb20 con 0x7f00741017b0 2023-07-14T18:13:57.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.671+0000 7f00737fe700 1 --2- 172.21.15.133:0/1538764398 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f005c0a11a0 0x7f005c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:13:57.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.671+0000 7f00737fe700 1 --2- 172.21.15.133:0/1538764398 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f005c0a11a0 0x7f005c0a3650 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0074100c80 tx=0x7f0064009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:13:57.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.673+0000 7f0070ff9700 1 -- 172.21.15.133:0/1538764398 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f006c097700 con 0x7f00741017b0 2023-07-14T18:13:57.953 INFO:journalctl@ceph.mds.c.smithi171.stdout:Jul 14 18:13:57 smithi171 systemd[1]: Starting Ceph mds.c for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:13:57.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.912+0000 7f007a1ff700 1 -- 172.21.15.133:0/1538764398 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0074106a60 con 0x7f00741017b0 2023-07-14T18:13:57.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.917+0000 7f0070ff9700 1 -- 172.21.15.133:0/1538764398 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5060 (secure 0 0 0) 0x7f006c09ad20 con 0x7f00741017b0 2023-07-14T18:13:57.960 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:13:57.960 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3168151271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3168151271},{"type":"v1","addr":"172.21.15.133:6837","nonce":3168151271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/4023127259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4023127259},{"type":"v1","addr":"172.21.15.161:6833","nonce":4023127259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2722373740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2722373740},{"type":"v1","addr":"172.21.15.171:6833","nonce":2722373740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2399005074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2399005074},{"type":"v1","addr":"172.21.15.133:6835","nonce":2399005074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3725502564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3725502564},{"type":"v1","addr":"172.21.15.161:6835","nonce":3725502564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1877362116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1877362116},{"type":"v1","addr":"172.21.15.171:6835","nonce":1877362116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[]} 2023-07-14T18:13:57.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.919+0000 7f007a1ff700 1 -- 172.21.15.133:0/1538764398 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f005c0a11a0 msgr2=0x7f005c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:57.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.919+0000 7f007a1ff700 1 --2- 172.21.15.133:0/1538764398 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f005c0a11a0 0x7f005c0a3650 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0074100c80 tx=0x7f0064009260 comp rx=0 tx=0).stop 2023-07-14T18:13:57.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.919+0000 7f007a1ff700 1 -- 172.21.15.133:0/1538764398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f00741017b0 msgr2=0x7f0074072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:13:57.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.919+0000 7f007a1ff700 1 --2- 172.21.15.133:0/1538764398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f00741017b0 0x7f0074072f70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f006c00d010 tx=0x7f006c00bc80 comp rx=0 tx=0).stop 2023-07-14T18:13:57.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.919+0000 7f007a1ff700 1 -- 172.21.15.133:0/1538764398 shutdown_connections 2023-07-14T18:13:57.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.919+0000 7f007a1ff700 1 --2- 172.21.15.133:0/1538764398 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f005c0a11a0 0x7f005c0a3650 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:57.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.919+0000 7f007a1ff700 1 --2- 172.21.15.133:0/1538764398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00740ffc30 0x7f00740743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:57.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.919+0000 7f007a1ff700 1 --2- 172.21.15.133:0/1538764398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f00741017b0 0x7f0074072f70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:57.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.919+0000 7f007a1ff700 1 --2- 172.21.15.133:0/1538764398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0074100e20 0x7f0074072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:13:57.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.919+0000 7f007a1ff700 1 -- 172.21.15.133:0/1538764398 >> 172.21.15.133:0/1538764398 conn(0x7f00740fb210 msgr2=0x7f00740fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:13:57.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.920+0000 7f007a1ff700 1 -- 172.21.15.133:0/1538764398 shutdown_connections 2023-07-14T18:13:57.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:13:57.920+0000 7f007a1ff700 1 -- 172.21.15.133:0/1538764398 wait complete. 2023-07-14T18:13:57.964 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 7 2023-07-14T18:13:58.226 INFO:journalctl@ceph.mds.c.smithi171.stdout:Jul 14 18:13:57 smithi171 podman[155263]: 2023-07-14T18:13:58.226 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:57 smithi171 ceph-mon[129241]: Deploying daemon mds.c on smithi171 2023-07-14T18:13:58.226 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:57 smithi171 ceph-mon[129241]: mds.? [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] up:boot 2023-07-14T18:13:58.226 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:57 smithi171 ceph-mon[129241]: fsmap 6 up:standby 2023-07-14T18:13:58.227 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:57 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T18:13:58.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:57 smithi161 ceph-mon[128993]: Deploying daemon mds.c on smithi171 2023-07-14T18:13:58.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:57 smithi161 ceph-mon[128993]: mds.? [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] up:boot 2023-07-14T18:13:58.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:57 smithi161 ceph-mon[128993]: fsmap 6 up:standby 2023-07-14T18:13:58.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:57 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T18:13:58.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:57 smithi133 ceph-mon[126384]: Deploying daemon mds.c on smithi171 2023-07-14T18:13:58.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:57 smithi133 ceph-mon[126384]: mds.? [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] up:boot 2023-07-14T18:13:58.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:57 smithi133 ceph-mon[126384]: fsmap 6 up:standby 2023-07-14T18:13:58.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:57 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T18:13:58.614 INFO:journalctl@ceph.mds.c.smithi171.stdout:Jul 14 18:13:58 smithi171 ceph-mds[155303]: -- 172.21.15.171:0/791336101 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d06a4371e0 con 0x55d06a40c800 2023-07-14T18:13:58.614 INFO:journalctl@ceph.mds.c.smithi171.stdout:Jul 14 18:13:58 smithi171 bash[155263]: 054f51021a87b95fdc8f25eba22055886c2935af7a183ae975b41fafb0406a13 2023-07-14T18:13:58.614 INFO:journalctl@ceph.mds.c.smithi171.stdout:Jul 14 18:13:58 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-c[155279]: starting mds.c at 2023-07-14T18:13:58.614 INFO:journalctl@ceph.mds.c.smithi171.stdout:Jul 14 18:13:58 smithi171 systemd[1]: Started Ceph mds.c for 06d0f5fa-2271-11ee-9b33-001a4aab830c. 2023-07-14T18:13:59.290 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:58 smithi161 ceph-mon[128993]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:59.290 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:58 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1538764398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:13:59.291 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:58 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:59.291 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:58 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:59.291 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:58 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:59.291 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:58 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:13:59.291 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:58 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:13:59.291 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:58 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:59.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:58 smithi171 ceph-mon[129241]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:59.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:58 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1538764398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:13:59.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:58 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:59.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:58 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:59.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:58 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:59.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:58 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:13:59.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:58 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:13:59.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:58 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:59.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:58 smithi133 ceph-mon[126384]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:13:59.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:58 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1538764398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:13:59.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:58 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:59.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:58 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:59.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:58 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:13:59.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:58 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:13:59.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:58 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:13:59.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:58 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:13:59.972 INFO:journalctl@ceph.mds.h.smithi161.stdout:Jul 14 18:13:59 smithi161 systemd[1]: Starting Ceph mds.h for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:13:59.981 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T18:14:00.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.146+0000 7ffa1bbff700 1 -- 172.21.15.133:0/2616281446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa1410c200 msgr2=0x7ffa1410e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:00.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.146+0000 7ffa1bbff700 1 --2- 172.21.15.133:0/2616281446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa1410c200 0x7ffa1410e5e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00b0a0 tx=0x7ffa0c00b3b0 comp rx=0 tx=0).stop 2023-07-14T18:14:00.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.146+0000 7ffa1bbff700 1 -- 172.21.15.133:0/2616281446 shutdown_connections 2023-07-14T18:14:00.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.146+0000 7ffa1bbff700 1 --2- 172.21.15.133:0/2616281446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa1410c200 0x7ffa1410e5e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:00.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.146+0000 7ffa1bbff700 1 --2- 172.21.15.133:0/2616281446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa14074680 0x7ffa14072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:00.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.146+0000 7ffa1bbff700 1 --2- 172.21.15.133:0/2616281446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa140730a0 0x7ffa14073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:00.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.146+0000 7ffa1bbff700 1 -- 172.21.15.133:0/2616281446 >> 172.21.15.133:0/2616281446 conn(0x7ffa140fb210 msgr2=0x7ffa140fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:00.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.146+0000 7ffa1bbff700 1 -- 172.21.15.133:0/2616281446 shutdown_connections 2023-07-14T18:14:00.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.146+0000 7ffa1bbff700 1 -- 172.21.15.133:0/2616281446 wait complete. 2023-07-14T18:14:00.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.147+0000 7ffa1bbff700 1 Processor -- start 2023-07-14T18:14:00.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.147+0000 7ffa1bbff700 1 -- start start 2023-07-14T18:14:00.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.148+0000 7ffa1bbff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa140730a0 0x7ffa1419c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:00.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.148+0000 7ffa1bbff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa14074680 0x7ffa1419c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:00.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.148+0000 7ffa1bbff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa1410c200 0x7ffa141a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:00.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.148+0000 7ffa1bbff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa141a3bd0 con 0x7ffa140730a0 2023-07-14T18:14:00.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.148+0000 7ffa1bbff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa141a3d10 con 0x7ffa1410c200 2023-07-14T18:14:00.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.148+0000 7ffa1bbff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa141a3ea0 con 0x7ffa14074680 2023-07-14T18:14:00.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.148+0000 7ffa1999b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa140730a0 0x7ffa1419c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:00.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.148+0000 7ffa1919a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa14074680 0x7ffa1419c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:00.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.148+0000 7ffa1919a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa14074680 0x7ffa1419c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:50138/0 (socket says 172.21.15.133:50138) 2023-07-14T18:14:00.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.148+0000 7ffa1919a700 1 -- 172.21.15.133:0/4011568760 learned_addr learned my addr 172.21.15.133:0/4011568760 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:00.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.148+0000 7ffa1a19c700 1 --2- 172.21.15.133:0/4011568760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa1410c200 0x7ffa141a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:00.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.148+0000 7ffa1999b700 1 -- 172.21.15.133:0/4011568760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa14074680 msgr2=0x7ffa1419c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:00.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.148+0000 7ffa1999b700 1 --2- 172.21.15.133:0/4011568760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa14074680 0x7ffa1419c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:00.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.148+0000 7ffa1999b700 1 -- 172.21.15.133:0/4011568760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa1410c200 msgr2=0x7ffa141a3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:00.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.148+0000 7ffa1999b700 1 --2- 172.21.15.133:0/4011568760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa1410c200 0x7ffa141a3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:00.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.148+0000 7ffa1999b700 1 -- 172.21.15.133:0/4011568760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0c009d00 con 0x7ffa140730a0 2023-07-14T18:14:00.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.148+0000 7ffa1919a700 1 --2- 172.21.15.133:0/4011568760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa14074680 0x7ffa1419c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:14:00.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.149+0000 7ffa1999b700 1 --2- 172.21.15.133:0/4011568760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa140730a0 0x7ffa1419c050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ffa14074a50 tx=0x7ffa1000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:00.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.149+0000 7ffa0affd700 1 -- 172.21.15.133:0/4011568760 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1000f890 con 0x7ffa140730a0 2023-07-14T18:14:00.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.149+0000 7ffa0affd700 1 -- 172.21.15.133:0/4011568760 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa10010460 con 0x7ffa140730a0 2023-07-14T18:14:00.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.149+0000 7ffa1bbff700 1 -- 172.21.15.133:0/4011568760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa141a4180 con 0x7ffa140730a0 2023-07-14T18:14:00.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.149+0000 7ffa0affd700 1 -- 172.21.15.133:0/4011568760 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1000e610 con 0x7ffa140730a0 2023-07-14T18:14:00.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.149+0000 7ffa1bbff700 1 -- 172.21.15.133:0/4011568760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa141a46e0 con 0x7ffa140730a0 2023-07-14T18:14:00.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.150+0000 7ffa1bbff700 1 -- 172.21.15.133:0/4011568760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa14102460 con 0x7ffa140730a0 2023-07-14T18:14:00.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.152+0000 7ffa0affd700 1 -- 172.21.15.133:0/4011568760 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ffa10010890 con 0x7ffa140730a0 2023-07-14T18:14:00.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.153+0000 7ffa0affd700 1 --2- 172.21.15.133:0/4011568760 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ffa000a12a0 0x7ffa000a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:00.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.154+0000 7ffa0affd700 1 -- 172.21.15.133:0/4011568760 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ffa100cc5c0 con 0x7ffa140730a0 2023-07-14T18:14:00.197 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.154+0000 7ffa1919a700 1 --2- 172.21.15.133:0/4011568760 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ffa000a12a0 0x7ffa000a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:00.197 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.154+0000 7ffa1919a700 1 --2- 172.21.15.133:0/4011568760 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ffa000a12a0 0x7ffa000a3750 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffa04005950 tx=0x7ffa04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:00.198 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.156+0000 7ffa0affd700 1 -- 172.21.15.133:0/4011568760 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7ffa10096110 con 0x7ffa140730a0 2023-07-14T18:14:00.294 INFO:journalctl@ceph.mds.h.smithi161.stdout:Jul 14 18:14:00 smithi161 podman[154014]: 2023-07-14T18:14:00.295 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:59 smithi161 ceph-mon[128993]: Deploying daemon mds.h on smithi161 2023-07-14T18:14:00.295 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:59 smithi161 ceph-mon[128993]: mds.? [v2:172.21.15.171:6836/106205332,v1:172.21.15.171:6837/106205332] up:boot 2023-07-14T18:14:00.295 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:59 smithi161 ceph-mon[128993]: fsmap 7 up:standby 2023-07-14T18:14:00.295 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:13:59 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T18:14:00.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:59 smithi171 ceph-mon[129241]: Deploying daemon mds.h on smithi161 2023-07-14T18:14:00.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:59 smithi171 ceph-mon[129241]: mds.? [v2:172.21.15.171:6836/106205332,v1:172.21.15.171:6837/106205332] up:boot 2023-07-14T18:14:00.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:59 smithi171 ceph-mon[129241]: fsmap 7 up:standby 2023-07-14T18:14:00.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:13:59 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T18:14:00.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:59 smithi133 ceph-mon[126384]: Deploying daemon mds.h on smithi161 2023-07-14T18:14:00.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:59 smithi133 ceph-mon[126384]: mds.? [v2:172.21.15.171:6836/106205332,v1:172.21.15.171:6837/106205332] up:boot 2023-07-14T18:14:00.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:59 smithi133 ceph-mon[126384]: fsmap 7 up:standby 2023-07-14T18:14:00.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:13:59 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T18:14:00.441 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.399+0000 7ffa1bbff700 1 -- 172.21.15.133:0/4011568760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa1402c8e0 con 0x7ffa140730a0 2023-07-14T18:14:00.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.399+0000 7ffa0affd700 1 -- 172.21.15.133:0/4011568760 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5817 (secure 0 0 0) 0x7ffa10016090 con 0x7ffa140730a0 2023-07-14T18:14:00.443 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:14:00.443 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3168151271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3168151271},{"type":"v1","addr":"172.21.15.133:6837","nonce":3168151271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/4023127259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4023127259},{"type":"v1","addr":"172.21.15.161:6833","nonce":4023127259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2722373740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2722373740},{"type":"v1","addr":"172.21.15.171:6833","nonce":2722373740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2399005074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2399005074},{"type":"v1","addr":"172.21.15.133:6835","nonce":2399005074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3725502564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3725502564},{"type":"v1","addr":"172.21.15.161:6835","nonce":3725502564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1877362116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1877362116},{"type":"v1","addr":"172.21.15.171:6835","nonce":1877362116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/106205332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":106205332},{"type":"v1","addr":"172.21.15.171:6837","nonce":106205332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-07-14T18:14:00.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.402+0000 7ffa1bbff700 1 -- 172.21.15.133:0/4011568760 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ffa000a12a0 msgr2=0x7ffa000a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:00.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.402+0000 7ffa1bbff700 1 --2- 172.21.15.133:0/4011568760 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ffa000a12a0 0x7ffa000a3750 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffa04005950 tx=0x7ffa04009260 comp rx=0 tx=0).stop 2023-07-14T18:14:00.446 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.402+0000 7ffa1bbff700 1 -- 172.21.15.133:0/4011568760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa140730a0 msgr2=0x7ffa1419c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:00.446 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.402+0000 7ffa1bbff700 1 --2- 172.21.15.133:0/4011568760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa140730a0 0x7ffa1419c050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ffa14074a50 tx=0x7ffa1000dc30 comp rx=0 tx=0).stop 2023-07-14T18:14:00.446 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.402+0000 7ffa1bbff700 1 -- 172.21.15.133:0/4011568760 shutdown_connections 2023-07-14T18:14:00.447 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.402+0000 7ffa1bbff700 1 --2- 172.21.15.133:0/4011568760 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ffa000a12a0 0x7ffa000a3750 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:00.447 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.402+0000 7ffa1bbff700 1 --2- 172.21.15.133:0/4011568760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa1410c200 0x7ffa141a3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:00.447 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.402+0000 7ffa1bbff700 1 --2- 172.21.15.133:0/4011568760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa140730a0 0x7ffa1419c050 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:00.447 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.402+0000 7ffa1bbff700 1 --2- 172.21.15.133:0/4011568760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa14074680 0x7ffa1419c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:00.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.402+0000 7ffa1bbff700 1 -- 172.21.15.133:0/4011568760 >> 172.21.15.133:0/4011568760 conn(0x7ffa140fb210 msgr2=0x7ffa140ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:00.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.402+0000 7ffa1bbff700 1 -- 172.21.15.133:0/4011568760 shutdown_connections 2023-07-14T18:14:00.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:00.402+0000 7ffa1bbff700 1 -- 172.21.15.133:0/4011568760 wait complete. 2023-07-14T18:14:00.449 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 8 2023-07-14T18:14:00.849 INFO:journalctl@ceph.mds.h.smithi161.stdout:Jul 14 18:14:00 smithi161 ceph-mds[154054]: -- 172.21.15.161:0/463244948 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5634a51891e0 con 0x5634a515f000 2023-07-14T18:14:00.849 INFO:journalctl@ceph.mds.h.smithi161.stdout:Jul 14 18:14:00 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-h[154030]: starting mds.h at 2023-07-14T18:14:00.849 INFO:journalctl@ceph.mds.h.smithi161.stdout:Jul 14 18:14:00 smithi161 bash[154014]: ff4ec1a7d739ac8806a8c33b6d2baec1de7388c2af13ae1eebcf5b272262232d 2023-07-14T18:14:00.849 INFO:journalctl@ceph.mds.h.smithi161.stdout:Jul 14 18:14:00 smithi161 systemd[1]: Started Ceph mds.h for 06d0f5fa-2271-11ee-9b33-001a4aab830c. 2023-07-14T18:14:01.226 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:00 smithi171 ceph-mon[129241]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:01.226 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:00 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/4011568760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:14:01.227 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:00 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:01.227 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:00 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:01.227 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:00 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:01.227 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:00 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:14:01.227 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:00 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:14:01.227 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:00 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:14:01.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:00 smithi161 ceph-mon[128993]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:01.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:00 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/4011568760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:14:01.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:00 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:01.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:00 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:01.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:00 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:01.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:00 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:14:01.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:00 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:14:01.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:00 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:14:01.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:00 smithi133 ceph-mon[126384]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:01.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:00 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/4011568760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:14:01.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:00 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:01.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:00 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:01.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:00 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:01.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:00 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:14:01.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:00 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:14:01.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:00 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:14:02.337 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:01 smithi171 ceph-mon[129241]: Deploying daemon mds.i on smithi171 2023-07-14T18:14:02.338 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:01 smithi171 ceph-mon[129241]: mds.? [v2:172.21.15.161:6836/3458064798,v1:172.21.15.161:6837/3458064798] up:boot 2023-07-14T18:14:02.338 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:01 smithi171 ceph-mon[129241]: fsmap 8 up:standby 2023-07-14T18:14:02.338 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:01 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T18:14:02.338 INFO:journalctl@ceph.mds.i.smithi171.stdout:Jul 14 18:14:01 smithi171 systemd[1]: Starting Ceph mds.i for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:14:02.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:01 smithi161 ceph-mon[128993]: Deploying daemon mds.i on smithi171 2023-07-14T18:14:02.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:01 smithi161 ceph-mon[128993]: mds.? [v2:172.21.15.161:6836/3458064798,v1:172.21.15.161:6837/3458064798] up:boot 2023-07-14T18:14:02.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:01 smithi161 ceph-mon[128993]: fsmap 8 up:standby 2023-07-14T18:14:02.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:01 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T18:14:02.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:01 smithi133 ceph-mon[126384]: Deploying daemon mds.i on smithi171 2023-07-14T18:14:02.452 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:01 smithi133 ceph-mon[126384]: mds.? [v2:172.21.15.161:6836/3458064798,v1:172.21.15.161:6837/3458064798] up:boot 2023-07-14T18:14:02.452 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:01 smithi133 ceph-mon[126384]: fsmap 8 up:standby 2023-07-14T18:14:02.452 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:01 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T18:14:02.465 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T18:14:02.609 INFO:journalctl@ceph.mds.i.smithi171.stdout:Jul 14 18:14:02 smithi171 podman[155601]: 2023-07-14T18:14:02.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.631+0000 7fba29dff700 1 -- 172.21.15.133:0/3907635224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba24100e20 msgr2=0x7fba24101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:02.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.631+0000 7fba29dff700 1 --2- 172.21.15.133:0/3907635224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba24100e20 0x7fba24101270 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fba18009990 tx=0x7fba18009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:02.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.632+0000 7fba29dff700 1 -- 172.21.15.133:0/3907635224 shutdown_connections 2023-07-14T18:14:02.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.632+0000 7fba29dff700 1 --2- 172.21.15.133:0/3907635224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba241017b0 0x7fba24108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:02.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.632+0000 7fba29dff700 1 --2- 172.21.15.133:0/3907635224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba240ffc30 0x7fba24100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:02.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.632+0000 7fba29dff700 1 --2- 172.21.15.133:0/3907635224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba24100e20 0x7fba24101270 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:02.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.632+0000 7fba29dff700 1 -- 172.21.15.133:0/3907635224 >> 172.21.15.133:0/3907635224 conn(0x7fba240fb210 msgr2=0x7fba240fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:02.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.632+0000 7fba29dff700 1 -- 172.21.15.133:0/3907635224 shutdown_connections 2023-07-14T18:14:02.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.632+0000 7fba29dff700 1 -- 172.21.15.133:0/3907635224 wait complete. 2023-07-14T18:14:02.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.633+0000 7fba29dff700 1 Processor -- start 2023-07-14T18:14:02.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.633+0000 7fba29dff700 1 -- start start 2023-07-14T18:14:02.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.633+0000 7fba29dff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba240ffc30 0x7fba24197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:02.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.633+0000 7fba29dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba24100e20 0x7fba24197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:02.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.633+0000 7fba29dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba241017b0 0x7fba2419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:02.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.633+0000 7fba29dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2419f5f0 con 0x7fba24100e20 2023-07-14T18:14:02.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.633+0000 7fba29dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2419f730 con 0x7fba241017b0 2023-07-14T18:14:02.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.633+0000 7fba29dff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2419f8c0 con 0x7fba240ffc30 2023-07-14T18:14:02.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.634+0000 7fba22ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba24100e20 0x7fba24197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:02.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.634+0000 7fba23fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba241017b0 0x7fba2419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:02.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.634+0000 7fba22ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba24100e20 0x7fba24197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44936/0 (socket says 172.21.15.133:44936) 2023-07-14T18:14:02.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.634+0000 7fba22ffd700 1 -- 172.21.15.133:0/395920544 learned_addr learned my addr 172.21.15.133:0/395920544 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:02.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.634+0000 7fba237fe700 1 --2- 172.21.15.133:0/395920544 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba240ffc30 0x7fba24197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:02.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.634+0000 7fba22ffd700 1 -- 172.21.15.133:0/395920544 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba240ffc30 msgr2=0x7fba24197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:02.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.634+0000 7fba22ffd700 1 --2- 172.21.15.133:0/395920544 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba240ffc30 0x7fba24197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:02.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.634+0000 7fba22ffd700 1 -- 172.21.15.133:0/395920544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba241017b0 msgr2=0x7fba2419f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:02.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.634+0000 7fba22ffd700 1 --2- 172.21.15.133:0/395920544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba241017b0 0x7fba2419f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:02.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.634+0000 7fba22ffd700 1 -- 172.21.15.133:0/395920544 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba14009520 con 0x7fba24100e20 2023-07-14T18:14:02.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.634+0000 7fba237fe700 1 --2- 172.21.15.133:0/395920544 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba240ffc30 0x7fba24197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:14:02.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.634+0000 7fba23fff700 1 --2- 172.21.15.133:0/395920544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba241017b0 0x7fba2419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:14:02.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.634+0000 7fba22ffd700 1 --2- 172.21.15.133:0/395920544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba24100e20 0x7fba24197fb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fba1800f6c0 tx=0x7fba1800f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:02.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.634+0000 7fba20ff9700 1 -- 172.21.15.133:0/395920544 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1801d070 con 0x7fba24100e20 2023-07-14T18:14:02.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.635+0000 7fba20ff9700 1 -- 172.21.15.133:0/395920544 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba1800fe40 con 0x7fba24100e20 2023-07-14T18:14:02.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.635+0000 7fba29dff700 1 -- 172.21.15.133:0/395920544 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba180095f0 con 0x7fba24100e20 2023-07-14T18:14:02.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.635+0000 7fba20ff9700 1 -- 172.21.15.133:0/395920544 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba180173f0 con 0x7fba24100e20 2023-07-14T18:14:02.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.635+0000 7fba29dff700 1 -- 172.21.15.133:0/395920544 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2419fec0 con 0x7fba24100e20 2023-07-14T18:14:02.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.636+0000 7fba29dff700 1 -- 172.21.15.133:0/395920544 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba2410b3a0 con 0x7fba24100e20 2023-07-14T18:14:02.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.637+0000 7fba20ff9700 1 -- 172.21.15.133:0/395920544 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fba18021410 con 0x7fba24100e20 2023-07-14T18:14:02.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.639+0000 7fba20ff9700 1 --2- 172.21.15.133:0/395920544 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fba0c0a11a0 0x7fba0c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:02.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.639+0000 7fba20ff9700 1 -- 172.21.15.133:0/395920544 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fba180cd6d0 con 0x7fba24100e20 2023-07-14T18:14:02.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.639+0000 7fba237fe700 1 --2- 172.21.15.133:0/395920544 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fba0c0a11a0 0x7fba0c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:02.683 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.640+0000 7fba237fe700 1 --2- 172.21.15.133:0/395920544 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fba0c0a11a0 0x7fba0c0a3650 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fba14009fd0 tx=0x7fba14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:02.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.643+0000 7fba20ff9700 1 -- 172.21.15.133:0/395920544 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fba180973e0 con 0x7fba24100e20 2023-07-14T18:14:02.864 INFO:journalctl@ceph.mds.i.smithi171.stdout:Jul 14 18:14:02 smithi171 bash[155601]: 3bb09798cd6d0ca190bf5e0d10015fa6bac9b2efe2443cc663907c7c0f7414bd 2023-07-14T18:14:02.864 INFO:journalctl@ceph.mds.i.smithi171.stdout:Jul 14 18:14:02 smithi171 ceph-mds[155641]: -- 172.21.15.171:0/122194530 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56349539d1e0 con 0x563495373000 2023-07-14T18:14:02.864 INFO:journalctl@ceph.mds.i.smithi171.stdout:Jul 14 18:14:02 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-i[155617]: starting mds.i at 2023-07-14T18:14:02.865 INFO:journalctl@ceph.mds.i.smithi171.stdout:Jul 14 18:14:02 smithi171 systemd[1]: Started Ceph mds.i for 06d0f5fa-2271-11ee-9b33-001a4aab830c. 2023-07-14T18:14:02.938 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.895+0000 7fba29dff700 1 -- 172.21.15.133:0/395920544 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba2410b590 con 0x7fba24100e20 2023-07-14T18:14:02.938 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.896+0000 7fba20ff9700 1 -- 172.21.15.133:0/395920544 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6577 (secure 0 0 0) 0x7fba1809aa00 con 0x7fba24100e20 2023-07-14T18:14:02.939 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:14:02.939 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3168151271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3168151271},{"type":"v1","addr":"172.21.15.133:6837","nonce":3168151271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/4023127259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4023127259},{"type":"v1","addr":"172.21.15.161:6833","nonce":4023127259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2722373740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2722373740},{"type":"v1","addr":"172.21.15.171:6833","nonce":2722373740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2399005074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2399005074},{"type":"v1","addr":"172.21.15.133:6835","nonce":2399005074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3725502564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3725502564},{"type":"v1","addr":"172.21.15.161:6835","nonce":3725502564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1877362116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1877362116},{"type":"v1","addr":"172.21.15.171:6835","nonce":1877362116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3458064798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3458064798},{"type":"v1","addr":"172.21.15.161:6837","nonce":3458064798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/106205332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":106205332},{"type":"v1","addr":"172.21.15.171:6837","nonce":106205332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-07-14T18:14:02.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.898+0000 7fba29dff700 1 -- 172.21.15.133:0/395920544 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fba0c0a11a0 msgr2=0x7fba0c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:02.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.898+0000 7fba29dff700 1 --2- 172.21.15.133:0/395920544 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fba0c0a11a0 0x7fba0c0a3650 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fba14009fd0 tx=0x7fba14009260 comp rx=0 tx=0).stop 2023-07-14T18:14:02.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.898+0000 7fba29dff700 1 -- 172.21.15.133:0/395920544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba24100e20 msgr2=0x7fba24197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:02.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.898+0000 7fba29dff700 1 --2- 172.21.15.133:0/395920544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba24100e20 0x7fba24197fb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fba1800f6c0 tx=0x7fba1800f6f0 comp rx=0 tx=0).stop 2023-07-14T18:14:02.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.898+0000 7fba29dff700 1 -- 172.21.15.133:0/395920544 shutdown_connections 2023-07-14T18:14:02.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.898+0000 7fba29dff700 1 --2- 172.21.15.133:0/395920544 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fba0c0a11a0 0x7fba0c0a3650 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:02.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.898+0000 7fba29dff700 1 --2- 172.21.15.133:0/395920544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba241017b0 0x7fba2419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:02.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.898+0000 7fba29dff700 1 --2- 172.21.15.133:0/395920544 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba240ffc30 0x7fba24197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:02.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.898+0000 7fba29dff700 1 --2- 172.21.15.133:0/395920544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba24100e20 0x7fba24197fb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:02.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.899+0000 7fba29dff700 1 -- 172.21.15.133:0/395920544 >> 172.21.15.133:0/395920544 conn(0x7fba240fb210 msgr2=0x7fba24106650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:02.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.899+0000 7fba29dff700 1 -- 172.21.15.133:0/395920544 shutdown_connections 2023-07-14T18:14:02.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:02.899+0000 7fba29dff700 1 -- 172.21.15.133:0/395920544 wait complete. 2023-07-14T18:14:02.943 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 9 2023-07-14T18:14:03.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:02 smithi161 ceph-mon[128993]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:03.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:02 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:03.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:02 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:03.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:02 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:03.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:02 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:14:03.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:02 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:14:03.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:02 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:14:03.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:02 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/395920544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:14:03.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:02 smithi171 ceph-mon[129241]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:03.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:02 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:03.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:02 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:03.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:02 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:03.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:02 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:14:03.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:02 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:14:03.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:02 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:14:03.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:02 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/395920544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:14:03.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:02 smithi133 ceph-mon[126384]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:03.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:02 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:03.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:02 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:03.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:02 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:03.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:02 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:14:03.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:02 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:14:03.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:02 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:14:03.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:02 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/395920544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:14:04.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:03 smithi133 ceph-mon[126384]: Deploying daemon mds.g on smithi133 2023-07-14T18:14:04.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:03 smithi133 ceph-mon[126384]: mds.? [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] up:boot 2023-07-14T18:14:04.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:03 smithi133 ceph-mon[126384]: fsmap 9 up:standby 2023-07-14T18:14:04.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:03 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T18:14:04.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:03 smithi161 ceph-mon[128993]: Deploying daemon mds.g on smithi133 2023-07-14T18:14:04.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:03 smithi161 ceph-mon[128993]: mds.? [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] up:boot 2023-07-14T18:14:04.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:03 smithi161 ceph-mon[128993]: fsmap 9 up:standby 2023-07-14T18:14:04.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:03 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T18:14:04.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:03 smithi171 ceph-mon[129241]: Deploying daemon mds.g on smithi133 2023-07-14T18:14:04.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:03 smithi171 ceph-mon[129241]: mds.? [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] up:boot 2023-07-14T18:14:04.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:03 smithi171 ceph-mon[129241]: fsmap 9 up:standby 2023-07-14T18:14:04.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:03 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T18:14:04.901 INFO:journalctl@ceph.mds.g.smithi133.stdout:Jul 14 18:14:04 smithi133 systemd[1]: Starting Ceph mds.g for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:14:04.962 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T18:14:05.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.165+0000 7f29e9bff700 1 -- 172.21.15.133:0/1360855369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29e4074680 msgr2=0x7f29e4072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:05.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.165+0000 7f29e9bff700 1 --2- 172.21.15.133:0/1360855369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29e4074680 0x7f29e4072ad0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f29d4009990 tx=0x7f29d4009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:05.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.165+0000 7f29e9bff700 1 -- 172.21.15.133:0/1360855369 shutdown_connections 2023-07-14T18:14:05.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.165+0000 7f29e9bff700 1 --2- 172.21.15.133:0/1360855369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29e410c200 0x7f29e410e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:05.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.165+0000 7f29e9bff700 1 --2- 172.21.15.133:0/1360855369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29e4074680 0x7f29e4072ad0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:05.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.165+0000 7f29e9bff700 1 --2- 172.21.15.133:0/1360855369 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29e40730a0 0x7f29e4073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:05.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.165+0000 7f29e9bff700 1 -- 172.21.15.133:0/1360855369 >> 172.21.15.133:0/1360855369 conn(0x7f29e40fb210 msgr2=0x7f29e40fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:05.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.165+0000 7f29e9bff700 1 -- 172.21.15.133:0/1360855369 shutdown_connections 2023-07-14T18:14:05.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.166+0000 7f29e9bff700 1 -- 172.21.15.133:0/1360855369 wait complete. 2023-07-14T18:14:05.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.167+0000 7f29e9bff700 1 Processor -- start 2023-07-14T18:14:05.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.167+0000 7f29e9bff700 1 -- start start 2023-07-14T18:14:05.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.167+0000 7f29e9bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29e40730a0 0x7f29e419bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:05.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.167+0000 7f29e9bff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29e4074680 0x7f29e419c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:05.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.168+0000 7f29e9bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29e410c200 0x7f29e41a3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:05.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.168+0000 7f29e9bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e41a3af0 con 0x7f29e40730a0 2023-07-14T18:14:05.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.168+0000 7f29e9bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e41a3c30 con 0x7f29e410c200 2023-07-14T18:14:05.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.168+0000 7f29e9bff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e41a3dc0 con 0x7f29e4074680 2023-07-14T18:14:05.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.168+0000 7f29e37fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29e40730a0 0x7f29e419bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:05.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.168+0000 7f29e37fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29e40730a0 0x7f29e419bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44956/0 (socket says 172.21.15.133:44956) 2023-07-14T18:14:05.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.168+0000 7f29e37fe700 1 -- 172.21.15.133:0/2764390997 learned_addr learned my addr 172.21.15.133:0/2764390997 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:05.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.168+0000 7f29e2ffd700 1 --2- 172.21.15.133:0/2764390997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29e4074680 0x7f29e419c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:05.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.168+0000 7f29e3fff700 1 --2- 172.21.15.133:0/2764390997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29e410c200 0x7f29e41a3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:05.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.168+0000 7f29e37fe700 1 -- 172.21.15.133:0/2764390997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29e4074680 msgr2=0x7f29e419c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:05.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.168+0000 7f29e37fe700 1 --2- 172.21.15.133:0/2764390997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29e4074680 0x7f29e419c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:05.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.168+0000 7f29e37fe700 1 -- 172.21.15.133:0/2764390997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29e410c200 msgr2=0x7f29e41a3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:05.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.168+0000 7f29e37fe700 1 --2- 172.21.15.133:0/2764390997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29e410c200 0x7f29e41a3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:05.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.168+0000 7f29e37fe700 1 -- 172.21.15.133:0/2764390997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d40095f0 con 0x7f29e40730a0 2023-07-14T18:14:05.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.168+0000 7f29e37fe700 1 --2- 172.21.15.133:0/2764390997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29e40730a0 0x7f29e419bf70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f29d4005f50 tx=0x7f29d40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:05.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.169+0000 7f29e0ff9700 1 -- 172.21.15.133:0/2764390997 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d401d070 con 0x7f29e40730a0 2023-07-14T18:14:05.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.169+0000 7f29e0ff9700 1 -- 172.21.15.133:0/2764390997 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29d400fbb0 con 0x7f29e40730a0 2023-07-14T18:14:05.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.169+0000 7f29e0ff9700 1 -- 172.21.15.133:0/2764390997 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4017650 con 0x7f29e40730a0 2023-07-14T18:14:05.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.169+0000 7f29e9bff700 1 -- 172.21.15.133:0/2764390997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e41a4040 con 0x7f29e40730a0 2023-07-14T18:14:05.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.169+0000 7f29e9bff700 1 -- 172.21.15.133:0/2764390997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e41a4410 con 0x7f29e40730a0 2023-07-14T18:14:05.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.170+0000 7f29e9bff700 1 -- 172.21.15.133:0/2764390997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e4101d50 con 0x7f29e40730a0 2023-07-14T18:14:05.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.173+0000 7f29e0ff9700 1 -- 172.21.15.133:0/2764390997 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f29d400fd20 con 0x7f29e40730a0 2023-07-14T18:14:05.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.175+0000 7f29e0ff9700 1 --2- 172.21.15.133:0/2764390997 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f29cc0a1220 0x7f29cc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:05.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.175+0000 7f29e0ff9700 1 -- 172.21.15.133:0/2764390997 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f29d40cdab0 con 0x7f29e40730a0 2023-07-14T18:14:05.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.175+0000 7f29e2ffd700 1 --2- 172.21.15.133:0/2764390997 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f29cc0a1220 0x7f29cc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:05.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.176+0000 7f29e2ffd700 1 --2- 172.21.15.133:0/2764390997 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f29cc0a1220 0x7f29cc0a36d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f29e40743a0 tx=0x7f29d800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:05.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.179+0000 7f29e0ff9700 1 -- 172.21.15.133:0/2764390997 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f29d4097690 con 0x7f29e40730a0 2023-07-14T18:14:05.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:04 smithi161 ceph-mon[128993]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:05.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:04 smithi171 ceph-mon[129241]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:05.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:04 smithi133 ceph-mon[126384]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:05.490 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.448+0000 7f29e9bff700 1 -- 172.21.15.133:0/2764390997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29e402c8e0 con 0x7f29e40730a0 2023-07-14T18:14:05.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.448+0000 7f29e0ff9700 1 -- 172.21.15.133:0/2764390997 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7336 (secure 0 0 0) 0x7f29d409acb0 con 0x7f29e40730a0 2023-07-14T18:14:05.491 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:14:05.492 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3168151271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3168151271},{"type":"v1","addr":"172.21.15.133:6837","nonce":3168151271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/4023127259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4023127259},{"type":"v1","addr":"172.21.15.161:6833","nonce":4023127259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2722373740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2722373740},{"type":"v1","addr":"172.21.15.171:6833","nonce":2722373740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2399005074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2399005074},{"type":"v1","addr":"172.21.15.133:6835","nonce":2399005074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3725502564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3725502564},{"type":"v1","addr":"172.21.15.161:6835","nonce":3725502564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1877362116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1877362116},{"type":"v1","addr":"172.21.15.171:6835","nonce":1877362116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3458064798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3458064798},{"type":"v1","addr":"172.21.15.161:6837","nonce":3458064798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/106205332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":106205332},{"type":"v1","addr":"172.21.15.171:6837","nonce":106205332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24541,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/816201198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":816201198},{"type":"v1","addr":"172.21.15.171:6839","nonce":816201198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-07-14T18:14:05.493 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.451+0000 7f29e9bff700 1 -- 172.21.15.133:0/2764390997 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f29cc0a1220 msgr2=0x7f29cc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:05.494 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.451+0000 7f29e9bff700 1 --2- 172.21.15.133:0/2764390997 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f29cc0a1220 0x7f29cc0a36d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f29e40743a0 tx=0x7f29d800b410 comp rx=0 tx=0).stop 2023-07-14T18:14:05.494 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.451+0000 7f29e9bff700 1 -- 172.21.15.133:0/2764390997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29e40730a0 msgr2=0x7f29e419bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:05.494 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.451+0000 7f29e9bff700 1 --2- 172.21.15.133:0/2764390997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29e40730a0 0x7f29e419bf70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f29d4005f50 tx=0x7f29d40057b0 comp rx=0 tx=0).stop 2023-07-14T18:14:05.494 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.451+0000 7f29e9bff700 1 -- 172.21.15.133:0/2764390997 shutdown_connections 2023-07-14T18:14:05.494 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.451+0000 7f29e9bff700 1 --2- 172.21.15.133:0/2764390997 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f29cc0a1220 0x7f29cc0a36d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:05.494 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.451+0000 7f29e9bff700 1 --2- 172.21.15.133:0/2764390997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29e410c200 0x7f29e41a3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:05.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.451+0000 7f29e9bff700 1 --2- 172.21.15.133:0/2764390997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29e40730a0 0x7f29e419bf70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:05.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.451+0000 7f29e9bff700 1 --2- 172.21.15.133:0/2764390997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29e4074680 0x7f29e419c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:05.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.451+0000 7f29e9bff700 1 -- 172.21.15.133:0/2764390997 >> 172.21.15.133:0/2764390997 conn(0x7f29e40fb210 msgr2=0x7f29e40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:05.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.451+0000 7f29e9bff700 1 -- 172.21.15.133:0/2764390997 shutdown_connections 2023-07-14T18:14:05.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:05.451+0000 7f29e9bff700 1 -- 172.21.15.133:0/2764390997 wait complete. 2023-07-14T18:14:05.495 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 10 2023-07-14T18:14:05.690 INFO:journalctl@ceph.mds.g.smithi133.stdout:Jul 14 18:14:05 smithi133 podman[160728]: 2023-07-14T18:14:05.690 INFO:journalctl@ceph.mds.g.smithi133.stdout:Jul 14 18:14:05 smithi133 ceph-mds[160770]: -- 172.21.15.133:0/1289402727 <== mon.0 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56253804d1e0 con 0x562538022c00 2023-07-14T18:14:05.690 INFO:journalctl@ceph.mds.g.smithi133.stdout:Jul 14 18:14:05 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-g[160746]: starting mds.g at 2023-07-14T18:14:06.001 INFO:journalctl@ceph.mds.g.smithi133.stdout:Jul 14 18:14:05 smithi133 bash[160728]: c3f959794e1e73754d728689289e6a38da952a2771cbcc906c58494cdcae4f03 2023-07-14T18:14:06.001 INFO:journalctl@ceph.mds.g.smithi133.stdout:Jul 14 18:14:05 smithi133 systemd[1]: Started Ceph mds.g for 06d0f5fa-2271-11ee-9b33-001a4aab830c. 2023-07-14T18:14:06.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:05 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2764390997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:14:06.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:05 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:06.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:05 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:06.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:05 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:06.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:05 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:14:06.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:05 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:14:06.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:05 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:14:06.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:05 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2764390997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:14:06.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:05 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:06.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:05 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:06.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:05 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:06.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:05 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:14:06.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:05 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:14:06.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:05 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:14:06.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:05 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2764390997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:14:06.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:05 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:06.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:05 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:06.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:05 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:06.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:05 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:14:06.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:05 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:14:06.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:05 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:14:07.289 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:06 smithi133 ceph-mon[126384]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:07.290 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:06 smithi133 ceph-mon[126384]: Deploying daemon mds.a on smithi133 2023-07-14T18:14:07.290 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:06 smithi133 ceph-mon[126384]: mds.? [v2:172.21.15.133:6838/3093144261,v1:172.21.15.133:6839/3093144261] up:boot 2023-07-14T18:14:07.290 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:06 smithi133 ceph-mon[126384]: fsmap 10 up:standby 2023-07-14T18:14:07.290 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:06 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-14T18:14:07.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:06 smithi161 ceph-mon[128993]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:07.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:06 smithi161 ceph-mon[128993]: Deploying daemon mds.a on smithi133 2023-07-14T18:14:07.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:06 smithi161 ceph-mon[128993]: mds.? [v2:172.21.15.133:6838/3093144261,v1:172.21.15.133:6839/3093144261] up:boot 2023-07-14T18:14:07.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:06 smithi161 ceph-mon[128993]: fsmap 10 up:standby 2023-07-14T18:14:07.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:06 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-14T18:14:07.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:06 smithi171 ceph-mon[129241]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:07.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:06 smithi171 ceph-mon[129241]: Deploying daemon mds.a on smithi133 2023-07-14T18:14:07.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:06 smithi171 ceph-mon[129241]: mds.? [v2:172.21.15.133:6838/3093144261,v1:172.21.15.133:6839/3093144261] up:boot 2023-07-14T18:14:07.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:06 smithi171 ceph-mon[129241]: fsmap 10 up:standby 2023-07-14T18:14:07.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:06 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-14T18:14:07.515 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T18:14:07.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.829+0000 7f012a5ff700 1 -- 172.21.15.133:0/890571123 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0124101e50 msgr2=0x7f0124102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:07.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.829+0000 7f012a5ff700 1 --2- 172.21.15.133:0/890571123 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0124101e50 0x7f0124102260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0114009990 tx=0x7f0114009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:07.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.829+0000 7f012a5ff700 1 -- 172.21.15.133:0/890571123 shutdown_connections 2023-07-14T18:14:07.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.829+0000 7f012a5ff700 1 --2- 172.21.15.133:0/890571123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0124103950 0x7f012410a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:07.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.829+0000 7f012a5ff700 1 --2- 172.21.15.133:0/890571123 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0124101e50 0x7f0124102260 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:07.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.829+0000 7f012a5ff700 1 --2- 172.21.15.133:0/890571123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0124102fa0 0x7f0124103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:07.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.829+0000 7f012a5ff700 1 -- 172.21.15.133:0/890571123 >> 172.21.15.133:0/890571123 conn(0x7f01240fd490 msgr2=0x7f01240ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:07.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.831+0000 7f012a5ff700 1 -- 172.21.15.133:0/890571123 shutdown_connections 2023-07-14T18:14:07.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.831+0000 7f012a5ff700 1 -- 172.21.15.133:0/890571123 wait complete. 2023-07-14T18:14:07.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.833+0000 7f012a5ff700 1 Processor -- start 2023-07-14T18:14:07.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.833+0000 7f012a5ff700 1 -- start start 2023-07-14T18:14:07.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.833+0000 7f012a5ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0124101e50 0x7f012410e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:07.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.833+0000 7f012a5ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0124102fa0 0x7f012410eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:07.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.834+0000 7f012a5ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0124103950 0x7f0124113f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:07.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.834+0000 7f012a5ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0124114520 con 0x7f0124103950 2023-07-14T18:14:07.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.834+0000 7f012a5ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0124114660 con 0x7f0124102fa0 2023-07-14T18:14:07.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.834+0000 7f012a5ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01241147f0 con 0x7f0124101e50 2023-07-14T18:14:07.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.834+0000 7f0128b9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0124103950 0x7f0124113f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:07.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.834+0000 7f0128b9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0124103950 0x7f0124113f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41902/0 (socket says 172.21.15.133:41902) 2023-07-14T18:14:07.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.834+0000 7f0128b9c700 1 -- 172.21.15.133:0/4290517150 learned_addr learned my addr 172.21.15.133:0/4290517150 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:07.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.835+0000 7f0123fff700 1 --2- 172.21.15.133:0/4290517150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0124101e50 0x7f012410e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:07.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.835+0000 7f0128b9c700 1 -- 172.21.15.133:0/4290517150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0124101e50 msgr2=0x7f012410e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:07.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.835+0000 7f0128b9c700 1 --2- 172.21.15.133:0/4290517150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0124101e50 0x7f012410e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:07.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.835+0000 7f0128b9c700 1 -- 172.21.15.133:0/4290517150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0124102fa0 msgr2=0x7f012410eef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T18:14:07.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.835+0000 7f0128b9c700 1 --2- 172.21.15.133:0/4290517150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0124102fa0 0x7f012410eef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:07.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.835+0000 7f0128b9c700 1 -- 172.21.15.133:0/4290517150 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01140095f0 con 0x7f0124103950 2023-07-14T18:14:07.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.835+0000 7f0128b9c700 1 --2- 172.21.15.133:0/4290517150 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0124103950 0x7f0124113f50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f011c00ef30 tx=0x7f011c00c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:07.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.836+0000 7f0121ffb700 1 -- 172.21.15.133:0/4290517150 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c018070 con 0x7f0124103950 2023-07-14T18:14:07.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.836+0000 7f0121ffb700 1 -- 172.21.15.133:0/4290517150 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f011c00f040 con 0x7f0124103950 2023-07-14T18:14:07.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.836+0000 7f0121ffb700 1 -- 172.21.15.133:0/4290517150 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c003de0 con 0x7f0124103950 2023-07-14T18:14:07.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.836+0000 7f012a5ff700 1 -- 172.21.15.133:0/4290517150 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0124114a70 con 0x7f0124103950 2023-07-14T18:14:07.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.836+0000 7f012a5ff700 1 -- 172.21.15.133:0/4290517150 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0124114f30 con 0x7f0124103950 2023-07-14T18:14:07.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.838+0000 7f012a5ff700 1 -- 172.21.15.133:0/4290517150 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0124067430 con 0x7f0124103950 2023-07-14T18:14:07.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.840+0000 7f0121ffb700 1 -- 172.21.15.133:0/4290517150 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f011c00caa0 con 0x7f0124103950 2023-07-14T18:14:07.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.841+0000 7f0121ffb700 1 --2- 172.21.15.133:0/4290517150 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f01100a1250 0x7f01100a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:07.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.841+0000 7f0121ffb700 1 -- 172.21.15.133:0/4290517150 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f011c0cc530 con 0x7f0124103950 2023-07-14T18:14:07.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.842+0000 7f0123fff700 1 --2- 172.21.15.133:0/4290517150 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f01100a1250 0x7f01100a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:07.885 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.842+0000 7f0123fff700 1 --2- 172.21.15.133:0/4290517150 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f01100a1250 0x7f01100a3700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0114006010 tx=0x7f0114005c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:07.886 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:07.844+0000 7f0121ffb700 1 -- 172.21.15.133:0/4290517150 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f011c0961b0 con 0x7f0124103950 2023-07-14T18:14:08.155 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:08.112+0000 7f012a5ff700 1 -- 172.21.15.133:0/4290517150 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f012406a060 con 0x7f0124103950 2023-07-14T18:14:08.156 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:08.113+0000 7f0121ffb700 1 -- 172.21.15.133:0/4290517150 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8097 (secure 0 0 0) 0x7f011c0997d0 con 0x7f0124103950 2023-07-14T18:14:08.156 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:14:08.156 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3168151271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3168151271},{"type":"v1","addr":"172.21.15.133:6837","nonce":3168151271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14544,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/3093144261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3093144261},{"type":"v1","addr":"172.21.15.133:6839","nonce":3093144261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/4023127259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4023127259},{"type":"v1","addr":"172.21.15.161:6833","nonce":4023127259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2722373740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2722373740},{"type":"v1","addr":"172.21.15.171:6833","nonce":2722373740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2399005074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2399005074},{"type":"v1","addr":"172.21.15.133:6835","nonce":2399005074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3725502564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3725502564},{"type":"v1","addr":"172.21.15.161:6835","nonce":3725502564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1877362116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1877362116},{"type":"v1","addr":"172.21.15.171:6835","nonce":1877362116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3458064798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3458064798},{"type":"v1","addr":"172.21.15.161:6837","nonce":3458064798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/106205332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":106205332},{"type":"v1","addr":"172.21.15.171:6837","nonce":106205332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24541,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/816201198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":816201198},{"type":"v1","addr":"172.21.15.171:6839","nonce":816201198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-07-14T18:14:08.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:08.116+0000 7f012a5ff700 1 -- 172.21.15.133:0/4290517150 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f01100a1250 msgr2=0x7f01100a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:08.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:08.116+0000 7f012a5ff700 1 --2- 172.21.15.133:0/4290517150 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f01100a1250 0x7f01100a3700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0114006010 tx=0x7f0114005c00 comp rx=0 tx=0).stop 2023-07-14T18:14:08.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:08.116+0000 7f012a5ff700 1 -- 172.21.15.133:0/4290517150 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0124103950 msgr2=0x7f0124113f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:08.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:08.116+0000 7f012a5ff700 1 --2- 172.21.15.133:0/4290517150 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0124103950 0x7f0124113f50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f011c00ef30 tx=0x7f011c00c450 comp rx=0 tx=0).stop 2023-07-14T18:14:08.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:08.116+0000 7f012a5ff700 1 -- 172.21.15.133:0/4290517150 shutdown_connections 2023-07-14T18:14:08.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:08.116+0000 7f012a5ff700 1 --2- 172.21.15.133:0/4290517150 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f01100a1250 0x7f01100a3700 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:08.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:08.116+0000 7f012a5ff700 1 --2- 172.21.15.133:0/4290517150 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0124103950 0x7f0124113f50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:08.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:08.116+0000 7f012a5ff700 1 --2- 172.21.15.133:0/4290517150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0124101e50 0x7f012410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:08.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:08.116+0000 7f012a5ff700 1 --2- 172.21.15.133:0/4290517150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0124102fa0 0x7f012410eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:08.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:08.116+0000 7f012a5ff700 1 -- 172.21.15.133:0/4290517150 >> 172.21.15.133:0/4290517150 conn(0x7f01240fd490 msgr2=0x7f01240ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:08.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:08.116+0000 7f012a5ff700 1 -- 172.21.15.133:0/4290517150 shutdown_connections 2023-07-14T18:14:08.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:08.116+0000 7f012a5ff700 1 -- 172.21.15.133:0/4290517150 wait complete. 2023-07-14T18:14:08.160 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 11 2023-07-14T18:14:08.404 INFO:journalctl@ceph.mds.a.smithi133.stdout:Jul 14 18:14:08 smithi133 systemd[1]: Starting Ceph mds.a for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:14:08.840 INFO:journalctl@ceph.mds.a.smithi133.stdout:Jul 14 18:14:08 smithi133 podman[161115]: 2023-07-14T18:14:08.840 INFO:journalctl@ceph.mds.a.smithi133.stdout:Jul 14 18:14:08 smithi133 bash[161115]: a0aa1db88f11924175dd3e5bae10e584a59ab0d7398babd896578b579871fd3a 2023-07-14T18:14:08.840 INFO:journalctl@ceph.mds.a.smithi133.stdout:Jul 14 18:14:08 smithi133 ceph-mds[161155]: -- 172.21.15.133:0/3881568457 <== mon.0 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cce2dd91e0 con 0x55cce2daec00 2023-07-14T18:14:08.840 INFO:journalctl@ceph.mds.a.smithi133.stdout:Jul 14 18:14:08 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-a[161131]: starting mds.a at 2023-07-14T18:14:09.155 INFO:journalctl@ceph.mds.a.smithi133.stdout:Jul 14 18:14:08 smithi133 systemd[1]: Started Ceph mds.a for 06d0f5fa-2271-11ee-9b33-001a4aab830c. 2023-07-14T18:14:09.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:08 smithi133 ceph-mon[126384]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:09.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:08 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/4290517150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:14:09.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:08 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:09.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:08 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:09.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:08 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:09.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:08 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:14:09.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:08 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:14:09.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:08 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:14:09.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:08 smithi161 ceph-mon[128993]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:09.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:08 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/4290517150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:14:09.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:08 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:09.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:08 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:09.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:08 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:09.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:08 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:14:09.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:08 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:14:09.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:08 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:14:09.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:08 smithi171 ceph-mon[129241]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:09.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:08 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/4290517150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:14:09.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:08 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:09.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:08 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:09.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:08 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:09.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:08 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T18:14:09.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:08 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T18:14:09.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:08 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:14:10.127 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:09 smithi161 ceph-mon[128993]: Deploying daemon mds.b on smithi161 2023-07-14T18:14:10.128 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:09 smithi161 ceph-mon[128993]: mds.? [v2:172.21.15.133:6840/3282902068,v1:172.21.15.133:6841/3282902068] up:boot 2023-07-14T18:14:10.128 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:09 smithi161 ceph-mon[128993]: fsmap 11 up:standby 2023-07-14T18:14:10.128 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:09 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-14T18:14:10.182 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T18:14:10.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:09 smithi171 ceph-mon[129241]: Deploying daemon mds.b on smithi161 2023-07-14T18:14:10.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:09 smithi171 ceph-mon[129241]: mds.? [v2:172.21.15.133:6840/3282902068,v1:172.21.15.133:6841/3282902068] up:boot 2023-07-14T18:14:10.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:09 smithi171 ceph-mon[129241]: fsmap 11 up:standby 2023-07-14T18:14:10.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:09 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-14T18:14:10.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.350+0000 7f4294bff700 1 -- 172.21.15.133:0/2614489253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42901017b0 msgr2=0x7f4290108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:10.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.350+0000 7f4294bff700 1 --2- 172.21.15.133:0/2614489253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42901017b0 0x7f4290108070 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f428800b0a0 tx=0x7f428800b3b0 comp rx=0 tx=0).stop 2023-07-14T18:14:10.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.350+0000 7f4294bff700 1 -- 172.21.15.133:0/2614489253 shutdown_connections 2023-07-14T18:14:10.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.350+0000 7f4294bff700 1 --2- 172.21.15.133:0/2614489253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42901017b0 0x7f4290108070 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:10.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.350+0000 7f4294bff700 1 --2- 172.21.15.133:0/2614489253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42900ffc30 0x7f4290100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:10.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.350+0000 7f4294bff700 1 --2- 172.21.15.133:0/2614489253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4290100e20 0x7f4290101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:10.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.350+0000 7f4294bff700 1 -- 172.21.15.133:0/2614489253 >> 172.21.15.133:0/2614489253 conn(0x7f42900fb210 msgr2=0x7f42900fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:10.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.351+0000 7f4294bff700 1 -- 172.21.15.133:0/2614489253 shutdown_connections 2023-07-14T18:14:10.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.351+0000 7f4294bff700 1 -- 172.21.15.133:0/2614489253 wait complete. 2023-07-14T18:14:10.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.351+0000 7f4294bff700 1 Processor -- start 2023-07-14T18:14:10.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.352+0000 7f4294bff700 1 -- start start 2023-07-14T18:14:10.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.352+0000 7f4294bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42900ffc30 0x7f42900743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:10.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.352+0000 7f4294bff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4290100e20 0x7f4290072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:10.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.352+0000 7f4294bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42901017b0 0x7f4290072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:10.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.352+0000 7f4294bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4290073710 con 0x7f42900ffc30 2023-07-14T18:14:10.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.352+0000 7f4294bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4290073880 con 0x7f42901017b0 2023-07-14T18:14:10.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.352+0000 7f4294bff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429019fa20 con 0x7f4290100e20 2023-07-14T18:14:10.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.352+0000 7f428e59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42900ffc30 0x7f42900743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:10.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.352+0000 7f428ed9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42901017b0 0x7f4290072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:10.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.352+0000 7f428e59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42900ffc30 0x7f42900743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41928/0 (socket says 172.21.15.133:41928) 2023-07-14T18:14:10.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.352+0000 7f428ed9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42901017b0 0x7f4290072f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:57580/0 (socket says 172.21.15.133:57580) 2023-07-14T18:14:10.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.352+0000 7f428e59c700 1 -- 172.21.15.133:0/116902383 learned_addr learned my addr 172.21.15.133:0/116902383 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:10.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.352+0000 7f428dd9b700 1 --2- 172.21.15.133:0/116902383 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4290100e20 0x7f4290072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:10.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.352+0000 7f428ed9d700 1 -- 172.21.15.133:0/116902383 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4290100e20 msgr2=0x7f4290072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:10.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.352+0000 7f428ed9d700 1 --2- 172.21.15.133:0/116902383 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4290100e20 0x7f4290072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:10.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.352+0000 7f428ed9d700 1 -- 172.21.15.133:0/116902383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42900ffc30 msgr2=0x7f42900743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:10.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.352+0000 7f428ed9d700 1 --2- 172.21.15.133:0/116902383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42900ffc30 0x7f42900743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:10.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.353+0000 7f428ed9d700 1 -- 172.21.15.133:0/116902383 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4288009d00 con 0x7f42901017b0 2023-07-14T18:14:10.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.353+0000 7f428dd9b700 1 --2- 172.21.15.133:0/116902383 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4290100e20 0x7f4290072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:14:10.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.353+0000 7f428e59c700 1 --2- 172.21.15.133:0/116902383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42900ffc30 0x7f42900743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:14:10.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.353+0000 7f428ed9d700 1 --2- 172.21.15.133:0/116902383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42901017b0 0x7f4290072f70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f428800b0a0 tx=0x7f4288007f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:10.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.353+0000 7f427f7fe700 1 -- 172.21.15.133:0/116902383 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428800e040 con 0x7f42901017b0 2023-07-14T18:14:10.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.353+0000 7f427f7fe700 1 -- 172.21.15.133:0/116902383 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4288004c50 con 0x7f42901017b0 2023-07-14T18:14:10.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.353+0000 7f4294bff700 1 -- 172.21.15.133:0/116902383 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f429019fca0 con 0x7f42901017b0 2023-07-14T18:14:10.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.353+0000 7f427f7fe700 1 -- 172.21.15.133:0/116902383 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428801b6b0 con 0x7f42901017b0 2023-07-14T18:14:10.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.353+0000 7f4294bff700 1 -- 172.21.15.133:0/116902383 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42901a0120 con 0x7f42901017b0 2023-07-14T18:14:10.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.354+0000 7f4294bff700 1 -- 172.21.15.133:0/116902383 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f429010b410 con 0x7f42901017b0 2023-07-14T18:14:10.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.356+0000 7f427f7fe700 1 -- 172.21.15.133:0/116902383 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f428801b810 con 0x7f42901017b0 2023-07-14T18:14:10.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.357+0000 7f427f7fe700 1 --2- 172.21.15.133:0/116902383 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f42780a11a0 0x7f42780a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:10.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.357+0000 7f427f7fe700 1 -- 172.21.15.133:0/116902383 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f42880cdf10 con 0x7f42901017b0 2023-07-14T18:14:10.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.357+0000 7f428e59c700 1 --2- 172.21.15.133:0/116902383 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f42780a11a0 0x7f42780a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:10.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.358+0000 7f428e59c700 1 --2- 172.21.15.133:0/116902383 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f42780a11a0 0x7f42780a3650 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4280005950 tx=0x7f4280009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:10.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.359+0000 7f427f7fe700 1 -- 172.21.15.133:0/116902383 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f4288097af0 con 0x7f42901017b0 2023-07-14T18:14:10.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:09 smithi133 ceph-mon[126384]: Deploying daemon mds.b on smithi161 2023-07-14T18:14:10.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:09 smithi133 ceph-mon[126384]: mds.? [v2:172.21.15.133:6840/3282902068,v1:172.21.15.133:6841/3282902068] up:boot 2023-07-14T18:14:10.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:09 smithi133 ceph-mon[126384]: fsmap 11 up:standby 2023-07-14T18:14:10.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:09 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-14T18:14:10.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.600+0000 7f4294bff700 1 -- 172.21.15.133:0/116902383 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42901069f0 con 0x7f42901017b0 2023-07-14T18:14:10.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.601+0000 7f427f7fe700 1 -- 172.21.15.133:0/116902383 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8858 (secure 0 0 0) 0x7f428809b110 con 0x7f42901017b0 2023-07-14T18:14:10.644 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:14:10.644 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3168151271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3168151271},{"type":"v1","addr":"172.21.15.133:6837","nonce":3168151271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14544,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/3093144261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3093144261},{"type":"v1","addr":"172.21.15.133:6839","nonce":3093144261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14556,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/3282902068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":3282902068},{"type":"v1","addr":"172.21.15.133:6841","nonce":3282902068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/4023127259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4023127259},{"type":"v1","addr":"172.21.15.161:6833","nonce":4023127259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2722373740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2722373740},{"type":"v1","addr":"172.21.15.171:6833","nonce":2722373740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2399005074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2399005074},{"type":"v1","addr":"172.21.15.133:6835","nonce":2399005074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3725502564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3725502564},{"type":"v1","addr":"172.21.15.161:6835","nonce":3725502564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1877362116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1877362116},{"type":"v1","addr":"172.21.15.171:6835","nonce":1877362116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3458064798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3458064798},{"type":"v1","addr":"172.21.15.161:6837","nonce":3458064798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/106205332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":106205332},{"type":"v1","addr":"172.21.15.171:6837","nonce":106205332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24541,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/816201198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":816201198},{"type":"v1","addr":"172.21.15.171:6839","nonce":816201198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-07-14T18:14:10.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.603+0000 7f4294bff700 1 -- 172.21.15.133:0/116902383 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f42780a11a0 msgr2=0x7f42780a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:10.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.603+0000 7f4294bff700 1 --2- 172.21.15.133:0/116902383 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f42780a11a0 0x7f42780a3650 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4280005950 tx=0x7f4280009260 comp rx=0 tx=0).stop 2023-07-14T18:14:10.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.603+0000 7f4294bff700 1 -- 172.21.15.133:0/116902383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42901017b0 msgr2=0x7f4290072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:10.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.603+0000 7f4294bff700 1 --2- 172.21.15.133:0/116902383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42901017b0 0x7f4290072f70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f428800b0a0 tx=0x7f4288007f30 comp rx=0 tx=0).stop 2023-07-14T18:14:10.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.603+0000 7f4294bff700 1 -- 172.21.15.133:0/116902383 shutdown_connections 2023-07-14T18:14:10.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.603+0000 7f4294bff700 1 --2- 172.21.15.133:0/116902383 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f42780a11a0 0x7f42780a3650 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:10.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.603+0000 7f4294bff700 1 --2- 172.21.15.133:0/116902383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42901017b0 0x7f4290072f70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:10.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.603+0000 7f4294bff700 1 --2- 172.21.15.133:0/116902383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42900ffc30 0x7f42900743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:10.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.603+0000 7f4294bff700 1 --2- 172.21.15.133:0/116902383 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4290100e20 0x7f4290072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:10.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.603+0000 7f4294bff700 1 -- 172.21.15.133:0/116902383 >> 172.21.15.133:0/116902383 conn(0x7f42900fb210 msgr2=0x7f42900fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:10.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.603+0000 7f4294bff700 1 -- 172.21.15.133:0/116902383 shutdown_connections 2023-07-14T18:14:10.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:10.603+0000 7f4294bff700 1 -- 172.21.15.133:0/116902383 wait complete. 2023-07-14T18:14:10.648 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 12 2023-07-14T18:14:10.699 INFO:journalctl@ceph.mds.b.smithi161.stdout:Jul 14 18:14:10 smithi161 systemd[1]: Starting Ceph mds.b for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:14:10.953 INFO:journalctl@ceph.mds.b.smithi161.stdout:Jul 14 18:14:10 smithi161 podman[154355]: 2023-07-14T18:14:11.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:10 smithi161 ceph-mon[128993]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:11.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:10 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/116902383' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:14:11.350 INFO:journalctl@ceph.mds.b.smithi161.stdout:Jul 14 18:14:10 smithi161 ceph-mds[154396]: -- 172.21.15.161:0/2860627290 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5589a7d091e0 con 0x5589a7cde800 2023-07-14T18:14:11.350 INFO:journalctl@ceph.mds.b.smithi161.stdout:Jul 14 18:14:10 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-b[154371]: starting mds.b at 2023-07-14T18:14:11.350 INFO:journalctl@ceph.mds.b.smithi161.stdout:Jul 14 18:14:10 smithi161 bash[154355]: 243dee7a58e56d4f4c48f7def5921b09031a9f0896a62cbac06d517b8b862169 2023-07-14T18:14:11.350 INFO:journalctl@ceph.mds.b.smithi161.stdout:Jul 14 18:14:10 smithi161 systemd[1]: Started Ceph mds.b for 06d0f5fa-2271-11ee-9b33-001a4aab830c. 2023-07-14T18:14:11.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:10 smithi171 ceph-mon[129241]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:11.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:10 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/116902383' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:14:11.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:10 smithi133 ceph-mon[126384]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:11.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:10 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/116902383' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:14:12.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:11 smithi161 ceph-mon[128993]: mds.? [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] up:boot 2023-07-14T18:14:12.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:11 smithi161 ceph-mon[128993]: fsmap 12 up:standby 2023-07-14T18:14:12.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:11 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T18:14:12.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:11 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:12.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:11 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:12.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:11 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:12.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:11 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:12.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:11 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:12.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:11 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:14:12.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:11 smithi171 ceph-mon[129241]: mds.? [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] up:boot 2023-07-14T18:14:12.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:11 smithi171 ceph-mon[129241]: fsmap 12 up:standby 2023-07-14T18:14:12.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:11 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T18:14:12.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:11 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:12.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:11 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:12.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:11 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:12.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:11 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:12.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:11 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:12.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:11 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:14:12.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:11 smithi133 ceph-mon[126384]: mds.? [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] up:boot 2023-07-14T18:14:12.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:11 smithi133 ceph-mon[126384]: fsmap 12 up:standby 2023-07-14T18:14:12.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:11 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T18:14:12.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:11 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:12.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:11 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:12.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:11 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:12.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:11 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:12.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:11 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:12.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:11 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:14:12.666 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T18:14:12.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.831+0000 7efef8bff700 1 -- 172.21.15.133:0/2558049747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efef410c200 msgr2=0x7efef410e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:12.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.831+0000 7efef8bff700 1 --2- 172.21.15.133:0/2558049747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efef410c200 0x7efef410e5e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efeec00b0a0 tx=0x7efeec00b3b0 comp rx=0 tx=0).stop 2023-07-14T18:14:12.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.831+0000 7efef8bff700 1 -- 172.21.15.133:0/2558049747 shutdown_connections 2023-07-14T18:14:12.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.831+0000 7efef8bff700 1 --2- 172.21.15.133:0/2558049747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efef410c200 0x7efef410e5e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:12.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.831+0000 7efef8bff700 1 --2- 172.21.15.133:0/2558049747 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efef4074680 0x7efef4072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:12.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.831+0000 7efef8bff700 1 --2- 172.21.15.133:0/2558049747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efef40730a0 0x7efef4073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:12.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.831+0000 7efef8bff700 1 -- 172.21.15.133:0/2558049747 >> 172.21.15.133:0/2558049747 conn(0x7efef40fb210 msgr2=0x7efef40fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:12.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.831+0000 7efef8bff700 1 -- 172.21.15.133:0/2558049747 shutdown_connections 2023-07-14T18:14:12.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.831+0000 7efef8bff700 1 -- 172.21.15.133:0/2558049747 wait complete. 2023-07-14T18:14:12.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.832+0000 7efef8bff700 1 Processor -- start 2023-07-14T18:14:12.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.833+0000 7efef8bff700 1 -- start start 2023-07-14T18:14:12.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.833+0000 7efef8bff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efef40730a0 0x7efef419c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:12.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.833+0000 7efef8bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efef4074680 0x7efef419c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:12.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.833+0000 7efef8bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efef410c200 0x7efef41a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:12.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.833+0000 7efef8bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef41a3c60 con 0x7efef410c200 2023-07-14T18:14:12.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.833+0000 7efef8bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef41a3da0 con 0x7efef4074680 2023-07-14T18:14:12.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.833+0000 7efef8bff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef41a3f30 con 0x7efef40730a0 2023-07-14T18:14:12.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.833+0000 7efef259c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efef40730a0 0x7efef419c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:12.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.833+0000 7efef1d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efef4074680 0x7efef419c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:12.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.833+0000 7efef1d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efef4074680 0x7efef419c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:57600/0 (socket says 172.21.15.133:57600) 2023-07-14T18:14:12.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.833+0000 7efef1d9b700 1 -- 172.21.15.133:0/108374667 learned_addr learned my addr 172.21.15.133:0/108374667 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:12.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.834+0000 7efef2d9d700 1 --2- 172.21.15.133:0/108374667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efef410c200 0x7efef41a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:12.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.834+0000 7efef259c700 1 -- 172.21.15.133:0/108374667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efef4074680 msgr2=0x7efef419c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:12.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.834+0000 7efef259c700 1 --2- 172.21.15.133:0/108374667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efef4074680 0x7efef419c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:12.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.834+0000 7efef259c700 1 -- 172.21.15.133:0/108374667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efef410c200 msgr2=0x7efef41a3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:12.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.834+0000 7efef259c700 1 --2- 172.21.15.133:0/108374667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efef410c200 0x7efef41a3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:12.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.834+0000 7efef259c700 1 -- 172.21.15.133:0/108374667 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeec009d00 con 0x7efef40730a0 2023-07-14T18:14:12.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.834+0000 7efef259c700 1 --2- 172.21.15.133:0/108374667 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efef40730a0 0x7efef419c0e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efef4074a50 tx=0x7efee400b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:12.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.835+0000 7efee37fe700 1 -- 172.21.15.133:0/108374667 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee4011830 con 0x7efef40730a0 2023-07-14T18:14:12.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.835+0000 7efee37fe700 1 -- 172.21.15.133:0/108374667 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efee4011e50 con 0x7efef40730a0 2023-07-14T18:14:12.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.835+0000 7efef8bff700 1 -- 172.21.15.133:0/108374667 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef41a4210 con 0x7efef40730a0 2023-07-14T18:14:12.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.835+0000 7efee37fe700 1 -- 172.21.15.133:0/108374667 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee4010d70 con 0x7efef40730a0 2023-07-14T18:14:12.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.835+0000 7efef8bff700 1 -- 172.21.15.133:0/108374667 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef41a47f0 con 0x7efef40730a0 2023-07-14T18:14:12.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.836+0000 7efef8bff700 1 -- 172.21.15.133:0/108374667 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef4102460 con 0x7efef40730a0 2023-07-14T18:14:12.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.838+0000 7efee37fe700 1 -- 172.21.15.133:0/108374667 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7efee4010410 con 0x7efef40730a0 2023-07-14T18:14:12.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.839+0000 7efee37fe700 1 --2- 172.21.15.133:0/108374667 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efedc0a1220 0x7efedc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:12.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.839+0000 7efee37fe700 1 -- 172.21.15.133:0/108374667 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7efee40cbfa0 con 0x7efef40730a0 2023-07-14T18:14:12.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.839+0000 7efef1d9b700 1 --2- 172.21.15.133:0/108374667 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efedc0a1220 0x7efedc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:12.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.840+0000 7efef1d9b700 1 --2- 172.21.15.133:0/108374667 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efedc0a1220 0x7efedc0a36d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efee8005950 tx=0x7efee8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:12.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:12.841+0000 7efee37fe700 1 -- 172.21.15.133:0/108374667 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7efee4095af0 con 0x7efef40730a0 2023-07-14T18:14:13.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.112+0000 7efef8bff700 1 -- 172.21.15.133:0/108374667 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efef402c8e0 con 0x7efef40730a0 2023-07-14T18:14:13.156 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.113+0000 7efee37fe700 1 -- 172.21.15.133:0/108374667 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9619 (secure 0 0 0) 0x7efee4010ed0 con 0x7efef40730a0 2023-07-14T18:14:13.157 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:14:13.157 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3168151271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3168151271},{"type":"v1","addr":"172.21.15.133:6837","nonce":3168151271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14544,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/3093144261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3093144261},{"type":"v1","addr":"172.21.15.133:6839","nonce":3093144261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14556,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/3282902068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":3282902068},{"type":"v1","addr":"172.21.15.133:6841","nonce":3282902068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/4023127259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4023127259},{"type":"v1","addr":"172.21.15.161:6833","nonce":4023127259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2722373740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2722373740},{"type":"v1","addr":"172.21.15.171:6833","nonce":2722373740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2399005074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2399005074},{"type":"v1","addr":"172.21.15.133:6835","nonce":2399005074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3725502564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3725502564},{"type":"v1","addr":"172.21.15.161:6835","nonce":3725502564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1877362116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1877362116},{"type":"v1","addr":"172.21.15.171:6835","nonce":1877362116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3458064798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3458064798},{"type":"v1","addr":"172.21.15.161:6837","nonce":3458064798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/106205332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":106205332},{"type":"v1","addr":"172.21.15.171:6837","nonce":106205332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24541,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/816201198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":816201198},{"type":"v1","addr":"172.21.15.171:6839","nonce":816201198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/2503865871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2503865871},{"type":"v1","addr":"172.21.15.161:6839","nonce":2503865871}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2023-07-14T18:14:13.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.116+0000 7efef8bff700 1 -- 172.21.15.133:0/108374667 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efedc0a1220 msgr2=0x7efedc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:13.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.116+0000 7efef8bff700 1 --2- 172.21.15.133:0/108374667 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efedc0a1220 0x7efedc0a36d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efee8005950 tx=0x7efee8009260 comp rx=0 tx=0).stop 2023-07-14T18:14:13.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.116+0000 7efef8bff700 1 -- 172.21.15.133:0/108374667 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efef40730a0 msgr2=0x7efef419c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:13.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.116+0000 7efef8bff700 1 --2- 172.21.15.133:0/108374667 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efef40730a0 0x7efef419c0e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efef4074a50 tx=0x7efee400b730 comp rx=0 tx=0).stop 2023-07-14T18:14:13.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.116+0000 7efef8bff700 1 -- 172.21.15.133:0/108374667 shutdown_connections 2023-07-14T18:14:13.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.116+0000 7efef8bff700 1 --2- 172.21.15.133:0/108374667 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efedc0a1220 0x7efedc0a36d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:13.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.116+0000 7efef8bff700 1 --2- 172.21.15.133:0/108374667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efef410c200 0x7efef41a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:13.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.116+0000 7efef8bff700 1 --2- 172.21.15.133:0/108374667 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efef40730a0 0x7efef419c0e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:13.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.116+0000 7efef8bff700 1 --2- 172.21.15.133:0/108374667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efef4074680 0x7efef419c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:13.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.116+0000 7efef8bff700 1 -- 172.21.15.133:0/108374667 >> 172.21.15.133:0/108374667 conn(0x7efef40fb210 msgr2=0x7efef40ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:13.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.116+0000 7efef8bff700 1 -- 172.21.15.133:0/108374667 shutdown_connections 2023-07-14T18:14:13.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.116+0000 7efef8bff700 1 -- 172.21.15.133:0/108374667 wait complete. 2023-07-14T18:14:13.161 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 13 2023-07-14T18:14:13.178 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T18:14:13.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:12 smithi161 ceph-mon[128993]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:13.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:12 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:13.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:12 smithi171 ceph-mon[129241]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:13.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:12 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:13.385 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.342+0000 7f5833fff700 1 -- 172.21.15.133:0/1153725026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f582c1017b0 msgr2=0x7f582c108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:13.385 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.342+0000 7f5833fff700 1 --2- 172.21.15.133:0/1153725026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f582c1017b0 0x7f582c108070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f582400b0a0 tx=0x7f582400b3b0 comp rx=0 tx=0).stop 2023-07-14T18:14:13.386 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.342+0000 7f5833fff700 1 -- 172.21.15.133:0/1153725026 shutdown_connections 2023-07-14T18:14:13.386 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.342+0000 7f5833fff700 1 --2- 172.21.15.133:0/1153725026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c0ffc30 0x7f582c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:13.386 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.342+0000 7f5833fff700 1 --2- 172.21.15.133:0/1153725026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f582c1017b0 0x7f582c108070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:13.387 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.342+0000 7f5833fff700 1 --2- 172.21.15.133:0/1153725026 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f582c100e20 0x7f582c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:13.387 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.342+0000 7f5833fff700 1 -- 172.21.15.133:0/1153725026 >> 172.21.15.133:0/1153725026 conn(0x7f582c0fb210 msgr2=0x7f582c0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:13.387 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.342+0000 7f5833fff700 1 -- 172.21.15.133:0/1153725026 shutdown_connections 2023-07-14T18:14:13.387 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.342+0000 7f5833fff700 1 -- 172.21.15.133:0/1153725026 wait complete. 2023-07-14T18:14:13.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.343+0000 7f5833fff700 1 Processor -- start 2023-07-14T18:14:13.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.344+0000 7f5833fff700 1 -- start start 2023-07-14T18:14:13.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.344+0000 7f5833fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f582c0ffc30 0x7f582c195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:13.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.344+0000 7f5833fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c100e20 0x7f582c195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:13.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.344+0000 7f5833fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f582c1017b0 0x7f582c19cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:13.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.344+0000 7f5833fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c19d5b0 con 0x7f582c0ffc30 2023-07-14T18:14:13.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.344+0000 7f5833fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c19d6f0 con 0x7f582c100e20 2023-07-14T18:14:13.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.344+0000 7f5833fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c19d880 con 0x7f582c1017b0 2023-07-14T18:14:13.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.344+0000 7f5831d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f582c0ffc30 0x7f582c195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:13.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.344+0000 7f5831d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f582c0ffc30 0x7f582c195a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41950/0 (socket says 172.21.15.133:41950) 2023-07-14T18:14:13.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.344+0000 7f5831d9b700 1 -- 172.21.15.133:0/2234237630 learned_addr learned my addr 172.21.15.133:0/2234237630 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:13.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.344+0000 7f583259c700 1 --2- 172.21.15.133:0/2234237630 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f582c1017b0 0x7f582c19cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:13.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.344+0000 7f583159a700 1 --2- 172.21.15.133:0/2234237630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c100e20 0x7f582c195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:13.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.345+0000 7f5831d9b700 1 -- 172.21.15.133:0/2234237630 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f582c1017b0 msgr2=0x7f582c19cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:13.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.345+0000 7f5831d9b700 1 --2- 172.21.15.133:0/2234237630 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f582c1017b0 0x7f582c19cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:13.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.345+0000 7f5831d9b700 1 -- 172.21.15.133:0/2234237630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c100e20 msgr2=0x7f582c195f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:13.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.345+0000 7f5831d9b700 1 --2- 172.21.15.133:0/2234237630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c100e20 0x7f582c195f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:13.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.345+0000 7f5831d9b700 1 -- 172.21.15.133:0/2234237630 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5824009d00 con 0x7f582c0ffc30 2023-07-14T18:14:13.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.345+0000 7f5831d9b700 1 --2- 172.21.15.133:0/2234237630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f582c0ffc30 0x7f582c195a30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f582800e970 tx=0x7f582800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:13.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.345+0000 7f5822ffd700 1 -- 172.21.15.133:0/2234237630 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5828009790 con 0x7f582c0ffc30 2023-07-14T18:14:13.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.345+0000 7f5822ffd700 1 -- 172.21.15.133:0/2234237630 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f582800c7e0 con 0x7f582c0ffc30 2023-07-14T18:14:13.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.345+0000 7f5833fff700 1 -- 172.21.15.133:0/2234237630 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582c19db60 con 0x7f582c0ffc30 2023-07-14T18:14:13.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.345+0000 7f5822ffd700 1 -- 172.21.15.133:0/2234237630 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5828003c70 con 0x7f582c0ffc30 2023-07-14T18:14:13.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.345+0000 7f5833fff700 1 -- 172.21.15.133:0/2234237630 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582c19e0c0 con 0x7f582c0ffc30 2023-07-14T18:14:13.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.346+0000 7f5833fff700 1 -- 172.21.15.133:0/2234237630 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582c1070f0 con 0x7f582c0ffc30 2023-07-14T18:14:13.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.348+0000 7f5822ffd700 1 -- 172.21.15.133:0/2234237630 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f582800cc50 con 0x7f582c0ffc30 2023-07-14T18:14:13.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.349+0000 7f5822ffd700 1 --2- 172.21.15.133:0/2234237630 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f58180a1220 0x7f58180a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:13.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.350+0000 7f5822ffd700 1 -- 172.21.15.133:0/2234237630 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5828014070 con 0x7f582c0ffc30 2023-07-14T18:14:13.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.350+0000 7f583159a700 1 --2- 172.21.15.133:0/2234237630 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f58180a1220 0x7f58180a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:13.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.350+0000 7f583159a700 1 --2- 172.21.15.133:0/2234237630 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f58180a1220 0x7f58180a36d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f581c009c60 tx=0x7f581c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:13.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.353+0000 7f5822ffd700 1 -- 172.21.15.133:0/2234237630 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f58280961a0 con 0x7f582c0ffc30 2023-07-14T18:14:13.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:12 smithi133 ceph-mon[126384]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:13.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:12 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:13.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.566+0000 7f5833fff700 1 -- 172.21.15.133:0/2234237630 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f582c19dcf0 con 0x7f582c0ffc30 2023-07-14T18:14:13.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.568+0000 7f5822ffd700 1 -- 172.21.15.133:0/2234237630 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18465 (secure 0 0 0) 0x7f582c19dcf0 con 0x7f582c0ffc30 2023-07-14T18:14:13.611 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:14:13.611 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":80,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","created":"2023-07-14T18:06:49.561582+0000","modified":"2023-07-14T18:13:30.945504+0000","last_up_change":"2023-07-14T18:13:29.836099+0000","last_in_change":"2023-07-14T18:13:11.606710+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T18:10:03.945039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"85a29de7-d712-422a-b647-d9e90091aaf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6803","nonce":3076510801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6805","nonce":3076510801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6809","nonce":3076510801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6807","nonce":3076510801}]},"public_addr":"172.21.15.133:6803/3076510801","cluster_addr":"172.21.15.133:6805/3076510801","heartbeat_back_addr":"172.21.15.133:6809/3076510801","heartbeat_front_addr":"172.21.15.133:6807/3076510801","state":["exists","up"]},{"osd":1,"uuid":"80ad4e78-7917-495e-b119-99f7c7c36243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6801","nonce":1841901852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6803","nonce":1841901852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6807","nonce":1841901852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6805","nonce":1841901852}]},"public_addr":"172.21.15.161:6801/1841901852","cluster_addr":"172.21.15.161:6803/1841901852","heartbeat_back_addr":"172.21.15.161:6807/1841901852","heartbeat_front_addr":"172.21.15.161:6805/1841901852","state":["exists","up"]},{"osd":2,"uuid":"fc4bfd37-1fdd-47c6-a48e-6d910daf9e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6801","nonce":346103099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6803","nonce":346103099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6807","nonce":346103099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6805","nonce":346103099}]},"public_addr":"172.21.15.171:6801/346103099","cluster_addr":"172.21.15.171:6803/346103099","heartbeat_back_addr":"172.21.15.171:6807/346103099","heartbeat_front_addr":"172.21.15.171:6805/346103099","state":["exists","up"]},{"osd":3,"uuid":"4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6811","nonce":2152994143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6813","nonce":2152994143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6817","nonce":2152994143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6815","nonce":2152994143}]},"public_addr":"172.21.15.133:6811/2152994143","cluster_addr":"172.21.15.133:6813/2152994143","heartbeat_back_addr":"172.21.15.133:6817/2152994143","heartbeat_front_addr":"172.21.15.133:6815/2152994143","state":["exists","up"]},{"osd":4,"uuid":"4601a1c2-254b-4c4d-9b15-8fc952b3d2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6809","nonce":47213291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6811","nonce":47213291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6815","nonce":47213291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6813","nonce":47213291}]},"public_addr":"172.21.15.161:6809/47213291","cluster_addr":"172.21.15.161:6811/47213291","heartbeat_back_addr":"172.21.15.161:6815/47213291","heartbeat_front_addr":"172.21.15.161:6813/47213291","state":["exists","up"]},{"osd":5,"uuid":"6cca180a-9140-4e49-835b-86842dae0574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6809","nonce":777556751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6811","nonce":777556751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6815","nonce":777556751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6813","nonce":777556751}]},"public_addr":"172.21.15.171:6809/777556751","cluster_addr":"172.21.15.171:6811/777556751","heartbeat_back_addr":"172.21.15.171:6815/777556751","heartbeat_front_addr":"172.21.15.171:6813/777556751","state":["exists","up"]},{"osd":6,"uuid":"ea0299ff-7740-4494-a3a9-1503b3fc0ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6819","nonce":2884152358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6821","nonce":2884152358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6825","nonce":2884152358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6823","nonce":2884152358}]},"public_addr":"172.21.15.133:6819/2884152358","cluster_addr":"172.21.15.133:6821/2884152358","heartbeat_back_addr":"172.21.15.133:6825/2884152358","heartbeat_front_addr":"172.21.15.133:6823/2884152358","state":["exists","up"]},{"osd":7,"uuid":"b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6817","nonce":4183705749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6819","nonce":4183705749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6823","nonce":4183705749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6821","nonce":4183705749}]},"public_addr":"172.21.15.161:6817/4183705749","cluster_addr":"172.21.15.161:6819/4183705749","heartbeat_back_addr":"172.21.15.161:6823/4183705749","heartbeat_front_addr":"172.21.15.161:6821/4183705749","state":["exists","up"]},{"osd":8,"uuid":"87dffc07-bf48-48c8-b418-5c4b509686d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6817","nonce":2771494483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6819","nonce":2771494483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6823","nonce":2771494483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6821","nonce":2771494483}]},"public_addr":"172.21.15.171:6817/2771494483","cluster_addr":"172.21.15.171:6819/2771494483","heartbeat_back_addr":"172.21.15.171:6823/2771494483","heartbeat_front_addr":"172.21.15.171:6821/2771494483","state":["exists","up"]},{"osd":9,"uuid":"e0832870-79ce-4bce-8045-58b7a7f6dc69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6827","nonce":2066164149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6829","nonce":2066164149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6833","nonce":2066164149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6831","nonce":2066164149}]},"public_addr":"172.21.15.133:6827/2066164149","cluster_addr":"172.21.15.133:6829/2066164149","heartbeat_back_addr":"172.21.15.133:6833/2066164149","heartbeat_front_addr":"172.21.15.133:6831/2066164149","state":["exists","up"]},{"osd":10,"uuid":"081f0484-d134-4b2e-9c48-ef8e6a646422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6825","nonce":2162333626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6827","nonce":2162333626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6831","nonce":2162333626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6829","nonce":2162333626}]},"public_addr":"172.21.15.161:6825/2162333626","cluster_addr":"172.21.15.161:6827/2162333626","heartbeat_back_addr":"172.21.15.161:6831/2162333626","heartbeat_front_addr":"172.21.15.161:6829/2162333626","state":["exists","up"]},{"osd":11,"uuid":"4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6825","nonce":3403894313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6827","nonce":3403894313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6831","nonce":3403894313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6829","nonce":3403894313}]},"public_addr":"172.21.15.171:6825/3403894313","cluster_addr":"172.21.15.171:6827/3403894313","heartbeat_back_addr":"172.21.15.171:6831/3403894313","heartbeat_front_addr":"172.21.15.171:6829/3403894313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:17.816229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:39.638543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:00.543317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:26.988921+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:48.644694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:10.049860+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:35.986508+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:57.753846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:19.018245+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:44.460450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:06.279960+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:27.955006+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/1832569241":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6801/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:0/4219815304":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:6801/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/432253615":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1840700480":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1807775439":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1089030734":"2023-07-15T18:07:47.346704+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T18:14:13.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.570+0000 7f5833fff700 1 -- 172.21.15.133:0/2234237630 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f58180a1220 msgr2=0x7f58180a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:13.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.570+0000 7f5833fff700 1 --2- 172.21.15.133:0/2234237630 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f58180a1220 0x7f58180a36d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f581c009c60 tx=0x7f581c009260 comp rx=0 tx=0).stop 2023-07-14T18:14:13.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.570+0000 7f5833fff700 1 -- 172.21.15.133:0/2234237630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f582c0ffc30 msgr2=0x7f582c195a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:13.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.570+0000 7f5833fff700 1 --2- 172.21.15.133:0/2234237630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f582c0ffc30 0x7f582c195a30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f582800e970 tx=0x7f582800ec80 comp rx=0 tx=0).stop 2023-07-14T18:14:13.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.570+0000 7f5833fff700 1 -- 172.21.15.133:0/2234237630 shutdown_connections 2023-07-14T18:14:13.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.570+0000 7f5833fff700 1 --2- 172.21.15.133:0/2234237630 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f58180a1220 0x7f58180a36d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:13.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.570+0000 7f5833fff700 1 --2- 172.21.15.133:0/2234237630 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f582c1017b0 0x7f582c19cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:13.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.570+0000 7f5833fff700 1 --2- 172.21.15.133:0/2234237630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f582c0ffc30 0x7f582c195a30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:13.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.570+0000 7f5833fff700 1 --2- 172.21.15.133:0/2234237630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c100e20 0x7f582c195f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:13.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.570+0000 7f5833fff700 1 -- 172.21.15.133:0/2234237630 >> 172.21.15.133:0/2234237630 conn(0x7f582c0fb210 msgr2=0x7f582c104230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:13.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.570+0000 7f5833fff700 1 -- 172.21.15.133:0/2234237630 shutdown_connections 2023-07-14T18:14:13.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.570+0000 7f5833fff700 1 -- 172.21.15.133:0/2234237630 wait complete. 2023-07-14T18:14:13.633 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T18:10:03.945039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T18:14:13.634 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T18:14:13.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.803+0000 7fd53c1ff700 1 -- 172.21.15.133:0/4182666468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5340ffc20 msgr2=0x7fd534100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:13.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.803+0000 7fd53c1ff700 1 --2- 172.21.15.133:0/4182666468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5340ffc20 0x7fd534100030 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd530009990 tx=0x7fd530009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:13.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.803+0000 7fd53c1ff700 1 -- 172.21.15.133:0/4182666468 shutdown_connections 2023-07-14T18:14:13.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.803+0000 7fd53c1ff700 1 --2- 172.21.15.133:0/4182666468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5341017a0 0x7fd534108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:13.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.803+0000 7fd53c1ff700 1 --2- 172.21.15.133:0/4182666468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5340ffc20 0x7fd534100030 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:13.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.803+0000 7fd53c1ff700 1 --2- 172.21.15.133:0/4182666468 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd534100e10 0x7fd534101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:13.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.803+0000 7fd53c1ff700 1 -- 172.21.15.133:0/4182666468 >> 172.21.15.133:0/4182666468 conn(0x7fd5340fb260 msgr2=0x7fd5340fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:13.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.803+0000 7fd53c1ff700 1 -- 172.21.15.133:0/4182666468 shutdown_connections 2023-07-14T18:14:13.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.803+0000 7fd53c1ff700 1 -- 172.21.15.133:0/4182666468 wait complete. 2023-07-14T18:14:13.852 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.804+0000 7fd53c1ff700 1 Processor -- start 2023-07-14T18:14:13.852 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.804+0000 7fd53c1ff700 1 -- start start 2023-07-14T18:14:13.852 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.804+0000 7fd53c1ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5340ffc20 0x7fd534197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:13.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.804+0000 7fd53c1ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd534100e10 0x7fd534198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:13.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.804+0000 7fd53c1ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5341017a0 0x7fd53419f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:13.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.804+0000 7fd53c1ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53419f750 con 0x7fd5340ffc20 2023-07-14T18:14:13.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.804+0000 7fd53c1ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53419f890 con 0x7fd5341017a0 2023-07-14T18:14:13.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.804+0000 7fd53c1ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53419fa20 con 0x7fd534100e10 2023-07-14T18:14:13.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.805+0000 7fd539f9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5340ffc20 0x7fd534197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:13.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.805+0000 7fd539f9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5340ffc20 0x7fd534197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41958/0 (socket says 172.21.15.133:41958) 2023-07-14T18:14:13.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.805+0000 7fd539f9b700 1 -- 172.21.15.133:0/3980193026 learned_addr learned my addr 172.21.15.133:0/3980193026 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:13.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.805+0000 7fd53979a700 1 --2- 172.21.15.133:0/3980193026 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd534100e10 0x7fd534198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:13.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.805+0000 7fd53a79c700 1 --2- 172.21.15.133:0/3980193026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5341017a0 0x7fd53419f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:13.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.805+0000 7fd539f9b700 1 -- 172.21.15.133:0/3980193026 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd534100e10 msgr2=0x7fd534198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:13.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.805+0000 7fd539f9b700 1 --2- 172.21.15.133:0/3980193026 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd534100e10 0x7fd534198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:13.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.805+0000 7fd539f9b700 1 -- 172.21.15.133:0/3980193026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5341017a0 msgr2=0x7fd53419f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:13.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.805+0000 7fd539f9b700 1 --2- 172.21.15.133:0/3980193026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5341017a0 0x7fd53419f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:13.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.805+0000 7fd539f9b700 1 -- 172.21.15.133:0/3980193026 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5300095f0 con 0x7fd5340ffc20 2023-07-14T18:14:13.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.805+0000 7fd53a79c700 1 --2- 172.21.15.133:0/3980193026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5341017a0 0x7fd53419f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:14:13.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.805+0000 7fd539f9b700 1 --2- 172.21.15.133:0/3980193026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5340ffc20 0x7fd534197bd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd53000b620 tx=0x7fd530005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:13.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.805+0000 7fd52affd700 1 -- 172.21.15.133:0/3980193026 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53001d070 con 0x7fd5340ffc20 2023-07-14T18:14:13.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.806+0000 7fd53c1ff700 1 -- 172.21.15.133:0/3980193026 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd53419fca0 con 0x7fd5340ffc20 2023-07-14T18:14:13.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.806+0000 7fd52affd700 1 -- 172.21.15.133:0/3980193026 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd530021410 con 0x7fd5340ffc20 2023-07-14T18:14:13.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.806+0000 7fd52affd700 1 -- 172.21.15.133:0/3980193026 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd530017650 con 0x7fd5340ffc20 2023-07-14T18:14:13.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.806+0000 7fd53c1ff700 1 -- 172.21.15.133:0/3980193026 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5341a00d0 con 0x7fd5340ffc20 2023-07-14T18:14:13.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.807+0000 7fd53c1ff700 1 -- 172.21.15.133:0/3980193026 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd534192300 con 0x7fd5340ffc20 2023-07-14T18:14:13.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.809+0000 7fd52affd700 1 -- 172.21.15.133:0/3980193026 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fd530021580 con 0x7fd5340ffc20 2023-07-14T18:14:13.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.811+0000 7fd52affd700 1 --2- 172.21.15.133:0/3980193026 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd5200a1150 0x7fd5200a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:13.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.811+0000 7fd52affd700 1 -- 172.21.15.133:0/3980193026 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd5300cda10 con 0x7fd5340ffc20 2023-07-14T18:14:13.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.812+0000 7fd53979a700 1 --2- 172.21.15.133:0/3980193026 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd5200a1150 0x7fd5200a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:13.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.812+0000 7fd53979a700 1 --2- 172.21.15.133:0/3980193026 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd5200a1150 0x7fd5200a3600 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd524007c60 tx=0x7fd5240073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:13.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:13.815+0000 7fd52affd700 1 -- 172.21.15.133:0/3980193026 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fd5300975f0 con 0x7fd5340ffc20 2023-07-14T18:14:14.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.034+0000 7fd53c1ff700 1 -- 172.21.15.133:0/3980193026 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd534061250 con 0x7fd5340ffc20 2023-07-14T18:14:14.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.038+0000 7fd52affd700 1 -- 172.21.15.133:0/3980193026 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd53009ac10 con 0x7fd5340ffc20 2023-07-14T18:14:14.081 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-07-14T18:14:14.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.040+0000 7fd53c1ff700 1 -- 172.21.15.133:0/3980193026 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd5200a1150 msgr2=0x7fd5200a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:14.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.040+0000 7fd53c1ff700 1 --2- 172.21.15.133:0/3980193026 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd5200a1150 0x7fd5200a3600 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd524007c60 tx=0x7fd5240073d0 comp rx=0 tx=0).stop 2023-07-14T18:14:14.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.040+0000 7fd53c1ff700 1 -- 172.21.15.133:0/3980193026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5340ffc20 msgr2=0x7fd534197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:14.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.040+0000 7fd53c1ff700 1 --2- 172.21.15.133:0/3980193026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5340ffc20 0x7fd534197bd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd53000b620 tx=0x7fd530005740 comp rx=0 tx=0).stop 2023-07-14T18:14:14.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.041+0000 7fd53c1ff700 1 -- 172.21.15.133:0/3980193026 shutdown_connections 2023-07-14T18:14:14.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.041+0000 7fd53c1ff700 1 --2- 172.21.15.133:0/3980193026 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd5200a1150 0x7fd5200a3600 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:14.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.041+0000 7fd53c1ff700 1 --2- 172.21.15.133:0/3980193026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5341017a0 0x7fd53419f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:14.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.041+0000 7fd53c1ff700 1 --2- 172.21.15.133:0/3980193026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5340ffc20 0x7fd534197bd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:14.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.041+0000 7fd53c1ff700 1 --2- 172.21.15.133:0/3980193026 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd534100e10 0x7fd534198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:14.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.041+0000 7fd53c1ff700 1 -- 172.21.15.133:0/3980193026 >> 172.21.15.133:0/3980193026 conn(0x7fd5340fb260 msgr2=0x7fd5341066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:14.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.041+0000 7fd53c1ff700 1 -- 172.21.15.133:0/3980193026 shutdown_connections 2023-07-14T18:14:14.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.041+0000 7fd53c1ff700 1 -- 172.21.15.133:0/3980193026 wait complete. 2023-07-14T18:14:14.103 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-14T18:14:14.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.265+0000 7f6e8f9ff700 1 -- 172.21.15.133:0/412249083 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e880ffc30 msgr2=0x7f6e88100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:14.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.265+0000 7f6e8f9ff700 1 --2- 172.21.15.133:0/412249083 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e880ffc30 0x7f6e88100040 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6e84009990 tx=0x7f6e84009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:14.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.265+0000 7f6e8f9ff700 1 -- 172.21.15.133:0/412249083 shutdown_connections 2023-07-14T18:14:14.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.265+0000 7f6e8f9ff700 1 --2- 172.21.15.133:0/412249083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e881017b0 0x7f6e88108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:14.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.265+0000 7f6e8f9ff700 1 --2- 172.21.15.133:0/412249083 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e880ffc30 0x7f6e88100040 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:14.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.265+0000 7f6e8f9ff700 1 --2- 172.21.15.133:0/412249083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e88100e20 0x7f6e88101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:14.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.265+0000 7f6e8f9ff700 1 -- 172.21.15.133:0/412249083 >> 172.21.15.133:0/412249083 conn(0x7f6e880fb210 msgr2=0x7f6e880fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:14.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.265+0000 7f6e8f9ff700 1 -- 172.21.15.133:0/412249083 shutdown_connections 2023-07-14T18:14:14.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.265+0000 7f6e8f9ff700 1 -- 172.21.15.133:0/412249083 wait complete. 2023-07-14T18:14:14.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.266+0000 7f6e8f9ff700 1 Processor -- start 2023-07-14T18:14:14.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.267+0000 7f6e8f9ff700 1 -- start start 2023-07-14T18:14:14.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.267+0000 7f6e8f9ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e880ffc30 0x7f6e880743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:14.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.267+0000 7f6e8f9ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e88100e20 0x7f6e88072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:14.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.267+0000 7f6e8f9ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e881017b0 0x7f6e88072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:14.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.267+0000 7f6e8f9ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e88073710 con 0x7f6e88100e20 2023-07-14T18:14:14.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.267+0000 7f6e8f9ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e88073880 con 0x7f6e881017b0 2023-07-14T18:14:14.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.267+0000 7f6e8f9ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8819f960 con 0x7f6e880ffc30 2023-07-14T18:14:14.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.267+0000 7f6e8cf9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e88100e20 0x7f6e88072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:14.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.267+0000 7f6e8d79b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e880ffc30 0x7f6e880743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:14.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.267+0000 7f6e8cf9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e88100e20 0x7f6e88072a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41982/0 (socket says 172.21.15.133:41982) 2023-07-14T18:14:14.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.267+0000 7f6e8cf9a700 1 -- 172.21.15.133:0/1350270673 learned_addr learned my addr 172.21.15.133:0/1350270673 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:14.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.267+0000 7f6e8df9c700 1 --2- 172.21.15.133:0/1350270673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e881017b0 0x7f6e88072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:14.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.268+0000 7f6e8cf9a700 1 -- 172.21.15.133:0/1350270673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e880ffc30 msgr2=0x7f6e880743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:14.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.268+0000 7f6e8cf9a700 1 --2- 172.21.15.133:0/1350270673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e880ffc30 0x7f6e880743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:14.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.268+0000 7f6e8cf9a700 1 -- 172.21.15.133:0/1350270673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e881017b0 msgr2=0x7f6e88072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:14.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.268+0000 7f6e8cf9a700 1 --2- 172.21.15.133:0/1350270673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e881017b0 0x7f6e88072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:14.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.268+0000 7f6e8cf9a700 1 -- 172.21.15.133:0/1350270673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e78009520 con 0x7f6e88100e20 2023-07-14T18:14:14.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.268+0000 7f6e8d79b700 1 --2- 172.21.15.133:0/1350270673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e880ffc30 0x7f6e880743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:14:14.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.268+0000 7f6e8cf9a700 1 --2- 172.21.15.133:0/1350270673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e88100e20 0x7f6e88072a30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6e7800eb30 tx=0x7f6e7800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:14.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.268+0000 7f6e7e7fc700 1 -- 172.21.15.133:0/1350270673 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e78009880 con 0x7f6e88100e20 2023-07-14T18:14:14.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.268+0000 7f6e7e7fc700 1 -- 172.21.15.133:0/1350270673 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e780044e0 con 0x7f6e88100e20 2023-07-14T18:14:14.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.268+0000 7f6e8f9ff700 1 -- 172.21.15.133:0/1350270673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e840095f0 con 0x7f6e88100e20 2023-07-14T18:14:14.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.268+0000 7f6e7e7fc700 1 -- 172.21.15.133:0/1350270673 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e78003d80 con 0x7f6e88100e20 2023-07-14T18:14:14.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.268+0000 7f6e8f9ff700 1 -- 172.21.15.133:0/1350270673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e881a0010 con 0x7f6e88100e20 2023-07-14T18:14:14.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.269+0000 7f6e8f9ff700 1 -- 172.21.15.133:0/1350270673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e8810b3f0 con 0x7f6e88100e20 2023-07-14T18:14:14.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.271+0000 7f6e7e7fc700 1 -- 172.21.15.133:0/1350270673 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6e78003f20 con 0x7f6e88100e20 2023-07-14T18:14:14.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.274+0000 7f6e7e7fc700 1 --2- 172.21.15.133:0/1350270673 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6e740a1220 0x7f6e740a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:14.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.274+0000 7f6e7e7fc700 1 -- 172.21.15.133:0/1350270673 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6e78014070 con 0x7f6e88100e20 2023-07-14T18:14:14.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.275+0000 7f6e8d79b700 1 --2- 172.21.15.133:0/1350270673 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6e740a1220 0x7f6e740a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:14.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.275+0000 7f6e8d79b700 1 --2- 172.21.15.133:0/1350270673 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6e740a1220 0x7f6e740a36d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400b2e0 tx=0x7f6e84005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:14.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.278+0000 7f6e7e7fc700 1 -- 172.21.15.133:0/1350270673 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f6e78096540 con 0x7f6e88100e20 2023-07-14T18:14:14.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:13 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/108374667' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:14:14.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:13 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2234237630' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:14:14.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:13 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:14.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:13 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:14.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:13 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/108374667' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:14:14.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:13 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2234237630' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:14:14.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:13 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:14.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:13 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:14.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:13 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/108374667' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:14:14.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:13 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2234237630' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:14:14.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:13 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:14.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:13 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:14.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.489+0000 7f6e8f9ff700 1 -- 172.21.15.133:0/1350270673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6e8810b5e0 con 0x7f6e88100e20 2023-07-14T18:14:14.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.489+0000 7f6e7e7fc700 1 -- 172.21.15.133:0/1350270673 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6e7801e2e0 con 0x7f6e88100e20 2023-07-14T18:14:14.532 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:14:14.533 INFO:teuthology.orchestra.run.smithi133.stdout:[ 2023-07-14T18:14:14.533 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:14:14.533 INFO:teuthology.orchestra.run.smithi133.stdout: "poolnum": 1, 2023-07-14T18:14:14.533 INFO:teuthology.orchestra.run.smithi133.stdout: "poolname": ".mgr" 2023-07-14T18:14:14.533 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:14:14.533 INFO:teuthology.orchestra.run.smithi133.stdout:] 2023-07-14T18:14:14.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.492+0000 7f6e8f9ff700 1 -- 172.21.15.133:0/1350270673 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6e740a1220 msgr2=0x7f6e740a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:14.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.492+0000 7f6e8f9ff700 1 --2- 172.21.15.133:0/1350270673 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6e740a1220 0x7f6e740a36d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400b2e0 tx=0x7f6e84005fb0 comp rx=0 tx=0).stop 2023-07-14T18:14:14.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.492+0000 7f6e8f9ff700 1 -- 172.21.15.133:0/1350270673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e88100e20 msgr2=0x7f6e88072a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:14.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.492+0000 7f6e8f9ff700 1 --2- 172.21.15.133:0/1350270673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e88100e20 0x7f6e88072a30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6e7800eb30 tx=0x7f6e7800ee40 comp rx=0 tx=0).stop 2023-07-14T18:14:14.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.492+0000 7f6e8f9ff700 1 -- 172.21.15.133:0/1350270673 shutdown_connections 2023-07-14T18:14:14.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.492+0000 7f6e8f9ff700 1 --2- 172.21.15.133:0/1350270673 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6e740a1220 0x7f6e740a36d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:14.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.492+0000 7f6e8f9ff700 1 --2- 172.21.15.133:0/1350270673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e881017b0 0x7f6e88072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:14.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.492+0000 7f6e8f9ff700 1 --2- 172.21.15.133:0/1350270673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e880ffc30 0x7f6e880743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:14.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.492+0000 7f6e8f9ff700 1 --2- 172.21.15.133:0/1350270673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e88100e20 0x7f6e88072a30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:14.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.492+0000 7f6e8f9ff700 1 -- 172.21.15.133:0/1350270673 >> 172.21.15.133:0/1350270673 conn(0x7f6e880fb210 msgr2=0x7f6e880fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:14.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.492+0000 7f6e8f9ff700 1 -- 172.21.15.133:0/1350270673 shutdown_connections 2023-07-14T18:14:14.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.492+0000 7f6e8f9ff700 1 -- 172.21.15.133:0/1350270673 wait complete. 2023-07-14T18:14:14.555 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-14T18:14:14.556 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-14T18:14:14.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.736+0000 7f3e811ff700 1 -- 172.21.15.133:0/3909043782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e7c074680 msgr2=0x7f3e7c072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:14.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.736+0000 7f3e811ff700 1 --2- 172.21.15.133:0/3909043782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e7c074680 0x7f3e7c072ad0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c0098f0 tx=0x7f3e6c009c00 comp rx=0 tx=0).stop 2023-07-14T18:14:14.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.736+0000 7f3e811ff700 1 -- 172.21.15.133:0/3909043782 shutdown_connections 2023-07-14T18:14:14.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.736+0000 7f3e811ff700 1 --2- 172.21.15.133:0/3909043782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e7c107ad0 0x7f3e7c109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:14.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.736+0000 7f3e811ff700 1 --2- 172.21.15.133:0/3909043782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e7c074680 0x7f3e7c072ad0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:14.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.736+0000 7f3e811ff700 1 --2- 172.21.15.133:0/3909043782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e7c0730a0 0x7f3e7c073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:14.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.736+0000 7f3e811ff700 1 -- 172.21.15.133:0/3909043782 >> 172.21.15.133:0/3909043782 conn(0x7f3e7c0fb210 msgr2=0x7f3e7c0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:14.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.736+0000 7f3e811ff700 1 -- 172.21.15.133:0/3909043782 shutdown_connections 2023-07-14T18:14:14.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.736+0000 7f3e811ff700 1 -- 172.21.15.133:0/3909043782 wait complete. 2023-07-14T18:14:14.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.737+0000 7f3e811ff700 1 Processor -- start 2023-07-14T18:14:14.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.737+0000 7f3e811ff700 1 -- start start 2023-07-14T18:14:14.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.737+0000 7f3e811ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e7c0730a0 0x7f3e7c197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:14.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.737+0000 7f3e811ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e7c074680 0x7f3e7c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:14.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.737+0000 7f3e811ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e7c107ad0 0x7f3e7c19f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:14.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.737+0000 7f3e811ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c19f9d0 con 0x7f3e7c0730a0 2023-07-14T18:14:14.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.737+0000 7f3e811ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c19fb10 con 0x7f3e7c074680 2023-07-14T18:14:14.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.737+0000 7f3e811ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c19fca0 con 0x7f3e7c107ad0 2023-07-14T18:14:14.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.738+0000 7f3e7a59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e7c074680 0x7f3e7c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:14.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.738+0000 7f3e7ad9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e7c0730a0 0x7f3e7c197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:14.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.738+0000 7f3e7a59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e7c074680 0x7f3e7c198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:57674/0 (socket says 172.21.15.133:57674) 2023-07-14T18:14:14.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.738+0000 7f3e7b59e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e7c107ad0 0x7f3e7c19f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:14.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.738+0000 7f3e7a59c700 1 -- 172.21.15.133:0/2851339995 learned_addr learned my addr 172.21.15.133:0/2851339995 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:14.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.738+0000 7f3e7a59c700 1 -- 172.21.15.133:0/2851339995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e7c107ad0 msgr2=0x7f3e7c19f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:14.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.738+0000 7f3e7a59c700 1 --2- 172.21.15.133:0/2851339995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e7c107ad0 0x7f3e7c19f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:14.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.738+0000 7f3e7a59c700 1 -- 172.21.15.133:0/2851339995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e7c0730a0 msgr2=0x7f3e7c197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:14.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.738+0000 7f3e7a59c700 1 --2- 172.21.15.133:0/2851339995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e7c0730a0 0x7f3e7c197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:14.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.738+0000 7f3e7a59c700 1 -- 172.21.15.133:0/2851339995 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e74009c40 con 0x7f3e7c074680 2023-07-14T18:14:14.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.738+0000 7f3e7ad9d700 1 --2- 172.21.15.133:0/2851339995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e7c0730a0 0x7f3e7c197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:14:14.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.738+0000 7f3e7a59c700 1 --2- 172.21.15.133:0/2851339995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e7c074680 0x7f3e7c198390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000e9f0 tx=0x7f3e7000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:14.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.738+0000 7f3e63fff700 1 -- 172.21.15.133:0/2851339995 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e70004220 con 0x7f3e7c074680 2023-07-14T18:14:14.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.738+0000 7f3e63fff700 1 -- 172.21.15.133:0/2851339995 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e70004380 con 0x7f3e7c074680 2023-07-14T18:14:14.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.738+0000 7f3e63fff700 1 -- 172.21.15.133:0/2851339995 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e70003b40 con 0x7f3e7c074680 2023-07-14T18:14:14.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.738+0000 7f3e811ff700 1 -- 172.21.15.133:0/2851339995 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e6c009520 con 0x7f3e7c074680 2023-07-14T18:14:14.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.739+0000 7f3e811ff700 1 -- 172.21.15.133:0/2851339995 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e7c1a0270 con 0x7f3e7c074680 2023-07-14T18:14:14.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.740+0000 7f3e811ff700 1 -- 172.21.15.133:0/2851339995 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e7c10d640 con 0x7f3e7c074680 2023-07-14T18:14:14.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.741+0000 7f3e63fff700 1 -- 172.21.15.133:0/2851339995 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f3e70004870 con 0x7f3e7c074680 2023-07-14T18:14:14.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.743+0000 7f3e63fff700 1 --2- 172.21.15.133:0/2851339995 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3e640a11d0 0x7f3e640a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:14.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.743+0000 7f3e63fff700 1 -- 172.21.15.133:0/2851339995 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3e70014070 con 0x7f3e7c074680 2023-07-14T18:14:14.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.743+0000 7f3e7ad9d700 1 --2- 172.21.15.133:0/2851339995 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3e640a11d0 0x7f3e640a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:14.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.744+0000 7f3e7ad9d700 1 --2- 172.21.15.133:0/2851339995 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3e640a11d0 0x7f3e640a3680 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009fd0 tx=0x7f3e6c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:14.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.745+0000 7f3e63fff700 1 -- 172.21.15.133:0/2851339995 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f3e70096880 con 0x7f3e7c074680 2023-07-14T18:14:15.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:14.963+0000 7f3e811ff700 1 -- 172.21.15.133:0/2851339995 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3e7c10d7d0 con 0x7f3e7c074680 2023-07-14T18:14:15.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:15 smithi161 ceph-mon[128993]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:15.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:15 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:15.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:15 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:15.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:15 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3980193026' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:14:15.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:15 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1350270673' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-14T18:14:15.850 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:15 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:15.850 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:15 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:15.850 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:15 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:14:15.850 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:15 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:14:15.850 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:15 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:15.864 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:15 smithi171 ceph-mon[129241]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:15.864 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:15 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:15.865 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:15 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:15.865 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:15 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3980193026' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:14:15.865 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:15 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1350270673' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-14T18:14:15.865 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:15 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:15.865 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:15 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:15.865 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:15 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:14:15.865 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:15 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:14:15.866 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:15 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:15.905 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:15 smithi133 ceph-mon[126384]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:15.905 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:15 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:15.905 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:15 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:15.905 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:15 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3980193026' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:14:15.905 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:15 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1350270673' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-14T18:14:15.906 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:15 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:15.906 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:15 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:15.906 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:15 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:14:15.906 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:15 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:14:15.906 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:15 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:15.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:15.906+0000 7f3e63fff700 1 -- 172.21.15.133:0/2851339995 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3e70099ea0 con 0x7f3e7c074680 2023-07-14T18:14:15.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:15.912+0000 7f3e811ff700 1 -- 172.21.15.133:0/2851339995 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3e640a11d0 msgr2=0x7f3e640a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:15.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:15.912+0000 7f3e811ff700 1 --2- 172.21.15.133:0/2851339995 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3e640a11d0 0x7f3e640a3680 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009fd0 tx=0x7f3e6c019040 comp rx=0 tx=0).stop 2023-07-14T18:14:15.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:15.912+0000 7f3e811ff700 1 -- 172.21.15.133:0/2851339995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e7c074680 msgr2=0x7f3e7c198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:15.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:15.912+0000 7f3e811ff700 1 --2- 172.21.15.133:0/2851339995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e7c074680 0x7f3e7c198390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000e9f0 tx=0x7f3e7000ed00 comp rx=0 tx=0).stop 2023-07-14T18:14:15.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:15.912+0000 7f3e811ff700 1 -- 172.21.15.133:0/2851339995 shutdown_connections 2023-07-14T18:14:15.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:15.912+0000 7f3e811ff700 1 --2- 172.21.15.133:0/2851339995 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3e640a11d0 0x7f3e640a3680 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:15.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:15.912+0000 7f3e811ff700 1 --2- 172.21.15.133:0/2851339995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e7c107ad0 0x7f3e7c19f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:15.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:15.912+0000 7f3e811ff700 1 --2- 172.21.15.133:0/2851339995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e7c0730a0 0x7f3e7c197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:15.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:15.912+0000 7f3e811ff700 1 --2- 172.21.15.133:0/2851339995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e7c074680 0x7f3e7c198390 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:15.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:15.912+0000 7f3e811ff700 1 -- 172.21.15.133:0/2851339995 >> 172.21.15.133:0/2851339995 conn(0x7f3e7c0fb210 msgr2=0x7f3e7c0fbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:15.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:15.912+0000 7f3e811ff700 1 -- 172.21.15.133:0/2851339995 shutdown_connections 2023-07-14T18:14:15.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:15.913+0000 7f3e811ff700 1 -- 172.21.15.133:0/2851339995 wait complete. 2023-07-14T18:14:15.958 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' created 2023-07-14T18:14:15.976 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-07-14T18:14:16.187 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.144+0000 7fba9f9ff700 1 -- 172.21.15.133:0/3469260665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba980730a0 msgr2=0x7fba98073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:16.188 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.144+0000 7fba9f9ff700 1 --2- 172.21.15.133:0/3469260665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba980730a0 0x7fba98073510 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fba88009990 tx=0x7fba88009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:16.188 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.144+0000 7fba9f9ff700 1 -- 172.21.15.133:0/3469260665 shutdown_connections 2023-07-14T18:14:16.188 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.144+0000 7fba9f9ff700 1 --2- 172.21.15.133:0/3469260665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba9810c1e0 0x7fba9810e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:16.188 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.144+0000 7fba9f9ff700 1 --2- 172.21.15.133:0/3469260665 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba98074680 0x7fba98072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:16.188 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.144+0000 7fba9f9ff700 1 --2- 172.21.15.133:0/3469260665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba980730a0 0x7fba98073510 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:16.188 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.144+0000 7fba9f9ff700 1 -- 172.21.15.133:0/3469260665 >> 172.21.15.133:0/3469260665 conn(0x7fba980fb210 msgr2=0x7fba980fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:16.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.145+0000 7fba9f9ff700 1 -- 172.21.15.133:0/3469260665 shutdown_connections 2023-07-14T18:14:16.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.145+0000 7fba9f9ff700 1 -- 172.21.15.133:0/3469260665 wait complete. 2023-07-14T18:14:16.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.145+0000 7fba9f9ff700 1 Processor -- start 2023-07-14T18:14:16.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.146+0000 7fba9f9ff700 1 -- start start 2023-07-14T18:14:16.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.146+0000 7fba9f9ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba980730a0 0x7fba9819c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:16.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.146+0000 7fba9f9ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba98074680 0x7fba9819c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:16.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.146+0000 7fba9f9ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba9810c1e0 0x7fba981a35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:16.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.146+0000 7fba9f9ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba981a3b80 con 0x7fba9810c1e0 2023-07-14T18:14:16.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.146+0000 7fba9f9ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba981a3cc0 con 0x7fba980730a0 2023-07-14T18:14:16.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.146+0000 7fba9f9ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba981a3e50 con 0x7fba98074680 2023-07-14T18:14:16.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.146+0000 7fba9d79b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba980730a0 0x7fba9819c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:16.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.146+0000 7fba9d79b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba980730a0 0x7fba9819c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:57698/0 (socket says 172.21.15.133:57698) 2023-07-14T18:14:16.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.146+0000 7fba9d79b700 1 -- 172.21.15.133:0/2671945742 learned_addr learned my addr 172.21.15.133:0/2671945742 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:16.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.146+0000 7fba9cf9a700 1 --2- 172.21.15.133:0/2671945742 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba98074680 0x7fba9819c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:16.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.146+0000 7fba9df9c700 1 --2- 172.21.15.133:0/2671945742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba9810c1e0 0x7fba981a35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:16.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.147+0000 7fba9cf9a700 1 -- 172.21.15.133:0/2671945742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba980730a0 msgr2=0x7fba9819c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:16.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.147+0000 7fba9cf9a700 1 --2- 172.21.15.133:0/2671945742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba980730a0 0x7fba9819c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:16.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.147+0000 7fba9cf9a700 1 -- 172.21.15.133:0/2671945742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba9810c1e0 msgr2=0x7fba981a35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:16.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.147+0000 7fba9cf9a700 1 --2- 172.21.15.133:0/2671945742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba9810c1e0 0x7fba981a35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:16.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.147+0000 7fba9cf9a700 1 -- 172.21.15.133:0/2671945742 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba880095f0 con 0x7fba98074680 2023-07-14T18:14:16.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.147+0000 7fba9cf9a700 1 --2- 172.21.15.133:0/2671945742 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba98074680 0x7fba9819c540 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fba8800b680 tx=0x7fba88009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:16.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.147+0000 7fba8e7fc700 1 -- 172.21.15.133:0/2671945742 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8801d070 con 0x7fba98074680 2023-07-14T18:14:16.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.147+0000 7fba8e7fc700 1 -- 172.21.15.133:0/2671945742 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba88021410 con 0x7fba98074680 2023-07-14T18:14:16.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.147+0000 7fba9f9ff700 1 -- 172.21.15.133:0/2671945742 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba981a40d0 con 0x7fba98074680 2023-07-14T18:14:16.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.147+0000 7fba8e7fc700 1 -- 172.21.15.133:0/2671945742 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba88017650 con 0x7fba98074680 2023-07-14T18:14:16.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.147+0000 7fba9f9ff700 1 -- 172.21.15.133:0/2671945742 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba981a44f0 con 0x7fba98074680 2023-07-14T18:14:16.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.148+0000 7fba9f9ff700 1 -- 172.21.15.133:0/2671945742 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba98101d30 con 0x7fba98074680 2023-07-14T18:14:16.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.150+0000 7fba8e7fc700 1 -- 172.21.15.133:0/2671945742 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fba88021810 con 0x7fba98074680 2023-07-14T18:14:16.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.151+0000 7fba8e7fc700 1 --2- 172.21.15.133:0/2671945742 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fba840a1150 0x7fba840a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:16.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.152+0000 7fba8e7fc700 1 -- 172.21.15.133:0/2671945742 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fba880cd9f0 con 0x7fba98074680 2023-07-14T18:14:16.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.152+0000 7fba9d79b700 1 --2- 172.21.15.133:0/2671945742 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fba840a1150 0x7fba840a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:16.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.152+0000 7fba9d79b700 1 --2- 172.21.15.133:0/2671945742 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fba840a1150 0x7fba840a3600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fba94005950 tx=0x7fba94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:16.198 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.155+0000 7fba8e7fc700 1 -- 172.21.15.133:0/2671945742 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fba88097460 con 0x7fba98074680 2023-07-14T18:14:16.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.386+0000 7fba9f9ff700 1 -- 172.21.15.133:0/2671945742 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fba9802c8e0 con 0x7fba98074680 2023-07-14T18:14:16.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.913+0000 7fba8e7fc700 1 -- 172.21.15.133:0/2671945742 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v82) v1 ==== 171+0+0 (secure 0 0 0) 0x7fba8809aa80 con 0x7fba98074680 2023-07-14T18:14:16.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.919+0000 7fba9f9ff700 1 -- 172.21.15.133:0/2671945742 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fba840a1150 msgr2=0x7fba840a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:16.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.919+0000 7fba9f9ff700 1 --2- 172.21.15.133:0/2671945742 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fba840a1150 0x7fba840a3600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fba94005950 tx=0x7fba94009260 comp rx=0 tx=0).stop 2023-07-14T18:14:16.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.920+0000 7fba9f9ff700 1 -- 172.21.15.133:0/2671945742 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba98074680 msgr2=0x7fba9819c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:16.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.920+0000 7fba9f9ff700 1 --2- 172.21.15.133:0/2671945742 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba98074680 0x7fba9819c540 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fba8800b680 tx=0x7fba88009580 comp rx=0 tx=0).stop 2023-07-14T18:14:16.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.920+0000 7fba9f9ff700 1 -- 172.21.15.133:0/2671945742 shutdown_connections 2023-07-14T18:14:16.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.920+0000 7fba9f9ff700 1 --2- 172.21.15.133:0/2671945742 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fba840a1150 0x7fba840a3600 secure :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fba94005950 tx=0x7fba94009260 comp rx=0 tx=0).stop 2023-07-14T18:14:16.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.920+0000 7fba9f9ff700 1 --2- 172.21.15.133:0/2671945742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba980730a0 0x7fba9819c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:16.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.920+0000 7fba9f9ff700 1 --2- 172.21.15.133:0/2671945742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba9810c1e0 0x7fba981a35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:16.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.920+0000 7fba9f9ff700 1 --2- 172.21.15.133:0/2671945742 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba98074680 0x7fba9819c540 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:16.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.920+0000 7fba9f9ff700 1 -- 172.21.15.133:0/2671945742 >> 172.21.15.133:0/2671945742 conn(0x7fba980fb210 msgr2=0x7fba980fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:16.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.925+0000 7fba9f9ff700 1 -- 172.21.15.133:0/2671945742 shutdown_connections 2023-07-14T18:14:16.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:16.928+0000 7fba9f9ff700 1 -- 172.21.15.133:0/2671945742 wait complete. 2023-07-14T18:14:16.977 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' created 2023-07-14T18:14:16.996 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-14T18:14:17.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.168+0000 7f6059dff700 1 -- 172.21.15.133:0/971154782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6054074680 msgr2=0x7f6054072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:17.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.168+0000 7f6059dff700 1 --2- 172.21.15.133:0/971154782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6054074680 0x7f6054072ad0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6044009990 tx=0x7f6044009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:17.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.168+0000 7f6059dff700 1 -- 172.21.15.133:0/971154782 shutdown_connections 2023-07-14T18:14:17.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.168+0000 7f6059dff700 1 --2- 172.21.15.133:0/971154782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f605410c1e0 0x7f605410e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:17.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.168+0000 7f6059dff700 1 --2- 172.21.15.133:0/971154782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6054074680 0x7f6054072ad0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:17.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.168+0000 7f6059dff700 1 --2- 172.21.15.133:0/971154782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60540730a0 0x7f6054073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:17.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.168+0000 7f6059dff700 1 -- 172.21.15.133:0/971154782 >> 172.21.15.133:0/971154782 conn(0x7f60540fb210 msgr2=0x7f60540fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:17.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.168+0000 7f6059dff700 1 -- 172.21.15.133:0/971154782 shutdown_connections 2023-07-14T18:14:17.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.168+0000 7f6059dff700 1 -- 172.21.15.133:0/971154782 wait complete. 2023-07-14T18:14:17.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.170+0000 7f6059dff700 1 Processor -- start 2023-07-14T18:14:17.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.170+0000 7f6059dff700 1 -- start start 2023-07-14T18:14:17.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.170+0000 7f6059dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60540730a0 0x7f605419c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:17.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.170+0000 7f6059dff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6054074680 0x7f605419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:17.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.170+0000 7f6059dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f605410c1e0 0x7f60541a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:17.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.170+0000 7f6059dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60541a3b90 con 0x7f60540730a0 2023-07-14T18:14:17.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.170+0000 7f6059dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60541a3cd0 con 0x7f605410c1e0 2023-07-14T18:14:17.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.170+0000 7f6059dff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60541a3e60 con 0x7f6054074680 2023-07-14T18:14:17.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.171+0000 7f6052ffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6054074680 0x7f605419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:17.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.171+0000 7f6052ffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6054074680 0x7f605419c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:43964/0 (socket says 172.21.15.133:43964) 2023-07-14T18:14:17.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.171+0000 7f6052ffd700 1 -- 172.21.15.133:0/2100421707 learned_addr learned my addr 172.21.15.133:0/2100421707 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:17.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.171+0000 7f6053fff700 1 --2- 172.21.15.133:0/2100421707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f605410c1e0 0x7f60541a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:17.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.171+0000 7f60537fe700 1 --2- 172.21.15.133:0/2100421707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60540730a0 0x7f605419c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:17.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.171+0000 7f6052ffd700 1 -- 172.21.15.133:0/2100421707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f605410c1e0 msgr2=0x7f60541a35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:17.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.171+0000 7f6052ffd700 1 --2- 172.21.15.133:0/2100421707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f605410c1e0 0x7f60541a35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:17.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.171+0000 7f6052ffd700 1 -- 172.21.15.133:0/2100421707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60540730a0 msgr2=0x7f605419c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:17.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.171+0000 7f6052ffd700 1 --2- 172.21.15.133:0/2100421707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60540730a0 0x7f605419c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:17.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.171+0000 7f6052ffd700 1 -- 172.21.15.133:0/2100421707 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60440095f0 con 0x7f6054074680 2023-07-14T18:14:17.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.171+0000 7f6052ffd700 1 --2- 172.21.15.133:0/2100421707 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6054074680 0x7f605419c550 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f604800e970 tx=0x7f604800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:17.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.172+0000 7f6050ff9700 1 -- 172.21.15.133:0/2100421707 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6048009790 con 0x7f6054074680 2023-07-14T18:14:17.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.172+0000 7f6050ff9700 1 -- 172.21.15.133:0/2100421707 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f604800c7e0 con 0x7f6054074680 2023-07-14T18:14:17.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.172+0000 7f6059dff700 1 -- 172.21.15.133:0/2100421707 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60541a4140 con 0x7f6054074680 2023-07-14T18:14:17.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.172+0000 7f6050ff9700 1 -- 172.21.15.133:0/2100421707 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6048003c70 con 0x7f6054074680 2023-07-14T18:14:17.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.172+0000 7f6059dff700 1 -- 172.21.15.133:0/2100421707 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60541a46a0 con 0x7f6054074680 2023-07-14T18:14:17.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.173+0000 7f6059dff700 1 -- 172.21.15.133:0/2100421707 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6054101d30 con 0x7f6054074680 2023-07-14T18:14:17.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.175+0000 7f6050ff9700 1 -- 172.21.15.133:0/2100421707 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f604800cc50 con 0x7f6054074680 2023-07-14T18:14:17.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.176+0000 7f6050ff9700 1 --2- 172.21.15.133:0/2100421707 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f603c0a11d0 0x7f603c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:17.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.176+0000 7f6050ff9700 1 -- 172.21.15.133:0/2100421707 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f6048014070 con 0x7f6054074680 2023-07-14T18:14:17.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.176+0000 7f60537fe700 1 --2- 172.21.15.133:0/2100421707 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f603c0a11d0 0x7f603c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:17.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.177+0000 7f60537fe700 1 --2- 172.21.15.133:0/2100421707 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f603c0a11d0 0x7f603c0a3680 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f604400b2e0 tx=0x7f6044005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:17.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.178+0000 7f6050ff9700 1 -- 172.21.15.133:0/2100421707 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f6048096780 con 0x7f6054074680 2023-07-14T18:14:17.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:16 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2851339995' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-14T18:14:17.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:16 smithi161 ceph-mon[128993]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-14T18:14:17.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:16 smithi161 ceph-mon[128993]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:17.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:16 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:14:17.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:16 smithi161 ceph-mon[128993]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-14T18:14:17.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:16 smithi161 ceph-mon[128993]: osdmap e81: 12 total, 12 up, 12 in 2023-07-14T18:14:17.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:16 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2671945742' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-14T18:14:17.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:16 smithi161 ceph-mon[128993]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-14T18:14:17.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:16 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2851339995' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-14T18:14:17.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:16 smithi171 ceph-mon[129241]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-14T18:14:17.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:16 smithi171 ceph-mon[129241]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:17.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:16 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:14:17.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:16 smithi171 ceph-mon[129241]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-14T18:14:17.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:16 smithi171 ceph-mon[129241]: osdmap e81: 12 total, 12 up, 12 in 2023-07-14T18:14:17.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:16 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2671945742' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-14T18:14:17.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:16 smithi171 ceph-mon[129241]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-14T18:14:17.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:16 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2851339995' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-14T18:14:17.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:16 smithi133 ceph-mon[126384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-14T18:14:17.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:16 smithi133 ceph-mon[126384]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:17.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:16 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:14:17.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:16 smithi133 ceph-mon[126384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-14T18:14:17.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:16 smithi133 ceph-mon[126384]: osdmap e81: 12 total, 12 up, 12 in 2023-07-14T18:14:17.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:16 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2671945742' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-14T18:14:17.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:16 smithi133 ceph-mon[126384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-14T18:14:17.481 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.438+0000 7f6059dff700 1 -- 172.21.15.133:0/2100421707 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f605402cd60 con 0x7f6054074680 2023-07-14T18:14:17.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.935+0000 7f6050ff9700 1 -- 172.21.15.133:0/2100421707 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f605402cd60 con 0x7f6054074680 2023-07-14T18:14:17.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.943+0000 7f6059dff700 1 -- 172.21.15.133:0/2100421707 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f603c0a11d0 msgr2=0x7f603c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:17.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.943+0000 7f6059dff700 1 --2- 172.21.15.133:0/2100421707 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f603c0a11d0 0x7f603c0a3680 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f604400b2e0 tx=0x7f6044005fb0 comp rx=0 tx=0).stop 2023-07-14T18:14:17.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.943+0000 7f6059dff700 1 -- 172.21.15.133:0/2100421707 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6054074680 msgr2=0x7f605419c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:17.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.943+0000 7f6059dff700 1 --2- 172.21.15.133:0/2100421707 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6054074680 0x7f605419c550 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f604800e970 tx=0x7f604800ec80 comp rx=0 tx=0).stop 2023-07-14T18:14:17.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.943+0000 7f6059dff700 1 -- 172.21.15.133:0/2100421707 shutdown_connections 2023-07-14T18:14:17.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.943+0000 7f6059dff700 1 --2- 172.21.15.133:0/2100421707 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f603c0a11d0 0x7f603c0a3680 secure :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f604400b2e0 tx=0x7f6044005fb0 comp rx=0 tx=0).stop 2023-07-14T18:14:17.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.943+0000 7f6059dff700 1 --2- 172.21.15.133:0/2100421707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f605410c1e0 0x7f60541a35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:17.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.943+0000 7f6059dff700 1 --2- 172.21.15.133:0/2100421707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60540730a0 0x7f605419c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:17.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.943+0000 7f6059dff700 1 --2- 172.21.15.133:0/2100421707 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6054074680 0x7f605419c550 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f604800e970 tx=0x7f604800ec80 comp rx=0 tx=0).stop 2023-07-14T18:14:17.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.943+0000 7f6059dff700 1 -- 172.21.15.133:0/2100421707 >> 172.21.15.133:0/2100421707 conn(0x7f60540fb210 msgr2=0x7f60540fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:17.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.944+0000 7f6059dff700 1 -- 172.21.15.133:0/2100421707 shutdown_connections 2023-07-14T18:14:17.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:17.944+0000 7f6059dff700 1 -- 172.21.15.133:0/2100421707 wait complete. 2023-07-14T18:14:17.988 INFO:teuthology.orchestra.run.smithi133.stderr:new fs with metadata pool 2 and data pool 3 2023-07-14T18:14:18.011 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-07-14T18:14:18.011 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-07-14T18:14:18.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.182+0000 7f62bdfff700 1 -- 172.21.15.133:0/3041981077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62b8074680 msgr2=0x7f62b8072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:18.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.182+0000 7f62bdfff700 1 --2- 172.21.15.133:0/3041981077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62b8074680 0x7f62b8072ad0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009990 tx=0x7f62a8009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:18.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.182+0000 7f62bdfff700 1 -- 172.21.15.133:0/3041981077 shutdown_connections 2023-07-14T18:14:18.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.182+0000 7f62bdfff700 1 --2- 172.21.15.133:0/3041981077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62b8107ae0 0x7f62b8109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:18.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.182+0000 7f62bdfff700 1 --2- 172.21.15.133:0/3041981077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62b8074680 0x7f62b8072ad0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:18.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.182+0000 7f62bdfff700 1 --2- 172.21.15.133:0/3041981077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62b80730a0 0x7f62b8073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:18.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.182+0000 7f62bdfff700 1 -- 172.21.15.133:0/3041981077 >> 172.21.15.133:0/3041981077 conn(0x7f62b80fb260 msgr2=0x7f62b80fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:18.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.182+0000 7f62bdfff700 1 -- 172.21.15.133:0/3041981077 shutdown_connections 2023-07-14T18:14:18.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.182+0000 7f62bdfff700 1 -- 172.21.15.133:0/3041981077 wait complete. 2023-07-14T18:14:18.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.184+0000 7f62bdfff700 1 Processor -- start 2023-07-14T18:14:18.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.184+0000 7f62bdfff700 1 -- start start 2023-07-14T18:14:18.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.185+0000 7f62bdfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62b80730a0 0x7f62b8197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:18.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.185+0000 7f62bdfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62b8074680 0x7f62b8198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:18.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.185+0000 7f62bdfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62b8107ae0 0x7f62b819f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:18.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.185+0000 7f62bdfff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b819f940 con 0x7f62b8107ae0 2023-07-14T18:14:18.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.185+0000 7f62bdfff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b819fa80 con 0x7f62b80730a0 2023-07-14T18:14:18.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.185+0000 7f62bdfff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b819fc10 con 0x7f62b8074680 2023-07-14T18:14:18.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.185+0000 7f62b6ffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62b8074680 0x7f62b8198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:18.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.185+0000 7f62b6ffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62b8074680 0x7f62b8198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:43982/0 (socket says 172.21.15.133:43982) 2023-07-14T18:14:18.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.185+0000 7f62b6ffd700 1 -- 172.21.15.133:0/615401246 learned_addr learned my addr 172.21.15.133:0/615401246 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:18.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.185+0000 7f62b77fe700 1 --2- 172.21.15.133:0/615401246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62b80730a0 0x7f62b8197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:18.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.185+0000 7f62b7fff700 1 --2- 172.21.15.133:0/615401246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62b8107ae0 0x7f62b819f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:18.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.185+0000 7f62b6ffd700 1 -- 172.21.15.133:0/615401246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62b80730a0 msgr2=0x7f62b8197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:18.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.185+0000 7f62b6ffd700 1 --2- 172.21.15.133:0/615401246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62b80730a0 0x7f62b8197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:18.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.185+0000 7f62b6ffd700 1 -- 172.21.15.133:0/615401246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62b8107ae0 msgr2=0x7f62b819f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:18.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.185+0000 7f62b6ffd700 1 --2- 172.21.15.133:0/615401246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62b8107ae0 0x7f62b819f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:18.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.185+0000 7f62b6ffd700 1 -- 172.21.15.133:0/615401246 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a80095f0 con 0x7f62b8074680 2023-07-14T18:14:18.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.186+0000 7f62b6ffd700 1 --2- 172.21.15.133:0/615401246 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62b8074680 0x7f62b8198300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00e970 tx=0x7f62ac00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:18.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.186+0000 7f62b4ff9700 1 -- 172.21.15.133:0/615401246 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac009790 con 0x7f62b8074680 2023-07-14T18:14:18.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.186+0000 7f62b4ff9700 1 -- 172.21.15.133:0/615401246 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62ac00c7e0 con 0x7f62b8074680 2023-07-14T18:14:18.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.186+0000 7f62bdfff700 1 -- 172.21.15.133:0/615401246 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b819fef0 con 0x7f62b8074680 2023-07-14T18:14:18.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.186+0000 7f62b4ff9700 1 -- 172.21.15.133:0/615401246 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac003c70 con 0x7f62b8074680 2023-07-14T18:14:18.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.186+0000 7f62bdfff700 1 -- 172.21.15.133:0/615401246 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b81a03c0 con 0x7f62b8074680 2023-07-14T18:14:18.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.187+0000 7f62bdfff700 1 -- 172.21.15.133:0/615401246 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b810d650 con 0x7f62b8074680 2023-07-14T18:14:18.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.189+0000 7f62b4ff9700 1 -- 172.21.15.133:0/615401246 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f62ac004220 con 0x7f62b8074680 2023-07-14T18:14:18.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.190+0000 7f62b4ff9700 1 --2- 172.21.15.133:0/615401246 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f62a00a11d0 0x7f62a00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:18.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.190+0000 7f62b4ff9700 1 -- 172.21.15.133:0/615401246 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f62ac014070 con 0x7f62b8074680 2023-07-14T18:14:18.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.190+0000 7f62b77fe700 1 --2- 172.21.15.133:0/615401246 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f62a00a11d0 0x7f62a00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:18.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.191+0000 7f62b77fe700 1 --2- 172.21.15.133:0/615401246 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f62a00a11d0 0x7f62a00a3680 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f62a800b2e0 tx=0x7f62a8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:18.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.193+0000 7f62b4ff9700 1 -- 172.21.15.133:0/615401246 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f62ac096160 con 0x7f62b8074680 2023-07-14T18:14:18.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:17 smithi161 ceph-mon[128993]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-07-14T18:14:18.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:17 smithi161 ceph-mon[128993]: osdmap e82: 12 total, 12 up, 12 in 2023-07-14T18:14:18.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:17 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2100421707' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-14T18:14:18.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:17 smithi161 ceph-mon[128993]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-14T18:14:18.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:17 smithi161 ceph-mon[128993]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-14T18:14:18.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:17 smithi171 ceph-mon[129241]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-07-14T18:14:18.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:17 smithi171 ceph-mon[129241]: osdmap e82: 12 total, 12 up, 12 in 2023-07-14T18:14:18.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:17 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2100421707' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-14T18:14:18.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:17 smithi171 ceph-mon[129241]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-14T18:14:18.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:17 smithi171 ceph-mon[129241]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-14T18:14:18.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:17 smithi133 ceph-mon[126384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-07-14T18:14:18.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:17 smithi133 ceph-mon[126384]: osdmap e82: 12 total, 12 up, 12 in 2023-07-14T18:14:18.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:17 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2100421707' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-14T18:14:18.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:17 smithi133 ceph-mon[126384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-14T18:14:18.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:17 smithi133 ceph-mon[126384]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-14T18:14:18.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.418+0000 7f62bdfff700 1 -- 172.21.15.133:0/615401246 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f62b81a0d10 con 0x7f62b8074680 2023-07-14T18:14:18.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.911+0000 7f62b4ff9700 1 -- 172.21.15.133:0/615401246 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v84) v1 ==== 157+0+0 (secure 0 0 0) 0x7f62ac099780 con 0x7f62b8074680 2023-07-14T18:14:18.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.913+0000 7f62bdfff700 1 -- 172.21.15.133:0/615401246 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f62a00a11d0 msgr2=0x7f62a00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:18.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.913+0000 7f62bdfff700 1 --2- 172.21.15.133:0/615401246 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f62a00a11d0 0x7f62a00a3680 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f62a800b2e0 tx=0x7f62a8005fb0 comp rx=0 tx=0).stop 2023-07-14T18:14:18.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.913+0000 7f62bdfff700 1 -- 172.21.15.133:0/615401246 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62b8074680 msgr2=0x7f62b8198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:18.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.913+0000 7f62bdfff700 1 --2- 172.21.15.133:0/615401246 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62b8074680 0x7f62b8198300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00e970 tx=0x7f62ac00ec80 comp rx=0 tx=0).stop 2023-07-14T18:14:18.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.914+0000 7f62bdfff700 1 -- 172.21.15.133:0/615401246 shutdown_connections 2023-07-14T18:14:18.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.914+0000 7f62bdfff700 1 --2- 172.21.15.133:0/615401246 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f62a00a11d0 0x7f62a00a3680 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:18.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.914+0000 7f62bdfff700 1 --2- 172.21.15.133:0/615401246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62b80730a0 0x7f62b8197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:18.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.914+0000 7f62bdfff700 1 --2- 172.21.15.133:0/615401246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62b8107ae0 0x7f62b819f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:18.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.914+0000 7f62bdfff700 1 --2- 172.21.15.133:0/615401246 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62b8074680 0x7f62b8198300 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:18.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.914+0000 7f62bdfff700 1 -- 172.21.15.133:0/615401246 >> 172.21.15.133:0/615401246 conn(0x7f62b80fb260 msgr2=0x7f62b8109e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:18.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.914+0000 7f62bdfff700 1 -- 172.21.15.133:0/615401246 shutdown_connections 2023-07-14T18:14:18.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:18.914+0000 7f62bdfff700 1 -- 172.21.15.133:0/615401246 wait complete. 2023-07-14T18:14:18.976 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-07-14T18:14:19.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.153+0000 7f499abff700 1 -- 172.21.15.133:0/294698588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49940ffc20 msgr2=0x7f4994100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:19.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.153+0000 7f499abff700 1 --2- 172.21.15.133:0/294698588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49940ffc20 0x7f4994100030 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f499000b0a0 tx=0x7f499000b3b0 comp rx=0 tx=0).stop 2023-07-14T18:14:19.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.153+0000 7f499abff700 1 -- 172.21.15.133:0/294698588 shutdown_connections 2023-07-14T18:14:19.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.153+0000 7f499abff700 1 --2- 172.21.15.133:0/294698588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49941017a0 0x7f4994108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:19.197 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.153+0000 7f499abff700 1 --2- 172.21.15.133:0/294698588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49940ffc20 0x7f4994100030 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:19.197 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.153+0000 7f499abff700 1 --2- 172.21.15.133:0/294698588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4994100e10 0x7f4994101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:19.197 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.153+0000 7f499abff700 1 -- 172.21.15.133:0/294698588 >> 172.21.15.133:0/294698588 conn(0x7f49940fb200 msgr2=0x7f49940fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:19.197 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.153+0000 7f499abff700 1 -- 172.21.15.133:0/294698588 shutdown_connections 2023-07-14T18:14:19.197 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.153+0000 7f499abff700 1 -- 172.21.15.133:0/294698588 wait complete. 2023-07-14T18:14:19.198 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.154+0000 7f499abff700 1 Processor -- start 2023-07-14T18:14:19.199 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.154+0000 7f499abff700 1 -- start start 2023-07-14T18:14:19.199 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.154+0000 7f499abff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49940ffc20 0x7f4994195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:19.199 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.154+0000 7f499abff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4994100e10 0x7f4994195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:19.199 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.154+0000 7f499abff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49941017a0 0x7f499419cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:19.199 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.154+0000 7f499abff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499419d5c0 con 0x7f4994100e10 2023-07-14T18:14:19.199 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.154+0000 7f499abff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499419d700 con 0x7f49940ffc20 2023-07-14T18:14:19.200 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.154+0000 7f499abff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499419d890 con 0x7f49941017a0 2023-07-14T18:14:19.200 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.154+0000 7f498bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4994100e10 0x7f4994195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:19.200 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.154+0000 7f499899b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49940ffc20 0x7f4994195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:19.200 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.155+0000 7f499919c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49941017a0 0x7f499419cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:19.200 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.155+0000 7f499899b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49940ffc20 0x7f4994195a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:34598/0 (socket says 172.21.15.133:34598) 2023-07-14T18:14:19.200 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.155+0000 7f499899b700 1 -- 172.21.15.133:0/1724691163 learned_addr learned my addr 172.21.15.133:0/1724691163 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:19.201 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.155+0000 7f499899b700 1 -- 172.21.15.133:0/1724691163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49941017a0 msgr2=0x7f499419cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:19.201 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.155+0000 7f499899b700 1 --2- 172.21.15.133:0/1724691163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49941017a0 0x7f499419cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:19.201 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.155+0000 7f499899b700 1 -- 172.21.15.133:0/1724691163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4994100e10 msgr2=0x7f4994195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:19.201 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.155+0000 7f499899b700 1 --2- 172.21.15.133:0/1724691163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4994100e10 0x7f4994195f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:19.201 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.155+0000 7f499899b700 1 -- 172.21.15.133:0/1724691163 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4990009d00 con 0x7f49940ffc20 2023-07-14T18:14:19.201 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.155+0000 7f499899b700 1 --2- 172.21.15.133:0/1724691163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49940ffc20 0x7f4994195a40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f499000b070 tx=0x7f499000ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:19.201 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.155+0000 7f4989ffb700 1 -- 172.21.15.133:0/1724691163 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499000e040 con 0x7f49940ffc20 2023-07-14T18:14:19.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.155+0000 7f499abff700 1 -- 172.21.15.133:0/1724691163 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499419db10 con 0x7f49940ffc20 2023-07-14T18:14:19.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.155+0000 7f4989ffb700 1 -- 172.21.15.133:0/1724691163 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4990003c00 con 0x7f49940ffc20 2023-07-14T18:14:19.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.155+0000 7f4989ffb700 1 -- 172.21.15.133:0/1724691163 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49900052a0 con 0x7f49940ffc20 2023-07-14T18:14:19.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.155+0000 7f499abff700 1 -- 172.21.15.133:0/1724691163 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499419df90 con 0x7f49940ffc20 2023-07-14T18:14:19.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.156+0000 7f499abff700 1 -- 172.21.15.133:0/1724691163 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499410b440 con 0x7f49940ffc20 2023-07-14T18:14:19.203 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.159+0000 7f4989ffb700 1 -- 172.21.15.133:0/1724691163 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f499001d480 con 0x7f49940ffc20 2023-07-14T18:14:19.203 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.160+0000 7f4989ffb700 1 --2- 172.21.15.133:0/1724691163 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f497c0a11f0 0x7f497c0a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:19.203 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.160+0000 7f4989ffb700 1 -- 172.21.15.133:0/1724691163 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8205+0+0 (secure 0 0 0) 0x7f49900cd9d0 con 0x7f49940ffc20 2023-07-14T18:14:19.204 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.160+0000 7f498bfff700 1 --2- 172.21.15.133:0/1724691163 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f497c0a11f0 0x7f497c0a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:19.204 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.161+0000 7f498bfff700 1 --2- 172.21.15.133:0/1724691163 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f497c0a11f0 0x7f497c0a36a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4984009c60 tx=0x7f4984009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:19.205 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.162+0000 7f4989ffb700 1 -- 172.21.15.133:0/1724691163 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f4990097170 con 0x7f49940ffc20 2023-07-14T18:14:19.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:18 smithi161 ceph-mon[128993]: pgmap v267: 97 pgs: 15 creating+peering, 10 active+clean, 72 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:19.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:18 smithi161 ceph-mon[128993]: osdmap e83: 12 total, 12 up, 12 in 2023-07-14T18:14:19.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:18 smithi161 ceph-mon[128993]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-14T18:14:19.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:18 smithi161 ceph-mon[128993]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-14T18:14:19.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:18 smithi161 ceph-mon[128993]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-14T18:14:19.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:18 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:19.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:18 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:19.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:18 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:19.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:18 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:19.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:18 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/615401246' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-07-14T18:14:19.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:18 smithi161 ceph-mon[128993]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-07-14T18:14:19.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:18 smithi161 ceph-mon[128993]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-07-14T18:14:19.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:18 smithi161 ceph-mon[128993]: osdmap e84: 12 total, 12 up, 12 in 2023-07-14T18:14:19.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:18 smithi171 ceph-mon[129241]: pgmap v267: 97 pgs: 15 creating+peering, 10 active+clean, 72 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:19.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:18 smithi171 ceph-mon[129241]: osdmap e83: 12 total, 12 up, 12 in 2023-07-14T18:14:19.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:18 smithi171 ceph-mon[129241]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-14T18:14:19.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:18 smithi171 ceph-mon[129241]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-14T18:14:19.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:18 smithi171 ceph-mon[129241]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-14T18:14:19.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:18 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:19.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:18 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:19.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:18 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:19.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:18 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:19.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:18 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/615401246' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-07-14T18:14:19.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:18 smithi171 ceph-mon[129241]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-07-14T18:14:19.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:18 smithi171 ceph-mon[129241]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-07-14T18:14:19.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:18 smithi171 ceph-mon[129241]: osdmap e84: 12 total, 12 up, 12 in 2023-07-14T18:14:19.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:18 smithi133 ceph-mon[126384]: pgmap v267: 97 pgs: 15 creating+peering, 10 active+clean, 72 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T18:14:19.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:18 smithi133 ceph-mon[126384]: osdmap e83: 12 total, 12 up, 12 in 2023-07-14T18:14:19.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:18 smithi133 ceph-mon[126384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-14T18:14:19.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:18 smithi133 ceph-mon[126384]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-14T18:14:19.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:18 smithi133 ceph-mon[126384]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-14T18:14:19.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:18 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:19.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:18 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:19.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:18 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:19.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:18 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:19.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:18 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/615401246' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-07-14T18:14:19.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:18 smithi133 ceph-mon[126384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-07-14T18:14:19.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:18 smithi133 ceph-mon[126384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-07-14T18:14:19.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:18 smithi133 ceph-mon[126384]: osdmap e84: 12 total, 12 up, 12 in 2023-07-14T18:14:19.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:19.364+0000 7f499abff700 1 -- 172.21.15.133:0/1724691163 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4994106a80 con 0x7f49940ffc20 2023-07-14T18:14:20.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:19 smithi161 ceph-mon[128993]: mds.? [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] up:active 2023-07-14T18:14:20.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:19 smithi161 ceph-mon[128993]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T18:14:20.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:19 smithi161 ceph-mon[128993]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T18:14:20.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:19 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:20.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:19 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:20.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:19 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:14:20.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:19 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:14:20.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:19 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:20.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:19 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1724691163' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-14T18:14:20.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:19 smithi161 ceph-mon[128993]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-14T18:14:20.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:19 smithi171 ceph-mon[129241]: mds.? [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] up:active 2023-07-14T18:14:20.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:19 smithi171 ceph-mon[129241]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T18:14:20.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:19 smithi171 ceph-mon[129241]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T18:14:20.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:19 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:20.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:19 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:20.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:19 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:14:20.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:19 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:14:20.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:19 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:20.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:19 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1724691163' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-14T18:14:20.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:19 smithi171 ceph-mon[129241]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-14T18:14:20.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:19 smithi133 ceph-mon[126384]: mds.? [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] up:active 2023-07-14T18:14:20.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:19 smithi133 ceph-mon[126384]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T18:14:20.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:19 smithi133 ceph-mon[126384]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T18:14:20.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:19 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:20.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:19 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:20.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:19 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:14:20.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:19 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:14:20.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:19 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:14:20.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:19 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1724691163' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-14T18:14:20.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:19 smithi133 ceph-mon[126384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-14T18:14:21.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.013+0000 7f4989ffb700 1 -- 172.21.15.133:0/1724691163 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v86) v1 ==== 228+0+0 (secure 0 0 0) 0x7f499009a790 con 0x7f49940ffc20 2023-07-14T18:14:21.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.020+0000 7f499abff700 1 -- 172.21.15.133:0/1724691163 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f497c0a11f0 msgr2=0x7f497c0a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:21.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.020+0000 7f499abff700 1 --2- 172.21.15.133:0/1724691163 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f497c0a11f0 0x7f497c0a36a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4984009c60 tx=0x7f4984009260 comp rx=0 tx=0).stop 2023-07-14T18:14:21.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.020+0000 7f499abff700 1 -- 172.21.15.133:0/1724691163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49940ffc20 msgr2=0x7f4994195a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:21.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.020+0000 7f499abff700 1 --2- 172.21.15.133:0/1724691163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49940ffc20 0x7f4994195a40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f499000b070 tx=0x7f499000ba50 comp rx=0 tx=0).stop 2023-07-14T18:14:21.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.022+0000 7f499abff700 1 -- 172.21.15.133:0/1724691163 shutdown_connections 2023-07-14T18:14:21.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.022+0000 7f499abff700 1 --2- 172.21.15.133:0/1724691163 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f497c0a11f0 0x7f497c0a36a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:21.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.022+0000 7f499abff700 1 --2- 172.21.15.133:0/1724691163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49940ffc20 0x7f4994195a40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:21.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.022+0000 7f499abff700 1 --2- 172.21.15.133:0/1724691163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49941017a0 0x7f499419cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:21.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.022+0000 7f499abff700 1 --2- 172.21.15.133:0/1724691163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4994100e10 0x7f4994195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:21.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.022+0000 7f499abff700 1 -- 172.21.15.133:0/1724691163 >> 172.21.15.133:0/1724691163 conn(0x7f49940fb200 msgr2=0x7f49940fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:21.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.022+0000 7f499abff700 1 -- 172.21.15.133:0/1724691163 shutdown_connections 2023-07-14T18:14:21.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.023+0000 7f499abff700 1 -- 172.21.15.133:0/1724691163 wait complete. 2023-07-14T18:14:21.067 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data_ec' created 2023-07-14T18:14:21.086 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-07-14T18:14:21.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.245+0000 7f366c7ff700 1 -- 172.21.15.133:0/2519535996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3664074680 msgr2=0x7f3664072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:21.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.245+0000 7f366c7ff700 1 --2- 172.21.15.133:0/2519535996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3664074680 0x7f3664072ad0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3660009990 tx=0x7f3660009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:21.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.245+0000 7f366c7ff700 1 -- 172.21.15.133:0/2519535996 shutdown_connections 2023-07-14T18:14:21.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.245+0000 7f366c7ff700 1 --2- 172.21.15.133:0/2519535996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3664107ae0 0x7f3664109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:21.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.245+0000 7f366c7ff700 1 --2- 172.21.15.133:0/2519535996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3664074680 0x7f3664072ad0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:21.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.245+0000 7f366c7ff700 1 --2- 172.21.15.133:0/2519535996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36640730a0 0x7f3664073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:21.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.245+0000 7f366c7ff700 1 -- 172.21.15.133:0/2519535996 >> 172.21.15.133:0/2519535996 conn(0x7f36640fb260 msgr2=0x7f36640fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:21.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.245+0000 7f366c7ff700 1 -- 172.21.15.133:0/2519535996 shutdown_connections 2023-07-14T18:14:21.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.245+0000 7f366c7ff700 1 -- 172.21.15.133:0/2519535996 wait complete. 2023-07-14T18:14:21.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.246+0000 7f366c7ff700 1 Processor -- start 2023-07-14T18:14:21.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.246+0000 7f366c7ff700 1 -- start start 2023-07-14T18:14:21.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.246+0000 7f366c7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36640730a0 0x7f3664197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:21.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.246+0000 7f366c7ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3664074680 0x7f3664198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:21.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.246+0000 7f366c7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3664107ae0 0x7f366419f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:21.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.246+0000 7f366c7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366419f940 con 0x7f36640730a0 2023-07-14T18:14:21.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.246+0000 7f366c7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366419fa80 con 0x7f3664107ae0 2023-07-14T18:14:21.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.246+0000 7f366c7ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366419fc10 con 0x7f3664074680 2023-07-14T18:14:21.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.247+0000 7f366a59b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36640730a0 0x7f3664197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:21.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.247+0000 7f366a59b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36640730a0 0x7f3664197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39138/0 (socket says 172.21.15.133:39138) 2023-07-14T18:14:21.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.247+0000 7f366a59b700 1 -- 172.21.15.133:0/1086480739 learned_addr learned my addr 172.21.15.133:0/1086480739 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:21.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.247+0000 7f366ad9c700 1 --2- 172.21.15.133:0/1086480739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3664107ae0 0x7f366419f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:21.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.247+0000 7f3669d9a700 1 --2- 172.21.15.133:0/1086480739 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3664074680 0x7f3664198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:21.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.247+0000 7f366a59b700 1 -- 172.21.15.133:0/1086480739 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3664074680 msgr2=0x7f3664198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:21.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.247+0000 7f366a59b700 1 --2- 172.21.15.133:0/1086480739 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3664074680 0x7f3664198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:21.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.247+0000 7f366a59b700 1 -- 172.21.15.133:0/1086480739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3664107ae0 msgr2=0x7f366419f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:21.294 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.247+0000 7f366a59b700 1 --2- 172.21.15.133:0/1086480739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3664107ae0 0x7f366419f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:21.294 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.247+0000 7f366a59b700 1 -- 172.21.15.133:0/1086480739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36600095f0 con 0x7f36640730a0 2023-07-14T18:14:21.294 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.247+0000 7f366ad9c700 1 --2- 172.21.15.133:0/1086480739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3664107ae0 0x7f366419f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:14:21.294 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.247+0000 7f366a59b700 1 --2- 172.21.15.133:0/1086480739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36640730a0 0x7f3664197dc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3660005720 tx=0x7f3660009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:21.294 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.247+0000 7f365b7fe700 1 -- 172.21.15.133:0/1086480739 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366001d070 con 0x7f36640730a0 2023-07-14T18:14:21.294 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.248+0000 7f365b7fe700 1 -- 172.21.15.133:0/1086480739 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3660021410 con 0x7f36640730a0 2023-07-14T18:14:21.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.248+0000 7f366c7ff700 1 -- 172.21.15.133:0/1086480739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f366419fe90 con 0x7f36640730a0 2023-07-14T18:14:21.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.248+0000 7f365b7fe700 1 -- 172.21.15.133:0/1086480739 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3660017650 con 0x7f36640730a0 2023-07-14T18:14:21.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.248+0000 7f366c7ff700 1 -- 172.21.15.133:0/1086480739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36641a0250 con 0x7f36640730a0 2023-07-14T18:14:21.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.249+0000 7f366c7ff700 1 -- 172.21.15.133:0/1086480739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f366410d650 con 0x7f36640730a0 2023-07-14T18:14:21.296 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.251+0000 7f365b7fe700 1 -- 172.21.15.133:0/1086480739 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f3660021580 con 0x7f36640730a0 2023-07-14T18:14:21.296 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.252+0000 7f365b7fe700 1 --2- 172.21.15.133:0/1086480739 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f36500a11d0 0x7f36500a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:21.296 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.252+0000 7f365b7fe700 1 -- 172.21.15.133:0/1086480739 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f36600ce070 con 0x7f36640730a0 2023-07-14T18:14:21.296 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.252+0000 7f3669d9a700 1 --2- 172.21.15.133:0/1086480739 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f36500a11d0 0x7f36500a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:21.296 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.253+0000 7f3669d9a700 1 --2- 172.21.15.133:0/1086480739 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f36500a11d0 0x7f36500a3680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f36640743a0 tx=0x7f3654009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:21.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.254+0000 7f365b7fe700 1 -- 172.21.15.133:0/1086480739 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f3660097620 con 0x7f36640730a0 2023-07-14T18:14:21.349 INFO:journalctl@ceph.mds.b.smithi161.stdout:Jul 14 18:14:20 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-b[154371]: 2023-07-14T18:14:20.986+0000 7faba7315700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-14T18:14:21.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:20 smithi161 ceph-mon[128993]: pgmap v270: 97 pgs: 23 creating+peering, 35 active+clean, 39 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-07-14T18:14:21.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:20 smithi161 ceph-mon[128993]: osdmap e85: 12 total, 12 up, 12 in 2023-07-14T18:14:21.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:20 smithi171 ceph-mon[129241]: pgmap v270: 97 pgs: 23 creating+peering, 35 active+clean, 39 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-07-14T18:14:21.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:20 smithi171 ceph-mon[129241]: osdmap e85: 12 total, 12 up, 12 in 2023-07-14T18:14:21.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:20 smithi133 ceph-mon[126384]: pgmap v270: 97 pgs: 23 creating+peering, 35 active+clean, 39 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-07-14T18:14:21.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:20 smithi133 ceph-mon[126384]: osdmap e85: 12 total, 12 up, 12 in 2023-07-14T18:14:21.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:21.452+0000 7f366c7ff700 1 -- 172.21.15.133:0/1086480739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f366410d7e0 con 0x7f36640730a0 2023-07-14T18:14:22.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.024+0000 7f365b7fe700 1 -- 172.21.15.133:0/1086480739 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v87) v1 ==== 169+0+0 (secure 0 0 0) 0x7f366009ac40 con 0x7f36640730a0 2023-07-14T18:14:22.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.029+0000 7f366c7ff700 1 -- 172.21.15.133:0/1086480739 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f36500a11d0 msgr2=0x7f36500a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:22.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.029+0000 7f366c7ff700 1 --2- 172.21.15.133:0/1086480739 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f36500a11d0 0x7f36500a3680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f36640743a0 tx=0x7f3654009260 comp rx=0 tx=0).stop 2023-07-14T18:14:22.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.029+0000 7f366c7ff700 1 -- 172.21.15.133:0/1086480739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36640730a0 msgr2=0x7f3664197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:22.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.029+0000 7f366c7ff700 1 --2- 172.21.15.133:0/1086480739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36640730a0 0x7f3664197dc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3660005720 tx=0x7f3660009580 comp rx=0 tx=0).stop 2023-07-14T18:14:22.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.032+0000 7f366c7ff700 1 -- 172.21.15.133:0/1086480739 shutdown_connections 2023-07-14T18:14:22.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.032+0000 7f366c7ff700 1 --2- 172.21.15.133:0/1086480739 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f36500a11d0 0x7f36500a3680 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:22.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.032+0000 7f366c7ff700 1 --2- 172.21.15.133:0/1086480739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3664107ae0 0x7f366419f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:22.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.032+0000 7f366c7ff700 1 --2- 172.21.15.133:0/1086480739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36640730a0 0x7f3664197dc0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:22.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.032+0000 7f366c7ff700 1 --2- 172.21.15.133:0/1086480739 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3664074680 0x7f3664198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:22.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.032+0000 7f366c7ff700 1 -- 172.21.15.133:0/1086480739 >> 172.21.15.133:0/1086480739 conn(0x7f36640fb260 msgr2=0x7f3664109e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:22.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.038+0000 7f366c7ff700 1 -- 172.21.15.133:0/1086480739 shutdown_connections 2023-07-14T18:14:22.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.040+0000 7f366c7ff700 1 -- 172.21.15.133:0/1086480739 wait complete. 2023-07-14T18:14:22.089 INFO:teuthology.orchestra.run.smithi133.stderr:set pool 4 allow_ec_overwrites to true 2023-07-14T18:14:22.108 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-07-14T18:14:22.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.263+0000 7f76e5fff700 1 -- 172.21.15.133:0/227683885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76e0100e10 msgr2=0x7f76e0101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:22.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.263+0000 7f76e5fff700 1 --2- 172.21.15.133:0/227683885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76e0100e10 0x7f76e0101260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f76d4009990 tx=0x7f76d4009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:22.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.263+0000 7f76e5fff700 1 -- 172.21.15.133:0/227683885 shutdown_connections 2023-07-14T18:14:22.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.263+0000 7f76e5fff700 1 --2- 172.21.15.133:0/227683885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76e00ffc20 0x7f76e0100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:22.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.263+0000 7f76e5fff700 1 --2- 172.21.15.133:0/227683885 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76e01017a0 0x7f76e0108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:22.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.263+0000 7f76e5fff700 1 --2- 172.21.15.133:0/227683885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76e0100e10 0x7f76e0101260 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:22.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.263+0000 7f76e5fff700 1 -- 172.21.15.133:0/227683885 >> 172.21.15.133:0/227683885 conn(0x7f76e00fb260 msgr2=0x7f76e00fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:22.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.263+0000 7f76e5fff700 1 -- 172.21.15.133:0/227683885 shutdown_connections 2023-07-14T18:14:22.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.263+0000 7f76e5fff700 1 -- 172.21.15.133:0/227683885 wait complete. 2023-07-14T18:14:22.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.264+0000 7f76e5fff700 1 Processor -- start 2023-07-14T18:14:22.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.265+0000 7f76e5fff700 1 -- start start 2023-07-14T18:14:22.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.265+0000 7f76e5fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76e00ffc20 0x7f76e00743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:22.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.265+0000 7f76e5fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76e0100e10 0x7f76e0072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:22.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.265+0000 7f76e5fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76e01017a0 0x7f76e0072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:22.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.265+0000 7f76e5fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e0073710 con 0x7f76e0100e10 2023-07-14T18:14:22.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.265+0000 7f76e5fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e0073880 con 0x7f76e01017a0 2023-07-14T18:14:22.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.265+0000 7f76e5fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e019f960 con 0x7f76e00ffc20 2023-07-14T18:14:22.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.265+0000 7f76deffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76e0100e10 0x7f76e0072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:22.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.265+0000 7f76df7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76e00ffc20 0x7f76e00743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:22.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.265+0000 7f76df7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76e00ffc20 0x7f76e00743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:44032/0 (socket says 172.21.15.133:44032) 2023-07-14T18:14:22.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.265+0000 7f76df7fe700 1 -- 172.21.15.133:0/3137844495 learned_addr learned my addr 172.21.15.133:0/3137844495 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:22.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.265+0000 7f76deffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76e0100e10 0x7f76e0072a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39158/0 (socket says 172.21.15.133:39158) 2023-07-14T18:14:22.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.265+0000 7f76dffff700 1 --2- 172.21.15.133:0/3137844495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76e01017a0 0x7f76e0072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:22.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.265+0000 7f76df7fe700 1 -- 172.21.15.133:0/3137844495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76e01017a0 msgr2=0x7f76e0072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:22.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.265+0000 7f76df7fe700 1 --2- 172.21.15.133:0/3137844495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76e01017a0 0x7f76e0072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:22.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.265+0000 7f76df7fe700 1 -- 172.21.15.133:0/3137844495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76e0100e10 msgr2=0x7f76e0072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:22.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.265+0000 7f76df7fe700 1 --2- 172.21.15.133:0/3137844495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76e0100e10 0x7f76e0072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:22.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.265+0000 7f76df7fe700 1 -- 172.21.15.133:0/3137844495 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d40095f0 con 0x7f76e00ffc20 2023-07-14T18:14:22.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.265+0000 7f76dffff700 1 --2- 172.21.15.133:0/3137844495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76e01017a0 0x7f76e0072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:14:22.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.266+0000 7f76df7fe700 1 --2- 172.21.15.133:0/3137844495 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76e00ffc20 0x7f76e00743f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f76d000e970 tx=0x7f76d000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:22.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.266+0000 7f76dcff9700 1 -- 172.21.15.133:0/3137844495 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d0009790 con 0x7f76e00ffc20 2023-07-14T18:14:22.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.266+0000 7f76e5fff700 1 -- 172.21.15.133:0/3137844495 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e019fc40 con 0x7f76e00ffc20 2023-07-14T18:14:22.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.266+0000 7f76dcff9700 1 -- 172.21.15.133:0/3137844495 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76d000eec0 con 0x7f76e00ffc20 2023-07-14T18:14:22.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.266+0000 7f76dcff9700 1 -- 172.21.15.133:0/3137844495 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d0003cb0 con 0x7f76e00ffc20 2023-07-14T18:14:22.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.266+0000 7f76e5fff700 1 -- 172.21.15.133:0/3137844495 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e01a01a0 con 0x7f76e00ffc20 2023-07-14T18:14:22.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.267+0000 7f76e5fff700 1 -- 172.21.15.133:0/3137844495 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e010b440 con 0x7f76e00ffc20 2023-07-14T18:14:22.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.269+0000 7f76dcff9700 1 -- 172.21.15.133:0/3137844495 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f76d000cbd0 con 0x7f76e00ffc20 2023-07-14T18:14:22.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.270+0000 7f76dcff9700 1 --2- 172.21.15.133:0/3137844495 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f76c80a11d0 0x7f76c80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:22.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.271+0000 7f76dcff9700 1 -- 172.21.15.133:0/3137844495 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f76d0014070 con 0x7f76e00ffc20 2023-07-14T18:14:22.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.271+0000 7f76deffd700 1 --2- 172.21.15.133:0/3137844495 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f76c80a11d0 0x7f76c80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:22.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.271+0000 7f76deffd700 1 --2- 172.21.15.133:0/3137844495 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f76c80a11d0 0x7f76c80a3680 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f76d400b2e0 tx=0x7f76d4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:22.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.272+0000 7f76dcff9700 1 -- 172.21.15.133:0/3137844495 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f76d00965d0 con 0x7f76e00ffc20 2023-07-14T18:14:22.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:22 smithi161 ceph-mon[128993]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-14T18:14:22.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:22 smithi161 ceph-mon[128993]: osdmap e86: 12 total, 12 up, 12 in 2023-07-14T18:14:22.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:22 smithi161 ceph-mon[128993]: pgmap v273: 129 pgs: 20 creating+peering, 73 active+clean, 36 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1 op/s 2023-07-14T18:14:22.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:22 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1086480739' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-07-14T18:14:22.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:22 smithi171 ceph-mon[129241]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-14T18:14:22.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:22 smithi171 ceph-mon[129241]: osdmap e86: 12 total, 12 up, 12 in 2023-07-14T18:14:22.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:22 smithi171 ceph-mon[129241]: pgmap v273: 129 pgs: 20 creating+peering, 73 active+clean, 36 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1 op/s 2023-07-14T18:14:22.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:22 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1086480739' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-07-14T18:14:22.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:22 smithi133 ceph-mon[126384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-14T18:14:22.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:22 smithi133 ceph-mon[126384]: osdmap e86: 12 total, 12 up, 12 in 2023-07-14T18:14:22.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:22 smithi133 ceph-mon[126384]: pgmap v273: 129 pgs: 20 creating+peering, 73 active+clean, 36 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1 op/s 2023-07-14T18:14:22.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:22 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1086480739' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-07-14T18:14:22.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:22.522+0000 7f76e5fff700 1 -- 172.21.15.133:0/3137844495 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f76e0106b20 con 0x7f76e00ffc20 2023-07-14T18:14:23.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.043+0000 7f76dcff9700 1 -- 172.21.15.133:0/3137844495 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7f76d0099bf0 con 0x7f76e00ffc20 2023-07-14T18:14:23.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.046+0000 7f76e5fff700 1 -- 172.21.15.133:0/3137844495 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f76c80a11d0 msgr2=0x7f76c80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:23.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.046+0000 7f76e5fff700 1 --2- 172.21.15.133:0/3137844495 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f76c80a11d0 0x7f76c80a3680 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f76d400b2e0 tx=0x7f76d4005fb0 comp rx=0 tx=0).stop 2023-07-14T18:14:23.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.046+0000 7f76e5fff700 1 -- 172.21.15.133:0/3137844495 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76e00ffc20 msgr2=0x7f76e00743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:23.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.046+0000 7f76e5fff700 1 --2- 172.21.15.133:0/3137844495 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76e00ffc20 0x7f76e00743f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f76d000e970 tx=0x7f76d000ec80 comp rx=0 tx=0).stop 2023-07-14T18:14:23.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.046+0000 7f76e5fff700 1 -- 172.21.15.133:0/3137844495 shutdown_connections 2023-07-14T18:14:23.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.046+0000 7f76e5fff700 1 --2- 172.21.15.133:0/3137844495 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f76c80a11d0 0x7f76c80a3680 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:23.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.046+0000 7f76e5fff700 1 --2- 172.21.15.133:0/3137844495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76e01017a0 0x7f76e0072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:23.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.046+0000 7f76e5fff700 1 --2- 172.21.15.133:0/3137844495 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76e00ffc20 0x7f76e00743f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:23.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.046+0000 7f76e5fff700 1 --2- 172.21.15.133:0/3137844495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76e0100e10 0x7f76e0072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:23.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.046+0000 7f76e5fff700 1 -- 172.21.15.133:0/3137844495 >> 172.21.15.133:0/3137844495 conn(0x7f76e00fb260 msgr2=0x7f76e00fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:23.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.046+0000 7f76e5fff700 1 -- 172.21.15.133:0/3137844495 shutdown_connections 2023-07-14T18:14:23.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.046+0000 7f76e5fff700 1 -- 172.21.15.133:0/3137844495 wait complete. 2023-07-14T18:14:23.091 INFO:teuthology.orchestra.run.smithi133.stderr:added data pool 4 to fsmap 2023-07-14T18:14:23.109 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T18:14:23.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.262+0000 7f19243ff700 1 -- 172.21.15.133:0/315342768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f191c1017b0 msgr2=0x7f191c108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:23.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.262+0000 7f19243ff700 1 --2- 172.21.15.133:0/315342768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f191c1017b0 0x7f191c108070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f191400b0a0 tx=0x7f191400b3b0 comp rx=0 tx=0).stop 2023-07-14T18:14:23.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.262+0000 7f19243ff700 1 -- 172.21.15.133:0/315342768 shutdown_connections 2023-07-14T18:14:23.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.262+0000 7f19243ff700 1 --2- 172.21.15.133:0/315342768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f191c1017b0 0x7f191c108070 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:23.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.262+0000 7f19243ff700 1 --2- 172.21.15.133:0/315342768 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f191c0ffc30 0x7f191c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:23.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.262+0000 7f19243ff700 1 --2- 172.21.15.133:0/315342768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f191c100e20 0x7f191c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:23.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.262+0000 7f19243ff700 1 -- 172.21.15.133:0/315342768 >> 172.21.15.133:0/315342768 conn(0x7f191c0fb210 msgr2=0x7f191c0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:23.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.262+0000 7f19243ff700 1 -- 172.21.15.133:0/315342768 shutdown_connections 2023-07-14T18:14:23.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.262+0000 7f19243ff700 1 -- 172.21.15.133:0/315342768 wait complete. 2023-07-14T18:14:23.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.263+0000 7f19243ff700 1 Processor -- start 2023-07-14T18:14:23.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.264+0000 7f19243ff700 1 -- start start 2023-07-14T18:14:23.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.264+0000 7f19243ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f191c0ffc30 0x7f191c197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:23.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.264+0000 7f19243ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f191c100e20 0x7f191c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:23.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.264+0000 7f19243ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f191c1017b0 0x7f191c19f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:23.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.264+0000 7f19243ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191c19f7e0 con 0x7f191c1017b0 2023-07-14T18:14:23.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.264+0000 7f19243ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191c19f950 con 0x7f191c0ffc30 2023-07-14T18:14:23.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.264+0000 7f19243ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191c19fb10 con 0x7f191c100e20 2023-07-14T18:14:23.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.264+0000 7f192299c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f191c1017b0 0x7f191c19f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:23.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.264+0000 7f192219b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f191c0ffc30 0x7f191c197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:23.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.264+0000 7f192299c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f191c1017b0 0x7f191c19f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39176/0 (socket says 172.21.15.133:39176) 2023-07-14T18:14:23.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.264+0000 7f192219b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f191c0ffc30 0x7f191c197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:34662/0 (socket says 172.21.15.133:34662) 2023-07-14T18:14:23.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.264+0000 7f192299c700 1 -- 172.21.15.133:0/2706818086 learned_addr learned my addr 172.21.15.133:0/2706818086 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:23.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.264+0000 7f192199a700 1 --2- 172.21.15.133:0/2706818086 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f191c100e20 0x7f191c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:23.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.264+0000 7f192299c700 1 -- 172.21.15.133:0/2706818086 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f191c100e20 msgr2=0x7f191c1981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:23.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.264+0000 7f192299c700 1 --2- 172.21.15.133:0/2706818086 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f191c100e20 0x7f191c1981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:23.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.264+0000 7f192299c700 1 -- 172.21.15.133:0/2706818086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f191c0ffc30 msgr2=0x7f191c197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:23.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.264+0000 7f192299c700 1 --2- 172.21.15.133:0/2706818086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f191c0ffc30 0x7f191c197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:23.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.265+0000 7f192299c700 1 -- 172.21.15.133:0/2706818086 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1914009d00 con 0x7f191c1017b0 2023-07-14T18:14:23.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.265+0000 7f192199a700 1 --2- 172.21.15.133:0/2706818086 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f191c100e20 0x7f191c1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:14:23.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.265+0000 7f192299c700 1 --2- 172.21.15.133:0/2706818086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f191c1017b0 0x7f191c19f210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f191400d010 tx=0x7f191400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:23.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.265+0000 7f19137fe700 1 -- 172.21.15.133:0/2706818086 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191400e040 con 0x7f191c1017b0 2023-07-14T18:14:23.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.265+0000 7f19137fe700 1 -- 172.21.15.133:0/2706818086 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1914003e10 con 0x7f191c1017b0 2023-07-14T18:14:23.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.265+0000 7f19243ff700 1 -- 172.21.15.133:0/2706818086 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f191c19fd90 con 0x7f191c1017b0 2023-07-14T18:14:23.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.265+0000 7f19137fe700 1 -- 172.21.15.133:0/2706818086 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19140052a0 con 0x7f191c1017b0 2023-07-14T18:14:23.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.265+0000 7f19243ff700 1 -- 172.21.15.133:0/2706818086 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f191c1a01c0 con 0x7f191c1017b0 2023-07-14T18:14:23.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.266+0000 7f19243ff700 1 -- 172.21.15.133:0/2706818086 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f191c10b460 con 0x7f191c1017b0 2023-07-14T18:14:23.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.268+0000 7f19137fe700 1 -- 172.21.15.133:0/2706818086 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f1914005400 con 0x7f191c1017b0 2023-07-14T18:14:23.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.271+0000 7f19137fe700 1 --2- 172.21.15.133:0/2706818086 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f19080a1220 0x7f19080a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:23.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.271+0000 7f19137fe700 1 -- 172.21.15.133:0/2706818086 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f19140cdcc0 con 0x7f191c1017b0 2023-07-14T18:14:23.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.271+0000 7f192219b700 1 --2- 172.21.15.133:0/2706818086 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f19080a1220 0x7f19080a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:23.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.271+0000 7f192219b700 1 --2- 172.21.15.133:0/2706818086 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f19080a1220 0x7f19080a36d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f191c198b40 tx=0x7f1918009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:23.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.275+0000 7f19137fe700 1 -- 172.21.15.133:0/2706818086 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f19140d4080 con 0x7f191c1017b0 2023-07-14T18:14:23.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:23 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1086480739' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-07-14T18:14:23.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:23 smithi161 ceph-mon[128993]: osdmap e87: 12 total, 12 up, 12 in 2023-07-14T18:14:23.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:23 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3137844495' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-07-14T18:14:23.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:23 smithi161 ceph-mon[128993]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-07-14T18:14:23.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:23 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1086480739' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-07-14T18:14:23.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:23 smithi171 ceph-mon[129241]: osdmap e87: 12 total, 12 up, 12 in 2023-07-14T18:14:23.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:23 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3137844495' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-07-14T18:14:23.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:23 smithi171 ceph-mon[129241]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-07-14T18:14:23.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:23 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1086480739' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-07-14T18:14:23.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:23 smithi133 ceph-mon[126384]: osdmap e87: 12 total, 12 up, 12 in 2023-07-14T18:14:23.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:23 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3137844495' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-07-14T18:14:23.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:23 smithi133 ceph-mon[126384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-07-14T18:14:23.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.518+0000 7f19243ff700 1 -- 172.21.15.133:0/2706818086 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f191c1a07d0 con 0x7f191c1017b0 2023-07-14T18:14:23.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.519+0000 7f19137fe700 1 -- 172.21.15.133:0/2706818086 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10790 (secure 0 0 0) 0x7f1914097250 con 0x7f191c1017b0 2023-07-14T18:14:23.562 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:14:23.562 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3168151271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3168151271},{"type":"v1","addr":"172.21.15.133:6837","nonce":3168151271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14544,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/3093144261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3093144261},{"type":"v1","addr":"172.21.15.133:6839","nonce":3093144261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14556,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/3282902068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":3282902068},{"type":"v1","addr":"172.21.15.133:6841","nonce":3282902068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/4023127259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4023127259},{"type":"v1","addr":"172.21.15.161:6833","nonce":4023127259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2722373740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2722373740},{"type":"v1","addr":"172.21.15.171:6833","nonce":2722373740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2399005074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2399005074},{"type":"v1","addr":"172.21.15.133:6835","nonce":2399005074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3725502564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3725502564},{"type":"v1","addr":"172.21.15.161:6835","nonce":3725502564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1877362116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1877362116},{"type":"v1","addr":"172.21.15.171:6835","nonce":1877362116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3458064798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3458064798},{"type":"v1","addr":"172.21.15.161:6837","nonce":3458064798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/106205332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":106205332},{"type":"v1","addr":"172.21.15.171:6837","nonce":106205332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24541,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/816201198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":816201198},{"type":"v1","addr":"172.21.15.171:6839","nonce":816201198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T18:14:17.440873+0000","modified":"2023-07-14T18:14:23.035911+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/2503865871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2503865871},{"type":"v1","addr":"172.21.15.161:6839","nonce":2503865871}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-07-14T18:14:23.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.521+0000 7f19243ff700 1 -- 172.21.15.133:0/2706818086 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f19080a1220 msgr2=0x7f19080a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:23.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.521+0000 7f19243ff700 1 --2- 172.21.15.133:0/2706818086 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f19080a1220 0x7f19080a36d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f191c198b40 tx=0x7f1918009260 comp rx=0 tx=0).stop 2023-07-14T18:14:23.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.521+0000 7f19243ff700 1 -- 172.21.15.133:0/2706818086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f191c1017b0 msgr2=0x7f191c19f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:23.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.521+0000 7f19243ff700 1 --2- 172.21.15.133:0/2706818086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f191c1017b0 0x7f191c19f210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f191400d010 tx=0x7f191400bc80 comp rx=0 tx=0).stop 2023-07-14T18:14:23.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.521+0000 7f19243ff700 1 -- 172.21.15.133:0/2706818086 shutdown_connections 2023-07-14T18:14:23.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.521+0000 7f19243ff700 1 --2- 172.21.15.133:0/2706818086 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f19080a1220 0x7f19080a36d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:23.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.521+0000 7f19243ff700 1 --2- 172.21.15.133:0/2706818086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f191c0ffc30 0x7f191c197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:23.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.521+0000 7f19243ff700 1 --2- 172.21.15.133:0/2706818086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f191c1017b0 0x7f191c19f210 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:23.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.521+0000 7f19243ff700 1 --2- 172.21.15.133:0/2706818086 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f191c100e20 0x7f191c1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:23.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.521+0000 7f19243ff700 1 -- 172.21.15.133:0/2706818086 >> 172.21.15.133:0/2706818086 conn(0x7f191c0fb210 msgr2=0x7f191c106660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:23.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.521+0000 7f19243ff700 1 -- 172.21.15.133:0/2706818086 shutdown_connections 2023-07-14T18:14:23.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.521+0000 7f19243ff700 1 -- 172.21.15.133:0/2706818086 wait complete. 2023-07-14T18:14:23.566 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 17 2023-07-14T18:14:23.583 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T18:14:23.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.736+0000 7efd2b7ff700 1 -- 172.21.15.133:0/3302470304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd241017b0 msgr2=0x7efd24108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:23.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.736+0000 7efd2b7ff700 1 --2- 172.21.15.133:0/3302470304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd241017b0 0x7efd24108070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efd1c00b0a0 tx=0x7efd1c00b3b0 comp rx=0 tx=0).stop 2023-07-14T18:14:23.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.736+0000 7efd2b7ff700 1 -- 172.21.15.133:0/3302470304 shutdown_connections 2023-07-14T18:14:23.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.736+0000 7efd2b7ff700 1 --2- 172.21.15.133:0/3302470304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd240ffc30 0x7efd24100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:23.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.736+0000 7efd2b7ff700 1 --2- 172.21.15.133:0/3302470304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd241017b0 0x7efd24108070 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:23.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.736+0000 7efd2b7ff700 1 --2- 172.21.15.133:0/3302470304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd24100e20 0x7efd24101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:23.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.736+0000 7efd2b7ff700 1 -- 172.21.15.133:0/3302470304 >> 172.21.15.133:0/3302470304 conn(0x7efd240fb210 msgr2=0x7efd240fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:23.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.736+0000 7efd2b7ff700 1 -- 172.21.15.133:0/3302470304 shutdown_connections 2023-07-14T18:14:23.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.736+0000 7efd2b7ff700 1 -- 172.21.15.133:0/3302470304 wait complete. 2023-07-14T18:14:23.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.737+0000 7efd2b7ff700 1 Processor -- start 2023-07-14T18:14:23.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.737+0000 7efd2b7ff700 1 -- start start 2023-07-14T18:14:23.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.738+0000 7efd2b7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd240ffc30 0x7efd241959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:23.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.738+0000 7efd2b7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd24100e20 0x7efd24195f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:23.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.738+0000 7efd2b7ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd241017b0 0x7efd2419cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:23.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.738+0000 7efd2b7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2419d540 con 0x7efd24100e20 2023-07-14T18:14:23.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.738+0000 7efd2b7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2419d680 con 0x7efd240ffc30 2023-07-14T18:14:23.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.738+0000 7efd2b7ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2419d810 con 0x7efd241017b0 2023-07-14T18:14:23.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.738+0000 7efd28d9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd24100e20 0x7efd24195f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:23.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.738+0000 7efd28d9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd24100e20 0x7efd24195f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39194/0 (socket says 172.21.15.133:39194) 2023-07-14T18:14:23.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.738+0000 7efd28d9a700 1 -- 172.21.15.133:0/2203716506 learned_addr learned my addr 172.21.15.133:0/2203716506 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:23.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.738+0000 7efd2959b700 1 --2- 172.21.15.133:0/2203716506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd240ffc30 0x7efd241959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:23.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.738+0000 7efd29d9c700 1 --2- 172.21.15.133:0/2203716506 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd241017b0 0x7efd2419cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:23.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.738+0000 7efd28d9a700 1 -- 172.21.15.133:0/2203716506 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd241017b0 msgr2=0x7efd2419cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:23.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.738+0000 7efd28d9a700 1 --2- 172.21.15.133:0/2203716506 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd241017b0 0x7efd2419cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:23.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.738+0000 7efd28d9a700 1 -- 172.21.15.133:0/2203716506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd240ffc30 msgr2=0x7efd241959c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:23.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.738+0000 7efd28d9a700 1 --2- 172.21.15.133:0/2203716506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd240ffc30 0x7efd241959c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:23.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.738+0000 7efd28d9a700 1 -- 172.21.15.133:0/2203716506 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1c009d00 con 0x7efd24100e20 2023-07-14T18:14:23.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.738+0000 7efd28d9a700 1 --2- 172.21.15.133:0/2203716506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd24100e20 0x7efd24195f00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efd1400e9a0 tx=0x7efd1400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:23.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.739+0000 7efd1a7fc700 1 -- 172.21.15.133:0/2203716506 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd14009790 con 0x7efd24100e20 2023-07-14T18:14:23.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.739+0000 7efd1a7fc700 1 -- 172.21.15.133:0/2203716506 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd14010430 con 0x7efd24100e20 2023-07-14T18:14:23.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.739+0000 7efd2b7ff700 1 -- 172.21.15.133:0/2203716506 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd2419daf0 con 0x7efd24100e20 2023-07-14T18:14:23.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.739+0000 7efd1a7fc700 1 -- 172.21.15.133:0/2203716506 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd14018680 con 0x7efd24100e20 2023-07-14T18:14:23.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.739+0000 7efd2b7ff700 1 -- 172.21.15.133:0/2203716506 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd2419e050 con 0x7efd24100e20 2023-07-14T18:14:23.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.740+0000 7efd2b7ff700 1 -- 172.21.15.133:0/2203716506 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd241070f0 con 0x7efd24100e20 2023-07-14T18:14:23.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.742+0000 7efd1a7fc700 1 -- 172.21.15.133:0/2203716506 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7efd140105a0 con 0x7efd24100e20 2023-07-14T18:14:23.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.743+0000 7efd1a7fc700 1 --2- 172.21.15.133:0/2203716506 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efd100a1220 0x7efd100a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:23.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.743+0000 7efd1a7fc700 1 -- 172.21.15.133:0/2203716506 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7efd14014070 con 0x7efd24100e20 2023-07-14T18:14:23.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.743+0000 7efd2959b700 1 --2- 172.21.15.133:0/2203716506 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efd100a1220 0x7efd100a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:23.793 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.744+0000 7efd2959b700 1 --2- 172.21.15.133:0/2203716506 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efd100a1220 0x7efd100a36d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efd24100c80 tx=0x7efd20009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:23.793 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.747+0000 7efd1a7fc700 1 -- 172.21.15.133:0/2203716506 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7efd14051780 con 0x7efd24100e20 2023-07-14T18:14:23.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.948+0000 7efd2b7ff700 1 -- 172.21.15.133:0/2203716506 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd2402c8e0 con 0x7efd24100e20 2023-07-14T18:14:23.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.950+0000 7efd1a7fc700 1 -- 172.21.15.133:0/2203716506 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24003 (secure 0 0 0) 0x7efd140961b0 con 0x7efd24100e20 2023-07-14T18:14:23.993 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:14:23.994 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":88,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","created":"2023-07-14T18:06:49.561582+0000","modified":"2023-07-14T18:14:22.524878+0000","last_up_change":"2023-07-14T18:13:29.836099+0000","last_in_change":"2023-07-14T18:13:11.606710+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T18:10:03.945039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T18:14:15.451340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T18:14:16.892215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T18:14:20.493175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"85a29de7-d712-422a-b647-d9e90091aaf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6803","nonce":3076510801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6805","nonce":3076510801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6809","nonce":3076510801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6807","nonce":3076510801}]},"public_addr":"172.21.15.133:6803/3076510801","cluster_addr":"172.21.15.133:6805/3076510801","heartbeat_back_addr":"172.21.15.133:6809/3076510801","heartbeat_front_addr":"172.21.15.133:6807/3076510801","state":["exists","up"]},{"osd":1,"uuid":"80ad4e78-7917-495e-b119-99f7c7c36243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6801","nonce":1841901852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6803","nonce":1841901852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6807","nonce":1841901852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6805","nonce":1841901852}]},"public_addr":"172.21.15.161:6801/1841901852","cluster_addr":"172.21.15.161:6803/1841901852","heartbeat_back_addr":"172.21.15.161:6807/1841901852","heartbeat_front_addr":"172.21.15.161:6805/1841901852","state":["exists","up"]},{"osd":2,"uuid":"fc4bfd37-1fdd-47c6-a48e-6d910daf9e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6801","nonce":346103099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6803","nonce":346103099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6807","nonce":346103099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6805","nonce":346103099}]},"public_addr":"172.21.15.171:6801/346103099","cluster_addr":"172.21.15.171:6803/346103099","heartbeat_back_addr":"172.21.15.171:6807/346103099","heartbeat_front_addr":"172.21.15.171:6805/346103099","state":["exists","up"]},{"osd":3,"uuid":"4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6811","nonce":2152994143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6813","nonce":2152994143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6817","nonce":2152994143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6815","nonce":2152994143}]},"public_addr":"172.21.15.133:6811/2152994143","cluster_addr":"172.21.15.133:6813/2152994143","heartbeat_back_addr":"172.21.15.133:6817/2152994143","heartbeat_front_addr":"172.21.15.133:6815/2152994143","state":["exists","up"]},{"osd":4,"uuid":"4601a1c2-254b-4c4d-9b15-8fc952b3d2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6809","nonce":47213291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6811","nonce":47213291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6815","nonce":47213291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6813","nonce":47213291}]},"public_addr":"172.21.15.161:6809/47213291","cluster_addr":"172.21.15.161:6811/47213291","heartbeat_back_addr":"172.21.15.161:6815/47213291","heartbeat_front_addr":"172.21.15.161:6813/47213291","state":["exists","up"]},{"osd":5,"uuid":"6cca180a-9140-4e49-835b-86842dae0574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6809","nonce":777556751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6811","nonce":777556751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6815","nonce":777556751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6813","nonce":777556751}]},"public_addr":"172.21.15.171:6809/777556751","cluster_addr":"172.21.15.171:6811/777556751","heartbeat_back_addr":"172.21.15.171:6815/777556751","heartbeat_front_addr":"172.21.15.171:6813/777556751","state":["exists","up"]},{"osd":6,"uuid":"ea0299ff-7740-4494-a3a9-1503b3fc0ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6819","nonce":2884152358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6821","nonce":2884152358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6825","nonce":2884152358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6823","nonce":2884152358}]},"public_addr":"172.21.15.133:6819/2884152358","cluster_addr":"172.21.15.133:6821/2884152358","heartbeat_back_addr":"172.21.15.133:6825/2884152358","heartbeat_front_addr":"172.21.15.133:6823/2884152358","state":["exists","up"]},{"osd":7,"uuid":"b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6817","nonce":4183705749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6819","nonce":4183705749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6823","nonce":4183705749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6821","nonce":4183705749}]},"public_addr":"172.21.15.161:6817/4183705749","cluster_addr":"172.21.15.161:6819/4183705749","heartbeat_back_addr":"172.21.15.161:6823/4183705749","heartbeat_front_addr":"172.21.15.161:6821/4183705749","state":["exists","up"]},{"osd":8,"uuid":"87dffc07-bf48-48c8-b418-5c4b509686d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6817","nonce":2771494483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6819","nonce":2771494483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6823","nonce":2771494483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6821","nonce":2771494483}]},"public_addr":"172.21.15.171:6817/2771494483","cluster_addr":"172.21.15.171:6819/2771494483","heartbeat_back_addr":"172.21.15.171:6823/2771494483","heartbeat_front_addr":"172.21.15.171:6821/2771494483","state":["exists","up"]},{"osd":9,"uuid":"e0832870-79ce-4bce-8045-58b7a7f6dc69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6827","nonce":2066164149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6829","nonce":2066164149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6833","nonce":2066164149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6831","nonce":2066164149}]},"public_addr":"172.21.15.133:6827/2066164149","cluster_addr":"172.21.15.133:6829/2066164149","heartbeat_back_addr":"172.21.15.133:6833/2066164149","heartbeat_front_addr":"172.21.15.133:6831/2066164149","state":["exists","up"]},{"osd":10,"uuid":"081f0484-d134-4b2e-9c48-ef8e6a646422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6825","nonce":2162333626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6827","nonce":2162333626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6831","nonce":2162333626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6829","nonce":2162333626}]},"public_addr":"172.21.15.161:6825/2162333626","cluster_addr":"172.21.15.161:6827/2162333626","heartbeat_back_addr":"172.21.15.161:6831/2162333626","heartbeat_front_addr":"172.21.15.161:6829/2162333626","state":["exists","up"]},{"osd":11,"uuid":"4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6825","nonce":3403894313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6827","nonce":3403894313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6831","nonce":3403894313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6829","nonce":3403894313}]},"public_addr":"172.21.15.171:6825/3403894313","cluster_addr":"172.21.15.171:6827/3403894313","heartbeat_back_addr":"172.21.15.171:6831/3403894313","heartbeat_front_addr":"172.21.15.171:6829/3403894313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:17.816229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:39.638543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:00.543317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:26.988921+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:48.644694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:10.049860+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:35.986508+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:57.753846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:19.018245+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:44.460450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:06.279960+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:27.955006+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/1832569241":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6801/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:0/4219815304":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:6801/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/432253615":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1840700480":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1807775439":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1089030734":"2023-07-15T18:07:47.346704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T18:14:23.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.952+0000 7efd2b7ff700 1 -- 172.21.15.133:0/2203716506 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efd100a1220 msgr2=0x7efd100a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:23.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.952+0000 7efd2b7ff700 1 --2- 172.21.15.133:0/2203716506 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efd100a1220 0x7efd100a36d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efd24100c80 tx=0x7efd20009260 comp rx=0 tx=0).stop 2023-07-14T18:14:23.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.952+0000 7efd2b7ff700 1 -- 172.21.15.133:0/2203716506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd24100e20 msgr2=0x7efd24195f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:23.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.952+0000 7efd2b7ff700 1 --2- 172.21.15.133:0/2203716506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd24100e20 0x7efd24195f00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efd1400e9a0 tx=0x7efd1400ecb0 comp rx=0 tx=0).stop 2023-07-14T18:14:23.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.952+0000 7efd2b7ff700 1 -- 172.21.15.133:0/2203716506 shutdown_connections 2023-07-14T18:14:23.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.952+0000 7efd2b7ff700 1 --2- 172.21.15.133:0/2203716506 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efd100a1220 0x7efd100a36d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:23.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.952+0000 7efd2b7ff700 1 --2- 172.21.15.133:0/2203716506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd240ffc30 0x7efd241959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:23.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.952+0000 7efd2b7ff700 1 --2- 172.21.15.133:0/2203716506 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd241017b0 0x7efd2419cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:23.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.952+0000 7efd2b7ff700 1 --2- 172.21.15.133:0/2203716506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd24100e20 0x7efd24195f00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:23.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.952+0000 7efd2b7ff700 1 -- 172.21.15.133:0/2203716506 >> 172.21.15.133:0/2203716506 conn(0x7efd240fb210 msgr2=0x7efd24104230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:23.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.952+0000 7efd2b7ff700 1 -- 172.21.15.133:0/2203716506 shutdown_connections 2023-07-14T18:14:23.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:23.952+0000 7efd2b7ff700 1 -- 172.21.15.133:0/2203716506 wait complete. 2023-07-14T18:14:24.014 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T18:14:24.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.169+0000 7fcd8efff700 1 -- 172.21.15.133:0/222296708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd880ffc30 msgr2=0x7fcd88100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:24.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.169+0000 7fcd8efff700 1 --2- 172.21.15.133:0/222296708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd880ffc30 0x7fcd88100040 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcd84009990 tx=0x7fcd84009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:24.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.169+0000 7fcd8efff700 1 -- 172.21.15.133:0/222296708 shutdown_connections 2023-07-14T18:14:24.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.169+0000 7fcd8efff700 1 --2- 172.21.15.133:0/222296708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd881017b0 0x7fcd88108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:24.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.169+0000 7fcd8efff700 1 --2- 172.21.15.133:0/222296708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd880ffc30 0x7fcd88100040 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:24.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.169+0000 7fcd8efff700 1 --2- 172.21.15.133:0/222296708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcd88100e20 0x7fcd88101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:24.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.169+0000 7fcd8efff700 1 -- 172.21.15.133:0/222296708 >> 172.21.15.133:0/222296708 conn(0x7fcd880fb210 msgr2=0x7fcd880fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:24.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.169+0000 7fcd8efff700 1 -- 172.21.15.133:0/222296708 shutdown_connections 2023-07-14T18:14:24.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.169+0000 7fcd8efff700 1 -- 172.21.15.133:0/222296708 wait complete. 2023-07-14T18:14:24.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.170+0000 7fcd8efff700 1 Processor -- start 2023-07-14T18:14:24.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.170+0000 7fcd8efff700 1 -- start start 2023-07-14T18:14:24.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.171+0000 7fcd8efff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd880ffc30 0x7fcd880743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:24.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.171+0000 7fcd8efff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd88100e20 0x7fcd88072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:24.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.171+0000 7fcd8efff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcd881017b0 0x7fcd88072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:24.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.171+0000 7fcd8efff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd88073710 con 0x7fcd880ffc30 2023-07-14T18:14:24.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.171+0000 7fcd8efff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd88073880 con 0x7fcd88100e20 2023-07-14T18:14:24.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.171+0000 7fcd8efff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8819f960 con 0x7fcd881017b0 2023-07-14T18:14:24.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.171+0000 7fcd8cd9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd880ffc30 0x7fcd880743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:24.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.171+0000 7fcd8cd9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd880ffc30 0x7fcd880743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39206/0 (socket says 172.21.15.133:39206) 2023-07-14T18:14:24.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.171+0000 7fcd8cd9b700 1 -- 172.21.15.133:0/4065634544 learned_addr learned my addr 172.21.15.133:0/4065634544 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:24.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.171+0000 7fcd7ffff700 1 --2- 172.21.15.133:0/4065634544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd88100e20 0x7fcd88072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:24.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.171+0000 7fcd8d59c700 1 --2- 172.21.15.133:0/4065634544 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcd881017b0 0x7fcd88072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:24.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.171+0000 7fcd8cd9b700 1 -- 172.21.15.133:0/4065634544 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcd881017b0 msgr2=0x7fcd88072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:24.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.171+0000 7fcd8cd9b700 1 --2- 172.21.15.133:0/4065634544 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcd881017b0 0x7fcd88072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:24.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.171+0000 7fcd8cd9b700 1 -- 172.21.15.133:0/4065634544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd88100e20 msgr2=0x7fcd88072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:24.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.171+0000 7fcd8cd9b700 1 --2- 172.21.15.133:0/4065634544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd88100e20 0x7fcd88072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:24.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.171+0000 7fcd8cd9b700 1 -- 172.21.15.133:0/4065634544 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd840095f0 con 0x7fcd880ffc30 2023-07-14T18:14:24.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.171+0000 7fcd7ffff700 1 --2- 172.21.15.133:0/4065634544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd88100e20 0x7fcd88072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:14:24.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.171+0000 7fcd8cd9b700 1 --2- 172.21.15.133:0/4065634544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd880ffc30 0x7fcd880743f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400b620 tx=0x7fcd84005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:24.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.172+0000 7fcd7dffb700 1 -- 172.21.15.133:0/4065634544 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8401d070 con 0x7fcd880ffc30 2023-07-14T18:14:24.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.172+0000 7fcd8efff700 1 -- 172.21.15.133:0/4065634544 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8819fbe0 con 0x7fcd880ffc30 2023-07-14T18:14:24.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.172+0000 7fcd7dffb700 1 -- 172.21.15.133:0/4065634544 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd840047e0 con 0x7fcd880ffc30 2023-07-14T18:14:24.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.172+0000 7fcd7dffb700 1 -- 172.21.15.133:0/4065634544 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84005560 con 0x7fcd880ffc30 2023-07-14T18:14:24.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.172+0000 7fcd8efff700 1 -- 172.21.15.133:0/4065634544 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd881a0010 con 0x7fcd880ffc30 2023-07-14T18:14:24.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.173+0000 7fcd8efff700 1 -- 172.21.15.133:0/4065634544 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd8810b400 con 0x7fcd880ffc30 2023-07-14T18:14:24.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.174+0000 7fcd7dffb700 1 -- 172.21.15.133:0/4065634544 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fcd84004950 con 0x7fcd880ffc30 2023-07-14T18:14:24.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.176+0000 7fcd7dffb700 1 --2- 172.21.15.133:0/4065634544 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fcd700a11a0 0x7fcd700a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:24.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.176+0000 7fcd7dffb700 1 -- 172.21.15.133:0/4065634544 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fcd840cdc70 con 0x7fcd880ffc30 2023-07-14T18:14:24.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.176+0000 7fcd7ffff700 1 --2- 172.21.15.133:0/4065634544 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fcd700a11a0 0x7fcd700a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:24.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.177+0000 7fcd7ffff700 1 --2- 172.21.15.133:0/4065634544 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fcd700a11a0 0x7fcd700a3650 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcd78009cb0 tx=0x7fcd78009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:24.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.180+0000 7fcd7dffb700 1 -- 172.21.15.133:0/4065634544 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fcd84097200 con 0x7fcd880ffc30 2023-07-14T18:14:24.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:24 smithi161 ceph-mon[128993]: osdmap e88: 12 total, 12 up, 12 in 2023-07-14T18:14:24.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:24 smithi161 ceph-mon[128993]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-07-14T18:14:24.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:24 smithi161 ceph-mon[128993]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T18:14:24.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:24 smithi161 ceph-mon[128993]: pgmap v276: 129 pgs: 10 creating+peering, 104 active+clean, 15 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2023-07-14T18:14:24.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:24 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2706818086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:14:24.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:24 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2203716506' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:14:24.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:24 smithi171 ceph-mon[129241]: osdmap e88: 12 total, 12 up, 12 in 2023-07-14T18:14:24.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:24 smithi171 ceph-mon[129241]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-07-14T18:14:24.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:24 smithi171 ceph-mon[129241]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T18:14:24.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:24 smithi171 ceph-mon[129241]: pgmap v276: 129 pgs: 10 creating+peering, 104 active+clean, 15 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2023-07-14T18:14:24.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:24 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2706818086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:14:24.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:24 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2203716506' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:14:24.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:24 smithi133 ceph-mon[126384]: osdmap e88: 12 total, 12 up, 12 in 2023-07-14T18:14:24.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:24 smithi133 ceph-mon[126384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-07-14T18:14:24.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:24 smithi133 ceph-mon[126384]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T18:14:24.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:24 smithi133 ceph-mon[126384]: pgmap v276: 129 pgs: 10 creating+peering, 104 active+clean, 15 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2023-07-14T18:14:24.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:24 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2706818086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:14:24.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:24 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2203716506' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:14:24.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.380+0000 7fcd8efff700 1 -- 172.21.15.133:0/4065634544 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd88106b00 con 0x7fcd880ffc30 2023-07-14T18:14:24.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.382+0000 7fcd7dffb700 1 -- 172.21.15.133:0/4065634544 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24003 (secure 0 0 0) 0x7fcd8409a820 con 0x7fcd880ffc30 2023-07-14T18:14:24.426 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:14:24.426 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":88,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","created":"2023-07-14T18:06:49.561582+0000","modified":"2023-07-14T18:14:22.524878+0000","last_up_change":"2023-07-14T18:13:29.836099+0000","last_in_change":"2023-07-14T18:13:11.606710+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T18:10:03.945039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T18:14:15.451340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T18:14:16.892215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T18:14:20.493175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"85a29de7-d712-422a-b647-d9e90091aaf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6803","nonce":3076510801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6805","nonce":3076510801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6809","nonce":3076510801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6807","nonce":3076510801}]},"public_addr":"172.21.15.133:6803/3076510801","cluster_addr":"172.21.15.133:6805/3076510801","heartbeat_back_addr":"172.21.15.133:6809/3076510801","heartbeat_front_addr":"172.21.15.133:6807/3076510801","state":["exists","up"]},{"osd":1,"uuid":"80ad4e78-7917-495e-b119-99f7c7c36243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6801","nonce":1841901852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6803","nonce":1841901852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6807","nonce":1841901852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6805","nonce":1841901852}]},"public_addr":"172.21.15.161:6801/1841901852","cluster_addr":"172.21.15.161:6803/1841901852","heartbeat_back_addr":"172.21.15.161:6807/1841901852","heartbeat_front_addr":"172.21.15.161:6805/1841901852","state":["exists","up"]},{"osd":2,"uuid":"fc4bfd37-1fdd-47c6-a48e-6d910daf9e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6801","nonce":346103099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6803","nonce":346103099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6807","nonce":346103099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6805","nonce":346103099}]},"public_addr":"172.21.15.171:6801/346103099","cluster_addr":"172.21.15.171:6803/346103099","heartbeat_back_addr":"172.21.15.171:6807/346103099","heartbeat_front_addr":"172.21.15.171:6805/346103099","state":["exists","up"]},{"osd":3,"uuid":"4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6811","nonce":2152994143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6813","nonce":2152994143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6817","nonce":2152994143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6815","nonce":2152994143}]},"public_addr":"172.21.15.133:6811/2152994143","cluster_addr":"172.21.15.133:6813/2152994143","heartbeat_back_addr":"172.21.15.133:6817/2152994143","heartbeat_front_addr":"172.21.15.133:6815/2152994143","state":["exists","up"]},{"osd":4,"uuid":"4601a1c2-254b-4c4d-9b15-8fc952b3d2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6809","nonce":47213291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6811","nonce":47213291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6815","nonce":47213291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6813","nonce":47213291}]},"public_addr":"172.21.15.161:6809/47213291","cluster_addr":"172.21.15.161:6811/47213291","heartbeat_back_addr":"172.21.15.161:6815/47213291","heartbeat_front_addr":"172.21.15.161:6813/47213291","state":["exists","up"]},{"osd":5,"uuid":"6cca180a-9140-4e49-835b-86842dae0574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6809","nonce":777556751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6811","nonce":777556751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6815","nonce":777556751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6813","nonce":777556751}]},"public_addr":"172.21.15.171:6809/777556751","cluster_addr":"172.21.15.171:6811/777556751","heartbeat_back_addr":"172.21.15.171:6815/777556751","heartbeat_front_addr":"172.21.15.171:6813/777556751","state":["exists","up"]},{"osd":6,"uuid":"ea0299ff-7740-4494-a3a9-1503b3fc0ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6819","nonce":2884152358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6821","nonce":2884152358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6825","nonce":2884152358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6823","nonce":2884152358}]},"public_addr":"172.21.15.133:6819/2884152358","cluster_addr":"172.21.15.133:6821/2884152358","heartbeat_back_addr":"172.21.15.133:6825/2884152358","heartbeat_front_addr":"172.21.15.133:6823/2884152358","state":["exists","up"]},{"osd":7,"uuid":"b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6817","nonce":4183705749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6819","nonce":4183705749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6823","nonce":4183705749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6821","nonce":4183705749}]},"public_addr":"172.21.15.161:6817/4183705749","cluster_addr":"172.21.15.161:6819/4183705749","heartbeat_back_addr":"172.21.15.161:6823/4183705749","heartbeat_front_addr":"172.21.15.161:6821/4183705749","state":["exists","up"]},{"osd":8,"uuid":"87dffc07-bf48-48c8-b418-5c4b509686d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6817","nonce":2771494483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6819","nonce":2771494483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6823","nonce":2771494483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6821","nonce":2771494483}]},"public_addr":"172.21.15.171:6817/2771494483","cluster_addr":"172.21.15.171:6819/2771494483","heartbeat_back_addr":"172.21.15.171:6823/2771494483","heartbeat_front_addr":"172.21.15.171:6821/2771494483","state":["exists","up"]},{"osd":9,"uuid":"e0832870-79ce-4bce-8045-58b7a7f6dc69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6827","nonce":2066164149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6829","nonce":2066164149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6833","nonce":2066164149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6831","nonce":2066164149}]},"public_addr":"172.21.15.133:6827/2066164149","cluster_addr":"172.21.15.133:6829/2066164149","heartbeat_back_addr":"172.21.15.133:6833/2066164149","heartbeat_front_addr":"172.21.15.133:6831/2066164149","state":["exists","up"]},{"osd":10,"uuid":"081f0484-d134-4b2e-9c48-ef8e6a646422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6825","nonce":2162333626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6827","nonce":2162333626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6831","nonce":2162333626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6829","nonce":2162333626}]},"public_addr":"172.21.15.161:6825/2162333626","cluster_addr":"172.21.15.161:6827/2162333626","heartbeat_back_addr":"172.21.15.161:6831/2162333626","heartbeat_front_addr":"172.21.15.161:6829/2162333626","state":["exists","up"]},{"osd":11,"uuid":"4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6825","nonce":3403894313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6827","nonce":3403894313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6831","nonce":3403894313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6829","nonce":3403894313}]},"public_addr":"172.21.15.171:6825/3403894313","cluster_addr":"172.21.15.171:6827/3403894313","heartbeat_back_addr":"172.21.15.171:6831/3403894313","heartbeat_front_addr":"172.21.15.171:6829/3403894313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:17.816229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:39.638543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:00.543317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:26.988921+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:48.644694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:10.049860+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:35.986508+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:57.753846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:19.018245+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:44.460450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:06.279960+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:27.955006+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/1832569241":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6801/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:0/4219815304":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:6801/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/432253615":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1840700480":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1807775439":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1089030734":"2023-07-15T18:07:47.346704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T18:14:24.427 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.384+0000 7fcd8efff700 1 -- 172.21.15.133:0/4065634544 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fcd700a11a0 msgr2=0x7fcd700a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:24.428 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.384+0000 7fcd8efff700 1 --2- 172.21.15.133:0/4065634544 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fcd700a11a0 0x7fcd700a3650 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcd78009cb0 tx=0x7fcd78009260 comp rx=0 tx=0).stop 2023-07-14T18:14:24.428 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.384+0000 7fcd8efff700 1 -- 172.21.15.133:0/4065634544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd880ffc30 msgr2=0x7fcd880743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:24.428 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.384+0000 7fcd8efff700 1 --2- 172.21.15.133:0/4065634544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd880ffc30 0x7fcd880743f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400b620 tx=0x7fcd84005ef0 comp rx=0 tx=0).stop 2023-07-14T18:14:24.428 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.385+0000 7fcd8efff700 1 -- 172.21.15.133:0/4065634544 shutdown_connections 2023-07-14T18:14:24.428 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.385+0000 7fcd8efff700 1 --2- 172.21.15.133:0/4065634544 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fcd700a11a0 0x7fcd700a3650 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:24.428 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.385+0000 7fcd8efff700 1 --2- 172.21.15.133:0/4065634544 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcd881017b0 0x7fcd88072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:24.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.385+0000 7fcd8efff700 1 --2- 172.21.15.133:0/4065634544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd880ffc30 0x7fcd880743f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:24.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.385+0000 7fcd8efff700 1 --2- 172.21.15.133:0/4065634544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd88100e20 0x7fcd88072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:24.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.385+0000 7fcd8efff700 1 -- 172.21.15.133:0/4065634544 >> 172.21.15.133:0/4065634544 conn(0x7fcd880fb210 msgr2=0x7fcd880fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:24.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.385+0000 7fcd8efff700 1 -- 172.21.15.133:0/4065634544 shutdown_connections 2023-07-14T18:14:24.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.385+0000 7fcd8efff700 1 -- 172.21.15.133:0/4065634544 wait complete. 2023-07-14T18:14:24.464 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T18:14:24.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.618+0000 7f2f07fff700 1 -- 172.21.15.133:0/324140791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f00107af0 msgr2=0x7f2f00109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:24.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.618+0000 7f2f07fff700 1 --2- 172.21.15.133:0/324140791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f00107af0 0x7f2f00109f70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800b0a0 tx=0x7f2ef800b3b0 comp rx=0 tx=0).stop 2023-07-14T18:14:24.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.618+0000 7f2f07fff700 1 -- 172.21.15.133:0/324140791 shutdown_connections 2023-07-14T18:14:24.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.618+0000 7f2f07fff700 1 --2- 172.21.15.133:0/324140791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f00074680 0x7f2f00072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:24.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.618+0000 7f2f07fff700 1 --2- 172.21.15.133:0/324140791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f00107af0 0x7f2f00109f70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:24.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.618+0000 7f2f07fff700 1 --2- 172.21.15.133:0/324140791 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f000730a0 0x7f2f00073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:24.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.618+0000 7f2f07fff700 1 -- 172.21.15.133:0/324140791 >> 172.21.15.133:0/324140791 conn(0x7f2f000fb210 msgr2=0x7f2f000fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:24.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.618+0000 7f2f07fff700 1 -- 172.21.15.133:0/324140791 shutdown_connections 2023-07-14T18:14:24.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.618+0000 7f2f07fff700 1 -- 172.21.15.133:0/324140791 wait complete. 2023-07-14T18:14:24.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.619+0000 7f2f07fff700 1 Processor -- start 2023-07-14T18:14:24.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.619+0000 7f2f07fff700 1 -- start start 2023-07-14T18:14:24.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.619+0000 7f2f07fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f000730a0 0x7f2f00197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:24.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.619+0000 7f2f07fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f00074680 0x7f2f001981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:24.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.619+0000 7f2f07fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f00107af0 0x7f2f0019f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:24.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.619+0000 7f2f07fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0019f800 con 0x7f2f000730a0 2023-07-14T18:14:24.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.619+0000 7f2f07fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0019f970 con 0x7f2f00074680 2023-07-14T18:14:24.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.619+0000 7f2f07fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0019fb30 con 0x7f2f00107af0 2023-07-14T18:14:24.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.620+0000 7f2f05d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f000730a0 0x7f2f00197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:24.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.620+0000 7f2f0559a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f00074680 0x7f2f001981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:24.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.620+0000 7f2f05d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f000730a0 0x7f2f00197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39226/0 (socket says 172.21.15.133:39226) 2023-07-14T18:14:24.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.620+0000 7f2f05d9b700 1 -- 172.21.15.133:0/914812907 learned_addr learned my addr 172.21.15.133:0/914812907 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:24.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.620+0000 7f2f0659c700 1 --2- 172.21.15.133:0/914812907 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f00107af0 0x7f2f0019f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:24.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.620+0000 7f2f05d9b700 1 -- 172.21.15.133:0/914812907 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f00107af0 msgr2=0x7f2f0019f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:24.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.620+0000 7f2f05d9b700 1 --2- 172.21.15.133:0/914812907 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f00107af0 0x7f2f0019f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:24.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.620+0000 7f2f05d9b700 1 -- 172.21.15.133:0/914812907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f00074680 msgr2=0x7f2f001981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:24.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.620+0000 7f2f05d9b700 1 --2- 172.21.15.133:0/914812907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f00074680 0x7f2f001981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:24.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.620+0000 7f2f05d9b700 1 -- 172.21.15.133:0/914812907 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef8009d00 con 0x7f2f000730a0 2023-07-14T18:14:24.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.620+0000 7f2f05d9b700 1 --2- 172.21.15.133:0/914812907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f000730a0 0x7f2f00197c90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2efc00d980 tx=0x7f2efc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:24.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.620+0000 7f2ef6ffd700 1 -- 172.21.15.133:0/914812907 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc00f890 con 0x7f2f000730a0 2023-07-14T18:14:24.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.620+0000 7f2ef6ffd700 1 -- 172.21.15.133:0/914812907 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2efc010460 con 0x7f2f000730a0 2023-07-14T18:14:24.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.620+0000 7f2f07fff700 1 -- 172.21.15.133:0/914812907 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f0019fe10 con 0x7f2f000730a0 2023-07-14T18:14:24.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.621+0000 7f2ef6ffd700 1 -- 172.21.15.133:0/914812907 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc00e610 con 0x7f2f000730a0 2023-07-14T18:14:24.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.621+0000 7f2f07fff700 1 -- 172.21.15.133:0/914812907 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f001a0290 con 0x7f2f000730a0 2023-07-14T18:14:24.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.622+0000 7f2f07fff700 1 -- 172.21.15.133:0/914812907 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f001068b0 con 0x7f2f000730a0 2023-07-14T18:14:24.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.624+0000 7f2ef6ffd700 1 -- 172.21.15.133:0/914812907 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f2efc0105d0 con 0x7f2f000730a0 2023-07-14T18:14:24.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.625+0000 7f2ef6ffd700 1 --2- 172.21.15.133:0/914812907 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f2eec0a1220 0x7f2eec0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:24.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.625+0000 7f2ef6ffd700 1 -- 172.21.15.133:0/914812907 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2efc0cc7f0 con 0x7f2f000730a0 2023-07-14T18:14:24.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.625+0000 7f2f0559a700 1 --2- 172.21.15.133:0/914812907 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f2eec0a1220 0x7f2eec0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:24.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.626+0000 7f2f0559a700 1 --2- 172.21.15.133:0/914812907 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f2eec0a1220 0x7f2eec0a36d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2f00199190 tx=0x7f2ef0007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:24.671 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.628+0000 7f2ef6ffd700 1 -- 172.21.15.133:0/914812907 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f2efc095cf0 con 0x7f2f000730a0 2023-07-14T18:14:24.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.873+0000 7f2f07fff700 1 -- 172.21.15.133:0/914812907 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f0002c8e0 con 0x7f2f000730a0 2023-07-14T18:14:24.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.874+0000 7f2ef6ffd700 1 -- 172.21.15.133:0/914812907 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10790 (secure 0 0 0) 0x7f2efc099310 con 0x7f2f000730a0 2023-07-14T18:14:24.917 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:14:24.917 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3168151271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3168151271},{"type":"v1","addr":"172.21.15.133:6837","nonce":3168151271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14544,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/3093144261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3093144261},{"type":"v1","addr":"172.21.15.133:6839","nonce":3093144261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14556,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/3282902068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":3282902068},{"type":"v1","addr":"172.21.15.133:6841","nonce":3282902068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/4023127259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4023127259},{"type":"v1","addr":"172.21.15.161:6833","nonce":4023127259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2722373740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2722373740},{"type":"v1","addr":"172.21.15.171:6833","nonce":2722373740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2399005074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2399005074},{"type":"v1","addr":"172.21.15.133:6835","nonce":2399005074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3725502564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3725502564},{"type":"v1","addr":"172.21.15.161:6835","nonce":3725502564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1877362116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1877362116},{"type":"v1","addr":"172.21.15.171:6835","nonce":1877362116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3458064798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3458064798},{"type":"v1","addr":"172.21.15.161:6837","nonce":3458064798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/106205332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":106205332},{"type":"v1","addr":"172.21.15.171:6837","nonce":106205332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24541,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/816201198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":816201198},{"type":"v1","addr":"172.21.15.171:6839","nonce":816201198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T18:14:17.440873+0000","modified":"2023-07-14T18:14:23.035911+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/2503865871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2503865871},{"type":"v1","addr":"172.21.15.161:6839","nonce":2503865871}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-07-14T18:14:24.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.876+0000 7f2f07fff700 1 -- 172.21.15.133:0/914812907 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f2eec0a1220 msgr2=0x7f2eec0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:24.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.876+0000 7f2f07fff700 1 --2- 172.21.15.133:0/914812907 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f2eec0a1220 0x7f2eec0a36d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2f00199190 tx=0x7f2ef0007400 comp rx=0 tx=0).stop 2023-07-14T18:14:24.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.876+0000 7f2f07fff700 1 -- 172.21.15.133:0/914812907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f000730a0 msgr2=0x7f2f00197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:24.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.876+0000 7f2f07fff700 1 --2- 172.21.15.133:0/914812907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f000730a0 0x7f2f00197c90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2efc00d980 tx=0x7f2efc00dc90 comp rx=0 tx=0).stop 2023-07-14T18:14:24.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.876+0000 7f2f07fff700 1 -- 172.21.15.133:0/914812907 shutdown_connections 2023-07-14T18:14:24.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.876+0000 7f2f07fff700 1 --2- 172.21.15.133:0/914812907 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f2eec0a1220 0x7f2eec0a36d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:24.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.876+0000 7f2f07fff700 1 --2- 172.21.15.133:0/914812907 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f00107af0 0x7f2f0019f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:24.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.876+0000 7f2f07fff700 1 --2- 172.21.15.133:0/914812907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f000730a0 0x7f2f00197c90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:24.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.876+0000 7f2f07fff700 1 --2- 172.21.15.133:0/914812907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f00074680 0x7f2f001981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:24.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.876+0000 7f2f07fff700 1 -- 172.21.15.133:0/914812907 >> 172.21.15.133:0/914812907 conn(0x7f2f000fb210 msgr2=0x7f2f001039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:24.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.876+0000 7f2f07fff700 1 -- 172.21.15.133:0/914812907 shutdown_connections 2023-07-14T18:14:24.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:24.876+0000 7f2f07fff700 1 -- 172.21.15.133:0/914812907 wait complete. 2023-07-14T18:14:24.921 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 17 2023-07-14T18:14:24.939 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T18:14:25.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.097+0000 7fdaf29ff700 1 -- 172.21.15.133:0/3079282402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdaec1017b0 msgr2=0x7fdaec108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:25.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.097+0000 7fdaf29ff700 1 --2- 172.21.15.133:0/3079282402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdaec1017b0 0x7fdaec108070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdae400b0a0 tx=0x7fdae400b3b0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.097+0000 7fdaf29ff700 1 -- 172.21.15.133:0/3079282402 shutdown_connections 2023-07-14T18:14:25.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.097+0000 7fdaf29ff700 1 --2- 172.21.15.133:0/3079282402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaec0ffc30 0x7fdaec100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.097+0000 7fdaf29ff700 1 --2- 172.21.15.133:0/3079282402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdaec1017b0 0x7fdaec108070 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.097+0000 7fdaf29ff700 1 --2- 172.21.15.133:0/3079282402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdaec100e20 0x7fdaec101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.097+0000 7fdaf29ff700 1 -- 172.21.15.133:0/3079282402 >> 172.21.15.133:0/3079282402 conn(0x7fdaec0fb210 msgr2=0x7fdaec0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:25.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.097+0000 7fdaf29ff700 1 -- 172.21.15.133:0/3079282402 shutdown_connections 2023-07-14T18:14:25.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.097+0000 7fdaf29ff700 1 -- 172.21.15.133:0/3079282402 wait complete. 2023-07-14T18:14:25.143 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.098+0000 7fdaf29ff700 1 Processor -- start 2023-07-14T18:14:25.143 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.098+0000 7fdaf29ff700 1 -- start start 2023-07-14T18:14:25.143 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.098+0000 7fdaf29ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaec0ffc30 0x7fdaec0743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:25.143 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.098+0000 7fdaf29ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdaec100e20 0x7fdaec072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:25.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.098+0000 7fdaf29ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdaec1017b0 0x7fdaec072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:25.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.098+0000 7fdaf29ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec073710 con 0x7fdaec100e20 2023-07-14T18:14:25.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.098+0000 7fdaf29ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec073880 con 0x7fdaec0ffc30 2023-07-14T18:14:25.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.098+0000 7fdaf29ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec19fa90 con 0x7fdaec1017b0 2023-07-14T18:14:25.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.098+0000 7fdaeb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdaec100e20 0x7fdaec072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:25.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.098+0000 7fdaeb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdaec100e20 0x7fdaec072a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39236/0 (socket says 172.21.15.133:39236) 2023-07-14T18:14:25.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.098+0000 7fdaeb7fe700 1 -- 172.21.15.133:0/2011446169 learned_addr learned my addr 172.21.15.133:0/2011446169 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:25.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.098+0000 7fdaebfff700 1 --2- 172.21.15.133:0/2011446169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaec0ffc30 0x7fdaec0743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:25.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.099+0000 7fdaf0f9c700 1 --2- 172.21.15.133:0/2011446169 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdaec1017b0 0x7fdaec072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:25.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.099+0000 7fdaeb7fe700 1 -- 172.21.15.133:0/2011446169 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdaec1017b0 msgr2=0x7fdaec072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:25.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.099+0000 7fdaeb7fe700 1 --2- 172.21.15.133:0/2011446169 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdaec1017b0 0x7fdaec072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.099+0000 7fdaeb7fe700 1 -- 172.21.15.133:0/2011446169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaec0ffc30 msgr2=0x7fdaec0743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:25.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.099+0000 7fdaeb7fe700 1 --2- 172.21.15.133:0/2011446169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaec0ffc30 0x7fdaec0743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.099+0000 7fdaeb7fe700 1 -- 172.21.15.133:0/2011446169 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae4009d00 con 0x7fdaec100e20 2023-07-14T18:14:25.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.099+0000 7fdaeb7fe700 1 --2- 172.21.15.133:0/2011446169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdaec100e20 0x7fdaec072a30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdae000e9a0 tx=0x7fdae000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:25.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.099+0000 7fdae97fa700 1 -- 172.21.15.133:0/2011446169 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0009790 con 0x7fdaec100e20 2023-07-14T18:14:25.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.099+0000 7fdae97fa700 1 -- 172.21.15.133:0/2011446169 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdae000cbd0 con 0x7fdaec100e20 2023-07-14T18:14:25.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.099+0000 7fdaf29ff700 1 -- 172.21.15.133:0/2011446169 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaec19fd70 con 0x7fdaec100e20 2023-07-14T18:14:25.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.099+0000 7fdae97fa700 1 -- 172.21.15.133:0/2011446169 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0018680 con 0x7fdaec100e20 2023-07-14T18:14:25.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.099+0000 7fdaf29ff700 1 -- 172.21.15.133:0/2011446169 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaec1a0220 con 0x7fdaec100e20 2023-07-14T18:14:25.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.100+0000 7fdaf29ff700 1 -- 172.21.15.133:0/2011446169 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaec1070f0 con 0x7fdaec100e20 2023-07-14T18:14:25.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.102+0000 7fdae97fa700 1 -- 172.21.15.133:0/2011446169 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fdae000cd40 con 0x7fdaec100e20 2023-07-14T18:14:25.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.103+0000 7fdae97fa700 1 --2- 172.21.15.133:0/2011446169 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fdad40a11a0 0x7fdad40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:25.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.104+0000 7fdae97fa700 1 -- 172.21.15.133:0/2011446169 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fdae0014070 con 0x7fdaec100e20 2023-07-14T18:14:25.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.104+0000 7fdaebfff700 1 --2- 172.21.15.133:0/2011446169 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fdad40a11a0 0x7fdad40a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:25.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.104+0000 7fdaebfff700 1 --2- 172.21.15.133:0/2011446169 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fdad40a11a0 0x7fdad40a3650 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdaec100c80 tx=0x7fdadc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:25.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.107+0000 7fdae97fa700 1 -- 172.21.15.133:0/2011446169 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fdae0051780 con 0x7fdaec100e20 2023-07-14T18:14:25.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:25 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/4065634544' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:14:25.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:25 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/914812907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:14:25.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.310+0000 7fdaf29ff700 1 -- 172.21.15.133:0/2011446169 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdaec02c8e0 con 0x7fdaec100e20 2023-07-14T18:14:25.354 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.311+0000 7fdae97fa700 1 -- 172.21.15.133:0/2011446169 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24003 (secure 0 0 0) 0x7fdae0096300 con 0x7fdaec100e20 2023-07-14T18:14:25.355 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:14:25.355 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":88,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","created":"2023-07-14T18:06:49.561582+0000","modified":"2023-07-14T18:14:22.524878+0000","last_up_change":"2023-07-14T18:13:29.836099+0000","last_in_change":"2023-07-14T18:13:11.606710+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T18:10:03.945039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T18:14:15.451340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T18:14:16.892215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T18:14:20.493175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"85a29de7-d712-422a-b647-d9e90091aaf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6803","nonce":3076510801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6805","nonce":3076510801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6809","nonce":3076510801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6807","nonce":3076510801}]},"public_addr":"172.21.15.133:6803/3076510801","cluster_addr":"172.21.15.133:6805/3076510801","heartbeat_back_addr":"172.21.15.133:6809/3076510801","heartbeat_front_addr":"172.21.15.133:6807/3076510801","state":["exists","up"]},{"osd":1,"uuid":"80ad4e78-7917-495e-b119-99f7c7c36243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6801","nonce":1841901852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6803","nonce":1841901852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6807","nonce":1841901852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6805","nonce":1841901852}]},"public_addr":"172.21.15.161:6801/1841901852","cluster_addr":"172.21.15.161:6803/1841901852","heartbeat_back_addr":"172.21.15.161:6807/1841901852","heartbeat_front_addr":"172.21.15.161:6805/1841901852","state":["exists","up"]},{"osd":2,"uuid":"fc4bfd37-1fdd-47c6-a48e-6d910daf9e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6801","nonce":346103099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6803","nonce":346103099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6807","nonce":346103099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6805","nonce":346103099}]},"public_addr":"172.21.15.171:6801/346103099","cluster_addr":"172.21.15.171:6803/346103099","heartbeat_back_addr":"172.21.15.171:6807/346103099","heartbeat_front_addr":"172.21.15.171:6805/346103099","state":["exists","up"]},{"osd":3,"uuid":"4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6811","nonce":2152994143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6813","nonce":2152994143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6817","nonce":2152994143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6815","nonce":2152994143}]},"public_addr":"172.21.15.133:6811/2152994143","cluster_addr":"172.21.15.133:6813/2152994143","heartbeat_back_addr":"172.21.15.133:6817/2152994143","heartbeat_front_addr":"172.21.15.133:6815/2152994143","state":["exists","up"]},{"osd":4,"uuid":"4601a1c2-254b-4c4d-9b15-8fc952b3d2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6809","nonce":47213291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6811","nonce":47213291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6815","nonce":47213291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6813","nonce":47213291}]},"public_addr":"172.21.15.161:6809/47213291","cluster_addr":"172.21.15.161:6811/47213291","heartbeat_back_addr":"172.21.15.161:6815/47213291","heartbeat_front_addr":"172.21.15.161:6813/47213291","state":["exists","up"]},{"osd":5,"uuid":"6cca180a-9140-4e49-835b-86842dae0574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6809","nonce":777556751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6811","nonce":777556751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6815","nonce":777556751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6813","nonce":777556751}]},"public_addr":"172.21.15.171:6809/777556751","cluster_addr":"172.21.15.171:6811/777556751","heartbeat_back_addr":"172.21.15.171:6815/777556751","heartbeat_front_addr":"172.21.15.171:6813/777556751","state":["exists","up"]},{"osd":6,"uuid":"ea0299ff-7740-4494-a3a9-1503b3fc0ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6819","nonce":2884152358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6821","nonce":2884152358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6825","nonce":2884152358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6823","nonce":2884152358}]},"public_addr":"172.21.15.133:6819/2884152358","cluster_addr":"172.21.15.133:6821/2884152358","heartbeat_back_addr":"172.21.15.133:6825/2884152358","heartbeat_front_addr":"172.21.15.133:6823/2884152358","state":["exists","up"]},{"osd":7,"uuid":"b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6817","nonce":4183705749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6819","nonce":4183705749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6823","nonce":4183705749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6821","nonce":4183705749}]},"public_addr":"172.21.15.161:6817/4183705749","cluster_addr":"172.21.15.161:6819/4183705749","heartbeat_back_addr":"172.21.15.161:6823/4183705749","heartbeat_front_addr":"172.21.15.161:6821/4183705749","state":["exists","up"]},{"osd":8,"uuid":"87dffc07-bf48-48c8-b418-5c4b509686d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6817","nonce":2771494483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6819","nonce":2771494483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6823","nonce":2771494483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6821","nonce":2771494483}]},"public_addr":"172.21.15.171:6817/2771494483","cluster_addr":"172.21.15.171:6819/2771494483","heartbeat_back_addr":"172.21.15.171:6823/2771494483","heartbeat_front_addr":"172.21.15.171:6821/2771494483","state":["exists","up"]},{"osd":9,"uuid":"e0832870-79ce-4bce-8045-58b7a7f6dc69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6827","nonce":2066164149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6829","nonce":2066164149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6833","nonce":2066164149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6831","nonce":2066164149}]},"public_addr":"172.21.15.133:6827/2066164149","cluster_addr":"172.21.15.133:6829/2066164149","heartbeat_back_addr":"172.21.15.133:6833/2066164149","heartbeat_front_addr":"172.21.15.133:6831/2066164149","state":["exists","up"]},{"osd":10,"uuid":"081f0484-d134-4b2e-9c48-ef8e6a646422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6825","nonce":2162333626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6827","nonce":2162333626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6831","nonce":2162333626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6829","nonce":2162333626}]},"public_addr":"172.21.15.161:6825/2162333626","cluster_addr":"172.21.15.161:6827/2162333626","heartbeat_back_addr":"172.21.15.161:6831/2162333626","heartbeat_front_addr":"172.21.15.161:6829/2162333626","state":["exists","up"]},{"osd":11,"uuid":"4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6825","nonce":3403894313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6827","nonce":3403894313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6831","nonce":3403894313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6829","nonce":3403894313}]},"public_addr":"172.21.15.171:6825/3403894313","cluster_addr":"172.21.15.171:6827/3403894313","heartbeat_back_addr":"172.21.15.171:6831/3403894313","heartbeat_front_addr":"172.21.15.171:6829/3403894313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:17.816229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:39.638543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:00.543317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:26.988921+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:48.644694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:10.049860+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:35.986508+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:57.753846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:19.018245+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:44.460450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:06.279960+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:27.955006+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/1832569241":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6801/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:0/4219815304":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:6801/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/432253615":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1840700480":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1807775439":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1089030734":"2023-07-15T18:07:47.346704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T18:14:25.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.314+0000 7fdaf29ff700 1 -- 172.21.15.133:0/2011446169 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fdad40a11a0 msgr2=0x7fdad40a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:25.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.314+0000 7fdaf29ff700 1 --2- 172.21.15.133:0/2011446169 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fdad40a11a0 0x7fdad40a3650 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdaec100c80 tx=0x7fdadc009260 comp rx=0 tx=0).stop 2023-07-14T18:14:25.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.314+0000 7fdaf29ff700 1 -- 172.21.15.133:0/2011446169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdaec100e20 msgr2=0x7fdaec072a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:25.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.314+0000 7fdaf29ff700 1 --2- 172.21.15.133:0/2011446169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdaec100e20 0x7fdaec072a30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdae000e9a0 tx=0x7fdae000ecb0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.314+0000 7fdaf29ff700 1 -- 172.21.15.133:0/2011446169 shutdown_connections 2023-07-14T18:14:25.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.314+0000 7fdaf29ff700 1 --2- 172.21.15.133:0/2011446169 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fdad40a11a0 0x7fdad40a3650 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.314+0000 7fdaf29ff700 1 --2- 172.21.15.133:0/2011446169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaec0ffc30 0x7fdaec0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.314+0000 7fdaf29ff700 1 --2- 172.21.15.133:0/2011446169 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdaec1017b0 0x7fdaec072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.314+0000 7fdaf29ff700 1 --2- 172.21.15.133:0/2011446169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdaec100e20 0x7fdaec072a30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.314+0000 7fdaf29ff700 1 -- 172.21.15.133:0/2011446169 >> 172.21.15.133:0/2011446169 conn(0x7fdaec0fb210 msgr2=0x7fdaec104230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:25.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.314+0000 7fdaf29ff700 1 -- 172.21.15.133:0/2011446169 shutdown_connections 2023-07-14T18:14:25.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.314+0000 7fdaf29ff700 1 -- 172.21.15.133:0/2011446169 wait complete. 2023-07-14T18:14:25.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:25 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/4065634544' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:14:25.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:25 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/914812907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:14:25.374 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:25 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/4065634544' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:14:25.374 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:25 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/914812907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:14:25.377 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-14T18:14:17.440873+0000', 'modified': '2023-07-14T18:14:23.035911+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24557}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24557': {'gid': 24557, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6839/2503865871', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 2503865871}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 2503865871}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-07-14T18:14:25.377 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-14T18:14:25.378 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-14T18:14:25.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.533+0000 7f2ac29ff700 1 -- 172.21.15.133:0/722019262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2abc1017a0 msgr2=0x7f2abc108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:25.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.533+0000 7f2ac29ff700 1 --2- 172.21.15.133:0/722019262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2abc1017a0 0x7f2abc108060 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2ab400b0a0 tx=0x7f2ab400b3b0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.533+0000 7f2ac29ff700 1 -- 172.21.15.133:0/722019262 shutdown_connections 2023-07-14T18:14:25.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.533+0000 7f2ac29ff700 1 --2- 172.21.15.133:0/722019262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2abc1017a0 0x7f2abc108060 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.533+0000 7f2ac29ff700 1 --2- 172.21.15.133:0/722019262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2abc0ffc20 0x7f2abc100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.533+0000 7f2ac29ff700 1 --2- 172.21.15.133:0/722019262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2abc100e10 0x7f2abc101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.533+0000 7f2ac29ff700 1 -- 172.21.15.133:0/722019262 >> 172.21.15.133:0/722019262 conn(0x7f2abc0fb260 msgr2=0x7f2abc0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:25.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.533+0000 7f2ac29ff700 1 -- 172.21.15.133:0/722019262 shutdown_connections 2023-07-14T18:14:25.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.533+0000 7f2ac29ff700 1 -- 172.21.15.133:0/722019262 wait complete. 2023-07-14T18:14:25.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.534+0000 7f2ac29ff700 1 Processor -- start 2023-07-14T18:14:25.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.534+0000 7f2ac29ff700 1 -- start start 2023-07-14T18:14:25.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.534+0000 7f2ac29ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2abc0ffc20 0x7f2abc195a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:25.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.534+0000 7f2ac29ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2abc100e10 0x7f2abc195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:25.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.534+0000 7f2ac29ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2abc1017a0 0x7f2abc19d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:25.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.534+0000 7f2ac29ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc19d5f0 con 0x7f2abc1017a0 2023-07-14T18:14:25.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.534+0000 7f2ac29ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc19d730 con 0x7f2abc100e10 2023-07-14T18:14:25.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.534+0000 7f2ac29ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc19d8c0 con 0x7f2abc0ffc20 2023-07-14T18:14:25.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.535+0000 7f2abb7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2abc100e10 0x7f2abc195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:25.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.535+0000 7f2ac0f9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2abc1017a0 0x7f2abc19d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:25.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.535+0000 7f2abb7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2abc100e10 0x7f2abc195fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:34754/0 (socket says 172.21.15.133:34754) 2023-07-14T18:14:25.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.535+0000 7f2ac0f9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2abc1017a0 0x7f2abc19d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39268/0 (socket says 172.21.15.133:39268) 2023-07-14T18:14:25.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.535+0000 7f2abb7fe700 1 -- 172.21.15.133:0/3303853976 learned_addr learned my addr 172.21.15.133:0/3303853976 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:25.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.535+0000 7f2abbfff700 1 --2- 172.21.15.133:0/3303853976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2abc0ffc20 0x7f2abc195a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:25.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.535+0000 7f2ac0f9c700 1 -- 172.21.15.133:0/3303853976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2abc0ffc20 msgr2=0x7f2abc195a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:25.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.535+0000 7f2ac0f9c700 1 --2- 172.21.15.133:0/3303853976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2abc0ffc20 0x7f2abc195a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.535+0000 7f2ac0f9c700 1 -- 172.21.15.133:0/3303853976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2abc100e10 msgr2=0x7f2abc195fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:25.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.535+0000 7f2ac0f9c700 1 --2- 172.21.15.133:0/3303853976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2abc100e10 0x7f2abc195fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.535+0000 7f2ac0f9c700 1 -- 172.21.15.133:0/3303853976 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aac009520 con 0x7f2abc1017a0 2023-07-14T18:14:25.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.535+0000 7f2ac0f9c700 1 --2- 172.21.15.133:0/3303853976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2abc1017a0 0x7f2abc19d020 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2ab400b070 tx=0x7f2ab400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:25.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.535+0000 7f2ab97fa700 1 -- 172.21.15.133:0/3303853976 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab400e040 con 0x7f2abc1017a0 2023-07-14T18:14:25.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.535+0000 7f2ac29ff700 1 -- 172.21.15.133:0/3303853976 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab4009d00 con 0x7f2abc1017a0 2023-07-14T18:14:25.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.536+0000 7f2ac29ff700 1 -- 172.21.15.133:0/3303853976 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2abc19de30 con 0x7f2abc1017a0 2023-07-14T18:14:25.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.536+0000 7f2ab97fa700 1 -- 172.21.15.133:0/3303853976 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ab4009480 con 0x7f2abc1017a0 2023-07-14T18:14:25.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.536+0000 7f2ab97fa700 1 -- 172.21.15.133:0/3303853976 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab40052a0 con 0x7f2abc1017a0 2023-07-14T18:14:25.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.539+0000 7f2ab97fa700 1 -- 172.21.15.133:0/3303853976 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f2ab401d480 con 0x7f2abc1017a0 2023-07-14T18:14:25.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.540+0000 7f2ad629eb80 1 Processor -- start 2023-07-14T18:14:25.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.540+0000 7f2ad629eb80 1 -- start start 2023-07-14T18:14:25.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.540+0000 7f2ab97fa700 1 --2- 172.21.15.133:0/3303853976 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f2aa40a11d0 0x7f2aa40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:25.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.540+0000 7f2ab97fa700 1 -- 172.21.15.133:0/3303853976 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2ab40cdb30 con 0x7f2abc1017a0 2023-07-14T18:14:25.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.540+0000 7f2ad629eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635ca349f10 0x5635ca34a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:25.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.541+0000 7f2ad629eb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635ca34a860 0x5635ca34ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:25.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.541+0000 7f2abb7fe700 1 --2- 172.21.15.133:0/3303853976 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f2aa40a11d0 0x7f2aa40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:25.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.541+0000 7f2ad629eb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635ca34b460 0x5635ca34b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:25.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.541+0000 7f2ad629eb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635ca24ec60 con 0x5635ca34a860 2023-07-14T18:14:25.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.541+0000 7f2ad629eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635ca2a19f0 con 0x5635ca349f10 2023-07-14T18:14:25.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.541+0000 7f2ad629eb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635ca29ae80 con 0x5635ca34b460 2023-07-14T18:14:25.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.541+0000 7f2abb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635ca34a860 0x5635ca34ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:25.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.541+0000 7f2abbfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635ca349f10 0x5635ca34a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:25.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.541+0000 7f2abbfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635ca349f10 0x5635ca34a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:34768/0 (socket says 172.21.15.133:34768) 2023-07-14T18:14:25.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.541+0000 7f2abbfff700 1 -- 172.21.15.133:0/1527421670 learned_addr learned my addr 172.21.15.133:0/1527421670 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:25.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.541+0000 7f2ac0f9c700 1 --2- 172.21.15.133:0/1527421670 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635ca34b460 0x5635ca34b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:25.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.541+0000 7f2abbfff700 1 -- 172.21.15.133:0/1527421670 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635ca34b460 msgr2=0x5635ca34b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:14:25.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.541+0000 7f2abbfff700 1 --2- 172.21.15.133:0/1527421670 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635ca34b460 0x5635ca34b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.541+0000 7f2abbfff700 1 -- 172.21.15.133:0/1527421670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635ca34a860 msgr2=0x5635ca34ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:14:25.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.541+0000 7f2abbfff700 1 --2- 172.21.15.133:0/1527421670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635ca34a860 0x5635ca34ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.541+0000 7f2abbfff700 1 -- 172.21.15.133:0/1527421670 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635ca0c8d90 con 0x5635ca349f10 2023-07-14T18:14:25.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.541+0000 7f2abb7fe700 1 --2- 172.21.15.133:0/3303853976 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f2aa40a11d0 0x7f2aa40a3680 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0009880 tx=0x7f2ab0009c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:25.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.542+0000 7f2abbfff700 1 --2- 172.21.15.133:0/1527421670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635ca349f10 0x5635ca34a320 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2aac009eb0 tx=0x7f2aac011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dffa5235d6736c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:25.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.542+0000 7f2aa27fc700 1 -- 172.21.15.133:0/1527421670 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac019630 con 0x5635ca349f10 2023-07-14T18:14:25.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.542+0000 7f2aa27fc700 1 -- 172.21.15.133:0/1527421670 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2aac019c50 con 0x5635ca349f10 2023-07-14T18:14:25.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.542+0000 7f2aa27fc700 1 -- 172.21.15.133:0/1527421670 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac00ed10 con 0x5635ca349f10 2023-07-14T18:14:25.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.542+0000 7f2ad629eb80 1 -- 172.21.15.133:0/1527421670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635ca349f10 msgr2=0x5635ca34a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:25.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.542+0000 7f2ad629eb80 1 --2- 172.21.15.133:0/1527421670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635ca349f10 0x5635ca34a320 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2aac009eb0 tx=0x7f2aac011720 comp rx=0 tx=0).stop 2023-07-14T18:14:25.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.543+0000 7f2ad629eb80 1 -- 172.21.15.133:0/1527421670 shutdown_connections 2023-07-14T18:14:25.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.543+0000 7f2ad629eb80 1 --2- 172.21.15.133:0/1527421670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635ca349f10 0x5635ca34a320 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.543+0000 7f2ad629eb80 1 --2- 172.21.15.133:0/1527421670 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635ca34b460 0x5635ca34b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.543+0000 7f2ad629eb80 1 --2- 172.21.15.133:0/1527421670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635ca34a860 0x5635ca34ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.543+0000 7f2ad629eb80 1 -- 172.21.15.133:0/1527421670 >> 172.21.15.133:0/1527421670 conn(0x5635ca33f630 msgr2=0x5635ca344980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:25.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.543+0000 7f2ad629eb80 1 -- 172.21.15.133:0/1527421670 shutdown_connections 2023-07-14T18:14:25.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.543+0000 7f2ad629eb80 1 -- 172.21.15.133:0/1527421670 wait complete. 2023-07-14T18:14:25.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.543+0000 7f2ad629eb80 1 -- start start 2023-07-14T18:14:25.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.543+0000 7f2ad629eb80 1 Processor -- start 2023-07-14T18:14:25.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.544+0000 7f2ad629eb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635ca349f10 0x5635ca353900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:25.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.544+0000 7f2ad629eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635ca34a860 0x5635ca353e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:25.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.544+0000 7f2ad629eb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635ca34b460 0x5635ca35ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:25.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.544+0000 7f2ad629eb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635ca23aa70 con 0x5635ca349f10 2023-07-14T18:14:25.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.544+0000 7f2ad629eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635ca23a770 con 0x5635ca34a860 2023-07-14T18:14:25.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.544+0000 7f2ad629eb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635ca11e8d0 con 0x5635ca34b460 2023-07-14T18:14:25.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.544+0000 7f2ad629eb80 10 client.0 _ll_register_callbacks cb 0x5635ca238710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:14:25.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.544+0000 7f2abbfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635ca349f10 0x5635ca353900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:25.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.544+0000 7f2abbfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635ca349f10 0x5635ca353900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39296/0 (socket says 172.21.15.133:39296) 2023-07-14T18:14:25.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.544+0000 7f2abbfff700 1 -- 172.21.15.133:0/3862868815 learned_addr learned my addr 172.21.15.133:0/3862868815 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:25.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.544+0000 7f2ac0f9c700 1 --2- 172.21.15.133:0/3862868815 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635ca34b460 0x5635ca35ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:25.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.544+0000 7f2abb7fe700 1 --2- 172.21.15.133:0/3862868815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635ca34a860 0x5635ca353e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:25.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.544+0000 7f2abbfff700 1 -- 172.21.15.133:0/3862868815 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635ca34b460 msgr2=0x5635ca35ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:14:25.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.544+0000 7f2abbfff700 1 --2- 172.21.15.133:0/3862868815 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635ca34b460 0x5635ca35ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.544+0000 7f2abbfff700 1 -- 172.21.15.133:0/3862868815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635ca34a860 msgr2=0x5635ca353e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:14:25.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.544+0000 7f2abbfff700 1 --2- 172.21.15.133:0/3862868815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635ca34a860 0x5635ca353e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.545+0000 7f2abbfff700 1 -- 172.21.15.133:0/3862868815 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aac009ae0 con 0x5635ca349f10 2023-07-14T18:14:25.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.545+0000 7f2abbfff700 1 --2- 172.21.15.133:0/3862868815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635ca349f10 0x5635ca353900 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0046e0 tx=0x7f2aac00ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744258258e4a6ad6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:25.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.545+0000 7f2a9b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T18:14:25.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.545+0000 7f2a9b7fe700 1 -- 172.21.15.133:0/3862868815 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac019840 con 0x5635ca349f10 2023-07-14T18:14:25.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.545+0000 7f2ad629eb80 1 -- 172.21.15.133:0/3862868815 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635ca1f44a0 con 0x5635ca349f10 2023-07-14T18:14:25.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.545+0000 7f2a9b7fe700 1 -- 172.21.15.133:0/3862868815 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2aac0199a0 con 0x5635ca349f10 2023-07-14T18:14:25.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.545+0000 7f2a9b7fe700 1 -- 172.21.15.133:0/3862868815 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2aac01ae80 con 0x5635ca349f10 2023-07-14T18:14:25.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.545+0000 7f2ad629eb80 10 client.14673 fetch_fsmap learned FSMap version 17 2023-07-14T18:14:25.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.545+0000 7f2ad629eb80 1 -- 172.21.15.133:0/3862868815 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x5635ca0b1c10 con 0x5635ca349f10 2023-07-14T18:14:25.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.546+0000 7f2a9b7fe700 1 -- 172.21.15.133:0/3862868815 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5635ca1f44a0 con 0x5635ca349f10 2023-07-14T18:14:25.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.546+0000 7f2a9b7fe700 1 -- 172.21.15.133:0/3862868815 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f2aac004b30 con 0x5635ca349f10 2023-07-14T18:14:25.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.546+0000 7f2ad629eb80 10 client.14673 fetch_fsmap finished waiting for FSMap version 17 2023-07-14T18:14:25.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.546+0000 7f2ad629eb80 10 client.14673 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-14T18:14:25.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.546+0000 7f2ad629eb80 20 client.14673 populate_metadata read hostname 'smithi133' 2023-07-14T18:14:25.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.546+0000 7f2ad629eb80 1 --2- 172.21.15.133:0/3862868815 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5635ca35ea70 0x5635ca35ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:25.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.546+0000 7f2ad629eb80 4 client.14673 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:14:25.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.546+0000 7f2ad629eb80 1 -- 172.21.15.133:0/3862868815 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635ca318d50 con 0x5635ca35ea70 2023-07-14T18:14:25.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.546+0000 7f2abb7fe700 1 --2- 172.21.15.133:0/3862868815 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5635ca35ea70 0x5635ca35ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:25.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.547+0000 7f2a9b7fe700 10 client.14673 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:14:25.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.547+0000 7f2abb7fe700 1 --2- 172.21.15.133:0/3862868815 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5635ca35ea70 0x5635ca35ded0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21a212efa88aad87 server_cookie=8155da3fd181e510 in_seq=0 out_seq=0 2023-07-14T18:14:25.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.548+0000 7f2a9b7fe700 1 -- 172.21.15.133:0/3862868815 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5635ca318d50 con 0x5635ca35ea70 2023-07-14T18:14:25.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.548+0000 7f2a9b7fe700 10 client.14673 handle_command_reply: tid=0 2023-07-14T18:14:25.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.548+0000 7f2ad629eb80 1 client.14673 shutdown 2023-07-14T18:14:25.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.548+0000 7f2ad629eb80 1 -- 172.21.15.133:0/3862868815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635ca349f10 msgr2=0x5635ca353900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:25.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.548+0000 7f2ad629eb80 1 --2- 172.21.15.133:0/3862868815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635ca349f10 0x5635ca353900 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0046e0 tx=0x7f2aac00ed20 comp rx=0 tx=0).stop 2023-07-14T18:14:25.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.548+0000 7f2ad629eb80 1 -- 172.21.15.133:0/3862868815 shutdown_connections 2023-07-14T18:14:25.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.548+0000 7f2ad629eb80 1 --2- 172.21.15.133:0/3862868815 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5635ca35ea70 0x5635ca35ded0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.548+0000 7f2ad629eb80 1 --2- 172.21.15.133:0/3862868815 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635ca34b460 0x5635ca35ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.548+0000 7f2ad629eb80 1 --2- 172.21.15.133:0/3862868815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635ca349f10 0x5635ca353900 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.548+0000 7f2ad629eb80 1 --2- 172.21.15.133:0/3862868815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635ca34a860 0x5635ca353e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.548+0000 7f2a9b7fe700 0 client.14673 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:14:25.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.548+0000 7f2ad629eb80 1 -- 172.21.15.133:0/3862868815 >> 172.21.15.133:0/3862868815 conn(0x5635ca33f630 msgr2=0x5635ca343600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:25.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.548+0000 7f2ad629eb80 1 -- 172.21.15.133:0/3862868815 shutdown_connections 2023-07-14T18:14:25.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.548+0000 7f2ad629eb80 1 -- 172.21.15.133:0/3862868815 wait complete. 2023-07-14T18:14:25.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.548+0000 7f2ad629eb80 20 client.14673 trim_cache size 0 max 16384 2023-07-14T18:14:25.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.566+0000 7f2ad629eb80 1 Processor -- start 2023-07-14T18:14:25.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.566+0000 7f2ad629eb80 1 -- start start 2023-07-14T18:14:25.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.567+0000 7f2ad629eb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635ca349f10 0x5635ca3440a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:25.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.567+0000 7f2ad629eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635ca34a860 0x5635ca367800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:25.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.567+0000 7f2ad629eb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635ca34b460 0x5635ca35ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:25.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.567+0000 7f2ad629eb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635ca34bd50 con 0x5635ca34b460 2023-07-14T18:14:25.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.567+0000 7f2ad629eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635ca269790 con 0x5635ca34a860 2023-07-14T18:14:25.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.567+0000 7f2ad629eb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635ca259480 con 0x5635ca349f10 2023-07-14T18:14:25.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.567+0000 7f2ac0f9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635ca34b460 0x5635ca35ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:25.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.567+0000 7f2ac0f9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635ca34b460 0x5635ca35ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39300/0 (socket says 172.21.15.133:39300) 2023-07-14T18:14:25.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.567+0000 7f2ac0f9c700 1 -- 172.21.15.133:0/1096029754 learned_addr learned my addr 172.21.15.133:0/1096029754 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:25.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.567+0000 7f2abbfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635ca349f10 0x5635ca3440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:25.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.567+0000 7f2abb7fe700 1 --2- 172.21.15.133:0/1096029754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635ca34a860 0x5635ca367800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:25.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.567+0000 7f2ac0f9c700 1 -- 172.21.15.133:0/1096029754 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635ca349f10 msgr2=0x5635ca3440a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:14:25.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.567+0000 7f2ac0f9c700 1 --2- 172.21.15.133:0/1096029754 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635ca349f10 0x5635ca3440a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.567+0000 7f2ac0f9c700 1 -- 172.21.15.133:0/1096029754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635ca34a860 msgr2=0x5635ca367800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:14:25.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.567+0000 7f2ac0f9c700 1 --2- 172.21.15.133:0/1096029754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635ca34a860 0x5635ca367800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.567+0000 7f2ac0f9c700 1 -- 172.21.15.133:0/1096029754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635ca0ef200 con 0x5635ca34b460 2023-07-14T18:14:25.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.567+0000 7f2abb7fe700 1 --2- 172.21.15.133:0/1096029754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635ca34a860 0x5635ca367800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:14:25.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.568+0000 7f2ac0f9c700 1 --2- 172.21.15.133:0/1096029754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635ca34b460 0x5635ca35ded0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2ab409ad70 tx=0x7f2ab401bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d441a816f41a642 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:25.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.568+0000 7f2aa2ffd700 1 -- 172.21.15.133:0/1096029754 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab409ade0 con 0x5635ca34b460 2023-07-14T18:14:25.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.568+0000 7f2aa2ffd700 1 -- 172.21.15.133:0/1096029754 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ab409ee10 con 0x5635ca34b460 2023-07-14T18:14:25.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.568+0000 7f2aa2ffd700 1 -- 172.21.15.133:0/1096029754 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab409a2d0 con 0x5635ca34b460 2023-07-14T18:14:25.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.568+0000 7f2ad629eb80 1 -- 172.21.15.133:0/1096029754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635ca34b460 msgr2=0x5635ca35ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:25.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.568+0000 7f2ad629eb80 1 --2- 172.21.15.133:0/1096029754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635ca34b460 0x5635ca35ded0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2ab409ad70 tx=0x7f2ab401bf90 comp rx=0 tx=0).stop 2023-07-14T18:14:25.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.568+0000 7f2ad629eb80 1 -- 172.21.15.133:0/1096029754 shutdown_connections 2023-07-14T18:14:25.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.568+0000 7f2ad629eb80 1 --2- 172.21.15.133:0/1096029754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635ca34b460 0x5635ca35ded0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.568+0000 7f2ad629eb80 1 --2- 172.21.15.133:0/1096029754 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635ca349f10 0x5635ca3440a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.568+0000 7f2ad629eb80 1 --2- 172.21.15.133:0/1096029754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635ca34a860 0x5635ca367800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.568+0000 7f2ad629eb80 1 -- 172.21.15.133:0/1096029754 >> 172.21.15.133:0/1096029754 conn(0x5635ca33f630 msgr2=0x5635ca344950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:25.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.568+0000 7f2ad629eb80 1 -- 172.21.15.133:0/1096029754 shutdown_connections 2023-07-14T18:14:25.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.569+0000 7f2ad629eb80 1 -- 172.21.15.133:0/1096029754 wait complete. 2023-07-14T18:14:25.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.569+0000 7f2ad629eb80 1 -- start start 2023-07-14T18:14:25.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.569+0000 7f2ad629eb80 1 Processor -- start 2023-07-14T18:14:25.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.569+0000 7f2ad629eb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635ca349f10 0x5635ca362a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:25.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.569+0000 7f2ad629eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635ca34a860 0x5635ca362fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:25.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.569+0000 7f2ad629eb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635ca34b460 0x5635ca365f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:25.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.569+0000 7f2ad629eb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635ca3664d0 con 0x5635ca34b460 2023-07-14T18:14:25.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.569+0000 7f2ad629eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635ca366640 con 0x5635ca34a860 2023-07-14T18:14:25.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.570+0000 7f2ad629eb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635ca366800 con 0x5635ca349f10 2023-07-14T18:14:25.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.570+0000 7f2ad629eb80 10 client.0 _ll_register_callbacks cb 0x5635ca238710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:14:25.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.570+0000 7f2ac0f9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635ca34b460 0x5635ca365f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:25.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.570+0000 7f2ac0f9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635ca34b460 0x5635ca365f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39308/0 (socket says 172.21.15.133:39308) 2023-07-14T18:14:25.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.570+0000 7f2ac0f9c700 1 -- 172.21.15.133:0/4216618514 learned_addr learned my addr 172.21.15.133:0/4216618514 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:25.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.570+0000 7f2abb7fe700 1 --2- 172.21.15.133:0/4216618514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635ca34a860 0x5635ca362fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:25.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.570+0000 7f2abbfff700 1 --2- 172.21.15.133:0/4216618514 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635ca349f10 0x5635ca362a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:25.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.570+0000 7f2ac0f9c700 1 -- 172.21.15.133:0/4216618514 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635ca349f10 msgr2=0x5635ca362a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:14:25.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.570+0000 7f2ac0f9c700 1 --2- 172.21.15.133:0/4216618514 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635ca349f10 0x5635ca362a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.570+0000 7f2ac0f9c700 1 -- 172.21.15.133:0/4216618514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635ca34a860 msgr2=0x5635ca362fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:14:25.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.570+0000 7f2ac0f9c700 1 --2- 172.21.15.133:0/4216618514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635ca34a860 0x5635ca362fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.570+0000 7f2ac0f9c700 1 -- 172.21.15.133:0/4216618514 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2abc19de30 con 0x5635ca34b460 2023-07-14T18:14:25.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.570+0000 7f2abbfff700 1 --2- 172.21.15.133:0/4216618514 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635ca349f10 0x5635ca362a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:14:25.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.570+0000 7f2ac0f9c700 1 --2- 172.21.15.133:0/4216618514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635ca34b460 0x5635ca365f90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2ab40972e0 tx=0x7f2ab4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cddee0a88f92ec server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:25.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.570+0000 7f2a9b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T18:14:25.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.570+0000 7f2a9b7fe700 1 -- 172.21.15.133:0/4216618514 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab40cb1b0 con 0x5635ca34b460 2023-07-14T18:14:25.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.570+0000 7f2a9b7fe700 1 -- 172.21.15.133:0/4216618514 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2ab409ade0 con 0x5635ca34b460 2023-07-14T18:14:25.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.570+0000 7f2ad629eb80 1 -- 172.21.15.133:0/4216618514 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2aac019c50 con 0x5635ca34b460 2023-07-14T18:14:25.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.571+0000 7f2a9b7fe700 1 -- 172.21.15.133:0/4216618514 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ab40a2d10 con 0x5635ca34b460 2023-07-14T18:14:25.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.571+0000 7f2ad629eb80 10 client.14679 fetch_fsmap learned FSMap version 17 2023-07-14T18:14:25.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.571+0000 7f2ad629eb80 1 -- 172.21.15.133:0/4216618514 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x5635ca0c7260 con 0x5635ca34b460 2023-07-14T18:14:25.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.571+0000 7f2a9b7fe700 1 -- 172.21.15.133:0/4216618514 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ab40c0080 con 0x5635ca34b460 2023-07-14T18:14:25.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.571+0000 7f2a9b7fe700 1 -- 172.21.15.133:0/4216618514 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f2ab409e040 con 0x5635ca34b460 2023-07-14T18:14:25.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.571+0000 7f2ad629eb80 10 client.14679 fetch_fsmap finished waiting for FSMap version 17 2023-07-14T18:14:25.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.571+0000 7f2ad629eb80 10 client.14679 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-14T18:14:25.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.571+0000 7f2ad629eb80 20 client.14679 populate_metadata read hostname 'smithi133' 2023-07-14T18:14:25.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.571+0000 7f2ad629eb80 1 --2- 172.21.15.133:0/4216618514 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5635ca3b7880 0x5635ca3b4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:25.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.571+0000 7f2ad629eb80 4 client.14679 mds_command: new command op to 24557 tid=0[{"prefix": "status"}] 2023-07-14T18:14:25.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.571+0000 7f2ad629eb80 1 -- 172.21.15.133:0/4216618514 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5635ca293e60 con 0x5635ca3b7880 2023-07-14T18:14:25.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.572+0000 7f2abbfff700 1 --2- 172.21.15.133:0/4216618514 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5635ca3b7880 0x5635ca3b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:25.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.572+0000 7f2abbfff700 1 --2- 172.21.15.133:0/4216618514 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5635ca3b7880 0x5635ca3b4720 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49a17aadea75d26c server_cookie=d80a01dc11db8baf in_seq=0 out_seq=0 2023-07-14T18:14:25.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.572+0000 7f2a9b7fe700 10 client.14679 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:14:25.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.572+0000 7f2a9b7fe700 1 -- 172.21.15.133:0/4216618514 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x5635ca293e60 con 0x5635ca3b7880 2023-07-14T18:14:25.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.572+0000 7f2a9b7fe700 10 client.14679 handle_command_reply: tid=0 2023-07-14T18:14:25.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.572+0000 7f2ad629eb80 1 client.14679 shutdown 2023-07-14T18:14:25.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.572+0000 7f2ad629eb80 1 -- 172.21.15.133:0/4216618514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635ca34b460 msgr2=0x5635ca365f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:25.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.572+0000 7f2ad629eb80 1 --2- 172.21.15.133:0/4216618514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635ca34b460 0x5635ca365f90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2ab40972e0 tx=0x7f2ab4009040 comp rx=0 tx=0).stop 2023-07-14T18:14:25.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.572+0000 7f2ad629eb80 1 -- 172.21.15.133:0/4216618514 shutdown_connections 2023-07-14T18:14:25.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.572+0000 7f2ad629eb80 1 --2- 172.21.15.133:0/4216618514 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5635ca3b7880 0x5635ca3b4720 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.572+0000 7f2ad629eb80 1 --2- 172.21.15.133:0/4216618514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635ca34b460 0x5635ca365f90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.572+0000 7f2ad629eb80 1 --2- 172.21.15.133:0/4216618514 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635ca349f10 0x5635ca362a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.572+0000 7f2a9b7fe700 0 client.14679 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:14:25.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.572+0000 7f2ad629eb80 1 --2- 172.21.15.133:0/4216618514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635ca34a860 0x5635ca362fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.572+0000 7f2ad629eb80 1 -- 172.21.15.133:0/4216618514 >> 172.21.15.133:0/4216618514 conn(0x5635ca33f630 msgr2=0x5635ca3517d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:25.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.573+0000 7f2ad629eb80 1 -- 172.21.15.133:0/4216618514 shutdown_connections 2023-07-14T18:14:25.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.573+0000 7f2ad629eb80 1 -- 172.21.15.133:0/4216618514 wait complete. 2023-07-14T18:14:25.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.573+0000 7f2ad629eb80 20 client.14679 trim_cache size 0 max 16384 2023-07-14T18:14:25.626 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:14:25.626 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "06d0f5fa-2271-11ee-9b33-001a4aab830c", 2023-07-14T18:14:25.626 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2023-07-14T18:14:25.626 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24557, 2023-07-14T18:14:25.626 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2023-07-14T18:14:25.626 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2023-07-14T18:14:25.627 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2023-07-14T18:14:25.627 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 7.6457831809999997, 2023-07-14T18:14:25.627 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 17, 2023-07-14T18:14:25.627 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 88, 2023-07-14T18:14:25.627 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 88, 2023-07-14T18:14:25.627 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 14.659563964 2023-07-14T18:14:25.627 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:14:25.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.573+0000 7f2ac29ff700 1 -- 172.21.15.133:0/3303853976 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f2aa40a11d0 msgr2=0x7f2aa40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:25.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.573+0000 7f2ac29ff700 1 --2- 172.21.15.133:0/3303853976 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f2aa40a11d0 0x7f2aa40a3680 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0009880 tx=0x7f2ab0009c30 comp rx=0 tx=0).stop 2023-07-14T18:14:25.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.573+0000 7f2ac29ff700 1 -- 172.21.15.133:0/3303853976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2abc1017a0 msgr2=0x7f2abc19d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:25.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.573+0000 7f2ac29ff700 1 --2- 172.21.15.133:0/3303853976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2abc1017a0 0x7f2abc19d020 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2ab400b070 tx=0x7f2ab400bc80 comp rx=0 tx=0).stop 2023-07-14T18:14:25.629 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.573+0000 7f2ac29ff700 1 -- 172.21.15.133:0/3303853976 shutdown_connections 2023-07-14T18:14:25.629 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.573+0000 7f2ac29ff700 1 --2- 172.21.15.133:0/3303853976 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f2aa40a11d0 0x7f2aa40a3680 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.629 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.573+0000 7f2ac29ff700 1 --2- 172.21.15.133:0/3303853976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2abc1017a0 0x7f2abc19d020 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.629 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.573+0000 7f2ac29ff700 1 --2- 172.21.15.133:0/3303853976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2abc0ffc20 0x7f2abc195a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.629 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.573+0000 7f2ac29ff700 1 --2- 172.21.15.133:0/3303853976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2abc100e10 0x7f2abc195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:25.629 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.573+0000 7f2ac29ff700 1 -- 172.21.15.133:0/3303853976 >> 172.21.15.133:0/3303853976 conn(0x7f2abc0fb260 msgr2=0x7f2abc0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:25.629 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.574+0000 7f2ac29ff700 1 -- 172.21.15.133:0/3303853976 shutdown_connections 2023-07-14T18:14:25.630 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:25.574+0000 7f2ac29ff700 1 -- 172.21.15.133:0/3303853976 wait complete. 2023-07-14T18:14:25.634 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-14T18:14:25.635 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-07-14T18:14:25.635 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph fs ls 2023-07-14T18:14:26.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:26 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2011446169' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:14:26.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:26 smithi161 ceph-mon[128993]: pgmap v277: 129 pgs: 10 creating+peering, 107 active+clean, 12 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 7 op/s 2023-07-14T18:14:26.350 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:14:26.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:26 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2011446169' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:14:26.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:26 smithi171 ceph-mon[129241]: pgmap v277: 129 pgs: 10 creating+peering, 107 active+clean, 12 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 7 op/s 2023-07-14T18:14:26.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:26 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2011446169' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:14:26.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:26 smithi133 ceph-mon[126384]: pgmap v277: 129 pgs: 10 creating+peering, 107 active+clean, 12 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 7 op/s 2023-07-14T18:14:27.976 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.932+0000 7fe1b801f700 1 -- 172.21.15.133:0/170347457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1b00fff30 msgr2=0x7fe1b0100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:27.976 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.932+0000 7fe1b801f700 1 --2- 172.21.15.133:0/170347457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1b00fff30 0x7fe1b0100300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac0099c0 tx=0x7fe1ac009cd0 comp rx=0 tx=0).stop 2023-07-14T18:14:27.977 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.932+0000 7fe1b801f700 1 -- 172.21.15.133:0/170347457 shutdown_connections 2023-07-14T18:14:27.977 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.932+0000 7fe1b801f700 1 --2- 172.21.15.133:0/170347457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1b0104fd0 0x7fe1b010cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:27.977 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.932+0000 7fe1b801f700 1 --2- 172.21.15.133:0/170347457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1b00fff30 0x7fe1b0100300 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:27.977 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.932+0000 7fe1b801f700 1 --2- 172.21.15.133:0/170347457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1b01008d0 0x7fe1b01048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:27.977 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.932+0000 7fe1b801f700 1 -- 172.21.15.133:0/170347457 >> 172.21.15.133:0/170347457 conn(0x7fe1b00fb790 msgr2=0x7fe1b00fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:27.977 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.932+0000 7fe1b801f700 1 -- 172.21.15.133:0/170347457 shutdown_connections 2023-07-14T18:14:27.977 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.933+0000 7fe1b801f700 1 -- 172.21.15.133:0/170347457 wait complete. 2023-07-14T18:14:27.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.933+0000 7fe1b801f700 1 Processor -- start 2023-07-14T18:14:27.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.934+0000 7fe1b801f700 1 -- start start 2023-07-14T18:14:27.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.934+0000 7fe1b801f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1b00fff30 0x7fe1b0197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:27.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.934+0000 7fe1b801f700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1b01008d0 0x7fe1b0198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:27.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.934+0000 7fe1b801f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1b0104fd0 0x7fe1b019c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:27.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.934+0000 7fe1b801f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b019cfb0 con 0x7fe1b00fff30 2023-07-14T18:14:27.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.934+0000 7fe1b801f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b019d120 con 0x7fe1b0104fd0 2023-07-14T18:14:27.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.934+0000 7fe1b801f700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b019d2e0 con 0x7fe1b01008d0 2023-07-14T18:14:27.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.934+0000 7fe1b5dbb700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1b00fff30 0x7fe1b0197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:27.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.934+0000 7fe1b5dbb700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1b00fff30 0x7fe1b0197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45412/0 (socket says 172.21.15.133:45412) 2023-07-14T18:14:27.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.934+0000 7fe1b55ba700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1b01008d0 0x7fe1b0198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:27.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.934+0000 7fe1b5dbb700 1 -- 172.21.15.133:0/3335289202 learned_addr learned my addr 172.21.15.133:0/3335289202 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:27.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.934+0000 7fe1b65bc700 1 --2- 172.21.15.133:0/3335289202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1b0104fd0 0x7fe1b019c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:27.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.935+0000 7fe1b5dbb700 1 -- 172.21.15.133:0/3335289202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1b01008d0 msgr2=0x7fe1b0198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:27.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.935+0000 7fe1b5dbb700 1 --2- 172.21.15.133:0/3335289202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1b01008d0 0x7fe1b0198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:27.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.935+0000 7fe1b5dbb700 1 -- 172.21.15.133:0/3335289202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1b0104fd0 msgr2=0x7fe1b019c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:27.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.935+0000 7fe1b5dbb700 1 --2- 172.21.15.133:0/3335289202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1b0104fd0 0x7fe1b019c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:27.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.935+0000 7fe1b5dbb700 1 -- 172.21.15.133:0/3335289202 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1ac0095f0 con 0x7fe1b00fff30 2023-07-14T18:14:27.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.935+0000 7fe1b65bc700 1 --2- 172.21.15.133:0/3335289202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1b0104fd0 0x7fe1b019c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:14:27.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.935+0000 7fe1b55ba700 1 --2- 172.21.15.133:0/3335289202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1b01008d0 0x7fe1b0198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:14:27.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.935+0000 7fe1b5dbb700 1 --2- 172.21.15.133:0/3335289202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1b00fff30 0x7fe1b0197ff0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac0098e0 tx=0x7fe1ac00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:27.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.935+0000 7fe1a6ffd700 1 -- 172.21.15.133:0/3335289202 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac01d070 con 0x7fe1b00fff30 2023-07-14T18:14:27.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.935+0000 7fe1a6ffd700 1 -- 172.21.15.133:0/3335289202 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1ac00fb80 con 0x7fe1b00fff30 2023-07-14T18:14:27.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.935+0000 7fe1b801f700 1 -- 172.21.15.133:0/3335289202 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b019d560 con 0x7fe1b00fff30 2023-07-14T18:14:27.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.935+0000 7fe1a6ffd700 1 -- 172.21.15.133:0/3335289202 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac0173f0 con 0x7fe1b00fff30 2023-07-14T18:14:27.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.935+0000 7fe1b801f700 1 -- 172.21.15.133:0/3335289202 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b01a4e20 con 0x7fe1b00fff30 2023-07-14T18:14:27.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.936+0000 7fe1b801f700 1 -- 172.21.15.133:0/3335289202 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b004e340 con 0x7fe1b00fff30 2023-07-14T18:14:27.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.938+0000 7fe1a6ffd700 1 -- 172.21.15.133:0/3335289202 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe1ac00fcf0 con 0x7fe1b00fff30 2023-07-14T18:14:27.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.939+0000 7fe1a6ffd700 1 --2- 172.21.15.133:0/3335289202 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe19c0a1150 0x7fe19c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:27.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.940+0000 7fe1a6ffd700 1 -- 172.21.15.133:0/3335289202 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe1ac0cd790 con 0x7fe1b00fff30 2023-07-14T18:14:27.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.940+0000 7fe1b55ba700 1 --2- 172.21.15.133:0/3335289202 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe19c0a1150 0x7fe19c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:27.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.940+0000 7fe1b55ba700 1 --2- 172.21.15.133:0/3335289202 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe19c0a1150 0x7fe19c0a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0199610 tx=0x7fe1a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:27.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:27.943+0000 7fe1a6ffd700 1 -- 172.21.15.133:0/3335289202 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fe1ac096d20 con 0x7fe1b00fff30 2023-07-14T18:14:28.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:28.182+0000 7fe1b801f700 1 -- 172.21.15.133:0/3335289202 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe1b01a4fb0 con 0x7fe1b00fff30 2023-07-14T18:14:28.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:28.182+0000 7fe1a6ffd700 1 -- 172.21.15.133:0/3335289202 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7fe1ac09a340 con 0x7fe1b00fff30 2023-07-14T18:14:28.226 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-07-14T18:14:28.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:28.184+0000 7fe1b801f700 1 -- 172.21.15.133:0/3335289202 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe19c0a1150 msgr2=0x7fe19c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:28.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:28.184+0000 7fe1b801f700 1 --2- 172.21.15.133:0/3335289202 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe19c0a1150 0x7fe19c0a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0199610 tx=0x7fe1a0009260 comp rx=0 tx=0).stop 2023-07-14T18:14:28.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:28.184+0000 7fe1b801f700 1 -- 172.21.15.133:0/3335289202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1b00fff30 msgr2=0x7fe1b0197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:28.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:28.184+0000 7fe1b801f700 1 --2- 172.21.15.133:0/3335289202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1b00fff30 0x7fe1b0197ff0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac0098e0 tx=0x7fe1ac00f690 comp rx=0 tx=0).stop 2023-07-14T18:14:28.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:28.185+0000 7fe1b801f700 1 -- 172.21.15.133:0/3335289202 shutdown_connections 2023-07-14T18:14:28.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:28.185+0000 7fe1b801f700 1 --2- 172.21.15.133:0/3335289202 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe19c0a1150 0x7fe19c0a3600 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:28.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:28.185+0000 7fe1b801f700 1 --2- 172.21.15.133:0/3335289202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1b0104fd0 0x7fe1b019c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:28.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:28.185+0000 7fe1b801f700 1 --2- 172.21.15.133:0/3335289202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1b00fff30 0x7fe1b0197ff0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:28.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:28.185+0000 7fe1b801f700 1 --2- 172.21.15.133:0/3335289202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1b01008d0 0x7fe1b0198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:28.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:28.185+0000 7fe1b801f700 1 -- 172.21.15.133:0/3335289202 >> 172.21.15.133:0/3335289202 conn(0x7fe1b00fb790 msgr2=0x7fe1b00fc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:28.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:28.185+0000 7fe1b801f700 1 -- 172.21.15.133:0/3335289202 shutdown_connections 2023-07-14T18:14:28.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:28.185+0000 7fe1b801f700 1 -- 172.21.15.133:0/3335289202 wait complete. 2023-07-14T18:14:28.724 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-07-14T18:14:28.725 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-14T18:14:28.725 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2023-07-14T18:14:28.725 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-07-14T18:14:28.726 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-07-14T18:14:28.726 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-07-14T18:14:28.726 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-14T18:14:28.726 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-14T18:14:28.726 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-07-14T18:14:28.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:14:28.727 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2023-07-14T18:14:28.771 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-14T18:14:28.772 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-14T18:14:28.772 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2023-07-14T18:14:28.772 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-07-14T18:14:28.772 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2023-07-14T18:14:28.772 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-07-14T18:14:28.772 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-14T18:14:28.772 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2023-07-14T18:14:28.773 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2023-07-14T18:14:28.773 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-14T18:14:28.773 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2023-07-14T18:14:28.773 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2023-07-14T18:14:28.773 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-14T18:14:28.773 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2023-07-14T18:14:28.773 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-14T18:14:28.773 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2023-07-14T18:14:28.774 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-14T18:14:28.774 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 41525sec preferred_lft 41525sec 2023-07-14T18:14:28.774 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::1cfd:5bbf:6db9:2d9/64 scope link noprefixroute 2023-07-14T18:14:28.774 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-07-14T18:14:28.775 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-14T18:14:28.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:14:28.776 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-07-14T18:14:28.776 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-07-14T18:14:28.777 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add name ceph-brx type bridge 2023-07-14T18:14:28.777 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr flush dev ceph-brx 2023-07-14T18:14:28.777 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set ceph-brx up 2023-07-14T18:14:28.777 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-14T18:14:28.778 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-07-14T18:14:28.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:28 smithi161 ceph-mon[128993]: pgmap v278: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2023-07-14T18:14:28.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:28 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3335289202' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-14T18:14:28.864 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:28 smithi171 ceph-mon[129241]: pgmap v278: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2023-07-14T18:14:28.865 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:28 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3335289202' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-14T18:14:28.905 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:28 smithi133 ceph-mon[126384]: pgmap v278: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2023-07-14T18:14:28.905 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:28 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3335289202' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-14T18:14:29.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:14:29.232 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-14T18:14:29.304 INFO:teuthology.orchestra.run.smithi133.stdout:1 2023-07-14T18:14:29.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:14:29.307 DEBUG:teuthology.orchestra.run.smithi133:> route 2023-07-14T18:14:29.372 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2023-07-14T18:14:29.372 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-14T18:14:29.373 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-14T18:14:29.373 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-14T18:14:29.373 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-14T18:14:29.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:14:29.374 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-07-14T18:14:29.375 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-07-14T18:14:29.375 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-14T18:14:29.375 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-14T18:14:29.375 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-14T18:14:29.376 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-07-14T18:14:30.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:14:30.066 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2023-07-14T18:14:30.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:14:30.114 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list-id 2023-07-14T18:14:30.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:14:30.197 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-07-14T18:14:30.198 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-07-14T18:14:30.198 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-07-14T18:14:30.199 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-07-14T18:14:30.199 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-07-14T18:14:30.383 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-07-14T18:14:30.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:14:30.384 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-07-14T18:14:30.385 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-07-14T18:14:30.385 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-07-14T18:14:30.385 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-14T18:14:30.385 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-07-14T18:14:30.386 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-07-14T18:14:30.386 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-07-14T18:14:30.386 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-07-14T18:14:30.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:14:30.666 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-07-14T18:14:30.666 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-07-14T18:14:30.667 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2023-07-14T18:14:30.667 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set dev brx.0 master ceph-brx 2023-07-14T18:14:30.667 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-07-14T18:14:30.840 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-07-14T18:14:30.841 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T18:14:30.841 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-07-14T18:14:30.885 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-07-14T18:14:30.886 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T18:14:30.886 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-07-14T18:14:30.970 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2023-07-14T18:14:31.080 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T18:14:31.080 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T18:14:31.148 INFO:teuthology.orchestra.run.smithi133.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T18:14:31.151 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T18:14:31.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:14:31.152 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2023-07-14T18:14:31.196 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-14T18:14:31.196 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-07-14T18:14:31.240 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2023-07-14T18:14:31.300 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:30 smithi133 ceph-mon[126384]: pgmap v279: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 4 op/s 2023-07-14T18:14:31.313 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T18:14:31.314 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T18:14:31.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:30 smithi161 ceph-mon[128993]: pgmap v279: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 4 op/s 2023-07-14T18:14:31.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:30 smithi171 ceph-mon[129241]: pgmap v279: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 4 op/s 2023-07-14T18:14:31.382 INFO:teuthology.orchestra.run.smithi133.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T18:14:31.384 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T18:14:31.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:14:31.386 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2023-07-14T18:14:31.436 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi133.stderr:2023-07-14T18:14:31.393+0000 7f7d2f80a5c0 -1 init, newargv = 0x55c625a3aee0 newargc=15 2023-07-14T18:14:31.436 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi133.stderr:ceph-fuse[163616]: starting ceph client 2023-07-14T18:14:31.445 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi133.stderr:ceph-fuse[163616]: starting fuse 2023-07-14T18:14:32.532 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2023-07-14T18:14:32.609 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T18:14:32.609 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T18:14:32.678 INFO:teuthology.orchestra.run.smithi133.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T18:14:32.681 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T18:14:32.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:14:32.682 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2023-07-14T18:14:32.725 INFO:teuthology.orchestra.run.smithi133.stdout:121 2023-07-14T18:14:32.726 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2023-07-14T18:14:32.726 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-14T18:14:32.726 DEBUG:teuthology.orchestra.run.smithi133:> import glob 2023-07-14T18:14:32.727 DEBUG:teuthology.orchestra.run.smithi133:> import re 2023-07-14T18:14:32.727 DEBUG:teuthology.orchestra.run.smithi133:> import os 2023-07-14T18:14:32.727 DEBUG:teuthology.orchestra.run.smithi133:> import subprocess 2023-07-14T18:14:32.728 DEBUG:teuthology.orchestra.run.smithi133:> 2023-07-14T18:14:32.728 DEBUG:teuthology.orchestra.run.smithi133:> def _find_admin_socket(client_name): 2023-07-14T18:14:32.728 DEBUG:teuthology.orchestra.run.smithi133:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-07-14T18:14:32.729 DEBUG:teuthology.orchestra.run.smithi133:> files = glob.glob(asok_path) 2023-07-14T18:14:32.729 DEBUG:teuthology.orchestra.run.smithi133:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-07-14T18:14:32.729 DEBUG:teuthology.orchestra.run.smithi133:> 2023-07-14T18:14:32.729 DEBUG:teuthology.orchestra.run.smithi133:> # Given a non-glob path, it better be there 2023-07-14T18:14:32.730 DEBUG:teuthology.orchestra.run.smithi133:> if "*" not in asok_path: 2023-07-14T18:14:32.730 DEBUG:teuthology.orchestra.run.smithi133:> assert(len(files) == 1) 2023-07-14T18:14:32.730 DEBUG:teuthology.orchestra.run.smithi133:> return files[0] 2023-07-14T18:14:32.731 DEBUG:teuthology.orchestra.run.smithi133:> 2023-07-14T18:14:32.731 DEBUG:teuthology.orchestra.run.smithi133:> for f in files: 2023-07-14T18:14:32.731 DEBUG:teuthology.orchestra.run.smithi133:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-14T18:14:32.731 DEBUG:teuthology.orchestra.run.smithi133:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-14T18:14:32.732 DEBUG:teuthology.orchestra.run.smithi133:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-14T18:14:32.732 DEBUG:teuthology.orchestra.run.smithi133:> contents = proc_f.read() 2023-07-14T18:14:32.732 DEBUG:teuthology.orchestra.run.smithi133:> if mountpoint in contents: 2023-07-14T18:14:32.732 DEBUG:teuthology.orchestra.run.smithi133:> return f 2023-07-14T18:14:32.733 DEBUG:teuthology.orchestra.run.smithi133:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-14T18:14:32.733 DEBUG:teuthology.orchestra.run.smithi133:> 2023-07-14T18:14:32.733 DEBUG:teuthology.orchestra.run.smithi133:> print(_find_admin_socket("client.admin")) 2023-07-14T18:14:32.733 DEBUG:teuthology.orchestra.run.smithi133:> ' 2023-07-14T18:14:32.910 INFO:teuthology.orchestra.run.smithi133.stdout:/var/run/ceph/ceph-client.admin.163616.asok 2023-07-14T18:14:33.088 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.163616.asok 2023-07-14T18:14:33.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:14:33.089 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.163616.asok status 2023-07-14T18:14:33.187 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:32 smithi133 ceph-mon[126384]: pgmap v280: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 0 op/s 2023-07-14T18:14:33.312 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:14:33.313 INFO:teuthology.orchestra.run.smithi133.stdout: "metadata": { 2023-07-14T18:14:33.313 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_sha1": "81c9387d2e7576da065747ec9c16d1c7d69016ec", 2023-07-14T18:14:33.313 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 18.0.0-4940-g81c9387d (81c9387d2e7576da065747ec9c16d1c7d69016ec) reef (dev)", 2023-07-14T18:14:33.313 INFO:teuthology.orchestra.run.smithi133.stdout: "entity_id": "admin", 2023-07-14T18:14:33.313 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": "smithi133", 2023-07-14T18:14:33.313 INFO:teuthology.orchestra.run.smithi133.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-07-14T18:14:33.314 INFO:teuthology.orchestra.run.smithi133.stdout: "pid": "163616", 2023-07-14T18:14:33.314 INFO:teuthology.orchestra.run.smithi133.stdout: "root": "/" 2023-07-14T18:14:33.314 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:14:33.314 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_count": 0, 2023-07-14T18:14:33.314 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_pinned_count": 0, 2023-07-14T18:14:33.314 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24706, 2023-07-14T18:14:33.314 INFO:teuthology.orchestra.run.smithi133.stdout: "inst": { 2023-07-14T18:14:33.314 INFO:teuthology.orchestra.run.smithi133.stdout: "name": { 2023-07-14T18:14:33.315 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "client", 2023-07-14T18:14:33.315 INFO:teuthology.orchestra.run.smithi133.stdout: "num": 24706 2023-07-14T18:14:33.315 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:14:33.315 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2023-07-14T18:14:33.315 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2023-07-14T18:14:33.315 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2023-07-14T18:14:33.315 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 2072111679 2023-07-14T18:14:33.315 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:14:33.316 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:14:33.316 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2023-07-14T18:14:33.316 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2023-07-14T18:14:33.316 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2023-07-14T18:14:33.316 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 2072111679 2023-07-14T18:14:33.316 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:14:33.316 INFO:teuthology.orchestra.run.smithi133.stdout: "inst_str": "client.24706 192.168.0.1:0/2072111679", 2023-07-14T18:14:33.317 INFO:teuthology.orchestra.run.smithi133.stdout: "addr_str": "192.168.0.1:0/2072111679", 2023-07-14T18:14:33.317 INFO:teuthology.orchestra.run.smithi133.stdout: "inode_count": 1, 2023-07-14T18:14:33.317 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_epoch": 17, 2023-07-14T18:14:33.317 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch": 88, 2023-07-14T18:14:33.317 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch_barrier": 0, 2023-07-14T18:14:33.317 INFO:teuthology.orchestra.run.smithi133.stdout: "blocklisted": false, 2023-07-14T18:14:33.317 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs" 2023-07-14T18:14:33.317 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:14:33.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:14:33.326 DEBUG:teuthology.orchestra.run.smithi133:> stat --file-system '--printf=%T 2023-07-14T18:14:33.326 DEBUG:teuthology.orchestra.run.smithi133:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-07-14T18:14:33.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:32 smithi161 ceph-mon[128993]: pgmap v280: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 0 op/s 2023-07-14T18:14:33.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:32 smithi171 ceph-mon[129241]: pgmap v280: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 0 op/s 2023-07-14T18:14:33.371 INFO:teuthology.orchestra.run.smithi133.stdout:fuseblk 2023-07-14T18:14:33.371 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-07-14T18:14:33.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:14:33.372 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-07-14T18:14:33.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:14:33.481 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-07-14T18:14:33.614 INFO:teuthology.orchestra.run.smithi133.stdout:# file: . 2023-07-14T18:14:33.615 INFO:teuthology.orchestra.run.smithi133.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-07-14T18:14:33.615 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:14:33.616 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T18:14:33.616 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T18:14:33.660 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T18:14:33.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.853+0000 7feff03ff700 1 -- 172.21.15.133:0/3692569085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefe80ffc30 msgr2=0x7fefe8100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:33.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.854+0000 7feff03ff700 1 --2- 172.21.15.133:0/3692569085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefe80ffc30 0x7fefe8100040 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fefe4009990 tx=0x7fefe4009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:33.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.854+0000 7feff03ff700 1 -- 172.21.15.133:0/3692569085 shutdown_connections 2023-07-14T18:14:33.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.854+0000 7feff03ff700 1 --2- 172.21.15.133:0/3692569085 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fefe81017b0 0x7fefe8108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:33.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.854+0000 7feff03ff700 1 --2- 172.21.15.133:0/3692569085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefe80ffc30 0x7fefe8100040 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:33.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.854+0000 7feff03ff700 1 --2- 172.21.15.133:0/3692569085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefe8100e20 0x7fefe8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:33.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.854+0000 7feff03ff700 1 -- 172.21.15.133:0/3692569085 >> 172.21.15.133:0/3692569085 conn(0x7fefe80fb210 msgr2=0x7fefe80fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:33.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.854+0000 7feff03ff700 1 -- 172.21.15.133:0/3692569085 shutdown_connections 2023-07-14T18:14:33.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.854+0000 7feff03ff700 1 -- 172.21.15.133:0/3692569085 wait complete. 2023-07-14T18:14:33.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.855+0000 7feff03ff700 1 Processor -- start 2023-07-14T18:14:33.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.855+0000 7feff03ff700 1 -- start start 2023-07-14T18:14:33.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.855+0000 7feff03ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefe80ffc30 0x7fefe80743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:33.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.855+0000 7feff03ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefe8100e20 0x7fefe8072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:33.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.855+0000 7feff03ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fefe81017b0 0x7fefe8072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:33.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.855+0000 7feff03ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe8073710 con 0x7fefe8100e20 2023-07-14T18:14:33.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.855+0000 7feff03ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe8073880 con 0x7fefe80ffc30 2023-07-14T18:14:33.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.855+0000 7feff03ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe819f970 con 0x7fefe81017b0 2023-07-14T18:14:33.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.855+0000 7fefed99a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefe8100e20 0x7fefe8072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:33.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.855+0000 7fefed99a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefe8100e20 0x7fefe8072a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45434/0 (socket says 172.21.15.133:45434) 2023-07-14T18:14:33.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.855+0000 7fefed99a700 1 -- 172.21.15.133:0/4052699988 learned_addr learned my addr 172.21.15.133:0/4052699988 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:33.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.856+0000 7fefee99c700 1 --2- 172.21.15.133:0/4052699988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fefe81017b0 0x7fefe8072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:33.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.856+0000 7fefee19b700 1 --2- 172.21.15.133:0/4052699988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefe80ffc30 0x7fefe80743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:33.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.856+0000 7fefed99a700 1 -- 172.21.15.133:0/4052699988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fefe81017b0 msgr2=0x7fefe8072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:33.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.856+0000 7fefed99a700 1 --2- 172.21.15.133:0/4052699988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fefe81017b0 0x7fefe8072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:33.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.856+0000 7fefed99a700 1 -- 172.21.15.133:0/4052699988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefe80ffc30 msgr2=0x7fefe80743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:33.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.856+0000 7fefed99a700 1 --2- 172.21.15.133:0/4052699988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefe80ffc30 0x7fefe80743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:33.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.856+0000 7fefed99a700 1 -- 172.21.15.133:0/4052699988 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe40095f0 con 0x7fefe8100e20 2023-07-14T18:14:33.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.856+0000 7fefed99a700 1 --2- 172.21.15.133:0/4052699988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefe8100e20 0x7fefe8072a30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fefd800e9a0 tx=0x7fefd800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:33.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.856+0000 7fefdf7fe700 1 -- 172.21.15.133:0/4052699988 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd8009790 con 0x7fefe8100e20 2023-07-14T18:14:33.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.856+0000 7feff03ff700 1 -- 172.21.15.133:0/4052699988 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe819fc50 con 0x7fefe8100e20 2023-07-14T18:14:33.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.856+0000 7fefdf7fe700 1 -- 172.21.15.133:0/4052699988 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefd8010430 con 0x7fefe8100e20 2023-07-14T18:14:33.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.856+0000 7fefdf7fe700 1 -- 172.21.15.133:0/4052699988 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd8018680 con 0x7fefe8100e20 2023-07-14T18:14:33.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.856+0000 7feff03ff700 1 -- 172.21.15.133:0/4052699988 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe81a0160 con 0x7fefe8100e20 2023-07-14T18:14:33.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.858+0000 7feff03ff700 1 -- 172.21.15.133:0/4052699988 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe810b400 con 0x7fefe8100e20 2023-07-14T18:14:33.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.859+0000 7fefdf7fe700 1 -- 172.21.15.133:0/4052699988 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fefd80105a0 con 0x7fefe8100e20 2023-07-14T18:14:33.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.861+0000 7fefdf7fe700 1 --2- 172.21.15.133:0/4052699988 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fefd40a11a0 0x7fefd40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:33.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.861+0000 7fefdf7fe700 1 -- 172.21.15.133:0/4052699988 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fefd8014070 con 0x7fefe8100e20 2023-07-14T18:14:33.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.861+0000 7fefee19b700 1 --2- 172.21.15.133:0/4052699988 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fefd40a11a0 0x7fefd40a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:33.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.862+0000 7fefee19b700 1 --2- 172.21.15.133:0/4052699988 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fefd40a11a0 0x7fefd40a3650 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fefe400b2e0 tx=0x7fefe4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:33.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:33.865+0000 7fefdf7fe700 1 -- 172.21.15.133:0/4052699988 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fefd8051780 con 0x7fefe8100e20 2023-07-14T18:14:34.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.073+0000 7feff03ff700 1 -- 172.21.15.133:0/4052699988 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefe8106ac0 con 0x7fefe8100e20 2023-07-14T18:14:34.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.074+0000 7fefdf7fe700 1 -- 172.21.15.133:0/4052699988 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24003 (secure 0 0 0) 0x7fefd8096210 con 0x7fefe8100e20 2023-07-14T18:14:34.118 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:14:34.118 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":88,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","created":"2023-07-14T18:06:49.561582+0000","modified":"2023-07-14T18:14:22.524878+0000","last_up_change":"2023-07-14T18:13:29.836099+0000","last_in_change":"2023-07-14T18:13:11.606710+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T18:10:03.945039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T18:14:15.451340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T18:14:16.892215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T18:14:20.493175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"85a29de7-d712-422a-b647-d9e90091aaf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6803","nonce":3076510801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6805","nonce":3076510801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6809","nonce":3076510801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6807","nonce":3076510801}]},"public_addr":"172.21.15.133:6803/3076510801","cluster_addr":"172.21.15.133:6805/3076510801","heartbeat_back_addr":"172.21.15.133:6809/3076510801","heartbeat_front_addr":"172.21.15.133:6807/3076510801","state":["exists","up"]},{"osd":1,"uuid":"80ad4e78-7917-495e-b119-99f7c7c36243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6801","nonce":1841901852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6803","nonce":1841901852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6807","nonce":1841901852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6805","nonce":1841901852}]},"public_addr":"172.21.15.161:6801/1841901852","cluster_addr":"172.21.15.161:6803/1841901852","heartbeat_back_addr":"172.21.15.161:6807/1841901852","heartbeat_front_addr":"172.21.15.161:6805/1841901852","state":["exists","up"]},{"osd":2,"uuid":"fc4bfd37-1fdd-47c6-a48e-6d910daf9e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6801","nonce":346103099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6803","nonce":346103099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6807","nonce":346103099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6805","nonce":346103099}]},"public_addr":"172.21.15.171:6801/346103099","cluster_addr":"172.21.15.171:6803/346103099","heartbeat_back_addr":"172.21.15.171:6807/346103099","heartbeat_front_addr":"172.21.15.171:6805/346103099","state":["exists","up"]},{"osd":3,"uuid":"4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6811","nonce":2152994143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6813","nonce":2152994143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6817","nonce":2152994143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6815","nonce":2152994143}]},"public_addr":"172.21.15.133:6811/2152994143","cluster_addr":"172.21.15.133:6813/2152994143","heartbeat_back_addr":"172.21.15.133:6817/2152994143","heartbeat_front_addr":"172.21.15.133:6815/2152994143","state":["exists","up"]},{"osd":4,"uuid":"4601a1c2-254b-4c4d-9b15-8fc952b3d2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6809","nonce":47213291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6811","nonce":47213291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6815","nonce":47213291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6813","nonce":47213291}]},"public_addr":"172.21.15.161:6809/47213291","cluster_addr":"172.21.15.161:6811/47213291","heartbeat_back_addr":"172.21.15.161:6815/47213291","heartbeat_front_addr":"172.21.15.161:6813/47213291","state":["exists","up"]},{"osd":5,"uuid":"6cca180a-9140-4e49-835b-86842dae0574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6809","nonce":777556751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6811","nonce":777556751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6815","nonce":777556751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6813","nonce":777556751}]},"public_addr":"172.21.15.171:6809/777556751","cluster_addr":"172.21.15.171:6811/777556751","heartbeat_back_addr":"172.21.15.171:6815/777556751","heartbeat_front_addr":"172.21.15.171:6813/777556751","state":["exists","up"]},{"osd":6,"uuid":"ea0299ff-7740-4494-a3a9-1503b3fc0ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6819","nonce":2884152358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6821","nonce":2884152358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6825","nonce":2884152358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6823","nonce":2884152358}]},"public_addr":"172.21.15.133:6819/2884152358","cluster_addr":"172.21.15.133:6821/2884152358","heartbeat_back_addr":"172.21.15.133:6825/2884152358","heartbeat_front_addr":"172.21.15.133:6823/2884152358","state":["exists","up"]},{"osd":7,"uuid":"b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6817","nonce":4183705749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6819","nonce":4183705749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6823","nonce":4183705749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6821","nonce":4183705749}]},"public_addr":"172.21.15.161:6817/4183705749","cluster_addr":"172.21.15.161:6819/4183705749","heartbeat_back_addr":"172.21.15.161:6823/4183705749","heartbeat_front_addr":"172.21.15.161:6821/4183705749","state":["exists","up"]},{"osd":8,"uuid":"87dffc07-bf48-48c8-b418-5c4b509686d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6817","nonce":2771494483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6819","nonce":2771494483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6823","nonce":2771494483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6821","nonce":2771494483}]},"public_addr":"172.21.15.171:6817/2771494483","cluster_addr":"172.21.15.171:6819/2771494483","heartbeat_back_addr":"172.21.15.171:6823/2771494483","heartbeat_front_addr":"172.21.15.171:6821/2771494483","state":["exists","up"]},{"osd":9,"uuid":"e0832870-79ce-4bce-8045-58b7a7f6dc69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6827","nonce":2066164149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6829","nonce":2066164149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6833","nonce":2066164149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6831","nonce":2066164149}]},"public_addr":"172.21.15.133:6827/2066164149","cluster_addr":"172.21.15.133:6829/2066164149","heartbeat_back_addr":"172.21.15.133:6833/2066164149","heartbeat_front_addr":"172.21.15.133:6831/2066164149","state":["exists","up"]},{"osd":10,"uuid":"081f0484-d134-4b2e-9c48-ef8e6a646422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6825","nonce":2162333626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6827","nonce":2162333626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6831","nonce":2162333626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6829","nonce":2162333626}]},"public_addr":"172.21.15.161:6825/2162333626","cluster_addr":"172.21.15.161:6827/2162333626","heartbeat_back_addr":"172.21.15.161:6831/2162333626","heartbeat_front_addr":"172.21.15.161:6829/2162333626","state":["exists","up"]},{"osd":11,"uuid":"4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6825","nonce":3403894313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6827","nonce":3403894313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6831","nonce":3403894313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6829","nonce":3403894313}]},"public_addr":"172.21.15.171:6825/3403894313","cluster_addr":"172.21.15.171:6827/3403894313","heartbeat_back_addr":"172.21.15.171:6831/3403894313","heartbeat_front_addr":"172.21.15.171:6829/3403894313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:17.816229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:39.638543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:00.543317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:26.988921+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:48.644694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:10.049860+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:35.986508+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:57.753846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:19.018245+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:44.460450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:06.279960+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:27.955006+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/1832569241":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6801/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:0/4219815304":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:6801/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/432253615":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1840700480":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1807775439":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1089030734":"2023-07-15T18:07:47.346704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T18:14:34.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.077+0000 7feff03ff700 1 -- 172.21.15.133:0/4052699988 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fefd40a11a0 msgr2=0x7fefd40a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:34.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.077+0000 7feff03ff700 1 --2- 172.21.15.133:0/4052699988 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fefd40a11a0 0x7fefd40a3650 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fefe400b2e0 tx=0x7fefe4005960 comp rx=0 tx=0).stop 2023-07-14T18:14:34.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.077+0000 7feff03ff700 1 -- 172.21.15.133:0/4052699988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefe8100e20 msgr2=0x7fefe8072a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:34.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.077+0000 7feff03ff700 1 --2- 172.21.15.133:0/4052699988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefe8100e20 0x7fefe8072a30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fefd800e9a0 tx=0x7fefd800ecb0 comp rx=0 tx=0).stop 2023-07-14T18:14:34.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.077+0000 7feff03ff700 1 -- 172.21.15.133:0/4052699988 shutdown_connections 2023-07-14T18:14:34.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.077+0000 7feff03ff700 1 --2- 172.21.15.133:0/4052699988 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fefd40a11a0 0x7fefd40a3650 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:34.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.077+0000 7feff03ff700 1 --2- 172.21.15.133:0/4052699988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefe80ffc30 0x7fefe80743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:34.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.077+0000 7feff03ff700 1 --2- 172.21.15.133:0/4052699988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fefe81017b0 0x7fefe8072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:34.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.077+0000 7feff03ff700 1 --2- 172.21.15.133:0/4052699988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefe8100e20 0x7fefe8072a30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:34.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.077+0000 7feff03ff700 1 -- 172.21.15.133:0/4052699988 >> 172.21.15.133:0/4052699988 conn(0x7fefe80fb210 msgr2=0x7fefe80fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:34.122 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.077+0000 7feff03ff700 1 -- 172.21.15.133:0/4052699988 shutdown_connections 2023-07-14T18:14:34.122 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.077+0000 7feff03ff700 1 -- 172.21.15.133:0/4052699988 wait complete. 2023-07-14T18:14:34.139 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T18:10:03.945039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T18:14:15.451340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T18:14:16.892215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T18:14:20.493175+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T18:14:34.140 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T18:14:34.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.297+0000 7ff4855ff700 1 -- 172.21.15.133:0/1532941667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff480100e10 msgr2=0x7ff480101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:34.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.297+0000 7ff4855ff700 1 --2- 172.21.15.133:0/1532941667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff480100e10 0x7ff480101260 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff474009990 tx=0x7ff474009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:34.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.297+0000 7ff4855ff700 1 -- 172.21.15.133:0/1532941667 shutdown_connections 2023-07-14T18:14:34.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.297+0000 7ff4855ff700 1 --2- 172.21.15.133:0/1532941667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4800ffc20 0x7ff480100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:34.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.297+0000 7ff4855ff700 1 --2- 172.21.15.133:0/1532941667 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4801017a0 0x7ff480108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:34.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.297+0000 7ff4855ff700 1 --2- 172.21.15.133:0/1532941667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff480100e10 0x7ff480101260 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:34.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.297+0000 7ff4855ff700 1 -- 172.21.15.133:0/1532941667 >> 172.21.15.133:0/1532941667 conn(0x7ff4800fb260 msgr2=0x7ff4800fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:34.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.297+0000 7ff4855ff700 1 -- 172.21.15.133:0/1532941667 shutdown_connections 2023-07-14T18:14:34.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.297+0000 7ff4855ff700 1 -- 172.21.15.133:0/1532941667 wait complete. 2023-07-14T18:14:34.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.299+0000 7ff4855ff700 1 Processor -- start 2023-07-14T18:14:34.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.299+0000 7ff4855ff700 1 -- start start 2023-07-14T18:14:34.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.299+0000 7ff4855ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4800ffc20 0x7ff480195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:34.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.299+0000 7ff4855ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff480100e10 0x7ff480195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:34.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.299+0000 7ff4855ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4801017a0 0x7ff48019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:34.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.299+0000 7ff4855ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48019d4f0 con 0x7ff480100e10 2023-07-14T18:14:34.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.299+0000 7ff4855ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48019d630 con 0x7ff4800ffc20 2023-07-14T18:14:34.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.299+0000 7ff4855ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48019d7c0 con 0x7ff4801017a0 2023-07-14T18:14:34.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.300+0000 7ff47effd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4800ffc20 0x7ff480195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:34.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.300+0000 7ff47effd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4800ffc20 0x7ff480195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:54730/0 (socket says 172.21.15.133:54730) 2023-07-14T18:14:34.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.300+0000 7ff47effd700 1 -- 172.21.15.133:0/3626059835 learned_addr learned my addr 172.21.15.133:0/3626059835 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:34.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.300+0000 7ff47e7fc700 1 --2- 172.21.15.133:0/3626059835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff480100e10 0x7ff480195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:34.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.300+0000 7ff47f7fe700 1 --2- 172.21.15.133:0/3626059835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4801017a0 0x7ff48019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:34.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.300+0000 7ff47effd700 1 -- 172.21.15.133:0/3626059835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4801017a0 msgr2=0x7ff48019cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:34.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.300+0000 7ff47effd700 1 --2- 172.21.15.133:0/3626059835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4801017a0 0x7ff48019cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:34.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.300+0000 7ff47effd700 1 -- 172.21.15.133:0/3626059835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff480100e10 msgr2=0x7ff480195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:34.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.300+0000 7ff47effd700 1 --2- 172.21.15.133:0/3626059835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff480100e10 0x7ff480195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:34.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.300+0000 7ff47effd700 1 -- 172.21.15.133:0/3626059835 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4740095f0 con 0x7ff4800ffc20 2023-07-14T18:14:34.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.300+0000 7ff47f7fe700 1 --2- 172.21.15.133:0/3626059835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4801017a0 0x7ff48019cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T18:14:34.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.300+0000 7ff47effd700 1 --2- 172.21.15.133:0/3626059835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4800ffc20 0x7ff480195970 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff47000e970 tx=0x7ff47000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:34.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.300+0000 7ff467fff700 1 -- 172.21.15.133:0/3626059835 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff470009790 con 0x7ff4800ffc20 2023-07-14T18:14:34.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.301+0000 7ff4855ff700 1 -- 172.21.15.133:0/3626059835 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff48019daa0 con 0x7ff4800ffc20 2023-07-14T18:14:34.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.301+0000 7ff467fff700 1 -- 172.21.15.133:0/3626059835 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff47000eec0 con 0x7ff4800ffc20 2023-07-14T18:14:34.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.301+0000 7ff467fff700 1 -- 172.21.15.133:0/3626059835 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff470003cf0 con 0x7ff4800ffc20 2023-07-14T18:14:34.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.301+0000 7ff4855ff700 1 -- 172.21.15.133:0/3626059835 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48019e000 con 0x7ff4800ffc20 2023-07-14T18:14:34.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.302+0000 7ff4855ff700 1 -- 172.21.15.133:0/3626059835 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff48010b440 con 0x7ff4800ffc20 2023-07-14T18:14:34.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.304+0000 7ff467fff700 1 -- 172.21.15.133:0/3626059835 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff47000ccd0 con 0x7ff4800ffc20 2023-07-14T18:14:34.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.305+0000 7ff467fff700 1 --2- 172.21.15.133:0/3626059835 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff4680a11d0 0x7ff4680a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:34.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.305+0000 7ff467fff700 1 -- 172.21.15.133:0/3626059835 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff470014070 con 0x7ff4800ffc20 2023-07-14T18:14:34.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.305+0000 7ff47e7fc700 1 --2- 172.21.15.133:0/3626059835 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff4680a11d0 0x7ff4680a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:34.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.306+0000 7ff47e7fc700 1 --2- 172.21.15.133:0/3626059835 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff4680a11d0 0x7ff4680a3680 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff47400b2e0 tx=0x7ff474005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:34.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.307+0000 7ff467fff700 1 -- 172.21.15.133:0/3626059835 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7ff4700963d0 con 0x7ff4800ffc20 2023-07-14T18:14:34.549 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.506+0000 7ff4855ff700 1 -- 172.21.15.133:0/3626059835 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff480106b20 con 0x7ff4800ffc20 2023-07-14T18:14:34.550 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.507+0000 7ff467fff700 1 -- 172.21.15.133:0/3626059835 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff4700999f0 con 0x7ff4800ffc20 2023-07-14T18:14:34.551 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-07-14T18:14:34.553 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.510+0000 7ff4855ff700 1 -- 172.21.15.133:0/3626059835 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff4680a11d0 msgr2=0x7ff4680a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:34.553 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.510+0000 7ff4855ff700 1 --2- 172.21.15.133:0/3626059835 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff4680a11d0 0x7ff4680a3680 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff47400b2e0 tx=0x7ff474005fb0 comp rx=0 tx=0).stop 2023-07-14T18:14:34.553 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.510+0000 7ff4855ff700 1 -- 172.21.15.133:0/3626059835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4800ffc20 msgr2=0x7ff480195970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:34.553 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.510+0000 7ff4855ff700 1 --2- 172.21.15.133:0/3626059835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4800ffc20 0x7ff480195970 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff47000e970 tx=0x7ff47000ec80 comp rx=0 tx=0).stop 2023-07-14T18:14:34.554 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.510+0000 7ff4855ff700 1 -- 172.21.15.133:0/3626059835 shutdown_connections 2023-07-14T18:14:34.554 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.510+0000 7ff4855ff700 1 --2- 172.21.15.133:0/3626059835 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff4680a11d0 0x7ff4680a3680 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:34.554 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.510+0000 7ff4855ff700 1 --2- 172.21.15.133:0/3626059835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4800ffc20 0x7ff480195970 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:34.554 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.510+0000 7ff4855ff700 1 --2- 172.21.15.133:0/3626059835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4801017a0 0x7ff48019cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:34.554 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.510+0000 7ff4855ff700 1 --2- 172.21.15.133:0/3626059835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff480100e10 0x7ff480195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:34.554 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.510+0000 7ff4855ff700 1 -- 172.21.15.133:0/3626059835 >> 172.21.15.133:0/3626059835 conn(0x7ff4800fb260 msgr2=0x7ff4800fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:34.554 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.510+0000 7ff4855ff700 1 -- 172.21.15.133:0/3626059835 shutdown_connections 2023-07-14T18:14:34.554 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.510+0000 7ff4855ff700 1 -- 172.21.15.133:0/3626059835 wait complete. 2023-07-14T18:14:34.571 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T18:14:34.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.728+0000 7f450bbff700 1 -- 172.21.15.133:0/4209726927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4504101790 msgr2=0x7f4504108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:34.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.728+0000 7f450bbff700 1 --2- 172.21.15.133:0/4209726927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4504101790 0x7f4504108050 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009f90 tx=0x7f44fc00b330 comp rx=0 tx=0).stop 2023-07-14T18:14:34.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.728+0000 7f450bbff700 1 -- 172.21.15.133:0/4209726927 shutdown_connections 2023-07-14T18:14:34.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.728+0000 7f450bbff700 1 --2- 172.21.15.133:0/4209726927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45040ffc10 0x7f4504100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:34.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.728+0000 7f450bbff700 1 --2- 172.21.15.133:0/4209726927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4504101790 0x7f4504108050 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:34.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.728+0000 7f450bbff700 1 --2- 172.21.15.133:0/4209726927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4504100e00 0x7f4504101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:34.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.728+0000 7f450bbff700 1 -- 172.21.15.133:0/4209726927 >> 172.21.15.133:0/4209726927 conn(0x7f45040fb210 msgr2=0x7f45040fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:34.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.728+0000 7f450bbff700 1 -- 172.21.15.133:0/4209726927 shutdown_connections 2023-07-14T18:14:34.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.728+0000 7f450bbff700 1 -- 172.21.15.133:0/4209726927 wait complete. 2023-07-14T18:14:34.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.729+0000 7f450bbff700 1 Processor -- start 2023-07-14T18:14:34.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.730+0000 7f450bbff700 1 -- start start 2023-07-14T18:14:34.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.730+0000 7f450bbff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45040ffc10 0x7f4504195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:34.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.730+0000 7f450bbff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4504100e00 0x7f4504196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:34.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.730+0000 7f450bbff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4504101790 0x7f450419d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:34.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.730+0000 7f450bbff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450419d640 con 0x7f4504100e00 2023-07-14T18:14:34.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.730+0000 7f450bbff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450419d780 con 0x7f45040ffc10 2023-07-14T18:14:34.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.730+0000 7f450bbff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450419d910 con 0x7f4504101790 2023-07-14T18:14:34.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.730+0000 7f450919a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4504100e00 0x7f4504196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:34.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.730+0000 7f450919a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4504100e00 0x7f4504196000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45468/0 (socket says 172.21.15.133:45468) 2023-07-14T18:14:34.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.730+0000 7f450919a700 1 -- 172.21.15.133:0/2161487037 learned_addr learned my addr 172.21.15.133:0/2161487037 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:34.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.730+0000 7f450999b700 1 --2- 172.21.15.133:0/2161487037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45040ffc10 0x7f4504195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:34.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.730+0000 7f450a19c700 1 --2- 172.21.15.133:0/2161487037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4504101790 0x7f450419d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:34.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.731+0000 7f450919a700 1 -- 172.21.15.133:0/2161487037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4504101790 msgr2=0x7f450419d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:34.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.731+0000 7f450919a700 1 --2- 172.21.15.133:0/2161487037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4504101790 0x7f450419d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:34.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.731+0000 7f450919a700 1 -- 172.21.15.133:0/2161487037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45040ffc10 msgr2=0x7f4504195ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:34.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.731+0000 7f450919a700 1 --2- 172.21.15.133:0/2161487037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45040ffc10 0x7f4504195ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:34.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.731+0000 7f450919a700 1 -- 172.21.15.133:0/2161487037 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44fc009c40 con 0x7f4504100e00 2023-07-14T18:14:34.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.731+0000 7f450919a700 1 --2- 172.21.15.133:0/2161487037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4504100e00 0x7f4504196000 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f44f400e9a0 tx=0x7f44f400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:34.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.731+0000 7f44faffd700 1 -- 172.21.15.133:0/2161487037 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f4009790 con 0x7f4504100e00 2023-07-14T18:14:34.778 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.731+0000 7f44faffd700 1 -- 172.21.15.133:0/2161487037 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44f400ce80 con 0x7f4504100e00 2023-07-14T18:14:34.778 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.731+0000 7f44faffd700 1 -- 172.21.15.133:0/2161487037 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f4018680 con 0x7f4504100e00 2023-07-14T18:14:34.778 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.731+0000 7f450bbff700 1 -- 172.21.15.133:0/2161487037 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f450419dbf0 con 0x7f4504100e00 2023-07-14T18:14:34.778 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.731+0000 7f450bbff700 1 -- 172.21.15.133:0/2161487037 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f450419e100 con 0x7f4504100e00 2023-07-14T18:14:34.778 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.732+0000 7f450bbff700 1 -- 172.21.15.133:0/2161487037 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4504107150 con 0x7f4504100e00 2023-07-14T18:14:34.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.734+0000 7f44faffd700 1 -- 172.21.15.133:0/2161487037 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f44f4010430 con 0x7f4504100e00 2023-07-14T18:14:34.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.736+0000 7f44faffd700 1 --2- 172.21.15.133:0/2161487037 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f44f00a11d0 0x7f44f00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:34.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.736+0000 7f44faffd700 1 -- 172.21.15.133:0/2161487037 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f44f4014070 con 0x7f4504100e00 2023-07-14T18:14:34.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.736+0000 7f450999b700 1 --2- 172.21.15.133:0/2161487037 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f44f00a11d0 0x7f44f00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:34.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.736+0000 7f450999b700 1 --2- 172.21.15.133:0/2161487037 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f44f00a11d0 0x7f44f00a3680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4504100c60 tx=0x7f4500009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:34.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.739+0000 7f44faffd700 1 -- 172.21.15.133:0/2161487037 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f44f4051780 con 0x7f4504100e00 2023-07-14T18:14:34.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.937+0000 7f450bbff700 1 -- 172.21.15.133:0/2161487037 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4504061350 con 0x7f4504100e00 2023-07-14T18:14:34.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.938+0000 7f44faffd700 1 -- 172.21.15.133:0/2161487037 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f44f4099c30 con 0x7f4504100e00 2023-07-14T18:14:34.981 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-07-14T18:14:34.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.940+0000 7f450bbff700 1 -- 172.21.15.133:0/2161487037 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f44f00a11d0 msgr2=0x7f44f00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:34.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.940+0000 7f450bbff700 1 --2- 172.21.15.133:0/2161487037 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f44f00a11d0 0x7f44f00a3680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4504100c60 tx=0x7f4500009310 comp rx=0 tx=0).stop 2023-07-14T18:14:34.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.940+0000 7f450bbff700 1 -- 172.21.15.133:0/2161487037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4504100e00 msgr2=0x7f4504196000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:34.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.940+0000 7f450bbff700 1 --2- 172.21.15.133:0/2161487037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4504100e00 0x7f4504196000 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f44f400e9a0 tx=0x7f44f400ecb0 comp rx=0 tx=0).stop 2023-07-14T18:14:34.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.940+0000 7f450bbff700 1 -- 172.21.15.133:0/2161487037 shutdown_connections 2023-07-14T18:14:34.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.940+0000 7f450bbff700 1 --2- 172.21.15.133:0/2161487037 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f44f00a11d0 0x7f44f00a3680 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:34.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.940+0000 7f450bbff700 1 --2- 172.21.15.133:0/2161487037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45040ffc10 0x7f4504195ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:34.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.940+0000 7f450bbff700 1 --2- 172.21.15.133:0/2161487037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4504101790 0x7f450419d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:34.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.940+0000 7f450bbff700 1 --2- 172.21.15.133:0/2161487037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4504100e00 0x7f4504196000 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:34.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.940+0000 7f450bbff700 1 -- 172.21.15.133:0/2161487037 >> 172.21.15.133:0/2161487037 conn(0x7f45040fb210 msgr2=0x7f4504104290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:34.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.940+0000 7f450bbff700 1 -- 172.21.15.133:0/2161487037 shutdown_connections 2023-07-14T18:14:34.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:34.940+0000 7f450bbff700 1 -- 172.21.15.133:0/2161487037 wait complete. 2023-07-14T18:14:35.001 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T18:14:35.201 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.158+0000 7fa4dc1ff700 1 -- 172.21.15.133:0/3596250223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4d41017a0 msgr2=0x7fa4d4108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:35.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.158+0000 7fa4dc1ff700 1 --2- 172.21.15.133:0/3596250223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4d41017a0 0x7fa4d4108060 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc00b0a0 tx=0x7fa4cc00b3b0 comp rx=0 tx=0).stop 2023-07-14T18:14:35.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.158+0000 7fa4dc1ff700 1 -- 172.21.15.133:0/3596250223 shutdown_connections 2023-07-14T18:14:35.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.158+0000 7fa4dc1ff700 1 --2- 172.21.15.133:0/3596250223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4d40ffc20 0x7fa4d4100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:35.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.158+0000 7fa4dc1ff700 1 --2- 172.21.15.133:0/3596250223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4d41017a0 0x7fa4d4108060 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:35.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.158+0000 7fa4dc1ff700 1 --2- 172.21.15.133:0/3596250223 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4d4100e10 0x7fa4d4101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:35.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.158+0000 7fa4dc1ff700 1 -- 172.21.15.133:0/3596250223 >> 172.21.15.133:0/3596250223 conn(0x7fa4d40fb260 msgr2=0x7fa4d40fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:35.203 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.158+0000 7fa4dc1ff700 1 -- 172.21.15.133:0/3596250223 shutdown_connections 2023-07-14T18:14:35.203 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.158+0000 7fa4dc1ff700 1 -- 172.21.15.133:0/3596250223 wait complete. 2023-07-14T18:14:35.204 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.159+0000 7fa4dc1ff700 1 Processor -- start 2023-07-14T18:14:35.205 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.159+0000 7fa4dc1ff700 1 -- start start 2023-07-14T18:14:35.205 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.160+0000 7fa4dc1ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4d40ffc20 0x7fa4d4197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:35.205 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.160+0000 7fa4dc1ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4d4100e10 0x7fa4d41981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:35.205 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.160+0000 7fa4dc1ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4d41017a0 0x7fa4d419f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:35.205 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.160+0000 7fa4dc1ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d419f820 con 0x7fa4d41017a0 2023-07-14T18:14:35.205 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.160+0000 7fa4dc1ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d419f990 con 0x7fa4d40ffc20 2023-07-14T18:14:35.205 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.160+0000 7fa4dc1ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d419fb50 con 0x7fa4d4100e10 2023-07-14T18:14:35.206 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.160+0000 7fa4da79c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4d41017a0 0x7fa4d419f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:35.206 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.160+0000 7fa4d9f9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4d40ffc20 0x7fa4d4197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:35.206 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.160+0000 7fa4d9f9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4d40ffc20 0x7fa4d4197cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:54754/0 (socket says 172.21.15.133:54754) 2023-07-14T18:14:35.206 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.160+0000 7fa4d9f9b700 1 -- 172.21.15.133:0/2908100753 learned_addr learned my addr 172.21.15.133:0/2908100753 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:35.206 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.160+0000 7fa4d979a700 1 --2- 172.21.15.133:0/2908100753 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4d4100e10 0x7fa4d41981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:35.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.160+0000 7fa4da79c700 1 -- 172.21.15.133:0/2908100753 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4d4100e10 msgr2=0x7fa4d41981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:35.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.160+0000 7fa4da79c700 1 --2- 172.21.15.133:0/2908100753 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4d4100e10 0x7fa4d41981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:35.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.160+0000 7fa4da79c700 1 -- 172.21.15.133:0/2908100753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4d40ffc20 msgr2=0x7fa4d4197cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:35.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.160+0000 7fa4da79c700 1 --2- 172.21.15.133:0/2908100753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4d40ffc20 0x7fa4d4197cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:35.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.160+0000 7fa4da79c700 1 -- 172.21.15.133:0/2908100753 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d0009520 con 0x7fa4d41017a0 2023-07-14T18:14:35.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.161+0000 7fa4da79c700 1 --2- 172.21.15.133:0/2908100753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4d41017a0 0x7fa4d419f250 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc00b070 tx=0x7fa4cc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:35.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.161+0000 7fa4caffd700 1 -- 172.21.15.133:0/2908100753 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc00e040 con 0x7fa4d41017a0 2023-07-14T18:14:35.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.161+0000 7fa4caffd700 1 -- 172.21.15.133:0/2908100753 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4cc009480 con 0x7fa4d41017a0 2023-07-14T18:14:35.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.161+0000 7fa4dc1ff700 1 -- 172.21.15.133:0/2908100753 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4cc009d00 con 0x7fa4d41017a0 2023-07-14T18:14:35.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.161+0000 7fa4caffd700 1 -- 172.21.15.133:0/2908100753 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc0052a0 con 0x7fa4d41017a0 2023-07-14T18:14:35.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.161+0000 7fa4dc1ff700 1 -- 172.21.15.133:0/2908100753 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d41a00c0 con 0x7fa4d41017a0 2023-07-14T18:14:35.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.162+0000 7fa4dc1ff700 1 -- 172.21.15.133:0/2908100753 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4d410b470 con 0x7fa4d41017a0 2023-07-14T18:14:35.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.164+0000 7fa4caffd700 1 -- 172.21.15.133:0/2908100753 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fa4cc01d480 con 0x7fa4d41017a0 2023-07-14T18:14:35.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.165+0000 7fa4caffd700 1 --2- 172.21.15.133:0/2908100753 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa4c00a11d0 0x7fa4c00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:35.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.165+0000 7fa4caffd700 1 -- 172.21.15.133:0/2908100753 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa4cc0cdac0 con 0x7fa4d41017a0 2023-07-14T18:14:35.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.165+0000 7fa4d9f9b700 1 --2- 172.21.15.133:0/2908100753 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa4c00a11d0 0x7fa4c00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:35.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.166+0000 7fa4d9f9b700 1 --2- 172.21.15.133:0/2908100753 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa4c00a11d0 0x7fa4c00a3680 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4198b80 tx=0x7fa4d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:35.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.169+0000 7fa4caffd700 1 -- 172.21.15.133:0/2908100753 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fa4cc097050 con 0x7fa4d41017a0 2023-07-14T18:14:35.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:34 smithi161 ceph-mon[128993]: pgmap v281: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 98 B/s wr, 0 op/s 2023-07-14T18:14:35.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:34 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/4052699988' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:14:35.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:34 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3626059835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:14:35.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:34 smithi171 ceph-mon[129241]: pgmap v281: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 98 B/s wr, 0 op/s 2023-07-14T18:14:35.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:34 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/4052699988' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:14:35.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:34 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3626059835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:14:35.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:34 smithi133 ceph-mon[126384]: pgmap v281: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 98 B/s wr, 0 op/s 2023-07-14T18:14:35.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:34 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/4052699988' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:14:35.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:34 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3626059835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:14:35.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.364+0000 7fa4dc1ff700 1 -- 172.21.15.133:0/2908100753 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa4d41a0670 con 0x7fa4d41017a0 2023-07-14T18:14:35.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.365+0000 7fa4caffd700 1 -- 172.21.15.133:0/2908100753 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa4cc09a670 con 0x7fa4d41017a0 2023-07-14T18:14:35.408 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2023-07-14T18:14:35.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.367+0000 7fa4dc1ff700 1 -- 172.21.15.133:0/2908100753 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa4c00a11d0 msgr2=0x7fa4c00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:35.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.367+0000 7fa4dc1ff700 1 --2- 172.21.15.133:0/2908100753 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa4c00a11d0 0x7fa4c00a3680 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4198b80 tx=0x7fa4d0009260 comp rx=0 tx=0).stop 2023-07-14T18:14:35.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.367+0000 7fa4dc1ff700 1 -- 172.21.15.133:0/2908100753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4d41017a0 msgr2=0x7fa4d419f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:35.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.367+0000 7fa4dc1ff700 1 --2- 172.21.15.133:0/2908100753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4d41017a0 0x7fa4d419f250 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc00b070 tx=0x7fa4cc00bc80 comp rx=0 tx=0).stop 2023-07-14T18:14:35.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.367+0000 7fa4dc1ff700 1 -- 172.21.15.133:0/2908100753 shutdown_connections 2023-07-14T18:14:35.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.367+0000 7fa4dc1ff700 1 --2- 172.21.15.133:0/2908100753 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa4c00a11d0 0x7fa4c00a3680 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:35.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.367+0000 7fa4dc1ff700 1 --2- 172.21.15.133:0/2908100753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4d40ffc20 0x7fa4d4197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:35.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.367+0000 7fa4dc1ff700 1 --2- 172.21.15.133:0/2908100753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4d41017a0 0x7fa4d419f250 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:35.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.367+0000 7fa4dc1ff700 1 --2- 172.21.15.133:0/2908100753 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4d4100e10 0x7fa4d41981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:35.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.367+0000 7fa4dc1ff700 1 -- 172.21.15.133:0/2908100753 >> 172.21.15.133:0/2908100753 conn(0x7fa4d40fb260 msgr2=0x7fa4d40fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:35.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.368+0000 7fa4dc1ff700 1 -- 172.21.15.133:0/2908100753 shutdown_connections 2023-07-14T18:14:35.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.368+0000 7fa4dc1ff700 1 -- 172.21.15.133:0/2908100753 wait complete. 2023-07-14T18:14:35.429 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T18:14:35.630 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.586+0000 7f152c3ff700 1 -- 172.21.15.133:0/2953270986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1524107ad0 msgr2=0x7f1524109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:35.630 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.586+0000 7f152c3ff700 1 --2- 172.21.15.133:0/2953270986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1524107ad0 0x7f1524109f50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f151c00b0a0 tx=0x7f151c00b3b0 comp rx=0 tx=0).stop 2023-07-14T18:14:35.630 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.586+0000 7f152c3ff700 1 -- 172.21.15.133:0/2953270986 shutdown_connections 2023-07-14T18:14:35.630 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.586+0000 7f152c3ff700 1 --2- 172.21.15.133:0/2953270986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1524107ad0 0x7f1524109f50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:35.630 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.586+0000 7f152c3ff700 1 --2- 172.21.15.133:0/2953270986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1524074680 0x7f1524072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:35.630 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.586+0000 7f152c3ff700 1 --2- 172.21.15.133:0/2953270986 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15240730a0 0x7f1524073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:35.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.586+0000 7f152c3ff700 1 -- 172.21.15.133:0/2953270986 >> 172.21.15.133:0/2953270986 conn(0x7f15240fb210 msgr2=0x7f15240fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:35.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.586+0000 7f152c3ff700 1 -- 172.21.15.133:0/2953270986 shutdown_connections 2023-07-14T18:14:35.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.586+0000 7f152c3ff700 1 -- 172.21.15.133:0/2953270986 wait complete. 2023-07-14T18:14:35.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.587+0000 7f152c3ff700 1 Processor -- start 2023-07-14T18:14:35.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.587+0000 7f152c3ff700 1 -- start start 2023-07-14T18:14:35.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.588+0000 7f152c3ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15240730a0 0x7f1524197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:35.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.588+0000 7f152c3ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1524074680 0x7f1524198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:35.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.588+0000 7f152c3ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1524107ad0 0x7f152419f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:35.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.588+0000 7f152c3ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152419fa50 con 0x7f15240730a0 2023-07-14T18:14:35.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.588+0000 7f152c3ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152419fb90 con 0x7f1524074680 2023-07-14T18:14:35.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.588+0000 7f152c3ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152419fd20 con 0x7f1524107ad0 2023-07-14T18:14:35.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.588+0000 7f152a19b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15240730a0 0x7f1524197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:35.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.588+0000 7f152999a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1524074680 0x7f1524198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:35.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.588+0000 7f152999a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1524074680 0x7f1524198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:54780/0 (socket says 172.21.15.133:54780) 2023-07-14T18:14:35.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.588+0000 7f152999a700 1 -- 172.21.15.133:0/3900019959 learned_addr learned my addr 172.21.15.133:0/3900019959 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:35.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.588+0000 7f152a99c700 1 --2- 172.21.15.133:0/3900019959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1524107ad0 0x7f152419f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:35.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.588+0000 7f152999a700 1 -- 172.21.15.133:0/3900019959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1524107ad0 msgr2=0x7f152419f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:35.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.588+0000 7f152999a700 1 --2- 172.21.15.133:0/3900019959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1524107ad0 0x7f152419f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:35.635 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.588+0000 7f152999a700 1 -- 172.21.15.133:0/3900019959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15240730a0 msgr2=0x7f1524197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:35.635 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.588+0000 7f152999a700 1 --2- 172.21.15.133:0/3900019959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15240730a0 0x7f1524197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:35.635 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.588+0000 7f152999a700 1 -- 172.21.15.133:0/3900019959 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151c009d00 con 0x7f1524074680 2023-07-14T18:14:35.635 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.589+0000 7f152999a700 1 --2- 172.21.15.133:0/3900019959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1524074680 0x7f1524198410 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f151400e970 tx=0x7f151400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:35.635 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.589+0000 7f151b7fe700 1 -- 172.21.15.133:0/3900019959 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1514009790 con 0x7f1524074680 2023-07-14T18:14:35.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.589+0000 7f151b7fe700 1 -- 172.21.15.133:0/3900019959 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f151400c7e0 con 0x7f1524074680 2023-07-14T18:14:35.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.589+0000 7f152c3ff700 1 -- 172.21.15.133:0/3900019959 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f152419ffa0 con 0x7f1524074680 2023-07-14T18:14:35.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.589+0000 7f151b7fe700 1 -- 172.21.15.133:0/3900019959 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1514003c70 con 0x7f1524074680 2023-07-14T18:14:35.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.589+0000 7f152c3ff700 1 -- 172.21.15.133:0/3900019959 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15241a0480 con 0x7f1524074680 2023-07-14T18:14:35.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.590+0000 7f152c3ff700 1 -- 172.21.15.133:0/3900019959 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1524106890 con 0x7f1524074680 2023-07-14T18:14:35.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.592+0000 7f151b7fe700 1 -- 172.21.15.133:0/3900019959 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f151400cc50 con 0x7f1524074680 2023-07-14T18:14:35.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.593+0000 7f151b7fe700 1 --2- 172.21.15.133:0/3900019959 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f15100a1150 0x7f15100a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:35.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.594+0000 7f151b7fe700 1 -- 172.21.15.133:0/3900019959 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1514014070 con 0x7f1524074680 2023-07-14T18:14:35.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.594+0000 7f152a19b700 1 --2- 172.21.15.133:0/3900019959 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f15100a1150 0x7f15100a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:35.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.594+0000 7f152a19b700 1 --2- 172.21.15.133:0/3900019959 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f15100a1150 0x7f15100a3600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1524074a50 tx=0x7f1520009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:35.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.596+0000 7f151b7fe700 1 -- 172.21.15.133:0/3900019959 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f1514096380 con 0x7f1524074680 2023-07-14T18:14:35.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.793+0000 7f152c3ff700 1 -- 172.21.15.133:0/3900019959 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1524061250 con 0x7f1524074680 2023-07-14T18:14:35.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.794+0000 7f151b7fe700 1 -- 172.21.15.133:0/3900019959 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1514003dd0 con 0x7f1524074680 2023-07-14T18:14:35.837 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-07-14T18:14:35.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.796+0000 7f152c3ff700 1 -- 172.21.15.133:0/3900019959 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f15100a1150 msgr2=0x7f15100a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:35.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.796+0000 7f152c3ff700 1 --2- 172.21.15.133:0/3900019959 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f15100a1150 0x7f15100a3600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1524074a50 tx=0x7f1520009260 comp rx=0 tx=0).stop 2023-07-14T18:14:35.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.796+0000 7f152c3ff700 1 -- 172.21.15.133:0/3900019959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1524074680 msgr2=0x7f1524198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:35.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.796+0000 7f152c3ff700 1 --2- 172.21.15.133:0/3900019959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1524074680 0x7f1524198410 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f151400e970 tx=0x7f151400ec80 comp rx=0 tx=0).stop 2023-07-14T18:14:35.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.796+0000 7f152c3ff700 1 -- 172.21.15.133:0/3900019959 shutdown_connections 2023-07-14T18:14:35.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.796+0000 7f152c3ff700 1 --2- 172.21.15.133:0/3900019959 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f15100a1150 0x7f15100a3600 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:35.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.796+0000 7f152c3ff700 1 --2- 172.21.15.133:0/3900019959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1524107ad0 0x7f152419f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:35.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.796+0000 7f152c3ff700 1 --2- 172.21.15.133:0/3900019959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15240730a0 0x7f1524197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:35.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.796+0000 7f152c3ff700 1 --2- 172.21.15.133:0/3900019959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1524074680 0x7f1524198410 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:35.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.796+0000 7f152c3ff700 1 -- 172.21.15.133:0/3900019959 >> 172.21.15.133:0/3900019959 conn(0x7f15240fb210 msgr2=0x7f15241039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:35.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.797+0000 7f152c3ff700 1 -- 172.21.15.133:0/3900019959 shutdown_connections 2023-07-14T18:14:35.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:35.797+0000 7f152c3ff700 1 -- 172.21.15.133:0/3900019959 wait complete. 2023-07-14T18:14:35.858 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T18:14:36.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.013+0000 7f639e9ff700 1 -- 172.21.15.133:0/1251764026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6398073040 msgr2=0x7f63980734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:36.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.013+0000 7f639e9ff700 1 --2- 172.21.15.133:0/1251764026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6398073040 0x7f63980734b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f638c009990 tx=0x7f638c009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:36.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.014+0000 7f639e9ff700 1 -- 172.21.15.133:0/1251764026 shutdown_connections 2023-07-14T18:14:36.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.014+0000 7f639e9ff700 1 --2- 172.21.15.133:0/1251764026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6398074620 0x7f6398072a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:36.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.014+0000 7f639e9ff700 1 --2- 172.21.15.133:0/1251764026 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6398107ab0 0x7f6398109f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:36.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.014+0000 7f639e9ff700 1 --2- 172.21.15.133:0/1251764026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6398073040 0x7f63980734b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:36.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.014+0000 7f639e9ff700 1 -- 172.21.15.133:0/1251764026 >> 172.21.15.133:0/1251764026 conn(0x7f63980fb260 msgr2=0x7f63980fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:36.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.014+0000 7f639e9ff700 1 -- 172.21.15.133:0/1251764026 shutdown_connections 2023-07-14T18:14:36.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.014+0000 7f639e9ff700 1 -- 172.21.15.133:0/1251764026 wait complete. 2023-07-14T18:14:36.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.015+0000 7f639e9ff700 1 Processor -- start 2023-07-14T18:14:36.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.015+0000 7f639e9ff700 1 -- start start 2023-07-14T18:14:36.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.015+0000 7f639e9ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6398073040 0x7f6398197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:36.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.015+0000 7f639e9ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6398074620 0x7f6398198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:36.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.015+0000 7f639e9ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6398107ab0 0x7f639819f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:36.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.015+0000 7f639e9ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639819f830 con 0x7f6398073040 2023-07-14T18:14:36.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.015+0000 7f639e9ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639819f9a0 con 0x7f6398074620 2023-07-14T18:14:36.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.015+0000 7f639e9ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639819fb60 con 0x7f6398107ab0 2023-07-14T18:14:36.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.015+0000 7f6397fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6398073040 0x7f6398197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:36.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.015+0000 7f6397fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6398073040 0x7f6398197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45528/0 (socket says 172.21.15.133:45528) 2023-07-14T18:14:36.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.015+0000 7f6397fff700 1 -- 172.21.15.133:0/3054315801 learned_addr learned my addr 172.21.15.133:0/3054315801 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:36.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.016+0000 7f63977fe700 1 --2- 172.21.15.133:0/3054315801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6398074620 0x7f6398198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.016+0000 7f639cf9c700 1 --2- 172.21.15.133:0/3054315801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6398107ab0 0x7f639819f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.016+0000 7f6397fff700 1 -- 172.21.15.133:0/3054315801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6398107ab0 msgr2=0x7f639819f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.016+0000 7f6397fff700 1 --2- 172.21.15.133:0/3054315801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6398107ab0 0x7f639819f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.016+0000 7f6397fff700 1 -- 172.21.15.133:0/3054315801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6398074620 msgr2=0x7f6398198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.016+0000 7f6397fff700 1 --2- 172.21.15.133:0/3054315801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6398074620 0x7f6398198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.016+0000 7f6397fff700 1 -- 172.21.15.133:0/3054315801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638c0095f0 con 0x7f6398073040 2023-07-14T18:14:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.016+0000 7f639cf9c700 1 --2- 172.21.15.133:0/3054315801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6398107ab0 0x7f639819f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:14:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.016+0000 7f6397fff700 1 --2- 172.21.15.133:0/3054315801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6398073040 0x7f6398197cf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f638800d980 tx=0x7f638800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.016+0000 7f63957fa700 1 -- 172.21.15.133:0/3054315801 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6388004220 con 0x7f6398073040 2023-07-14T18:14:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.016+0000 7f639e9ff700 1 -- 172.21.15.133:0/3054315801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639819fe40 con 0x7f6398073040 2023-07-14T18:14:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.016+0000 7f63957fa700 1 -- 172.21.15.133:0/3054315801 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6388007640 con 0x7f6398073040 2023-07-14T18:14:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.016+0000 7f63957fa700 1 -- 172.21.15.133:0/3054315801 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6388005610 con 0x7f6398073040 2023-07-14T18:14:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.017+0000 7f639e9ff700 1 -- 172.21.15.133:0/3054315801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63981a0270 con 0x7f6398073040 2023-07-14T18:14:36.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.018+0000 7f639e9ff700 1 -- 172.21.15.133:0/3054315801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f639810d650 con 0x7f6398073040 2023-07-14T18:14:36.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.019+0000 7f63957fa700 1 -- 172.21.15.133:0/3054315801 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6388004750 con 0x7f6398073040 2023-07-14T18:14:36.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.021+0000 7f63957fa700 1 --2- 172.21.15.133:0/3054315801 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f63800a1150 0x7f63800a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:36.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.021+0000 7f63957fa700 1 -- 172.21.15.133:0/3054315801 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f63880cca60 con 0x7f6398073040 2023-07-14T18:14:36.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.021+0000 7f63977fe700 1 --2- 172.21.15.133:0/3054315801 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f63800a1150 0x7f63800a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:36.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.022+0000 7f63977fe700 1 --2- 172.21.15.133:0/3054315801 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f63800a1150 0x7f63800a3600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f638c00b5c0 tx=0x7f638c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:36.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.025+0000 7f63957fa700 1 -- 172.21.15.133:0/3054315801 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f6388095f60 con 0x7f6398073040 2023-07-14T18:14:36.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:35 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2161487037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:14:36.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:35 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2908100753' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:14:36.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:35 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3900019959' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:14:36.282 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.238+0000 7f639e9ff700 1 -- 172.21.15.133:0/3054315801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f63981a0880 con 0x7f6398073040 2023-07-14T18:14:36.282 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.239+0000 7f63957fa700 1 -- 172.21.15.133:0/3054315801 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) v1 ==== 81+0+597 (secure 0 0 0) 0x7f6388016020 con 0x7f6398073040 2023-07-14T18:14:36.282 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/1832569241 2023-07-15T18:07:47.346704+0000 2023-07-14T18:14:36.283 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6801/2986586981 2023-07-15T18:07:47.346704+0000 2023-07-14T18:14:36.283 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6800/2986586981 2023-07-15T18:07:47.346704+0000 2023-07-14T18:14:36.283 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/4219815304 2023-07-15T18:07:47.346704+0000 2023-07-14T18:14:36.284 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6800/58641721 2023-07-15T18:07:19.109711+0000 2023-07-14T18:14:36.284 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6801/58641721 2023-07-15T18:07:19.109711+0000 2023-07-14T18:14:36.284 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/432253615 2023-07-15T18:07:19.109711+0000 2023-07-14T18:14:36.284 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/1840700480 2023-07-15T18:07:19.109711+0000 2023-07-14T18:14:36.284 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/1807775439 2023-07-15T18:07:19.109711+0000 2023-07-14T18:14:36.284 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/1089030734 2023-07-15T18:07:47.346704+0000 2023-07-14T18:14:36.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.241+0000 7f639e9ff700 1 -- 172.21.15.133:0/3054315801 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f63800a1150 msgr2=0x7f63800a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:36.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.241+0000 7f639e9ff700 1 --2- 172.21.15.133:0/3054315801 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f63800a1150 0x7f63800a3600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f638c00b5c0 tx=0x7f638c005fd0 comp rx=0 tx=0).stop 2023-07-14T18:14:36.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.241+0000 7f639e9ff700 1 -- 172.21.15.133:0/3054315801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6398073040 msgr2=0x7f6398197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:36.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.241+0000 7f639e9ff700 1 --2- 172.21.15.133:0/3054315801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6398073040 0x7f6398197cf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f638800d980 tx=0x7f638800dc90 comp rx=0 tx=0).stop 2023-07-14T18:14:36.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.242+0000 7f639e9ff700 1 -- 172.21.15.133:0/3054315801 shutdown_connections 2023-07-14T18:14:36.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.242+0000 7f639e9ff700 1 --2- 172.21.15.133:0/3054315801 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f63800a1150 0x7f63800a3600 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:36.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.242+0000 7f639e9ff700 1 --2- 172.21.15.133:0/3054315801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6398107ab0 0x7f639819f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:36.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.242+0000 7f639e9ff700 1 --2- 172.21.15.133:0/3054315801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6398073040 0x7f6398197cf0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:36.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.242+0000 7f639e9ff700 1 --2- 172.21.15.133:0/3054315801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6398074620 0x7f6398198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:36.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.242+0000 7f639e9ff700 1 -- 172.21.15.133:0/3054315801 >> 172.21.15.133:0/3054315801 conn(0x7f63980fb260 msgr2=0x7f6398109d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:36.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.242+0000 7f639e9ff700 1 -- 172.21.15.133:0/3054315801 shutdown_connections 2023-07-14T18:14:36.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.242+0000 7f639e9ff700 1 -- 172.21.15.133:0/3054315801 wait complete. 2023-07-14T18:14:36.287 INFO:teuthology.orchestra.run.smithi133.stderr:listed 10 entries 2023-07-14T18:14:36.303 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T18:14:36.304 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T18:14:36.347 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T18:14:36.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:35 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2161487037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:14:36.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:35 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2908100753' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:14:36.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:35 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3900019959' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:14:36.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:35 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2161487037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:14:36.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:35 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2908100753' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:14:36.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:35 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3900019959' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:14:36.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.539+0000 7f4d84fff700 1 -- 172.21.15.133:0/1122352867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d800ffc30 msgr2=0x7f4d80100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:36.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.539+0000 7f4d84fff700 1 --2- 172.21.15.133:0/1122352867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d800ffc30 0x7f4d80100040 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009990 tx=0x7f4d70009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:36.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.540+0000 7f4d84fff700 1 -- 172.21.15.133:0/1122352867 shutdown_connections 2023-07-14T18:14:36.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.540+0000 7f4d84fff700 1 --2- 172.21.15.133:0/1122352867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d801017b0 0x7f4d80108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:36.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.540+0000 7f4d84fff700 1 --2- 172.21.15.133:0/1122352867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d800ffc30 0x7f4d80100040 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:36.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.540+0000 7f4d84fff700 1 --2- 172.21.15.133:0/1122352867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d80100e20 0x7f4d80101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:36.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.540+0000 7f4d84fff700 1 -- 172.21.15.133:0/1122352867 >> 172.21.15.133:0/1122352867 conn(0x7f4d800fb210 msgr2=0x7f4d800fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:36.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.540+0000 7f4d84fff700 1 -- 172.21.15.133:0/1122352867 shutdown_connections 2023-07-14T18:14:36.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.540+0000 7f4d84fff700 1 -- 172.21.15.133:0/1122352867 wait complete. 2023-07-14T18:14:36.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.541+0000 7f4d84fff700 1 Processor -- start 2023-07-14T18:14:36.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.541+0000 7f4d84fff700 1 -- start start 2023-07-14T18:14:36.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.541+0000 7f4d84fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d800ffc30 0x7f4d80197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:36.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.541+0000 7f4d84fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d80100e20 0x7f4d80198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:36.587 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.541+0000 7f4d84fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d801017b0 0x7f4d8019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:36.587 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.541+0000 7f4d84fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8019f680 con 0x7f4d801017b0 2023-07-14T18:14:36.587 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.541+0000 7f4d84fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8019f7c0 con 0x7f4d800ffc30 2023-07-14T18:14:36.587 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.541+0000 7f4d84fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8019f950 con 0x7f4d80100e20 2023-07-14T18:14:36.587 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.541+0000 7f4d7ed9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d801017b0 0x7f4d8019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:36.588 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.541+0000 7f4d7ed9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d801017b0 0x7f4d8019f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45542/0 (socket says 172.21.15.133:45542) 2023-07-14T18:14:36.588 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.541+0000 7f4d7e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d800ffc30 0x7f4d80197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:36.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.541+0000 7f4d7ed9d700 1 -- 172.21.15.133:0/4068357232 learned_addr learned my addr 172.21.15.133:0/4068357232 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:36.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.542+0000 7f4d7dd9b700 1 --2- 172.21.15.133:0/4068357232 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d80100e20 0x7f4d80198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:36.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.542+0000 7f4d7ed9d700 1 -- 172.21.15.133:0/4068357232 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d80100e20 msgr2=0x7f4d80198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:36.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.542+0000 7f4d7ed9d700 1 --2- 172.21.15.133:0/4068357232 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d80100e20 0x7f4d80198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:36.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.542+0000 7f4d7ed9d700 1 -- 172.21.15.133:0/4068357232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d800ffc30 msgr2=0x7f4d80197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:36.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.542+0000 7f4d7ed9d700 1 --2- 172.21.15.133:0/4068357232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d800ffc30 0x7f4d80197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:36.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.542+0000 7f4d7ed9d700 1 -- 172.21.15.133:0/4068357232 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d700095f0 con 0x7f4d801017b0 2023-07-14T18:14:36.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.542+0000 7f4d7e59c700 1 --2- 172.21.15.133:0/4068357232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d800ffc30 0x7f4d80197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:14:36.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.542+0000 7f4d7ed9d700 1 --2- 172.21.15.133:0/4068357232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d801017b0 0x7f4d8019f0b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800ef30 tx=0x7f4d7800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:36.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.542+0000 7f4d6f7fe700 1 -- 172.21.15.133:0/4068357232 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d78018070 con 0x7f4d801017b0 2023-07-14T18:14:36.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.542+0000 7f4d84fff700 1 -- 172.21.15.133:0/4068357232 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d8019fbd0 con 0x7f4d801017b0 2023-07-14T18:14:36.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.542+0000 7f4d6f7fe700 1 -- 172.21.15.133:0/4068357232 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d7800f040 con 0x7f4d801017b0 2023-07-14T18:14:36.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.542+0000 7f4d6f7fe700 1 -- 172.21.15.133:0/4068357232 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7800cd00 con 0x7f4d801017b0 2023-07-14T18:14:36.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.542+0000 7f4d84fff700 1 -- 172.21.15.133:0/4068357232 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d801a0090 con 0x7f4d801017b0 2023-07-14T18:14:36.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.543+0000 7f4d84fff700 1 -- 172.21.15.133:0/4068357232 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d8010b400 con 0x7f4d801017b0 2023-07-14T18:14:36.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.545+0000 7f4d6f7fe700 1 -- 172.21.15.133:0/4068357232 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4d78003a00 con 0x7f4d801017b0 2023-07-14T18:14:36.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.547+0000 7f4d6f7fe700 1 --2- 172.21.15.133:0/4068357232 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4d680a11a0 0x7f4d680a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:36.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.547+0000 7f4d6f7fe700 1 -- 172.21.15.133:0/4068357232 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4d780cc610 con 0x7f4d801017b0 2023-07-14T18:14:36.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.547+0000 7f4d7e59c700 1 --2- 172.21.15.133:0/4068357232 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4d680a11a0 0x7f4d680a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:36.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.547+0000 7f4d7e59c700 1 --2- 172.21.15.133:0/4068357232 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4d680a11a0 0x7f4d680a3650 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4d70006010 tx=0x7f4d70005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:36.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.550+0000 7f4d6f7fe700 1 -- 172.21.15.133:0/4068357232 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f4d78095ba0 con 0x7f4d801017b0 2023-07-14T18:14:36.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.748+0000 7f4d84fff700 1 -- 172.21.15.133:0/4068357232 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d801a0640 con 0x7f4d801017b0 2023-07-14T18:14:36.793 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.750+0000 7f4d6f7fe700 1 -- 172.21.15.133:0/4068357232 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24003 (secure 0 0 0) 0x7f4d780991c0 con 0x7f4d801017b0 2023-07-14T18:14:36.793 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:14:36.794 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":88,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","created":"2023-07-14T18:06:49.561582+0000","modified":"2023-07-14T18:14:22.524878+0000","last_up_change":"2023-07-14T18:13:29.836099+0000","last_in_change":"2023-07-14T18:13:11.606710+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T18:10:03.945039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T18:14:15.451340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T18:14:16.892215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T18:14:20.493175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"85a29de7-d712-422a-b647-d9e90091aaf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6803","nonce":3076510801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6805","nonce":3076510801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6809","nonce":3076510801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6807","nonce":3076510801}]},"public_addr":"172.21.15.133:6803/3076510801","cluster_addr":"172.21.15.133:6805/3076510801","heartbeat_back_addr":"172.21.15.133:6809/3076510801","heartbeat_front_addr":"172.21.15.133:6807/3076510801","state":["exists","up"]},{"osd":1,"uuid":"80ad4e78-7917-495e-b119-99f7c7c36243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6801","nonce":1841901852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6803","nonce":1841901852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6807","nonce":1841901852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6805","nonce":1841901852}]},"public_addr":"172.21.15.161:6801/1841901852","cluster_addr":"172.21.15.161:6803/1841901852","heartbeat_back_addr":"172.21.15.161:6807/1841901852","heartbeat_front_addr":"172.21.15.161:6805/1841901852","state":["exists","up"]},{"osd":2,"uuid":"fc4bfd37-1fdd-47c6-a48e-6d910daf9e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6801","nonce":346103099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6803","nonce":346103099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6807","nonce":346103099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6805","nonce":346103099}]},"public_addr":"172.21.15.171:6801/346103099","cluster_addr":"172.21.15.171:6803/346103099","heartbeat_back_addr":"172.21.15.171:6807/346103099","heartbeat_front_addr":"172.21.15.171:6805/346103099","state":["exists","up"]},{"osd":3,"uuid":"4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6811","nonce":2152994143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6813","nonce":2152994143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6817","nonce":2152994143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6815","nonce":2152994143}]},"public_addr":"172.21.15.133:6811/2152994143","cluster_addr":"172.21.15.133:6813/2152994143","heartbeat_back_addr":"172.21.15.133:6817/2152994143","heartbeat_front_addr":"172.21.15.133:6815/2152994143","state":["exists","up"]},{"osd":4,"uuid":"4601a1c2-254b-4c4d-9b15-8fc952b3d2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6809","nonce":47213291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6811","nonce":47213291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6815","nonce":47213291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6813","nonce":47213291}]},"public_addr":"172.21.15.161:6809/47213291","cluster_addr":"172.21.15.161:6811/47213291","heartbeat_back_addr":"172.21.15.161:6815/47213291","heartbeat_front_addr":"172.21.15.161:6813/47213291","state":["exists","up"]},{"osd":5,"uuid":"6cca180a-9140-4e49-835b-86842dae0574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6809","nonce":777556751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6811","nonce":777556751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6815","nonce":777556751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6813","nonce":777556751}]},"public_addr":"172.21.15.171:6809/777556751","cluster_addr":"172.21.15.171:6811/777556751","heartbeat_back_addr":"172.21.15.171:6815/777556751","heartbeat_front_addr":"172.21.15.171:6813/777556751","state":["exists","up"]},{"osd":6,"uuid":"ea0299ff-7740-4494-a3a9-1503b3fc0ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6819","nonce":2884152358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6821","nonce":2884152358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6825","nonce":2884152358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6823","nonce":2884152358}]},"public_addr":"172.21.15.133:6819/2884152358","cluster_addr":"172.21.15.133:6821/2884152358","heartbeat_back_addr":"172.21.15.133:6825/2884152358","heartbeat_front_addr":"172.21.15.133:6823/2884152358","state":["exists","up"]},{"osd":7,"uuid":"b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6817","nonce":4183705749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6819","nonce":4183705749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6823","nonce":4183705749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6821","nonce":4183705749}]},"public_addr":"172.21.15.161:6817/4183705749","cluster_addr":"172.21.15.161:6819/4183705749","heartbeat_back_addr":"172.21.15.161:6823/4183705749","heartbeat_front_addr":"172.21.15.161:6821/4183705749","state":["exists","up"]},{"osd":8,"uuid":"87dffc07-bf48-48c8-b418-5c4b509686d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6817","nonce":2771494483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6819","nonce":2771494483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6823","nonce":2771494483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6821","nonce":2771494483}]},"public_addr":"172.21.15.171:6817/2771494483","cluster_addr":"172.21.15.171:6819/2771494483","heartbeat_back_addr":"172.21.15.171:6823/2771494483","heartbeat_front_addr":"172.21.15.171:6821/2771494483","state":["exists","up"]},{"osd":9,"uuid":"e0832870-79ce-4bce-8045-58b7a7f6dc69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6827","nonce":2066164149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6829","nonce":2066164149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6833","nonce":2066164149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6831","nonce":2066164149}]},"public_addr":"172.21.15.133:6827/2066164149","cluster_addr":"172.21.15.133:6829/2066164149","heartbeat_back_addr":"172.21.15.133:6833/2066164149","heartbeat_front_addr":"172.21.15.133:6831/2066164149","state":["exists","up"]},{"osd":10,"uuid":"081f0484-d134-4b2e-9c48-ef8e6a646422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6825","nonce":2162333626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6827","nonce":2162333626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6831","nonce":2162333626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6829","nonce":2162333626}]},"public_addr":"172.21.15.161:6825/2162333626","cluster_addr":"172.21.15.161:6827/2162333626","heartbeat_back_addr":"172.21.15.161:6831/2162333626","heartbeat_front_addr":"172.21.15.161:6829/2162333626","state":["exists","up"]},{"osd":11,"uuid":"4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6825","nonce":3403894313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6827","nonce":3403894313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6831","nonce":3403894313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6829","nonce":3403894313}]},"public_addr":"172.21.15.171:6825/3403894313","cluster_addr":"172.21.15.171:6827/3403894313","heartbeat_back_addr":"172.21.15.171:6831/3403894313","heartbeat_front_addr":"172.21.15.171:6829/3403894313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:17.816229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:39.638543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:00.543317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:26.988921+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:48.644694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:10.049860+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:35.986508+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:57.753846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:19.018245+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:44.460450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:06.279960+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:27.955006+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/1832569241":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6801/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:0/4219815304":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:6801/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/432253615":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1840700480":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1807775439":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1089030734":"2023-07-15T18:07:47.346704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T18:14:36.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.752+0000 7f4d84fff700 1 -- 172.21.15.133:0/4068357232 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4d680a11a0 msgr2=0x7f4d680a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:36.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.752+0000 7f4d84fff700 1 --2- 172.21.15.133:0/4068357232 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4d680a11a0 0x7f4d680a3650 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4d70006010 tx=0x7f4d70005960 comp rx=0 tx=0).stop 2023-07-14T18:14:36.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.752+0000 7f4d84fff700 1 -- 172.21.15.133:0/4068357232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d801017b0 msgr2=0x7f4d8019f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:36.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.752+0000 7f4d84fff700 1 --2- 172.21.15.133:0/4068357232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d801017b0 0x7f4d8019f0b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800ef30 tx=0x7f4d7800c3a0 comp rx=0 tx=0).stop 2023-07-14T18:14:36.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.752+0000 7f4d84fff700 1 -- 172.21.15.133:0/4068357232 shutdown_connections 2023-07-14T18:14:36.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.752+0000 7f4d84fff700 1 --2- 172.21.15.133:0/4068357232 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4d680a11a0 0x7f4d680a3650 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:36.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.752+0000 7f4d84fff700 1 --2- 172.21.15.133:0/4068357232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d800ffc30 0x7f4d80197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:36.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.752+0000 7f4d84fff700 1 --2- 172.21.15.133:0/4068357232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d801017b0 0x7f4d8019f0b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:36.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.752+0000 7f4d84fff700 1 --2- 172.21.15.133:0/4068357232 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d80100e20 0x7f4d80198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:36.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.752+0000 7f4d84fff700 1 -- 172.21.15.133:0/4068357232 >> 172.21.15.133:0/4068357232 conn(0x7f4d800fb210 msgr2=0x7f4d800fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:36.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.752+0000 7f4d84fff700 1 -- 172.21.15.133:0/4068357232 shutdown_connections 2023-07-14T18:14:36.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.752+0000 7f4d84fff700 1 -- 172.21.15.133:0/4068357232 wait complete. 2023-07-14T18:14:36.813 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T18:10:03.945039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T18:14:15.451340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T18:14:16.892215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T18:14:20.493175+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T18:14:36.814 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T18:14:37.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.971+0000 7fb7971ff700 1 -- 172.21.15.133:0/1135741807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7900730a0 msgr2=0x7fb790073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:37.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.971+0000 7fb7971ff700 1 --2- 172.21.15.133:0/1135741807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7900730a0 0x7fb790073510 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb77c009990 tx=0x7fb77c009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:37.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.972+0000 7fb7971ff700 1 -- 172.21.15.133:0/1135741807 shutdown_connections 2023-07-14T18:14:37.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.972+0000 7fb7971ff700 1 --2- 172.21.15.133:0/1135741807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb790107ae0 0x7fb790109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:37.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.972+0000 7fb7971ff700 1 --2- 172.21.15.133:0/1135741807 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb790074680 0x7fb790072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:37.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.972+0000 7fb7971ff700 1 --2- 172.21.15.133:0/1135741807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7900730a0 0x7fb790073510 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:37.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.972+0000 7fb7971ff700 1 -- 172.21.15.133:0/1135741807 >> 172.21.15.133:0/1135741807 conn(0x7fb7900fb260 msgr2=0x7fb7900fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:37.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.972+0000 7fb7971ff700 1 -- 172.21.15.133:0/1135741807 shutdown_connections 2023-07-14T18:14:37.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.972+0000 7fb7971ff700 1 -- 172.21.15.133:0/1135741807 wait complete. 2023-07-14T18:14:37.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.973+0000 7fb7971ff700 1 Processor -- start 2023-07-14T18:14:37.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.973+0000 7fb7971ff700 1 -- start start 2023-07-14T18:14:37.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.973+0000 7fb7971ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7900730a0 0x7fb790197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:37.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.973+0000 7fb7971ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb790074680 0x7fb7901980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:37.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.973+0000 7fb7971ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb790107ae0 0x7fb79019f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:37.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.973+0000 7fb7971ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79019f720 con 0x7fb790107ae0 2023-07-14T18:14:37.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.973+0000 7fb7971ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79019f860 con 0x7fb7900730a0 2023-07-14T18:14:37.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.973+0000 7fb7971ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79019f9f0 con 0x7fb790074680 2023-07-14T18:14:37.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.973+0000 7fb79579c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb790107ae0 0x7fb79019f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:37.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.973+0000 7fb79579c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb790107ae0 0x7fb79019f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46462/0 (socket says 172.21.15.133:46462) 2023-07-14T18:14:37.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.973+0000 7fb79579c700 1 -- 172.21.15.133:0/3167478920 learned_addr learned my addr 172.21.15.133:0/3167478920 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:37.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.973+0000 7fb78ffff700 1 --2- 172.21.15.133:0/3167478920 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb790074680 0x7fb7901980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:37.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.974+0000 7fb794f9b700 1 --2- 172.21.15.133:0/3167478920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7900730a0 0x7fb790197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:37.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.974+0000 7fb79579c700 1 -- 172.21.15.133:0/3167478920 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb790074680 msgr2=0x7fb7901980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:37.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.974+0000 7fb79579c700 1 --2- 172.21.15.133:0/3167478920 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb790074680 0x7fb7901980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:37.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.974+0000 7fb79579c700 1 -- 172.21.15.133:0/3167478920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7900730a0 msgr2=0x7fb790197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:37.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.974+0000 7fb79579c700 1 --2- 172.21.15.133:0/3167478920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7900730a0 0x7fb790197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:37.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.974+0000 7fb79579c700 1 -- 172.21.15.133:0/3167478920 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb77c0095f0 con 0x7fb790107ae0 2023-07-14T18:14:37.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.974+0000 7fb79579c700 1 --2- 172.21.15.133:0/3167478920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb790107ae0 0x7fb79019f150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb78800ef30 tx=0x7fb78800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:37.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.974+0000 7fb78dffb700 1 -- 172.21.15.133:0/3167478920 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb788018070 con 0x7fb790107ae0 2023-07-14T18:14:37.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.974+0000 7fb7971ff700 1 -- 172.21.15.133:0/3167478920 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb79019fc70 con 0x7fb790107ae0 2023-07-14T18:14:37.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.974+0000 7fb78dffb700 1 -- 172.21.15.133:0/3167478920 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb78800f040 con 0x7fb790107ae0 2023-07-14T18:14:37.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.974+0000 7fb78dffb700 1 -- 172.21.15.133:0/3167478920 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78800cd00 con 0x7fb790107ae0 2023-07-14T18:14:37.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.974+0000 7fb7971ff700 1 -- 172.21.15.133:0/3167478920 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7901a0130 con 0x7fb790107ae0 2023-07-14T18:14:37.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.975+0000 7fb7971ff700 1 -- 172.21.15.133:0/3167478920 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb790106190 con 0x7fb790107ae0 2023-07-14T18:14:37.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.977+0000 7fb78dffb700 1 -- 172.21.15.133:0/3167478920 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fb788003a00 con 0x7fb790107ae0 2023-07-14T18:14:37.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.979+0000 7fb78dffb700 1 --2- 172.21.15.133:0/3167478920 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb7800a1150 0x7fb7800a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:37.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.979+0000 7fb78dffb700 1 -- 172.21.15.133:0/3167478920 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb7880cc6b0 con 0x7fb790107ae0 2023-07-14T18:14:37.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.979+0000 7fb794f9b700 1 --2- 172.21.15.133:0/3167478920 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb7800a1150 0x7fb7800a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:37.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.979+0000 7fb794f9b700 1 --2- 172.21.15.133:0/3167478920 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb7800a1150 0x7fb7800a3600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb790074a50 tx=0x7fb784009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:37.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:36.982+0000 7fb78dffb700 1 -- 172.21.15.133:0/3167478920 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fb788095c40 con 0x7fb790107ae0 2023-07-14T18:14:37.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:36 smithi133 ceph-mon[126384]: pgmap v282: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-07-14T18:14:37.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:36 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3054315801' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T18:14:37.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:36 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/4068357232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:14:37.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.179+0000 7fb7971ff700 1 -- 172.21.15.133:0/3167478920 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb79002cd70 con 0x7fb790107ae0 2023-07-14T18:14:37.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.180+0000 7fb78dffb700 1 -- 172.21.15.133:0/3167478920 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb788099260 con 0x7fb790107ae0 2023-07-14T18:14:37.224 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-07-14T18:14:37.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.182+0000 7fb7971ff700 1 -- 172.21.15.133:0/3167478920 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb7800a1150 msgr2=0x7fb7800a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:37.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.182+0000 7fb7971ff700 1 --2- 172.21.15.133:0/3167478920 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb7800a1150 0x7fb7800a3600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb790074a50 tx=0x7fb784009260 comp rx=0 tx=0).stop 2023-07-14T18:14:37.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.182+0000 7fb7971ff700 1 -- 172.21.15.133:0/3167478920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb790107ae0 msgr2=0x7fb79019f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:37.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.182+0000 7fb7971ff700 1 --2- 172.21.15.133:0/3167478920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb790107ae0 0x7fb79019f150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb78800ef30 tx=0x7fb78800c3a0 comp rx=0 tx=0).stop 2023-07-14T18:14:37.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.182+0000 7fb7971ff700 1 -- 172.21.15.133:0/3167478920 shutdown_connections 2023-07-14T18:14:37.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.182+0000 7fb7971ff700 1 --2- 172.21.15.133:0/3167478920 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb7800a1150 0x7fb7800a3600 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:37.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.182+0000 7fb7971ff700 1 --2- 172.21.15.133:0/3167478920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7900730a0 0x7fb790197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:37.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.182+0000 7fb7971ff700 1 --2- 172.21.15.133:0/3167478920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb790107ae0 0x7fb79019f150 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:37.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.182+0000 7fb7971ff700 1 --2- 172.21.15.133:0/3167478920 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb790074680 0x7fb7901980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:37.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.182+0000 7fb7971ff700 1 -- 172.21.15.133:0/3167478920 >> 172.21.15.133:0/3167478920 conn(0x7fb7900fb260 msgr2=0x7fb7900fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:37.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.182+0000 7fb7971ff700 1 -- 172.21.15.133:0/3167478920 shutdown_connections 2023-07-14T18:14:37.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.182+0000 7fb7971ff700 1 -- 172.21.15.133:0/3167478920 wait complete. 2023-07-14T18:14:37.244 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T18:14:37.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:36 smithi161 ceph-mon[128993]: pgmap v282: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-07-14T18:14:37.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:36 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3054315801' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T18:14:37.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:36 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/4068357232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:14:37.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:36 smithi171 ceph-mon[129241]: pgmap v282: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-07-14T18:14:37.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:36 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3054315801' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T18:14:37.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:36 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/4068357232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:14:37.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.400+0000 7f02bb7ff700 1 -- 172.21.15.133:0/2714896644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02b4100e00 msgr2=0x7f02b4101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:37.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.400+0000 7f02bb7ff700 1 --2- 172.21.15.133:0/2714896644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02b4100e00 0x7f02b4101250 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f02a40098f0 tx=0x7f02a4009c00 comp rx=0 tx=0).stop 2023-07-14T18:14:37.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.400+0000 7f02bb7ff700 1 -- 172.21.15.133:0/2714896644 shutdown_connections 2023-07-14T18:14:37.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.400+0000 7f02bb7ff700 1 --2- 172.21.15.133:0/2714896644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02b4101790 0x7f02b4108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:37.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.400+0000 7f02bb7ff700 1 --2- 172.21.15.133:0/2714896644 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02b40ffc10 0x7f02b4100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:37.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.400+0000 7f02bb7ff700 1 --2- 172.21.15.133:0/2714896644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02b4100e00 0x7f02b4101250 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:37.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.400+0000 7f02bb7ff700 1 -- 172.21.15.133:0/2714896644 >> 172.21.15.133:0/2714896644 conn(0x7f02b40fb210 msgr2=0x7f02b40fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:37.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.400+0000 7f02bb7ff700 1 -- 172.21.15.133:0/2714896644 shutdown_connections 2023-07-14T18:14:37.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.400+0000 7f02bb7ff700 1 -- 172.21.15.133:0/2714896644 wait complete. 2023-07-14T18:14:37.446 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.401+0000 7f02bb7ff700 1 Processor -- start 2023-07-14T18:14:37.447 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.401+0000 7f02bb7ff700 1 -- start start 2023-07-14T18:14:37.447 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.402+0000 7f02bb7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02b40ffc10 0x7f02b41959d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:37.447 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.402+0000 7f02bb7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02b4100e00 0x7f02b4195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:37.447 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.402+0000 7f02bb7ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02b4101790 0x7f02b419cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:37.447 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.402+0000 7f02bb7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b419d550 con 0x7f02b40ffc10 2023-07-14T18:14:37.447 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.402+0000 7f02bb7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b419d690 con 0x7f02b4100e00 2023-07-14T18:14:37.447 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.402+0000 7f02bb7ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b419d820 con 0x7f02b4101790 2023-07-14T18:14:37.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.402+0000 7f02b959b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02b40ffc10 0x7f02b41959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:37.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.402+0000 7f02b959b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02b40ffc10 0x7f02b41959d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46484/0 (socket says 172.21.15.133:46484) 2023-07-14T18:14:37.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.402+0000 7f02b959b700 1 -- 172.21.15.133:0/223716008 learned_addr learned my addr 172.21.15.133:0/223716008 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:37.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.402+0000 7f02b8d9a700 1 --2- 172.21.15.133:0/223716008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02b4100e00 0x7f02b4195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:37.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.402+0000 7f02b9d9c700 1 --2- 172.21.15.133:0/223716008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02b4101790 0x7f02b419cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:37.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.402+0000 7f02b959b700 1 -- 172.21.15.133:0/223716008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02b4101790 msgr2=0x7f02b419cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:37.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.402+0000 7f02b959b700 1 --2- 172.21.15.133:0/223716008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02b4101790 0x7f02b419cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:37.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.402+0000 7f02b959b700 1 -- 172.21.15.133:0/223716008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02b4100e00 msgr2=0x7f02b4195f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:37.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.402+0000 7f02b959b700 1 --2- 172.21.15.133:0/223716008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02b4100e00 0x7f02b4195f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:37.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.402+0000 7f02b959b700 1 -- 172.21.15.133:0/223716008 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ac009d00 con 0x7f02b40ffc10 2023-07-14T18:14:37.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.403+0000 7f02b959b700 1 --2- 172.21.15.133:0/223716008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02b40ffc10 0x7f02b41959d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f02b000e970 tx=0x7f02b000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:37.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.403+0000 7f02aa7fc700 1 -- 172.21.15.133:0/223716008 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b0009790 con 0x7f02b40ffc10 2023-07-14T18:14:37.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.403+0000 7f02bb7ff700 1 -- 172.21.15.133:0/223716008 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a4009520 con 0x7f02b40ffc10 2023-07-14T18:14:37.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.403+0000 7f02aa7fc700 1 -- 172.21.15.133:0/223716008 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02b000c7e0 con 0x7f02b40ffc10 2023-07-14T18:14:37.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.403+0000 7f02aa7fc700 1 -- 172.21.15.133:0/223716008 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b0003c70 con 0x7f02b40ffc10 2023-07-14T18:14:37.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.403+0000 7f02bb7ff700 1 -- 172.21.15.133:0/223716008 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b419de80 con 0x7f02b40ffc10 2023-07-14T18:14:37.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.404+0000 7f02bb7ff700 1 -- 172.21.15.133:0/223716008 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b410b3e0 con 0x7f02b40ffc10 2023-07-14T18:14:37.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.406+0000 7f02aa7fc700 1 -- 172.21.15.133:0/223716008 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f02b000cc50 con 0x7f02b40ffc10 2023-07-14T18:14:37.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.407+0000 7f02aa7fc700 1 --2- 172.21.15.133:0/223716008 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f02a00a1150 0x7f02a00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:37.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.408+0000 7f02aa7fc700 1 -- 172.21.15.133:0/223716008 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f02b0014070 con 0x7f02b40ffc10 2023-07-14T18:14:37.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.408+0000 7f02b8d9a700 1 --2- 172.21.15.133:0/223716008 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f02a00a1150 0x7f02a00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:37.452 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.408+0000 7f02b8d9a700 1 --2- 172.21.15.133:0/223716008 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f02a00a1150 0x7f02a00a3600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f02a400f9e0 tx=0x7f02a4019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:37.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.411+0000 7f02aa7fc700 1 -- 172.21.15.133:0/223716008 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f02b00961f0 con 0x7f02b40ffc10 2023-07-14T18:14:37.654 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.611+0000 7f02bb7ff700 1 -- 172.21.15.133:0/223716008 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f02b4106b90 con 0x7f02b40ffc10 2023-07-14T18:14:37.655 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.611+0000 7f02aa7fc700 1 -- 172.21.15.133:0/223716008 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f02b0003dd0 con 0x7f02b40ffc10 2023-07-14T18:14:37.656 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-07-14T18:14:37.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.614+0000 7f02bb7ff700 1 -- 172.21.15.133:0/223716008 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f02a00a1150 msgr2=0x7f02a00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:37.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.614+0000 7f02bb7ff700 1 --2- 172.21.15.133:0/223716008 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f02a00a1150 0x7f02a00a3600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f02a400f9e0 tx=0x7f02a4019040 comp rx=0 tx=0).stop 2023-07-14T18:14:37.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.614+0000 7f02bb7ff700 1 -- 172.21.15.133:0/223716008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02b40ffc10 msgr2=0x7f02b41959d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:37.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.614+0000 7f02bb7ff700 1 --2- 172.21.15.133:0/223716008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02b40ffc10 0x7f02b41959d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f02b000e970 tx=0x7f02b000ec80 comp rx=0 tx=0).stop 2023-07-14T18:14:37.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.614+0000 7f02bb7ff700 1 -- 172.21.15.133:0/223716008 shutdown_connections 2023-07-14T18:14:37.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.614+0000 7f02bb7ff700 1 --2- 172.21.15.133:0/223716008 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f02a00a1150 0x7f02a00a3600 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:37.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.614+0000 7f02bb7ff700 1 --2- 172.21.15.133:0/223716008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02b4101790 0x7f02b419cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:37.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.614+0000 7f02bb7ff700 1 --2- 172.21.15.133:0/223716008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02b40ffc10 0x7f02b41959d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:37.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.614+0000 7f02bb7ff700 1 --2- 172.21.15.133:0/223716008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02b4100e00 0x7f02b4195f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:37.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.614+0000 7f02bb7ff700 1 -- 172.21.15.133:0/223716008 >> 172.21.15.133:0/223716008 conn(0x7f02b40fb210 msgr2=0x7f02b40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:37.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.614+0000 7f02bb7ff700 1 -- 172.21.15.133:0/223716008 shutdown_connections 2023-07-14T18:14:37.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.614+0000 7f02bb7ff700 1 -- 172.21.15.133:0/223716008 wait complete. 2023-07-14T18:14:37.675 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T18:14:37.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.833+0000 7f58699ff700 1 -- 172.21.15.133:0/3332182556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5864100e00 msgr2=0x7f5864101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:37.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.833+0000 7f58699ff700 1 --2- 172.21.15.133:0/3332182556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5864100e00 0x7f5864101250 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f58580098f0 tx=0x7f5858009c00 comp rx=0 tx=0).stop 2023-07-14T18:14:37.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.833+0000 7f58699ff700 1 -- 172.21.15.133:0/3332182556 shutdown_connections 2023-07-14T18:14:37.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.833+0000 7f58699ff700 1 --2- 172.21.15.133:0/3332182556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5864101790 0x7f5864108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:37.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.833+0000 7f58699ff700 1 --2- 172.21.15.133:0/3332182556 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58640ffc10 0x7f5864100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:37.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.833+0000 7f58699ff700 1 --2- 172.21.15.133:0/3332182556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5864100e00 0x7f5864101250 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:37.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.833+0000 7f58699ff700 1 -- 172.21.15.133:0/3332182556 >> 172.21.15.133:0/3332182556 conn(0x7f58640fb210 msgr2=0x7f58640fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:37.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.833+0000 7f58699ff700 1 -- 172.21.15.133:0/3332182556 shutdown_connections 2023-07-14T18:14:37.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.833+0000 7f58699ff700 1 -- 172.21.15.133:0/3332182556 wait complete. 2023-07-14T18:14:37.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.834+0000 7f58699ff700 1 Processor -- start 2023-07-14T18:14:37.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.834+0000 7f58699ff700 1 -- start start 2023-07-14T18:14:37.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.834+0000 7f58699ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58640ffc10 0x7f5864197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:37.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.835+0000 7f58699ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5864100e00 0x7f5864198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:37.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.835+0000 7f58699ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5864101790 0x7f586419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:37.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.835+0000 7f58699ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586419f760 con 0x7f5864100e00 2023-07-14T18:14:37.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.835+0000 7f58699ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586419f8d0 con 0x7f5864101790 2023-07-14T18:14:37.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.835+0000 7f58699ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586419fa90 con 0x7f58640ffc10 2023-07-14T18:14:37.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.835+0000 7f58627fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5864100e00 0x7f5864198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:37.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.835+0000 7f58627fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5864100e00 0x7f5864198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46504/0 (socket says 172.21.15.133:46504) 2023-07-14T18:14:37.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.835+0000 7f58627fc700 1 -- 172.21.15.133:0/3376345494 learned_addr learned my addr 172.21.15.133:0/3376345494 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:37.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.835+0000 7f58637fe700 1 --2- 172.21.15.133:0/3376345494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5864101790 0x7f586419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:37.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.835+0000 7f5862ffd700 1 --2- 172.21.15.133:0/3376345494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58640ffc10 0x7f5864197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:37.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.835+0000 7f58627fc700 1 -- 172.21.15.133:0/3376345494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58640ffc10 msgr2=0x7f5864197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:37.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.835+0000 7f58627fc700 1 --2- 172.21.15.133:0/3376345494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58640ffc10 0x7f5864197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:37.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.835+0000 7f58627fc700 1 -- 172.21.15.133:0/3376345494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5864101790 msgr2=0x7f586419f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:37.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.835+0000 7f58627fc700 1 --2- 172.21.15.133:0/3376345494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5864101790 0x7f586419f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:37.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.835+0000 7f58627fc700 1 -- 172.21.15.133:0/3376345494 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5858009520 con 0x7f5864100e00 2023-07-14T18:14:37.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.835+0000 7f58637fe700 1 --2- 172.21.15.133:0/3376345494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5864101790 0x7f586419f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:14:37.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.835+0000 7f58627fc700 1 --2- 172.21.15.133:0/3376345494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5864100e00 0x7f5864198160 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5858005e90 tx=0x7f585800b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:37.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.836+0000 7f58689fd700 1 -- 172.21.15.133:0/3376345494 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585801c070 con 0x7f5864100e00 2023-07-14T18:14:37.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.836+0000 7f58689fd700 1 -- 172.21.15.133:0/3376345494 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f585800f7f0 con 0x7f5864100e00 2023-07-14T18:14:37.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.836+0000 7f58699ff700 1 -- 172.21.15.133:0/3376345494 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f586419fd10 con 0x7f5864100e00 2023-07-14T18:14:37.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.836+0000 7f58689fd700 1 -- 172.21.15.133:0/3376345494 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5858004a40 con 0x7f5864100e00 2023-07-14T18:14:37.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.836+0000 7f58699ff700 1 -- 172.21.15.133:0/3376345494 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58641a0130 con 0x7f5864100e00 2023-07-14T18:14:37.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.837+0000 7f58699ff700 1 -- 172.21.15.133:0/3376345494 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58641922f0 con 0x7f5864100e00 2023-07-14T18:14:37.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.839+0000 7f58689fd700 1 -- 172.21.15.133:0/3376345494 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f5858020070 con 0x7f5864100e00 2023-07-14T18:14:37.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.840+0000 7f58689fd700 1 --2- 172.21.15.133:0/3376345494 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f584c0a1150 0x7f584c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:37.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.840+0000 7f58689fd700 1 -- 172.21.15.133:0/3376345494 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f58580cd870 con 0x7f5864100e00 2023-07-14T18:14:37.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.840+0000 7f5862ffd700 1 --2- 172.21.15.133:0/3376345494 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f584c0a1150 0x7f584c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:37.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.841+0000 7f5862ffd700 1 --2- 172.21.15.133:0/3376345494 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f584c0a1150 0x7f584c0a3600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f58641990f0 tx=0x7f5854009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:37.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:37.844+0000 7f58689fd700 1 -- 172.21.15.133:0/3376345494 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f5858096e00 con 0x7f5864100e00 2023-07-14T18:14:38.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.046+0000 7f58699ff700 1 -- 172.21.15.133:0/3376345494 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5864061250 con 0x7f5864100e00 2023-07-14T18:14:38.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.046+0000 7f58689fd700 1 -- 172.21.15.133:0/3376345494 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f585809a420 con 0x7f5864100e00 2023-07-14T18:14:38.090 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2023-07-14T18:14:38.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.049+0000 7f58699ff700 1 -- 172.21.15.133:0/3376345494 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f584c0a1150 msgr2=0x7f584c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:38.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.049+0000 7f58699ff700 1 --2- 172.21.15.133:0/3376345494 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f584c0a1150 0x7f584c0a3600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f58641990f0 tx=0x7f5854009310 comp rx=0 tx=0).stop 2023-07-14T18:14:38.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.049+0000 7f58699ff700 1 -- 172.21.15.133:0/3376345494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5864100e00 msgr2=0x7f5864198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:38.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.049+0000 7f58699ff700 1 --2- 172.21.15.133:0/3376345494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5864100e00 0x7f5864198160 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5858005e90 tx=0x7f585800b730 comp rx=0 tx=0).stop 2023-07-14T18:14:38.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.049+0000 7f58699ff700 1 -- 172.21.15.133:0/3376345494 shutdown_connections 2023-07-14T18:14:38.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.049+0000 7f58699ff700 1 --2- 172.21.15.133:0/3376345494 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f584c0a1150 0x7f584c0a3600 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:38.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.049+0000 7f58699ff700 1 --2- 172.21.15.133:0/3376345494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5864101790 0x7f586419f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:38.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.049+0000 7f58699ff700 1 --2- 172.21.15.133:0/3376345494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58640ffc10 0x7f5864197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:38.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.049+0000 7f58699ff700 1 --2- 172.21.15.133:0/3376345494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5864100e00 0x7f5864198160 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:38.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.049+0000 7f58699ff700 1 -- 172.21.15.133:0/3376345494 >> 172.21.15.133:0/3376345494 conn(0x7f58640fb210 msgr2=0x7f5864106760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:38.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.049+0000 7f58699ff700 1 -- 172.21.15.133:0/3376345494 shutdown_connections 2023-07-14T18:14:38.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.049+0000 7f58699ff700 1 -- 172.21.15.133:0/3376345494 wait complete. 2023-07-14T18:14:38.111 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T18:14:38.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.268+0000 7fe3ee3ff700 1 -- 172.21.15.133:0/3416387938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e80ffc10 msgr2=0x7fe3e8100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:38.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.268+0000 7fe3ee3ff700 1 --2- 172.21.15.133:0/3416387938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e80ffc10 0x7fe3e8100020 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8009990 tx=0x7fe3d8009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:38.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.269+0000 7fe3ee3ff700 1 -- 172.21.15.133:0/3416387938 shutdown_connections 2023-07-14T18:14:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.269+0000 7fe3ee3ff700 1 --2- 172.21.15.133:0/3416387938 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3e8101790 0x7fe3e8108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.269+0000 7fe3ee3ff700 1 --2- 172.21.15.133:0/3416387938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e80ffc10 0x7fe3e8100020 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.269+0000 7fe3ee3ff700 1 --2- 172.21.15.133:0/3416387938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e8100e00 0x7fe3e8101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.269+0000 7fe3ee3ff700 1 -- 172.21.15.133:0/3416387938 >> 172.21.15.133:0/3416387938 conn(0x7fe3e80fb210 msgr2=0x7fe3e80fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.269+0000 7fe3ee3ff700 1 -- 172.21.15.133:0/3416387938 shutdown_connections 2023-07-14T18:14:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.269+0000 7fe3ee3ff700 1 -- 172.21.15.133:0/3416387938 wait complete. 2023-07-14T18:14:38.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.270+0000 7fe3ee3ff700 1 Processor -- start 2023-07-14T18:14:38.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.270+0000 7fe3ee3ff700 1 -- start start 2023-07-14T18:14:38.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.270+0000 7fe3ee3ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3e80ffc10 0x7fe3e8197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:38.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.270+0000 7fe3ee3ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e8100e00 0x7fe3e81980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:38.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.270+0000 7fe3ee3ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e8101790 0x7fe3e819f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:38.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.270+0000 7fe3ee3ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e819f730 con 0x7fe3e8100e00 2023-07-14T18:14:38.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.270+0000 7fe3ee3ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e819f870 con 0x7fe3e8101790 2023-07-14T18:14:38.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.270+0000 7fe3ee3ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e819fa00 con 0x7fe3e80ffc10 2023-07-14T18:14:38.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.270+0000 7fe3e77fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e8100e00 0x7fe3e81980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:38.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.270+0000 7fe3e77fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e8100e00 0x7fe3e81980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46522/0 (socket says 172.21.15.133:46522) 2023-07-14T18:14:38.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.270+0000 7fe3e77fe700 1 -- 172.21.15.133:0/267371504 learned_addr learned my addr 172.21.15.133:0/267371504 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:38.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.270+0000 7fe3e7fff700 1 --2- 172.21.15.133:0/267371504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3e80ffc10 0x7fe3e8197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:38.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.270+0000 7fe3ec99c700 1 --2- 172.21.15.133:0/267371504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e8101790 0x7fe3e819f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:38.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.271+0000 7fe3e77fe700 1 -- 172.21.15.133:0/267371504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3e80ffc10 msgr2=0x7fe3e8197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:38.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.271+0000 7fe3e77fe700 1 --2- 172.21.15.133:0/267371504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3e80ffc10 0x7fe3e8197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:38.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.271+0000 7fe3e77fe700 1 -- 172.21.15.133:0/267371504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e8101790 msgr2=0x7fe3e819f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:38.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.271+0000 7fe3e77fe700 1 --2- 172.21.15.133:0/267371504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e8101790 0x7fe3e819f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:38.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.271+0000 7fe3e77fe700 1 -- 172.21.15.133:0/267371504 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d80095f0 con 0x7fe3e8100e00 2023-07-14T18:14:38.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.271+0000 7fe3e7fff700 1 --2- 172.21.15.133:0/267371504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3e80ffc10 0x7fe3e8197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:14:38.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.271+0000 7fe3e77fe700 1 --2- 172.21.15.133:0/267371504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e8100e00 0x7fe3e81980f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc00e9a0 tx=0x7fe3dc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:38.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.271+0000 7fe3e57fa700 1 -- 172.21.15.133:0/267371504 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc009790 con 0x7fe3e8100e00 2023-07-14T18:14:38.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.271+0000 7fe3e57fa700 1 -- 172.21.15.133:0/267371504 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3dc010430 con 0x7fe3e8100e00 2023-07-14T18:14:38.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.271+0000 7fe3ee3ff700 1 -- 172.21.15.133:0/267371504 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e819fce0 con 0x7fe3e8100e00 2023-07-14T18:14:38.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.271+0000 7fe3e57fa700 1 -- 172.21.15.133:0/267371504 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc018680 con 0x7fe3e8100e00 2023-07-14T18:14:38.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.271+0000 7fe3ee3ff700 1 -- 172.21.15.133:0/267371504 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e81a01e0 con 0x7fe3e8100e00 2023-07-14T18:14:38.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.272+0000 7fe3ee3ff700 1 -- 172.21.15.133:0/267371504 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e810b430 con 0x7fe3e8100e00 2023-07-14T18:14:38.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.274+0000 7fe3e57fa700 1 -- 172.21.15.133:0/267371504 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe3dc0105a0 con 0x7fe3e8100e00 2023-07-14T18:14:38.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.276+0000 7fe3e57fa700 1 --2- 172.21.15.133:0/267371504 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe3d00a11d0 0x7fe3d00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:38.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.276+0000 7fe3e57fa700 1 -- 172.21.15.133:0/267371504 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe3dc014070 con 0x7fe3e8100e00 2023-07-14T18:14:38.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.276+0000 7fe3e7fff700 1 --2- 172.21.15.133:0/267371504 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe3d00a11d0 0x7fe3d00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:38.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.276+0000 7fe3e7fff700 1 --2- 172.21.15.133:0/267371504 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe3d00a11d0 0x7fe3d00a3680 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800b2e0 tx=0x7fe3d8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:38.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.279+0000 7fe3e57fa700 1 -- 172.21.15.133:0/267371504 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fe3dc051780 con 0x7fe3e8100e00 2023-07-14T18:14:38.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:37 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3167478920' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:14:38.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:37 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/223716008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:14:38.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:37 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3167478920' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:14:38.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:37 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/223716008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:14:38.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:37 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3167478920' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:14:38.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:37 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/223716008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:14:38.523 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.480+0000 7fe3ee3ff700 1 -- 172.21.15.133:0/267371504 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe3e81a07f0 con 0x7fe3e8100e00 2023-07-14T18:14:38.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.481+0000 7fe3e57fa700 1 -- 172.21.15.133:0/267371504 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe3dc0961b0 con 0x7fe3e8100e00 2023-07-14T18:14:38.524 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-07-14T18:14:38.526 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.483+0000 7fe3ee3ff700 1 -- 172.21.15.133:0/267371504 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe3d00a11d0 msgr2=0x7fe3d00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:38.526 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.483+0000 7fe3ee3ff700 1 --2- 172.21.15.133:0/267371504 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe3d00a11d0 0x7fe3d00a3680 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800b2e0 tx=0x7fe3d8005fb0 comp rx=0 tx=0).stop 2023-07-14T18:14:38.526 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.483+0000 7fe3ee3ff700 1 -- 172.21.15.133:0/267371504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e8100e00 msgr2=0x7fe3e81980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:38.527 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.483+0000 7fe3ee3ff700 1 --2- 172.21.15.133:0/267371504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e8100e00 0x7fe3e81980f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc00e9a0 tx=0x7fe3dc00ecb0 comp rx=0 tx=0).stop 2023-07-14T18:14:38.527 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.483+0000 7fe3ee3ff700 1 -- 172.21.15.133:0/267371504 shutdown_connections 2023-07-14T18:14:38.527 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.483+0000 7fe3ee3ff700 1 --2- 172.21.15.133:0/267371504 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe3d00a11d0 0x7fe3d00a3680 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:38.527 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.483+0000 7fe3ee3ff700 1 --2- 172.21.15.133:0/267371504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e8101790 0x7fe3e819f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:38.527 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.483+0000 7fe3ee3ff700 1 --2- 172.21.15.133:0/267371504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3e80ffc10 0x7fe3e8197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:38.527 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.483+0000 7fe3ee3ff700 1 --2- 172.21.15.133:0/267371504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e8100e00 0x7fe3e81980f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:38.527 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.483+0000 7fe3ee3ff700 1 -- 172.21.15.133:0/267371504 >> 172.21.15.133:0/267371504 conn(0x7fe3e80fb210 msgr2=0x7fe3e80fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:38.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.483+0000 7fe3ee3ff700 1 -- 172.21.15.133:0/267371504 shutdown_connections 2023-07-14T18:14:38.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.483+0000 7fe3ee3ff700 1 -- 172.21.15.133:0/267371504 wait complete. 2023-07-14T18:14:38.544 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T18:14:38.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.700+0000 7ff56a5ff700 1 -- 172.21.15.133:0/4123986413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff564102f90 msgr2=0x7ff564103400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:38.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.700+0000 7ff56a5ff700 1 --2- 172.21.15.133:0/4123986413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff564102f90 0x7ff564103400 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff558009990 tx=0x7ff558009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:38.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.700+0000 7ff56a5ff700 1 -- 172.21.15.133:0/4123986413 shutdown_connections 2023-07-14T18:14:38.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.700+0000 7ff56a5ff700 1 --2- 172.21.15.133:0/4123986413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff564103940 0x7ff56410a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:38.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.700+0000 7ff56a5ff700 1 --2- 172.21.15.133:0/4123986413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff564101e40 0x7ff564102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:38.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.700+0000 7ff56a5ff700 1 --2- 172.21.15.133:0/4123986413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff564102f90 0x7ff564103400 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:38.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.700+0000 7ff56a5ff700 1 -- 172.21.15.133:0/4123986413 >> 172.21.15.133:0/4123986413 conn(0x7ff5640fd480 msgr2=0x7ff5640ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:38.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.700+0000 7ff56a5ff700 1 -- 172.21.15.133:0/4123986413 shutdown_connections 2023-07-14T18:14:38.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.700+0000 7ff56a5ff700 1 -- 172.21.15.133:0/4123986413 wait complete. 2023-07-14T18:14:38.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.701+0000 7ff56a5ff700 1 Processor -- start 2023-07-14T18:14:38.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.701+0000 7ff56a5ff700 1 -- start start 2023-07-14T18:14:38.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.702+0000 7ff56a5ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff564101e40 0x7ff564197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:38.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.702+0000 7ff56a5ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff564102f90 0x7ff564198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:38.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.702+0000 7ff56a5ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff564103940 0x7ff56419f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:38.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.702+0000 7ff56a5ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56419f940 con 0x7ff564103940 2023-07-14T18:14:38.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.702+0000 7ff56a5ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56419fa80 con 0x7ff564101e40 2023-07-14T18:14:38.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.702+0000 7ff56a5ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56419fc10 con 0x7ff564102f90 2023-07-14T18:14:38.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.702+0000 7ff568b9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff564103940 0x7ff56419f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:38.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.702+0000 7ff5637fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff564102f90 0x7ff564198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:38.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.702+0000 7ff568b9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff564103940 0x7ff56419f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46544/0 (socket says 172.21.15.133:46544) 2023-07-14T18:14:38.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.702+0000 7ff568b9c700 1 -- 172.21.15.133:0/2219568468 learned_addr learned my addr 172.21.15.133:0/2219568468 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:38.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.702+0000 7ff563fff700 1 --2- 172.21.15.133:0/2219568468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff564101e40 0x7ff564197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:38.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.702+0000 7ff568b9c700 1 -- 172.21.15.133:0/2219568468 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff564102f90 msgr2=0x7ff564198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:38.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.702+0000 7ff568b9c700 1 --2- 172.21.15.133:0/2219568468 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff564102f90 0x7ff564198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:38.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.702+0000 7ff568b9c700 1 -- 172.21.15.133:0/2219568468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff564101e40 msgr2=0x7ff564197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:38.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.702+0000 7ff568b9c700 1 --2- 172.21.15.133:0/2219568468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff564101e40 0x7ff564197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:38.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.702+0000 7ff568b9c700 1 -- 172.21.15.133:0/2219568468 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5580095f0 con 0x7ff564103940 2023-07-14T18:14:38.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.703+0000 7ff568b9c700 1 --2- 172.21.15.133:0/2219568468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff564103940 0x7ff56419f370 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00ef30 tx=0x7ff55c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:38.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.703+0000 7ff5617fa700 1 -- 172.21.15.133:0/2219568468 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c018070 con 0x7ff564103940 2023-07-14T18:14:38.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.703+0000 7ff5617fa700 1 -- 172.21.15.133:0/2219568468 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff55c00f040 con 0x7ff564103940 2023-07-14T18:14:38.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.703+0000 7ff56a5ff700 1 -- 172.21.15.133:0/2219568468 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff56419fe90 con 0x7ff564103940 2023-07-14T18:14:38.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.703+0000 7ff5617fa700 1 -- 172.21.15.133:0/2219568468 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c00cd00 con 0x7ff564103940 2023-07-14T18:14:38.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.703+0000 7ff56a5ff700 1 -- 172.21.15.133:0/2219568468 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5641a02c0 con 0x7ff564103940 2023-07-14T18:14:38.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.704+0000 7ff56a5ff700 1 -- 172.21.15.133:0/2219568468 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff564109110 con 0x7ff564103940 2023-07-14T18:14:38.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.706+0000 7ff5617fa700 1 -- 172.21.15.133:0/2219568468 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff55c003a00 con 0x7ff564103940 2023-07-14T18:14:38.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.707+0000 7ff5617fa700 1 --2- 172.21.15.133:0/2219568468 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff54c0a1150 0x7ff54c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:38.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.708+0000 7ff5617fa700 1 -- 172.21.15.133:0/2219568468 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff55c0cc640 con 0x7ff564103940 2023-07-14T18:14:38.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.708+0000 7ff563fff700 1 --2- 172.21.15.133:0/2219568468 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff54c0a1150 0x7ff54c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:38.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.709+0000 7ff563fff700 1 --2- 172.21.15.133:0/2219568468 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff54c0a1150 0x7ff54c0a3600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff554009c60 tx=0x7ff554009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:38.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.711+0000 7ff5617fa700 1 -- 172.21.15.133:0/2219568468 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7ff55c095bd0 con 0x7ff564103940 2023-07-14T18:14:38.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.926+0000 7ff56a5ff700 1 -- 172.21.15.133:0/2219568468 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff56402c8e0 con 0x7ff564103940 2023-07-14T18:14:38.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.926+0000 7ff5617fa700 1 -- 172.21.15.133:0/2219568468 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) v1 ==== 81+0+597 (secure 0 0 0) 0x7ff55c0991f0 con 0x7ff564103940 2023-07-14T18:14:38.970 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/1832569241 2023-07-15T18:07:47.346704+0000 2023-07-14T18:14:38.970 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6801/2986586981 2023-07-15T18:07:47.346704+0000 2023-07-14T18:14:38.970 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6800/2986586981 2023-07-15T18:07:47.346704+0000 2023-07-14T18:14:38.970 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/4219815304 2023-07-15T18:07:47.346704+0000 2023-07-14T18:14:38.970 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6800/58641721 2023-07-15T18:07:19.109711+0000 2023-07-14T18:14:38.970 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6801/58641721 2023-07-15T18:07:19.109711+0000 2023-07-14T18:14:38.971 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/432253615 2023-07-15T18:07:19.109711+0000 2023-07-14T18:14:38.971 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/1840700480 2023-07-15T18:07:19.109711+0000 2023-07-14T18:14:38.971 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/1807775439 2023-07-15T18:07:19.109711+0000 2023-07-14T18:14:38.971 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/1089030734 2023-07-15T18:07:47.346704+0000 2023-07-14T18:14:38.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.928+0000 7ff56a5ff700 1 -- 172.21.15.133:0/2219568468 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff54c0a1150 msgr2=0x7ff54c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:38.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.928+0000 7ff56a5ff700 1 --2- 172.21.15.133:0/2219568468 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff54c0a1150 0x7ff54c0a3600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff554009c60 tx=0x7ff554009260 comp rx=0 tx=0).stop 2023-07-14T18:14:38.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.928+0000 7ff56a5ff700 1 -- 172.21.15.133:0/2219568468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff564103940 msgr2=0x7ff56419f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:38.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.928+0000 7ff56a5ff700 1 --2- 172.21.15.133:0/2219568468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff564103940 0x7ff56419f370 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00ef30 tx=0x7ff55c00c3a0 comp rx=0 tx=0).stop 2023-07-14T18:14:38.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.929+0000 7ff56a5ff700 1 -- 172.21.15.133:0/2219568468 shutdown_connections 2023-07-14T18:14:38.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.929+0000 7ff56a5ff700 1 --2- 172.21.15.133:0/2219568468 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff54c0a1150 0x7ff54c0a3600 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:38.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.929+0000 7ff56a5ff700 1 --2- 172.21.15.133:0/2219568468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff564101e40 0x7ff564197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:38.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.929+0000 7ff56a5ff700 1 --2- 172.21.15.133:0/2219568468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff564103940 0x7ff56419f370 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:38.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.929+0000 7ff56a5ff700 1 --2- 172.21.15.133:0/2219568468 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff564102f90 0x7ff564198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:38.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.929+0000 7ff56a5ff700 1 -- 172.21.15.133:0/2219568468 >> 172.21.15.133:0/2219568468 conn(0x7ff5640fd480 msgr2=0x7ff564106250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:38.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.929+0000 7ff56a5ff700 1 -- 172.21.15.133:0/2219568468 shutdown_connections 2023-07-14T18:14:38.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:38.929+0000 7ff56a5ff700 1 -- 172.21.15.133:0/2219568468 wait complete. 2023-07-14T18:14:38.974 INFO:teuthology.orchestra.run.smithi133.stderr:listed 10 entries 2023-07-14T18:14:38.990 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi133.front.sepia.ceph.com... 2023-07-14T18:14:38.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:14:38.991 DEBUG:teuthology.orchestra.run.smithi133:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-07-14T18:14:39.056 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi133.stderr:ceph-fuse[163616]: fuse finished with error 0 and tester_r 0 2023-07-14T18:14:39.060 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T18:14:39.213 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:38 smithi133 ceph-mon[126384]: pgmap v283: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-07-14T18:14:39.213 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:38 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3376345494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:14:39.214 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:38 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/267371504' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:14:39.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:38 smithi161 ceph-mon[128993]: pgmap v283: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-07-14T18:14:39.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:38 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3376345494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:14:39.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:38 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/267371504' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:14:39.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:38 smithi171 ceph-mon[129241]: pgmap v283: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-07-14T18:14:39.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:38 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3376345494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:14:39.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:38 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/267371504' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:14:40.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:39 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2219568468' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T18:14:40.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:39 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2219568468' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T18:14:40.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:39 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2219568468' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T18:14:41.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:40 smithi161 ceph-mon[128993]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-14T18:14:41.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:40 smithi171 ceph-mon[129241]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-14T18:14:41.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:40 smithi133 ceph-mon[126384]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-14T18:14:43.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:42 smithi161 ceph-mon[128993]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-14T18:14:43.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:42 smithi171 ceph-mon[129241]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-14T18:14:43.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:42 smithi133 ceph-mon[126384]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-14T18:14:45.161 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2023-07-14T18:14:45.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:14:45.162 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-07-14T18:14:45.241 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T18:14:45.304 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:44 smithi133 ceph-mon[126384]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-07-14T18:14:45.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:44 smithi161 ceph-mon[128993]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-07-14T18:14:45.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:44 smithi171 ceph-mon[129241]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-07-14T18:14:45.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.400+0000 7efc47fff700 1 -- 172.21.15.133:0/1366103176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc400ffc30 msgr2=0x7efc40100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:45.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.400+0000 7efc47fff700 1 --2- 172.21.15.133:0/1366103176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc400ffc30 0x7efc40100040 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efc3c009990 tx=0x7efc3c009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:45.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.401+0000 7efc47fff700 1 -- 172.21.15.133:0/1366103176 shutdown_connections 2023-07-14T18:14:45.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.401+0000 7efc47fff700 1 --2- 172.21.15.133:0/1366103176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc401017b0 0x7efc40108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:45.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.401+0000 7efc47fff700 1 --2- 172.21.15.133:0/1366103176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc400ffc30 0x7efc40100040 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:45.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.401+0000 7efc47fff700 1 --2- 172.21.15.133:0/1366103176 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc40100e20 0x7efc40101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:45.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.401+0000 7efc47fff700 1 -- 172.21.15.133:0/1366103176 >> 172.21.15.133:0/1366103176 conn(0x7efc400fb210 msgr2=0x7efc400fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:45.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.401+0000 7efc47fff700 1 -- 172.21.15.133:0/1366103176 shutdown_connections 2023-07-14T18:14:45.446 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.401+0000 7efc47fff700 1 -- 172.21.15.133:0/1366103176 wait complete. 2023-07-14T18:14:45.447 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.402+0000 7efc47fff700 1 Processor -- start 2023-07-14T18:14:45.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.402+0000 7efc47fff700 1 -- start start 2023-07-14T18:14:45.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.402+0000 7efc47fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc400ffc30 0x7efc400743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:45.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.402+0000 7efc47fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc40100e20 0x7efc40072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:45.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.402+0000 7efc47fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc401017b0 0x7efc40072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:45.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.402+0000 7efc47fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc40073710 con 0x7efc40100e20 2023-07-14T18:14:45.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.402+0000 7efc47fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc40073880 con 0x7efc401017b0 2023-07-14T18:14:45.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.402+0000 7efc47fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4019f980 con 0x7efc400ffc30 2023-07-14T18:14:45.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.402+0000 7efc45d9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc400ffc30 0x7efc400743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:45.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.402+0000 7efc45d9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc400ffc30 0x7efc400743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:43878/0 (socket says 172.21.15.133:43878) 2023-07-14T18:14:45.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.403+0000 7efc45d9b700 1 -- 172.21.15.133:0/227637215 learned_addr learned my addr 172.21.15.133:0/227637215 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:45.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.403+0000 7efc4559a700 1 --2- 172.21.15.133:0/227637215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc40100e20 0x7efc40072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:45.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.403+0000 7efc4659c700 1 --2- 172.21.15.133:0/227637215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc401017b0 0x7efc40072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:45.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.403+0000 7efc45d9b700 1 -- 172.21.15.133:0/227637215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc401017b0 msgr2=0x7efc40072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:45.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.403+0000 7efc45d9b700 1 --2- 172.21.15.133:0/227637215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc401017b0 0x7efc40072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:45.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.403+0000 7efc45d9b700 1 -- 172.21.15.133:0/227637215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc40100e20 msgr2=0x7efc40072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:45.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.403+0000 7efc45d9b700 1 --2- 172.21.15.133:0/227637215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc40100e20 0x7efc40072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:45.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.403+0000 7efc45d9b700 1 -- 172.21.15.133:0/227637215 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3c0095f0 con 0x7efc400ffc30 2023-07-14T18:14:45.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.403+0000 7efc4659c700 1 --2- 172.21.15.133:0/227637215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc401017b0 0x7efc40072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:14:45.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.403+0000 7efc45d9b700 1 --2- 172.21.15.133:0/227637215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc400ffc30 0x7efc400743f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efc3c000c00 tx=0x7efc3c0057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:45.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.403+0000 7efc36ffd700 1 -- 172.21.15.133:0/227637215 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c01d070 con 0x7efc400ffc30 2023-07-14T18:14:45.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.403+0000 7efc47fff700 1 -- 172.21.15.133:0/227637215 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4019fc00 con 0x7efc400ffc30 2023-07-14T18:14:45.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.403+0000 7efc36ffd700 1 -- 172.21.15.133:0/227637215 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc3c00fc40 con 0x7efc400ffc30 2023-07-14T18:14:45.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.403+0000 7efc36ffd700 1 -- 172.21.15.133:0/227637215 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c017650 con 0x7efc400ffc30 2023-07-14T18:14:45.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.404+0000 7efc47fff700 1 -- 172.21.15.133:0/227637215 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc401a00b0 con 0x7efc400ffc30 2023-07-14T18:14:45.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.404+0000 7efc47fff700 1 -- 172.21.15.133:0/227637215 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc4010b400 con 0x7efc400ffc30 2023-07-14T18:14:45.452 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.407+0000 7efc36ffd700 1 -- 172.21.15.133:0/227637215 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7efc3c021410 con 0x7efc400ffc30 2023-07-14T18:14:45.452 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.408+0000 7efc36ffd700 1 --2- 172.21.15.133:0/227637215 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efc2c0a11a0 0x7efc2c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:45.452 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.408+0000 7efc36ffd700 1 -- 172.21.15.133:0/227637215 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7efc3c0ce750 con 0x7efc400ffc30 2023-07-14T18:14:45.452 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.408+0000 7efc4559a700 1 --2- 172.21.15.133:0/227637215 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efc2c0a11a0 0x7efc2c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:45.452 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.409+0000 7efc4559a700 1 --2- 172.21.15.133:0/227637215 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efc2c0a11a0 0x7efc2c0a3650 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efc30009c60 tx=0x7efc30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:45.453 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.410+0000 7efc36ffd700 1 -- 172.21.15.133:0/227637215 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7efc3c097d40 con 0x7efc400ffc30 2023-07-14T18:14:45.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.613+0000 7efc47fff700 1 -- 172.21.15.133:0/227637215 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc4010b5f0 con 0x7efc400ffc30 2023-07-14T18:14:45.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.615+0000 7efc36ffd700 1 -- 172.21.15.133:0/227637215 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24003 (secure 0 0 0) 0x7efc3c09b360 con 0x7efc400ffc30 2023-07-14T18:14:45.659 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:14:45.659 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":88,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","created":"2023-07-14T18:06:49.561582+0000","modified":"2023-07-14T18:14:22.524878+0000","last_up_change":"2023-07-14T18:13:29.836099+0000","last_in_change":"2023-07-14T18:13:11.606710+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T18:10:03.945039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T18:14:15.451340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T18:14:16.892215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T18:14:20.493175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"85a29de7-d712-422a-b647-d9e90091aaf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6803","nonce":3076510801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6805","nonce":3076510801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6809","nonce":3076510801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6807","nonce":3076510801}]},"public_addr":"172.21.15.133:6803/3076510801","cluster_addr":"172.21.15.133:6805/3076510801","heartbeat_back_addr":"172.21.15.133:6809/3076510801","heartbeat_front_addr":"172.21.15.133:6807/3076510801","state":["exists","up"]},{"osd":1,"uuid":"80ad4e78-7917-495e-b119-99f7c7c36243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6801","nonce":1841901852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6803","nonce":1841901852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6807","nonce":1841901852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6805","nonce":1841901852}]},"public_addr":"172.21.15.161:6801/1841901852","cluster_addr":"172.21.15.161:6803/1841901852","heartbeat_back_addr":"172.21.15.161:6807/1841901852","heartbeat_front_addr":"172.21.15.161:6805/1841901852","state":["exists","up"]},{"osd":2,"uuid":"fc4bfd37-1fdd-47c6-a48e-6d910daf9e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6801","nonce":346103099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6803","nonce":346103099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6807","nonce":346103099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6805","nonce":346103099}]},"public_addr":"172.21.15.171:6801/346103099","cluster_addr":"172.21.15.171:6803/346103099","heartbeat_back_addr":"172.21.15.171:6807/346103099","heartbeat_front_addr":"172.21.15.171:6805/346103099","state":["exists","up"]},{"osd":3,"uuid":"4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6811","nonce":2152994143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6813","nonce":2152994143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6817","nonce":2152994143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6815","nonce":2152994143}]},"public_addr":"172.21.15.133:6811/2152994143","cluster_addr":"172.21.15.133:6813/2152994143","heartbeat_back_addr":"172.21.15.133:6817/2152994143","heartbeat_front_addr":"172.21.15.133:6815/2152994143","state":["exists","up"]},{"osd":4,"uuid":"4601a1c2-254b-4c4d-9b15-8fc952b3d2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6809","nonce":47213291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6811","nonce":47213291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6815","nonce":47213291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6813","nonce":47213291}]},"public_addr":"172.21.15.161:6809/47213291","cluster_addr":"172.21.15.161:6811/47213291","heartbeat_back_addr":"172.21.15.161:6815/47213291","heartbeat_front_addr":"172.21.15.161:6813/47213291","state":["exists","up"]},{"osd":5,"uuid":"6cca180a-9140-4e49-835b-86842dae0574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6809","nonce":777556751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6811","nonce":777556751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6815","nonce":777556751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6813","nonce":777556751}]},"public_addr":"172.21.15.171:6809/777556751","cluster_addr":"172.21.15.171:6811/777556751","heartbeat_back_addr":"172.21.15.171:6815/777556751","heartbeat_front_addr":"172.21.15.171:6813/777556751","state":["exists","up"]},{"osd":6,"uuid":"ea0299ff-7740-4494-a3a9-1503b3fc0ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6819","nonce":2884152358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6821","nonce":2884152358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6825","nonce":2884152358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6823","nonce":2884152358}]},"public_addr":"172.21.15.133:6819/2884152358","cluster_addr":"172.21.15.133:6821/2884152358","heartbeat_back_addr":"172.21.15.133:6825/2884152358","heartbeat_front_addr":"172.21.15.133:6823/2884152358","state":["exists","up"]},{"osd":7,"uuid":"b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6817","nonce":4183705749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6819","nonce":4183705749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6823","nonce":4183705749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6821","nonce":4183705749}]},"public_addr":"172.21.15.161:6817/4183705749","cluster_addr":"172.21.15.161:6819/4183705749","heartbeat_back_addr":"172.21.15.161:6823/4183705749","heartbeat_front_addr":"172.21.15.161:6821/4183705749","state":["exists","up"]},{"osd":8,"uuid":"87dffc07-bf48-48c8-b418-5c4b509686d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6817","nonce":2771494483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6819","nonce":2771494483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6823","nonce":2771494483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6821","nonce":2771494483}]},"public_addr":"172.21.15.171:6817/2771494483","cluster_addr":"172.21.15.171:6819/2771494483","heartbeat_back_addr":"172.21.15.171:6823/2771494483","heartbeat_front_addr":"172.21.15.171:6821/2771494483","state":["exists","up"]},{"osd":9,"uuid":"e0832870-79ce-4bce-8045-58b7a7f6dc69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6827","nonce":2066164149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6829","nonce":2066164149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6833","nonce":2066164149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6831","nonce":2066164149}]},"public_addr":"172.21.15.133:6827/2066164149","cluster_addr":"172.21.15.133:6829/2066164149","heartbeat_back_addr":"172.21.15.133:6833/2066164149","heartbeat_front_addr":"172.21.15.133:6831/2066164149","state":["exists","up"]},{"osd":10,"uuid":"081f0484-d134-4b2e-9c48-ef8e6a646422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6825","nonce":2162333626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6827","nonce":2162333626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6831","nonce":2162333626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6829","nonce":2162333626}]},"public_addr":"172.21.15.161:6825/2162333626","cluster_addr":"172.21.15.161:6827/2162333626","heartbeat_back_addr":"172.21.15.161:6831/2162333626","heartbeat_front_addr":"172.21.15.161:6829/2162333626","state":["exists","up"]},{"osd":11,"uuid":"4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6825","nonce":3403894313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6827","nonce":3403894313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6831","nonce":3403894313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6829","nonce":3403894313}]},"public_addr":"172.21.15.171:6825/3403894313","cluster_addr":"172.21.15.171:6827/3403894313","heartbeat_back_addr":"172.21.15.171:6831/3403894313","heartbeat_front_addr":"172.21.15.171:6829/3403894313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:17.816229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:39.638543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:00.543317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:26.988921+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:48.644694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:10.049860+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:35.986508+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:57.753846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:19.018245+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:44.460450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:06.279960+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:27.955006+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/1832569241":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6801/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:0/4219815304":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:6801/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/432253615":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1840700480":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1807775439":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1089030734":"2023-07-15T18:07:47.346704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T18:14:45.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.617+0000 7efc47fff700 1 -- 172.21.15.133:0/227637215 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efc2c0a11a0 msgr2=0x7efc2c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:45.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.617+0000 7efc47fff700 1 --2- 172.21.15.133:0/227637215 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efc2c0a11a0 0x7efc2c0a3650 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efc30009c60 tx=0x7efc30009260 comp rx=0 tx=0).stop 2023-07-14T18:14:45.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.617+0000 7efc47fff700 1 -- 172.21.15.133:0/227637215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc400ffc30 msgr2=0x7efc400743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:45.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.617+0000 7efc47fff700 1 --2- 172.21.15.133:0/227637215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc400ffc30 0x7efc400743f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efc3c000c00 tx=0x7efc3c0057c0 comp rx=0 tx=0).stop 2023-07-14T18:14:45.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.618+0000 7efc47fff700 1 -- 172.21.15.133:0/227637215 shutdown_connections 2023-07-14T18:14:45.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.618+0000 7efc47fff700 1 --2- 172.21.15.133:0/227637215 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efc2c0a11a0 0x7efc2c0a3650 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:45.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.618+0000 7efc47fff700 1 --2- 172.21.15.133:0/227637215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc401017b0 0x7efc40072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:45.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.618+0000 7efc47fff700 1 --2- 172.21.15.133:0/227637215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc400ffc30 0x7efc400743f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:45.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.618+0000 7efc47fff700 1 --2- 172.21.15.133:0/227637215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc40100e20 0x7efc40072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:45.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.618+0000 7efc47fff700 1 -- 172.21.15.133:0/227637215 >> 172.21.15.133:0/227637215 conn(0x7efc400fb210 msgr2=0x7efc400fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:45.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.618+0000 7efc47fff700 1 -- 172.21.15.133:0/227637215 shutdown_connections 2023-07-14T18:14:45.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.618+0000 7efc47fff700 1 -- 172.21.15.133:0/227637215 wait complete. 2023-07-14T18:14:45.680 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T18:14:45.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.834+0000 7f48a3dff700 1 -- 172.21.15.133:0/2071491115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f489c1017b0 msgr2=0x7f489c108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:45.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.834+0000 7f48a3dff700 1 --2- 172.21.15.133:0/2071491115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f489c1017b0 0x7f489c108070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f489400b0a0 tx=0x7f489400b3b0 comp rx=0 tx=0).stop 2023-07-14T18:14:45.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.834+0000 7f48a3dff700 1 -- 172.21.15.133:0/2071491115 shutdown_connections 2023-07-14T18:14:45.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.834+0000 7f48a3dff700 1 --2- 172.21.15.133:0/2071491115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f489c1017b0 0x7f489c108070 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:45.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.834+0000 7f48a3dff700 1 --2- 172.21.15.133:0/2071491115 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f489c0ffc30 0x7f489c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:45.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.834+0000 7f48a3dff700 1 --2- 172.21.15.133:0/2071491115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f489c100e20 0x7f489c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:45.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.834+0000 7f48a3dff700 1 -- 172.21.15.133:0/2071491115 >> 172.21.15.133:0/2071491115 conn(0x7f489c0fb210 msgr2=0x7f489c0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:45.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.834+0000 7f48a3dff700 1 -- 172.21.15.133:0/2071491115 shutdown_connections 2023-07-14T18:14:45.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.834+0000 7f48a3dff700 1 -- 172.21.15.133:0/2071491115 wait complete. 2023-07-14T18:14:45.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.835+0000 7f48a3dff700 1 Processor -- start 2023-07-14T18:14:45.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.835+0000 7f48a3dff700 1 -- start start 2023-07-14T18:14:45.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.835+0000 7f48a3dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f489c0ffc30 0x7f489c197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:45.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.835+0000 7f48a3dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f489c100e20 0x7f489c198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:45.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.835+0000 7f48a3dff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f489c1017b0 0x7f489c19f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:45.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.835+0000 7f48a3dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489c19f770 con 0x7f489c0ffc30 2023-07-14T18:14:45.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.835+0000 7f48a3dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489c19f8e0 con 0x7f489c100e20 2023-07-14T18:14:45.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.835+0000 7f48a3dff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489c19faa0 con 0x7f489c1017b0 2023-07-14T18:14:45.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.835+0000 7f48a1b9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f489c0ffc30 0x7f489c197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:45.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.836+0000 7f48a1b9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f489c0ffc30 0x7f489c197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46582/0 (socket says 172.21.15.133:46582) 2023-07-14T18:14:45.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.836+0000 7f48a1b9b700 1 -- 172.21.15.133:0/1852232528 learned_addr learned my addr 172.21.15.133:0/1852232528 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:45.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.836+0000 7f48a139a700 1 --2- 172.21.15.133:0/1852232528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f489c100e20 0x7f489c198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:45.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.836+0000 7f48a239c700 1 --2- 172.21.15.133:0/1852232528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f489c1017b0 0x7f489c19f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:45.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.836+0000 7f48a1b9b700 1 -- 172.21.15.133:0/1852232528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f489c1017b0 msgr2=0x7f489c19f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:45.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.836+0000 7f48a1b9b700 1 --2- 172.21.15.133:0/1852232528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f489c1017b0 0x7f489c19f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:45.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.836+0000 7f48a1b9b700 1 -- 172.21.15.133:0/1852232528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f489c100e20 msgr2=0x7f489c198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:45.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.836+0000 7f48a1b9b700 1 --2- 172.21.15.133:0/1852232528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f489c100e20 0x7f489c198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:45.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.836+0000 7f48a1b9b700 1 -- 172.21.15.133:0/1852232528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4894009d00 con 0x7f489c0ffc30 2023-07-14T18:14:45.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.836+0000 7f48a1b9b700 1 --2- 172.21.15.133:0/1852232528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f489c0ffc30 0x7f489c197c30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f489800e970 tx=0x7f489800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:45.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.836+0000 7f4892ffd700 1 -- 172.21.15.133:0/1852232528 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4898009790 con 0x7f489c0ffc30 2023-07-14T18:14:45.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.836+0000 7f4892ffd700 1 -- 172.21.15.133:0/1852232528 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f489800c7e0 con 0x7f489c0ffc30 2023-07-14T18:14:45.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.836+0000 7f48a3dff700 1 -- 172.21.15.133:0/1852232528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f489c19fd80 con 0x7f489c0ffc30 2023-07-14T18:14:45.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.836+0000 7f4892ffd700 1 -- 172.21.15.133:0/1852232528 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4898003c70 con 0x7f489c0ffc30 2023-07-14T18:14:45.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.837+0000 7f48a3dff700 1 -- 172.21.15.133:0/1852232528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f489c1a0220 con 0x7f489c0ffc30 2023-07-14T18:14:45.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.838+0000 7f48a3dff700 1 -- 172.21.15.133:0/1852232528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f489c10b460 con 0x7f489c0ffc30 2023-07-14T18:14:45.885 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.839+0000 7f4892ffd700 1 -- 172.21.15.133:0/1852232528 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f489800cc50 con 0x7f489c0ffc30 2023-07-14T18:14:45.885 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.841+0000 7f4892ffd700 1 --2- 172.21.15.133:0/1852232528 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f48880a1220 0x7f48880a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:45.885 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.841+0000 7f4892ffd700 1 -- 172.21.15.133:0/1852232528 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4898014070 con 0x7f489c0ffc30 2023-07-14T18:14:45.885 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.841+0000 7f48a139a700 1 --2- 172.21.15.133:0/1852232528 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f48880a1220 0x7f48880a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:45.885 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.842+0000 7f48a139a700 1 --2- 172.21.15.133:0/1852232528 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f48880a1220 0x7f48880a36d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f488c009c60 tx=0x7f488c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:45.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:45.845+0000 7f4892ffd700 1 -- 172.21.15.133:0/1852232528 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f4898096140 con 0x7f489c0ffc30 2023-07-14T18:14:46.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.055+0000 7f48a3dff700 1 -- 172.21.15.133:0/1852232528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f489c10b650 con 0x7f489c0ffc30 2023-07-14T18:14:46.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.057+0000 7f4892ffd700 1 -- 172.21.15.133:0/1852232528 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24003 (secure 0 0 0) 0x7f4898003dd0 con 0x7f489c0ffc30 2023-07-14T18:14:46.101 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:14:46.101 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":88,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","created":"2023-07-14T18:06:49.561582+0000","modified":"2023-07-14T18:14:22.524878+0000","last_up_change":"2023-07-14T18:13:29.836099+0000","last_in_change":"2023-07-14T18:13:11.606710+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T18:10:03.945039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T18:14:15.451340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T18:14:16.892215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T18:14:20.493175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"85a29de7-d712-422a-b647-d9e90091aaf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6803","nonce":3076510801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6805","nonce":3076510801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6809","nonce":3076510801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6807","nonce":3076510801}]},"public_addr":"172.21.15.133:6803/3076510801","cluster_addr":"172.21.15.133:6805/3076510801","heartbeat_back_addr":"172.21.15.133:6809/3076510801","heartbeat_front_addr":"172.21.15.133:6807/3076510801","state":["exists","up"]},{"osd":1,"uuid":"80ad4e78-7917-495e-b119-99f7c7c36243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6801","nonce":1841901852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6803","nonce":1841901852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6807","nonce":1841901852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6805","nonce":1841901852}]},"public_addr":"172.21.15.161:6801/1841901852","cluster_addr":"172.21.15.161:6803/1841901852","heartbeat_back_addr":"172.21.15.161:6807/1841901852","heartbeat_front_addr":"172.21.15.161:6805/1841901852","state":["exists","up"]},{"osd":2,"uuid":"fc4bfd37-1fdd-47c6-a48e-6d910daf9e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6801","nonce":346103099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6803","nonce":346103099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6807","nonce":346103099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6805","nonce":346103099}]},"public_addr":"172.21.15.171:6801/346103099","cluster_addr":"172.21.15.171:6803/346103099","heartbeat_back_addr":"172.21.15.171:6807/346103099","heartbeat_front_addr":"172.21.15.171:6805/346103099","state":["exists","up"]},{"osd":3,"uuid":"4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6811","nonce":2152994143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6813","nonce":2152994143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6817","nonce":2152994143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6815","nonce":2152994143}]},"public_addr":"172.21.15.133:6811/2152994143","cluster_addr":"172.21.15.133:6813/2152994143","heartbeat_back_addr":"172.21.15.133:6817/2152994143","heartbeat_front_addr":"172.21.15.133:6815/2152994143","state":["exists","up"]},{"osd":4,"uuid":"4601a1c2-254b-4c4d-9b15-8fc952b3d2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6809","nonce":47213291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6811","nonce":47213291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6815","nonce":47213291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6813","nonce":47213291}]},"public_addr":"172.21.15.161:6809/47213291","cluster_addr":"172.21.15.161:6811/47213291","heartbeat_back_addr":"172.21.15.161:6815/47213291","heartbeat_front_addr":"172.21.15.161:6813/47213291","state":["exists","up"]},{"osd":5,"uuid":"6cca180a-9140-4e49-835b-86842dae0574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6809","nonce":777556751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6811","nonce":777556751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6815","nonce":777556751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6813","nonce":777556751}]},"public_addr":"172.21.15.171:6809/777556751","cluster_addr":"172.21.15.171:6811/777556751","heartbeat_back_addr":"172.21.15.171:6815/777556751","heartbeat_front_addr":"172.21.15.171:6813/777556751","state":["exists","up"]},{"osd":6,"uuid":"ea0299ff-7740-4494-a3a9-1503b3fc0ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6819","nonce":2884152358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6821","nonce":2884152358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6825","nonce":2884152358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6823","nonce":2884152358}]},"public_addr":"172.21.15.133:6819/2884152358","cluster_addr":"172.21.15.133:6821/2884152358","heartbeat_back_addr":"172.21.15.133:6825/2884152358","heartbeat_front_addr":"172.21.15.133:6823/2884152358","state":["exists","up"]},{"osd":7,"uuid":"b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6817","nonce":4183705749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6819","nonce":4183705749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6823","nonce":4183705749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6821","nonce":4183705749}]},"public_addr":"172.21.15.161:6817/4183705749","cluster_addr":"172.21.15.161:6819/4183705749","heartbeat_back_addr":"172.21.15.161:6823/4183705749","heartbeat_front_addr":"172.21.15.161:6821/4183705749","state":["exists","up"]},{"osd":8,"uuid":"87dffc07-bf48-48c8-b418-5c4b509686d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6817","nonce":2771494483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6819","nonce":2771494483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6823","nonce":2771494483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6821","nonce":2771494483}]},"public_addr":"172.21.15.171:6817/2771494483","cluster_addr":"172.21.15.171:6819/2771494483","heartbeat_back_addr":"172.21.15.171:6823/2771494483","heartbeat_front_addr":"172.21.15.171:6821/2771494483","state":["exists","up"]},{"osd":9,"uuid":"e0832870-79ce-4bce-8045-58b7a7f6dc69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6827","nonce":2066164149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6829","nonce":2066164149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6833","nonce":2066164149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6831","nonce":2066164149}]},"public_addr":"172.21.15.133:6827/2066164149","cluster_addr":"172.21.15.133:6829/2066164149","heartbeat_back_addr":"172.21.15.133:6833/2066164149","heartbeat_front_addr":"172.21.15.133:6831/2066164149","state":["exists","up"]},{"osd":10,"uuid":"081f0484-d134-4b2e-9c48-ef8e6a646422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6825","nonce":2162333626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6827","nonce":2162333626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6831","nonce":2162333626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6829","nonce":2162333626}]},"public_addr":"172.21.15.161:6825/2162333626","cluster_addr":"172.21.15.161:6827/2162333626","heartbeat_back_addr":"172.21.15.161:6831/2162333626","heartbeat_front_addr":"172.21.15.161:6829/2162333626","state":["exists","up"]},{"osd":11,"uuid":"4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6825","nonce":3403894313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6827","nonce":3403894313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6831","nonce":3403894313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6829","nonce":3403894313}]},"public_addr":"172.21.15.171:6825/3403894313","cluster_addr":"172.21.15.171:6827/3403894313","heartbeat_back_addr":"172.21.15.171:6831/3403894313","heartbeat_front_addr":"172.21.15.171:6829/3403894313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:17.816229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:39.638543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:00.543317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:26.988921+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:48.644694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:10.049860+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:35.986508+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:57.753846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:19.018245+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:44.460450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:06.279960+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:27.955006+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/1832569241":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6801/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:0/4219815304":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:6801/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/432253615":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1840700480":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1807775439":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1089030734":"2023-07-15T18:07:47.346704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T18:14:46.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.059+0000 7f48a3dff700 1 -- 172.21.15.133:0/1852232528 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f48880a1220 msgr2=0x7f48880a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:46.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.059+0000 7f48a3dff700 1 --2- 172.21.15.133:0/1852232528 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f48880a1220 0x7f48880a36d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f488c009c60 tx=0x7f488c009260 comp rx=0 tx=0).stop 2023-07-14T18:14:46.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.059+0000 7f48a3dff700 1 -- 172.21.15.133:0/1852232528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f489c0ffc30 msgr2=0x7f489c197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:46.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.059+0000 7f48a3dff700 1 --2- 172.21.15.133:0/1852232528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f489c0ffc30 0x7f489c197c30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f489800e970 tx=0x7f489800ec80 comp rx=0 tx=0).stop 2023-07-14T18:14:46.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.059+0000 7f48a3dff700 1 -- 172.21.15.133:0/1852232528 shutdown_connections 2023-07-14T18:14:46.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.059+0000 7f48a3dff700 1 --2- 172.21.15.133:0/1852232528 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f48880a1220 0x7f48880a36d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:46.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.059+0000 7f48a3dff700 1 --2- 172.21.15.133:0/1852232528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f489c1017b0 0x7f489c19f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:46.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.059+0000 7f48a3dff700 1 --2- 172.21.15.133:0/1852232528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f489c0ffc30 0x7f489c197c30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:46.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.059+0000 7f48a3dff700 1 --2- 172.21.15.133:0/1852232528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f489c100e20 0x7f489c198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:46.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.059+0000 7f48a3dff700 1 -- 172.21.15.133:0/1852232528 >> 172.21.15.133:0/1852232528 conn(0x7f489c0fb210 msgr2=0x7f489c106620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:46.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.059+0000 7f48a3dff700 1 -- 172.21.15.133:0/1852232528 shutdown_connections 2023-07-14T18:14:46.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.059+0000 7f48a3dff700 1 -- 172.21.15.133:0/1852232528 wait complete. 2023-07-14T18:14:46.121 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-14T18:14:46.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.278+0000 7fe5337ff700 1 -- 172.21.15.133:0/989425975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe52c0ffc20 msgr2=0x7fe52c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:46.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.278+0000 7fe5337ff700 1 --2- 172.21.15.133:0/989425975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe52c0ffc20 0x7fe52c100030 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe528009990 tx=0x7fe528009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:46.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.278+0000 7fe5337ff700 1 -- 172.21.15.133:0/989425975 shutdown_connections 2023-07-14T18:14:46.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.278+0000 7fe5337ff700 1 --2- 172.21.15.133:0/989425975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe52c1017a0 0x7fe52c108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:46.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.278+0000 7fe5337ff700 1 --2- 172.21.15.133:0/989425975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe52c0ffc20 0x7fe52c100030 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:46.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.278+0000 7fe5337ff700 1 --2- 172.21.15.133:0/989425975 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe52c100e10 0x7fe52c101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:46.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.278+0000 7fe5337ff700 1 -- 172.21.15.133:0/989425975 >> 172.21.15.133:0/989425975 conn(0x7fe52c0fb260 msgr2=0x7fe52c0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:46.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.278+0000 7fe5337ff700 1 -- 172.21.15.133:0/989425975 shutdown_connections 2023-07-14T18:14:46.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.278+0000 7fe5337ff700 1 -- 172.21.15.133:0/989425975 wait complete. 2023-07-14T18:14:46.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.279+0000 7fe5337ff700 1 Processor -- start 2023-07-14T18:14:46.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.279+0000 7fe5337ff700 1 -- start start 2023-07-14T18:14:46.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.280+0000 7fe5337ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe52c0ffc20 0x7fe52c197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:46.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.280+0000 7fe5337ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe52c100e10 0x7fe52c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:46.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.280+0000 7fe5337ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe52c1017a0 0x7fe52c19f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:46.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.280+0000 7fe5337ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52c19f750 con 0x7fe52c100e10 2023-07-14T18:14:46.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.280+0000 7fe5337ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52c19f890 con 0x7fe52c0ffc20 2023-07-14T18:14:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.280+0000 7fe5337ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52c19fa20 con 0x7fe52c1017a0 2023-07-14T18:14:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.280+0000 7fe530d9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe52c100e10 0x7fe52c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.280+0000 7fe530d9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe52c100e10 0x7fe52c198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46606/0 (socket says 172.21.15.133:46606) 2023-07-14T18:14:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.280+0000 7fe530d9a700 1 -- 172.21.15.133:0/556321830 learned_addr learned my addr 172.21.15.133:0/556321830 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.280+0000 7fe53159b700 1 --2- 172.21.15.133:0/556321830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe52c0ffc20 0x7fe52c197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.280+0000 7fe531d9c700 1 --2- 172.21.15.133:0/556321830 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe52c1017a0 0x7fe52c19f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.280+0000 7fe530d9a700 1 -- 172.21.15.133:0/556321830 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe52c1017a0 msgr2=0x7fe52c19f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:46.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.280+0000 7fe530d9a700 1 --2- 172.21.15.133:0/556321830 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe52c1017a0 0x7fe52c19f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:46.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.280+0000 7fe530d9a700 1 -- 172.21.15.133:0/556321830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe52c0ffc20 msgr2=0x7fe52c197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:46.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.280+0000 7fe530d9a700 1 --2- 172.21.15.133:0/556321830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe52c0ffc20 0x7fe52c197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:46.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.280+0000 7fe530d9a700 1 -- 172.21.15.133:0/556321830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5280095f0 con 0x7fe52c100e10 2023-07-14T18:14:46.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.280+0000 7fe530d9a700 1 --2- 172.21.15.133:0/556321830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe52c100e10 0x7fe52c198110 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00e9a0 tx=0x7fe51c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:46.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.281+0000 7fe5227fc700 1 -- 172.21.15.133:0/556321830 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c009790 con 0x7fe52c100e10 2023-07-14T18:14:46.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.281+0000 7fe5337ff700 1 -- 172.21.15.133:0/556321830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe52c19fd00 con 0x7fe52c100e10 2023-07-14T18:14:46.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.281+0000 7fe5227fc700 1 -- 172.21.15.133:0/556321830 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe51c010430 con 0x7fe52c100e10 2023-07-14T18:14:46.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.281+0000 7fe5227fc700 1 -- 172.21.15.133:0/556321830 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c018680 con 0x7fe52c100e10 2023-07-14T18:14:46.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.281+0000 7fe5337ff700 1 -- 172.21.15.133:0/556321830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe52c1a0200 con 0x7fe52c100e10 2023-07-14T18:14:46.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.282+0000 7fe5337ff700 1 -- 172.21.15.133:0/556321830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe52c192300 con 0x7fe52c100e10 2023-07-14T18:14:46.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.284+0000 7fe5227fc700 1 -- 172.21.15.133:0/556321830 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe51c0105a0 con 0x7fe52c100e10 2023-07-14T18:14:46.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.285+0000 7fe5227fc700 1 --2- 172.21.15.133:0/556321830 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe5180a1150 0x7fe5180a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:46.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.285+0000 7fe5227fc700 1 -- 172.21.15.133:0/556321830 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe51c014070 con 0x7fe52c100e10 2023-07-14T18:14:46.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.285+0000 7fe53159b700 1 --2- 172.21.15.133:0/556321830 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe5180a1150 0x7fe5180a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:46.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.286+0000 7fe53159b700 1 --2- 172.21.15.133:0/556321830 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe5180a1150 0x7fe5180a3600 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe52800b620 tx=0x7fe528005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:46.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.289+0000 7fe5227fc700 1 -- 172.21.15.133:0/556321830 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fe51c051780 con 0x7fe52c100e10 2023-07-14T18:14:46.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:45 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/227637215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:14:46.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:45 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/227637215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:14:46.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:45 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/227637215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:14:46.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.534+0000 7fe5337ff700 1 -- 172.21.15.133:0/556321830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe52c02c8e0 con 0x7fe52c100e10 2023-07-14T18:14:46.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.929+0000 7fe5227fc700 1 -- 172.21.15.133:0/556321830 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe51c096210 con 0x7fe52c100e10 2023-07-14T18:14:46.975 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.932+0000 7fe5337ff700 1 -- 172.21.15.133:0/556321830 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe5180a1150 msgr2=0x7fe5180a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:46.975 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.932+0000 7fe5337ff700 1 --2- 172.21.15.133:0/556321830 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe5180a1150 0x7fe5180a3600 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe52800b620 tx=0x7fe528005fd0 comp rx=0 tx=0).stop 2023-07-14T18:14:46.976 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.932+0000 7fe5337ff700 1 -- 172.21.15.133:0/556321830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe52c100e10 msgr2=0x7fe52c198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:46.976 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.932+0000 7fe5337ff700 1 --2- 172.21.15.133:0/556321830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe52c100e10 0x7fe52c198110 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00e9a0 tx=0x7fe51c00ecb0 comp rx=0 tx=0).stop 2023-07-14T18:14:46.976 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.932+0000 7fe5337ff700 1 -- 172.21.15.133:0/556321830 shutdown_connections 2023-07-14T18:14:46.976 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.932+0000 7fe5337ff700 1 --2- 172.21.15.133:0/556321830 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe5180a1150 0x7fe5180a3600 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:46.976 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.932+0000 7fe5337ff700 1 --2- 172.21.15.133:0/556321830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe52c0ffc20 0x7fe52c197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:46.976 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.932+0000 7fe5337ff700 1 --2- 172.21.15.133:0/556321830 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe52c1017a0 0x7fe52c19f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:46.976 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.932+0000 7fe5337ff700 1 --2- 172.21.15.133:0/556321830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe52c100e10 0x7fe52c198110 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:46.977 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.932+0000 7fe5337ff700 1 -- 172.21.15.133:0/556321830 >> 172.21.15.133:0/556321830 conn(0x7fe52c0fb260 msgr2=0x7fe52c1066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:46.977 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.932+0000 7fe5337ff700 1 -- 172.21.15.133:0/556321830 shutdown_connections 2023-07-14T18:14:46.977 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:46.932+0000 7fe5337ff700 1 -- 172.21.15.133:0/556321830 wait complete. 2023-07-14T18:14:46.994 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-07-14T18:14:46.995 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-07-14T18:14:47.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.150+0000 7f5bad5ff700 1 -- 172.21.15.133:0/1564917253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ba80730a0 msgr2=0x7f5ba8073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:47.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.150+0000 7f5bad5ff700 1 --2- 172.21.15.133:0/1564917253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ba80730a0 0x7f5ba8073510 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c009990 tx=0x7f5b9c009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:47.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.150+0000 7f5bad5ff700 1 -- 172.21.15.133:0/1564917253 shutdown_connections 2023-07-14T18:14:47.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.150+0000 7f5bad5ff700 1 --2- 172.21.15.133:0/1564917253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ba8074680 0x7f5ba8072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:47.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.150+0000 7f5bad5ff700 1 --2- 172.21.15.133:0/1564917253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ba8107ad0 0x7f5ba8109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:47.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.150+0000 7f5bad5ff700 1 --2- 172.21.15.133:0/1564917253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ba80730a0 0x7f5ba8073510 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:47.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.150+0000 7f5bad5ff700 1 -- 172.21.15.133:0/1564917253 >> 172.21.15.133:0/1564917253 conn(0x7f5ba80fb210 msgr2=0x7f5ba80fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:47.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.150+0000 7f5bad5ff700 1 -- 172.21.15.133:0/1564917253 shutdown_connections 2023-07-14T18:14:47.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.150+0000 7f5bad5ff700 1 -- 172.21.15.133:0/1564917253 wait complete. 2023-07-14T18:14:47.198 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.151+0000 7f5bad5ff700 1 Processor -- start 2023-07-14T18:14:47.198 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.151+0000 7f5bad5ff700 1 -- start start 2023-07-14T18:14:47.198 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.151+0000 7f5bad5ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ba80730a0 0x7f5ba8197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:47.198 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.151+0000 7f5bad5ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ba8074680 0x7f5ba8198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:47.198 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.151+0000 7f5bad5ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ba8107ad0 0x7f5ba819f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:47.198 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.151+0000 7f5bad5ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba819f950 con 0x7f5ba80730a0 2023-07-14T18:14:47.198 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.151+0000 7f5bad5ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba819fa90 con 0x7f5ba8074680 2023-07-14T18:14:47.199 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.151+0000 7f5bad5ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba819fc20 con 0x7f5ba8107ad0 2023-07-14T18:14:47.199 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.152+0000 7f5ba6ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ba80730a0 0x7f5ba8197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:47.199 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.152+0000 7f5ba6ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ba80730a0 0x7f5ba8197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39222/0 (socket says 172.21.15.133:39222) 2023-07-14T18:14:47.199 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.152+0000 7f5ba6ffd700 1 -- 172.21.15.133:0/431020081 learned_addr learned my addr 172.21.15.133:0/431020081 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:47.199 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.152+0000 7f5ba67fc700 1 --2- 172.21.15.133:0/431020081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ba8074680 0x7f5ba8198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:47.199 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.152+0000 7f5ba77fe700 1 --2- 172.21.15.133:0/431020081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ba8107ad0 0x7f5ba819f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:47.199 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.152+0000 7f5ba6ffd700 1 -- 172.21.15.133:0/431020081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ba8107ad0 msgr2=0x7f5ba819f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:47.200 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.152+0000 7f5ba6ffd700 1 --2- 172.21.15.133:0/431020081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ba8107ad0 0x7f5ba819f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:47.200 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.152+0000 7f5ba6ffd700 1 -- 172.21.15.133:0/431020081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ba8074680 msgr2=0x7f5ba8198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:47.200 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.152+0000 7f5ba6ffd700 1 --2- 172.21.15.133:0/431020081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ba8074680 0x7f5ba8198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:47.200 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.152+0000 7f5ba6ffd700 1 -- 172.21.15.133:0/431020081 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9c0095f0 con 0x7f5ba80730a0 2023-07-14T18:14:47.200 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.152+0000 7f5ba6ffd700 1 --2- 172.21.15.133:0/431020081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ba80730a0 0x7f5ba8197dd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5b9800d950 tx=0x7f5b9800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:47.200 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.152+0000 7f5b8ffff700 1 -- 172.21.15.133:0/431020081 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b98004220 con 0x7f5ba80730a0 2023-07-14T18:14:47.200 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.153+0000 7f5bad5ff700 1 -- 172.21.15.133:0/431020081 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba819ff00 con 0x7f5ba80730a0 2023-07-14T18:14:47.201 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.153+0000 7f5b8ffff700 1 -- 172.21.15.133:0/431020081 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b98007620 con 0x7f5ba80730a0 2023-07-14T18:14:47.201 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.153+0000 7f5b8ffff700 1 -- 172.21.15.133:0/431020081 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b98005670 con 0x7f5ba80730a0 2023-07-14T18:14:47.201 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.153+0000 7f5bad5ff700 1 -- 172.21.15.133:0/431020081 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba81a0320 con 0x7f5ba80730a0 2023-07-14T18:14:47.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.154+0000 7f5bad5ff700 1 -- 172.21.15.133:0/431020081 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba810d660 con 0x7f5ba80730a0 2023-07-14T18:14:47.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.156+0000 7f5b8ffff700 1 -- 172.21.15.133:0/431020081 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f5b98003680 con 0x7f5ba80730a0 2023-07-14T18:14:47.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.157+0000 7f5b8ffff700 1 --2- 172.21.15.133:0/431020081 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5b900a1150 0x7f5b900a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:47.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.157+0000 7f5b8ffff700 1 -- 172.21.15.133:0/431020081 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5b980ccdd0 con 0x7f5ba80730a0 2023-07-14T18:14:47.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.157+0000 7f5ba67fc700 1 --2- 172.21.15.133:0/431020081 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5b900a1150 0x7f5b900a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:47.203 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.158+0000 7f5ba67fc700 1 --2- 172.21.15.133:0/431020081 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5b900a1150 0x7f5b900a3600 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c00b2e0 tx=0x7f5b9c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:47.203 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.159+0000 7f5b8ffff700 1 -- 172.21.15.133:0/431020081 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f5b980962d0 con 0x7f5ba80730a0 2023-07-14T18:14:47.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:46 smithi161 ceph-mon[128993]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-14T18:14:47.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:46 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1852232528' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:14:47.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:46 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/556321830' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-14T18:14:47.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:46 smithi171 ceph-mon[129241]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-14T18:14:47.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:46 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1852232528' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:14:47.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:46 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/556321830' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-14T18:14:47.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:46 smithi133 ceph-mon[126384]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-14T18:14:47.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:46 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1852232528' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:14:47.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:46 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/556321830' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-14T18:14:47.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.406+0000 7f5bad5ff700 1 -- 172.21.15.133:0/431020081 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f5ba81a0930 con 0x7f5ba80730a0 2023-07-14T18:14:47.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.938+0000 7f5b8ffff700 1 -- 172.21.15.133:0/431020081 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f5b980998f0 con 0x7f5ba80730a0 2023-07-14T18:14:47.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.944+0000 7f5bad5ff700 1 -- 172.21.15.133:0/431020081 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5b900a1150 msgr2=0x7f5b900a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:47.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.944+0000 7f5bad5ff700 1 --2- 172.21.15.133:0/431020081 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5b900a1150 0x7f5b900a3600 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c00b2e0 tx=0x7f5b9c005fb0 comp rx=0 tx=0).stop 2023-07-14T18:14:47.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.944+0000 7f5bad5ff700 1 -- 172.21.15.133:0/431020081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ba80730a0 msgr2=0x7f5ba8197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:47.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.944+0000 7f5bad5ff700 1 --2- 172.21.15.133:0/431020081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ba80730a0 0x7f5ba8197dd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5b9800d950 tx=0x7f5b9800dc60 comp rx=0 tx=0).stop 2023-07-14T18:14:47.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.944+0000 7f5bad5ff700 1 -- 172.21.15.133:0/431020081 shutdown_connections 2023-07-14T18:14:47.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.945+0000 7f5bad5ff700 1 --2- 172.21.15.133:0/431020081 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5b900a1150 0x7f5b900a3600 secure :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c00b2e0 tx=0x7f5b9c005fb0 comp rx=0 tx=0).stop 2023-07-14T18:14:47.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.945+0000 7f5bad5ff700 1 --2- 172.21.15.133:0/431020081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ba8107ad0 0x7f5ba819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:47.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.945+0000 7f5bad5ff700 1 --2- 172.21.15.133:0/431020081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ba80730a0 0x7f5ba8197dd0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:47.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.945+0000 7f5bad5ff700 1 --2- 172.21.15.133:0/431020081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ba8074680 0x7f5ba8198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:47.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.945+0000 7f5bad5ff700 1 -- 172.21.15.133:0/431020081 >> 172.21.15.133:0/431020081 conn(0x7f5ba80fb210 msgr2=0x7f5ba8109e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:47.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.945+0000 7f5bad5ff700 1 -- 172.21.15.133:0/431020081 shutdown_connections 2023-07-14T18:14:47.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:47.945+0000 7f5bad5ff700 1 -- 172.21.15.133:0/431020081 wait complete. 2023-07-14T18:14:48.012 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-07-14T18:14:48.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.168+0000 7f45b2fff700 1 -- 172.21.15.133:0/1618212203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ac0ffc10 msgr2=0x7f45ac100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:48.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.168+0000 7f45b2fff700 1 --2- 172.21.15.133:0/1618212203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ac0ffc10 0x7f45ac100020 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009990 tx=0x7f45a0009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:48.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.168+0000 7f45b2fff700 1 -- 172.21.15.133:0/1618212203 shutdown_connections 2023-07-14T18:14:48.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.168+0000 7f45b2fff700 1 --2- 172.21.15.133:0/1618212203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45ac101790 0x7f45ac108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:48.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.168+0000 7f45b2fff700 1 --2- 172.21.15.133:0/1618212203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ac0ffc10 0x7f45ac100020 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:48.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.168+0000 7f45b2fff700 1 --2- 172.21.15.133:0/1618212203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45ac100e00 0x7f45ac101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:48.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.168+0000 7f45b2fff700 1 -- 172.21.15.133:0/1618212203 >> 172.21.15.133:0/1618212203 conn(0x7f45ac0fb210 msgr2=0x7f45ac0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:48.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.168+0000 7f45b2fff700 1 -- 172.21.15.133:0/1618212203 shutdown_connections 2023-07-14T18:14:48.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.168+0000 7f45b2fff700 1 -- 172.21.15.133:0/1618212203 wait complete. 2023-07-14T18:14:48.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.169+0000 7f45b2fff700 1 Processor -- start 2023-07-14T18:14:48.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.169+0000 7f45b2fff700 1 -- start start 2023-07-14T18:14:48.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.169+0000 7f45b2fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45ac0ffc10 0x7f45ac0743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:48.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.169+0000 7f45b2fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45ac100e00 0x7f45ac072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:48.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.169+0000 7f45b2fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ac101790 0x7f45ac072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:48.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.169+0000 7f45b2fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac073710 con 0x7f45ac101790 2023-07-14T18:14:48.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.169+0000 7f45b2fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac073880 con 0x7f45ac0ffc10 2023-07-14T18:14:48.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.169+0000 7f45b2fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac19fa20 con 0x7f45ac100e00 2023-07-14T18:14:48.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.170+0000 7f45b159c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ac101790 0x7f45ac072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:48.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.170+0000 7f45b159c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ac101790 0x7f45ac072f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39238/0 (socket says 172.21.15.133:39238) 2023-07-14T18:14:48.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.170+0000 7f45b159c700 1 -- 172.21.15.133:0/439977155 learned_addr learned my addr 172.21.15.133:0/439977155 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:48.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.170+0000 7f45abfff700 1 --2- 172.21.15.133:0/439977155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45ac100e00 0x7f45ac072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:48.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.170+0000 7f45b0d9b700 1 --2- 172.21.15.133:0/439977155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45ac0ffc10 0x7f45ac0743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:48.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.170+0000 7f45b159c700 1 -- 172.21.15.133:0/439977155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45ac100e00 msgr2=0x7f45ac072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:48.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.170+0000 7f45b159c700 1 --2- 172.21.15.133:0/439977155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45ac100e00 0x7f45ac072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:48.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.170+0000 7f45b159c700 1 -- 172.21.15.133:0/439977155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45ac0ffc10 msgr2=0x7f45ac0743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:48.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.170+0000 7f45b159c700 1 --2- 172.21.15.133:0/439977155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45ac0ffc10 0x7f45ac0743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:48.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.170+0000 7f45b159c700 1 -- 172.21.15.133:0/439977155 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4598009520 con 0x7f45ac101790 2023-07-14T18:14:48.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.170+0000 7f45b159c700 1 --2- 172.21.15.133:0/439977155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ac101790 0x7f45ac072f70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f45a400ef30 tx=0x7f45a400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:48.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.170+0000 7f45a9ffb700 1 -- 172.21.15.133:0/439977155 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a4018070 con 0x7f45ac101790 2023-07-14T18:14:48.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.171+0000 7f45b2fff700 1 -- 172.21.15.133:0/439977155 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a00095f0 con 0x7f45ac101790 2023-07-14T18:14:48.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.171+0000 7f45a9ffb700 1 -- 172.21.15.133:0/439977155 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45a400f040 con 0x7f45ac101790 2023-07-14T18:14:48.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.171+0000 7f45a9ffb700 1 -- 172.21.15.133:0/439977155 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a400cd00 con 0x7f45ac101790 2023-07-14T18:14:48.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.171+0000 7f45b2fff700 1 -- 172.21.15.133:0/439977155 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45ac1a0020 con 0x7f45ac101790 2023-07-14T18:14:48.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.172+0000 7f45b2fff700 1 -- 172.21.15.133:0/439977155 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45ac106b90 con 0x7f45ac101790 2023-07-14T18:14:48.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.174+0000 7f45a9ffb700 1 -- 172.21.15.133:0/439977155 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f45a4003a00 con 0x7f45ac101790 2023-07-14T18:14:48.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.175+0000 7f45a9ffb700 1 --2- 172.21.15.133:0/439977155 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f459c0a1150 0x7f459c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:48.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.175+0000 7f45a9ffb700 1 -- 172.21.15.133:0/439977155 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f45a40cc7d0 con 0x7f45ac101790 2023-07-14T18:14:48.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.175+0000 7f45b0d9b700 1 --2- 172.21.15.133:0/439977155 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f459c0a1150 0x7f459c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:48.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.176+0000 7f45b0d9b700 1 --2- 172.21.15.133:0/439977155 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f459c0a1150 0x7f459c0a3600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f45a000b2e0 tx=0x7f45a0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:48.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.179+0000 7f45a9ffb700 1 -- 172.21.15.133:0/439977155 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f45a4095c40 con 0x7f45ac101790 2023-07-14T18:14:48.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:47 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/556321830' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-14T18:14:48.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:47 smithi161 ceph-mon[128993]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T18:14:48.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:47 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2023-07-14T18:14:48.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:47 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2023-07-14T18:14:48.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:47 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2023-07-14T18:14:48.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:47 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2023-07-14T18:14:48.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:47 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-07-14T18:14:48.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:47 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2023-07-14T18:14:48.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:47 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2023-07-14T18:14:48.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:47 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2023-07-14T18:14:48.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:47 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2023-07-14T18:14:48.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:47 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2023-07-14T18:14:48.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:47 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/431020081' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-07-14T18:14:48.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:47 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/556321830' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-14T18:14:48.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:47 smithi171 ceph-mon[129241]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T18:14:48.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:47 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2023-07-14T18:14:48.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:47 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2023-07-14T18:14:48.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:47 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2023-07-14T18:14:48.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:47 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2023-07-14T18:14:48.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:47 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-07-14T18:14:48.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:47 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2023-07-14T18:14:48.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:47 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2023-07-14T18:14:48.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:47 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2023-07-14T18:14:48.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:47 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2023-07-14T18:14:48.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:47 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2023-07-14T18:14:48.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:47 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/431020081' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-07-14T18:14:48.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:47 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/556321830' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-14T18:14:48.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:47 smithi133 ceph-mon[126384]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T18:14:48.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:47 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2023-07-14T18:14:48.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:47 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2023-07-14T18:14:48.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:47 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2023-07-14T18:14:48.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:47 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2023-07-14T18:14:48.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:47 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-07-14T18:14:48.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:47 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2023-07-14T18:14:48.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:47 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2023-07-14T18:14:48.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:47 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2023-07-14T18:14:48.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:47 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2023-07-14T18:14:48.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:47 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2023-07-14T18:14:48.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:47 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/431020081' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-07-14T18:14:48.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.425+0000 7f45b2fff700 1 -- 172.21.15.133:0/439977155 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f45ac02c8e0 con 0x7f45ac101790 2023-07-14T18:14:48.994 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.949+0000 7f45a9ffb700 1 -- 172.21.15.133:0/439977155 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7f45a4099260 con 0x7f45ac101790 2023-07-14T18:14:48.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.952+0000 7f45b2fff700 1 -- 172.21.15.133:0/439977155 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f459c0a1150 msgr2=0x7f459c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:48.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.952+0000 7f45b2fff700 1 --2- 172.21.15.133:0/439977155 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f459c0a1150 0x7f459c0a3600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f45a000b2e0 tx=0x7f45a0005fb0 comp rx=0 tx=0).stop 2023-07-14T18:14:48.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.952+0000 7f45b2fff700 1 -- 172.21.15.133:0/439977155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ac101790 msgr2=0x7f45ac072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:48.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.952+0000 7f45b2fff700 1 --2- 172.21.15.133:0/439977155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ac101790 0x7f45ac072f70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f45a400ef30 tx=0x7f45a400c3a0 comp rx=0 tx=0).stop 2023-07-14T18:14:48.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.952+0000 7f45b2fff700 1 -- 172.21.15.133:0/439977155 shutdown_connections 2023-07-14T18:14:48.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.952+0000 7f45b2fff700 1 --2- 172.21.15.133:0/439977155 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f459c0a1150 0x7f459c0a3600 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:48.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.952+0000 7f45b2fff700 1 --2- 172.21.15.133:0/439977155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45ac0ffc10 0x7f45ac0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:48.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.952+0000 7f45b2fff700 1 --2- 172.21.15.133:0/439977155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ac101790 0x7f45ac072f70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:48.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.952+0000 7f45b2fff700 1 --2- 172.21.15.133:0/439977155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45ac100e00 0x7f45ac072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:48.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.952+0000 7f45b2fff700 1 -- 172.21.15.133:0/439977155 >> 172.21.15.133:0/439977155 conn(0x7f45ac0fb210 msgr2=0x7f45ac0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:48.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.953+0000 7f45b2fff700 1 -- 172.21.15.133:0/439977155 shutdown_connections 2023-07-14T18:14:48.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:48.953+0000 7f45b2fff700 1 -- 172.21.15.133:0/439977155 wait complete. 2023-07-14T18:14:49.017 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-07-14T18:14:49.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.172+0000 7f8d7a9ff700 1 -- 172.21.15.133:0/4293587618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d741006f0 msgr2=0x7f8d74102ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:49.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.172+0000 7f8d7a9ff700 1 --2- 172.21.15.133:0/4293587618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d741006f0 0x7f8d74102ad0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8d64009990 tx=0x7f8d64009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:49.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.172+0000 7f8d7a9ff700 1 -- 172.21.15.133:0/4293587618 shutdown_connections 2023-07-14T18:14:49.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.172+0000 7f8d7a9ff700 1 --2- 172.21.15.133:0/4293587618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d74105930 0x7f8d74107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:49.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.172+0000 7f8d7a9ff700 1 --2- 172.21.15.133:0/4293587618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d741006f0 0x7f8d74102ad0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:49.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.172+0000 7f8d7a9ff700 1 --2- 172.21.15.133:0/4293587618 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8d74103010 0x7f8d741053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:49.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.172+0000 7f8d7a9ff700 1 -- 172.21.15.133:0/4293587618 >> 172.21.15.133:0/4293587618 conn(0x7f8d740fa130 msgr2=0x7f8d740fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:49.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.172+0000 7f8d7a9ff700 1 -- 172.21.15.133:0/4293587618 shutdown_connections 2023-07-14T18:14:49.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.172+0000 7f8d7a9ff700 1 -- 172.21.15.133:0/4293587618 wait complete. 2023-07-14T18:14:49.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.173+0000 7f8d7a9ff700 1 Processor -- start 2023-07-14T18:14:49.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.173+0000 7f8d7a9ff700 1 -- start start 2023-07-14T18:14:49.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.174+0000 7f8d7a9ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d741006f0 0x7f8d74100060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:49.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.174+0000 7f8d7a9ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8d74103010 0x7f8d740fe6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:49.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.174+0000 7f8d7a9ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d74105930 0x7f8d740febe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:49.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.174+0000 7f8d7a9ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d740ff380 con 0x7f8d74105930 2023-07-14T18:14:49.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.174+0000 7f8d7a9ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d740ff4f0 con 0x7f8d741006f0 2023-07-14T18:14:49.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.174+0000 7f8d7a9ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7419f9e0 con 0x7f8d74103010 2023-07-14T18:14:49.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.174+0000 7f8d78f9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d74105930 0x7f8d740febe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:49.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.174+0000 7f8d78f9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d74105930 0x7f8d740febe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39256/0 (socket says 172.21.15.133:39256) 2023-07-14T18:14:49.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.174+0000 7f8d78f9c700 1 -- 172.21.15.133:0/2442581004 learned_addr learned my addr 172.21.15.133:0/2442581004 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:49.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.174+0000 7f8d6bfff700 1 --2- 172.21.15.133:0/2442581004 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8d74103010 0x7f8d740fe6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:49.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.174+0000 7f8d73fff700 1 --2- 172.21.15.133:0/2442581004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d741006f0 0x7f8d74100060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:49.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.174+0000 7f8d78f9c700 1 -- 172.21.15.133:0/2442581004 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8d74103010 msgr2=0x7f8d740fe6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:49.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.174+0000 7f8d78f9c700 1 --2- 172.21.15.133:0/2442581004 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8d74103010 0x7f8d740fe6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:49.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.174+0000 7f8d78f9c700 1 -- 172.21.15.133:0/2442581004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d741006f0 msgr2=0x7f8d74100060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:49.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.174+0000 7f8d78f9c700 1 --2- 172.21.15.133:0/2442581004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d741006f0 0x7f8d74100060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:49.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.174+0000 7f8d78f9c700 1 -- 172.21.15.133:0/2442581004 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d640095f0 con 0x7f8d74105930 2023-07-14T18:14:49.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.175+0000 7f8d78f9c700 1 --2- 172.21.15.133:0/2442581004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d74105930 0x7f8d740febe0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c007ee0 tx=0x7f8d6c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:49.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.175+0000 7f8d71ffb700 1 -- 172.21.15.133:0/2442581004 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c00dd20 con 0x7f8d74105930 2023-07-14T18:14:49.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.175+0000 7f8d7a9ff700 1 -- 172.21.15.133:0/2442581004 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d7419fcc0 con 0x7f8d74105930 2023-07-14T18:14:49.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.175+0000 7f8d71ffb700 1 -- 172.21.15.133:0/2442581004 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d6c00f040 con 0x7f8d74105930 2023-07-14T18:14:49.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.175+0000 7f8d71ffb700 1 -- 172.21.15.133:0/2442581004 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c0048a0 con 0x7f8d74105930 2023-07-14T18:14:49.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.175+0000 7f8d7a9ff700 1 -- 172.21.15.133:0/2442581004 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d741a01d0 con 0x7f8d74105930 2023-07-14T18:14:49.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.176+0000 7f8d7a9ff700 1 -- 172.21.15.133:0/2442581004 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d740fbca0 con 0x7f8d74105930 2023-07-14T18:14:49.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.178+0000 7f8d71ffb700 1 -- 172.21.15.133:0/2442581004 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f8d6c003a00 con 0x7f8d74105930 2023-07-14T18:14:49.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.179+0000 7f8d71ffb700 1 --2- 172.21.15.133:0/2442581004 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8d600a11d0 0x7f8d600a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:49.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.179+0000 7f8d71ffb700 1 -- 172.21.15.133:0/2442581004 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8d6c0cc600 con 0x7f8d74105930 2023-07-14T18:14:49.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.180+0000 7f8d73fff700 1 --2- 172.21.15.133:0/2442581004 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8d600a11d0 0x7f8d600a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:49.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.180+0000 7f8d73fff700 1 --2- 172.21.15.133:0/2442581004 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8d600a11d0 0x7f8d600a3680 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400b5c0 tx=0x7f8d64005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:49.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.181+0000 7f8d71ffb700 1 -- 172.21.15.133:0/2442581004 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f8d6c0959e0 con 0x7f8d74105930 2023-07-14T18:14:49.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:48 smithi161 ceph-mon[128993]: pgmap v288: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-14T18:14:49.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:48 smithi161 ceph-mon[128993]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-14T18:14:49.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:48 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/431020081' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-07-14T18:14:49.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:48 smithi161 ceph-mon[128993]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-14T18:14:49.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:48 smithi161 ceph-mon[128993]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T18:14:49.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2023-07-14T18:14:49.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2023-07-14T18:14:49.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2023-07-14T18:14:49.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2023-07-14T18:14:49.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-07-14T18:14:49.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2023-07-14T18:14:49.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2023-07-14T18:14:49.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2023-07-14T18:14:49.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2023-07-14T18:14:49.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2023-07-14T18:14:49.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:48 smithi161 ceph-mon[128993]: osdmap e89: 12 total, 12 up, 12 in 2023-07-14T18:14:49.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:48 smithi161 ceph-mon[128993]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-07-14T18:14:49.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:48 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/439977155' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-14T18:14:49.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:48 smithi171 ceph-mon[129241]: pgmap v288: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-14T18:14:49.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:48 smithi171 ceph-mon[129241]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-14T18:14:49.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:48 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/431020081' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-07-14T18:14:49.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:48 smithi171 ceph-mon[129241]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-14T18:14:49.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:48 smithi171 ceph-mon[129241]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T18:14:49.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2023-07-14T18:14:49.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2023-07-14T18:14:49.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2023-07-14T18:14:49.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2023-07-14T18:14:49.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-07-14T18:14:49.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2023-07-14T18:14:49.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2023-07-14T18:14:49.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2023-07-14T18:14:49.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2023-07-14T18:14:49.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2023-07-14T18:14:49.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:48 smithi171 ceph-mon[129241]: osdmap e89: 12 total, 12 up, 12 in 2023-07-14T18:14:49.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:48 smithi171 ceph-mon[129241]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-07-14T18:14:49.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:48 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/439977155' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-14T18:14:49.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:48 smithi133 ceph-mon[126384]: pgmap v288: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-14T18:14:49.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:48 smithi133 ceph-mon[126384]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-14T18:14:49.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:48 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/431020081' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-07-14T18:14:49.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:48 smithi133 ceph-mon[126384]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-14T18:14:49.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:48 smithi133 ceph-mon[126384]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T18:14:49.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2023-07-14T18:14:49.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2023-07-14T18:14:49.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2023-07-14T18:14:49.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2023-07-14T18:14:49.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-07-14T18:14:49.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2023-07-14T18:14:49.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2023-07-14T18:14:49.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2023-07-14T18:14:49.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2023-07-14T18:14:49.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2023-07-14T18:14:49.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:48 smithi133 ceph-mon[126384]: osdmap e89: 12 total, 12 up, 12 in 2023-07-14T18:14:49.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:48 smithi133 ceph-mon[126384]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-07-14T18:14:49.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:48 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/439977155' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-14T18:14:49.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.426+0000 7f8d7a9ff700 1 -- 172.21.15.133:0/2442581004 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f8d7402c8e0 con 0x7f8d74105930 2023-07-14T18:14:49.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.877+0000 7f8d71ffb700 1 -- 172.21.15.133:0/2442581004 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7f8d6c099000 con 0x7f8d74105930 2023-07-14T18:14:49.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.880+0000 7f8d7a9ff700 1 -- 172.21.15.133:0/2442581004 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8d600a11d0 msgr2=0x7f8d600a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:49.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.880+0000 7f8d7a9ff700 1 --2- 172.21.15.133:0/2442581004 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8d600a11d0 0x7f8d600a3680 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400b5c0 tx=0x7f8d64005fd0 comp rx=0 tx=0).stop 2023-07-14T18:14:49.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.880+0000 7f8d7a9ff700 1 -- 172.21.15.133:0/2442581004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d74105930 msgr2=0x7f8d740febe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:49.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.880+0000 7f8d7a9ff700 1 --2- 172.21.15.133:0/2442581004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d74105930 0x7f8d740febe0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c007ee0 tx=0x7f8d6c00d3b0 comp rx=0 tx=0).stop 2023-07-14T18:14:49.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.880+0000 7f8d7a9ff700 1 -- 172.21.15.133:0/2442581004 shutdown_connections 2023-07-14T18:14:49.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.880+0000 7f8d7a9ff700 1 --2- 172.21.15.133:0/2442581004 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8d600a11d0 0x7f8d600a3680 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:49.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.880+0000 7f8d7a9ff700 1 --2- 172.21.15.133:0/2442581004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d741006f0 0x7f8d74100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:49.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.880+0000 7f8d7a9ff700 1 --2- 172.21.15.133:0/2442581004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d74105930 0x7f8d740febe0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:49.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.880+0000 7f8d7a9ff700 1 --2- 172.21.15.133:0/2442581004 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8d74103010 0x7f8d740fe6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:49.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.880+0000 7f8d7a9ff700 1 -- 172.21.15.133:0/2442581004 >> 172.21.15.133:0/2442581004 conn(0x7f8d740fa130 msgr2=0x7f8d74101350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:49.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.880+0000 7f8d7a9ff700 1 -- 172.21.15.133:0/2442581004 shutdown_connections 2023-07-14T18:14:49.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:49.880+0000 7f8d7a9ff700 1 -- 172.21.15.133:0/2442581004 wait complete. 2023-07-14T18:14:49.947 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2023-07-14T18:14:49.948 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-07-14T18:14:50.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.109+0000 7f15c2dff700 1 -- 172.21.15.133:0/2062633728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15bc100e10 msgr2=0x7f15bc101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:50.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.109+0000 7f15c2dff700 1 --2- 172.21.15.133:0/2062633728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15bc100e10 0x7f15bc101260 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f15a8009990 tx=0x7f15a8009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:50.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.109+0000 7f15c2dff700 1 -- 172.21.15.133:0/2062633728 shutdown_connections 2023-07-14T18:14:50.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.109+0000 7f15c2dff700 1 --2- 172.21.15.133:0/2062633728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15bc0ffc20 0x7f15bc100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:50.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.109+0000 7f15c2dff700 1 --2- 172.21.15.133:0/2062633728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15bc1017a0 0x7f15bc108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:50.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.109+0000 7f15c2dff700 1 --2- 172.21.15.133:0/2062633728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15bc100e10 0x7f15bc101260 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:50.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.109+0000 7f15c2dff700 1 -- 172.21.15.133:0/2062633728 >> 172.21.15.133:0/2062633728 conn(0x7f15bc0fb260 msgr2=0x7f15bc0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:50.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.109+0000 7f15c2dff700 1 -- 172.21.15.133:0/2062633728 shutdown_connections 2023-07-14T18:14:50.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.109+0000 7f15c2dff700 1 -- 172.21.15.133:0/2062633728 wait complete. 2023-07-14T18:14:50.156 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.110+0000 7f15c2dff700 1 Processor -- start 2023-07-14T18:14:50.156 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.110+0000 7f15c2dff700 1 -- start start 2023-07-14T18:14:50.156 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.111+0000 7f15c2dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15bc0ffc20 0x7f15bc197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:50.157 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.111+0000 7f15c2dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15bc100e10 0x7f15bc1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:50.157 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.111+0000 7f15c2dff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15bc1017a0 0x7f15bc19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:50.157 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.111+0000 7f15c2dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc19f710 con 0x7f15bc100e10 2023-07-14T18:14:50.157 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.111+0000 7f15c2dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc19f850 con 0x7f15bc0ffc20 2023-07-14T18:14:50.157 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.111+0000 7f15c2dff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc19f9e0 con 0x7f15bc1017a0 2023-07-14T18:14:50.157 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.111+0000 7f15bbfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15bc100e10 0x7f15bc1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:50.157 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.111+0000 7f15c0b9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15bc0ffc20 0x7f15bc197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:50.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.111+0000 7f15bbfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15bc100e10 0x7f15bc1980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39272/0 (socket says 172.21.15.133:39272) 2023-07-14T18:14:50.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.111+0000 7f15bbfff700 1 -- 172.21.15.133:0/1290599766 learned_addr learned my addr 172.21.15.133:0/1290599766 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:50.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.111+0000 7f15c139c700 1 --2- 172.21.15.133:0/1290599766 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15bc1017a0 0x7f15bc19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:50.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.111+0000 7f15bbfff700 1 -- 172.21.15.133:0/1290599766 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15bc1017a0 msgr2=0x7f15bc19f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:50.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.111+0000 7f15bbfff700 1 --2- 172.21.15.133:0/1290599766 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15bc1017a0 0x7f15bc19f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:50.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.111+0000 7f15bbfff700 1 -- 172.21.15.133:0/1290599766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15bc0ffc20 msgr2=0x7f15bc197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:50.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.111+0000 7f15bbfff700 1 --2- 172.21.15.133:0/1290599766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15bc0ffc20 0x7f15bc197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:50.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.111+0000 7f15bbfff700 1 -- 172.21.15.133:0/1290599766 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a80095f0 con 0x7f15bc100e10 2023-07-14T18:14:50.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.111+0000 7f15c0b9b700 1 --2- 172.21.15.133:0/1290599766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15bc0ffc20 0x7f15bc197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:14:50.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.112+0000 7f15bbfff700 1 --2- 172.21.15.133:0/1290599766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15bc100e10 0x7f15bc1980d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f15a8004760 tx=0x7f15a8004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:50.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.112+0000 7f15b9ffb700 1 -- 172.21.15.133:0/1290599766 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a801d070 con 0x7f15bc100e10 2023-07-14T18:14:50.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.112+0000 7f15c2dff700 1 -- 172.21.15.133:0/1290599766 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15bc19fc60 con 0x7f15bc100e10 2023-07-14T18:14:50.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.112+0000 7f15b9ffb700 1 -- 172.21.15.133:0/1290599766 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15a8004830 con 0x7f15bc100e10 2023-07-14T18:14:50.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.112+0000 7f15b9ffb700 1 -- 172.21.15.133:0/1290599766 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a8005350 con 0x7f15bc100e10 2023-07-14T18:14:50.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.112+0000 7f15c2dff700 1 -- 172.21.15.133:0/1290599766 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15bc1a0020 con 0x7f15bc100e10 2023-07-14T18:14:50.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.113+0000 7f15c2dff700 1 -- 172.21.15.133:0/1290599766 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15bc10b440 con 0x7f15bc100e10 2023-07-14T18:14:50.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.115+0000 7f15b9ffb700 1 -- 172.21.15.133:0/1290599766 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f15a8005530 con 0x7f15bc100e10 2023-07-14T18:14:50.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.116+0000 7f15b9ffb700 1 --2- 172.21.15.133:0/1290599766 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f15ac0a1150 0x7f15ac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:50.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.116+0000 7f15b9ffb700 1 -- 172.21.15.133:0/1290599766 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f15a80cde30 con 0x7f15bc100e10 2023-07-14T18:14:50.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.116+0000 7f15c0b9b700 1 --2- 172.21.15.133:0/1290599766 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f15ac0a1150 0x7f15ac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:50.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.117+0000 7f15c0b9b700 1 --2- 172.21.15.133:0/1290599766 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f15ac0a1150 0x7f15ac0a3600 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f15bc100c70 tx=0x7f15b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:50.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.120+0000 7f15b9ffb700 1 -- 172.21.15.133:0/1290599766 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f15a805a780 con 0x7f15bc100e10 2023-07-14T18:14:50.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:49 smithi161 ceph-mon[128993]: osdmap e90: 12 total, 12 up, 12 in 2023-07-14T18:14:50.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:49 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/439977155' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-14T18:14:50.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:49 smithi161 ceph-mon[128993]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-07-14T18:14:50.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:49 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2442581004' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-14T18:14:50.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:49 smithi161 ceph-mon[128993]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-07-14T18:14:50.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:49 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2442581004' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-14T18:14:50.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:49 smithi161 ceph-mon[128993]: mds.? [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] up:active 2023-07-14T18:14:50.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:49 smithi161 ceph-mon[128993]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-14T18:14:50.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:49 smithi161 ceph-mon[128993]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-14T18:14:50.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:49 smithi161 ceph-mon[128993]: Cluster is now healthy 2023-07-14T18:14:50.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:49 smithi161 ceph-mon[128993]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2023-07-14T18:14:50.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:49 smithi161 ceph-mon[128993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2023-07-14T18:14:50.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:49 smithi161 ceph-mon[128993]: daemon mds.f is now active in filesystem cephfs as rank 2 2023-07-14T18:14:50.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:49 smithi171 ceph-mon[129241]: osdmap e90: 12 total, 12 up, 12 in 2023-07-14T18:14:50.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:49 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/439977155' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-14T18:14:50.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:49 smithi171 ceph-mon[129241]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-07-14T18:14:50.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:49 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2442581004' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-14T18:14:50.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:49 smithi171 ceph-mon[129241]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-07-14T18:14:50.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:49 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2442581004' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-14T18:14:50.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:49 smithi171 ceph-mon[129241]: mds.? [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] up:active 2023-07-14T18:14:50.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:49 smithi171 ceph-mon[129241]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-14T18:14:50.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:49 smithi171 ceph-mon[129241]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-14T18:14:50.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:49 smithi171 ceph-mon[129241]: Cluster is now healthy 2023-07-14T18:14:50.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:49 smithi171 ceph-mon[129241]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2023-07-14T18:14:50.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:49 smithi171 ceph-mon[129241]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2023-07-14T18:14:50.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:49 smithi171 ceph-mon[129241]: daemon mds.f is now active in filesystem cephfs as rank 2 2023-07-14T18:14:50.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.352+0000 7f15c2dff700 1 -- 172.21.15.133:0/1290599766 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f15bc107750 con 0x7f15ac0a1150 2023-07-14T18:14:50.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:49 smithi133 ceph-mon[126384]: osdmap e90: 12 total, 12 up, 12 in 2023-07-14T18:14:50.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:49 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/439977155' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-14T18:14:50.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:49 smithi133 ceph-mon[126384]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-07-14T18:14:50.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:49 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2442581004' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-14T18:14:50.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:49 smithi133 ceph-mon[126384]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-07-14T18:14:50.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:49 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2442581004' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-14T18:14:50.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:49 smithi133 ceph-mon[126384]: mds.? [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] up:active 2023-07-14T18:14:50.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:49 smithi133 ceph-mon[126384]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-14T18:14:50.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:49 smithi133 ceph-mon[126384]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-14T18:14:50.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:49 smithi133 ceph-mon[126384]: Cluster is now healthy 2023-07-14T18:14:50.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:49 smithi133 ceph-mon[126384]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2023-07-14T18:14:50.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:49 smithi133 ceph-mon[126384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2023-07-14T18:14:50.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:49 smithi133 ceph-mon[126384]: daemon mds.f is now active in filesystem cephfs as rank 2 2023-07-14T18:14:50.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.398+0000 7f15b9ffb700 1 -- 172.21.15.133:0/1290599766 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f15bc107750 con 0x7f15ac0a1150 2023-07-14T18:14:50.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.400+0000 7f15c2dff700 1 -- 172.21.15.133:0/1290599766 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f15ac0a1150 msgr2=0x7f15ac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:50.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.400+0000 7f15c2dff700 1 --2- 172.21.15.133:0/1290599766 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f15ac0a1150 0x7f15ac0a3600 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f15bc100c70 tx=0x7f15b0009260 comp rx=0 tx=0).stop 2023-07-14T18:14:50.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.400+0000 7f15c2dff700 1 -- 172.21.15.133:0/1290599766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15bc100e10 msgr2=0x7f15bc1980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:50.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.400+0000 7f15c2dff700 1 --2- 172.21.15.133:0/1290599766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15bc100e10 0x7f15bc1980d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f15a8004760 tx=0x7f15a8004790 comp rx=0 tx=0).stop 2023-07-14T18:14:50.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.400+0000 7f15c2dff700 1 -- 172.21.15.133:0/1290599766 shutdown_connections 2023-07-14T18:14:50.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.400+0000 7f15c2dff700 1 --2- 172.21.15.133:0/1290599766 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f15ac0a1150 0x7f15ac0a3600 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:50.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.400+0000 7f15c2dff700 1 --2- 172.21.15.133:0/1290599766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15bc0ffc20 0x7f15bc197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:50.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.400+0000 7f15c2dff700 1 --2- 172.21.15.133:0/1290599766 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15bc1017a0 0x7f15bc19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:50.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.400+0000 7f15c2dff700 1 --2- 172.21.15.133:0/1290599766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15bc100e10 0x7f15bc1980d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:50.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.400+0000 7f15c2dff700 1 -- 172.21.15.133:0/1290599766 >> 172.21.15.133:0/1290599766 conn(0x7f15bc0fb260 msgr2=0x7f15bc0fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:50.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.401+0000 7f15c2dff700 1 -- 172.21.15.133:0/1290599766 shutdown_connections 2023-07-14T18:14:50.446 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.401+0000 7f15c2dff700 1 -- 172.21.15.133:0/1290599766 wait complete. 2023-07-14T18:14:50.462 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-07-14T18:14:50.655 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:14:50 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:14:50.359+0000 7fa629cca700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T18:14:50.656 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:14:50 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:14:50.359+0000 7fa629cca700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T18:14:50.656 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:14:50 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:14:50.359+0000 7fa629cca700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T18:14:50.656 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:14:50 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:14:50.359+0000 7fa629cca700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T18:14:50.656 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:14:50 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:14:50.359+0000 7fa629cca700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T18:14:50.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.618+0000 7fb6c1fff700 1 -- 172.21.15.133:0/2607800020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6bc100e10 msgr2=0x7fb6bc101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:50.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.618+0000 7fb6c1fff700 1 --2- 172.21.15.133:0/2607800020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6bc100e10 0x7fb6bc101260 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009990 tx=0x7fb6b0009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:50.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.619+0000 7fb6c1fff700 1 -- 172.21.15.133:0/2607800020 shutdown_connections 2023-07-14T18:14:50.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.619+0000 7fb6c1fff700 1 --2- 172.21.15.133:0/2607800020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6bc1017a0 0x7fb6bc108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:50.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.619+0000 7fb6c1fff700 1 --2- 172.21.15.133:0/2607800020 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb6bc0ffc20 0x7fb6bc100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:50.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.619+0000 7fb6c1fff700 1 --2- 172.21.15.133:0/2607800020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6bc100e10 0x7fb6bc101260 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:50.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.619+0000 7fb6c1fff700 1 -- 172.21.15.133:0/2607800020 >> 172.21.15.133:0/2607800020 conn(0x7fb6bc0fb260 msgr2=0x7fb6bc0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:50.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.619+0000 7fb6c1fff700 1 -- 172.21.15.133:0/2607800020 shutdown_connections 2023-07-14T18:14:50.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.619+0000 7fb6c1fff700 1 -- 172.21.15.133:0/2607800020 wait complete. 2023-07-14T18:14:50.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.620+0000 7fb6c1fff700 1 Processor -- start 2023-07-14T18:14:50.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.620+0000 7fb6c1fff700 1 -- start start 2023-07-14T18:14:50.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.620+0000 7fb6c1fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb6bc0ffc20 0x7fb6bc0743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:50.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.620+0000 7fb6c1fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6bc100e10 0x7fb6bc072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:50.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.620+0000 7fb6c1fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6bc1017a0 0x7fb6bc072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:50.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.620+0000 7fb6c1fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc073710 con 0x7fb6bc100e10 2023-07-14T18:14:50.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.620+0000 7fb6c1fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc073880 con 0x7fb6bc1017a0 2023-07-14T18:14:50.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.620+0000 7fb6c1fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc19f9d0 con 0x7fb6bc0ffc20 2023-07-14T18:14:50.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.620+0000 7fb6baffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6bc100e10 0x7fb6bc072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:50.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.620+0000 7fb6bbfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6bc1017a0 0x7fb6bc072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:50.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.621+0000 7fb6bb7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb6bc0ffc20 0x7fb6bc0743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:50.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.621+0000 7fb6bbfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6bc1017a0 0x7fb6bc072f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:34022/0 (socket says 172.21.15.133:34022) 2023-07-14T18:14:50.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.621+0000 7fb6bbfff700 1 -- 172.21.15.133:0/2288139814 learned_addr learned my addr 172.21.15.133:0/2288139814 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:50.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.621+0000 7fb6baffd700 1 -- 172.21.15.133:0/2288139814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb6bc0ffc20 msgr2=0x7fb6bc0743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:50.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.621+0000 7fb6baffd700 1 --2- 172.21.15.133:0/2288139814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb6bc0ffc20 0x7fb6bc0743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:50.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.621+0000 7fb6baffd700 1 -- 172.21.15.133:0/2288139814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6bc1017a0 msgr2=0x7fb6bc072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:50.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.621+0000 7fb6baffd700 1 --2- 172.21.15.133:0/2288139814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6bc1017a0 0x7fb6bc072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:50.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.621+0000 7fb6baffd700 1 -- 172.21.15.133:0/2288139814 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b00095f0 con 0x7fb6bc100e10 2023-07-14T18:14:50.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.621+0000 7fb6bbfff700 1 --2- 172.21.15.133:0/2288139814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6bc1017a0 0x7fb6bc072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:14:50.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.621+0000 7fb6baffd700 1 --2- 172.21.15.133:0/2288139814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6bc100e10 0x7fb6bc072a30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb6b000b620 tx=0x7fb6b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:50.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.621+0000 7fb6b8ff9700 1 -- 172.21.15.133:0/2288139814 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b001d070 con 0x7fb6bc100e10 2023-07-14T18:14:50.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.621+0000 7fb6b8ff9700 1 -- 172.21.15.133:0/2288139814 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6b0021410 con 0x7fb6bc100e10 2023-07-14T18:14:50.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.621+0000 7fb6c1fff700 1 -- 172.21.15.133:0/2288139814 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6bc19fc50 con 0x7fb6bc100e10 2023-07-14T18:14:50.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.621+0000 7fb6b8ff9700 1 -- 172.21.15.133:0/2288139814 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0017650 con 0x7fb6bc100e10 2023-07-14T18:14:50.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.622+0000 7fb6c1fff700 1 -- 172.21.15.133:0/2288139814 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6bc1a00d0 con 0x7fb6bc100e10 2023-07-14T18:14:50.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.623+0000 7fb6c1fff700 1 -- 172.21.15.133:0/2288139814 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6bc10b440 con 0x7fb6bc100e10 2023-07-14T18:14:50.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.624+0000 7fb6b8ff9700 1 -- 172.21.15.133:0/2288139814 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fb6b00217f0 con 0x7fb6bc100e10 2023-07-14T18:14:50.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.626+0000 7fb6b8ff9700 1 --2- 172.21.15.133:0/2288139814 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb6a40a11d0 0x7fb6a40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:50.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.626+0000 7fb6b8ff9700 1 -- 172.21.15.133:0/2288139814 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb6b00cdfb0 con 0x7fb6bc100e10 2023-07-14T18:14:50.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.626+0000 7fb6bb7fe700 1 --2- 172.21.15.133:0/2288139814 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb6a40a11d0 0x7fb6a40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:50.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.626+0000 7fb6bb7fe700 1 --2- 172.21.15.133:0/2288139814 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb6a40a11d0 0x7fb6a40a3680 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc100c70 tx=0x7fb6ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:50.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.629+0000 7fb6b8ff9700 1 -- 172.21.15.133:0/2288139814 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fb6b0097420 con 0x7fb6bc100e10 2023-07-14T18:14:50.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.858+0000 7fb6c1fff700 1 -- 172.21.15.133:0/2288139814 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6bc060a80 con 0x7fb6a40a11d0 2023-07-14T18:14:50.906 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.863+0000 7fb6b8ff9700 1 -- 172.21.15.133:0/2288139814 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fb6bc060a80 con 0x7fb6a40a11d0 2023-07-14T18:14:50.906 INFO:teuthology.orchestra.run.smithi133.stdout:/volumes/_nogroup/sv_0/b7946942-424a-4638-9cd9-7826bdc1f26b 2023-07-14T18:14:50.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.865+0000 7fb6c1fff700 1 -- 172.21.15.133:0/2288139814 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb6a40a11d0 msgr2=0x7fb6a40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:50.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.865+0000 7fb6c1fff700 1 --2- 172.21.15.133:0/2288139814 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb6a40a11d0 0x7fb6a40a3680 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc100c70 tx=0x7fb6ac009260 comp rx=0 tx=0).stop 2023-07-14T18:14:50.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.865+0000 7fb6c1fff700 1 -- 172.21.15.133:0/2288139814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6bc100e10 msgr2=0x7fb6bc072a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:50.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.865+0000 7fb6c1fff700 1 --2- 172.21.15.133:0/2288139814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6bc100e10 0x7fb6bc072a30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb6b000b620 tx=0x7fb6b0004730 comp rx=0 tx=0).stop 2023-07-14T18:14:50.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.865+0000 7fb6c1fff700 1 -- 172.21.15.133:0/2288139814 shutdown_connections 2023-07-14T18:14:50.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.865+0000 7fb6c1fff700 1 --2- 172.21.15.133:0/2288139814 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb6a40a11d0 0x7fb6a40a3680 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:50.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.865+0000 7fb6c1fff700 1 --2- 172.21.15.133:0/2288139814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6bc1017a0 0x7fb6bc072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:50.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.865+0000 7fb6c1fff700 1 --2- 172.21.15.133:0/2288139814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb6bc0ffc20 0x7fb6bc0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:50.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.865+0000 7fb6c1fff700 1 --2- 172.21.15.133:0/2288139814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6bc100e10 0x7fb6bc072a30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:50.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.865+0000 7fb6c1fff700 1 -- 172.21.15.133:0/2288139814 >> 172.21.15.133:0/2288139814 conn(0x7fb6bc0fb260 msgr2=0x7fb6bc0fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:50.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.865+0000 7fb6c1fff700 1 -- 172.21.15.133:0/2288139814 shutdown_connections 2023-07-14T18:14:50.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:50.865+0000 7fb6c1fff700 1 -- 172.21.15.133:0/2288139814 wait complete. 2023-07-14T18:14:50.927 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-07-14T18:14:51.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.086+0000 7f412a7ff700 1 -- 172.21.15.133:0/3305479951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41240ffc10 msgr2=0x7f4124100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:51.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.086+0000 7f412a7ff700 1 --2- 172.21.15.133:0/3305479951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41240ffc10 0x7f4124100020 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f41140098f0 tx=0x7f4114009c00 comp rx=0 tx=0).stop 2023-07-14T18:14:51.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.086+0000 7f412a7ff700 1 -- 172.21.15.133:0/3305479951 shutdown_connections 2023-07-14T18:14:51.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.086+0000 7f412a7ff700 1 --2- 172.21.15.133:0/3305479951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4124101790 0x7f4124108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:51.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.086+0000 7f412a7ff700 1 --2- 172.21.15.133:0/3305479951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41240ffc10 0x7f4124100020 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:51.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.086+0000 7f412a7ff700 1 --2- 172.21.15.133:0/3305479951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4124100e00 0x7f4124101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:51.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.086+0000 7f412a7ff700 1 -- 172.21.15.133:0/3305479951 >> 172.21.15.133:0/3305479951 conn(0x7f41240fb210 msgr2=0x7f41240fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:51.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.086+0000 7f412a7ff700 1 -- 172.21.15.133:0/3305479951 shutdown_connections 2023-07-14T18:14:51.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.086+0000 7f412a7ff700 1 -- 172.21.15.133:0/3305479951 wait complete. 2023-07-14T18:14:51.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.087+0000 7f412a7ff700 1 Processor -- start 2023-07-14T18:14:51.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.087+0000 7f412a7ff700 1 -- start start 2023-07-14T18:14:51.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.088+0000 7f412a7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41240ffc10 0x7f41241959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:51.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.088+0000 7f412a7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4124100e00 0x7f4124195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:51.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.088+0000 7f412a7ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4124101790 0x7f412419cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:51.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.088+0000 7f412a7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412419d570 con 0x7f41240ffc10 2023-07-14T18:14:51.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.088+0000 7f412a7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412419d6b0 con 0x7f4124100e00 2023-07-14T18:14:51.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.088+0000 7f412a7ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412419d840 con 0x7f4124101790 2023-07-14T18:14:51.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.088+0000 7f4123fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41240ffc10 0x7f41241959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:51.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.088+0000 7f4123fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41240ffc10 0x7f41241959f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39318/0 (socket says 172.21.15.133:39318) 2023-07-14T18:14:51.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.088+0000 7f4123fff700 1 -- 172.21.15.133:0/729575507 learned_addr learned my addr 172.21.15.133:0/729575507 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:51.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.088+0000 7f41237fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4124100e00 0x7f4124195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:51.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.088+0000 7f4128d9c700 1 --2- 172.21.15.133:0/729575507 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4124101790 0x7f412419cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:51.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.088+0000 7f4123fff700 1 -- 172.21.15.133:0/729575507 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4124101790 msgr2=0x7f412419cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:51.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.088+0000 7f4123fff700 1 --2- 172.21.15.133:0/729575507 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4124101790 0x7f412419cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:51.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.088+0000 7f4123fff700 1 -- 172.21.15.133:0/729575507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4124100e00 msgr2=0x7f4124195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:51.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.088+0000 7f4123fff700 1 --2- 172.21.15.133:0/729575507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4124100e00 0x7f4124195f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:51.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.088+0000 7f4123fff700 1 -- 172.21.15.133:0/729575507 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4114009520 con 0x7f41240ffc10 2023-07-14T18:14:51.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.088+0000 7f4123fff700 1 --2- 172.21.15.133:0/729575507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41240ffc10 0x7f41241959f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4114005e90 tx=0x7f411400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:51.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.089+0000 7f41217fa700 1 -- 172.21.15.133:0/729575507 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411401c070 con 0x7f41240ffc10 2023-07-14T18:14:51.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.089+0000 7f412a7ff700 1 -- 172.21.15.133:0/729575507 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f412419dac0 con 0x7f41240ffc10 2023-07-14T18:14:51.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.089+0000 7f41217fa700 1 -- 172.21.15.133:0/729575507 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f411400f7f0 con 0x7f41240ffc10 2023-07-14T18:14:51.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.089+0000 7f41217fa700 1 -- 172.21.15.133:0/729575507 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114003990 con 0x7f41240ffc10 2023-07-14T18:14:51.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.089+0000 7f412a7ff700 1 -- 172.21.15.133:0/729575507 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f412419dee0 con 0x7f41240ffc10 2023-07-14T18:14:51.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.090+0000 7f412a7ff700 1 -- 172.21.15.133:0/729575507 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4124107220 con 0x7f41240ffc10 2023-07-14T18:14:51.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.092+0000 7f41217fa700 1 -- 172.21.15.133:0/729575507 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4114020030 con 0x7f41240ffc10 2023-07-14T18:14:51.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.093+0000 7f41217fa700 1 --2- 172.21.15.133:0/729575507 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f410c0a1150 0x7f410c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:51.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.093+0000 7f41217fa700 1 -- 172.21.15.133:0/729575507 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f41140cdbe0 con 0x7f41240ffc10 2023-07-14T18:14:51.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.093+0000 7f41237fe700 1 --2- 172.21.15.133:0/729575507 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f410c0a1150 0x7f410c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:51.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.094+0000 7f41237fe700 1 --2- 172.21.15.133:0/729575507 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f410c0a1150 0x7f410c0a3600 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4118005fd0 tx=0x7f4118009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:51.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.097+0000 7f41217fa700 1 -- 172.21.15.133:0/729575507 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f4114097050 con 0x7f41240ffc10 2023-07-14T18:14:51.350 INFO:journalctl@ceph.mds.b.smithi161.stdout:Jul 14 18:14:50 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-b[154371]: 2023-07-14T18:14:50.988+0000 7faba7315700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-07-14T18:14:51.350 INFO:journalctl@ceph.mds.b.smithi161.stdout:Jul 14 18:14:50 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-b[154371]: 2023-07-14T18:14:50.988+0000 7faba7315700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-07-14T18:14:51.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:50 smithi161 ceph-mon[128993]: pgmap v291: 129 pgs: 3 peering, 126 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-07-14T18:14:51.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:50 smithi161 ceph-mon[128993]: mds.? [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] up:active 2023-07-14T18:14:51.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:50 smithi161 ceph-mon[128993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2023-07-14T18:14:51.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:50 smithi161 ceph-mon[128993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:14:51.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:50 smithi171 ceph-mon[129241]: pgmap v291: 129 pgs: 3 peering, 126 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-07-14T18:14:51.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:50 smithi171 ceph-mon[129241]: mds.? [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] up:active 2023-07-14T18:14:51.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:50 smithi171 ceph-mon[129241]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2023-07-14T18:14:51.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:50 smithi171 ceph-mon[129241]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:14:51.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.325+0000 7f412a7ff700 1 -- 172.21.15.133:0/729575507 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f4124060b10 con 0x7f410c0a1150 2023-07-14T18:14:51.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.350+0000 7f41217fa700 1 -- 172.21.15.133:0/729575507 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4124060b10 con 0x7f410c0a1150 2023-07-14T18:14:51.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.353+0000 7f412a7ff700 1 -- 172.21.15.133:0/729575507 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f410c0a1150 msgr2=0x7f410c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:51.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.353+0000 7f412a7ff700 1 --2- 172.21.15.133:0/729575507 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f410c0a1150 0x7f410c0a3600 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4118005fd0 tx=0x7f4118009310 comp rx=0 tx=0).stop 2023-07-14T18:14:51.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.353+0000 7f412a7ff700 1 -- 172.21.15.133:0/729575507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41240ffc10 msgr2=0x7f41241959f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:51.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.353+0000 7f412a7ff700 1 --2- 172.21.15.133:0/729575507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41240ffc10 0x7f41241959f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4114005e90 tx=0x7f411400b730 comp rx=0 tx=0).stop 2023-07-14T18:14:51.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.353+0000 7f412a7ff700 1 -- 172.21.15.133:0/729575507 shutdown_connections 2023-07-14T18:14:51.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.353+0000 7f412a7ff700 1 --2- 172.21.15.133:0/729575507 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f410c0a1150 0x7f410c0a3600 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:51.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.353+0000 7f412a7ff700 1 --2- 172.21.15.133:0/729575507 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4124101790 0x7f412419cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:51.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.353+0000 7f412a7ff700 1 --2- 172.21.15.133:0/729575507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41240ffc10 0x7f41241959f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:51.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.353+0000 7f412a7ff700 1 --2- 172.21.15.133:0/729575507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4124100e00 0x7f4124195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:51.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.353+0000 7f412a7ff700 1 -- 172.21.15.133:0/729575507 >> 172.21.15.133:0/729575507 conn(0x7f41240fb210 msgr2=0x7f4124104360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:51.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.353+0000 7f412a7ff700 1 -- 172.21.15.133:0/729575507 shutdown_connections 2023-07-14T18:14:51.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.353+0000 7f412a7ff700 1 -- 172.21.15.133:0/729575507 wait complete. 2023-07-14T18:14:51.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:50 smithi133 ceph-mon[126384]: pgmap v291: 129 pgs: 3 peering, 126 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-07-14T18:14:51.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:50 smithi133 ceph-mon[126384]: mds.? [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] up:active 2023-07-14T18:14:51.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:50 smithi133 ceph-mon[126384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2023-07-14T18:14:51.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:50 smithi133 ceph-mon[126384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:14:51.416 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-07-14T18:14:51.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.572+0000 7f60e41ff700 1 -- 172.21.15.133:0/4042850354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60dc0ffc20 msgr2=0x7f60dc100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:51.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.572+0000 7f60e41ff700 1 --2- 172.21.15.133:0/4042850354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60dc0ffc20 0x7f60dc100030 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f60d8009990 tx=0x7f60d8009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:51.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.572+0000 7f60e41ff700 1 -- 172.21.15.133:0/4042850354 shutdown_connections 2023-07-14T18:14:51.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.572+0000 7f60e41ff700 1 --2- 172.21.15.133:0/4042850354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60dc0ffc20 0x7f60dc100030 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:51.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.572+0000 7f60e41ff700 1 --2- 172.21.15.133:0/4042850354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60dc1017a0 0x7f60dc108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:51.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.572+0000 7f60e41ff700 1 --2- 172.21.15.133:0/4042850354 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60dc100e10 0x7f60dc101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:51.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.572+0000 7f60e41ff700 1 -- 172.21.15.133:0/4042850354 >> 172.21.15.133:0/4042850354 conn(0x7f60dc0fb260 msgr2=0x7f60dc0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:51.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.573+0000 7f60e41ff700 1 -- 172.21.15.133:0/4042850354 shutdown_connections 2023-07-14T18:14:51.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.573+0000 7f60e41ff700 1 -- 172.21.15.133:0/4042850354 wait complete. 2023-07-14T18:14:51.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.573+0000 7f60e41ff700 1 Processor -- start 2023-07-14T18:14:51.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.574+0000 7f60e41ff700 1 -- start start 2023-07-14T18:14:51.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.574+0000 7f60e41ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60dc0ffc20 0x7f60dc197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:51.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.574+0000 7f60e41ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60dc100e10 0x7f60dc1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:51.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.574+0000 7f60e41ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60dc1017a0 0x7f60dc19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:51.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.574+0000 7f60e41ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc19f6e0 con 0x7f60dc100e10 2023-07-14T18:14:51.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.574+0000 7f60e41ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc19f820 con 0x7f60dc0ffc20 2023-07-14T18:14:51.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.574+0000 7f60e41ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc19f9b0 con 0x7f60dc1017a0 2023-07-14T18:14:51.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.574+0000 7f60e179a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60dc100e10 0x7f60dc1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:51.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.574+0000 7f60e179a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60dc100e10 0x7f60dc1980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39336/0 (socket says 172.21.15.133:39336) 2023-07-14T18:14:51.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.574+0000 7f60e179a700 1 -- 172.21.15.133:0/67080708 learned_addr learned my addr 172.21.15.133:0/67080708 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:51.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.574+0000 7f60e1f9b700 1 --2- 172.21.15.133:0/67080708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60dc0ffc20 0x7f60dc197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:51.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.574+0000 7f60e279c700 1 --2- 172.21.15.133:0/67080708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60dc1017a0 0x7f60dc19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:51.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.575+0000 7f60e179a700 1 -- 172.21.15.133:0/67080708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60dc1017a0 msgr2=0x7f60dc19f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:51.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.575+0000 7f60e179a700 1 --2- 172.21.15.133:0/67080708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60dc1017a0 0x7f60dc19f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:51.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.575+0000 7f60e179a700 1 -- 172.21.15.133:0/67080708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60dc0ffc20 msgr2=0x7f60dc197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:51.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.575+0000 7f60e179a700 1 --2- 172.21.15.133:0/67080708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60dc0ffc20 0x7f60dc197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:51.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.575+0000 7f60e179a700 1 -- 172.21.15.133:0/67080708 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d80095f0 con 0x7f60dc100e10 2023-07-14T18:14:51.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.575+0000 7f60e1f9b700 1 --2- 172.21.15.133:0/67080708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60dc0ffc20 0x7f60dc197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:14:51.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.575+0000 7f60e179a700 1 --2- 172.21.15.133:0/67080708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60dc100e10 0x7f60dc1980a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00e9a0 tx=0x7f60cc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:51.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.575+0000 7f60d2ffd700 1 -- 172.21.15.133:0/67080708 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc009790 con 0x7f60dc100e10 2023-07-14T18:14:51.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.575+0000 7f60d2ffd700 1 -- 172.21.15.133:0/67080708 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60cc010430 con 0x7f60dc100e10 2023-07-14T18:14:51.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.575+0000 7f60e41ff700 1 -- 172.21.15.133:0/67080708 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60dc19fc90 con 0x7f60dc100e10 2023-07-14T18:14:51.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.575+0000 7f60d2ffd700 1 -- 172.21.15.133:0/67080708 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc018680 con 0x7f60dc100e10 2023-07-14T18:14:51.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.575+0000 7f60e41ff700 1 -- 172.21.15.133:0/67080708 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60dc1a01a0 con 0x7f60dc100e10 2023-07-14T18:14:51.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.576+0000 7f60e41ff700 1 -- 172.21.15.133:0/67080708 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60dc10b3f0 con 0x7f60dc100e10 2023-07-14T18:14:51.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.578+0000 7f60d2ffd700 1 -- 172.21.15.133:0/67080708 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f60cc0105a0 con 0x7f60dc100e10 2023-07-14T18:14:51.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.580+0000 7f60d2ffd700 1 --2- 172.21.15.133:0/67080708 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f60c80a11d0 0x7f60c80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:51.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.580+0000 7f60d2ffd700 1 -- 172.21.15.133:0/67080708 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f60cc014070 con 0x7f60dc100e10 2023-07-14T18:14:51.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.580+0000 7f60e1f9b700 1 --2- 172.21.15.133:0/67080708 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f60c80a11d0 0x7f60c80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:51.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.580+0000 7f60e1f9b700 1 --2- 172.21.15.133:0/67080708 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f60c80a11d0 0x7f60c80a3680 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f60d800b2e0 tx=0x7f60d8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:51.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.583+0000 7f60d2ffd700 1 -- 172.21.15.133:0/67080708 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f60cc051780 con 0x7f60dc100e10 2023-07-14T18:14:51.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.813+0000 7f60e41ff700 1 -- 172.21.15.133:0/67080708 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f60dc107760 con 0x7f60c80a11d0 2023-07-14T18:14:51.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.817+0000 7f60d2ffd700 1 -- 172.21.15.133:0/67080708 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f60dc107760 con 0x7f60c80a11d0 2023-07-14T18:14:51.861 INFO:teuthology.orchestra.run.smithi133.stdout:/volumes/_nogroup/sv_1/63cee60f-94c5-4324-af48-1f2bdf1c218e 2023-07-14T18:14:51.863 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.819+0000 7f60e41ff700 1 -- 172.21.15.133:0/67080708 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f60c80a11d0 msgr2=0x7f60c80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:51.863 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.819+0000 7f60e41ff700 1 --2- 172.21.15.133:0/67080708 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f60c80a11d0 0x7f60c80a3680 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f60d800b2e0 tx=0x7f60d8005960 comp rx=0 tx=0).stop 2023-07-14T18:14:51.863 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.819+0000 7f60e41ff700 1 -- 172.21.15.133:0/67080708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60dc100e10 msgr2=0x7f60dc1980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:51.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.819+0000 7f60e41ff700 1 --2- 172.21.15.133:0/67080708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60dc100e10 0x7f60dc1980a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00e9a0 tx=0x7f60cc00ecb0 comp rx=0 tx=0).stop 2023-07-14T18:14:51.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.820+0000 7f60e41ff700 1 -- 172.21.15.133:0/67080708 shutdown_connections 2023-07-14T18:14:51.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.820+0000 7f60e41ff700 1 --2- 172.21.15.133:0/67080708 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f60c80a11d0 0x7f60c80a3680 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:51.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.820+0000 7f60e41ff700 1 --2- 172.21.15.133:0/67080708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60dc0ffc20 0x7f60dc197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:51.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.820+0000 7f60e41ff700 1 --2- 172.21.15.133:0/67080708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60dc1017a0 0x7f60dc19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:51.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.820+0000 7f60e41ff700 1 --2- 172.21.15.133:0/67080708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60dc100e10 0x7f60dc1980a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:51.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.820+0000 7f60e41ff700 1 -- 172.21.15.133:0/67080708 >> 172.21.15.133:0/67080708 conn(0x7f60dc0fb260 msgr2=0x7f60dc0fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:51.865 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.820+0000 7f60e41ff700 1 -- 172.21.15.133:0/67080708 shutdown_connections 2023-07-14T18:14:51.865 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:51.820+0000 7f60e41ff700 1 -- 172.21.15.133:0/67080708 wait complete. 2023-07-14T18:14:51.882 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T18:14:52.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.046+0000 7f62e47ff700 1 -- 172.21.15.133:0/2616906430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62dc100e20 msgr2=0x7f62dc101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:52.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.046+0000 7f62e47ff700 1 --2- 172.21.15.133:0/2616906430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62dc100e20 0x7f62dc101270 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009990 tx=0x7f62cc009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.047+0000 7f62e47ff700 1 -- 172.21.15.133:0/2616906430 shutdown_connections 2023-07-14T18:14:52.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.047+0000 7f62e47ff700 1 --2- 172.21.15.133:0/2616906430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62dc0ffc30 0x7f62dc100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.047+0000 7f62e47ff700 1 --2- 172.21.15.133:0/2616906430 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62dc1017b0 0x7f62dc108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.047+0000 7f62e47ff700 1 --2- 172.21.15.133:0/2616906430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62dc100e20 0x7f62dc101270 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.047+0000 7f62e47ff700 1 -- 172.21.15.133:0/2616906430 >> 172.21.15.133:0/2616906430 conn(0x7f62dc0fb210 msgr2=0x7f62dc0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:52.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.047+0000 7f62e47ff700 1 -- 172.21.15.133:0/2616906430 shutdown_connections 2023-07-14T18:14:52.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.047+0000 7f62e47ff700 1 -- 172.21.15.133:0/2616906430 wait complete. 2023-07-14T18:14:52.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.048+0000 7f62e47ff700 1 Processor -- start 2023-07-14T18:14:52.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.048+0000 7f62e47ff700 1 -- start start 2023-07-14T18:14:52.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.048+0000 7f62e47ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62dc0ffc30 0x7f62dc195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:52.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.048+0000 7f62e47ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62dc100e20 0x7f62dc195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:52.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.048+0000 7f62e47ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62dc1017b0 0x7f62dc19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:52.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.048+0000 7f62e47ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc19d4b0 con 0x7f62dc1017b0 2023-07-14T18:14:52.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.048+0000 7f62e47ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc19d5f0 con 0x7f62dc100e20 2023-07-14T18:14:52.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.048+0000 7f62e47ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc19d780 con 0x7f62dc0ffc30 2023-07-14T18:14:52.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.048+0000 7f62e259b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62dc0ffc30 0x7f62dc195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:52.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.048+0000 7f62e1d9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62dc100e20 0x7f62dc195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:52.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.048+0000 7f62e1d9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62dc100e20 0x7f62dc195e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:34096/0 (socket says 172.21.15.133:34096) 2023-07-14T18:14:52.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.048+0000 7f62e1d9a700 1 -- 172.21.15.133:0/4139464148 learned_addr learned my addr 172.21.15.133:0/4139464148 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:52.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.049+0000 7f62e2d9c700 1 --2- 172.21.15.133:0/4139464148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62dc1017b0 0x7f62dc19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:52.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.049+0000 7f62e259b700 1 -- 172.21.15.133:0/4139464148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62dc100e20 msgr2=0x7f62dc195e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:52.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.049+0000 7f62e259b700 1 --2- 172.21.15.133:0/4139464148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62dc100e20 0x7f62dc195e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.049+0000 7f62e259b700 1 -- 172.21.15.133:0/4139464148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62dc1017b0 msgr2=0x7f62dc19cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:52.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.049+0000 7f62e259b700 1 --2- 172.21.15.133:0/4139464148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62dc1017b0 0x7f62dc19cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.049+0000 7f62e259b700 1 -- 172.21.15.133:0/4139464148 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc0095f0 con 0x7f62dc0ffc30 2023-07-14T18:14:52.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.049+0000 7f62e1d9a700 1 --2- 172.21.15.133:0/4139464148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62dc100e20 0x7f62dc195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:14:52.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.049+0000 7f62e2d9c700 1 --2- 172.21.15.133:0/4139464148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62dc1017b0 0x7f62dc19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:14:52.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.049+0000 7f62e259b700 1 --2- 172.21.15.133:0/4139464148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62dc0ffc30 0x7f62dc195930 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f62d800e9c0 tx=0x7f62d800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:52.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.049+0000 7f62d37fe700 1 -- 172.21.15.133:0/4139464148 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d8004220 con 0x7f62dc0ffc30 2023-07-14T18:14:52.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.049+0000 7f62d37fe700 1 -- 172.21.15.133:0/4139464148 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62d8004380 con 0x7f62dc0ffc30 2023-07-14T18:14:52.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.049+0000 7f62e47ff700 1 -- 172.21.15.133:0/4139464148 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62dc19da60 con 0x7f62dc0ffc30 2023-07-14T18:14:52.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.049+0000 7f62d37fe700 1 -- 172.21.15.133:0/4139464148 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d8003be0 con 0x7f62dc0ffc30 2023-07-14T18:14:52.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.050+0000 7f62e47ff700 1 -- 172.21.15.133:0/4139464148 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62dc19df70 con 0x7f62dc0ffc30 2023-07-14T18:14:52.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.050+0000 7f62e47ff700 1 -- 172.21.15.133:0/4139464148 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62dc10b450 con 0x7f62dc0ffc30 2023-07-14T18:14:52.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.052+0000 7f62d37fe700 1 -- 172.21.15.133:0/4139464148 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f62d8003d40 con 0x7f62dc0ffc30 2023-07-14T18:14:52.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.053+0000 7f62d37fe700 1 --2- 172.21.15.133:0/4139464148 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f62c80a1220 0x7f62c80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:52.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.054+0000 7f62d37fe700 1 -- 172.21.15.133:0/4139464148 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f62d8014070 con 0x7f62dc0ffc30 2023-07-14T18:14:52.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.054+0000 7f62e1d9a700 1 --2- 172.21.15.133:0/4139464148 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f62c80a1220 0x7f62c80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:52.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.054+0000 7f62e1d9a700 1 --2- 172.21.15.133:0/4139464148 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f62c80a1220 0x7f62c80a36d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00b2e0 tx=0x7f62cc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:52.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.056+0000 7f62d37fe700 1 -- 172.21.15.133:0/4139464148 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f62d80969a0 con 0x7f62dc0ffc30 2023-07-14T18:14:52.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:51 smithi133 ceph-mon[126384]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:14:52.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:51 smithi133 ceph-mon[126384]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:14:52.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.293+0000 7f62e47ff700 1 -- 172.21.15.133:0/4139464148 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62dc106a90 con 0x7f62dc0ffc30 2023-07-14T18:14:52.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.294+0000 7f62d37fe700 1 -- 172.21.15.133:0/4139464148 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10851 (secure 0 0 0) 0x7f62d801f340 con 0x7f62dc0ffc30 2023-07-14T18:14:52.337 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:14:52.338 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3168151271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3168151271},{"type":"v1","addr":"172.21.15.133:6837","nonce":3168151271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14544,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/3093144261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3093144261},{"type":"v1","addr":"172.21.15.133:6839","nonce":3093144261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14556,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/3282902068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":3282902068},{"type":"v1","addr":"172.21.15.133:6841","nonce":3282902068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/4023127259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4023127259},{"type":"v1","addr":"172.21.15.161:6833","nonce":4023127259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2722373740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2722373740},{"type":"v1","addr":"172.21.15.171:6833","nonce":2722373740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2399005074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2399005074},{"type":"v1","addr":"172.21.15.133:6835","nonce":2399005074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T18:14:17.440873+0000","modified":"2023-07-14T18:14:50.886364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24557,"mds_1":24541,"mds_2":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3725502564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3725502564},{"type":"v1","addr":"172.21.15.161:6835","nonce":3725502564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6835/1877362116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1877362116},{"type":"v1","addr":"172.21.15.171:6835","nonce":1877362116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/3458064798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3458064798},{"type":"v1","addr":"172.21.15.161:6837","nonce":3458064798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/106205332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":106205332},{"type":"v1","addr":"172.21.15.171:6837","nonce":106205332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.171:6839/816201198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":816201198},{"type":"v1","addr":"172.21.15.171:6839","nonce":816201198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/2503865871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2503865871},{"type":"v1","addr":"172.21.15.161:6839","nonce":2503865871}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T18:14:52.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.296+0000 7f62e47ff700 1 -- 172.21.15.133:0/4139464148 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f62c80a1220 msgr2=0x7f62c80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:52.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.296+0000 7f62e47ff700 1 --2- 172.21.15.133:0/4139464148 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f62c80a1220 0x7f62c80a36d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00b2e0 tx=0x7f62cc005fb0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.296+0000 7f62e47ff700 1 -- 172.21.15.133:0/4139464148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62dc0ffc30 msgr2=0x7f62dc195930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:52.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.296+0000 7f62e47ff700 1 --2- 172.21.15.133:0/4139464148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62dc0ffc30 0x7f62dc195930 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f62d800e9c0 tx=0x7f62d800ecd0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.296+0000 7f62e47ff700 1 -- 172.21.15.133:0/4139464148 shutdown_connections 2023-07-14T18:14:52.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.296+0000 7f62e47ff700 1 --2- 172.21.15.133:0/4139464148 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f62c80a1220 0x7f62c80a36d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.296+0000 7f62e47ff700 1 --2- 172.21.15.133:0/4139464148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62dc1017b0 0x7f62dc19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.296+0000 7f62e47ff700 1 --2- 172.21.15.133:0/4139464148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62dc0ffc30 0x7f62dc195930 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.296+0000 7f62e47ff700 1 --2- 172.21.15.133:0/4139464148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62dc100e20 0x7f62dc195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.296+0000 7f62e47ff700 1 -- 172.21.15.133:0/4139464148 >> 172.21.15.133:0/4139464148 conn(0x7f62dc0fb210 msgr2=0x7f62dc0fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:52.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.296+0000 7f62e47ff700 1 -- 172.21.15.133:0/4139464148 shutdown_connections 2023-07-14T18:14:52.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.296+0000 7f62e47ff700 1 -- 172.21.15.133:0/4139464148 wait complete. 2023-07-14T18:14:52.342 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 25 2023-07-14T18:14:52.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:51 smithi161 ceph-mon[128993]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:14:52.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:51 smithi161 ceph-mon[128993]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:14:52.359 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T18:14:52.364 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:51 smithi171 ceph-mon[129241]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:14:52.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:51 smithi171 ceph-mon[129241]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:14:52.556 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.512+0000 7fe29cdff700 1 -- 172.21.15.133:0/283072288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2980ffc30 msgr2=0x7fe298100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:52.556 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.512+0000 7fe29cdff700 1 --2- 172.21.15.133:0/283072288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2980ffc30 0x7fe298100040 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe288009990 tx=0x7fe288009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.512+0000 7fe29cdff700 1 -- 172.21.15.133:0/283072288 shutdown_connections 2023-07-14T18:14:52.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.512+0000 7fe29cdff700 1 --2- 172.21.15.133:0/283072288 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2981017b0 0x7fe298108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.512+0000 7fe29cdff700 1 --2- 172.21.15.133:0/283072288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2980ffc30 0x7fe298100040 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.512+0000 7fe29cdff700 1 --2- 172.21.15.133:0/283072288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe298100e20 0x7fe298101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.513+0000 7fe29cdff700 1 -- 172.21.15.133:0/283072288 >> 172.21.15.133:0/283072288 conn(0x7fe2980fb210 msgr2=0x7fe2980fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:52.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.513+0000 7fe29cdff700 1 -- 172.21.15.133:0/283072288 shutdown_connections 2023-07-14T18:14:52.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.513+0000 7fe29cdff700 1 -- 172.21.15.133:0/283072288 wait complete. 2023-07-14T18:14:52.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.514+0000 7fe29cdff700 1 Processor -- start 2023-07-14T18:14:52.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.514+0000 7fe29cdff700 1 -- start start 2023-07-14T18:14:52.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.514+0000 7fe29cdff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2980ffc30 0x7fe2980743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:52.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.514+0000 7fe29cdff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe298100e20 0x7fe298072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:52.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.514+0000 7fe29cdff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2981017b0 0x7fe298072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:52.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.514+0000 7fe29cdff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe298073710 con 0x7fe2980ffc30 2023-07-14T18:14:52.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.514+0000 7fe29cdff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe298073880 con 0x7fe298100e20 2023-07-14T18:14:52.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.514+0000 7fe29cdff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29819f9d0 con 0x7fe2981017b0 2023-07-14T18:14:52.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.514+0000 7fe29659c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2980ffc30 0x7fe2980743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:52.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.514+0000 7fe296d9d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2981017b0 0x7fe298072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:52.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.514+0000 7fe295d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe298100e20 0x7fe298072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:52.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.514+0000 7fe295d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe298100e20 0x7fe298072a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:34126/0 (socket says 172.21.15.133:34126) 2023-07-14T18:14:52.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.514+0000 7fe29659c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2980ffc30 0x7fe2980743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39372/0 (socket says 172.21.15.133:39372) 2023-07-14T18:14:52.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.514+0000 7fe296d9d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2981017b0 0x7fe298072f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:60332/0 (socket says 172.21.15.133:60332) 2023-07-14T18:14:52.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.514+0000 7fe295d9b700 1 -- 172.21.15.133:0/3341175425 learned_addr learned my addr 172.21.15.133:0/3341175425 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:52.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.515+0000 7fe296d9d700 1 -- 172.21.15.133:0/3341175425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe298100e20 msgr2=0x7fe298072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:52.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.515+0000 7fe296d9d700 1 --2- 172.21.15.133:0/3341175425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe298100e20 0x7fe298072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.515+0000 7fe296d9d700 1 -- 172.21.15.133:0/3341175425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2980ffc30 msgr2=0x7fe2980743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:52.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.515+0000 7fe296d9d700 1 --2- 172.21.15.133:0/3341175425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2980ffc30 0x7fe2980743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.515+0000 7fe296d9d700 1 -- 172.21.15.133:0/3341175425 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2880095f0 con 0x7fe2981017b0 2023-07-14T18:14:52.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.515+0000 7fe296d9d700 1 --2- 172.21.15.133:0/3341175425 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2981017b0 0x7fe298072f70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe29000efc0 tx=0x7fe29000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:52.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.515+0000 7fe2877fe700 1 -- 172.21.15.133:0/3341175425 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe290018070 con 0x7fe2981017b0 2023-07-14T18:14:52.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.515+0000 7fe2877fe700 1 -- 172.21.15.133:0/3341175425 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe29000f040 con 0x7fe2981017b0 2023-07-14T18:14:52.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.515+0000 7fe29cdff700 1 -- 172.21.15.133:0/3341175425 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29819fc50 con 0x7fe2981017b0 2023-07-14T18:14:52.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.515+0000 7fe2877fe700 1 -- 172.21.15.133:0/3341175425 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29000cd40 con 0x7fe2981017b0 2023-07-14T18:14:52.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.515+0000 7fe29cdff700 1 -- 172.21.15.133:0/3341175425 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2981a0160 con 0x7fe2981017b0 2023-07-14T18:14:52.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.516+0000 7fe29cdff700 1 -- 172.21.15.133:0/3341175425 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe29810b450 con 0x7fe2981017b0 2023-07-14T18:14:52.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.518+0000 7fe2877fe700 1 -- 172.21.15.133:0/3341175425 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fe290003a00 con 0x7fe2981017b0 2023-07-14T18:14:52.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.519+0000 7fe2877fe700 1 --2- 172.21.15.133:0/3341175425 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe2800a12a0 0x7fe2800a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:52.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.520+0000 7fe2877fe700 1 -- 172.21.15.133:0/3341175425 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe2900cca70 con 0x7fe2981017b0 2023-07-14T18:14:52.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.520+0000 7fe29659c700 1 --2- 172.21.15.133:0/3341175425 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe2800a12a0 0x7fe2800a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:52.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.520+0000 7fe29659c700 1 --2- 172.21.15.133:0/3341175425 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe2800a12a0 0x7fe2800a3750 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe28800b2e0 tx=0x7fe288005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:52.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.521+0000 7fe2877fe700 1 -- 172.21.15.133:0/3341175425 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fe290095d80 con 0x7fe2981017b0 2023-07-14T18:14:52.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.721+0000 7fe29cdff700 1 -- 172.21.15.133:0/3341175425 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe298106ad0 con 0x7fe2981017b0 2023-07-14T18:14:52.767 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.724+0000 7fe2877fe700 1 -- 172.21.15.133:0/3341175425 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24473 (secure 0 0 0) 0x7fe29004dfb0 con 0x7fe2981017b0 2023-07-14T18:14:52.768 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:14:52.769 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":90,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","created":"2023-07-14T18:06:49.561582+0000","modified":"2023-07-14T18:14:48.934538+0000","last_up_change":"2023-07-14T18:13:29.836099+0000","last_in_change":"2023-07-14T18:13:11.606710+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T18:10:03.945039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T18:14:15.451340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T18:14:16.892215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T18:14:20.493175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"85a29de7-d712-422a-b647-d9e90091aaf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6803","nonce":3076510801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6805","nonce":3076510801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6809","nonce":3076510801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6807","nonce":3076510801}]},"public_addr":"172.21.15.133:6803/3076510801","cluster_addr":"172.21.15.133:6805/3076510801","heartbeat_back_addr":"172.21.15.133:6809/3076510801","heartbeat_front_addr":"172.21.15.133:6807/3076510801","state":["exists","up"]},{"osd":1,"uuid":"80ad4e78-7917-495e-b119-99f7c7c36243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6801","nonce":1841901852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6803","nonce":1841901852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6807","nonce":1841901852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6805","nonce":1841901852}]},"public_addr":"172.21.15.161:6801/1841901852","cluster_addr":"172.21.15.161:6803/1841901852","heartbeat_back_addr":"172.21.15.161:6807/1841901852","heartbeat_front_addr":"172.21.15.161:6805/1841901852","state":["exists","up"]},{"osd":2,"uuid":"fc4bfd37-1fdd-47c6-a48e-6d910daf9e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6801","nonce":346103099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6803","nonce":346103099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6807","nonce":346103099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6805","nonce":346103099}]},"public_addr":"172.21.15.171:6801/346103099","cluster_addr":"172.21.15.171:6803/346103099","heartbeat_back_addr":"172.21.15.171:6807/346103099","heartbeat_front_addr":"172.21.15.171:6805/346103099","state":["exists","up"]},{"osd":3,"uuid":"4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6811","nonce":2152994143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6813","nonce":2152994143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6817","nonce":2152994143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6815","nonce":2152994143}]},"public_addr":"172.21.15.133:6811/2152994143","cluster_addr":"172.21.15.133:6813/2152994143","heartbeat_back_addr":"172.21.15.133:6817/2152994143","heartbeat_front_addr":"172.21.15.133:6815/2152994143","state":["exists","up"]},{"osd":4,"uuid":"4601a1c2-254b-4c4d-9b15-8fc952b3d2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6809","nonce":47213291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6811","nonce":47213291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6815","nonce":47213291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6813","nonce":47213291}]},"public_addr":"172.21.15.161:6809/47213291","cluster_addr":"172.21.15.161:6811/47213291","heartbeat_back_addr":"172.21.15.161:6815/47213291","heartbeat_front_addr":"172.21.15.161:6813/47213291","state":["exists","up"]},{"osd":5,"uuid":"6cca180a-9140-4e49-835b-86842dae0574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6809","nonce":777556751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6811","nonce":777556751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6815","nonce":777556751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6813","nonce":777556751}]},"public_addr":"172.21.15.171:6809/777556751","cluster_addr":"172.21.15.171:6811/777556751","heartbeat_back_addr":"172.21.15.171:6815/777556751","heartbeat_front_addr":"172.21.15.171:6813/777556751","state":["exists","up"]},{"osd":6,"uuid":"ea0299ff-7740-4494-a3a9-1503b3fc0ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6819","nonce":2884152358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6821","nonce":2884152358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6825","nonce":2884152358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6823","nonce":2884152358}]},"public_addr":"172.21.15.133:6819/2884152358","cluster_addr":"172.21.15.133:6821/2884152358","heartbeat_back_addr":"172.21.15.133:6825/2884152358","heartbeat_front_addr":"172.21.15.133:6823/2884152358","state":["exists","up"]},{"osd":7,"uuid":"b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6817","nonce":4183705749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6819","nonce":4183705749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6823","nonce":4183705749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6821","nonce":4183705749}]},"public_addr":"172.21.15.161:6817/4183705749","cluster_addr":"172.21.15.161:6819/4183705749","heartbeat_back_addr":"172.21.15.161:6823/4183705749","heartbeat_front_addr":"172.21.15.161:6821/4183705749","state":["exists","up"]},{"osd":8,"uuid":"87dffc07-bf48-48c8-b418-5c4b509686d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6817","nonce":2771494483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6819","nonce":2771494483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6823","nonce":2771494483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6821","nonce":2771494483}]},"public_addr":"172.21.15.171:6817/2771494483","cluster_addr":"172.21.15.171:6819/2771494483","heartbeat_back_addr":"172.21.15.171:6823/2771494483","heartbeat_front_addr":"172.21.15.171:6821/2771494483","state":["exists","up"]},{"osd":9,"uuid":"e0832870-79ce-4bce-8045-58b7a7f6dc69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6827","nonce":2066164149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6829","nonce":2066164149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6833","nonce":2066164149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6831","nonce":2066164149}]},"public_addr":"172.21.15.133:6827/2066164149","cluster_addr":"172.21.15.133:6829/2066164149","heartbeat_back_addr":"172.21.15.133:6833/2066164149","heartbeat_front_addr":"172.21.15.133:6831/2066164149","state":["exists","up"]},{"osd":10,"uuid":"081f0484-d134-4b2e-9c48-ef8e6a646422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6825","nonce":2162333626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6827","nonce":2162333626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6831","nonce":2162333626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6829","nonce":2162333626}]},"public_addr":"172.21.15.161:6825/2162333626","cluster_addr":"172.21.15.161:6827/2162333626","heartbeat_back_addr":"172.21.15.161:6831/2162333626","heartbeat_front_addr":"172.21.15.161:6829/2162333626","state":["exists","up"]},{"osd":11,"uuid":"4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6825","nonce":3403894313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6827","nonce":3403894313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6831","nonce":3403894313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6829","nonce":3403894313}]},"public_addr":"172.21.15.171:6825/3403894313","cluster_addr":"172.21.15.171:6827/3403894313","heartbeat_back_addr":"172.21.15.171:6831/3403894313","heartbeat_front_addr":"172.21.15.171:6829/3403894313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:17.816229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:39.638543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:00.543317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:26.988921+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:48.644694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:10.049860+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:35.986508+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:57.753846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:19.018245+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:44.460450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:06.279960+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:27.955006+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/1832569241":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6801/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:0/4219815304":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:6801/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/432253615":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1840700480":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1807775439":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1089030734":"2023-07-15T18:07:47.346704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T18:14:52.770 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.726+0000 7fe29cdff700 1 -- 172.21.15.133:0/3341175425 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe2800a12a0 msgr2=0x7fe2800a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:52.770 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.726+0000 7fe29cdff700 1 --2- 172.21.15.133:0/3341175425 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe2800a12a0 0x7fe2800a3750 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe28800b2e0 tx=0x7fe288005fb0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.770 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.726+0000 7fe29cdff700 1 -- 172.21.15.133:0/3341175425 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2981017b0 msgr2=0x7fe298072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:52.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.726+0000 7fe29cdff700 1 --2- 172.21.15.133:0/3341175425 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2981017b0 0x7fe298072f70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe29000efc0 tx=0x7fe29000c3a0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.726+0000 7fe29cdff700 1 -- 172.21.15.133:0/3341175425 shutdown_connections 2023-07-14T18:14:52.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.726+0000 7fe29cdff700 1 --2- 172.21.15.133:0/3341175425 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe2800a12a0 0x7fe2800a3750 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.726+0000 7fe29cdff700 1 --2- 172.21.15.133:0/3341175425 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2981017b0 0x7fe298072f70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.726+0000 7fe29cdff700 1 --2- 172.21.15.133:0/3341175425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2980ffc30 0x7fe2980743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.726+0000 7fe29cdff700 1 --2- 172.21.15.133:0/3341175425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe298100e20 0x7fe298072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.726+0000 7fe29cdff700 1 -- 172.21.15.133:0/3341175425 >> 172.21.15.133:0/3341175425 conn(0x7fe2980fb210 msgr2=0x7fe2980fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:52.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.727+0000 7fe29cdff700 1 -- 172.21.15.133:0/3341175425 shutdown_connections 2023-07-14T18:14:52.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.727+0000 7fe29cdff700 1 -- 172.21.15.133:0/3341175425 wait complete. 2023-07-14T18:14:52.789 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-14T18:14:52.790 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T18:14:52.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.943+0000 7ff8263ff700 1 -- 172.21.15.133:0/526734523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8200ffc30 msgr2=0x7ff820100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:52.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.943+0000 7ff8263ff700 1 --2- 172.21.15.133:0/526734523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8200ffc30 0x7ff820100040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff810009990 tx=0x7ff810009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.943+0000 7ff8263ff700 1 -- 172.21.15.133:0/526734523 shutdown_connections 2023-07-14T18:14:52.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.943+0000 7ff8263ff700 1 --2- 172.21.15.133:0/526734523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff8201017b0 0x7ff820108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.943+0000 7ff8263ff700 1 --2- 172.21.15.133:0/526734523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8200ffc30 0x7ff820100040 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.943+0000 7ff8263ff700 1 --2- 172.21.15.133:0/526734523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff820100e20 0x7ff820101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.943+0000 7ff8263ff700 1 -- 172.21.15.133:0/526734523 >> 172.21.15.133:0/526734523 conn(0x7ff8200fb210 msgr2=0x7ff8200fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:52.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.944+0000 7ff8263ff700 1 -- 172.21.15.133:0/526734523 shutdown_connections 2023-07-14T18:14:52.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.944+0000 7ff8263ff700 1 -- 172.21.15.133:0/526734523 wait complete. 2023-07-14T18:14:52.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.944+0000 7ff8263ff700 1 Processor -- start 2023-07-14T18:14:52.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.945+0000 7ff8263ff700 1 -- start start 2023-07-14T18:14:52.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.945+0000 7ff8263ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8200ffc30 0x7ff8200743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:52.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.945+0000 7ff8263ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff820100e20 0x7ff820072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:52.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.945+0000 7ff8263ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8201017b0 0x7ff820072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.945+0000 7ff8263ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff820073710 con 0x7ff8201017b0 2023-07-14T18:14:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.945+0000 7ff8263ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff820073880 con 0x7ff8200ffc30 2023-07-14T18:14:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.945+0000 7ff8263ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82019f950 con 0x7ff820100e20 2023-07-14T18:14:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.945+0000 7ff82499c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8201017b0 0x7ff820072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.945+0000 7ff81f7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff820100e20 0x7ff820072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.945+0000 7ff81ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8200ffc30 0x7ff8200743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.945+0000 7ff82499c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8201017b0 0x7ff820072f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39392/0 (socket says 172.21.15.133:39392) 2023-07-14T18:14:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.945+0000 7ff82499c700 1 -- 172.21.15.133:0/348352429 learned_addr learned my addr 172.21.15.133:0/348352429 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.946+0000 7ff82499c700 1 -- 172.21.15.133:0/348352429 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff820100e20 msgr2=0x7ff820072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.946+0000 7ff82499c700 1 --2- 172.21.15.133:0/348352429 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff820100e20 0x7ff820072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.946+0000 7ff82499c700 1 -- 172.21.15.133:0/348352429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8200ffc30 msgr2=0x7ff8200743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:52.994 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.946+0000 7ff82499c700 1 --2- 172.21.15.133:0/348352429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8200ffc30 0x7ff8200743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:52.994 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.946+0000 7ff82499c700 1 -- 172.21.15.133:0/348352429 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8100095f0 con 0x7ff8201017b0 2023-07-14T18:14:52.994 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.946+0000 7ff81ffff700 1 --2- 172.21.15.133:0/348352429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8200ffc30 0x7ff8200743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:14:52.994 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.946+0000 7ff82499c700 1 --2- 172.21.15.133:0/348352429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8201017b0 0x7ff820072f70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff81800ef30 tx=0x7ff81800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:52.994 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.946+0000 7ff81d7fa700 1 -- 172.21.15.133:0/348352429 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff818018070 con 0x7ff8201017b0 2023-07-14T18:14:52.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.946+0000 7ff8263ff700 1 -- 172.21.15.133:0/348352429 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff82019fbd0 con 0x7ff8201017b0 2023-07-14T18:14:52.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.946+0000 7ff81d7fa700 1 -- 172.21.15.133:0/348352429 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff81800f040 con 0x7ff8201017b0 2023-07-14T18:14:52.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.946+0000 7ff81d7fa700 1 -- 172.21.15.133:0/348352429 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81800cd00 con 0x7ff8201017b0 2023-07-14T18:14:52.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.946+0000 7ff8263ff700 1 -- 172.21.15.133:0/348352429 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8201a00e0 con 0x7ff8201017b0 2023-07-14T18:14:52.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.947+0000 7ff8263ff700 1 -- 172.21.15.133:0/348352429 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff82010b400 con 0x7ff8201017b0 2023-07-14T18:14:52.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.949+0000 7ff81d7fa700 1 -- 172.21.15.133:0/348352429 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7ff818003a00 con 0x7ff8201017b0 2023-07-14T18:14:52.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.951+0000 7ff81d7fa700 1 --2- 172.21.15.133:0/348352429 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff8080a1220 0x7ff8080a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:52.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.952+0000 7ff81d7fa700 1 -- 172.21.15.133:0/348352429 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff8180cc960 con 0x7ff8201017b0 2023-07-14T18:14:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.952+0000 7ff81ffff700 1 --2- 172.21.15.133:0/348352429 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff8080a1220 0x7ff8080a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.952+0000 7ff81ffff700 1 --2- 172.21.15.133:0/348352429 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff8080a1220 0x7ff8080a36d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff81000b2e0 tx=0x7ff810005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:52.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:52.955+0000 7ff81d7fa700 1 -- 172.21.15.133:0/348352429 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7ff818095c70 con 0x7ff8201017b0 2023-07-14T18:14:53.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:52 smithi133 ceph-mon[126384]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:14:53.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:52 smithi133 ceph-mon[126384]: pgmap v292: 129 pgs: 3 peering, 126 active+clean; 582 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1.1 KiB/s wr, 3 op/s 2023-07-14T18:14:53.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:52 smithi133 ceph-mon[126384]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:14:53.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:52 smithi133 ceph-mon[126384]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-07-14T18:14:53.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:52 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/4139464148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:14:53.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:52 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3341175425' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:14:53.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.153+0000 7ff8263ff700 1 -- 172.21.15.133:0/348352429 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff820060a80 con 0x7ff8080a1220 2023-07-14T18:14:53.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.167+0000 7ff81d7fa700 1 -- 172.21.15.133:0/348352429 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422648 (secure 0 0 0) 0x7ff820060a80 con 0x7ff8080a1220 2023-07-14T18:14:53.216 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:14:53.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.172+0000 7ff8263ff700 1 -- 172.21.15.133:0/348352429 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff8080a1220 msgr2=0x7ff8080a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:53.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.172+0000 7ff8263ff700 1 --2- 172.21.15.133:0/348352429 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff8080a1220 0x7ff8080a36d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff81000b2e0 tx=0x7ff810005960 comp rx=0 tx=0).stop 2023-07-14T18:14:53.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.172+0000 7ff8263ff700 1 -- 172.21.15.133:0/348352429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8201017b0 msgr2=0x7ff820072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:53.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.172+0000 7ff8263ff700 1 --2- 172.21.15.133:0/348352429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8201017b0 0x7ff820072f70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff81800ef30 tx=0x7ff81800c3a0 comp rx=0 tx=0).stop 2023-07-14T18:14:53.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.172+0000 7ff8263ff700 1 -- 172.21.15.133:0/348352429 shutdown_connections 2023-07-14T18:14:53.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.172+0000 7ff8263ff700 1 --2- 172.21.15.133:0/348352429 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff8080a1220 0x7ff8080a36d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:53.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.172+0000 7ff8263ff700 1 --2- 172.21.15.133:0/348352429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8200ffc30 0x7ff8200743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:53.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.172+0000 7ff8263ff700 1 --2- 172.21.15.133:0/348352429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8201017b0 0x7ff820072f70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:53.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.172+0000 7ff8263ff700 1 --2- 172.21.15.133:0/348352429 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff820100e20 0x7ff820072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:53.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.172+0000 7ff8263ff700 1 -- 172.21.15.133:0/348352429 >> 172.21.15.133:0/348352429 conn(0x7ff8200fb210 msgr2=0x7ff8200fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:53.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.172+0000 7ff8263ff700 1 -- 172.21.15.133:0/348352429 shutdown_connections 2023-07-14T18:14:53.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.172+0000 7ff8263ff700 1 -- 172.21.15.133:0/348352429 wait complete. 2023-07-14T18:14:53.220 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2023-07-14T18:14:53.235 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2023-07-14T18:14:51.436610+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595624,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":133,"num_read_kb":115,"num_write":252,"num_write_kb":2966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":421,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335796,"kb_used_data":11544,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124606028,"statfs":{"total":1151940427776,"available":1151596572672,"internally_reserved":0,"allocated":11821056,"data_stored":7518900,"data_compressed":280140,"data_compressed_allocated":376832,"data_compressed_original":2252800,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":664,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":22,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002199"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945666+0000","last_change":"2023-07-14T18:14:17.927179+0000","last_active":"2023-07-14T18:14:48.945666+0000","last_peered":"2023-07-14T18:14:48.945666+0000","last_clean":"2023-07-14T18:14:48.945666+0000","last_became_active":"2023-07-14T18:14:17.926880+0000","last_became_peered":"2023-07-14T18:14:17.926880+0000","last_unstale":"2023-07-14T18:14:48.945666+0000","last_undegraded":"2023-07-14T18:14:48.945666+0000","last_fullsized":"2023-07-14T18:14:48.945666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:26:26.468709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541115+0000","last_change":"2023-07-14T18:14:17.933834+0000","last_active":"2023-07-14T18:14:49.541115+0000","last_peered":"2023-07-14T18:14:49.541115+0000","last_clean":"2023-07-14T18:14:49.541115+0000","last_became_active":"2023-07-14T18:14:17.933529+0000","last_became_peered":"2023-07-14T18:14:17.933529+0000","last_unstale":"2023-07-14T18:14:49.541115+0000","last_undegraded":"2023-07-14T18:14:49.541115+0000","last_fullsized":"2023-07-14T18:14:49.541115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:34:13.388837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540772+0000","last_change":"2023-07-14T18:14:17.928400+0000","last_active":"2023-07-14T18:14:49.540772+0000","last_peered":"2023-07-14T18:14:49.540772+0000","last_clean":"2023-07-14T18:14:49.540772+0000","last_became_active":"2023-07-14T18:14:17.928077+0000","last_became_peered":"2023-07-14T18:14:17.928077+0000","last_unstale":"2023-07-14T18:14:49.540772+0000","last_undegraded":"2023-07-14T18:14:49.540772+0000","last_fullsized":"2023-07-14T18:14:49.540772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:24.522613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.947528+0000","last_change":"2023-07-14T18:14:17.932623+0000","last_active":"2023-07-14T18:14:48.947528+0000","last_peered":"2023-07-14T18:14:48.947528+0000","last_clean":"2023-07-14T18:14:48.947528+0000","last_became_active":"2023-07-14T18:14:17.932298+0000","last_became_peered":"2023-07-14T18:14:17.932298+0000","last_unstale":"2023-07-14T18:14:48.947528+0000","last_undegraded":"2023-07-14T18:14:48.947528+0000","last_fullsized":"2023-07-14T18:14:48.947528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:17:46.564398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.042494+0000","last_change":"2023-07-14T18:14:17.927109+0000","last_active":"2023-07-14T18:14:23.042494+0000","last_peered":"2023-07-14T18:14:23.042494+0000","last_clean":"2023-07-14T18:14:23.042494+0000","last_became_active":"2023-07-14T18:14:17.926800+0000","last_became_peered":"2023-07-14T18:14:17.926800+0000","last_unstale":"2023-07-14T18:14:23.042494+0000","last_undegraded":"2023-07-14T18:14:23.042494+0000","last_fullsized":"2023-07-14T18:14:23.042494+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:41:23.319498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.043209+0000","last_change":"2023-07-14T18:14:17.935380+0000","last_active":"2023-07-14T18:14:23.043209+0000","last_peered":"2023-07-14T18:14:23.043209+0000","last_clean":"2023-07-14T18:14:23.043209+0000","last_became_active":"2023-07-14T18:14:17.935099+0000","last_became_peered":"2023-07-14T18:14:17.935099+0000","last_unstale":"2023-07-14T18:14:23.043209+0000","last_undegraded":"2023-07-14T18:14:23.043209+0000","last_fullsized":"2023-07-14T18:14:23.043209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:09:17.841473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.943679+0000","last_change":"2023-07-14T18:14:17.930207+0000","last_active":"2023-07-14T18:14:48.943679+0000","last_peered":"2023-07-14T18:14:48.943679+0000","last_clean":"2023-07-14T18:14:48.943679+0000","last_became_active":"2023-07-14T18:14:17.929898+0000","last_became_peered":"2023-07-14T18:14:17.929898+0000","last_unstale":"2023-07-14T18:14:48.943679+0000","last_undegraded":"2023-07-14T18:14:48.943679+0000","last_fullsized":"2023-07-14T18:14:48.943679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:33:33.679907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.936854+0000","last_change":"2023-07-14T18:14:17.928645+0000","last_active":"2023-07-14T18:14:47.936854+0000","last_peered":"2023-07-14T18:14:47.936854+0000","last_clean":"2023-07-14T18:14:47.936854+0000","last_became_active":"2023-07-14T18:14:17.928288+0000","last_became_peered":"2023-07-14T18:14:17.928288+0000","last_unstale":"2023-07-14T18:14:47.936854+0000","last_undegraded":"2023-07-14T18:14:47.936854+0000","last_fullsized":"2023-07-14T18:14:47.936854+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:09:18.003654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.948678+0000","last_change":"2023-07-14T18:14:17.925888+0000","last_active":"2023-07-14T18:14:48.948678+0000","last_peered":"2023-07-14T18:14:48.948678+0000","last_clean":"2023-07-14T18:14:48.948678+0000","last_became_active":"2023-07-14T18:14:17.925580+0000","last_became_peered":"2023-07-14T18:14:17.925580+0000","last_unstale":"2023-07-14T18:14:48.948678+0000","last_undegraded":"2023-07-14T18:14:48.948678+0000","last_fullsized":"2023-07-14T18:14:48.948678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:28:42.610213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.043961+0000","last_change":"2023-07-14T18:14:17.924440+0000","last_active":"2023-07-14T18:14:23.043961+0000","last_peered":"2023-07-14T18:14:23.043961+0000","last_clean":"2023-07-14T18:14:23.043961+0000","last_became_active":"2023-07-14T18:14:17.923839+0000","last_became_peered":"2023-07-14T18:14:17.923839+0000","last_unstale":"2023-07-14T18:14:23.043961+0000","last_undegraded":"2023-07-14T18:14:23.043961+0000","last_fullsized":"2023-07-14T18:14:23.043961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:05:37.167764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944134+0000","last_change":"2023-07-14T18:14:17.931067+0000","last_active":"2023-07-14T18:14:48.944134+0000","last_peered":"2023-07-14T18:14:48.944134+0000","last_clean":"2023-07-14T18:14:48.944134+0000","last_became_active":"2023-07-14T18:14:17.930633+0000","last_became_peered":"2023-07-14T18:14:17.930633+0000","last_unstale":"2023-07-14T18:14:48.944134+0000","last_undegraded":"2023-07-14T18:14:48.944134+0000","last_fullsized":"2023-07-14T18:14:48.944134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:35:03.441365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.947103+0000","last_change":"2023-07-14T18:14:17.932770+0000","last_active":"2023-07-14T18:14:48.947103+0000","last_peered":"2023-07-14T18:14:48.947103+0000","last_clean":"2023-07-14T18:14:48.947103+0000","last_became_active":"2023-07-14T18:14:17.932438+0000","last_became_peered":"2023-07-14T18:14:17.932438+0000","last_unstale":"2023-07-14T18:14:48.947103+0000","last_undegraded":"2023-07-14T18:14:48.947103+0000","last_fullsized":"2023-07-14T18:14:48.947103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:45:58.878384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.941615+0000","last_change":"2023-07-14T18:14:17.938213+0000","last_active":"2023-07-14T18:14:47.941615+0000","last_peered":"2023-07-14T18:14:47.941615+0000","last_clean":"2023-07-14T18:14:47.941615+0000","last_became_active":"2023-07-14T18:14:17.937565+0000","last_became_peered":"2023-07-14T18:14:17.937565+0000","last_unstale":"2023-07-14T18:14:47.941615+0000","last_undegraded":"2023-07-14T18:14:47.941615+0000","last_fullsized":"2023-07-14T18:14:47.941615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:02:12.833666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541023+0000","last_change":"2023-07-14T18:14:17.934677+0000","last_active":"2023-07-14T18:14:49.541023+0000","last_peered":"2023-07-14T18:14:49.541023+0000","last_clean":"2023-07-14T18:14:49.541023+0000","last_became_active":"2023-07-14T18:14:17.933622+0000","last_became_peered":"2023-07-14T18:14:17.933622+0000","last_unstale":"2023-07-14T18:14:49.541023+0000","last_undegraded":"2023-07-14T18:14:49.541023+0000","last_fullsized":"2023-07-14T18:14:49.541023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:44:58.693553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540682+0000","last_change":"2023-07-14T18:14:17.935317+0000","last_active":"2023-07-14T18:14:49.540682+0000","last_peered":"2023-07-14T18:14:49.540682+0000","last_clean":"2023-07-14T18:14:49.540682+0000","last_became_active":"2023-07-14T18:14:17.935034+0000","last_became_peered":"2023-07-14T18:14:17.935034+0000","last_unstale":"2023-07-14T18:14:49.540682+0000","last_undegraded":"2023-07-14T18:14:49.540682+0000","last_fullsized":"2023-07-14T18:14:49.540682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:45:23.409187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946950+0000","last_change":"2023-07-14T18:14:17.928222+0000","last_active":"2023-07-14T18:14:48.946950+0000","last_peered":"2023-07-14T18:14:48.946950+0000","last_clean":"2023-07-14T18:14:48.946950+0000","last_became_active":"2023-07-14T18:14:17.927964+0000","last_became_peered":"2023-07-14T18:14:17.927964+0000","last_unstale":"2023-07-14T18:14:48.946950+0000","last_undegraded":"2023-07-14T18:14:48.946950+0000","last_fullsized":"2023-07-14T18:14:48.946950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:44:57.639864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944940+0000","last_change":"2023-07-14T18:14:17.927276+0000","last_active":"2023-07-14T18:14:48.944940+0000","last_peered":"2023-07-14T18:14:48.944940+0000","last_clean":"2023-07-14T18:14:48.944940+0000","last_became_active":"2023-07-14T18:14:17.926952+0000","last_became_peered":"2023-07-14T18:14:17.926952+0000","last_unstale":"2023-07-14T18:14:48.944940+0000","last_undegraded":"2023-07-14T18:14:48.944940+0000","last_fullsized":"2023-07-14T18:14:48.944940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:48:53.419369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.044606+0000","last_change":"2023-07-14T18:14:17.937364+0000","last_active":"2023-07-14T18:14:23.044606+0000","last_peered":"2023-07-14T18:14:23.044606+0000","last_clean":"2023-07-14T18:14:23.044606+0000","last_became_active":"2023-07-14T18:14:17.937014+0000","last_became_peered":"2023-07-14T18:14:17.937014+0000","last_unstale":"2023-07-14T18:14:23.044606+0000","last_undegraded":"2023-07-14T18:14:23.044606+0000","last_fullsized":"2023-07-14T18:14:23.044606+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:28:49.560380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.043696+0000","last_change":"2023-07-14T18:14:17.920358+0000","last_active":"2023-07-14T18:14:23.043696+0000","last_peered":"2023-07-14T18:14:23.043696+0000","last_clean":"2023-07-14T18:14:23.043696+0000","last_became_active":"2023-07-14T18:14:17.920042+0000","last_became_peered":"2023-07-14T18:14:17.920042+0000","last_unstale":"2023-07-14T18:14:23.043696+0000","last_undegraded":"2023-07-14T18:14:23.043696+0000","last_fullsized":"2023-07-14T18:14:23.043696+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:23:16.585987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.042640+0000","last_change":"2023-07-14T18:14:22.045376+0000","last_active":"2023-07-14T18:14:23.042640+0000","last_peered":"2023-07-14T18:14:23.042640+0000","last_clean":"2023-07-14T18:14:23.042640+0000","last_became_active":"2023-07-14T18:14:22.044984+0000","last_became_peered":"2023-07-14T18:14:22.044984+0000","last_unstale":"2023-07-14T18:14:23.042640+0000","last_undegraded":"2023-07-14T18:14:23.042640+0000","last_fullsized":"2023-07-14T18:14:23.042640+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:02:18.566020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.044506+0000","last_change":"2023-07-14T18:14:16.927936+0000","last_active":"2023-07-14T18:14:23.044506+0000","last_peered":"2023-07-14T18:14:23.044506+0000","last_clean":"2023-07-14T18:14:23.044506+0000","last_became_active":"2023-07-14T18:14:16.927556+0000","last_became_peered":"2023-07-14T18:14:16.927556+0000","last_unstale":"2023-07-14T18:14:23.044506+0000","last_undegraded":"2023-07-14T18:14:23.044506+0000","last_fullsized":"2023-07-14T18:14:23.044506+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:21:35.559367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.938611+0000","last_change":"2023-07-14T18:14:17.925822+0000","last_active":"2023-07-14T18:14:47.938611+0000","last_peered":"2023-07-14T18:14:47.938611+0000","last_clean":"2023-07-14T18:14:47.938611+0000","last_became_active":"2023-07-14T18:14:17.925508+0000","last_became_peered":"2023-07-14T18:14:17.925508+0000","last_unstale":"2023-07-14T18:14:47.938611+0000","last_undegraded":"2023-07-14T18:14:47.938611+0000","last_fullsized":"2023-07-14T18:14:47.938611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:23:07.664586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946499+0000","last_change":"2023-07-14T18:14:22.043098+0000","last_active":"2023-07-14T18:14:48.946499+0000","last_peered":"2023-07-14T18:14:48.946499+0000","last_clean":"2023-07-14T18:14:48.946499+0000","last_became_active":"2023-07-14T18:14:22.042517+0000","last_became_peered":"2023-07-14T18:14:22.042517+0000","last_unstale":"2023-07-14T18:14:48.946499+0000","last_undegraded":"2023-07-14T18:14:48.946499+0000","last_fullsized":"2023-07-14T18:14:48.946499+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:33:54.491558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.938925+0000","last_change":"2023-07-14T18:14:16.927612+0000","last_active":"2023-07-14T18:14:47.938925+0000","last_peered":"2023-07-14T18:14:47.938925+0000","last_clean":"2023-07-14T18:14:47.938925+0000","last_became_active":"2023-07-14T18:14:16.927284+0000","last_became_peered":"2023-07-14T18:14:16.927284+0000","last_unstale":"2023-07-14T18:14:47.938925+0000","last_undegraded":"2023-07-14T18:14:47.938925+0000","last_fullsized":"2023-07-14T18:14:47.938925+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:33:50.082251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540962+0000","last_change":"2023-07-14T18:14:17.930624+0000","last_active":"2023-07-14T18:14:49.540962+0000","last_peered":"2023-07-14T18:14:49.540962+0000","last_clean":"2023-07-14T18:14:49.540962+0000","last_became_active":"2023-07-14T18:14:17.930273+0000","last_became_peered":"2023-07-14T18:14:17.930273+0000","last_unstale":"2023-07-14T18:14:49.540962+0000","last_undegraded":"2023-07-14T18:14:49.540962+0000","last_fullsized":"2023-07-14T18:14:49.540962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:19:40.573682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.543208+0000","last_change":"2023-07-14T18:14:49.543208+0000","last_active":"2023-07-14T18:14:49.543208+0000","last_peered":"2023-07-14T18:14:49.543208+0000","last_clean":"2023-07-14T18:14:49.543208+0000","last_became_active":"2023-07-14T18:14:49.542685+0000","last_became_peered":"2023-07-14T18:14:49.542685+0000","last_unstale":"2023-07-14T18:14:49.543208+0000","last_undegraded":"2023-07-14T18:14:49.543208+0000","last_fullsized":"2023-07-14T18:14:49.543208+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:32:12.710336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.043944+0000","last_change":"2023-07-14T18:14:16.926185+0000","last_active":"2023-07-14T18:14:23.043944+0000","last_peered":"2023-07-14T18:14:23.043944+0000","last_clean":"2023-07-14T18:14:23.043944+0000","last_became_active":"2023-07-14T18:14:16.925874+0000","last_became_peered":"2023-07-14T18:14:16.925874+0000","last_unstale":"2023-07-14T18:14:23.043944+0000","last_undegraded":"2023-07-14T18:14:23.043944+0000","last_fullsized":"2023-07-14T18:14:23.043944+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:25:25.661288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.947867+0000","last_change":"2023-07-14T18:14:17.936601+0000","last_active":"2023-07-14T18:14:48.947867+0000","last_peered":"2023-07-14T18:14:48.947867+0000","last_clean":"2023-07-14T18:14:48.947867+0000","last_became_active":"2023-07-14T18:14:17.936291+0000","last_became_peered":"2023-07-14T18:14:17.936291+0000","last_unstale":"2023-07-14T18:14:48.947867+0000","last_undegraded":"2023-07-14T18:14:48.947867+0000","last_fullsized":"2023-07-14T18:14:48.947867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:50:25.290580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946635+0000","last_change":"2023-07-14T18:14:22.042952+0000","last_active":"2023-07-14T18:14:48.946635+0000","last_peered":"2023-07-14T18:14:48.946635+0000","last_clean":"2023-07-14T18:14:48.946635+0000","last_became_active":"2023-07-14T18:14:22.042256+0000","last_became_peered":"2023-07-14T18:14:22.042256+0000","last_unstale":"2023-07-14T18:14:48.946635+0000","last_undegraded":"2023-07-14T18:14:48.946635+0000","last_fullsized":"2023-07-14T18:14:48.946635+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:20:29.914375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.042627+0000","last_change":"2023-07-14T18:14:16.920902+0000","last_active":"2023-07-14T18:14:23.042627+0000","last_peered":"2023-07-14T18:14:23.042627+0000","last_clean":"2023-07-14T18:14:23.042627+0000","last_became_active":"2023-07-14T18:14:16.920554+0000","last_became_peered":"2023-07-14T18:14:16.920554+0000","last_unstale":"2023-07-14T18:14:23.042627+0000","last_undegraded":"2023-07-14T18:14:23.042627+0000","last_fullsized":"2023-07-14T18:14:23.042627+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:46:09.916824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.043608+0000","last_change":"2023-07-14T18:14:17.925005+0000","last_active":"2023-07-14T18:14:23.043608+0000","last_peered":"2023-07-14T18:14:23.043608+0000","last_clean":"2023-07-14T18:14:23.043608+0000","last_became_active":"2023-07-14T18:14:17.924723+0000","last_became_peered":"2023-07-14T18:14:17.924723+0000","last_unstale":"2023-07-14T18:14:23.043608+0000","last_undegraded":"2023-07-14T18:14:23.043608+0000","last_fullsized":"2023-07-14T18:14:23.043608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:05:07.389626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.043424+0000","last_change":"2023-07-14T18:14:22.038482+0000","last_active":"2023-07-14T18:14:23.043424+0000","last_peered":"2023-07-14T18:14:23.043424+0000","last_clean":"2023-07-14T18:14:23.043424+0000","last_became_active":"2023-07-14T18:14:22.038059+0000","last_became_peered":"2023-07-14T18:14:22.038059+0000","last_unstale":"2023-07-14T18:14:23.043424+0000","last_undegraded":"2023-07-14T18:14:23.043424+0000","last_fullsized":"2023-07-14T18:14:23.043424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:52:35.222394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.044070+0000","last_change":"2023-07-14T18:14:16.924159+0000","last_active":"2023-07-14T18:14:23.044070+0000","last_peered":"2023-07-14T18:14:23.044070+0000","last_clean":"2023-07-14T18:14:23.044070+0000","last_became_active":"2023-07-14T18:14:16.923808+0000","last_became_peered":"2023-07-14T18:14:16.923808+0000","last_unstale":"2023-07-14T18:14:23.044070+0000","last_undegraded":"2023-07-14T18:14:23.044070+0000","last_fullsized":"2023-07-14T18:14:23.044070+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:17:32.885985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.043642+0000","last_change":"2023-07-14T18:14:17.934894+0000","last_active":"2023-07-14T18:14:23.043642+0000","last_peered":"2023-07-14T18:14:23.043642+0000","last_clean":"2023-07-14T18:14:23.043642+0000","last_became_active":"2023-07-14T18:14:17.934651+0000","last_became_peered":"2023-07-14T18:14:17.934651+0000","last_unstale":"2023-07-14T18:14:23.043642+0000","last_undegraded":"2023-07-14T18:14:23.043642+0000","last_fullsized":"2023-07-14T18:14:23.043642+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:11:05.980662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.042761+0000","last_change":"2023-07-14T18:14:22.043611+0000","last_active":"2023-07-14T18:14:23.042761+0000","last_peered":"2023-07-14T18:14:23.042761+0000","last_clean":"2023-07-14T18:14:23.042761+0000","last_became_active":"2023-07-14T18:14:22.043101+0000","last_became_peered":"2023-07-14T18:14:22.043101+0000","last_unstale":"2023-07-14T18:14:23.042761+0000","last_undegraded":"2023-07-14T18:14:23.042761+0000","last_fullsized":"2023-07-14T18:14:23.042761+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:43:59.148422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.043055+0000","last_change":"2023-07-14T18:14:16.925321+0000","last_active":"2023-07-14T18:14:23.043055+0000","last_peered":"2023-07-14T18:14:23.043055+0000","last_clean":"2023-07-14T18:14:23.043055+0000","last_became_active":"2023-07-14T18:14:16.925016+0000","last_became_peered":"2023-07-14T18:14:16.925016+0000","last_unstale":"2023-07-14T18:14:23.043055+0000","last_undegraded":"2023-07-14T18:14:23.043055+0000","last_fullsized":"2023-07-14T18:14:23.043055+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:42:48.366016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.044022+0000","last_change":"2023-07-14T18:14:17.928838+0000","last_active":"2023-07-14T18:14:23.044022+0000","last_peered":"2023-07-14T18:14:23.044022+0000","last_clean":"2023-07-14T18:14:23.044022+0000","last_became_active":"2023-07-14T18:14:17.928511+0000","last_became_peered":"2023-07-14T18:14:17.928511+0000","last_unstale":"2023-07-14T18:14:23.044022+0000","last_undegraded":"2023-07-14T18:14:23.044022+0000","last_fullsized":"2023-07-14T18:14:23.044022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:58:39.019902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944791+0000","last_change":"2023-07-14T18:14:22.043182+0000","last_active":"2023-07-14T18:14:48.944791+0000","last_peered":"2023-07-14T18:14:48.944791+0000","last_clean":"2023-07-14T18:14:48.944791+0000","last_became_active":"2023-07-14T18:14:22.042779+0000","last_became_peered":"2023-07-14T18:14:22.042779+0000","last_unstale":"2023-07-14T18:14:48.944791+0000","last_undegraded":"2023-07-14T18:14:48.944791+0000","last_fullsized":"2023-07-14T18:14:48.944791+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:15:03.016184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.941119+0000","last_change":"2023-07-14T18:14:16.935321+0000","last_active":"2023-07-14T18:14:47.941119+0000","last_peered":"2023-07-14T18:14:47.941119+0000","last_clean":"2023-07-14T18:14:47.941119+0000","last_became_active":"2023-07-14T18:14:16.934494+0000","last_became_peered":"2023-07-14T18:14:16.934494+0000","last_unstale":"2023-07-14T18:14:47.941119+0000","last_undegraded":"2023-07-14T18:14:47.941119+0000","last_fullsized":"2023-07-14T18:14:47.941119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:50:11.459219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.936730+0000","last_change":"2023-07-14T18:14:17.922139+0000","last_active":"2023-07-14T18:14:47.936730+0000","last_peered":"2023-07-14T18:14:47.936730+0000","last_clean":"2023-07-14T18:14:47.936730+0000","last_became_active":"2023-07-14T18:14:17.921829+0000","last_became_peered":"2023-07-14T18:14:17.921829+0000","last_unstale":"2023-07-14T18:14:47.936730+0000","last_undegraded":"2023-07-14T18:14:47.936730+0000","last_fullsized":"2023-07-14T18:14:47.936730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:01:35.248837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540988+0000","last_change":"2023-07-14T18:14:22.033625+0000","last_active":"2023-07-14T18:14:49.540988+0000","last_peered":"2023-07-14T18:14:49.540988+0000","last_clean":"2023-07-14T18:14:49.540988+0000","last_became_active":"2023-07-14T18:14:22.033253+0000","last_became_peered":"2023-07-14T18:14:22.033253+0000","last_unstale":"2023-07-14T18:14:49.540988+0000","last_undegraded":"2023-07-14T18:14:49.540988+0000","last_fullsized":"2023-07-14T18:14:49.540988+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:16:24.641370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.042648+0000","last_change":"2023-07-14T18:14:16.923864+0000","last_active":"2023-07-14T18:14:23.042648+0000","last_peered":"2023-07-14T18:14:23.042648+0000","last_clean":"2023-07-14T18:14:23.042648+0000","last_became_active":"2023-07-14T18:14:16.923546+0000","last_became_peered":"2023-07-14T18:14:16.923546+0000","last_unstale":"2023-07-14T18:14:23.042648+0000","last_undegraded":"2023-07-14T18:14:23.042648+0000","last_fullsized":"2023-07-14T18:14:23.042648+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:38:31.657819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.043075+0000","last_change":"2023-07-14T18:14:17.936473+0000","last_active":"2023-07-14T18:14:23.043075+0000","last_peered":"2023-07-14T18:14:23.043075+0000","last_clean":"2023-07-14T18:14:23.043075+0000","last_became_active":"2023-07-14T18:14:17.935387+0000","last_became_peered":"2023-07-14T18:14:17.935387+0000","last_unstale":"2023-07-14T18:14:23.043075+0000","last_undegraded":"2023-07-14T18:14:23.043075+0000","last_fullsized":"2023-07-14T18:14:23.043075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:31:24.561502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.936752+0000","last_change":"2023-07-14T18:14:22.041544+0000","last_active":"2023-07-14T18:14:47.936752+0000","last_peered":"2023-07-14T18:14:47.936752+0000","last_clean":"2023-07-14T18:14:47.936752+0000","last_became_active":"2023-07-14T18:14:22.041183+0000","last_became_peered":"2023-07-14T18:14:22.041183+0000","last_unstale":"2023-07-14T18:14:47.936752+0000","last_undegraded":"2023-07-14T18:14:47.936752+0000","last_fullsized":"2023-07-14T18:14:47.936752+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:58:13.466720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.043436+0000","last_change":"2023-07-14T18:14:16.928860+0000","last_active":"2023-07-14T18:14:23.043436+0000","last_peered":"2023-07-14T18:14:23.043436+0000","last_clean":"2023-07-14T18:14:23.043436+0000","last_became_active":"2023-07-14T18:14:16.928567+0000","last_became_peered":"2023-07-14T18:14:16.928567+0000","last_unstale":"2023-07-14T18:14:23.043436+0000","last_undegraded":"2023-07-14T18:14:23.043436+0000","last_fullsized":"2023-07-14T18:14:23.043436+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:48:08.885397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.948806+0000","last_change":"2023-07-14T18:14:17.938456+0000","last_active":"2023-07-14T18:14:48.948806+0000","last_peered":"2023-07-14T18:14:48.948806+0000","last_clean":"2023-07-14T18:14:48.948806+0000","last_became_active":"2023-07-14T18:14:17.938086+0000","last_became_peered":"2023-07-14T18:14:17.938086+0000","last_unstale":"2023-07-14T18:14:48.948806+0000","last_undegraded":"2023-07-14T18:14:48.948806+0000","last_fullsized":"2023-07-14T18:14:48.948806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:16:01.795875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.044141+0000","last_change":"2023-07-14T18:14:22.042785+0000","last_active":"2023-07-14T18:14:23.044141+0000","last_peered":"2023-07-14T18:14:23.044141+0000","last_clean":"2023-07-14T18:14:23.044141+0000","last_became_active":"2023-07-14T18:14:22.042346+0000","last_became_peered":"2023-07-14T18:14:22.042346+0000","last_unstale":"2023-07-14T18:14:23.044141+0000","last_undegraded":"2023-07-14T18:14:23.044141+0000","last_fullsized":"2023-07-14T18:14:23.044141+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:00:14.408038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944005+0000","last_change":"2023-07-14T18:14:16.921168+0000","last_active":"2023-07-14T18:14:48.944005+0000","last_peered":"2023-07-14T18:14:48.944005+0000","last_clean":"2023-07-14T18:14:48.944005+0000","last_became_active":"2023-07-14T18:14:16.920808+0000","last_became_peered":"2023-07-14T18:14:16.920808+0000","last_unstale":"2023-07-14T18:14:48.944005+0000","last_undegraded":"2023-07-14T18:14:48.944005+0000","last_fullsized":"2023-07-14T18:14:48.944005+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:41:57.870102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541090+0000","last_change":"2023-07-14T18:14:17.935100+0000","last_active":"2023-07-14T18:14:49.541090+0000","last_peered":"2023-07-14T18:14:49.541090+0000","last_clean":"2023-07-14T18:14:49.541090+0000","last_became_active":"2023-07-14T18:14:17.934794+0000","last_became_peered":"2023-07-14T18:14:17.934794+0000","last_unstale":"2023-07-14T18:14:49.541090+0000","last_undegraded":"2023-07-14T18:14:49.541090+0000","last_fullsized":"2023-07-14T18:14:49.541090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:37:28.349127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.948310+0000","last_change":"2023-07-14T18:14:22.043495+0000","last_active":"2023-07-14T18:14:48.948310+0000","last_peered":"2023-07-14T18:14:48.948310+0000","last_clean":"2023-07-14T18:14:48.948310+0000","last_became_active":"2023-07-14T18:14:22.042686+0000","last_became_peered":"2023-07-14T18:14:22.042686+0000","last_unstale":"2023-07-14T18:14:48.948310+0000","last_undegraded":"2023-07-14T18:14:48.948310+0000","last_fullsized":"2023-07-14T18:14:48.948310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:16:19.904966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.043464+0000","last_change":"2023-07-14T18:14:16.926006+0000","last_active":"2023-07-14T18:14:23.043464+0000","last_peered":"2023-07-14T18:14:23.043464+0000","last_clean":"2023-07-14T18:14:23.043464+0000","last_became_active":"2023-07-14T18:14:16.925719+0000","last_became_peered":"2023-07-14T18:14:16.925719+0000","last_unstale":"2023-07-14T18:14:23.043464+0000","last_undegraded":"2023-07-14T18:14:23.043464+0000","last_fullsized":"2023-07-14T18:14:23.043464+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:34:22.563247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.948103+0000","last_change":"2023-07-14T18:14:17.934155+0000","last_active":"2023-07-14T18:14:48.948103+0000","last_peered":"2023-07-14T18:14:48.948103+0000","last_clean":"2023-07-14T18:14:48.948103+0000","last_became_active":"2023-07-14T18:14:17.933756+0000","last_became_peered":"2023-07-14T18:14:17.933756+0000","last_unstale":"2023-07-14T18:14:48.948103+0000","last_undegraded":"2023-07-14T18:14:48.948103+0000","last_fullsized":"2023-07-14T18:14:48.948103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:04:42.915865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.042929+0000","last_change":"2023-07-14T18:14:22.039198+0000","last_active":"2023-07-14T18:14:23.042929+0000","last_peered":"2023-07-14T18:14:23.042929+0000","last_clean":"2023-07-14T18:14:23.042929+0000","last_became_active":"2023-07-14T18:14:22.038842+0000","last_became_peered":"2023-07-14T18:14:22.038842+0000","last_unstale":"2023-07-14T18:14:23.042929+0000","last_undegraded":"2023-07-14T18:14:23.042929+0000","last_fullsized":"2023-07-14T18:14:23.042929+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:47:57.598573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:33.565591+0000","last_change":"2023-07-14T18:14:16.920918+0000","last_active":"2023-07-14T18:14:33.565591+0000","last_peered":"2023-07-14T18:14:33.565591+0000","last_clean":"2023-07-14T18:14:33.565591+0000","last_became_active":"2023-07-14T18:14:16.920572+0000","last_became_peered":"2023-07-14T18:14:16.920572+0000","last_unstale":"2023-07-14T18:14:33.565591+0000","last_undegraded":"2023-07-14T18:14:33.565591+0000","last_fullsized":"2023-07-14T18:14:33.565591+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:01:52.488489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3794,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.043769+0000","last_change":"2023-07-14T18:14:17.929413+0000","last_active":"2023-07-14T18:14:23.043769+0000","last_peered":"2023-07-14T18:14:23.043769+0000","last_clean":"2023-07-14T18:14:23.043769+0000","last_became_active":"2023-07-14T18:14:17.929064+0000","last_became_peered":"2023-07-14T18:14:17.929064+0000","last_unstale":"2023-07-14T18:14:23.043769+0000","last_undegraded":"2023-07-14T18:14:23.043769+0000","last_fullsized":"2023-07-14T18:14:23.043769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:29:41.216292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.044322+0000","last_change":"2023-07-14T18:14:22.041983+0000","last_active":"2023-07-14T18:14:23.044322+0000","last_peered":"2023-07-14T18:14:23.044322+0000","last_clean":"2023-07-14T18:14:23.044322+0000","last_became_active":"2023-07-14T18:14:22.041553+0000","last_became_peered":"2023-07-14T18:14:22.041553+0000","last_unstale":"2023-07-14T18:14:23.044322+0000","last_undegraded":"2023-07-14T18:14:23.044322+0000","last_fullsized":"2023-07-14T18:14:23.044322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:38:50.903603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.043388+0000","last_change":"2023-07-14T18:14:16.925444+0000","last_active":"2023-07-14T18:14:23.043388+0000","last_peered":"2023-07-14T18:14:23.043388+0000","last_clean":"2023-07-14T18:14:23.043388+0000","last_became_active":"2023-07-14T18:14:16.925138+0000","last_became_peered":"2023-07-14T18:14:16.925138+0000","last_unstale":"2023-07-14T18:14:23.043388+0000","last_undegraded":"2023-07-14T18:14:23.043388+0000","last_fullsized":"2023-07-14T18:14:23.043388+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:10:22.448371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.941098+0000","last_change":"2023-07-14T18:14:17.922351+0000","last_active":"2023-07-14T18:14:47.941098+0000","last_peered":"2023-07-14T18:14:47.941098+0000","last_clean":"2023-07-14T18:14:47.941098+0000","last_became_active":"2023-07-14T18:14:17.921973+0000","last_became_peered":"2023-07-14T18:14:17.921973+0000","last_unstale":"2023-07-14T18:14:47.941098+0000","last_undegraded":"2023-07-14T18:14:47.941098+0000","last_fullsized":"2023-07-14T18:14:47.941098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:44:32.533360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945943+0000","last_change":"2023-07-14T18:14:22.048274+0000","last_active":"2023-07-14T18:14:48.945943+0000","last_peered":"2023-07-14T18:14:48.945943+0000","last_clean":"2023-07-14T18:14:48.945943+0000","last_became_active":"2023-07-14T18:14:22.047946+0000","last_became_peered":"2023-07-14T18:14:22.047946+0000","last_unstale":"2023-07-14T18:14:48.945943+0000","last_undegraded":"2023-07-14T18:14:48.945943+0000","last_fullsized":"2023-07-14T18:14:48.945943+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:29:47.580140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.042941+0000","last_change":"2023-07-14T18:14:16.933566+0000","last_active":"2023-07-14T18:14:23.042941+0000","last_peered":"2023-07-14T18:14:23.042941+0000","last_clean":"2023-07-14T18:14:23.042941+0000","last_became_active":"2023-07-14T18:14:16.933300+0000","last_became_peered":"2023-07-14T18:14:16.933300+0000","last_unstale":"2023-07-14T18:14:23.042941+0000","last_undegraded":"2023-07-14T18:14:23.042941+0000","last_fullsized":"2023-07-14T18:14:23.042941+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:57:38.599337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944233+0000","last_change":"2023-07-14T18:14:17.928521+0000","last_active":"2023-07-14T18:14:48.944233+0000","last_peered":"2023-07-14T18:14:48.944233+0000","last_clean":"2023-07-14T18:14:48.944233+0000","last_became_active":"2023-07-14T18:14:17.928025+0000","last_became_peered":"2023-07-14T18:14:17.928025+0000","last_unstale":"2023-07-14T18:14:48.944233+0000","last_undegraded":"2023-07-14T18:14:48.944233+0000","last_fullsized":"2023-07-14T18:14:48.944233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:21:23.554363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.044311+0000","last_change":"2023-07-14T18:14:22.046820+0000","last_active":"2023-07-14T18:14:23.044311+0000","last_peered":"2023-07-14T18:14:23.044311+0000","last_clean":"2023-07-14T18:14:23.044311+0000","last_became_active":"2023-07-14T18:14:22.046440+0000","last_became_peered":"2023-07-14T18:14:22.046440+0000","last_unstale":"2023-07-14T18:14:23.044311+0000","last_undegraded":"2023-07-14T18:14:23.044311+0000","last_fullsized":"2023-07-14T18:14:23.044311+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:13:04.976696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:50.894777+0000","last_change":"2023-07-14T18:14:16.922075+0000","last_active":"2023-07-14T18:14:50.894777+0000","last_peered":"2023-07-14T18:14:50.894777+0000","last_clean":"2023-07-14T18:14:50.894777+0000","last_became_active":"2023-07-14T18:14:16.921570+0000","last_became_peered":"2023-07-14T18:14:16.921570+0000","last_unstale":"2023-07-14T18:14:50.894777+0000","last_undegraded":"2023-07-14T18:14:50.894777+0000","last_fullsized":"2023-07-14T18:14:50.894777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:08:37.059574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.943804+0000","last_change":"2023-07-14T18:14:17.936439+0000","last_active":"2023-07-14T18:14:48.943804+0000","last_peered":"2023-07-14T18:14:48.943804+0000","last_clean":"2023-07-14T18:14:48.943804+0000","last_became_active":"2023-07-14T18:14:17.936138+0000","last_became_peered":"2023-07-14T18:14:17.936138+0000","last_unstale":"2023-07-14T18:14:48.943804+0000","last_undegraded":"2023-07-14T18:14:48.943804+0000","last_fullsized":"2023-07-14T18:14:48.943804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:58:02.987288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"peering","last_fresh":"2023-07-14T18:14:23.043196+0000","last_change":"2023-07-14T18:14:47.941773+0000","last_active":"2023-07-14T18:14:47.941773+0000","last_peered":"2023-07-14T18:14:23.043196+0000","last_clean":"2023-07-14T18:14:23.043196+0000","last_became_active":"2023-07-14T18:14:22.048479+0000","last_became_peered":"2023-07-14T18:14:22.048479+0000","last_unstale":"2023-07-14T18:14:23.043196+0000","last_undegraded":"2023-07-14T18:14:23.043196+0000","last_fullsized":"2023-07-14T18:14:23.043196+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,8,9,11],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'8","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.043077+0000","last_change":"2023-07-14T18:14:16.926417+0000","last_active":"2023-07-14T18:14:23.043077+0000","last_peered":"2023-07-14T18:14:23.043077+0000","last_clean":"2023-07-14T18:14:23.043077+0000","last_became_active":"2023-07-14T18:14:16.926101+0000","last_became_peered":"2023-07-14T18:14:16.926101+0000","last_unstale":"2023-07-14T18:14:23.043077+0000","last_undegraded":"2023-07-14T18:14:23.043077+0000","last_fullsized":"2023-07-14T18:14:23.043077+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:47:26.940643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.043986+0000","last_change":"2023-07-14T18:14:17.923222+0000","last_active":"2023-07-14T18:14:23.043986+0000","last_peered":"2023-07-14T18:14:23.043986+0000","last_clean":"2023-07-14T18:14:23.043986+0000","last_became_active":"2023-07-14T18:14:17.922416+0000","last_became_peered":"2023-07-14T18:14:17.922416+0000","last_unstale":"2023-07-14T18:14:23.043986+0000","last_undegraded":"2023-07-14T18:14:23.043986+0000","last_fullsized":"2023-07-14T18:14:23.043986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:27:08.510679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.043959+0000","last_change":"2023-07-14T18:14:22.042171+0000","last_active":"2023-07-14T18:14:23.043959+0000","last_peered":"2023-07-14T18:14:23.043959+0000","last_clean":"2023-07-14T18:14:23.043959+0000","last_became_active":"2023-07-14T18:14:22.041692+0000","last_became_peered":"2023-07-14T18:14:22.041692+0000","last_unstale":"2023-07-14T18:14:23.043959+0000","last_undegraded":"2023-07-14T18:14:23.043959+0000","last_fullsized":"2023-07-14T18:14:23.043959+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:02:17.710154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.042912+0000","last_change":"2023-07-14T18:14:16.922883+0000","last_active":"2023-07-14T18:14:23.042912+0000","last_peered":"2023-07-14T18:14:23.042912+0000","last_clean":"2023-07-14T18:14:23.042912+0000","last_became_active":"2023-07-14T18:14:16.922501+0000","last_became_peered":"2023-07-14T18:14:16.922501+0000","last_unstale":"2023-07-14T18:14:23.042912+0000","last_undegraded":"2023-07-14T18:14:23.042912+0000","last_fullsized":"2023-07-14T18:14:23.042912+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:01:19.995117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946509+0000","last_change":"2023-07-14T18:14:17.922899+0000","last_active":"2023-07-14T18:14:48.946509+0000","last_peered":"2023-07-14T18:14:48.946509+0000","last_clean":"2023-07-14T18:14:48.946509+0000","last_became_active":"2023-07-14T18:14:17.922608+0000","last_became_peered":"2023-07-14T18:14:17.922608+0000","last_unstale":"2023-07-14T18:14:48.946509+0000","last_undegraded":"2023-07-14T18:14:48.946509+0000","last_fullsized":"2023-07-14T18:14:48.946509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:53:19.741997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.043503+0000","last_change":"2023-07-14T18:14:22.327484+0000","last_active":"2023-07-14T18:14:23.043503+0000","last_peered":"2023-07-14T18:14:23.043503+0000","last_clean":"2023-07-14T18:14:23.043503+0000","last_became_active":"2023-07-14T18:14:22.327010+0000","last_became_peered":"2023-07-14T18:14:22.327010+0000","last_unstale":"2023-07-14T18:14:23.043503+0000","last_undegraded":"2023-07-14T18:14:23.043503+0000","last_fullsized":"2023-07-14T18:14:23.043503+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:03:46.949712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.043591+0000","last_change":"2023-07-14T18:14:16.930606+0000","last_active":"2023-07-14T18:14:23.043591+0000","last_peered":"2023-07-14T18:14:23.043591+0000","last_clean":"2023-07-14T18:14:23.043591+0000","last_became_active":"2023-07-14T18:14:16.930175+0000","last_became_peered":"2023-07-14T18:14:16.930175+0000","last_unstale":"2023-07-14T18:14:23.043591+0000","last_undegraded":"2023-07-14T18:14:23.043591+0000","last_fullsized":"2023-07-14T18:14:23.043591+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:14:52.887300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.043658+0000","last_change":"2023-07-14T18:14:17.931962+0000","last_active":"2023-07-14T18:14:23.043658+0000","last_peered":"2023-07-14T18:14:23.043658+0000","last_clean":"2023-07-14T18:14:23.043658+0000","last_became_active":"2023-07-14T18:14:17.931621+0000","last_became_peered":"2023-07-14T18:14:17.931621+0000","last_unstale":"2023-07-14T18:14:23.043658+0000","last_undegraded":"2023-07-14T18:14:23.043658+0000","last_fullsized":"2023-07-14T18:14:23.043658+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:00:55.115418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.941942+0000","last_change":"2023-07-14T18:14:22.038977+0000","last_active":"2023-07-14T18:14:47.941942+0000","last_peered":"2023-07-14T18:14:47.941942+0000","last_clean":"2023-07-14T18:14:47.941942+0000","last_became_active":"2023-07-14T18:14:22.038515+0000","last_became_peered":"2023-07-14T18:14:22.038515+0000","last_unstale":"2023-07-14T18:14:47.941942+0000","last_undegraded":"2023-07-14T18:14:47.941942+0000","last_fullsized":"2023-07-14T18:14:47.941942+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:06:22.910162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.943866+0000","last_change":"2023-07-14T18:14:16.925457+0000","last_active":"2023-07-14T18:14:48.943866+0000","last_peered":"2023-07-14T18:14:48.943866+0000","last_clean":"2023-07-14T18:14:48.943866+0000","last_became_active":"2023-07-14T18:14:16.925150+0000","last_became_peered":"2023-07-14T18:14:16.925150+0000","last_unstale":"2023-07-14T18:14:48.943866+0000","last_undegraded":"2023-07-14T18:14:48.943866+0000","last_fullsized":"2023-07-14T18:14:48.943866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:46:38.768309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.043000+0000","last_change":"2023-07-14T18:14:17.926492+0000","last_active":"2023-07-14T18:14:23.043000+0000","last_peered":"2023-07-14T18:14:23.043000+0000","last_clean":"2023-07-14T18:14:23.043000+0000","last_became_active":"2023-07-14T18:14:17.926218+0000","last_became_peered":"2023-07-14T18:14:17.926218+0000","last_unstale":"2023-07-14T18:14:23.043000+0000","last_undegraded":"2023-07-14T18:14:23.043000+0000","last_fullsized":"2023-07-14T18:14:23.043000+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:34:09.581079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.942201+0000","last_change":"2023-07-14T18:14:22.327242+0000","last_active":"2023-07-14T18:14:47.942201+0000","last_peered":"2023-07-14T18:14:47.942201+0000","last_clean":"2023-07-14T18:14:47.942201+0000","last_became_active":"2023-07-14T18:14:22.326787+0000","last_became_peered":"2023-07-14T18:14:22.326787+0000","last_unstale":"2023-07-14T18:14:47.942201+0000","last_undegraded":"2023-07-14T18:14:47.942201+0000","last_fullsized":"2023-07-14T18:14:47.942201+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:16:34.451163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.042925+0000","last_change":"2023-07-14T18:14:17.925035+0000","last_active":"2023-07-14T18:14:23.042925+0000","last_peered":"2023-07-14T18:14:23.042925+0000","last_clean":"2023-07-14T18:14:23.042925+0000","last_became_active":"2023-07-14T18:14:17.924713+0000","last_became_peered":"2023-07-14T18:14:17.924713+0000","last_unstale":"2023-07-14T18:14:23.042925+0000","last_undegraded":"2023-07-14T18:14:23.042925+0000","last_fullsized":"2023-07-14T18:14:23.042925+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:05:55.756247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.941016+0000","last_change":"2023-07-14T18:14:16.935561+0000","last_active":"2023-07-14T18:14:47.941016+0000","last_peered":"2023-07-14T18:14:47.941016+0000","last_clean":"2023-07-14T18:14:47.941016+0000","last_became_active":"2023-07-14T18:14:16.935022+0000","last_became_peered":"2023-07-14T18:14:16.935022+0000","last_unstale":"2023-07-14T18:14:47.941016+0000","last_undegraded":"2023-07-14T18:14:47.941016+0000","last_fullsized":"2023-07-14T18:14:47.941016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:56:29.171270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.943861+0000","last_change":"2023-07-14T18:14:22.043818+0000","last_active":"2023-07-14T18:14:48.943861+0000","last_peered":"2023-07-14T18:14:48.943861+0000","last_clean":"2023-07-14T18:14:48.943861+0000","last_became_active":"2023-07-14T18:14:22.043449+0000","last_became_peered":"2023-07-14T18:14:22.043449+0000","last_unstale":"2023-07-14T18:14:48.943861+0000","last_undegraded":"2023-07-14T18:14:48.943861+0000","last_fullsized":"2023-07-14T18:14:48.943861+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:25:52.282417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.936894+0000","last_change":"2023-07-14T18:14:17.930211+0000","last_active":"2023-07-14T18:14:47.936894+0000","last_peered":"2023-07-14T18:14:47.936894+0000","last_clean":"2023-07-14T18:14:47.936894+0000","last_became_active":"2023-07-14T18:14:17.929851+0000","last_became_peered":"2023-07-14T18:14:17.929851+0000","last_unstale":"2023-07-14T18:14:47.936894+0000","last_undegraded":"2023-07-14T18:14:47.936894+0000","last_fullsized":"2023-07-14T18:14:47.936894+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:05:44.957698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.894113+0000","last_change":"2023-07-14T18:14:16.928995+0000","last_active":"2023-07-14T18:14:49.894113+0000","last_peered":"2023-07-14T18:14:49.894113+0000","last_clean":"2023-07-14T18:14:49.894113+0000","last_became_active":"2023-07-14T18:14:16.928612+0000","last_became_peered":"2023-07-14T18:14:16.928612+0000","last_unstale":"2023-07-14T18:14:49.894113+0000","last_undegraded":"2023-07-14T18:14:49.894113+0000","last_fullsized":"2023-07-14T18:14:49.894113+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:33:55.215553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":340,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.943619+0000","last_change":"2023-07-14T18:13:11.614507+0000","last_active":"2023-07-14T18:14:48.943619+0000","last_peered":"2023-07-14T18:14:48.943619+0000","last_clean":"2023-07-14T18:14:48.943619+0000","last_became_active":"2023-07-14T18:13:11.614063+0000","last_became_peered":"2023-07-14T18:13:11.614063+0000","last_unstale":"2023-07-14T18:14:48.943619+0000","last_undegraded":"2023-07-14T18:14:48.943619+0000","last_fullsized":"2023-07-14T18:14:48.943619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:10:03.995193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:10:03.995193+0000","last_clean_scrub_stamp":"2023-07-14T18:10:03.995193+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:35:25.729988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.044245+0000","last_change":"2023-07-14T18:14:22.327767+0000","last_active":"2023-07-14T18:14:23.044245+0000","last_peered":"2023-07-14T18:14:23.044245+0000","last_clean":"2023-07-14T18:14:23.044245+0000","last_became_active":"2023-07-14T18:14:22.327352+0000","last_became_peered":"2023-07-14T18:14:22.327352+0000","last_unstale":"2023-07-14T18:14:23.044245+0000","last_undegraded":"2023-07-14T18:14:23.044245+0000","last_fullsized":"2023-07-14T18:14:23.044245+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:57:34.977504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"peering","last_fresh":"2023-07-14T18:14:23.042553+0000","last_change":"2023-07-14T18:14:47.940476+0000","last_active":"2023-07-14T18:14:47.940476+0000","last_peered":"2023-07-14T18:14:23.042553+0000","last_clean":"2023-07-14T18:14:23.042553+0000","last_became_active":"2023-07-14T18:14:17.922463+0000","last_became_peered":"2023-07-14T18:14:17.922463+0000","last_unstale":"2023-07-14T18:14:23.042553+0000","last_undegraded":"2023-07-14T18:14:23.042553+0000","last_fullsized":"2023-07-14T18:14:23.042553+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4,9],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.894434+0000","last_change":"2023-07-14T18:14:48.950022+0000","last_active":"2023-07-14T18:14:49.894434+0000","last_peered":"2023-07-14T18:14:49.894434+0000","last_clean":"2023-07-14T18:14:49.894434+0000","last_became_active":"2023-07-14T18:14:48.949149+0000","last_became_peered":"2023-07-14T18:14:48.949149+0000","last_unstale":"2023-07-14T18:14:49.894434+0000","last_undegraded":"2023-07-14T18:14:49.894434+0000","last_fullsized":"2023-07-14T18:14:49.894434+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:10:25.993299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945005+0000","last_change":"2023-07-14T18:14:22.038428+0000","last_active":"2023-07-14T18:14:48.945005+0000","last_peered":"2023-07-14T18:14:48.945005+0000","last_clean":"2023-07-14T18:14:48.945005+0000","last_became_active":"2023-07-14T18:14:22.038038+0000","last_became_peered":"2023-07-14T18:14:22.038038+0000","last_unstale":"2023-07-14T18:14:48.945005+0000","last_undegraded":"2023-07-14T18:14:48.945005+0000","last_fullsized":"2023-07-14T18:14:48.945005+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:58:40.233180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.043668+0000","last_change":"2023-07-14T18:14:16.933110+0000","last_active":"2023-07-14T18:14:23.043668+0000","last_peered":"2023-07-14T18:14:23.043668+0000","last_clean":"2023-07-14T18:14:23.043668+0000","last_became_active":"2023-07-14T18:14:16.932844+0000","last_became_peered":"2023-07-14T18:14:16.932844+0000","last_unstale":"2023-07-14T18:14:23.043668+0000","last_undegraded":"2023-07-14T18:14:23.043668+0000","last_fullsized":"2023-07-14T18:14:23.043668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:23:39.914998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.942146+0000","last_change":"2023-07-14T18:14:17.920168+0000","last_active":"2023-07-14T18:14:47.942146+0000","last_peered":"2023-07-14T18:14:47.942146+0000","last_clean":"2023-07-14T18:14:47.942146+0000","last_became_active":"2023-07-14T18:14:17.919576+0000","last_became_peered":"2023-07-14T18:14:17.919576+0000","last_unstale":"2023-07-14T18:14:47.942146+0000","last_undegraded":"2023-07-14T18:14:47.942146+0000","last_fullsized":"2023-07-14T18:14:47.942146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:50:05.490741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.936519+0000","last_change":"2023-07-14T18:14:22.327206+0000","last_active":"2023-07-14T18:14:47.936519+0000","last_peered":"2023-07-14T18:14:47.936519+0000","last_clean":"2023-07-14T18:14:47.936519+0000","last_became_active":"2023-07-14T18:14:22.326743+0000","last_became_peered":"2023-07-14T18:14:22.326743+0000","last_unstale":"2023-07-14T18:14:47.936519+0000","last_undegraded":"2023-07-14T18:14:47.936519+0000","last_fullsized":"2023-07-14T18:14:47.936519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:07:53.561873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.044507+0000","last_change":"2023-07-14T18:14:16.928994+0000","last_active":"2023-07-14T18:14:23.044507+0000","last_peered":"2023-07-14T18:14:23.044507+0000","last_clean":"2023-07-14T18:14:23.044507+0000","last_became_active":"2023-07-14T18:14:16.928468+0000","last_became_peered":"2023-07-14T18:14:16.928468+0000","last_unstale":"2023-07-14T18:14:23.044507+0000","last_undegraded":"2023-07-14T18:14:23.044507+0000","last_fullsized":"2023-07-14T18:14:23.044507+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:26:00.943782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.937743+0000","last_change":"2023-07-14T18:14:17.919147+0000","last_active":"2023-07-14T18:14:47.937743+0000","last_peered":"2023-07-14T18:14:47.937743+0000","last_clean":"2023-07-14T18:14:47.937743+0000","last_became_active":"2023-07-14T18:14:17.918811+0000","last_became_peered":"2023-07-14T18:14:17.918811+0000","last_unstale":"2023-07-14T18:14:47.937743+0000","last_undegraded":"2023-07-14T18:14:47.937743+0000","last_fullsized":"2023-07-14T18:14:47.937743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:54:37.460727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.043013+0000","last_change":"2023-07-14T18:14:22.032632+0000","last_active":"2023-07-14T18:14:23.043013+0000","last_peered":"2023-07-14T18:14:23.043013+0000","last_clean":"2023-07-14T18:14:23.043013+0000","last_became_active":"2023-07-14T18:14:22.032233+0000","last_became_peered":"2023-07-14T18:14:22.032233+0000","last_unstale":"2023-07-14T18:14:23.043013+0000","last_undegraded":"2023-07-14T18:14:23.043013+0000","last_fullsized":"2023-07-14T18:14:23.043013+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:04:38.414089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.947795+0000","last_change":"2023-07-14T18:14:16.924993+0000","last_active":"2023-07-14T18:14:48.947795+0000","last_peered":"2023-07-14T18:14:48.947795+0000","last_clean":"2023-07-14T18:14:48.947795+0000","last_became_active":"2023-07-14T18:14:16.924702+0000","last_became_peered":"2023-07-14T18:14:16.924702+0000","last_unstale":"2023-07-14T18:14:48.947795+0000","last_undegraded":"2023-07-14T18:14:48.947795+0000","last_fullsized":"2023-07-14T18:14:48.947795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:21:25.188427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541234+0000","last_change":"2023-07-14T18:14:17.943397+0000","last_active":"2023-07-14T18:14:49.541234+0000","last_peered":"2023-07-14T18:14:49.541234+0000","last_clean":"2023-07-14T18:14:49.541234+0000","last_became_active":"2023-07-14T18:14:17.942114+0000","last_became_peered":"2023-07-14T18:14:17.942114+0000","last_unstale":"2023-07-14T18:14:49.541234+0000","last_undegraded":"2023-07-14T18:14:49.541234+0000","last_fullsized":"2023-07-14T18:14:49.541234+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:46:40.372375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.947250+0000","last_change":"2023-07-14T18:14:22.328273+0000","last_active":"2023-07-14T18:14:48.947250+0000","last_peered":"2023-07-14T18:14:48.947250+0000","last_clean":"2023-07-14T18:14:48.947250+0000","last_became_active":"2023-07-14T18:14:22.327843+0000","last_became_peered":"2023-07-14T18:14:22.327843+0000","last_unstale":"2023-07-14T18:14:48.947250+0000","last_undegraded":"2023-07-14T18:14:48.947250+0000","last_fullsized":"2023-07-14T18:14:48.947250+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:23:58.886753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.545792+0000","last_change":"2023-07-14T18:14:49.543525+0000","last_active":"2023-07-14T18:14:49.545792+0000","last_peered":"2023-07-14T18:14:49.545792+0000","last_clean":"2023-07-14T18:14:49.545792+0000","last_became_active":"2023-07-14T18:14:49.542962+0000","last_became_peered":"2023-07-14T18:14:49.542962+0000","last_unstale":"2023-07-14T18:14:49.545792+0000","last_undegraded":"2023-07-14T18:14:49.545792+0000","last_fullsized":"2023-07-14T18:14:49.545792+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:11:59.955363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.042312+0000","last_change":"2023-07-14T18:14:17.932181+0000","last_active":"2023-07-14T18:14:23.042312+0000","last_peered":"2023-07-14T18:14:23.042312+0000","last_clean":"2023-07-14T18:14:23.042312+0000","last_became_active":"2023-07-14T18:14:17.931811+0000","last_became_peered":"2023-07-14T18:14:17.931811+0000","last_unstale":"2023-07-14T18:14:23.042312+0000","last_undegraded":"2023-07-14T18:14:23.042312+0000","last_fullsized":"2023-07-14T18:14:23.042312+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:04:25.241107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.044528+0000","last_change":"2023-07-14T18:14:22.047281+0000","last_active":"2023-07-14T18:14:23.044528+0000","last_peered":"2023-07-14T18:14:23.044528+0000","last_clean":"2023-07-14T18:14:23.044528+0000","last_became_active":"2023-07-14T18:14:22.045531+0000","last_became_peered":"2023-07-14T18:14:22.045531+0000","last_unstale":"2023-07-14T18:14:23.044528+0000","last_undegraded":"2023-07-14T18:14:23.044528+0000","last_fullsized":"2023-07-14T18:14:23.044528+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:39:44.705154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.957885+0000","last_change":"2023-07-14T18:14:16.926904+0000","last_active":"2023-07-14T18:14:47.957885+0000","last_peered":"2023-07-14T18:14:47.957885+0000","last_clean":"2023-07-14T18:14:47.957885+0000","last_became_active":"2023-07-14T18:14:16.926579+0000","last_became_peered":"2023-07-14T18:14:16.926579+0000","last_unstale":"2023-07-14T18:14:47.957885+0000","last_undegraded":"2023-07-14T18:14:47.957885+0000","last_fullsized":"2023-07-14T18:14:47.957885+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:00:13.082750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946719+0000","last_change":"2023-07-14T18:14:17.926005+0000","last_active":"2023-07-14T18:14:48.946719+0000","last_peered":"2023-07-14T18:14:48.946719+0000","last_clean":"2023-07-14T18:14:48.946719+0000","last_became_active":"2023-07-14T18:14:17.925266+0000","last_became_peered":"2023-07-14T18:14:17.925266+0000","last_unstale":"2023-07-14T18:14:48.946719+0000","last_undegraded":"2023-07-14T18:14:48.946719+0000","last_fullsized":"2023-07-14T18:14:48.946719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:03:29.763528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946669+0000","last_change":"2023-07-14T18:14:22.043219+0000","last_active":"2023-07-14T18:14:48.946669+0000","last_peered":"2023-07-14T18:14:48.946669+0000","last_clean":"2023-07-14T18:14:48.946669+0000","last_became_active":"2023-07-14T18:14:22.042654+0000","last_became_peered":"2023-07-14T18:14:22.042654+0000","last_unstale":"2023-07-14T18:14:48.946669+0000","last_undegraded":"2023-07-14T18:14:48.946669+0000","last_fullsized":"2023-07-14T18:14:48.946669+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:46:41.786917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.042563+0000","last_change":"2023-07-14T18:14:16.923130+0000","last_active":"2023-07-14T18:14:23.042563+0000","last_peered":"2023-07-14T18:14:23.042563+0000","last_clean":"2023-07-14T18:14:23.042563+0000","last_became_active":"2023-07-14T18:14:16.922835+0000","last_became_peered":"2023-07-14T18:14:16.922835+0000","last_unstale":"2023-07-14T18:14:23.042563+0000","last_undegraded":"2023-07-14T18:14:23.042563+0000","last_fullsized":"2023-07-14T18:14:23.042563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:41.783624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.937816+0000","last_change":"2023-07-14T18:14:17.931024+0000","last_active":"2023-07-14T18:14:47.937816+0000","last_peered":"2023-07-14T18:14:47.937816+0000","last_clean":"2023-07-14T18:14:47.937816+0000","last_became_active":"2023-07-14T18:14:17.929529+0000","last_became_peered":"2023-07-14T18:14:17.929529+0000","last_unstale":"2023-07-14T18:14:47.937816+0000","last_undegraded":"2023-07-14T18:14:47.937816+0000","last_fullsized":"2023-07-14T18:14:47.937816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:40:41.156046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946795+0000","last_change":"2023-07-14T18:14:22.327313+0000","last_active":"2023-07-14T18:14:48.946795+0000","last_peered":"2023-07-14T18:14:48.946795+0000","last_clean":"2023-07-14T18:14:48.946795+0000","last_became_active":"2023-07-14T18:14:22.326922+0000","last_became_peered":"2023-07-14T18:14:22.326922+0000","last_unstale":"2023-07-14T18:14:48.946795+0000","last_undegraded":"2023-07-14T18:14:48.946795+0000","last_fullsized":"2023-07-14T18:14:48.946795+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:48:34.325995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.042645+0000","last_change":"2023-07-14T18:14:16.926632+0000","last_active":"2023-07-14T18:14:23.042645+0000","last_peered":"2023-07-14T18:14:23.042645+0000","last_clean":"2023-07-14T18:14:23.042645+0000","last_became_active":"2023-07-14T18:14:16.926279+0000","last_became_peered":"2023-07-14T18:14:16.926279+0000","last_unstale":"2023-07-14T18:14:23.042645+0000","last_undegraded":"2023-07-14T18:14:23.042645+0000","last_fullsized":"2023-07-14T18:14:23.042645+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:21:49.512723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944432+0000","last_change":"2023-07-14T18:14:17.924416+0000","last_active":"2023-07-14T18:14:48.944432+0000","last_peered":"2023-07-14T18:14:48.944432+0000","last_clean":"2023-07-14T18:14:48.944432+0000","last_became_active":"2023-07-14T18:14:17.924140+0000","last_became_peered":"2023-07-14T18:14:17.924140+0000","last_unstale":"2023-07-14T18:14:48.944432+0000","last_undegraded":"2023-07-14T18:14:48.944432+0000","last_fullsized":"2023-07-14T18:14:48.944432+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:36:11.267149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946249+0000","last_change":"2023-07-14T18:14:22.037127+0000","last_active":"2023-07-14T18:14:48.946249+0000","last_peered":"2023-07-14T18:14:48.946249+0000","last_clean":"2023-07-14T18:14:48.946249+0000","last_became_active":"2023-07-14T18:14:22.036647+0000","last_became_peered":"2023-07-14T18:14:22.036647+0000","last_unstale":"2023-07-14T18:14:48.946249+0000","last_undegraded":"2023-07-14T18:14:48.946249+0000","last_fullsized":"2023-07-14T18:14:48.946249+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:45:53.709735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.043204+0000","last_change":"2023-07-14T18:14:16.921494+0000","last_active":"2023-07-14T18:14:23.043204+0000","last_peered":"2023-07-14T18:14:23.043204+0000","last_clean":"2023-07-14T18:14:23.043204+0000","last_became_active":"2023-07-14T18:14:16.919622+0000","last_became_peered":"2023-07-14T18:14:16.919622+0000","last_unstale":"2023-07-14T18:14:23.043204+0000","last_undegraded":"2023-07-14T18:14:23.043204+0000","last_fullsized":"2023-07-14T18:14:23.043204+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:46:08.638644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944208+0000","last_change":"2023-07-14T18:14:17.922266+0000","last_active":"2023-07-14T18:14:48.944208+0000","last_peered":"2023-07-14T18:14:48.944208+0000","last_clean":"2023-07-14T18:14:48.944208+0000","last_became_active":"2023-07-14T18:14:17.921461+0000","last_became_peered":"2023-07-14T18:14:17.921461+0000","last_unstale":"2023-07-14T18:14:48.944208+0000","last_undegraded":"2023-07-14T18:14:48.944208+0000","last_fullsized":"2023-07-14T18:14:48.944208+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:27:06.427711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541171+0000","last_change":"2023-07-14T18:14:22.044080+0000","last_active":"2023-07-14T18:14:49.541171+0000","last_peered":"2023-07-14T18:14:49.541171+0000","last_clean":"2023-07-14T18:14:49.541171+0000","last_became_active":"2023-07-14T18:14:22.043612+0000","last_became_peered":"2023-07-14T18:14:22.043612+0000","last_unstale":"2023-07-14T18:14:49.541171+0000","last_undegraded":"2023-07-14T18:14:49.541171+0000","last_fullsized":"2023-07-14T18:14:49.541171+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:41:00.904598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.043753+0000","last_change":"2023-07-14T18:14:16.931871+0000","last_active":"2023-07-14T18:14:23.043753+0000","last_peered":"2023-07-14T18:14:23.043753+0000","last_clean":"2023-07-14T18:14:23.043753+0000","last_became_active":"2023-07-14T18:14:16.931549+0000","last_became_peered":"2023-07-14T18:14:16.931549+0000","last_unstale":"2023-07-14T18:14:23.043753+0000","last_undegraded":"2023-07-14T18:14:23.043753+0000","last_fullsized":"2023-07-14T18:14:23.043753+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:12:21.938967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944103+0000","last_change":"2023-07-14T18:14:17.929550+0000","last_active":"2023-07-14T18:14:48.944103+0000","last_peered":"2023-07-14T18:14:48.944103+0000","last_clean":"2023-07-14T18:14:48.944103+0000","last_became_active":"2023-07-14T18:14:17.929204+0000","last_became_peered":"2023-07-14T18:14:17.929204+0000","last_unstale":"2023-07-14T18:14:48.944103+0000","last_undegraded":"2023-07-14T18:14:48.944103+0000","last_fullsized":"2023-07-14T18:14:48.944103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:46:07.635189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.042413+0000","last_change":"2023-07-14T18:14:22.034899+0000","last_active":"2023-07-14T18:14:23.042413+0000","last_peered":"2023-07-14T18:14:23.042413+0000","last_clean":"2023-07-14T18:14:23.042413+0000","last_became_active":"2023-07-14T18:14:22.034524+0000","last_became_peered":"2023-07-14T18:14:22.034524+0000","last_unstale":"2023-07-14T18:14:23.042413+0000","last_undegraded":"2023-07-14T18:14:23.042413+0000","last_fullsized":"2023-07-14T18:14:23.042413+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:48:57.308042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.943046+0000","last_change":"2023-07-14T18:14:16.935285+0000","last_active":"2023-07-14T18:14:47.943046+0000","last_peered":"2023-07-14T18:14:47.943046+0000","last_clean":"2023-07-14T18:14:47.943046+0000","last_became_active":"2023-07-14T18:14:16.934888+0000","last_became_peered":"2023-07-14T18:14:16.934888+0000","last_unstale":"2023-07-14T18:14:47.943046+0000","last_undegraded":"2023-07-14T18:14:47.943046+0000","last_fullsized":"2023-07-14T18:14:47.943046+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:07:28.573759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944372+0000","last_change":"2023-07-14T18:14:17.930982+0000","last_active":"2023-07-14T18:14:48.944372+0000","last_peered":"2023-07-14T18:14:48.944372+0000","last_clean":"2023-07-14T18:14:48.944372+0000","last_became_active":"2023-07-14T18:14:17.930661+0000","last_became_peered":"2023-07-14T18:14:17.930661+0000","last_unstale":"2023-07-14T18:14:48.944372+0000","last_undegraded":"2023-07-14T18:14:48.944372+0000","last_fullsized":"2023-07-14T18:14:48.944372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:42:10.943678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946615+0000","last_change":"2023-07-14T18:14:17.926204+0000","last_active":"2023-07-14T18:14:48.946615+0000","last_peered":"2023-07-14T18:14:48.946615+0000","last_clean":"2023-07-14T18:14:48.946615+0000","last_became_active":"2023-07-14T18:14:17.925689+0000","last_became_peered":"2023-07-14T18:14:17.925689+0000","last_unstale":"2023-07-14T18:14:48.946615+0000","last_undegraded":"2023-07-14T18:14:48.946615+0000","last_fullsized":"2023-07-14T18:14:48.946615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:32:10.441762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.042949+0000","last_change":"2023-07-14T18:14:17.923146+0000","last_active":"2023-07-14T18:14:23.042949+0000","last_peered":"2023-07-14T18:14:23.042949+0000","last_clean":"2023-07-14T18:14:23.042949+0000","last_became_active":"2023-07-14T18:14:17.922734+0000","last_became_peered":"2023-07-14T18:14:17.922734+0000","last_unstale":"2023-07-14T18:14:23.042949+0000","last_undegraded":"2023-07-14T18:14:23.042949+0000","last_fullsized":"2023-07-14T18:14:23.042949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:19:46.064120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.042404+0000","last_change":"2023-07-14T18:14:17.927397+0000","last_active":"2023-07-14T18:14:23.042404+0000","last_peered":"2023-07-14T18:14:23.042404+0000","last_clean":"2023-07-14T18:14:23.042404+0000","last_became_active":"2023-07-14T18:14:17.927000+0000","last_became_peered":"2023-07-14T18:14:17.927000+0000","last_unstale":"2023-07-14T18:14:23.042404+0000","last_undegraded":"2023-07-14T18:14:23.042404+0000","last_fullsized":"2023-07-14T18:14:23.042404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:47:44.090071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541451+0000","last_change":"2023-07-14T18:14:17.935266+0000","last_active":"2023-07-14T18:14:49.541451+0000","last_peered":"2023-07-14T18:14:49.541451+0000","last_clean":"2023-07-14T18:14:49.541451+0000","last_became_active":"2023-07-14T18:14:17.934944+0000","last_became_peered":"2023-07-14T18:14:17.934944+0000","last_unstale":"2023-07-14T18:14:49.541451+0000","last_undegraded":"2023-07-14T18:14:49.541451+0000","last_fullsized":"2023-07-14T18:14:49.541451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:36:06.516397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"peering","last_fresh":"2023-07-14T18:14:23.043134+0000","last_change":"2023-07-14T18:14:47.942726+0000","last_active":"2023-07-14T18:14:47.942726+0000","last_peered":"2023-07-14T18:14:23.043134+0000","last_clean":"2023-07-14T18:14:23.043134+0000","last_became_active":"2023-07-14T18:14:17.929066+0000","last_became_peered":"2023-07-14T18:14:17.929066+0000","last_unstale":"2023-07-14T18:14:23.043134+0000","last_undegraded":"2023-07-14T18:14:23.043134+0000","last_fullsized":"2023-07-14T18:14:23.043134+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4,9],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946187+0000","last_change":"2023-07-14T18:14:17.930757+0000","last_active":"2023-07-14T18:14:48.946187+0000","last_peered":"2023-07-14T18:14:48.946187+0000","last_clean":"2023-07-14T18:14:48.946187+0000","last_became_active":"2023-07-14T18:14:17.930302+0000","last_became_peered":"2023-07-14T18:14:17.930302+0000","last_unstale":"2023-07-14T18:14:48.946187+0000","last_undegraded":"2023-07-14T18:14:48.946187+0000","last_fullsized":"2023-07-14T18:14:48.946187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:44:33.862208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.043048+0000","last_change":"2023-07-14T18:14:17.930337+0000","last_active":"2023-07-14T18:14:23.043048+0000","last_peered":"2023-07-14T18:14:23.043048+0000","last_clean":"2023-07-14T18:14:23.043048+0000","last_became_active":"2023-07-14T18:14:17.930037+0000","last_became_peered":"2023-07-14T18:14:17.930037+0000","last_unstale":"2023-07-14T18:14:23.043048+0000","last_undegraded":"2023-07-14T18:14:23.043048+0000","last_fullsized":"2023-07-14T18:14:23.043048+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:38:04.631807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.939298+0000","last_change":"2023-07-14T18:14:17.912750+0000","last_active":"2023-07-14T18:14:47.939298+0000","last_peered":"2023-07-14T18:14:47.939298+0000","last_clean":"2023-07-14T18:14:47.939298+0000","last_became_active":"2023-07-14T18:14:17.912402+0000","last_became_peered":"2023-07-14T18:14:17.912402+0000","last_unstale":"2023-07-14T18:14:47.939298+0000","last_undegraded":"2023-07-14T18:14:47.939298+0000","last_fullsized":"2023-07-14T18:14:47.939298+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:55:52.861047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T18:14:23.042792+0000","last_change":"2023-07-14T18:14:17.930004+0000","last_active":"2023-07-14T18:14:23.042792+0000","last_peered":"2023-07-14T18:14:23.042792+0000","last_clean":"2023-07-14T18:14:23.042792+0000","last_became_active":"2023-07-14T18:14:17.929719+0000","last_became_peered":"2023-07-14T18:14:17.929719+0000","last_unstale":"2023-07-14T18:14:23.042792+0000","last_undegraded":"2023-07-14T18:14:23.042792+0000","last_fullsized":"2023-07-14T18:14:23.042792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:05:10.971979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.936529+0000","last_change":"2023-07-14T18:14:17.919044+0000","last_active":"2023-07-14T18:14:47.936529+0000","last_peered":"2023-07-14T18:14:47.936529+0000","last_clean":"2023-07-14T18:14:47.936529+0000","last_became_active":"2023-07-14T18:14:17.918079+0000","last_became_peered":"2023-07-14T18:14:17.918079+0000","last_unstale":"2023-07-14T18:14:47.936529+0000","last_undegraded":"2023-07-14T18:14:47.936529+0000","last_fullsized":"2023-07-14T18:14:47.936529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:41:26.331275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946548+0000","last_change":"2023-07-14T18:14:17.921560+0000","last_active":"2023-07-14T18:14:48.946548+0000","last_peered":"2023-07-14T18:14:48.946548+0000","last_clean":"2023-07-14T18:14:48.946548+0000","last_became_active":"2023-07-14T18:14:17.921243+0000","last_became_peered":"2023-07-14T18:14:17.921243+0000","last_unstale":"2023-07-14T18:14:48.946548+0000","last_undegraded":"2023-07-14T18:14:48.946548+0000","last_fullsized":"2023-07-14T18:14:48.946548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:03:15.907251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.941389+0000","last_change":"2023-07-14T18:14:17.927504+0000","last_active":"2023-07-14T18:14:47.941389+0000","last_peered":"2023-07-14T18:14:47.941389+0000","last_clean":"2023-07-14T18:14:47.941389+0000","last_became_active":"2023-07-14T18:14:17.927233+0000","last_became_peered":"2023-07-14T18:14:17.927233+0000","last_unstale":"2023-07-14T18:14:47.941389+0000","last_undegraded":"2023-07-14T18:14:47.941389+0000","last_fullsized":"2023-07-14T18:14:47.941389+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:12:03.525006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946334+0000","last_change":"2023-07-14T18:14:17.932317+0000","last_active":"2023-07-14T18:14:48.946334+0000","last_peered":"2023-07-14T18:14:48.946334+0000","last_clean":"2023-07-14T18:14:48.946334+0000","last_became_active":"2023-07-14T18:14:17.931973+0000","last_became_peered":"2023-07-14T18:14:17.931973+0000","last_unstale":"2023-07-14T18:14:48.946334+0000","last_undegraded":"2023-07-14T18:14:48.946334+0000","last_fullsized":"2023-07-14T18:14:48.946334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:01:51.245001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281026,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5256,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":71,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":188416,"data_stored":41080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416402,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":488509,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.39900000000000002}]},{"osd":1,"last update":"Fri Jul 14 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.70699999999999996}]},{"osd":2,"last update":"Fri Jul 14 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.502},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.52900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":4,"last update":"Fri Jul 14 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.752}]},{"osd":5,"last update":"Fri Jul 14 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.60699999999999998}]},{"osd":6,"last update":"Fri Jul 14 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.55100000000000005}]},{"osd":7,"last update":"Fri Jul 14 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.65700000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":9,"last update":"Fri Jul 14 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.83199999999999996}]},{"osd":10,"last update":"Fri Jul 14 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.81200000000000006}]}]},{"osd":10,"up_from":72,"seq":309237645334,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":1062876,"data_compressed":47436,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":1,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.52500000000000002}]},{"osd":2,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.75},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.503},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.63400000000000001}]},{"osd":4,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.81399999999999995}]},{"osd":5,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.71999999999999997}]},{"osd":6,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.78100000000000003}]},{"osd":7,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.42199999999999999}]},{"osd":8,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.85099999999999998}]},{"osd":9,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.91400000000000003}]},{"osd":11,"last update":"Fri Jul 14 18:14:32 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.86499999999999999}]}]},{"osd":9,"up_from":67,"seq":287762808859,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":469433,"data_compressed":11692,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.59199999999999997}]},{"osd":1,"last update":"Fri Jul 14 18:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.879}]},{"osd":2,"last update":"Fri Jul 14 18:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.64900000000000002}]},{"osd":3,"last update":"Fri Jul 14 18:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.53900000000000003}]},{"osd":4,"last update":"Fri Jul 14 18:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.66700000000000004}]},{"osd":5,"last update":"Fri Jul 14 18:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.82799999999999996}]},{"osd":6,"last update":"Fri Jul 14 18:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.75900000000000001}]},{"osd":7,"last update":"Fri Jul 14 18:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.86599999999999999}]},{"osd":8,"last update":"Fri Jul 14 18:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.84899999999999998}]},{"osd":10,"last update":"Fri Jul 14 18:14:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.95499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]}]},{"osd":8,"up_from":59,"seq":253403070496,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":469455,"data_compressed":11692,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81899999999999995,"5min":1.238,"15min":1.238},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56299999999999994,"15min":0.56000000000000005},"min":{"1min":0.40300000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79700000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.86799999999999999}]},{"osd":1,"last update":"Fri Jul 14 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56599999999999995,"15min":0.56000000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.58499999999999996,"15min":0.58099999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.68300000000000005}]},{"osd":2,"last update":"Fri Jul 14 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47199999999999998,"15min":0.47099999999999997},"min":{"1min":0.308,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.71399999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.504,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.57799999999999996}]},{"osd":3,"last update":"Fri Jul 14 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.625,"15min":0.622},"min":{"1min":0.46000000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84999999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.81399999999999995}]},{"osd":4,"last update":"Fri Jul 14 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.624,"15min":0.622},"min":{"1min":0.41799999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.65800000000000003,"15min":0.65400000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.83299999999999996}]},{"osd":5,"last update":"Fri Jul 14 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.38300000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.375,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.75600000000000001}]},{"osd":6,"last update":"Fri Jul 14 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.71699999999999997,"15min":0.72099999999999997},"min":{"1min":0.42399999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92900000000000005,"5min":1.29,"15min":1.29},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.755,"5min":0.71499999999999997,"15min":0.70799999999999996},"min":{"1min":0.55200000000000005,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.90600000000000003}]},{"osd":7,"last update":"Fri Jul 14 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.70499999999999996,"15min":0.69999999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.95099999999999996}]},{"osd":9,"last update":"Fri Jul 14 18:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":1.02},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.91900000000000004}]},{"osd":10,"last update":"Fri Jul 14 18:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":1.002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]}]},{"osd":7,"up_from":51,"seq":219043332131,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":1079451,"data_compressed":53365,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.755,"15min":0.77300000000000002},"min":{"1min":0.38200000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.024,"5min":5.843,"15min":5.843},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.51100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.81299999999999994,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.40899999999999997}]},{"osd":1,"last update":"Fri Jul 14 18:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42399999999999999,"15min":0.42599999999999999},"min":{"1min":0.246,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.621,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.27900000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.45900000000000002}]},{"osd":2,"last update":"Fri Jul 14 18:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.41499999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.38700000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.61799999999999999}]},{"osd":3,"last update":"Fri Jul 14 18:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.65300000000000002,"15min":0.64800000000000002},"min":{"1min":0.46899999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.86899999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.66200000000000003,"15min":0.66400000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77300000000000002,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.76300000000000001}]},{"osd":4,"last update":"Fri Jul 14 18:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76700000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.499,"15min":0.497},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70499999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.41699999999999998}]},{"osd":5,"last update":"Fri Jul 14 18:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68200000000000005,"15min":0.68400000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88100000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.67200000000000004}]},{"osd":6,"last update":"Fri Jul 14 18:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":1.109,"15min":1.175},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.85899999999999999,"5min":13.189,"15min":13.189},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.499,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88100000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.71299999999999997}]},{"osd":8,"last update":"Fri Jul 14 18:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.70299999999999996,"15min":0.69299999999999995},"min":{"1min":0.51700000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69299999999999995,"15min":0.68999999999999995},"min":{"1min":0.46999999999999997,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.55700000000000005}]},{"osd":9,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.94599999999999995,"15min":0.97599999999999998},"min":{"1min":0.58099999999999996,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.91000000000000003,"5min":5.8250000000000002,"15min":5.8250000000000002},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.77500000000000002,"15min":0.77100000000000002},"min":{"1min":0.61799999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.78400000000000003}]},{"osd":10,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.748},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.73099999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":950272,"data_stored":468851,"data_compressed":11692,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.46500000000000002,"15min":0.45500000000000002},"min":{"1min":0.32700000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.61299999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.438},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.49299999999999999,"15min":0.48199999999999998},"min":{"1min":0.35399999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65200000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.56000000000000005}]},{"osd":1,"last update":"Fri Jul 14 18:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.64700000000000002,"15min":0.64000000000000001},"min":{"1min":0.46000000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83499999999999996,"5min":0.87,"15min":0.87},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67700000000000005,"15min":0.67400000000000004},"min":{"1min":0.438,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91400000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.63300000000000001}]},{"osd":2,"last update":"Fri Jul 14 18:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63600000000000001,"15min":0.626},"min":{"1min":0.46100000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0309999999999999,"5min":1.087,"15min":1.087},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68400000000000005,"15min":0.67800000000000005},"min":{"1min":0.52200000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.86099999999999999,"5min":1.097,"15min":1.097},"last":0.65200000000000002}]},{"osd":3,"last update":"Fri Jul 14 18:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.39200000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84499999999999997,"5min":1.03,"15min":1.03},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.64100000000000001,"15min":0.64900000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83699999999999997,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.83299999999999996}]},{"osd":4,"last update":"Fri Jul 14 18:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86199999999999999,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71799999999999997,"15min":0.72099999999999997},"min":{"1min":0.48099999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.80200000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.80100000000000005}]},{"osd":5,"last update":"Fri Jul 14 18:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.75700000000000001,"15min":0.76300000000000001},"min":{"1min":0.52300000000000002,"5min":0.435,"15min":0.435},"max":{"1min":1.0429999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.52900000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94899999999999995,"5min":1.163,"15min":1.163},"last":0.78200000000000003}]},{"osd":7,"last update":"Fri Jul 14 18:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.73299999999999998,"15min":0.72499999999999998},"min":{"1min":0.57499999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.752},{"interface":"front","average":{"1min":0.75,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.85699999999999998}]},{"osd":8,"last update":"Fri Jul 14 18:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.80300000000000005,"15min":0.80700000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0900000000000001,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.746,"15min":0.75},"min":{"1min":0.56499999999999995,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.96599999999999997,"5min":1.079,"15min":1.079},"last":0.997}]},{"osd":9,"last update":"Fri Jul 14 18:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.88800000000000001}]},{"osd":10,"last update":"Fri Jul 14 18:14:11 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.873},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":1.028}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":492303,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53400000000000003,"15min":0.50800000000000001},"min":{"1min":0.40899999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.79000000000000004,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.55600000000000005,"15min":0.52500000000000002},"min":{"1min":0.39600000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.0660000000000001,"5min":1.272,"15min":1.272},"last":0.69699999999999995}]},{"osd":2,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.56100000000000005,"15min":0.53300000000000003},"min":{"1min":0.434,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.54500000000000004,"15min":0.51700000000000002},"min":{"1min":0.50700000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77800000000000002,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.72999999999999998}]},{"osd":3,"last update":"Fri Jul 14 18:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.56499999999999995,"15min":0.52400000000000002},"min":{"1min":0.39700000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.93500000000000005,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.60199999999999998,"15min":0.58199999999999996},"min":{"1min":0.45600000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91300000000000003,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.60099999999999998}]},{"osd":4,"last update":"Fri Jul 14 18:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.46100000000000002,"15min":0.44700000000000001},"min":{"1min":0.29699999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.45600000000000002,"15min":0.44400000000000001},"min":{"1min":0.316,"5min":0.255,"15min":0.255},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.45800000000000002}]},{"osd":5,"last update":"Fri Jul 14 18:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82099999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.439,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.66900000000000004}]},{"osd":6,"last update":"Fri Jul 14 18:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.67200000000000004,"15min":0.66900000000000004},"min":{"1min":0.46000000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90200000000000002,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.629},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67600000000000005,"15min":0.66800000000000004},"min":{"1min":0.47499999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.0029999999999999,"5min":1.23,"15min":1.23},"last":0.55400000000000005}]},{"osd":7,"last update":"Fri Jul 14 18:14:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55600000000000005,"15min":0.54800000000000004},"min":{"1min":0.35399999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57899999999999996,"15min":0.57299999999999995},"min":{"1min":0.36499999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.68700000000000006}]},{"osd":8,"last update":"Fri Jul 14 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.64500000000000002,"15min":0.63600000000000001},"min":{"1min":0.42799999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.69099999999999995,"15min":0.67900000000000005},"min":{"1min":0.51900000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.84699999999999998}]},{"osd":9,"last update":"Fri Jul 14 18:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.90000000000000002}]},{"osd":10,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.875},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.79300000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":488621,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 18:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.56999999999999995,"15min":0.54200000000000004},"min":{"1min":0.35999999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.58599999999999997,"15min":0.57199999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.57499999999999996}]},{"osd":2,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.59699999999999998,"15min":0.55500000000000005},"min":{"1min":0.39800000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61199999999999999,"15min":0.57199999999999995},"min":{"1min":0.434,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.754}]},{"osd":3,"last update":"Fri Jul 14 18:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.443,"15min":0.42799999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.438},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.48299999999999998,"15min":0.46300000000000002},"min":{"1min":0.36399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.38600000000000001}]},{"osd":4,"last update":"Fri Jul 14 18:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.57799999999999996,"15min":0.55700000000000005},"min":{"1min":0.41799999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.64000000000000001,"15min":0.61699999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.627}]},{"osd":5,"last update":"Fri Jul 14 18:14:20 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.71599999999999997,"15min":0.68400000000000005},"min":{"1min":0.47699999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":3.3740000000000001,"5min":3.3740000000000001,"15min":3.3740000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.749,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.78400000000000003}]},{"osd":6,"last update":"Fri Jul 14 18:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64100000000000001,"15min":0.61699999999999999},"min":{"1min":0.44700000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.66400000000000003,"15min":0.64000000000000001},"min":{"1min":0.433,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.52100000000000002}]},{"osd":7,"last update":"Fri Jul 14 18:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.68500000000000005,"15min":0.67500000000000004},"min":{"1min":0.58399999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.76100000000000001,"15min":0.74099999999999999},"min":{"1min":0.56599999999999995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":0.74099999999999999}]},{"osd":8,"last update":"Fri Jul 14 18:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.80400000000000005,"15min":0.79400000000000004},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.874,"5min":0.78000000000000003,"15min":0.76400000000000001},"min":{"1min":0.58199999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.82199999999999995}]},{"osd":9,"last update":"Fri Jul 14 18:14:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.77000000000000002,"15min":0.77200000000000002},"min":{"1min":0.57099999999999995,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":1.157,"15min":1.2150000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0389999999999999,"5min":7.8689999999999998,"15min":7.8689999999999998},"last":0.53900000000000003}]},{"osd":10,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.76900000000000002}]},{"osd":11,"last update":"Fri Jul 14 18:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.80400000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":472679,"data_compressed":11692,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.36099999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.54100000000000004,"15min":0.51900000000000002},"min":{"1min":0.39700000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.84099999999999997}]},{"osd":1,"last update":"Fri Jul 14 18:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.54400000000000004,"15min":0.50900000000000001},"min":{"1min":0.41199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.54300000000000004,"15min":0.50900000000000001},"min":{"1min":0.39000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.59699999999999998}]},{"osd":3,"last update":"Fri Jul 14 18:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60699999999999998,"15min":0.57299999999999995},"min":{"1min":0.441,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.877,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65100000000000002,"15min":0.65700000000000003},"min":{"1min":0.53200000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81299999999999994,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":0.79100000000000004}]},{"osd":4,"last update":"Fri Jul 14 18:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.59199999999999997,"15min":0.57199999999999995},"min":{"1min":0.47999999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88400000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.623,"15min":0.60299999999999998},"min":{"1min":0.443,"5min":0.433,"15min":0.433},"max":{"1min":0.82799999999999996,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.66900000000000004}]},{"osd":5,"last update":"Fri Jul 14 18:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.63600000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.48799999999999999,"15min":0.47999999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.51900000000000002}]},{"osd":6,"last update":"Fri Jul 14 18:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64400000000000002,"15min":0.63400000000000001},"min":{"1min":0.52100000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84199999999999997,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.65600000000000003,"15min":0.63100000000000001},"min":{"1min":0.54600000000000004,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84799999999999998,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.94599999999999995}]},{"osd":7,"last update":"Fri Jul 14 18:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.52400000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.92100000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.65400000000000003,"15min":0.65600000000000003},"min":{"1min":0.44900000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.873,"5min":1.014,"15min":1.014},"last":0.93100000000000005}]},{"osd":8,"last update":"Fri Jul 14 18:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63500000000000001,"15min":0.63900000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.873,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.63200000000000001,"15min":0.63300000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.74099999999999999}]},{"osd":9,"last update":"Fri Jul 14 18:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":1.069},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":1.0029999999999999}]},{"osd":10,"last update":"Fri Jul 14 18:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.82199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]}]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":478555,"data_compressed":14651,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.43099999999999999,"15min":0.39100000000000001},"min":{"1min":0.32700000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45000000000000001,"15min":0.41399999999999998},"min":{"1min":0.34000000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.92400000000000004,"5min":0.998,"15min":0.998},"last":0.45600000000000002}]},{"osd":1,"last update":"Fri Jul 14 18:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.59799999999999998,"15min":0.57599999999999996},"min":{"1min":0.44500000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.85299999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.625,"15min":0.58999999999999997},"min":{"1min":0.54500000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.032,"5min":1.29,"15min":1.29},"last":0.752}]},{"osd":2,"last update":"Fri Jul 14 18:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.61399999999999999,"15min":0.57899999999999996},"min":{"1min":0.52100000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.81599999999999995,"5min":1.087,"15min":1.087},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61799999999999999,"15min":0.59999999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95499999999999996,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.63500000000000001}]},{"osd":4,"last update":"Fri Jul 14 18:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.61599999999999999,"15min":0.58499999999999996},"min":{"1min":0.47899999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.1200000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.65900000000000003,"15min":0.63300000000000001},"min":{"1min":0.60499999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.135,"5min":1.153,"15min":1.153},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Jul 14 18:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.64600000000000002,"15min":0.63},"min":{"1min":0.54800000000000004,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.621},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.63900000000000001,"15min":0.61299999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.93100000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.80800000000000005}]},{"osd":6,"last update":"Fri Jul 14 18:14:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.621,"15min":0.59899999999999998},"min":{"1min":0.40699999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.432},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65400000000000003,"15min":0.622},"min":{"1min":0.47999999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.39500000000000002}]},{"osd":7,"last update":"Fri Jul 14 18:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.70399999999999996,"15min":0.68600000000000005},"min":{"1min":0.57199999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.71099999999999997,"15min":0.69899999999999995},"min":{"1min":0.59399999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.76700000000000002}]},{"osd":8,"last update":"Fri Jul 14 18:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.72799999999999998}]},{"osd":9,"last update":"Fri Jul 14 18:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.498}]},{"osd":10,"last update":"Fri Jul 14 18:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.66300000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":489097,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.48999999999999999,"15min":0.46000000000000002},"min":{"1min":0.39700000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.83799999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.50800000000000001,"15min":0.47799999999999998},"min":{"1min":0.46700000000000003,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.747,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.39600000000000002}]},{"osd":1,"last update":"Fri Jul 14 18:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.36899999999999999,"15min":0.35499999999999998},"min":{"1min":0.26900000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":3.7890000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.38600000000000001,"15min":0.377},"min":{"1min":0.28699999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.71899999999999997,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":1.3320000000000001}]},{"osd":2,"last update":"Fri Jul 14 18:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.55100000000000005,"15min":0.54000000000000004},"min":{"1min":0.42499999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73899999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":2.3090000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.56399999999999995,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.88800000000000001,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":3.6989999999999998}]},{"osd":3,"last update":"Fri Jul 14 18:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.54500000000000004,"15min":0.51700000000000002},"min":{"1min":0.48699999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.91800000000000004,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.54300000000000004,"15min":0.52300000000000002},"min":{"1min":0.46400000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82799999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":2.5640000000000001}]},{"osd":5,"last update":"Fri Jul 14 18:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.59199999999999997,"15min":0.56000000000000005},"min":{"1min":0.51000000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":3.7490000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.59799999999999998,"15min":0.56599999999999995},"min":{"1min":0.53400000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.72899999999999998}]},{"osd":6,"last update":"Fri Jul 14 18:14:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67000000000000004,"15min":0.65800000000000003},"min":{"1min":0.50800000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.98799999999999999,"5min":1.075,"15min":1.075},"last":3.4780000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.68300000000000005,"15min":0.66600000000000004},"min":{"1min":0.498,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.006,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":3.8620000000000001}]},{"osd":7,"last update":"Fri Jul 14 18:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.58699999999999997,"15min":0.56999999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.97399999999999998,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":3.8999999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.59799999999999998,"15min":0.58399999999999996},"min":{"1min":0.40000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0249999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":3.2250000000000001}]},{"osd":8,"last update":"Fri Jul 14 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64600000000000002,"15min":0.63900000000000001},"min":{"1min":0.48599999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.877,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":2.359},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.755,"15min":0.749},"min":{"1min":0.52500000000000002,"5min":0.5,"15min":0.5},"max":{"1min":0.96799999999999997,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":3.1739999999999999}]},{"osd":9,"last update":"Fri Jul 14 18:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":2.3889999999999998}]},{"osd":10,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":3.8239999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":2.52}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.548}]}]},{"osd":5,"up_from":38,"seq":163208757293,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":1059070,"data_compressed":47436,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:14:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55600000000000005,"15min":0.55100000000000005},"min":{"1min":0.39000000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.626},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56999999999999995,"15min":0.57899999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.876,"5min":1.645,"15min":1.645},"last":0.61299999999999999}]},{"osd":1,"last update":"Fri Jul 14 18:14:18 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.56699999999999995,"15min":0.55100000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.36299999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80400000000000005,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.48799999999999999}]},{"osd":2,"last update":"Fri Jul 14 18:14:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.41999999999999998,"15min":0.42399999999999999},"min":{"1min":0.26100000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.47599999999999998,"15min":0.49199999999999999},"min":{"1min":0.27500000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.72299999999999998,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.39400000000000002}]},{"osd":3,"last update":"Fri Jul 14 18:14:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60899999999999999,"15min":0.62},"min":{"1min":0.378,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84599999999999997,"5min":1.591,"15min":1.591},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.40500000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.81299999999999994,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.71099999999999997}]},{"osd":4,"last update":"Fri Jul 14 18:14:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61099999999999999,"15min":0.61499999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93400000000000005,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84499999999999997,"5min":1.504,"15min":1.504},"last":0.503}]},{"osd":6,"last update":"Fri Jul 14 18:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.46700000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0840000000000001,"5min":1.627,"15min":1.627},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.64300000000000002,"15min":0.63300000000000001},"min":{"1min":0.47799999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.75600000000000001}]},{"osd":7,"last update":"Fri Jul 14 18:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91700000000000004,"5min":1.554,"15min":1.554},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65200000000000002,"15min":0.64900000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.66400000000000003}]},{"osd":8,"last update":"Fri Jul 14 18:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58599999999999997,"15min":0.57699999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.38300000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.69399999999999995}]},{"osd":9,"last update":"Fri Jul 14 18:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.745},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.81699999999999995}]},{"osd":10,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.79100000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34102,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34102,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34102,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":26386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T18:14:53.248 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T18:14:53.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:52 smithi161 ceph-mon[128993]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:14:53.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:52 smithi161 ceph-mon[128993]: pgmap v292: 129 pgs: 3 peering, 126 active+clean; 582 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1.1 KiB/s wr, 3 op/s 2023-07-14T18:14:53.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:52 smithi161 ceph-mon[128993]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:14:53.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:52 smithi161 ceph-mon[128993]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-07-14T18:14:53.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:52 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/4139464148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:14:53.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:52 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3341175425' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:14:53.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:52 smithi171 ceph-mon[129241]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:14:53.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:52 smithi171 ceph-mon[129241]: pgmap v292: 129 pgs: 3 peering, 126 active+clean; 582 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1.1 KiB/s wr, 3 op/s 2023-07-14T18:14:53.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:52 smithi171 ceph-mon[129241]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:14:53.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:52 smithi171 ceph-mon[129241]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-07-14T18:14:53.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:52 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/4139464148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:14:53.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:52 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3341175425' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:14:53.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.429+0000 7fb51ddff700 1 -- 172.21.15.133:0/3173820180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5180ffc30 msgr2=0x7fb518100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:53.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.429+0000 7fb51ddff700 1 --2- 172.21.15.133:0/3173820180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5180ffc30 0x7fb518100040 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb508009990 tx=0x7fb508009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:53.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.429+0000 7fb51ddff700 1 -- 172.21.15.133:0/3173820180 shutdown_connections 2023-07-14T18:14:53.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.429+0000 7fb51ddff700 1 --2- 172.21.15.133:0/3173820180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5180ffc30 0x7fb518100040 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:53.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.429+0000 7fb51ddff700 1 --2- 172.21.15.133:0/3173820180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5181017b0 0x7fb518108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:53.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.429+0000 7fb51ddff700 1 --2- 172.21.15.133:0/3173820180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb518100e20 0x7fb518101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.429+0000 7fb51ddff700 1 -- 172.21.15.133:0/3173820180 >> 172.21.15.133:0/3173820180 conn(0x7fb5180fb210 msgr2=0x7fb5180fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.429+0000 7fb51ddff700 1 -- 172.21.15.133:0/3173820180 shutdown_connections 2023-07-14T18:14:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.429+0000 7fb51ddff700 1 -- 172.21.15.133:0/3173820180 wait complete. 2023-07-14T18:14:53.476 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.430+0000 7fb51ddff700 1 Processor -- start 2023-07-14T18:14:53.477 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.430+0000 7fb51ddff700 1 -- start start 2023-07-14T18:14:53.477 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.430+0000 7fb51ddff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5180ffc30 0x7fb518197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:53.477 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.430+0000 7fb51ddff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb518100e20 0x7fb518198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:53.477 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.430+0000 7fb51ddff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5181017b0 0x7fb51819f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:53.477 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.430+0000 7fb51ddff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51819f6c0 con 0x7fb5180ffc30 2023-07-14T18:14:53.477 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.430+0000 7fb51ddff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51819f800 con 0x7fb518100e20 2023-07-14T18:14:53.477 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.430+0000 7fb51ddff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51819f990 con 0x7fb5181017b0 2023-07-14T18:14:53.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.430+0000 7fb5177fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5180ffc30 0x7fb518197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:53.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.431+0000 7fb5177fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5180ffc30 0x7fb518197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39400/0 (socket says 172.21.15.133:39400) 2023-07-14T18:14:53.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.431+0000 7fb5177fe700 1 -- 172.21.15.133:0/3025291429 learned_addr learned my addr 172.21.15.133:0/3025291429 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:53.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.431+0000 7fb516ffd700 1 --2- 172.21.15.133:0/3025291429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb518100e20 0x7fb518198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:53.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.431+0000 7fb517fff700 1 --2- 172.21.15.133:0/3025291429 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5181017b0 0x7fb51819f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:53.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.431+0000 7fb5177fe700 1 -- 172.21.15.133:0/3025291429 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5181017b0 msgr2=0x7fb51819f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:53.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.431+0000 7fb5177fe700 1 --2- 172.21.15.133:0/3025291429 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5181017b0 0x7fb51819f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:53.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.431+0000 7fb5177fe700 1 -- 172.21.15.133:0/3025291429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb518100e20 msgr2=0x7fb518198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:53.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.431+0000 7fb5177fe700 1 --2- 172.21.15.133:0/3025291429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb518100e20 0x7fb518198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:53.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.431+0000 7fb5177fe700 1 -- 172.21.15.133:0/3025291429 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50c009520 con 0x7fb5180ffc30 2023-07-14T18:14:53.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.431+0000 7fb5177fe700 1 --2- 172.21.15.133:0/3025291429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5180ffc30 0x7fb518197b40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb50800b620 tx=0x7fb508005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:53.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.431+0000 7fb514ff9700 1 -- 172.21.15.133:0/3025291429 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50801d070 con 0x7fb5180ffc30 2023-07-14T18:14:53.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.431+0000 7fb51ddff700 1 -- 172.21.15.133:0/3025291429 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5080095f0 con 0x7fb5180ffc30 2023-07-14T18:14:53.480 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.431+0000 7fb514ff9700 1 -- 172.21.15.133:0/3025291429 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb508021410 con 0x7fb5180ffc30 2023-07-14T18:14:53.480 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.431+0000 7fb514ff9700 1 -- 172.21.15.133:0/3025291429 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb508017650 con 0x7fb5180ffc30 2023-07-14T18:14:53.480 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.432+0000 7fb51ddff700 1 -- 172.21.15.133:0/3025291429 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51819fea0 con 0x7fb5180ffc30 2023-07-14T18:14:53.480 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.433+0000 7fb51ddff700 1 -- 172.21.15.133:0/3025291429 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb51810b450 con 0x7fb5180ffc30 2023-07-14T18:14:53.481 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.435+0000 7fb514ff9700 1 -- 172.21.15.133:0/3025291429 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fb508021580 con 0x7fb5180ffc30 2023-07-14T18:14:53.481 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.436+0000 7fb514ff9700 1 --2- 172.21.15.133:0/3025291429 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb5000a1220 0x7fb5000a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:53.481 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.436+0000 7fb514ff9700 1 -- 172.21.15.133:0/3025291429 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb5080ce970 con 0x7fb5180ffc30 2023-07-14T18:14:53.481 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.436+0000 7fb516ffd700 1 --2- 172.21.15.133:0/3025291429 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb5000a1220 0x7fb5000a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:53.481 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.437+0000 7fb516ffd700 1 --2- 172.21.15.133:0/3025291429 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb5000a1220 0x7fb5000a36d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009df0 tx=0x7fb50c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:53.482 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.438+0000 7fb514ff9700 1 -- 172.21.15.133:0/3025291429 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fb508097c80 con 0x7fb5180ffc30 2023-07-14T18:14:53.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.637+0000 7fb51ddff700 1 -- 172.21.15.133:0/3025291429 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb518107de0 con 0x7fb5000a1220 2023-07-14T18:14:53.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.650+0000 7fb514ff9700 1 -- 172.21.15.133:0/3025291429 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422936 (secure 0 0 0) 0x7fb518107de0 con 0x7fb5000a1220 2023-07-14T18:14:53.697 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:14:53.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.655+0000 7fb51ddff700 1 -- 172.21.15.133:0/3025291429 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb5000a1220 msgr2=0x7fb5000a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:53.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.655+0000 7fb51ddff700 1 --2- 172.21.15.133:0/3025291429 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb5000a1220 0x7fb5000a36d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009df0 tx=0x7fb50c009260 comp rx=0 tx=0).stop 2023-07-14T18:14:53.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.655+0000 7fb51ddff700 1 -- 172.21.15.133:0/3025291429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5180ffc30 msgr2=0x7fb518197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:53.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.655+0000 7fb51ddff700 1 --2- 172.21.15.133:0/3025291429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5180ffc30 0x7fb518197b40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb50800b620 tx=0x7fb508005740 comp rx=0 tx=0).stop 2023-07-14T18:14:53.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.655+0000 7fb51ddff700 1 -- 172.21.15.133:0/3025291429 shutdown_connections 2023-07-14T18:14:53.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.655+0000 7fb51ddff700 1 --2- 172.21.15.133:0/3025291429 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb5000a1220 0x7fb5000a36d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:53.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.655+0000 7fb51ddff700 1 --2- 172.21.15.133:0/3025291429 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5181017b0 0x7fb51819f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:53.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.655+0000 7fb51ddff700 1 --2- 172.21.15.133:0/3025291429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5180ffc30 0x7fb518197b40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:53.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.655+0000 7fb51ddff700 1 --2- 172.21.15.133:0/3025291429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb518100e20 0x7fb518198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:53.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.655+0000 7fb51ddff700 1 -- 172.21.15.133:0/3025291429 >> 172.21.15.133:0/3025291429 conn(0x7fb5180fb210 msgr2=0x7fb5181066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:53.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.655+0000 7fb51ddff700 1 -- 172.21.15.133:0/3025291429 shutdown_connections 2023-07-14T18:14:53.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:53.655+0000 7fb51ddff700 1 -- 172.21.15.133:0/3025291429 wait complete. 2023-07-14T18:14:53.701 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2023-07-14T18:14:53.717 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2023-07-14T18:14:53.437102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":632787,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":164,"num_read_kb":176,"num_write":325,"num_write_kb":3022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":263,"ondisk_log_size":263,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":426,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336088,"kb_used_data":11852,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124605736,"statfs":{"total":1151940427776,"available":1151596273664,"internally_reserved":0,"allocated":12136448,"data_stored":7722726,"data_compressed":315714,"data_compressed_allocated":425984,"data_compressed_original":2351104,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":37827,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":38,"num_read_kb":67,"num_write":95,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002059"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945666+0000","last_change":"2023-07-14T18:14:17.927179+0000","last_active":"2023-07-14T18:14:48.945666+0000","last_peered":"2023-07-14T18:14:48.945666+0000","last_clean":"2023-07-14T18:14:48.945666+0000","last_became_active":"2023-07-14T18:14:17.926880+0000","last_became_peered":"2023-07-14T18:14:17.926880+0000","last_unstale":"2023-07-14T18:14:48.945666+0000","last_undegraded":"2023-07-14T18:14:48.945666+0000","last_fullsized":"2023-07-14T18:14:48.945666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:26:26.468709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541115+0000","last_change":"2023-07-14T18:14:17.933834+0000","last_active":"2023-07-14T18:14:49.541115+0000","last_peered":"2023-07-14T18:14:49.541115+0000","last_clean":"2023-07-14T18:14:49.541115+0000","last_became_active":"2023-07-14T18:14:17.933529+0000","last_became_peered":"2023-07-14T18:14:17.933529+0000","last_unstale":"2023-07-14T18:14:49.541115+0000","last_undegraded":"2023-07-14T18:14:49.541115+0000","last_fullsized":"2023-07-14T18:14:49.541115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:34:13.388837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540772+0000","last_change":"2023-07-14T18:14:17.928400+0000","last_active":"2023-07-14T18:14:49.540772+0000","last_peered":"2023-07-14T18:14:49.540772+0000","last_clean":"2023-07-14T18:14:49.540772+0000","last_became_active":"2023-07-14T18:14:17.928077+0000","last_became_peered":"2023-07-14T18:14:17.928077+0000","last_unstale":"2023-07-14T18:14:49.540772+0000","last_undegraded":"2023-07-14T18:14:49.540772+0000","last_fullsized":"2023-07-14T18:14:49.540772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:24.522613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.947528+0000","last_change":"2023-07-14T18:14:17.932623+0000","last_active":"2023-07-14T18:14:48.947528+0000","last_peered":"2023-07-14T18:14:48.947528+0000","last_clean":"2023-07-14T18:14:48.947528+0000","last_became_active":"2023-07-14T18:14:17.932298+0000","last_became_peered":"2023-07-14T18:14:17.932298+0000","last_unstale":"2023-07-14T18:14:48.947528+0000","last_undegraded":"2023-07-14T18:14:48.947528+0000","last_fullsized":"2023-07-14T18:14:48.947528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:17:46.564398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945288+0000","last_change":"2023-07-14T18:14:17.927109+0000","last_active":"2023-07-14T18:14:48.945288+0000","last_peered":"2023-07-14T18:14:48.945288+0000","last_clean":"2023-07-14T18:14:48.945288+0000","last_became_active":"2023-07-14T18:14:17.926800+0000","last_became_peered":"2023-07-14T18:14:17.926800+0000","last_unstale":"2023-07-14T18:14:48.945288+0000","last_undegraded":"2023-07-14T18:14:48.945288+0000","last_fullsized":"2023-07-14T18:14:48.945288+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:43:24.990024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541350+0000","last_change":"2023-07-14T18:14:17.935380+0000","last_active":"2023-07-14T18:14:49.541350+0000","last_peered":"2023-07-14T18:14:49.541350+0000","last_clean":"2023-07-14T18:14:49.541350+0000","last_became_active":"2023-07-14T18:14:17.935099+0000","last_became_peered":"2023-07-14T18:14:17.935099+0000","last_unstale":"2023-07-14T18:14:49.541350+0000","last_undegraded":"2023-07-14T18:14:49.541350+0000","last_fullsized":"2023-07-14T18:14:49.541350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:04:06.339952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.943679+0000","last_change":"2023-07-14T18:14:17.930207+0000","last_active":"2023-07-14T18:14:48.943679+0000","last_peered":"2023-07-14T18:14:48.943679+0000","last_clean":"2023-07-14T18:14:48.943679+0000","last_became_active":"2023-07-14T18:14:17.929898+0000","last_became_peered":"2023-07-14T18:14:17.929898+0000","last_unstale":"2023-07-14T18:14:48.943679+0000","last_undegraded":"2023-07-14T18:14:48.943679+0000","last_fullsized":"2023-07-14T18:14:48.943679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:33:33.679907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.936854+0000","last_change":"2023-07-14T18:14:17.928645+0000","last_active":"2023-07-14T18:14:47.936854+0000","last_peered":"2023-07-14T18:14:47.936854+0000","last_clean":"2023-07-14T18:14:47.936854+0000","last_became_active":"2023-07-14T18:14:17.928288+0000","last_became_peered":"2023-07-14T18:14:17.928288+0000","last_unstale":"2023-07-14T18:14:47.936854+0000","last_undegraded":"2023-07-14T18:14:47.936854+0000","last_fullsized":"2023-07-14T18:14:47.936854+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:09:18.003654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.948678+0000","last_change":"2023-07-14T18:14:17.925888+0000","last_active":"2023-07-14T18:14:48.948678+0000","last_peered":"2023-07-14T18:14:48.948678+0000","last_clean":"2023-07-14T18:14:48.948678+0000","last_became_active":"2023-07-14T18:14:17.925580+0000","last_became_peered":"2023-07-14T18:14:17.925580+0000","last_unstale":"2023-07-14T18:14:48.948678+0000","last_undegraded":"2023-07-14T18:14:48.948678+0000","last_fullsized":"2023-07-14T18:14:48.948678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:28:42.610213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541751+0000","last_change":"2023-07-14T18:14:17.924440+0000","last_active":"2023-07-14T18:14:49.541751+0000","last_peered":"2023-07-14T18:14:49.541751+0000","last_clean":"2023-07-14T18:14:49.541751+0000","last_became_active":"2023-07-14T18:14:17.923839+0000","last_became_peered":"2023-07-14T18:14:17.923839+0000","last_unstale":"2023-07-14T18:14:49.541751+0000","last_undegraded":"2023-07-14T18:14:49.541751+0000","last_fullsized":"2023-07-14T18:14:49.541751+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:11:20.578050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944134+0000","last_change":"2023-07-14T18:14:17.931067+0000","last_active":"2023-07-14T18:14:48.944134+0000","last_peered":"2023-07-14T18:14:48.944134+0000","last_clean":"2023-07-14T18:14:48.944134+0000","last_became_active":"2023-07-14T18:14:17.930633+0000","last_became_peered":"2023-07-14T18:14:17.930633+0000","last_unstale":"2023-07-14T18:14:48.944134+0000","last_undegraded":"2023-07-14T18:14:48.944134+0000","last_fullsized":"2023-07-14T18:14:48.944134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:35:03.441365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.947103+0000","last_change":"2023-07-14T18:14:17.932770+0000","last_active":"2023-07-14T18:14:48.947103+0000","last_peered":"2023-07-14T18:14:48.947103+0000","last_clean":"2023-07-14T18:14:48.947103+0000","last_became_active":"2023-07-14T18:14:17.932438+0000","last_became_peered":"2023-07-14T18:14:17.932438+0000","last_unstale":"2023-07-14T18:14:48.947103+0000","last_undegraded":"2023-07-14T18:14:48.947103+0000","last_fullsized":"2023-07-14T18:14:48.947103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:45:58.878384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942853+0000","last_change":"2023-07-14T18:14:17.938213+0000","last_active":"2023-07-14T18:14:48.942853+0000","last_peered":"2023-07-14T18:14:48.942853+0000","last_clean":"2023-07-14T18:14:48.942853+0000","last_became_active":"2023-07-14T18:14:17.937565+0000","last_became_peered":"2023-07-14T18:14:17.937565+0000","last_unstale":"2023-07-14T18:14:48.942853+0000","last_undegraded":"2023-07-14T18:14:48.942853+0000","last_fullsized":"2023-07-14T18:14:48.942853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:18:33.559796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541023+0000","last_change":"2023-07-14T18:14:17.934677+0000","last_active":"2023-07-14T18:14:49.541023+0000","last_peered":"2023-07-14T18:14:49.541023+0000","last_clean":"2023-07-14T18:14:49.541023+0000","last_became_active":"2023-07-14T18:14:17.933622+0000","last_became_peered":"2023-07-14T18:14:17.933622+0000","last_unstale":"2023-07-14T18:14:49.541023+0000","last_undegraded":"2023-07-14T18:14:49.541023+0000","last_fullsized":"2023-07-14T18:14:49.541023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:44:58.693553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540682+0000","last_change":"2023-07-14T18:14:17.935317+0000","last_active":"2023-07-14T18:14:49.540682+0000","last_peered":"2023-07-14T18:14:49.540682+0000","last_clean":"2023-07-14T18:14:49.540682+0000","last_became_active":"2023-07-14T18:14:17.935034+0000","last_became_peered":"2023-07-14T18:14:17.935034+0000","last_unstale":"2023-07-14T18:14:49.540682+0000","last_undegraded":"2023-07-14T18:14:49.540682+0000","last_fullsized":"2023-07-14T18:14:49.540682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:45:23.409187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946950+0000","last_change":"2023-07-14T18:14:17.928222+0000","last_active":"2023-07-14T18:14:48.946950+0000","last_peered":"2023-07-14T18:14:48.946950+0000","last_clean":"2023-07-14T18:14:48.946950+0000","last_became_active":"2023-07-14T18:14:17.927964+0000","last_became_peered":"2023-07-14T18:14:17.927964+0000","last_unstale":"2023-07-14T18:14:48.946950+0000","last_undegraded":"2023-07-14T18:14:48.946950+0000","last_fullsized":"2023-07-14T18:14:48.946950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:44:57.639864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944940+0000","last_change":"2023-07-14T18:14:17.927276+0000","last_active":"2023-07-14T18:14:48.944940+0000","last_peered":"2023-07-14T18:14:48.944940+0000","last_clean":"2023-07-14T18:14:48.944940+0000","last_became_active":"2023-07-14T18:14:17.926952+0000","last_became_peered":"2023-07-14T18:14:17.926952+0000","last_unstale":"2023-07-14T18:14:48.944940+0000","last_undegraded":"2023-07-14T18:14:48.944940+0000","last_fullsized":"2023-07-14T18:14:48.944940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:48:53.419369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945947+0000","last_change":"2023-07-14T18:14:17.937364+0000","last_active":"2023-07-14T18:14:48.945947+0000","last_peered":"2023-07-14T18:14:48.945947+0000","last_clean":"2023-07-14T18:14:48.945947+0000","last_became_active":"2023-07-14T18:14:17.937014+0000","last_became_peered":"2023-07-14T18:14:17.937014+0000","last_unstale":"2023-07-14T18:14:48.945947+0000","last_undegraded":"2023-07-14T18:14:48.945947+0000","last_fullsized":"2023-07-14T18:14:48.945947+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:57:26.423684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541653+0000","last_change":"2023-07-14T18:14:17.920358+0000","last_active":"2023-07-14T18:14:49.541653+0000","last_peered":"2023-07-14T18:14:49.541653+0000","last_clean":"2023-07-14T18:14:49.541653+0000","last_became_active":"2023-07-14T18:14:17.920042+0000","last_became_peered":"2023-07-14T18:14:17.920042+0000","last_unstale":"2023-07-14T18:14:49.541653+0000","last_undegraded":"2023-07-14T18:14:49.541653+0000","last_fullsized":"2023-07-14T18:14:49.541653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:53:36.975517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541158+0000","last_change":"2023-07-14T18:14:22.045376+0000","last_active":"2023-07-14T18:14:49.541158+0000","last_peered":"2023-07-14T18:14:49.541158+0000","last_clean":"2023-07-14T18:14:49.541158+0000","last_became_active":"2023-07-14T18:14:22.044984+0000","last_became_peered":"2023-07-14T18:14:22.044984+0000","last_unstale":"2023-07-14T18:14:49.541158+0000","last_undegraded":"2023-07-14T18:14:49.541158+0000","last_fullsized":"2023-07-14T18:14:49.541158+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:47:15.523409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.894895+0000","last_change":"2023-07-14T18:14:48.954270+0000","last_active":"2023-07-14T18:14:49.894895+0000","last_peered":"2023-07-14T18:14:49.894895+0000","last_clean":"2023-07-14T18:14:49.894895+0000","last_became_active":"2023-07-14T18:14:48.947868+0000","last_became_peered":"2023-07-14T18:14:48.947868+0000","last_unstale":"2023-07-14T18:14:49.894895+0000","last_undegraded":"2023-07-14T18:14:49.894895+0000","last_fullsized":"2023-07-14T18:14:49.894895+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:13:41.862407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.938611+0000","last_change":"2023-07-14T18:14:17.925822+0000","last_active":"2023-07-14T18:14:47.938611+0000","last_peered":"2023-07-14T18:14:47.938611+0000","last_clean":"2023-07-14T18:14:47.938611+0000","last_became_active":"2023-07-14T18:14:17.925508+0000","last_became_peered":"2023-07-14T18:14:17.925508+0000","last_unstale":"2023-07-14T18:14:47.938611+0000","last_undegraded":"2023-07-14T18:14:47.938611+0000","last_fullsized":"2023-07-14T18:14:47.938611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:23:07.664586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946499+0000","last_change":"2023-07-14T18:14:22.043098+0000","last_active":"2023-07-14T18:14:48.946499+0000","last_peered":"2023-07-14T18:14:48.946499+0000","last_clean":"2023-07-14T18:14:48.946499+0000","last_became_active":"2023-07-14T18:14:22.042517+0000","last_became_peered":"2023-07-14T18:14:22.042517+0000","last_unstale":"2023-07-14T18:14:48.946499+0000","last_undegraded":"2023-07-14T18:14:48.946499+0000","last_fullsized":"2023-07-14T18:14:48.946499+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:33:54.491558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.938925+0000","last_change":"2023-07-14T18:14:16.927612+0000","last_active":"2023-07-14T18:14:47.938925+0000","last_peered":"2023-07-14T18:14:47.938925+0000","last_clean":"2023-07-14T18:14:47.938925+0000","last_became_active":"2023-07-14T18:14:16.927284+0000","last_became_peered":"2023-07-14T18:14:16.927284+0000","last_unstale":"2023-07-14T18:14:47.938925+0000","last_undegraded":"2023-07-14T18:14:47.938925+0000","last_fullsized":"2023-07-14T18:14:47.938925+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:33:50.082251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540962+0000","last_change":"2023-07-14T18:14:17.930624+0000","last_active":"2023-07-14T18:14:49.540962+0000","last_peered":"2023-07-14T18:14:49.540962+0000","last_clean":"2023-07-14T18:14:49.540962+0000","last_became_active":"2023-07-14T18:14:17.930273+0000","last_became_peered":"2023-07-14T18:14:17.930273+0000","last_unstale":"2023-07-14T18:14:49.540962+0000","last_undegraded":"2023-07-14T18:14:49.540962+0000","last_fullsized":"2023-07-14T18:14:49.540962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:19:40.573682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.543208+0000","last_change":"2023-07-14T18:14:49.543208+0000","last_active":"2023-07-14T18:14:49.543208+0000","last_peered":"2023-07-14T18:14:49.543208+0000","last_clean":"2023-07-14T18:14:49.543208+0000","last_became_active":"2023-07-14T18:14:49.542685+0000","last_became_peered":"2023-07-14T18:14:49.542685+0000","last_unstale":"2023-07-14T18:14:49.543208+0000","last_undegraded":"2023-07-14T18:14:49.543208+0000","last_fullsized":"2023-07-14T18:14:49.543208+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:32:12.710336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.894625+0000","last_change":"2023-07-14T18:14:16.926185+0000","last_active":"2023-07-14T18:14:49.894625+0000","last_peered":"2023-07-14T18:14:49.894625+0000","last_clean":"2023-07-14T18:14:49.894625+0000","last_became_active":"2023-07-14T18:14:16.925874+0000","last_became_peered":"2023-07-14T18:14:16.925874+0000","last_unstale":"2023-07-14T18:14:49.894625+0000","last_undegraded":"2023-07-14T18:14:49.894625+0000","last_fullsized":"2023-07-14T18:14:49.894625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:26:07.280801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.947867+0000","last_change":"2023-07-14T18:14:17.936601+0000","last_active":"2023-07-14T18:14:48.947867+0000","last_peered":"2023-07-14T18:14:48.947867+0000","last_clean":"2023-07-14T18:14:48.947867+0000","last_became_active":"2023-07-14T18:14:17.936291+0000","last_became_peered":"2023-07-14T18:14:17.936291+0000","last_unstale":"2023-07-14T18:14:48.947867+0000","last_undegraded":"2023-07-14T18:14:48.947867+0000","last_fullsized":"2023-07-14T18:14:48.947867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:50:25.290580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946635+0000","last_change":"2023-07-14T18:14:22.042952+0000","last_active":"2023-07-14T18:14:48.946635+0000","last_peered":"2023-07-14T18:14:48.946635+0000","last_clean":"2023-07-14T18:14:48.946635+0000","last_became_active":"2023-07-14T18:14:22.042256+0000","last_became_peered":"2023-07-14T18:14:22.042256+0000","last_unstale":"2023-07-14T18:14:48.946635+0000","last_undegraded":"2023-07-14T18:14:48.946635+0000","last_fullsized":"2023-07-14T18:14:48.946635+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:20:29.914375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945839+0000","last_change":"2023-07-14T18:14:16.920902+0000","last_active":"2023-07-14T18:14:48.945839+0000","last_peered":"2023-07-14T18:14:48.945839+0000","last_clean":"2023-07-14T18:14:48.945839+0000","last_became_active":"2023-07-14T18:14:16.920554+0000","last_became_peered":"2023-07-14T18:14:16.920554+0000","last_unstale":"2023-07-14T18:14:48.945839+0000","last_undegraded":"2023-07-14T18:14:48.945839+0000","last_fullsized":"2023-07-14T18:14:48.945839+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:05:37.194027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945093+0000","last_change":"2023-07-14T18:14:17.925005+0000","last_active":"2023-07-14T18:14:48.945093+0000","last_peered":"2023-07-14T18:14:48.945093+0000","last_clean":"2023-07-14T18:14:48.945093+0000","last_became_active":"2023-07-14T18:14:17.924723+0000","last_became_peered":"2023-07-14T18:14:17.924723+0000","last_unstale":"2023-07-14T18:14:48.945093+0000","last_undegraded":"2023-07-14T18:14:48.945093+0000","last_fullsized":"2023-07-14T18:14:48.945093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:34:45.456102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541394+0000","last_change":"2023-07-14T18:14:22.038482+0000","last_active":"2023-07-14T18:14:49.541394+0000","last_peered":"2023-07-14T18:14:49.541394+0000","last_clean":"2023-07-14T18:14:49.541394+0000","last_became_active":"2023-07-14T18:14:22.038059+0000","last_became_peered":"2023-07-14T18:14:22.038059+0000","last_unstale":"2023-07-14T18:14:49.541394+0000","last_undegraded":"2023-07-14T18:14:49.541394+0000","last_fullsized":"2023-07-14T18:14:49.541394+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:40:04.711668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:50.885816+0000","last_change":"2023-07-14T18:14:48.949585+0000","last_active":"2023-07-14T18:14:50.885816+0000","last_peered":"2023-07-14T18:14:50.885816+0000","last_clean":"2023-07-14T18:14:50.885816+0000","last_became_active":"2023-07-14T18:14:48.949106+0000","last_became_peered":"2023-07-14T18:14:48.949106+0000","last_unstale":"2023-07-14T18:14:50.885816+0000","last_undegraded":"2023-07-14T18:14:50.885816+0000","last_fullsized":"2023-07-14T18:14:50.885816+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:27:44.232817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945805+0000","last_change":"2023-07-14T18:14:17.934894+0000","last_active":"2023-07-14T18:14:48.945805+0000","last_peered":"2023-07-14T18:14:48.945805+0000","last_clean":"2023-07-14T18:14:48.945805+0000","last_became_active":"2023-07-14T18:14:17.934651+0000","last_became_peered":"2023-07-14T18:14:17.934651+0000","last_unstale":"2023-07-14T18:14:48.945805+0000","last_undegraded":"2023-07-14T18:14:48.945805+0000","last_fullsized":"2023-07-14T18:14:48.945805+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:07:18.984143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541406+0000","last_change":"2023-07-14T18:14:22.043611+0000","last_active":"2023-07-14T18:14:49.541406+0000","last_peered":"2023-07-14T18:14:49.541406+0000","last_clean":"2023-07-14T18:14:49.541406+0000","last_became_active":"2023-07-14T18:14:22.043101+0000","last_became_peered":"2023-07-14T18:14:22.043101+0000","last_unstale":"2023-07-14T18:14:49.541406+0000","last_undegraded":"2023-07-14T18:14:49.541406+0000","last_fullsized":"2023-07-14T18:14:49.541406+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:09:27.645400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.894491+0000","last_change":"2023-07-14T18:14:16.925321+0000","last_active":"2023-07-14T18:14:49.894491+0000","last_peered":"2023-07-14T18:14:49.894491+0000","last_clean":"2023-07-14T18:14:49.894491+0000","last_became_active":"2023-07-14T18:14:16.925016+0000","last_became_peered":"2023-07-14T18:14:16.925016+0000","last_unstale":"2023-07-14T18:14:49.894491+0000","last_undegraded":"2023-07-14T18:14:49.894491+0000","last_fullsized":"2023-07-14T18:14:49.894491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:51:09.710945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944916+0000","last_change":"2023-07-14T18:14:17.928838+0000","last_active":"2023-07-14T18:14:48.944916+0000","last_peered":"2023-07-14T18:14:48.944916+0000","last_clean":"2023-07-14T18:14:48.944916+0000","last_became_active":"2023-07-14T18:14:17.928511+0000","last_became_peered":"2023-07-14T18:14:17.928511+0000","last_unstale":"2023-07-14T18:14:48.944916+0000","last_undegraded":"2023-07-14T18:14:48.944916+0000","last_fullsized":"2023-07-14T18:14:48.944916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:25:59.076594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944791+0000","last_change":"2023-07-14T18:14:22.043182+0000","last_active":"2023-07-14T18:14:48.944791+0000","last_peered":"2023-07-14T18:14:48.944791+0000","last_clean":"2023-07-14T18:14:48.944791+0000","last_became_active":"2023-07-14T18:14:22.042779+0000","last_became_peered":"2023-07-14T18:14:22.042779+0000","last_unstale":"2023-07-14T18:14:48.944791+0000","last_undegraded":"2023-07-14T18:14:48.944791+0000","last_fullsized":"2023-07-14T18:14:48.944791+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:15:03.016184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.895882+0000","last_change":"2023-07-14T18:14:16.935321+0000","last_active":"2023-07-14T18:14:49.895882+0000","last_peered":"2023-07-14T18:14:49.895882+0000","last_clean":"2023-07-14T18:14:49.895882+0000","last_became_active":"2023-07-14T18:14:16.934494+0000","last_became_peered":"2023-07-14T18:14:16.934494+0000","last_unstale":"2023-07-14T18:14:49.895882+0000","last_undegraded":"2023-07-14T18:14:49.895882+0000","last_fullsized":"2023-07-14T18:14:49.895882+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:00:50.926893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.936730+0000","last_change":"2023-07-14T18:14:17.922139+0000","last_active":"2023-07-14T18:14:47.936730+0000","last_peered":"2023-07-14T18:14:47.936730+0000","last_clean":"2023-07-14T18:14:47.936730+0000","last_became_active":"2023-07-14T18:14:17.921829+0000","last_became_peered":"2023-07-14T18:14:17.921829+0000","last_unstale":"2023-07-14T18:14:47.936730+0000","last_undegraded":"2023-07-14T18:14:47.936730+0000","last_fullsized":"2023-07-14T18:14:47.936730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:01:35.248837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540988+0000","last_change":"2023-07-14T18:14:22.033625+0000","last_active":"2023-07-14T18:14:49.540988+0000","last_peered":"2023-07-14T18:14:49.540988+0000","last_clean":"2023-07-14T18:14:49.540988+0000","last_became_active":"2023-07-14T18:14:22.033253+0000","last_became_peered":"2023-07-14T18:14:22.033253+0000","last_unstale":"2023-07-14T18:14:49.540988+0000","last_undegraded":"2023-07-14T18:14:49.540988+0000","last_fullsized":"2023-07-14T18:14:49.540988+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:16:24.641370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942449+0000","last_change":"2023-07-14T18:14:16.923864+0000","last_active":"2023-07-14T18:14:48.942449+0000","last_peered":"2023-07-14T18:14:48.942449+0000","last_clean":"2023-07-14T18:14:48.942449+0000","last_became_active":"2023-07-14T18:14:16.923546+0000","last_became_peered":"2023-07-14T18:14:16.923546+0000","last_unstale":"2023-07-14T18:14:48.942449+0000","last_undegraded":"2023-07-14T18:14:48.942449+0000","last_fullsized":"2023-07-14T18:14:48.942449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:31:33.868703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945453+0000","last_change":"2023-07-14T18:14:17.936473+0000","last_active":"2023-07-14T18:14:48.945453+0000","last_peered":"2023-07-14T18:14:48.945453+0000","last_clean":"2023-07-14T18:14:48.945453+0000","last_became_active":"2023-07-14T18:14:17.935387+0000","last_became_peered":"2023-07-14T18:14:17.935387+0000","last_unstale":"2023-07-14T18:14:48.945453+0000","last_undegraded":"2023-07-14T18:14:48.945453+0000","last_fullsized":"2023-07-14T18:14:48.945453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:09:31.140915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.936752+0000","last_change":"2023-07-14T18:14:22.041544+0000","last_active":"2023-07-14T18:14:47.936752+0000","last_peered":"2023-07-14T18:14:47.936752+0000","last_clean":"2023-07-14T18:14:47.936752+0000","last_became_active":"2023-07-14T18:14:22.041183+0000","last_became_peered":"2023-07-14T18:14:22.041183+0000","last_unstale":"2023-07-14T18:14:47.936752+0000","last_undegraded":"2023-07-14T18:14:47.936752+0000","last_fullsized":"2023-07-14T18:14:47.936752+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:58:13.466720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:50.885733+0000","last_change":"2023-07-14T18:14:16.928860+0000","last_active":"2023-07-14T18:14:50.885733+0000","last_peered":"2023-07-14T18:14:50.885733+0000","last_clean":"2023-07-14T18:14:50.885733+0000","last_became_active":"2023-07-14T18:14:16.928567+0000","last_became_peered":"2023-07-14T18:14:16.928567+0000","last_unstale":"2023-07-14T18:14:50.885733+0000","last_undegraded":"2023-07-14T18:14:50.885733+0000","last_fullsized":"2023-07-14T18:14:50.885733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:44:11.774963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.948806+0000","last_change":"2023-07-14T18:14:17.938456+0000","last_active":"2023-07-14T18:14:48.948806+0000","last_peered":"2023-07-14T18:14:48.948806+0000","last_clean":"2023-07-14T18:14:48.948806+0000","last_became_active":"2023-07-14T18:14:17.938086+0000","last_became_peered":"2023-07-14T18:14:17.938086+0000","last_unstale":"2023-07-14T18:14:48.948806+0000","last_undegraded":"2023-07-14T18:14:48.948806+0000","last_fullsized":"2023-07-14T18:14:48.948806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:16:01.795875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945337+0000","last_change":"2023-07-14T18:14:22.042785+0000","last_active":"2023-07-14T18:14:48.945337+0000","last_peered":"2023-07-14T18:14:48.945337+0000","last_clean":"2023-07-14T18:14:48.945337+0000","last_became_active":"2023-07-14T18:14:22.042346+0000","last_became_peered":"2023-07-14T18:14:22.042346+0000","last_unstale":"2023-07-14T18:14:48.945337+0000","last_undegraded":"2023-07-14T18:14:48.945337+0000","last_fullsized":"2023-07-14T18:14:48.945337+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:26:17.301312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944005+0000","last_change":"2023-07-14T18:14:16.921168+0000","last_active":"2023-07-14T18:14:48.944005+0000","last_peered":"2023-07-14T18:14:48.944005+0000","last_clean":"2023-07-14T18:14:48.944005+0000","last_became_active":"2023-07-14T18:14:16.920808+0000","last_became_peered":"2023-07-14T18:14:16.920808+0000","last_unstale":"2023-07-14T18:14:48.944005+0000","last_undegraded":"2023-07-14T18:14:48.944005+0000","last_fullsized":"2023-07-14T18:14:48.944005+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:41:57.870102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541090+0000","last_change":"2023-07-14T18:14:17.935100+0000","last_active":"2023-07-14T18:14:49.541090+0000","last_peered":"2023-07-14T18:14:49.541090+0000","last_clean":"2023-07-14T18:14:49.541090+0000","last_became_active":"2023-07-14T18:14:17.934794+0000","last_became_peered":"2023-07-14T18:14:17.934794+0000","last_unstale":"2023-07-14T18:14:49.541090+0000","last_undegraded":"2023-07-14T18:14:49.541090+0000","last_fullsized":"2023-07-14T18:14:49.541090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:37:28.349127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.948310+0000","last_change":"2023-07-14T18:14:22.043495+0000","last_active":"2023-07-14T18:14:48.948310+0000","last_peered":"2023-07-14T18:14:48.948310+0000","last_clean":"2023-07-14T18:14:48.948310+0000","last_became_active":"2023-07-14T18:14:22.042686+0000","last_became_peered":"2023-07-14T18:14:22.042686+0000","last_unstale":"2023-07-14T18:14:48.948310+0000","last_undegraded":"2023-07-14T18:14:48.948310+0000","last_fullsized":"2023-07-14T18:14:48.948310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:16:19.904966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944994+0000","last_change":"2023-07-14T18:14:16.926006+0000","last_active":"2023-07-14T18:14:48.944994+0000","last_peered":"2023-07-14T18:14:48.944994+0000","last_clean":"2023-07-14T18:14:48.944994+0000","last_became_active":"2023-07-14T18:14:16.925719+0000","last_became_peered":"2023-07-14T18:14:16.925719+0000","last_unstale":"2023-07-14T18:14:48.944994+0000","last_undegraded":"2023-07-14T18:14:48.944994+0000","last_fullsized":"2023-07-14T18:14:48.944994+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:03:27.517349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.948103+0000","last_change":"2023-07-14T18:14:17.934155+0000","last_active":"2023-07-14T18:14:48.948103+0000","last_peered":"2023-07-14T18:14:48.948103+0000","last_clean":"2023-07-14T18:14:48.948103+0000","last_became_active":"2023-07-14T18:14:17.933756+0000","last_became_peered":"2023-07-14T18:14:17.933756+0000","last_unstale":"2023-07-14T18:14:48.948103+0000","last_undegraded":"2023-07-14T18:14:48.948103+0000","last_fullsized":"2023-07-14T18:14:48.948103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:04:42.915865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945422+0000","last_change":"2023-07-14T18:14:22.039198+0000","last_active":"2023-07-14T18:14:48.945422+0000","last_peered":"2023-07-14T18:14:48.945422+0000","last_clean":"2023-07-14T18:14:48.945422+0000","last_became_active":"2023-07-14T18:14:22.038842+0000","last_became_peered":"2023-07-14T18:14:22.038842+0000","last_unstale":"2023-07-14T18:14:48.945422+0000","last_undegraded":"2023-07-14T18:14:48.945422+0000","last_fullsized":"2023-07-14T18:14:48.945422+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:32:54.838956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'24","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:51.893171+0000","last_change":"2023-07-14T18:14:16.920918+0000","last_active":"2023-07-14T18:14:51.893171+0000","last_peered":"2023-07-14T18:14:51.893171+0000","last_clean":"2023-07-14T18:14:51.893171+0000","last_became_active":"2023-07-14T18:14:16.920572+0000","last_became_peered":"2023-07-14T18:14:16.920572+0000","last_unstale":"2023-07-14T18:14:51.893171+0000","last_undegraded":"2023-07-14T18:14:51.893171+0000","last_fullsized":"2023-07-14T18:14:51.893171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:55:35.900625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38515,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":10,"num_read_kb":42,"num_write":25,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945963+0000","last_change":"2023-07-14T18:14:17.929413+0000","last_active":"2023-07-14T18:14:48.945963+0000","last_peered":"2023-07-14T18:14:48.945963+0000","last_clean":"2023-07-14T18:14:48.945963+0000","last_became_active":"2023-07-14T18:14:17.929064+0000","last_became_peered":"2023-07-14T18:14:17.929064+0000","last_unstale":"2023-07-14T18:14:48.945963+0000","last_undegraded":"2023-07-14T18:14:48.945963+0000","last_fullsized":"2023-07-14T18:14:48.945963+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:29:27.786039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945372+0000","last_change":"2023-07-14T18:14:22.041983+0000","last_active":"2023-07-14T18:14:48.945372+0000","last_peered":"2023-07-14T18:14:48.945372+0000","last_clean":"2023-07-14T18:14:48.945372+0000","last_became_active":"2023-07-14T18:14:22.041553+0000","last_became_peered":"2023-07-14T18:14:22.041553+0000","last_unstale":"2023-07-14T18:14:48.945372+0000","last_undegraded":"2023-07-14T18:14:48.945372+0000","last_fullsized":"2023-07-14T18:14:48.945372+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:59:09.096024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.892826+0000","last_change":"2023-07-14T18:14:16.925444+0000","last_active":"2023-07-14T18:14:49.892826+0000","last_peered":"2023-07-14T18:14:49.892826+0000","last_clean":"2023-07-14T18:14:49.892826+0000","last_became_active":"2023-07-14T18:14:16.925138+0000","last_became_peered":"2023-07-14T18:14:16.925138+0000","last_unstale":"2023-07-14T18:14:49.892826+0000","last_undegraded":"2023-07-14T18:14:49.892826+0000","last_fullsized":"2023-07-14T18:14:49.892826+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:10:33.163171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942332+0000","last_change":"2023-07-14T18:14:17.922351+0000","last_active":"2023-07-14T18:14:48.942332+0000","last_peered":"2023-07-14T18:14:48.942332+0000","last_clean":"2023-07-14T18:14:48.942332+0000","last_became_active":"2023-07-14T18:14:17.921973+0000","last_became_peered":"2023-07-14T18:14:17.921973+0000","last_unstale":"2023-07-14T18:14:48.942332+0000","last_undegraded":"2023-07-14T18:14:48.942332+0000","last_fullsized":"2023-07-14T18:14:48.942332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:16:09.934284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945943+0000","last_change":"2023-07-14T18:14:22.048274+0000","last_active":"2023-07-14T18:14:48.945943+0000","last_peered":"2023-07-14T18:14:48.945943+0000","last_clean":"2023-07-14T18:14:48.945943+0000","last_became_active":"2023-07-14T18:14:22.047946+0000","last_became_peered":"2023-07-14T18:14:22.047946+0000","last_unstale":"2023-07-14T18:14:48.945943+0000","last_undegraded":"2023-07-14T18:14:48.945943+0000","last_fullsized":"2023-07-14T18:14:48.945943+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:29:47.580140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944169+0000","last_change":"2023-07-14T18:14:16.933566+0000","last_active":"2023-07-14T18:14:48.944169+0000","last_peered":"2023-07-14T18:14:48.944169+0000","last_clean":"2023-07-14T18:14:48.944169+0000","last_became_active":"2023-07-14T18:14:16.933300+0000","last_became_peered":"2023-07-14T18:14:16.933300+0000","last_unstale":"2023-07-14T18:14:48.944169+0000","last_undegraded":"2023-07-14T18:14:48.944169+0000","last_fullsized":"2023-07-14T18:14:48.944169+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:05:57.086039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944233+0000","last_change":"2023-07-14T18:14:17.928521+0000","last_active":"2023-07-14T18:14:48.944233+0000","last_peered":"2023-07-14T18:14:48.944233+0000","last_clean":"2023-07-14T18:14:48.944233+0000","last_became_active":"2023-07-14T18:14:17.928025+0000","last_became_peered":"2023-07-14T18:14:17.928025+0000","last_unstale":"2023-07-14T18:14:48.944233+0000","last_undegraded":"2023-07-14T18:14:48.944233+0000","last_fullsized":"2023-07-14T18:14:48.944233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:21:23.554363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945666+0000","last_change":"2023-07-14T18:14:22.046820+0000","last_active":"2023-07-14T18:14:48.945666+0000","last_peered":"2023-07-14T18:14:48.945666+0000","last_clean":"2023-07-14T18:14:48.945666+0000","last_became_active":"2023-07-14T18:14:22.046440+0000","last_became_peered":"2023-07-14T18:14:22.046440+0000","last_unstale":"2023-07-14T18:14:48.945666+0000","last_undegraded":"2023-07-14T18:14:48.945666+0000","last_fullsized":"2023-07-14T18:14:48.945666+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:46:22.890726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:50.894777+0000","last_change":"2023-07-14T18:14:16.922075+0000","last_active":"2023-07-14T18:14:50.894777+0000","last_peered":"2023-07-14T18:14:50.894777+0000","last_clean":"2023-07-14T18:14:50.894777+0000","last_became_active":"2023-07-14T18:14:16.921570+0000","last_became_peered":"2023-07-14T18:14:16.921570+0000","last_unstale":"2023-07-14T18:14:50.894777+0000","last_undegraded":"2023-07-14T18:14:50.894777+0000","last_fullsized":"2023-07-14T18:14:50.894777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:08:37.059574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.943804+0000","last_change":"2023-07-14T18:14:17.936439+0000","last_active":"2023-07-14T18:14:48.943804+0000","last_peered":"2023-07-14T18:14:48.943804+0000","last_clean":"2023-07-14T18:14:48.943804+0000","last_became_active":"2023-07-14T18:14:17.936138+0000","last_became_peered":"2023-07-14T18:14:17.936138+0000","last_unstale":"2023-07-14T18:14:48.943804+0000","last_undegraded":"2023-07-14T18:14:48.943804+0000","last_fullsized":"2023-07-14T18:14:48.943804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:58:02.987288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.543302+0000","last_change":"2023-07-14T18:14:49.543302+0000","last_active":"2023-07-14T18:14:49.543302+0000","last_peered":"2023-07-14T18:14:49.543302+0000","last_clean":"2023-07-14T18:14:49.543302+0000","last_became_active":"2023-07-14T18:14:49.542768+0000","last_became_peered":"2023-07-14T18:14:49.542768+0000","last_unstale":"2023-07-14T18:14:49.543302+0000","last_undegraded":"2023-07-14T18:14:49.543302+0000","last_fullsized":"2023-07-14T18:14:49.543302+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:14:41.906790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.896057+0000","last_change":"2023-07-14T18:14:16.926417+0000","last_active":"2023-07-14T18:14:49.896057+0000","last_peered":"2023-07-14T18:14:49.896057+0000","last_clean":"2023-07-14T18:14:49.896057+0000","last_became_active":"2023-07-14T18:14:16.926101+0000","last_became_peered":"2023-07-14T18:14:16.926101+0000","last_unstale":"2023-07-14T18:14:49.896057+0000","last_undegraded":"2023-07-14T18:14:49.896057+0000","last_fullsized":"2023-07-14T18:14:49.896057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:09:38.329843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.950979+0000","last_change":"2023-07-14T18:14:48.950649+0000","last_active":"2023-07-14T18:14:48.950979+0000","last_peered":"2023-07-14T18:14:48.950979+0000","last_clean":"2023-07-14T18:14:48.950979+0000","last_became_active":"2023-07-14T18:14:48.950303+0000","last_became_peered":"2023-07-14T18:14:48.950303+0000","last_unstale":"2023-07-14T18:14:48.950979+0000","last_undegraded":"2023-07-14T18:14:48.950979+0000","last_fullsized":"2023-07-14T18:14:48.950979+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:40:15.252731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942609+0000","last_change":"2023-07-14T18:14:22.042171+0000","last_active":"2023-07-14T18:14:48.942609+0000","last_peered":"2023-07-14T18:14:48.942609+0000","last_clean":"2023-07-14T18:14:48.942609+0000","last_became_active":"2023-07-14T18:14:22.041692+0000","last_became_peered":"2023-07-14T18:14:22.041692+0000","last_unstale":"2023-07-14T18:14:48.942609+0000","last_undegraded":"2023-07-14T18:14:48.942609+0000","last_fullsized":"2023-07-14T18:14:48.942609+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:03:01.650548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.881565+0000","last_change":"2023-07-14T18:14:16.922883+0000","last_active":"2023-07-14T18:14:49.881565+0000","last_peered":"2023-07-14T18:14:49.881565+0000","last_clean":"2023-07-14T18:14:49.881565+0000","last_became_active":"2023-07-14T18:14:16.922501+0000","last_became_peered":"2023-07-14T18:14:16.922501+0000","last_unstale":"2023-07-14T18:14:49.881565+0000","last_undegraded":"2023-07-14T18:14:49.881565+0000","last_fullsized":"2023-07-14T18:14:49.881565+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:51:12.741737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946509+0000","last_change":"2023-07-14T18:14:17.922899+0000","last_active":"2023-07-14T18:14:48.946509+0000","last_peered":"2023-07-14T18:14:48.946509+0000","last_clean":"2023-07-14T18:14:48.946509+0000","last_became_active":"2023-07-14T18:14:17.922608+0000","last_became_peered":"2023-07-14T18:14:17.922608+0000","last_unstale":"2023-07-14T18:14:48.946509+0000","last_undegraded":"2023-07-14T18:14:48.946509+0000","last_fullsized":"2023-07-14T18:14:48.946509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:53:19.741997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945671+0000","last_change":"2023-07-14T18:14:22.327484+0000","last_active":"2023-07-14T18:14:48.945671+0000","last_peered":"2023-07-14T18:14:48.945671+0000","last_clean":"2023-07-14T18:14:48.945671+0000","last_became_active":"2023-07-14T18:14:22.327010+0000","last_became_peered":"2023-07-14T18:14:22.327010+0000","last_unstale":"2023-07-14T18:14:48.945671+0000","last_undegraded":"2023-07-14T18:14:48.945671+0000","last_fullsized":"2023-07-14T18:14:48.945671+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:32:16.310729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945788+0000","last_change":"2023-07-14T18:14:16.930606+0000","last_active":"2023-07-14T18:14:48.945788+0000","last_peered":"2023-07-14T18:14:48.945788+0000","last_clean":"2023-07-14T18:14:48.945788+0000","last_became_active":"2023-07-14T18:14:16.930175+0000","last_became_peered":"2023-07-14T18:14:16.930175+0000","last_unstale":"2023-07-14T18:14:48.945788+0000","last_undegraded":"2023-07-14T18:14:48.945788+0000","last_fullsized":"2023-07-14T18:14:48.945788+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:39:16.156926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541700+0000","last_change":"2023-07-14T18:14:17.931962+0000","last_active":"2023-07-14T18:14:49.541700+0000","last_peered":"2023-07-14T18:14:49.541700+0000","last_clean":"2023-07-14T18:14:49.541700+0000","last_became_active":"2023-07-14T18:14:17.931621+0000","last_became_peered":"2023-07-14T18:14:17.931621+0000","last_unstale":"2023-07-14T18:14:49.541700+0000","last_undegraded":"2023-07-14T18:14:49.541700+0000","last_fullsized":"2023-07-14T18:14:49.541700+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:32:19.781881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942854+0000","last_change":"2023-07-14T18:14:22.038977+0000","last_active":"2023-07-14T18:14:48.942854+0000","last_peered":"2023-07-14T18:14:48.942854+0000","last_clean":"2023-07-14T18:14:48.942854+0000","last_became_active":"2023-07-14T18:14:22.038515+0000","last_became_peered":"2023-07-14T18:14:22.038515+0000","last_unstale":"2023-07-14T18:14:48.942854+0000","last_undegraded":"2023-07-14T18:14:48.942854+0000","last_fullsized":"2023-07-14T18:14:48.942854+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:40:29.285675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.943866+0000","last_change":"2023-07-14T18:14:16.925457+0000","last_active":"2023-07-14T18:14:48.943866+0000","last_peered":"2023-07-14T18:14:48.943866+0000","last_clean":"2023-07-14T18:14:48.943866+0000","last_became_active":"2023-07-14T18:14:16.925150+0000","last_became_peered":"2023-07-14T18:14:16.925150+0000","last_unstale":"2023-07-14T18:14:48.943866+0000","last_undegraded":"2023-07-14T18:14:48.943866+0000","last_fullsized":"2023-07-14T18:14:48.943866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:46:38.768309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541290+0000","last_change":"2023-07-14T18:14:17.926492+0000","last_active":"2023-07-14T18:14:49.541290+0000","last_peered":"2023-07-14T18:14:49.541290+0000","last_clean":"2023-07-14T18:14:49.541290+0000","last_became_active":"2023-07-14T18:14:17.926218+0000","last_became_peered":"2023-07-14T18:14:17.926218+0000","last_unstale":"2023-07-14T18:14:49.541290+0000","last_undegraded":"2023-07-14T18:14:49.541290+0000","last_fullsized":"2023-07-14T18:14:49.541290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:08:20.317990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942594+0000","last_change":"2023-07-14T18:14:22.327242+0000","last_active":"2023-07-14T18:14:48.942594+0000","last_peered":"2023-07-14T18:14:48.942594+0000","last_clean":"2023-07-14T18:14:48.942594+0000","last_became_active":"2023-07-14T18:14:22.326787+0000","last_became_peered":"2023-07-14T18:14:22.326787+0000","last_unstale":"2023-07-14T18:14:48.942594+0000","last_undegraded":"2023-07-14T18:14:48.942594+0000","last_fullsized":"2023-07-14T18:14:48.942594+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:08:03.165732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942605+0000","last_change":"2023-07-14T18:14:17.925035+0000","last_active":"2023-07-14T18:14:48.942605+0000","last_peered":"2023-07-14T18:14:48.942605+0000","last_clean":"2023-07-14T18:14:48.942605+0000","last_became_active":"2023-07-14T18:14:17.924713+0000","last_became_peered":"2023-07-14T18:14:17.924713+0000","last_unstale":"2023-07-14T18:14:48.942605+0000","last_undegraded":"2023-07-14T18:14:48.942605+0000","last_fullsized":"2023-07-14T18:14:48.942605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:40:51.683089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:51.897592+0000","last_change":"2023-07-14T18:14:16.935561+0000","last_active":"2023-07-14T18:14:51.897592+0000","last_peered":"2023-07-14T18:14:51.897592+0000","last_clean":"2023-07-14T18:14:51.897592+0000","last_became_active":"2023-07-14T18:14:16.935022+0000","last_became_peered":"2023-07-14T18:14:16.935022+0000","last_unstale":"2023-07-14T18:14:51.897592+0000","last_undegraded":"2023-07-14T18:14:51.897592+0000","last_fullsized":"2023-07-14T18:14:51.897592+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:31:09.667735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.943861+0000","last_change":"2023-07-14T18:14:22.043818+0000","last_active":"2023-07-14T18:14:48.943861+0000","last_peered":"2023-07-14T18:14:48.943861+0000","last_clean":"2023-07-14T18:14:48.943861+0000","last_became_active":"2023-07-14T18:14:22.043449+0000","last_became_peered":"2023-07-14T18:14:22.043449+0000","last_unstale":"2023-07-14T18:14:48.943861+0000","last_undegraded":"2023-07-14T18:14:48.943861+0000","last_fullsized":"2023-07-14T18:14:48.943861+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:25:52.282417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.936894+0000","last_change":"2023-07-14T18:14:17.930211+0000","last_active":"2023-07-14T18:14:47.936894+0000","last_peered":"2023-07-14T18:14:47.936894+0000","last_clean":"2023-07-14T18:14:47.936894+0000","last_became_active":"2023-07-14T18:14:17.929851+0000","last_became_peered":"2023-07-14T18:14:17.929851+0000","last_unstale":"2023-07-14T18:14:47.936894+0000","last_undegraded":"2023-07-14T18:14:47.936894+0000","last_fullsized":"2023-07-14T18:14:47.936894+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:05:44.957698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.894113+0000","last_change":"2023-07-14T18:14:16.928995+0000","last_active":"2023-07-14T18:14:49.894113+0000","last_peered":"2023-07-14T18:14:49.894113+0000","last_clean":"2023-07-14T18:14:49.894113+0000","last_became_active":"2023-07-14T18:14:16.928612+0000","last_became_peered":"2023-07-14T18:14:16.928612+0000","last_unstale":"2023-07-14T18:14:49.894113+0000","last_undegraded":"2023-07-14T18:14:49.894113+0000","last_fullsized":"2023-07-14T18:14:49.894113+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:33:55.215553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":340,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.943619+0000","last_change":"2023-07-14T18:13:11.614507+0000","last_active":"2023-07-14T18:14:48.943619+0000","last_peered":"2023-07-14T18:14:48.943619+0000","last_clean":"2023-07-14T18:14:48.943619+0000","last_became_active":"2023-07-14T18:13:11.614063+0000","last_became_peered":"2023-07-14T18:13:11.614063+0000","last_unstale":"2023-07-14T18:14:48.943619+0000","last_undegraded":"2023-07-14T18:14:48.943619+0000","last_fullsized":"2023-07-14T18:14:48.943619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:10:03.995193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:10:03.995193+0000","last_clean_scrub_stamp":"2023-07-14T18:10:03.995193+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:35:25.729988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.543558+0000","last_change":"2023-07-14T18:14:49.542604+0000","last_active":"2023-07-14T18:14:49.543558+0000","last_peered":"2023-07-14T18:14:49.543558+0000","last_clean":"2023-07-14T18:14:49.543558+0000","last_became_active":"2023-07-14T18:14:49.542111+0000","last_became_peered":"2023-07-14T18:14:49.542111+0000","last_unstale":"2023-07-14T18:14:49.543558+0000","last_undegraded":"2023-07-14T18:14:49.543558+0000","last_fullsized":"2023-07-14T18:14:49.543558+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:40:56.970642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.543511+0000","last_change":"2023-07-14T18:14:49.543511+0000","last_active":"2023-07-14T18:14:49.543511+0000","last_peered":"2023-07-14T18:14:49.543511+0000","last_clean":"2023-07-14T18:14:49.543511+0000","last_became_active":"2023-07-14T18:14:49.542407+0000","last_became_peered":"2023-07-14T18:14:49.542407+0000","last_unstale":"2023-07-14T18:14:49.543511+0000","last_undegraded":"2023-07-14T18:14:49.543511+0000","last_fullsized":"2023-07-14T18:14:49.543511+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:55:54.330638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.894434+0000","last_change":"2023-07-14T18:14:48.950022+0000","last_active":"2023-07-14T18:14:49.894434+0000","last_peered":"2023-07-14T18:14:49.894434+0000","last_clean":"2023-07-14T18:14:49.894434+0000","last_became_active":"2023-07-14T18:14:48.949149+0000","last_became_peered":"2023-07-14T18:14:48.949149+0000","last_unstale":"2023-07-14T18:14:49.894434+0000","last_undegraded":"2023-07-14T18:14:49.894434+0000","last_fullsized":"2023-07-14T18:14:49.894434+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:10:25.993299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945005+0000","last_change":"2023-07-14T18:14:22.038428+0000","last_active":"2023-07-14T18:14:48.945005+0000","last_peered":"2023-07-14T18:14:48.945005+0000","last_clean":"2023-07-14T18:14:48.945005+0000","last_became_active":"2023-07-14T18:14:22.038038+0000","last_became_peered":"2023-07-14T18:14:22.038038+0000","last_unstale":"2023-07-14T18:14:48.945005+0000","last_undegraded":"2023-07-14T18:14:48.945005+0000","last_fullsized":"2023-07-14T18:14:48.945005+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:58:40.233180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946196+0000","last_change":"2023-07-14T18:14:16.933110+0000","last_active":"2023-07-14T18:14:48.946196+0000","last_peered":"2023-07-14T18:14:48.946196+0000","last_clean":"2023-07-14T18:14:48.946196+0000","last_became_active":"2023-07-14T18:14:16.932844+0000","last_became_peered":"2023-07-14T18:14:16.932844+0000","last_unstale":"2023-07-14T18:14:48.946196+0000","last_undegraded":"2023-07-14T18:14:48.946196+0000","last_fullsized":"2023-07-14T18:14:48.946196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:38:45.588165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942980+0000","last_change":"2023-07-14T18:14:17.920168+0000","last_active":"2023-07-14T18:14:48.942980+0000","last_peered":"2023-07-14T18:14:48.942980+0000","last_clean":"2023-07-14T18:14:48.942980+0000","last_became_active":"2023-07-14T18:14:17.919576+0000","last_became_peered":"2023-07-14T18:14:17.919576+0000","last_unstale":"2023-07-14T18:14:48.942980+0000","last_undegraded":"2023-07-14T18:14:48.942980+0000","last_fullsized":"2023-07-14T18:14:48.942980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:22:46.766040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.936519+0000","last_change":"2023-07-14T18:14:22.327206+0000","last_active":"2023-07-14T18:14:47.936519+0000","last_peered":"2023-07-14T18:14:47.936519+0000","last_clean":"2023-07-14T18:14:47.936519+0000","last_became_active":"2023-07-14T18:14:22.326743+0000","last_became_peered":"2023-07-14T18:14:22.326743+0000","last_unstale":"2023-07-14T18:14:47.936519+0000","last_undegraded":"2023-07-14T18:14:47.936519+0000","last_fullsized":"2023-07-14T18:14:47.936519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:07:53.561873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.895509+0000","last_change":"2023-07-14T18:14:16.928994+0000","last_active":"2023-07-14T18:14:49.895509+0000","last_peered":"2023-07-14T18:14:49.895509+0000","last_clean":"2023-07-14T18:14:49.895509+0000","last_became_active":"2023-07-14T18:14:16.928468+0000","last_became_peered":"2023-07-14T18:14:16.928468+0000","last_unstale":"2023-07-14T18:14:49.895509+0000","last_undegraded":"2023-07-14T18:14:49.895509+0000","last_fullsized":"2023-07-14T18:14:49.895509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:19:19.175018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.937743+0000","last_change":"2023-07-14T18:14:17.919147+0000","last_active":"2023-07-14T18:14:47.937743+0000","last_peered":"2023-07-14T18:14:47.937743+0000","last_clean":"2023-07-14T18:14:47.937743+0000","last_became_active":"2023-07-14T18:14:17.918811+0000","last_became_peered":"2023-07-14T18:14:17.918811+0000","last_unstale":"2023-07-14T18:14:47.937743+0000","last_undegraded":"2023-07-14T18:14:47.937743+0000","last_fullsized":"2023-07-14T18:14:47.937743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:54:37.460727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540750+0000","last_change":"2023-07-14T18:14:22.032632+0000","last_active":"2023-07-14T18:14:49.540750+0000","last_peered":"2023-07-14T18:14:49.540750+0000","last_clean":"2023-07-14T18:14:49.540750+0000","last_became_active":"2023-07-14T18:14:22.032233+0000","last_became_peered":"2023-07-14T18:14:22.032233+0000","last_unstale":"2023-07-14T18:14:49.540750+0000","last_undegraded":"2023-07-14T18:14:49.540750+0000","last_fullsized":"2023-07-14T18:14:49.540750+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:16:49.157181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.947795+0000","last_change":"2023-07-14T18:14:16.924993+0000","last_active":"2023-07-14T18:14:48.947795+0000","last_peered":"2023-07-14T18:14:48.947795+0000","last_clean":"2023-07-14T18:14:48.947795+0000","last_became_active":"2023-07-14T18:14:16.924702+0000","last_became_peered":"2023-07-14T18:14:16.924702+0000","last_unstale":"2023-07-14T18:14:48.947795+0000","last_undegraded":"2023-07-14T18:14:48.947795+0000","last_fullsized":"2023-07-14T18:14:48.947795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:21:25.188427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541234+0000","last_change":"2023-07-14T18:14:17.943397+0000","last_active":"2023-07-14T18:14:49.541234+0000","last_peered":"2023-07-14T18:14:49.541234+0000","last_clean":"2023-07-14T18:14:49.541234+0000","last_became_active":"2023-07-14T18:14:17.942114+0000","last_became_peered":"2023-07-14T18:14:17.942114+0000","last_unstale":"2023-07-14T18:14:49.541234+0000","last_undegraded":"2023-07-14T18:14:49.541234+0000","last_fullsized":"2023-07-14T18:14:49.541234+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:46:40.372375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.947250+0000","last_change":"2023-07-14T18:14:22.328273+0000","last_active":"2023-07-14T18:14:48.947250+0000","last_peered":"2023-07-14T18:14:48.947250+0000","last_clean":"2023-07-14T18:14:48.947250+0000","last_became_active":"2023-07-14T18:14:22.327843+0000","last_became_peered":"2023-07-14T18:14:22.327843+0000","last_unstale":"2023-07-14T18:14:48.947250+0000","last_undegraded":"2023-07-14T18:14:48.947250+0000","last_fullsized":"2023-07-14T18:14:48.947250+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:23:58.886753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.545792+0000","last_change":"2023-07-14T18:14:49.543525+0000","last_active":"2023-07-14T18:14:49.545792+0000","last_peered":"2023-07-14T18:14:49.545792+0000","last_clean":"2023-07-14T18:14:49.545792+0000","last_became_active":"2023-07-14T18:14:49.542962+0000","last_became_peered":"2023-07-14T18:14:49.542962+0000","last_unstale":"2023-07-14T18:14:49.545792+0000","last_undegraded":"2023-07-14T18:14:49.545792+0000","last_fullsized":"2023-07-14T18:14:49.545792+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:11:59.955363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945273+0000","last_change":"2023-07-14T18:14:17.932181+0000","last_active":"2023-07-14T18:14:48.945273+0000","last_peered":"2023-07-14T18:14:48.945273+0000","last_clean":"2023-07-14T18:14:48.945273+0000","last_became_active":"2023-07-14T18:14:17.931811+0000","last_became_peered":"2023-07-14T18:14:17.931811+0000","last_unstale":"2023-07-14T18:14:48.945273+0000","last_undegraded":"2023-07-14T18:14:48.945273+0000","last_fullsized":"2023-07-14T18:14:48.945273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:24:11.968765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945853+0000","last_change":"2023-07-14T18:14:22.047281+0000","last_active":"2023-07-14T18:14:48.945853+0000","last_peered":"2023-07-14T18:14:48.945853+0000","last_clean":"2023-07-14T18:14:48.945853+0000","last_became_active":"2023-07-14T18:14:22.045531+0000","last_became_peered":"2023-07-14T18:14:22.045531+0000","last_unstale":"2023-07-14T18:14:48.945853+0000","last_undegraded":"2023-07-14T18:14:48.945853+0000","last_fullsized":"2023-07-14T18:14:48.945853+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:59:25.065086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.957885+0000","last_change":"2023-07-14T18:14:16.926904+0000","last_active":"2023-07-14T18:14:47.957885+0000","last_peered":"2023-07-14T18:14:47.957885+0000","last_clean":"2023-07-14T18:14:47.957885+0000","last_became_active":"2023-07-14T18:14:16.926579+0000","last_became_peered":"2023-07-14T18:14:16.926579+0000","last_unstale":"2023-07-14T18:14:47.957885+0000","last_undegraded":"2023-07-14T18:14:47.957885+0000","last_fullsized":"2023-07-14T18:14:47.957885+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:00:13.082750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946719+0000","last_change":"2023-07-14T18:14:17.926005+0000","last_active":"2023-07-14T18:14:48.946719+0000","last_peered":"2023-07-14T18:14:48.946719+0000","last_clean":"2023-07-14T18:14:48.946719+0000","last_became_active":"2023-07-14T18:14:17.925266+0000","last_became_peered":"2023-07-14T18:14:17.925266+0000","last_unstale":"2023-07-14T18:14:48.946719+0000","last_undegraded":"2023-07-14T18:14:48.946719+0000","last_fullsized":"2023-07-14T18:14:48.946719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:03:29.763528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946669+0000","last_change":"2023-07-14T18:14:22.043219+0000","last_active":"2023-07-14T18:14:48.946669+0000","last_peered":"2023-07-14T18:14:48.946669+0000","last_clean":"2023-07-14T18:14:48.946669+0000","last_became_active":"2023-07-14T18:14:22.042654+0000","last_became_peered":"2023-07-14T18:14:22.042654+0000","last_unstale":"2023-07-14T18:14:48.946669+0000","last_undegraded":"2023-07-14T18:14:48.946669+0000","last_fullsized":"2023-07-14T18:14:48.946669+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:46:41.786917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540895+0000","last_change":"2023-07-14T18:14:16.923130+0000","last_active":"2023-07-14T18:14:49.540895+0000","last_peered":"2023-07-14T18:14:49.540895+0000","last_clean":"2023-07-14T18:14:49.540895+0000","last_became_active":"2023-07-14T18:14:16.922835+0000","last_became_peered":"2023-07-14T18:14:16.922835+0000","last_unstale":"2023-07-14T18:14:49.540895+0000","last_undegraded":"2023-07-14T18:14:49.540895+0000","last_fullsized":"2023-07-14T18:14:49.540895+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:48:50.705976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.937816+0000","last_change":"2023-07-14T18:14:17.931024+0000","last_active":"2023-07-14T18:14:47.937816+0000","last_peered":"2023-07-14T18:14:47.937816+0000","last_clean":"2023-07-14T18:14:47.937816+0000","last_became_active":"2023-07-14T18:14:17.929529+0000","last_became_peered":"2023-07-14T18:14:17.929529+0000","last_unstale":"2023-07-14T18:14:47.937816+0000","last_undegraded":"2023-07-14T18:14:47.937816+0000","last_fullsized":"2023-07-14T18:14:47.937816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:40:41.156046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946795+0000","last_change":"2023-07-14T18:14:22.327313+0000","last_active":"2023-07-14T18:14:48.946795+0000","last_peered":"2023-07-14T18:14:48.946795+0000","last_clean":"2023-07-14T18:14:48.946795+0000","last_became_active":"2023-07-14T18:14:22.326922+0000","last_became_peered":"2023-07-14T18:14:22.326922+0000","last_unstale":"2023-07-14T18:14:48.946795+0000","last_undegraded":"2023-07-14T18:14:48.946795+0000","last_fullsized":"2023-07-14T18:14:48.946795+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:48:34.325995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945190+0000","last_change":"2023-07-14T18:14:16.926632+0000","last_active":"2023-07-14T18:14:48.945190+0000","last_peered":"2023-07-14T18:14:48.945190+0000","last_clean":"2023-07-14T18:14:48.945190+0000","last_became_active":"2023-07-14T18:14:16.926279+0000","last_became_peered":"2023-07-14T18:14:16.926279+0000","last_unstale":"2023-07-14T18:14:48.945190+0000","last_undegraded":"2023-07-14T18:14:48.945190+0000","last_fullsized":"2023-07-14T18:14:48.945190+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:25:51.630494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944432+0000","last_change":"2023-07-14T18:14:17.924416+0000","last_active":"2023-07-14T18:14:48.944432+0000","last_peered":"2023-07-14T18:14:48.944432+0000","last_clean":"2023-07-14T18:14:48.944432+0000","last_became_active":"2023-07-14T18:14:17.924140+0000","last_became_peered":"2023-07-14T18:14:17.924140+0000","last_unstale":"2023-07-14T18:14:48.944432+0000","last_undegraded":"2023-07-14T18:14:48.944432+0000","last_fullsized":"2023-07-14T18:14:48.944432+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:36:11.267149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946249+0000","last_change":"2023-07-14T18:14:22.037127+0000","last_active":"2023-07-14T18:14:48.946249+0000","last_peered":"2023-07-14T18:14:48.946249+0000","last_clean":"2023-07-14T18:14:48.946249+0000","last_became_active":"2023-07-14T18:14:22.036647+0000","last_became_peered":"2023-07-14T18:14:22.036647+0000","last_unstale":"2023-07-14T18:14:48.946249+0000","last_undegraded":"2023-07-14T18:14:48.946249+0000","last_fullsized":"2023-07-14T18:14:48.946249+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:45:53.709735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.895800+0000","last_change":"2023-07-14T18:14:16.921494+0000","last_active":"2023-07-14T18:14:49.895800+0000","last_peered":"2023-07-14T18:14:49.895800+0000","last_clean":"2023-07-14T18:14:49.895800+0000","last_became_active":"2023-07-14T18:14:16.919622+0000","last_became_peered":"2023-07-14T18:14:16.919622+0000","last_unstale":"2023-07-14T18:14:49.895800+0000","last_undegraded":"2023-07-14T18:14:49.895800+0000","last_fullsized":"2023-07-14T18:14:49.895800+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:49:14.678276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944208+0000","last_change":"2023-07-14T18:14:17.922266+0000","last_active":"2023-07-14T18:14:48.944208+0000","last_peered":"2023-07-14T18:14:48.944208+0000","last_clean":"2023-07-14T18:14:48.944208+0000","last_became_active":"2023-07-14T18:14:17.921461+0000","last_became_peered":"2023-07-14T18:14:17.921461+0000","last_unstale":"2023-07-14T18:14:48.944208+0000","last_undegraded":"2023-07-14T18:14:48.944208+0000","last_fullsized":"2023-07-14T18:14:48.944208+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:27:06.427711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541171+0000","last_change":"2023-07-14T18:14:22.044080+0000","last_active":"2023-07-14T18:14:49.541171+0000","last_peered":"2023-07-14T18:14:49.541171+0000","last_clean":"2023-07-14T18:14:49.541171+0000","last_became_active":"2023-07-14T18:14:22.043612+0000","last_became_peered":"2023-07-14T18:14:22.043612+0000","last_unstale":"2023-07-14T18:14:49.541171+0000","last_undegraded":"2023-07-14T18:14:49.541171+0000","last_fullsized":"2023-07-14T18:14:49.541171+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:41:00.904598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945287+0000","last_change":"2023-07-14T18:14:16.931871+0000","last_active":"2023-07-14T18:14:48.945287+0000","last_peered":"2023-07-14T18:14:48.945287+0000","last_clean":"2023-07-14T18:14:48.945287+0000","last_became_active":"2023-07-14T18:14:16.931549+0000","last_became_peered":"2023-07-14T18:14:16.931549+0000","last_unstale":"2023-07-14T18:14:48.945287+0000","last_undegraded":"2023-07-14T18:14:48.945287+0000","last_fullsized":"2023-07-14T18:14:48.945287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:39:50.087515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944103+0000","last_change":"2023-07-14T18:14:17.929550+0000","last_active":"2023-07-14T18:14:48.944103+0000","last_peered":"2023-07-14T18:14:48.944103+0000","last_clean":"2023-07-14T18:14:48.944103+0000","last_became_active":"2023-07-14T18:14:17.929204+0000","last_became_peered":"2023-07-14T18:14:17.929204+0000","last_unstale":"2023-07-14T18:14:48.944103+0000","last_undegraded":"2023-07-14T18:14:48.944103+0000","last_fullsized":"2023-07-14T18:14:48.944103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:46:07.635189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942348+0000","last_change":"2023-07-14T18:14:22.034899+0000","last_active":"2023-07-14T18:14:48.942348+0000","last_peered":"2023-07-14T18:14:48.942348+0000","last_clean":"2023-07-14T18:14:48.942348+0000","last_became_active":"2023-07-14T18:14:22.034524+0000","last_became_peered":"2023-07-14T18:14:22.034524+0000","last_unstale":"2023-07-14T18:14:48.942348+0000","last_undegraded":"2023-07-14T18:14:48.942348+0000","last_fullsized":"2023-07-14T18:14:48.942348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:44:56.300609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942705+0000","last_change":"2023-07-14T18:14:16.935285+0000","last_active":"2023-07-14T18:14:48.942705+0000","last_peered":"2023-07-14T18:14:48.942705+0000","last_clean":"2023-07-14T18:14:48.942705+0000","last_became_active":"2023-07-14T18:14:16.934888+0000","last_became_peered":"2023-07-14T18:14:16.934888+0000","last_unstale":"2023-07-14T18:14:48.942705+0000","last_undegraded":"2023-07-14T18:14:48.942705+0000","last_fullsized":"2023-07-14T18:14:48.942705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:46:09.021281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944372+0000","last_change":"2023-07-14T18:14:17.930982+0000","last_active":"2023-07-14T18:14:48.944372+0000","last_peered":"2023-07-14T18:14:48.944372+0000","last_clean":"2023-07-14T18:14:48.944372+0000","last_became_active":"2023-07-14T18:14:17.930661+0000","last_became_peered":"2023-07-14T18:14:17.930661+0000","last_unstale":"2023-07-14T18:14:48.944372+0000","last_undegraded":"2023-07-14T18:14:48.944372+0000","last_fullsized":"2023-07-14T18:14:48.944372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:42:10.943678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946615+0000","last_change":"2023-07-14T18:14:17.926204+0000","last_active":"2023-07-14T18:14:48.946615+0000","last_peered":"2023-07-14T18:14:48.946615+0000","last_clean":"2023-07-14T18:14:48.946615+0000","last_became_active":"2023-07-14T18:14:17.925689+0000","last_became_peered":"2023-07-14T18:14:17.925689+0000","last_unstale":"2023-07-14T18:14:48.946615+0000","last_undegraded":"2023-07-14T18:14:48.946615+0000","last_fullsized":"2023-07-14T18:14:48.946615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:32:10.441762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541397+0000","last_change":"2023-07-14T18:14:17.923146+0000","last_active":"2023-07-14T18:14:49.541397+0000","last_peered":"2023-07-14T18:14:49.541397+0000","last_clean":"2023-07-14T18:14:49.541397+0000","last_became_active":"2023-07-14T18:14:17.922734+0000","last_became_peered":"2023-07-14T18:14:17.922734+0000","last_unstale":"2023-07-14T18:14:49.541397+0000","last_undegraded":"2023-07-14T18:14:49.541397+0000","last_fullsized":"2023-07-14T18:14:49.541397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:01.011677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540904+0000","last_change":"2023-07-14T18:14:17.927397+0000","last_active":"2023-07-14T18:14:49.540904+0000","last_peered":"2023-07-14T18:14:49.540904+0000","last_clean":"2023-07-14T18:14:49.540904+0000","last_became_active":"2023-07-14T18:14:17.927000+0000","last_became_peered":"2023-07-14T18:14:17.927000+0000","last_unstale":"2023-07-14T18:14:49.540904+0000","last_undegraded":"2023-07-14T18:14:49.540904+0000","last_fullsized":"2023-07-14T18:14:49.540904+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:57:52.235148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541451+0000","last_change":"2023-07-14T18:14:17.935266+0000","last_active":"2023-07-14T18:14:49.541451+0000","last_peered":"2023-07-14T18:14:49.541451+0000","last_clean":"2023-07-14T18:14:49.541451+0000","last_became_active":"2023-07-14T18:14:17.934944+0000","last_became_peered":"2023-07-14T18:14:17.934944+0000","last_unstale":"2023-07-14T18:14:49.541451+0000","last_undegraded":"2023-07-14T18:14:49.541451+0000","last_fullsized":"2023-07-14T18:14:49.541451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:36:06.516397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.542810+0000","last_change":"2023-07-14T18:14:49.542810+0000","last_active":"2023-07-14T18:14:49.542810+0000","last_peered":"2023-07-14T18:14:49.542810+0000","last_clean":"2023-07-14T18:14:49.542810+0000","last_became_active":"2023-07-14T18:14:49.542421+0000","last_became_peered":"2023-07-14T18:14:49.542421+0000","last_unstale":"2023-07-14T18:14:49.542810+0000","last_undegraded":"2023-07-14T18:14:49.542810+0000","last_fullsized":"2023-07-14T18:14:49.542810+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:16:57.268229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946187+0000","last_change":"2023-07-14T18:14:17.930757+0000","last_active":"2023-07-14T18:14:48.946187+0000","last_peered":"2023-07-14T18:14:48.946187+0000","last_clean":"2023-07-14T18:14:48.946187+0000","last_became_active":"2023-07-14T18:14:17.930302+0000","last_became_peered":"2023-07-14T18:14:17.930302+0000","last_unstale":"2023-07-14T18:14:48.946187+0000","last_undegraded":"2023-07-14T18:14:48.946187+0000","last_fullsized":"2023-07-14T18:14:48.946187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:44:33.862208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541685+0000","last_change":"2023-07-14T18:14:17.930337+0000","last_active":"2023-07-14T18:14:49.541685+0000","last_peered":"2023-07-14T18:14:49.541685+0000","last_clean":"2023-07-14T18:14:49.541685+0000","last_became_active":"2023-07-14T18:14:17.930037+0000","last_became_peered":"2023-07-14T18:14:17.930037+0000","last_unstale":"2023-07-14T18:14:49.541685+0000","last_undegraded":"2023-07-14T18:14:49.541685+0000","last_fullsized":"2023-07-14T18:14:49.541685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:56:10.479187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.939298+0000","last_change":"2023-07-14T18:14:17.912750+0000","last_active":"2023-07-14T18:14:47.939298+0000","last_peered":"2023-07-14T18:14:47.939298+0000","last_clean":"2023-07-14T18:14:47.939298+0000","last_became_active":"2023-07-14T18:14:17.912402+0000","last_became_peered":"2023-07-14T18:14:17.912402+0000","last_unstale":"2023-07-14T18:14:47.939298+0000","last_undegraded":"2023-07-14T18:14:47.939298+0000","last_fullsized":"2023-07-14T18:14:47.939298+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:55:52.861047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942521+0000","last_change":"2023-07-14T18:14:17.930004+0000","last_active":"2023-07-14T18:14:48.942521+0000","last_peered":"2023-07-14T18:14:48.942521+0000","last_clean":"2023-07-14T18:14:48.942521+0000","last_became_active":"2023-07-14T18:14:17.929719+0000","last_became_peered":"2023-07-14T18:14:17.929719+0000","last_unstale":"2023-07-14T18:14:48.942521+0000","last_undegraded":"2023-07-14T18:14:48.942521+0000","last_fullsized":"2023-07-14T18:14:48.942521+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:51:50.359987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-14T18:14:47.936529+0000","last_change":"2023-07-14T18:14:17.919044+0000","last_active":"2023-07-14T18:14:47.936529+0000","last_peered":"2023-07-14T18:14:47.936529+0000","last_clean":"2023-07-14T18:14:47.936529+0000","last_became_active":"2023-07-14T18:14:17.918079+0000","last_became_peered":"2023-07-14T18:14:17.918079+0000","last_unstale":"2023-07-14T18:14:47.936529+0000","last_undegraded":"2023-07-14T18:14:47.936529+0000","last_fullsized":"2023-07-14T18:14:47.936529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:41:26.331275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946548+0000","last_change":"2023-07-14T18:14:17.921560+0000","last_active":"2023-07-14T18:14:48.946548+0000","last_peered":"2023-07-14T18:14:48.946548+0000","last_clean":"2023-07-14T18:14:48.946548+0000","last_became_active":"2023-07-14T18:14:17.921243+0000","last_became_peered":"2023-07-14T18:14:17.921243+0000","last_unstale":"2023-07-14T18:14:48.946548+0000","last_undegraded":"2023-07-14T18:14:48.946548+0000","last_fullsized":"2023-07-14T18:14:48.946548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:03:15.907251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942971+0000","last_change":"2023-07-14T18:14:17.927504+0000","last_active":"2023-07-14T18:14:48.942971+0000","last_peered":"2023-07-14T18:14:48.942971+0000","last_clean":"2023-07-14T18:14:48.942971+0000","last_became_active":"2023-07-14T18:14:17.927233+0000","last_became_peered":"2023-07-14T18:14:17.927233+0000","last_unstale":"2023-07-14T18:14:48.942971+0000","last_undegraded":"2023-07-14T18:14:48.942971+0000","last_fullsized":"2023-07-14T18:14:48.942971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:37:40.310852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946334+0000","last_change":"2023-07-14T18:14:17.932317+0000","last_active":"2023-07-14T18:14:48.946334+0000","last_peered":"2023-07-14T18:14:48.946334+0000","last_clean":"2023-07-14T18:14:48.946334+0000","last_became_active":"2023-07-14T18:14:17.931973+0000","last_became_peered":"2023-07-14T18:14:17.931973+0000","last_unstale":"2023-07-14T18:14:48.946334+0000","last_undegraded":"2023-07-14T18:14:48.946334+0000","last_fullsized":"2023-07-14T18:14:48.946334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:01:51.245001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":20480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281026,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":42419,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":38,"num_read_kb":67,"num_write":144,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":114066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416402,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":488509,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.39900000000000002}]},{"osd":1,"last update":"Fri Jul 14 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.70699999999999996}]},{"osd":2,"last update":"Fri Jul 14 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.502},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.52900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":4,"last update":"Fri Jul 14 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.752}]},{"osd":5,"last update":"Fri Jul 14 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.60699999999999998}]},{"osd":6,"last update":"Fri Jul 14 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.55100000000000005}]},{"osd":7,"last update":"Fri Jul 14 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.65700000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":9,"last update":"Fri Jul 14 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.83199999999999996}]},{"osd":10,"last update":"Fri Jul 14 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.81200000000000006}]}]},{"osd":10,"up_from":72,"seq":309237645335,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":1118009,"data_compressed":53365,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":1,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.71999999999999997}]},{"osd":2,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.74399999999999999}]},{"osd":3,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.57499999999999996}]},{"osd":4,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.502}]},{"osd":5,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.83899999999999997}]},{"osd":6,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.77700000000000002}]},{"osd":7,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.42199999999999999}]},{"osd":8,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.80500000000000005}]},{"osd":9,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.85799999999999998}]},{"osd":11,"last update":"Fri Jul 14 18:14:32 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.875}]}]},{"osd":9,"up_from":67,"seq":287762808860,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":496937,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.53000000000000003,"15min":0.54200000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.59999999999999998,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.55600000000000005,"15min":0.56999999999999995},"min":{"1min":0.35399999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85299999999999998,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.57299999999999995}]},{"osd":1,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.65500000000000003,"15min":0.66000000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.72499999999999998,"15min":0.73699999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.879,"5min":1.085,"15min":1.085},"last":0.55300000000000005}]},{"osd":2,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.51300000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93200000000000005,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.745,"15min":0.75900000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.91300000000000003,"5min":1.147,"15min":1.147},"last":0.82099999999999995}]},{"osd":3,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.65200000000000002,"15min":0.67400000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.81100000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.65100000000000002,"15min":0.66800000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77700000000000002,"5min":1.018,"15min":1.018},"last":0.73599999999999999}]},{"osd":4,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.77800000000000002,"15min":0.79200000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.89600000000000002,"5min":1.198,"15min":1.198},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.77500000000000002,"15min":0.78900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.96299999999999997,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.75700000000000001}]},{"osd":5,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.79600000000000004,"15min":0.81200000000000006},"min":{"1min":0.54800000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.84199999999999997,"5min":1.224,"15min":1.224},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.75,"15min":0.75800000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95799999999999996,"5min":1.113,"15min":1.113},"last":0.95799999999999996}]},{"osd":6,"last update":"Fri Jul 14 18:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.75900000000000001}]},{"osd":7,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.85799999999999998,"15min":0.88200000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.92800000000000005,"5min":1.323,"15min":1.323},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.85099999999999998,"15min":0.86899999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.89500000000000002,"5min":1.411,"15min":1.411},"last":0.85699999999999998}]},{"osd":8,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.90100000000000002,"15min":0.92100000000000004},"min":{"1min":0.63,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0620000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.90200000000000002,"15min":0.92100000000000004},"min":{"1min":0.60799999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0109999999999999,"5min":1.371,"15min":1.371},"last":1.0109999999999999}]},{"osd":10,"last update":"Fri Jul 14 18:14:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.872},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.90300000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]}]},{"osd":8,"up_from":59,"seq":253403070497,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":492179,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81899999999999995,"5min":1.238,"15min":1.238},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56299999999999994,"15min":0.56000000000000005},"min":{"1min":0.40300000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79700000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.47399999999999998}]},{"osd":1,"last update":"Fri Jul 14 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56599999999999995,"15min":0.56000000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.58499999999999996,"15min":0.58099999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.89900000000000002}]},{"osd":2,"last update":"Fri Jul 14 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47199999999999998,"15min":0.47099999999999997},"min":{"1min":0.308,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.71399999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.40899999999999997}]},{"osd":3,"last update":"Fri Jul 14 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.625,"15min":0.622},"min":{"1min":0.46000000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84999999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.55400000000000005}]},{"osd":4,"last update":"Fri Jul 14 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.624,"15min":0.622},"min":{"1min":0.41799999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.65800000000000003,"15min":0.65400000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.93200000000000005}]},{"osd":5,"last update":"Fri Jul 14 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.38300000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.375,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.437}]},{"osd":6,"last update":"Fri Jul 14 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.71699999999999997,"15min":0.72099999999999997},"min":{"1min":0.42399999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92900000000000005,"5min":1.29,"15min":1.29},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.755,"5min":0.71499999999999997,"15min":0.70799999999999996},"min":{"1min":0.55200000000000005,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.57699999999999996}]},{"osd":7,"last update":"Fri Jul 14 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.70499999999999996,"15min":0.69999999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.80900000000000005}]},{"osd":9,"last update":"Fri Jul 14 18:14:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.78900000000000003,"15min":0.78800000000000003},"min":{"1min":0.58699999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.499},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.78900000000000003,"15min":0.79400000000000004},"min":{"1min":0.54300000000000004,"5min":0.502,"15min":0.502},"max":{"1min":0.91900000000000004,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.61899999999999999}]},{"osd":10,"last update":"Fri Jul 14 18:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.76700000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332131,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":1079451,"data_compressed":53365,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.755,"15min":0.77300000000000002},"min":{"1min":0.38200000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.024,"5min":5.843,"15min":5.843},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.51100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.81299999999999994,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.40899999999999997}]},{"osd":1,"last update":"Fri Jul 14 18:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42399999999999999,"15min":0.42599999999999999},"min":{"1min":0.246,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.621,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.27900000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.45900000000000002}]},{"osd":2,"last update":"Fri Jul 14 18:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.41499999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.38700000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.61799999999999999}]},{"osd":3,"last update":"Fri Jul 14 18:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.65300000000000002,"15min":0.64800000000000002},"min":{"1min":0.46899999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.86899999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.66200000000000003,"15min":0.66400000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77300000000000002,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.76300000000000001}]},{"osd":4,"last update":"Fri Jul 14 18:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76700000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.499,"15min":0.497},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70499999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.41699999999999998}]},{"osd":5,"last update":"Fri Jul 14 18:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68200000000000005,"15min":0.68400000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88100000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.67200000000000004}]},{"osd":6,"last update":"Fri Jul 14 18:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":1.109,"15min":1.175},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.85899999999999999,"5min":13.189,"15min":13.189},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.499,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88100000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.71299999999999997}]},{"osd":8,"last update":"Fri Jul 14 18:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.70299999999999996,"15min":0.69299999999999995},"min":{"1min":0.51700000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69299999999999995,"15min":0.68999999999999995},"min":{"1min":0.46999999999999997,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.55700000000000005}]},{"osd":9,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.94599999999999995,"15min":0.97599999999999998},"min":{"1min":0.58099999999999996,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.91000000000000003,"5min":5.8250000000000002,"15min":5.8250000000000002},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.77500000000000002,"15min":0.77100000000000002},"min":{"1min":0.61799999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.78400000000000003}]},{"osd":10,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.748},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.73099999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495656,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":488017,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.46500000000000002,"15min":0.45500000000000002},"min":{"1min":0.32700000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.61299999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.49299999999999999,"15min":0.48199999999999998},"min":{"1min":0.35399999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65200000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.55500000000000005}]},{"osd":1,"last update":"Fri Jul 14 18:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.64700000000000002,"15min":0.64000000000000001},"min":{"1min":0.46000000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83499999999999996,"5min":0.87,"15min":0.87},"last":0.745},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67700000000000005,"15min":0.67400000000000004},"min":{"1min":0.438,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91400000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.69999999999999996}]},{"osd":2,"last update":"Fri Jul 14 18:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63600000000000001,"15min":0.626},"min":{"1min":0.46100000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0309999999999999,"5min":1.087,"15min":1.087},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68400000000000005,"15min":0.67800000000000005},"min":{"1min":0.52200000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.86099999999999999,"5min":1.097,"15min":1.097},"last":0.46000000000000002}]},{"osd":3,"last update":"Fri Jul 14 18:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.39200000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84499999999999997,"5min":1.03,"15min":1.03},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.64100000000000001,"15min":0.64900000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83699999999999997,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.66300000000000003}]},{"osd":4,"last update":"Fri Jul 14 18:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86199999999999999,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71799999999999997,"15min":0.72099999999999997},"min":{"1min":0.48099999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.80200000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.70799999999999996}]},{"osd":5,"last update":"Fri Jul 14 18:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.75700000000000001,"15min":0.76300000000000001},"min":{"1min":0.52300000000000002,"5min":0.435,"15min":0.435},"max":{"1min":1.0429999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.621},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.52900000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94899999999999995,"5min":1.163,"15min":1.163},"last":0.42899999999999999}]},{"osd":7,"last update":"Fri Jul 14 18:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.73299999999999998,"15min":0.72499999999999998},"min":{"1min":0.57499999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.69199999999999995}]},{"osd":8,"last update":"Fri Jul 14 18:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.80300000000000005,"15min":0.80700000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0900000000000001,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.746,"15min":0.75},"min":{"1min":0.56499999999999995,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.96599999999999997,"5min":1.079,"15min":1.079},"last":0.63400000000000001}]},{"osd":9,"last update":"Fri Jul 14 18:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.80500000000000005,"15min":0.81899999999999995},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.99199999999999999,"5min":1.147,"15min":1.147},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.83499999999999996,"15min":0.84599999999999997},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":0.93500000000000005,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.67900000000000005}]},{"osd":10,"last update":"Fri Jul 14 18:14:11 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.76000000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":492303,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53400000000000003,"15min":0.50800000000000001},"min":{"1min":0.40899999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.79000000000000004,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.55600000000000005,"15min":0.52500000000000002},"min":{"1min":0.39600000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.0660000000000001,"5min":1.272,"15min":1.272},"last":0.69699999999999995}]},{"osd":2,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.56100000000000005,"15min":0.53300000000000003},"min":{"1min":0.434,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.54500000000000004,"15min":0.51700000000000002},"min":{"1min":0.50700000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77800000000000002,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.72999999999999998}]},{"osd":3,"last update":"Fri Jul 14 18:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.56499999999999995,"15min":0.52400000000000002},"min":{"1min":0.39700000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.93500000000000005,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.60199999999999998,"15min":0.58199999999999996},"min":{"1min":0.45600000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91300000000000003,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.60099999999999998}]},{"osd":4,"last update":"Fri Jul 14 18:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.46100000000000002,"15min":0.44700000000000001},"min":{"1min":0.29699999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.45600000000000002,"15min":0.44400000000000001},"min":{"1min":0.316,"5min":0.255,"15min":0.255},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.45800000000000002}]},{"osd":5,"last update":"Fri Jul 14 18:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82099999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.439,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.66900000000000004}]},{"osd":6,"last update":"Fri Jul 14 18:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.67200000000000004,"15min":0.66900000000000004},"min":{"1min":0.46000000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90200000000000002,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.629},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67600000000000005,"15min":0.66800000000000004},"min":{"1min":0.47499999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.0029999999999999,"5min":1.23,"15min":1.23},"last":0.55400000000000005}]},{"osd":7,"last update":"Fri Jul 14 18:14:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55600000000000005,"15min":0.54800000000000004},"min":{"1min":0.35399999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57899999999999996,"15min":0.57299999999999995},"min":{"1min":0.36499999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.68700000000000006}]},{"osd":8,"last update":"Fri Jul 14 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.64500000000000002,"15min":0.63600000000000001},"min":{"1min":0.42799999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.69099999999999995,"15min":0.67900000000000005},"min":{"1min":0.51900000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.84699999999999998}]},{"osd":9,"last update":"Fri Jul 14 18:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.90000000000000002}]},{"osd":10,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.875},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.79300000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":488621,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 18:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.56999999999999995,"15min":0.54200000000000004},"min":{"1min":0.35999999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.58599999999999997,"15min":0.57199999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.57499999999999996}]},{"osd":2,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.59699999999999998,"15min":0.55500000000000005},"min":{"1min":0.39800000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61199999999999999,"15min":0.57199999999999995},"min":{"1min":0.434,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.754}]},{"osd":3,"last update":"Fri Jul 14 18:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.443,"15min":0.42799999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.438},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.48299999999999998,"15min":0.46300000000000002},"min":{"1min":0.36399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.38600000000000001}]},{"osd":4,"last update":"Fri Jul 14 18:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.57799999999999996,"15min":0.55700000000000005},"min":{"1min":0.41799999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.64000000000000001,"15min":0.61699999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.627}]},{"osd":5,"last update":"Fri Jul 14 18:14:20 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.71599999999999997,"15min":0.68400000000000005},"min":{"1min":0.47699999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":3.3740000000000001,"5min":3.3740000000000001,"15min":3.3740000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.749,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.78400000000000003}]},{"osd":6,"last update":"Fri Jul 14 18:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64100000000000001,"15min":0.61699999999999999},"min":{"1min":0.44700000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.66400000000000003,"15min":0.64000000000000001},"min":{"1min":0.433,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.52100000000000002}]},{"osd":7,"last update":"Fri Jul 14 18:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.68500000000000005,"15min":0.67500000000000004},"min":{"1min":0.58399999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.76100000000000001,"15min":0.74099999999999999},"min":{"1min":0.56599999999999995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":0.74099999999999999}]},{"osd":8,"last update":"Fri Jul 14 18:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.80400000000000005,"15min":0.79400000000000004},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.874,"5min":0.78000000000000003,"15min":0.76400000000000001},"min":{"1min":0.58199999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.82199999999999995}]},{"osd":9,"last update":"Fri Jul 14 18:14:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.77000000000000002,"15min":0.77200000000000002},"min":{"1min":0.57099999999999995,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":1.157,"15min":1.2150000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0389999999999999,"5min":7.8689999999999998,"15min":7.8689999999999998},"last":0.53900000000000003}]},{"osd":10,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.76900000000000002}]},{"osd":11,"last update":"Fri Jul 14 18:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.80400000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":527170,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.36099999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.54100000000000004,"15min":0.51900000000000002},"min":{"1min":0.39700000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.84099999999999997}]},{"osd":1,"last update":"Fri Jul 14 18:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.54400000000000004,"15min":0.50900000000000001},"min":{"1min":0.41199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.54300000000000004,"15min":0.50900000000000001},"min":{"1min":0.39000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.59699999999999998}]},{"osd":3,"last update":"Fri Jul 14 18:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60699999999999998,"15min":0.57299999999999995},"min":{"1min":0.441,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.877,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65100000000000002,"15min":0.65700000000000003},"min":{"1min":0.53200000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81299999999999994,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":0.79100000000000004}]},{"osd":4,"last update":"Fri Jul 14 18:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.59199999999999997,"15min":0.57199999999999995},"min":{"1min":0.47999999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88400000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.623,"15min":0.60299999999999998},"min":{"1min":0.443,"5min":0.433,"15min":0.433},"max":{"1min":0.82799999999999996,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.66900000000000004}]},{"osd":5,"last update":"Fri Jul 14 18:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.63600000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.48799999999999999,"15min":0.47999999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.51900000000000002}]},{"osd":6,"last update":"Fri Jul 14 18:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64400000000000002,"15min":0.63400000000000001},"min":{"1min":0.52100000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84199999999999997,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.65600000000000003,"15min":0.63100000000000001},"min":{"1min":0.54600000000000004,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84799999999999998,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.94599999999999995}]},{"osd":7,"last update":"Fri Jul 14 18:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.52400000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.92100000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.65400000000000003,"15min":0.65600000000000003},"min":{"1min":0.44900000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.873,"5min":1.014,"15min":1.014},"last":0.93100000000000005}]},{"osd":8,"last update":"Fri Jul 14 18:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63500000000000001,"15min":0.63900000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.873,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.63200000000000001,"15min":0.63300000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.74099999999999999}]},{"osd":9,"last update":"Fri Jul 14 18:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":1.069},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":1.0029999999999999}]},{"osd":10,"last update":"Fri Jul 14 18:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.82199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]}]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":478555,"data_compressed":14651,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.43099999999999999,"15min":0.39100000000000001},"min":{"1min":0.32700000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45000000000000001,"15min":0.41399999999999998},"min":{"1min":0.34000000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.92400000000000004,"5min":0.998,"15min":0.998},"last":0.45600000000000002}]},{"osd":1,"last update":"Fri Jul 14 18:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.59799999999999998,"15min":0.57599999999999996},"min":{"1min":0.44500000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.85299999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.625,"15min":0.58999999999999997},"min":{"1min":0.54500000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.032,"5min":1.29,"15min":1.29},"last":0.752}]},{"osd":2,"last update":"Fri Jul 14 18:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.61399999999999999,"15min":0.57899999999999996},"min":{"1min":0.52100000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.81599999999999995,"5min":1.087,"15min":1.087},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61799999999999999,"15min":0.59999999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95499999999999996,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.63500000000000001}]},{"osd":4,"last update":"Fri Jul 14 18:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.61599999999999999,"15min":0.58499999999999996},"min":{"1min":0.47899999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.1200000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.65900000000000003,"15min":0.63300000000000001},"min":{"1min":0.60499999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.135,"5min":1.153,"15min":1.153},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Jul 14 18:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.64600000000000002,"15min":0.63},"min":{"1min":0.54800000000000004,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.621},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.63900000000000001,"15min":0.61299999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.93100000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.80800000000000005}]},{"osd":6,"last update":"Fri Jul 14 18:14:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.621,"15min":0.59899999999999998},"min":{"1min":0.40699999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.432},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65400000000000003,"15min":0.622},"min":{"1min":0.47999999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.39500000000000002}]},{"osd":7,"last update":"Fri Jul 14 18:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.70399999999999996,"15min":0.68600000000000005},"min":{"1min":0.57199999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.71099999999999997,"15min":0.69899999999999995},"min":{"1min":0.59399999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.76700000000000002}]},{"osd":8,"last update":"Fri Jul 14 18:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.72799999999999998}]},{"osd":9,"last update":"Fri Jul 14 18:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.498}]},{"osd":10,"last update":"Fri Jul 14 18:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.66300000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":489097,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.48999999999999999,"15min":0.46000000000000002},"min":{"1min":0.39700000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.83799999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.50800000000000001,"15min":0.47799999999999998},"min":{"1min":0.46700000000000003,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.747,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.39600000000000002}]},{"osd":1,"last update":"Fri Jul 14 18:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.36899999999999999,"15min":0.35499999999999998},"min":{"1min":0.26900000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":3.7890000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.38600000000000001,"15min":0.377},"min":{"1min":0.28699999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.71899999999999997,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":1.3320000000000001}]},{"osd":2,"last update":"Fri Jul 14 18:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.55100000000000005,"15min":0.54000000000000004},"min":{"1min":0.42499999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73899999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":2.3090000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.56399999999999995,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.88800000000000001,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":3.6989999999999998}]},{"osd":3,"last update":"Fri Jul 14 18:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.54500000000000004,"15min":0.51700000000000002},"min":{"1min":0.48699999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.91800000000000004,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.54300000000000004,"15min":0.52300000000000002},"min":{"1min":0.46400000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82799999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":2.5640000000000001}]},{"osd":5,"last update":"Fri Jul 14 18:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.59199999999999997,"15min":0.56000000000000005},"min":{"1min":0.51000000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":3.7490000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.59799999999999998,"15min":0.56599999999999995},"min":{"1min":0.53400000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.72899999999999998}]},{"osd":6,"last update":"Fri Jul 14 18:14:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67000000000000004,"15min":0.65800000000000003},"min":{"1min":0.50800000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.98799999999999999,"5min":1.075,"15min":1.075},"last":3.4780000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.68300000000000005,"15min":0.66600000000000004},"min":{"1min":0.498,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.006,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":3.8620000000000001}]},{"osd":7,"last update":"Fri Jul 14 18:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.58699999999999997,"15min":0.56999999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.97399999999999998,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":3.8999999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.59799999999999998,"15min":0.58399999999999996},"min":{"1min":0.40000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0249999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":3.2250000000000001}]},{"osd":8,"last update":"Fri Jul 14 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64600000000000002,"15min":0.63900000000000001},"min":{"1min":0.48599999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.877,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":2.359},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.755,"15min":0.749},"min":{"1min":0.52500000000000002,"5min":0.5,"15min":0.5},"max":{"1min":0.96799999999999997,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":3.1739999999999999}]},{"osd":9,"last update":"Fri Jul 14 18:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":2.3889999999999998}]},{"osd":10,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":3.8239999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":2.52}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.548}]}]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":1083878,"data_compressed":53365,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:14:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55600000000000005,"15min":0.55100000000000005},"min":{"1min":0.39000000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56999999999999995,"15min":0.57899999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.876,"5min":1.645,"15min":1.645},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Jul 14 18:14:18 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.56699999999999995,"15min":0.55100000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.36299999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80400000000000005,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.72799999999999998}]},{"osd":2,"last update":"Fri Jul 14 18:14:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.41999999999999998,"15min":0.42399999999999999},"min":{"1min":0.26100000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.441},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.47599999999999998,"15min":0.49199999999999999},"min":{"1min":0.27500000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.72299999999999998,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.61099999999999999}]},{"osd":3,"last update":"Fri Jul 14 18:14:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60899999999999999,"15min":0.62},"min":{"1min":0.378,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84599999999999997,"5min":1.591,"15min":1.591},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.40500000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.81299999999999994,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.64300000000000002}]},{"osd":4,"last update":"Fri Jul 14 18:14:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61099999999999999,"15min":0.61499999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93400000000000005,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84499999999999997,"5min":1.504,"15min":1.504},"last":0.57499999999999996}]},{"osd":6,"last update":"Fri Jul 14 18:14:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.65400000000000003,"15min":0.66000000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.79300000000000004,"5min":1.627,"15min":1.627},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65100000000000002,"15min":0.63600000000000001},"min":{"1min":0.51000000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.83099999999999996,"5min":1.036,"15min":1.036},"last":0.746}]},{"osd":7,"last update":"Fri Jul 14 18:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91700000000000004,"5min":1.554,"15min":1.554},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65200000000000002,"15min":0.64900000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.69899999999999995}]},{"osd":8,"last update":"Fri Jul 14 18:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58599999999999997,"15min":0.57699999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.38300000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.874}]},{"osd":9,"last update":"Fri Jul 14 18:14:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72899999999999998,"15min":0.72999999999999998},"min":{"1min":0.51800000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92600000000000005,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.76500000000000001,"15min":0.76800000000000002},"min":{"1min":0.46700000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.98199999999999998,"5min":1.149,"15min":1.149},"last":0.89200000000000002}]},{"osd":10,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.85399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34102,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34102,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34102,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":26386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":39243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":40045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T18:14:53.728 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-14T18:14:53.728 INFO:tasks.cephadm:Setting up client nodes... 2023-07-14T18:14:53.729 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-07-14T18:14:54.483 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:14:55.069 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:54 smithi133 ceph-mon[126384]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T18:14:55.070 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:54 smithi133 ceph-mon[126384]: pgmap v293: 129 pgs: 129 active+clean; 618 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.1 KiB/s wr, 16 op/s; 72 B/s, 0 objects/s recovering 2023-07-14T18:14:55.070 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:54 smithi133 ceph-mon[126384]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T18:14:55.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:54 smithi161 ceph-mon[128993]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T18:14:55.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:54 smithi161 ceph-mon[128993]: pgmap v293: 129 pgs: 129 active+clean; 618 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.1 KiB/s wr, 16 op/s; 72 B/s, 0 objects/s recovering 2023-07-14T18:14:55.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:54 smithi161 ceph-mon[128993]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T18:14:55.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:54 smithi171 ceph-mon[129241]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T18:14:55.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:54 smithi171 ceph-mon[129241]: pgmap v293: 129 pgs: 129 active+clean; 618 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.1 KiB/s wr, 16 op/s; 72 B/s, 0 objects/s recovering 2023-07-14T18:14:55.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:54 smithi171 ceph-mon[129241]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T18:14:56.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.005+0000 7f3c523b9700 1 -- 172.21.15.133:0/3542112847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c4c100ee0 msgr2=0x7f3c4c101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:56.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.005+0000 7f3c523b9700 1 --2- 172.21.15.133:0/3542112847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c4c100ee0 0x7f3c4c101330 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009990 tx=0x7f3c40009ca0 comp rx=0 tx=0).stop 2023-07-14T18:14:56.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.005+0000 7f3c523b9700 1 -- 172.21.15.133:0/3542112847 shutdown_connections 2023-07-14T18:14:56.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.005+0000 7f3c523b9700 1 --2- 172.21.15.133:0/3542112847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c4c0ffce0 0x7f3c4c1000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:56.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.005+0000 7f3c523b9700 1 --2- 172.21.15.133:0/3542112847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c4c101870 0x7f3c4c108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:56.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.005+0000 7f3c523b9700 1 --2- 172.21.15.133:0/3542112847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c4c100ee0 0x7f3c4c101330 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:56.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.005+0000 7f3c523b9700 1 -- 172.21.15.133:0/3542112847 >> 172.21.15.133:0/3542112847 conn(0x7f3c4c0fb330 msgr2=0x7f3c4c0fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:56.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.005+0000 7f3c523b9700 1 -- 172.21.15.133:0/3542112847 shutdown_connections 2023-07-14T18:14:56.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.005+0000 7f3c523b9700 1 -- 172.21.15.133:0/3542112847 wait complete. 2023-07-14T18:14:56.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.007+0000 7f3c523b9700 1 Processor -- start 2023-07-14T18:14:56.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.007+0000 7f3c523b9700 1 -- start start 2023-07-14T18:14:56.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.007+0000 7f3c523b9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c4c0ffce0 0x7f3c4c197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:56.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.007+0000 7f3c523b9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c4c100ee0 0x7f3c4c198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:56.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.008+0000 7f3c523b9700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c4c101870 0x7f3c4c19f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:56.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.008+0000 7f3c523b9700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c19f770 con 0x7f3c4c100ee0 2023-07-14T18:14:56.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.008+0000 7f3c523b9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c19f8b0 con 0x7f3c4c0ffce0 2023-07-14T18:14:56.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.008+0000 7f3c523b9700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c19fa70 con 0x7f3c4c101870 2023-07-14T18:14:56.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.008+0000 7f3c4bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c4c0ffce0 0x7f3c4c197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:56.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.008+0000 7f3c4b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c4c100ee0 0x7f3c4c198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:56.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.008+0000 7f3c4bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c4c0ffce0 0x7f3c4c197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:34186/0 (socket says 172.21.15.133:34186) 2023-07-14T18:14:56.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.008+0000 7f3c4bfff700 1 -- 172.21.15.133:0/3740455811 learned_addr learned my addr 172.21.15.133:0/3740455811 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:56.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.008+0000 7f3c50956700 1 --2- 172.21.15.133:0/3740455811 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c4c101870 0x7f3c4c19f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:56.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.008+0000 7f3c4b7fe700 1 -- 172.21.15.133:0/3740455811 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c4c101870 msgr2=0x7f3c4c19f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:56.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.008+0000 7f3c4b7fe700 1 --2- 172.21.15.133:0/3740455811 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c4c101870 0x7f3c4c19f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:56.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.008+0000 7f3c4b7fe700 1 -- 172.21.15.133:0/3740455811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c4c0ffce0 msgr2=0x7f3c4c197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:56.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.008+0000 7f3c4b7fe700 1 --2- 172.21.15.133:0/3740455811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c4c0ffce0 0x7f3c4c197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:56.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.008+0000 7f3c4b7fe700 1 -- 172.21.15.133:0/3740455811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c400095f0 con 0x7f3c4c100ee0 2023-07-14T18:14:56.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.008+0000 7f3c4bfff700 1 --2- 172.21.15.133:0/3740455811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c4c0ffce0 0x7f3c4c197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:14:56.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.008+0000 7f3c4b7fe700 1 --2- 172.21.15.133:0/3740455811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c4c100ee0 0x7f3c4c198160 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3c4000b680 tx=0x7f3c40009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:56.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.009+0000 7f3c497fa700 1 -- 172.21.15.133:0/3740455811 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4001d070 con 0x7f3c4c100ee0 2023-07-14T18:14:56.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.009+0000 7f3c523b9700 1 -- 172.21.15.133:0/3740455811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c4c19fcf0 con 0x7f3c4c100ee0 2023-07-14T18:14:56.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.009+0000 7f3c497fa700 1 -- 172.21.15.133:0/3740455811 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c40021410 con 0x7f3c4c100ee0 2023-07-14T18:14:56.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.009+0000 7f3c497fa700 1 -- 172.21.15.133:0/3740455811 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c40017650 con 0x7f3c4c100ee0 2023-07-14T18:14:56.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.009+0000 7f3c523b9700 1 -- 172.21.15.133:0/3740455811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4c1a0160 con 0x7f3c4c100ee0 2023-07-14T18:14:56.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.010+0000 7f3c523b9700 1 -- 172.21.15.133:0/3740455811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c4c10b640 con 0x7f3c4c100ee0 2023-07-14T18:14:56.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.012+0000 7f3c497fa700 1 -- 172.21.15.133:0/3740455811 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f3c40021810 con 0x7f3c4c100ee0 2023-07-14T18:14:56.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.013+0000 7f3c497fa700 1 --2- 172.21.15.133:0/3740455811 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3c340a11e0 0x7f3c340a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:56.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.013+0000 7f3c497fa700 1 -- 172.21.15.133:0/3740455811 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3c400ce180 con 0x7f3c4c100ee0 2023-07-14T18:14:56.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.014+0000 7f3c4bfff700 1 --2- 172.21.15.133:0/3740455811 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3c340a11e0 0x7f3c340a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:56.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.014+0000 7f3c4bfff700 1 --2- 172.21.15.133:0/3740455811 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3c340a11e0 0x7f3c340a3690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c199100 tx=0x7f3c3c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:56.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.017+0000 7f3c497fa700 1 -- 172.21.15.133:0/3740455811 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f3c40097490 con 0x7f3c4c100ee0 2023-07-14T18:14:56.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.291+0000 7f3c523b9700 1 -- 172.21.15.133:0/3740455811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f3c4c1a07b0 con 0x7f3c4c100ee0 2023-07-14T18:14:56.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.298+0000 7f3c497fa700 1 -- 172.21.15.133:0/3740455811 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f3c4009aab0 con 0x7f3c4c100ee0 2023-07-14T18:14:56.341 INFO:teuthology.orchestra.run.smithi133.stdout:[client.0] 2023-07-14T18:14:56.342 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQCgkLFkp+t2ERAAiv+CuhB1P3yWFjIwpQBl8g== 2023-07-14T18:14:56.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.300+0000 7f3c523b9700 1 -- 172.21.15.133:0/3740455811 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3c340a11e0 msgr2=0x7f3c340a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:56.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.300+0000 7f3c523b9700 1 --2- 172.21.15.133:0/3740455811 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3c340a11e0 0x7f3c340a3690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c199100 tx=0x7f3c3c009260 comp rx=0 tx=0).stop 2023-07-14T18:14:56.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.300+0000 7f3c523b9700 1 -- 172.21.15.133:0/3740455811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c4c100ee0 msgr2=0x7f3c4c198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:56.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.300+0000 7f3c523b9700 1 --2- 172.21.15.133:0/3740455811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c4c100ee0 0x7f3c4c198160 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3c4000b680 tx=0x7f3c40009580 comp rx=0 tx=0).stop 2023-07-14T18:14:56.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.300+0000 7f3c523b9700 1 -- 172.21.15.133:0/3740455811 shutdown_connections 2023-07-14T18:14:56.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.300+0000 7f3c523b9700 1 --2- 172.21.15.133:0/3740455811 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3c340a11e0 0x7f3c340a3690 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:56.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.300+0000 7f3c523b9700 1 --2- 172.21.15.133:0/3740455811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c4c0ffce0 0x7f3c4c197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:56.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.300+0000 7f3c523b9700 1 --2- 172.21.15.133:0/3740455811 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c4c101870 0x7f3c4c19f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:56.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.300+0000 7f3c523b9700 1 --2- 172.21.15.133:0/3740455811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c4c100ee0 0x7f3c4c198160 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:56.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.300+0000 7f3c523b9700 1 -- 172.21.15.133:0/3740455811 >> 172.21.15.133:0/3740455811 conn(0x7f3c4c0fb330 msgr2=0x7f3c4c0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:56.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.301+0000 7f3c523b9700 1 -- 172.21.15.133:0/3740455811 shutdown_connections 2023-07-14T18:14:56.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:56.301+0000 7f3c523b9700 1 -- 172.21.15.133:0/3740455811 wait complete. 2023-07-14T18:14:56.883 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T18:14:56.883 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-07-14T18:14:56.883 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-07-14T18:14:56.983 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-07-14T18:14:56.984 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-07-14T18:14:56.984 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph mgr dump --format=json 2023-07-14T18:14:57.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:56 smithi133 ceph-mon[126384]: pgmap v294: 129 pgs: 129 active+clean; 618 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.2 KiB/s wr, 16 op/s; 72 B/s, 0 objects/s recovering 2023-07-14T18:14:57.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:56 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3740455811' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-14T18:14:57.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:56 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3740455811' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-14T18:14:57.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:56 smithi161 ceph-mon[128993]: pgmap v294: 129 pgs: 129 active+clean; 618 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.2 KiB/s wr, 16 op/s; 72 B/s, 0 objects/s recovering 2023-07-14T18:14:57.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:56 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3740455811' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-14T18:14:57.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:56 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3740455811' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-14T18:14:57.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:56 smithi171 ceph-mon[129241]: pgmap v294: 129 pgs: 129 active+clean; 618 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.2 KiB/s wr, 16 op/s; 72 B/s, 0 objects/s recovering 2023-07-14T18:14:57.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:56 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3740455811' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-14T18:14:57.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:56 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3740455811' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-14T18:14:57.742 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:14:59.279 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.234+0000 7fc362088700 1 -- 172.21.15.133:0/3414232906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc35c104fd0 msgr2=0x7fc35c10cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:59.279 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.234+0000 7fc362088700 1 --2- 172.21.15.133:0/3414232906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc35c104fd0 0x7fc35c10cb50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc354009f90 tx=0x7fc35400b330 comp rx=0 tx=0).stop 2023-07-14T18:14:59.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.234+0000 7fc362088700 1 -- 172.21.15.133:0/3414232906 shutdown_connections 2023-07-14T18:14:59.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.234+0000 7fc362088700 1 --2- 172.21.15.133:0/3414232906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc35c104fd0 0x7fc35c10cb50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:59.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.234+0000 7fc362088700 1 --2- 172.21.15.133:0/3414232906 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc35c0fff30 0x7fc35c100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:59.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.234+0000 7fc362088700 1 --2- 172.21.15.133:0/3414232906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc35c1008d0 0x7fc35c1048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:59.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.234+0000 7fc362088700 1 -- 172.21.15.133:0/3414232906 >> 172.21.15.133:0/3414232906 conn(0x7fc35c0fb790 msgr2=0x7fc35c0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:59.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.234+0000 7fc362088700 1 -- 172.21.15.133:0/3414232906 shutdown_connections 2023-07-14T18:14:59.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.234+0000 7fc362088700 1 -- 172.21.15.133:0/3414232906 wait complete. 2023-07-14T18:14:59.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.235+0000 7fc362088700 1 Processor -- start 2023-07-14T18:14:59.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.235+0000 7fc362088700 1 -- start start 2023-07-14T18:14:59.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.236+0000 7fc362088700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc35c0fff30 0x7fc35c1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:59.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.236+0000 7fc362088700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc35c1008d0 0x7fc35c198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:59.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.236+0000 7fc362088700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc35c104fd0 0x7fc35c19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:59.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.236+0000 7fc362088700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c19d0b0 con 0x7fc35c104fd0 2023-07-14T18:14:59.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.236+0000 7fc362088700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c19d220 con 0x7fc35c1008d0 2023-07-14T18:14:59.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.236+0000 7fc362088700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c19d3e0 con 0x7fc35c0fff30 2023-07-14T18:14:59.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.236+0000 7fc35bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc35c104fd0 0x7fc35c19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:59.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.236+0000 7fc35affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc35c1008d0 0x7fc35c198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:59.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.236+0000 7fc35b7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc35c0fff30 0x7fc35c1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:59.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.236+0000 7fc35bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc35c104fd0 0x7fc35c19caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50038/0 (socket says 172.21.15.133:50038) 2023-07-14T18:14:59.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.236+0000 7fc35affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc35c1008d0 0x7fc35c198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:55658/0 (socket says 172.21.15.133:55658) 2023-07-14T18:14:59.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.236+0000 7fc35bfff700 1 -- 172.21.15.133:0/1927000307 learned_addr learned my addr 172.21.15.133:0/1927000307 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:14:59.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.236+0000 7fc35bfff700 1 -- 172.21.15.133:0/1927000307 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc35c0fff30 msgr2=0x7fc35c1980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:59.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.236+0000 7fc35bfff700 1 --2- 172.21.15.133:0/1927000307 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc35c0fff30 0x7fc35c1980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:59.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.236+0000 7fc35bfff700 1 -- 172.21.15.133:0/1927000307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc35c1008d0 msgr2=0x7fc35c198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:59.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.236+0000 7fc35bfff700 1 --2- 172.21.15.133:0/1927000307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc35c1008d0 0x7fc35c198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:59.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.236+0000 7fc35bfff700 1 -- 172.21.15.133:0/1927000307 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3500095f0 con 0x7fc35c104fd0 2023-07-14T18:14:59.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.237+0000 7fc35bfff700 1 --2- 172.21.15.133:0/1927000307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc35c104fd0 0x7fc35c19caa0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc35400b6c0 tx=0x7fc354004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:59.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.237+0000 7fc358ff9700 1 -- 172.21.15.133:0/1927000307 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35400e040 con 0x7fc35c104fd0 2023-07-14T18:14:59.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.237+0000 7fc358ff9700 1 -- 172.21.15.133:0/1927000307 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc354008e80 con 0x7fc35c104fd0 2023-07-14T18:14:59.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.237+0000 7fc362088700 1 -- 172.21.15.133:0/1927000307 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc354009c40 con 0x7fc35c104fd0 2023-07-14T18:14:59.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.237+0000 7fc358ff9700 1 -- 172.21.15.133:0/1927000307 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc354005040 con 0x7fc35c104fd0 2023-07-14T18:14:59.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.237+0000 7fc362088700 1 -- 172.21.15.133:0/1927000307 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc35c19d8f0 con 0x7fc35c104fd0 2023-07-14T18:14:59.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.238+0000 7fc362088700 1 -- 172.21.15.133:0/1927000307 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc35c04e340 con 0x7fc35c104fd0 2023-07-14T18:14:59.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.240+0000 7fc358ff9700 1 -- 172.21.15.133:0/1927000307 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fc35401d470 con 0x7fc35c104fd0 2023-07-14T18:14:59.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.241+0000 7fc358ff9700 1 --2- 172.21.15.133:0/1927000307 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc3440a11e0 0x7fc3440a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:14:59.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.241+0000 7fc358ff9700 1 -- 172.21.15.133:0/1927000307 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc3540ce2d0 con 0x7fc35c104fd0 2023-07-14T18:14:59.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.242+0000 7fc35b7fe700 1 --2- 172.21.15.133:0/1927000307 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc3440a11e0 0x7fc3440a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:14:59.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.242+0000 7fc35b7fe700 1 --2- 172.21.15.133:0/1927000307 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc3440a11e0 0x7fc3440a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc34c009c60 tx=0x7fc34c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:14:59.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.245+0000 7fc358ff9700 1 -- 172.21.15.133:0/1927000307 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fc3540975e0 con 0x7fc35c104fd0 2023-07-14T18:14:59.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:58 smithi161 ceph-mon[128993]: pgmap v295: 129 pgs: 129 active+clean; 618 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.2 KiB/s wr, 15 op/s; 61 B/s, 0 objects/s recovering 2023-07-14T18:14:59.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:58 smithi171 ceph-mon[129241]: pgmap v295: 129 pgs: 129 active+clean; 618 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.2 KiB/s wr, 15 op/s; 61 B/s, 0 objects/s recovering 2023-07-14T18:14:59.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:58 smithi133 ceph-mon[126384]: pgmap v295: 129 pgs: 129 active+clean; 618 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.2 KiB/s wr, 15 op/s; 61 B/s, 0 objects/s recovering 2023-07-14T18:14:59.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.495+0000 7fc362088700 1 -- 172.21.15.133:0/1927000307 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fc35c1a4f30 con 0x7fc35c104fd0 2023-07-14T18:14:59.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.500+0000 7fc358ff9700 1 -- 172.21.15.133:0/1927000307 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260130 (secure 0 0 0) 0x7fc35402e350 con 0x7fc35c104fd0 2023-07-14T18:14:59.547 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:14:59.549 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.505+0000 7fc362088700 1 -- 172.21.15.133:0/1927000307 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc3440a11e0 msgr2=0x7fc3440a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:59.549 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.505+0000 7fc362088700 1 --2- 172.21.15.133:0/1927000307 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc3440a11e0 0x7fc3440a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc34c009c60 tx=0x7fc34c009260 comp rx=0 tx=0).stop 2023-07-14T18:14:59.549 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.505+0000 7fc362088700 1 -- 172.21.15.133:0/1927000307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc35c104fd0 msgr2=0x7fc35c19caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:14:59.549 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.505+0000 7fc362088700 1 --2- 172.21.15.133:0/1927000307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc35c104fd0 0x7fc35c19caa0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc35400b6c0 tx=0x7fc354004ce0 comp rx=0 tx=0).stop 2023-07-14T18:14:59.549 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.505+0000 7fc362088700 1 -- 172.21.15.133:0/1927000307 shutdown_connections 2023-07-14T18:14:59.550 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.505+0000 7fc362088700 1 --2- 172.21.15.133:0/1927000307 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc3440a11e0 0x7fc3440a3690 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:59.550 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.505+0000 7fc362088700 1 --2- 172.21.15.133:0/1927000307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc35c104fd0 0x7fc35c19caa0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:59.550 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.505+0000 7fc362088700 1 --2- 172.21.15.133:0/1927000307 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc35c0fff30 0x7fc35c1980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:59.550 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.505+0000 7fc362088700 1 --2- 172.21.15.133:0/1927000307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc35c1008d0 0x7fc35c198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:14:59.550 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.505+0000 7fc362088700 1 -- 172.21.15.133:0/1927000307 >> 172.21.15.133:0/1927000307 conn(0x7fc35c0fb790 msgr2=0x7fc35c10b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:14:59.550 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.505+0000 7fc362088700 1 -- 172.21.15.133:0/1927000307 shutdown_connections 2023-07-14T18:14:59.550 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:14:59.505+0000 7fc362088700 1 -- 172.21.15.133:0/1927000307 wait complete. 2023-07-14T18:15:00.028 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":17,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":196197529},{"type":"v1","addr":"172.21.15.133:6801","nonce":196197529}]},"active_addr":"172.21.15.133:6801/196197529","active_change":"2023-07-14T18:07:47.346809+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.133:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":624488340}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":2547348609}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":2817674038}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":605654724}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":4177398123}]}]} 2023-07-14T18:15:00.038 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-07-14T18:15:00.039 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-07-14T18:15:00.039 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph osd dump --format=json 2023-07-14T18:15:00.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:14:59 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1927000307' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-14T18:15:00.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:14:59 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1927000307' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-14T18:15:00.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:14:59 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1927000307' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-14T18:15:00.785 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:01.319 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:00 smithi133 ceph-mon[126384]: pgmap v296: 129 pgs: 129 active+clean; 627 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.4 KiB/s wr, 16 op/s; 55 B/s, 0 objects/s recovering 2023-07-14T18:15:01.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:00 smithi161 ceph-mon[128993]: pgmap v296: 129 pgs: 129 active+clean; 627 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.4 KiB/s wr, 16 op/s; 55 B/s, 0 objects/s recovering 2023-07-14T18:15:01.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:00 smithi171 ceph-mon[129241]: pgmap v296: 129 pgs: 129 active+clean; 627 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.4 KiB/s wr, 16 op/s; 55 B/s, 0 objects/s recovering 2023-07-14T18:15:02.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.224+0000 7fb8028c1700 1 -- 172.21.15.133:0/4268288532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7fc0fff30 msgr2=0x7fb7fc100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:02.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.224+0000 7fb8028c1700 1 --2- 172.21.15.133:0/4268288532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7fc0fff30 0x7fb7fc100300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec0099c0 tx=0x7fb7ec009cd0 comp rx=0 tx=0).stop 2023-07-14T18:15:02.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.224+0000 7fb8028c1700 1 -- 172.21.15.133:0/4268288532 shutdown_connections 2023-07-14T18:15:02.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.224+0000 7fb8028c1700 1 --2- 172.21.15.133:0/4268288532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7fc104fd0 0x7fb7fc10cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:02.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.224+0000 7fb8028c1700 1 --2- 172.21.15.133:0/4268288532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7fc0fff30 0x7fb7fc100300 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:02.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.224+0000 7fb8028c1700 1 --2- 172.21.15.133:0/4268288532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7fc1008d0 0x7fb7fc1048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:02.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.224+0000 7fb8028c1700 1 -- 172.21.15.133:0/4268288532 >> 172.21.15.133:0/4268288532 conn(0x7fb7fc0fb790 msgr2=0x7fb7fc0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:02.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.224+0000 7fb8028c1700 1 -- 172.21.15.133:0/4268288532 shutdown_connections 2023-07-14T18:15:02.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.224+0000 7fb8028c1700 1 -- 172.21.15.133:0/4268288532 wait complete. 2023-07-14T18:15:02.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.225+0000 7fb8028c1700 1 Processor -- start 2023-07-14T18:15:02.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.226+0000 7fb8028c1700 1 -- start start 2023-07-14T18:15:02.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.226+0000 7fb8028c1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7fc0fff30 0x7fb7fc195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:02.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.226+0000 7fb8028c1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7fc1008d0 0x7fb7fc1962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:02.272 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.226+0000 7fb8028c1700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7fc104fd0 0x7fb7fc19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:02.272 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.226+0000 7fb8028c1700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc19ad60 con 0x7fb7fc1008d0 2023-07-14T18:15:02.272 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.226+0000 7fb8028c1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc19aed0 con 0x7fb7fc0fff30 2023-07-14T18:15:02.272 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.226+0000 7fb8028c1700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc19b090 con 0x7fb7fc104fd0 2023-07-14T18:15:02.272 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.226+0000 7fb7fb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7fc1008d0 0x7fb7fc1962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:02.272 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.226+0000 7fb7fb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7fc1008d0 0x7fb7fc1962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50048/0 (socket says 172.21.15.133:50048) 2023-07-14T18:15:02.272 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.226+0000 7fb7fb7fe700 1 -- 172.21.15.133:0/2526320007 learned_addr learned my addr 172.21.15.133:0/2526320007 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:02.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.226+0000 7fb7fbfff700 1 --2- 172.21.15.133:0/2526320007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7fc0fff30 0x7fb7fc195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:02.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.226+0000 7fb800e5e700 1 --2- 172.21.15.133:0/2526320007 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7fc104fd0 0x7fb7fc19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:02.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.227+0000 7fb7fb7fe700 1 -- 172.21.15.133:0/2526320007 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7fc104fd0 msgr2=0x7fb7fc19a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:02.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.227+0000 7fb7fb7fe700 1 --2- 172.21.15.133:0/2526320007 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7fc104fd0 0x7fb7fc19a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:02.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.227+0000 7fb7fb7fe700 1 -- 172.21.15.133:0/2526320007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7fc0fff30 msgr2=0x7fb7fc195da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:02.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.227+0000 7fb7fb7fe700 1 --2- 172.21.15.133:0/2526320007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7fc0fff30 0x7fb7fc195da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:02.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.227+0000 7fb7fb7fe700 1 -- 172.21.15.133:0/2526320007 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f0009520 con 0x7fb7fc1008d0 2023-07-14T18:15:02.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.227+0000 7fb7fb7fe700 1 --2- 172.21.15.133:0/2526320007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7fc1008d0 0x7fb7fc1962e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000ea30 tx=0x7fb7f000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:02.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.227+0000 7fb7f97fa700 1 -- 172.21.15.133:0/2526320007 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f000cbd0 con 0x7fb7fc1008d0 2023-07-14T18:15:02.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.227+0000 7fb8028c1700 1 -- 172.21.15.133:0/2526320007 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7ec0095f0 con 0x7fb7fc1008d0 2023-07-14T18:15:02.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.227+0000 7fb7f97fa700 1 -- 172.21.15.133:0/2526320007 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7f000eed0 con 0x7fb7fc1008d0 2023-07-14T18:15:02.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.227+0000 7fb7f97fa700 1 -- 172.21.15.133:0/2526320007 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0003d20 con 0x7fb7fc1008d0 2023-07-14T18:15:02.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.227+0000 7fb8028c1700 1 -- 172.21.15.133:0/2526320007 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7fc1a2a00 con 0x7fb7fc1008d0 2023-07-14T18:15:02.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.228+0000 7fb8028c1700 1 -- 172.21.15.133:0/2526320007 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7fc04e340 con 0x7fb7fc1008d0 2023-07-14T18:15:02.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.230+0000 7fb7f97fa700 1 -- 172.21.15.133:0/2526320007 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fb7f0003ea0 con 0x7fb7fc1008d0 2023-07-14T18:15:02.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.232+0000 7fb7f97fa700 1 --2- 172.21.15.133:0/2526320007 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb7e40a11e0 0x7fb7e40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:02.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.232+0000 7fb7f97fa700 1 -- 172.21.15.133:0/2526320007 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb7f0014070 con 0x7fb7fc1008d0 2023-07-14T18:15:02.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.232+0000 7fb7fbfff700 1 --2- 172.21.15.133:0/2526320007 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb7e40a11e0 0x7fb7e40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:02.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.233+0000 7fb7fbfff700 1 --2- 172.21.15.133:0/2526320007 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb7e40a11e0 0x7fb7e40a3690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec009990 tx=0x7fb7ec005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:02.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.234+0000 7fb7f97fa700 1 -- 172.21.15.133:0/2526320007 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fb7f00964c0 con 0x7fb7fc1008d0 2023-07-14T18:15:02.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.430+0000 7fb8028c1700 1 -- 172.21.15.133:0/2526320007 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7fc197100 con 0x7fb7fc1008d0 2023-07-14T18:15:02.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.432+0000 7fb7f97fa700 1 -- 172.21.15.133:0/2526320007 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24473 (secure 0 0 0) 0x7fb7f001f340 con 0x7fb7fc1008d0 2023-07-14T18:15:02.476 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:15:02.476 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":90,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","created":"2023-07-14T18:06:49.561582+0000","modified":"2023-07-14T18:14:48.934538+0000","last_up_change":"2023-07-14T18:13:29.836099+0000","last_in_change":"2023-07-14T18:13:11.606710+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T18:10:03.945039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T18:14:15.451340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T18:14:16.892215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T18:14:20.493175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"85a29de7-d712-422a-b647-d9e90091aaf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6803","nonce":3076510801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6805","nonce":3076510801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6809","nonce":3076510801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6807","nonce":3076510801}]},"public_addr":"172.21.15.133:6803/3076510801","cluster_addr":"172.21.15.133:6805/3076510801","heartbeat_back_addr":"172.21.15.133:6809/3076510801","heartbeat_front_addr":"172.21.15.133:6807/3076510801","state":["exists","up"]},{"osd":1,"uuid":"80ad4e78-7917-495e-b119-99f7c7c36243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6801","nonce":1841901852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6803","nonce":1841901852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6807","nonce":1841901852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6805","nonce":1841901852}]},"public_addr":"172.21.15.161:6801/1841901852","cluster_addr":"172.21.15.161:6803/1841901852","heartbeat_back_addr":"172.21.15.161:6807/1841901852","heartbeat_front_addr":"172.21.15.161:6805/1841901852","state":["exists","up"]},{"osd":2,"uuid":"fc4bfd37-1fdd-47c6-a48e-6d910daf9e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6801","nonce":346103099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6803","nonce":346103099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6807","nonce":346103099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6805","nonce":346103099}]},"public_addr":"172.21.15.171:6801/346103099","cluster_addr":"172.21.15.171:6803/346103099","heartbeat_back_addr":"172.21.15.171:6807/346103099","heartbeat_front_addr":"172.21.15.171:6805/346103099","state":["exists","up"]},{"osd":3,"uuid":"4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6811","nonce":2152994143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6813","nonce":2152994143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6817","nonce":2152994143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6815","nonce":2152994143}]},"public_addr":"172.21.15.133:6811/2152994143","cluster_addr":"172.21.15.133:6813/2152994143","heartbeat_back_addr":"172.21.15.133:6817/2152994143","heartbeat_front_addr":"172.21.15.133:6815/2152994143","state":["exists","up"]},{"osd":4,"uuid":"4601a1c2-254b-4c4d-9b15-8fc952b3d2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6809","nonce":47213291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6811","nonce":47213291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6815","nonce":47213291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6813","nonce":47213291}]},"public_addr":"172.21.15.161:6809/47213291","cluster_addr":"172.21.15.161:6811/47213291","heartbeat_back_addr":"172.21.15.161:6815/47213291","heartbeat_front_addr":"172.21.15.161:6813/47213291","state":["exists","up"]},{"osd":5,"uuid":"6cca180a-9140-4e49-835b-86842dae0574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6809","nonce":777556751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6811","nonce":777556751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6815","nonce":777556751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6813","nonce":777556751}]},"public_addr":"172.21.15.171:6809/777556751","cluster_addr":"172.21.15.171:6811/777556751","heartbeat_back_addr":"172.21.15.171:6815/777556751","heartbeat_front_addr":"172.21.15.171:6813/777556751","state":["exists","up"]},{"osd":6,"uuid":"ea0299ff-7740-4494-a3a9-1503b3fc0ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6819","nonce":2884152358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6821","nonce":2884152358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6825","nonce":2884152358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6823","nonce":2884152358}]},"public_addr":"172.21.15.133:6819/2884152358","cluster_addr":"172.21.15.133:6821/2884152358","heartbeat_back_addr":"172.21.15.133:6825/2884152358","heartbeat_front_addr":"172.21.15.133:6823/2884152358","state":["exists","up"]},{"osd":7,"uuid":"b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6817","nonce":4183705749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6819","nonce":4183705749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6823","nonce":4183705749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6821","nonce":4183705749}]},"public_addr":"172.21.15.161:6817/4183705749","cluster_addr":"172.21.15.161:6819/4183705749","heartbeat_back_addr":"172.21.15.161:6823/4183705749","heartbeat_front_addr":"172.21.15.161:6821/4183705749","state":["exists","up"]},{"osd":8,"uuid":"87dffc07-bf48-48c8-b418-5c4b509686d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6817","nonce":2771494483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6819","nonce":2771494483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6823","nonce":2771494483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6821","nonce":2771494483}]},"public_addr":"172.21.15.171:6817/2771494483","cluster_addr":"172.21.15.171:6819/2771494483","heartbeat_back_addr":"172.21.15.171:6823/2771494483","heartbeat_front_addr":"172.21.15.171:6821/2771494483","state":["exists","up"]},{"osd":9,"uuid":"e0832870-79ce-4bce-8045-58b7a7f6dc69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6827","nonce":2066164149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6829","nonce":2066164149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6833","nonce":2066164149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6831","nonce":2066164149}]},"public_addr":"172.21.15.133:6827/2066164149","cluster_addr":"172.21.15.133:6829/2066164149","heartbeat_back_addr":"172.21.15.133:6833/2066164149","heartbeat_front_addr":"172.21.15.133:6831/2066164149","state":["exists","up"]},{"osd":10,"uuid":"081f0484-d134-4b2e-9c48-ef8e6a646422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6825","nonce":2162333626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6827","nonce":2162333626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6831","nonce":2162333626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6829","nonce":2162333626}]},"public_addr":"172.21.15.161:6825/2162333626","cluster_addr":"172.21.15.161:6827/2162333626","heartbeat_back_addr":"172.21.15.161:6831/2162333626","heartbeat_front_addr":"172.21.15.161:6829/2162333626","state":["exists","up"]},{"osd":11,"uuid":"4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6825","nonce":3403894313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6827","nonce":3403894313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6831","nonce":3403894313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6829","nonce":3403894313}]},"public_addr":"172.21.15.171:6825/3403894313","cluster_addr":"172.21.15.171:6827/3403894313","heartbeat_back_addr":"172.21.15.171:6831/3403894313","heartbeat_front_addr":"172.21.15.171:6829/3403894313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:17.816229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:39.638543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:00.543317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:26.988921+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:48.644694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:10.049860+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:35.986508+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:57.753846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:19.018245+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:44.460450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:06.279960+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:27.955006+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/1832569241":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6801/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:0/4219815304":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:6801/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/432253615":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1840700480":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1807775439":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1089030734":"2023-07-15T18:07:47.346704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T18:15:02.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.434+0000 7fb8028c1700 1 -- 172.21.15.133:0/2526320007 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb7e40a11e0 msgr2=0x7fb7e40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:02.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.434+0000 7fb8028c1700 1 --2- 172.21.15.133:0/2526320007 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb7e40a11e0 0x7fb7e40a3690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec009990 tx=0x7fb7ec005fb0 comp rx=0 tx=0).stop 2023-07-14T18:15:02.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.434+0000 7fb8028c1700 1 -- 172.21.15.133:0/2526320007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7fc1008d0 msgr2=0x7fb7fc1962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:02.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.434+0000 7fb8028c1700 1 --2- 172.21.15.133:0/2526320007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7fc1008d0 0x7fb7fc1962e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000ea30 tx=0x7fb7f000ed40 comp rx=0 tx=0).stop 2023-07-14T18:15:02.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.434+0000 7fb8028c1700 1 -- 172.21.15.133:0/2526320007 shutdown_connections 2023-07-14T18:15:02.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.434+0000 7fb8028c1700 1 --2- 172.21.15.133:0/2526320007 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb7e40a11e0 0x7fb7e40a3690 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:02.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.434+0000 7fb8028c1700 1 --2- 172.21.15.133:0/2526320007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7fc0fff30 0x7fb7fc195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:02.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.434+0000 7fb8028c1700 1 --2- 172.21.15.133:0/2526320007 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7fc104fd0 0x7fb7fc19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:02.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.434+0000 7fb8028c1700 1 --2- 172.21.15.133:0/2526320007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7fc1008d0 0x7fb7fc1962e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:02.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.434+0000 7fb8028c1700 1 -- 172.21.15.133:0/2526320007 >> 172.21.15.133:0/2526320007 conn(0x7fb7fc0fb790 msgr2=0x7fb7fc0fc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:02.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.434+0000 7fb8028c1700 1 -- 172.21.15.133:0/2526320007 shutdown_connections 2023-07-14T18:15:02.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:02.434+0000 7fb8028c1700 1 -- 172.21.15.133:0/2526320007 wait complete. 2023-07-14T18:15:02.971 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-07-14T18:15:02.971 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph osd dump --format=json 2023-07-14T18:15:03.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:02 smithi161 ceph-mon[128993]: pgmap v297: 129 pgs: 129 active+clean; 627 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.5 KiB/s wr, 14 op/s; 48 B/s, 0 objects/s recovering 2023-07-14T18:15:03.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:02 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2526320007' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:15:03.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:02 smithi171 ceph-mon[129241]: pgmap v297: 129 pgs: 129 active+clean; 627 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.5 KiB/s wr, 14 op/s; 48 B/s, 0 objects/s recovering 2023-07-14T18:15:03.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:02 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2526320007' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:15:03.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:02 smithi133 ceph-mon[126384]: pgmap v297: 129 pgs: 129 active+clean; 627 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.5 KiB/s wr, 14 op/s; 48 B/s, 0 objects/s recovering 2023-07-14T18:15:03.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:02 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2526320007' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:15:03.693 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:05.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.217+0000 7f7ac2df7700 1 -- 172.21.15.133:0/218610161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7abc0fff30 msgr2=0x7f7abc100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:05.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.217+0000 7f7ac2df7700 1 --2- 172.21.15.133:0/218610161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7abc0fff30 0x7f7abc100300 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7ab00099c0 tx=0x7f7ab0009cd0 comp rx=0 tx=0).stop 2023-07-14T18:15:05.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.217+0000 7f7ac2df7700 1 -- 172.21.15.133:0/218610161 shutdown_connections 2023-07-14T18:15:05.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.217+0000 7f7ac2df7700 1 --2- 172.21.15.133:0/218610161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7abc104fd0 0x7f7abc10cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:05.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.217+0000 7f7ac2df7700 1 --2- 172.21.15.133:0/218610161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7abc0fff30 0x7f7abc100300 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:05.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.217+0000 7f7ac2df7700 1 --2- 172.21.15.133:0/218610161 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7abc1008d0 0x7f7abc1048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:05.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.217+0000 7f7ac2df7700 1 -- 172.21.15.133:0/218610161 >> 172.21.15.133:0/218610161 conn(0x7f7abc0fb790 msgr2=0x7f7abc0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:05.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.218+0000 7f7ac2df7700 1 -- 172.21.15.133:0/218610161 shutdown_connections 2023-07-14T18:15:05.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.218+0000 7f7ac2df7700 1 -- 172.21.15.133:0/218610161 wait complete. 2023-07-14T18:15:05.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.219+0000 7f7ac2df7700 1 Processor -- start 2023-07-14T18:15:05.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.219+0000 7f7ac2df7700 1 -- start start 2023-07-14T18:15:05.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.219+0000 7f7ac2df7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7abc0fff30 0x7f7abc195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:05.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.219+0000 7f7ac2df7700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7abc1008d0 0x7f7abc1962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:05.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.219+0000 7f7ac2df7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7abc104fd0 0x7f7abc19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:05.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.219+0000 7f7ac2df7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc19ad60 con 0x7f7abc0fff30 2023-07-14T18:15:05.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.219+0000 7f7ac2df7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc19aed0 con 0x7f7abc104fd0 2023-07-14T18:15:05.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.219+0000 7f7ac2df7700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc19b090 con 0x7f7abc1008d0 2023-07-14T18:15:05.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.220+0000 7f7abbfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7abc1008d0 0x7f7abc1962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:05.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.220+0000 7f7abbfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7abc1008d0 0x7f7abc1962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:60672/0 (socket says 172.21.15.133:60672) 2023-07-14T18:15:05.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.220+0000 7f7ac0b93700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7abc0fff30 0x7f7abc195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:05.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.220+0000 7f7abbfff700 1 -- 172.21.15.133:0/3396159203 learned_addr learned my addr 172.21.15.133:0/3396159203 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:05.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.220+0000 7f7ac1394700 1 --2- 172.21.15.133:0/3396159203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7abc104fd0 0x7f7abc19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:05.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.220+0000 7f7abbfff700 1 -- 172.21.15.133:0/3396159203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7abc104fd0 msgr2=0x7f7abc19a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:05.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.220+0000 7f7abbfff700 1 --2- 172.21.15.133:0/3396159203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7abc104fd0 0x7f7abc19a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:05.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.220+0000 7f7abbfff700 1 -- 172.21.15.133:0/3396159203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7abc0fff30 msgr2=0x7f7abc195da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:05.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.220+0000 7f7abbfff700 1 --2- 172.21.15.133:0/3396159203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7abc0fff30 0x7f7abc195da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:05.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.220+0000 7f7abbfff700 1 -- 172.21.15.133:0/3396159203 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab4009c40 con 0x7f7abc1008d0 2023-07-14T18:15:05.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.220+0000 7f7abbfff700 1 --2- 172.21.15.133:0/3396159203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7abc1008d0 0x7f7abc1962e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800d980 tx=0x7f7aa800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:05.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.221+0000 7f7ab9ffb700 1 -- 172.21.15.133:0/3396159203 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8009930 con 0x7f7abc1008d0 2023-07-14T18:15:05.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.221+0000 7f7ac2df7700 1 -- 172.21.15.133:0/3396159203 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab00095f0 con 0x7f7abc1008d0 2023-07-14T18:15:05.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.221+0000 7f7ab9ffb700 1 -- 172.21.15.133:0/3396159203 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7aa800b640 con 0x7f7abc1008d0 2023-07-14T18:15:05.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.221+0000 7f7ab9ffb700 1 -- 172.21.15.133:0/3396159203 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa800f3c0 con 0x7f7abc1008d0 2023-07-14T18:15:05.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.221+0000 7f7ac2df7700 1 -- 172.21.15.133:0/3396159203 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7abc1a2a00 con 0x7f7abc1008d0 2023-07-14T18:15:05.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.222+0000 7f7ac2df7700 1 -- 172.21.15.133:0/3396159203 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7abc04e340 con 0x7f7abc1008d0 2023-07-14T18:15:05.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.224+0000 7f7ab9ffb700 1 -- 172.21.15.133:0/3396159203 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f7aa8010680 con 0x7f7abc1008d0 2023-07-14T18:15:05.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.225+0000 7f7ab9ffb700 1 --2- 172.21.15.133:0/3396159203 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f7aac0a11e0 0x7f7aac0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:05.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.225+0000 7f7ab9ffb700 1 -- 172.21.15.133:0/3396159203 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7aa80cc830 con 0x7f7abc1008d0 2023-07-14T18:15:05.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.225+0000 7f7ac0b93700 1 --2- 172.21.15.133:0/3396159203 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f7aac0a11e0 0x7f7aac0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:05.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.226+0000 7f7ac0b93700 1 --2- 172.21.15.133:0/3396159203 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f7aac0a11e0 0x7f7aac0a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0009fd0 tx=0x7f7ab0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:05.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.227+0000 7f7ab9ffb700 1 -- 172.21.15.133:0/3396159203 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f7aa8095ab0 con 0x7f7abc1008d0 2023-07-14T18:15:05.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:04 smithi161 ceph-mon[128993]: pgmap v298: 129 pgs: 129 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.6 KiB/s wr, 14 op/s; 48 B/s, 0 objects/s recovering 2023-07-14T18:15:05.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:04 smithi171 ceph-mon[129241]: pgmap v298: 129 pgs: 129 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.6 KiB/s wr, 14 op/s; 48 B/s, 0 objects/s recovering 2023-07-14T18:15:05.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:04 smithi133 ceph-mon[126384]: pgmap v298: 129 pgs: 129 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.6 KiB/s wr, 14 op/s; 48 B/s, 0 objects/s recovering 2023-07-14T18:15:05.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.426+0000 7f7ac2df7700 1 -- 172.21.15.133:0/3396159203 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7abc197100 con 0x7f7abc1008d0 2023-07-14T18:15:05.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.428+0000 7f7ab9ffb700 1 -- 172.21.15.133:0/3396159203 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24473 (secure 0 0 0) 0x7f7aa8020090 con 0x7f7abc1008d0 2023-07-14T18:15:05.472 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:15:05.473 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":90,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","created":"2023-07-14T18:06:49.561582+0000","modified":"2023-07-14T18:14:48.934538+0000","last_up_change":"2023-07-14T18:13:29.836099+0000","last_in_change":"2023-07-14T18:13:11.606710+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T18:10:03.945039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T18:14:15.451340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T18:14:16.892215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T18:14:20.493175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"85a29de7-d712-422a-b647-d9e90091aaf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6803","nonce":3076510801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6805","nonce":3076510801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6809","nonce":3076510801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6807","nonce":3076510801}]},"public_addr":"172.21.15.133:6803/3076510801","cluster_addr":"172.21.15.133:6805/3076510801","heartbeat_back_addr":"172.21.15.133:6809/3076510801","heartbeat_front_addr":"172.21.15.133:6807/3076510801","state":["exists","up"]},{"osd":1,"uuid":"80ad4e78-7917-495e-b119-99f7c7c36243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6801","nonce":1841901852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6803","nonce":1841901852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6807","nonce":1841901852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6805","nonce":1841901852}]},"public_addr":"172.21.15.161:6801/1841901852","cluster_addr":"172.21.15.161:6803/1841901852","heartbeat_back_addr":"172.21.15.161:6807/1841901852","heartbeat_front_addr":"172.21.15.161:6805/1841901852","state":["exists","up"]},{"osd":2,"uuid":"fc4bfd37-1fdd-47c6-a48e-6d910daf9e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6801","nonce":346103099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6803","nonce":346103099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6807","nonce":346103099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6805","nonce":346103099}]},"public_addr":"172.21.15.171:6801/346103099","cluster_addr":"172.21.15.171:6803/346103099","heartbeat_back_addr":"172.21.15.171:6807/346103099","heartbeat_front_addr":"172.21.15.171:6805/346103099","state":["exists","up"]},{"osd":3,"uuid":"4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6811","nonce":2152994143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6813","nonce":2152994143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6817","nonce":2152994143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6815","nonce":2152994143}]},"public_addr":"172.21.15.133:6811/2152994143","cluster_addr":"172.21.15.133:6813/2152994143","heartbeat_back_addr":"172.21.15.133:6817/2152994143","heartbeat_front_addr":"172.21.15.133:6815/2152994143","state":["exists","up"]},{"osd":4,"uuid":"4601a1c2-254b-4c4d-9b15-8fc952b3d2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6809","nonce":47213291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6811","nonce":47213291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6815","nonce":47213291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6813","nonce":47213291}]},"public_addr":"172.21.15.161:6809/47213291","cluster_addr":"172.21.15.161:6811/47213291","heartbeat_back_addr":"172.21.15.161:6815/47213291","heartbeat_front_addr":"172.21.15.161:6813/47213291","state":["exists","up"]},{"osd":5,"uuid":"6cca180a-9140-4e49-835b-86842dae0574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6809","nonce":777556751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6811","nonce":777556751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6815","nonce":777556751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6813","nonce":777556751}]},"public_addr":"172.21.15.171:6809/777556751","cluster_addr":"172.21.15.171:6811/777556751","heartbeat_back_addr":"172.21.15.171:6815/777556751","heartbeat_front_addr":"172.21.15.171:6813/777556751","state":["exists","up"]},{"osd":6,"uuid":"ea0299ff-7740-4494-a3a9-1503b3fc0ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6819","nonce":2884152358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6821","nonce":2884152358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6825","nonce":2884152358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6823","nonce":2884152358}]},"public_addr":"172.21.15.133:6819/2884152358","cluster_addr":"172.21.15.133:6821/2884152358","heartbeat_back_addr":"172.21.15.133:6825/2884152358","heartbeat_front_addr":"172.21.15.133:6823/2884152358","state":["exists","up"]},{"osd":7,"uuid":"b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6817","nonce":4183705749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6819","nonce":4183705749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6823","nonce":4183705749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6821","nonce":4183705749}]},"public_addr":"172.21.15.161:6817/4183705749","cluster_addr":"172.21.15.161:6819/4183705749","heartbeat_back_addr":"172.21.15.161:6823/4183705749","heartbeat_front_addr":"172.21.15.161:6821/4183705749","state":["exists","up"]},{"osd":8,"uuid":"87dffc07-bf48-48c8-b418-5c4b509686d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6817","nonce":2771494483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6819","nonce":2771494483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6823","nonce":2771494483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6821","nonce":2771494483}]},"public_addr":"172.21.15.171:6817/2771494483","cluster_addr":"172.21.15.171:6819/2771494483","heartbeat_back_addr":"172.21.15.171:6823/2771494483","heartbeat_front_addr":"172.21.15.171:6821/2771494483","state":["exists","up"]},{"osd":9,"uuid":"e0832870-79ce-4bce-8045-58b7a7f6dc69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6827","nonce":2066164149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6829","nonce":2066164149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6833","nonce":2066164149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6831","nonce":2066164149}]},"public_addr":"172.21.15.133:6827/2066164149","cluster_addr":"172.21.15.133:6829/2066164149","heartbeat_back_addr":"172.21.15.133:6833/2066164149","heartbeat_front_addr":"172.21.15.133:6831/2066164149","state":["exists","up"]},{"osd":10,"uuid":"081f0484-d134-4b2e-9c48-ef8e6a646422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6825","nonce":2162333626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6827","nonce":2162333626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6831","nonce":2162333626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6829","nonce":2162333626}]},"public_addr":"172.21.15.161:6825/2162333626","cluster_addr":"172.21.15.161:6827/2162333626","heartbeat_back_addr":"172.21.15.161:6831/2162333626","heartbeat_front_addr":"172.21.15.161:6829/2162333626","state":["exists","up"]},{"osd":11,"uuid":"4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6825","nonce":3403894313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6827","nonce":3403894313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6831","nonce":3403894313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6829","nonce":3403894313}]},"public_addr":"172.21.15.171:6825/3403894313","cluster_addr":"172.21.15.171:6827/3403894313","heartbeat_back_addr":"172.21.15.171:6831/3403894313","heartbeat_front_addr":"172.21.15.171:6829/3403894313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:17.816229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:39.638543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:00.543317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:26.988921+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:48.644694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:10.049860+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:35.986508+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:57.753846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:19.018245+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:44.460450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:06.279960+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:27.955006+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/1832569241":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6801/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:0/4219815304":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:6801/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/432253615":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1840700480":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1807775439":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1089030734":"2023-07-15T18:07:47.346704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T18:15:05.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.430+0000 7f7ac2df7700 1 -- 172.21.15.133:0/3396159203 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f7aac0a11e0 msgr2=0x7f7aac0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:05.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.430+0000 7f7ac2df7700 1 --2- 172.21.15.133:0/3396159203 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f7aac0a11e0 0x7f7aac0a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0009fd0 tx=0x7f7ab0005fb0 comp rx=0 tx=0).stop 2023-07-14T18:15:05.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.430+0000 7f7ac2df7700 1 -- 172.21.15.133:0/3396159203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7abc1008d0 msgr2=0x7f7abc1962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:05.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.430+0000 7f7ac2df7700 1 --2- 172.21.15.133:0/3396159203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7abc1008d0 0x7f7abc1962e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800d980 tx=0x7f7aa800dc90 comp rx=0 tx=0).stop 2023-07-14T18:15:05.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.430+0000 7f7ac2df7700 1 -- 172.21.15.133:0/3396159203 shutdown_connections 2023-07-14T18:15:05.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.430+0000 7f7ac2df7700 1 --2- 172.21.15.133:0/3396159203 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f7aac0a11e0 0x7f7aac0a3690 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:05.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.430+0000 7f7ac2df7700 1 --2- 172.21.15.133:0/3396159203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7abc104fd0 0x7f7abc19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:05.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.430+0000 7f7ac2df7700 1 --2- 172.21.15.133:0/3396159203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7abc0fff30 0x7f7abc195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:05.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.430+0000 7f7ac2df7700 1 --2- 172.21.15.133:0/3396159203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7abc1008d0 0x7f7abc1962e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:05.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.430+0000 7f7ac2df7700 1 -- 172.21.15.133:0/3396159203 >> 172.21.15.133:0/3396159203 conn(0x7f7abc0fb790 msgr2=0x7f7abc103880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:05.476 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.430+0000 7f7ac2df7700 1 -- 172.21.15.133:0/3396159203 shutdown_connections 2023-07-14T18:15:05.476 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:05.430+0000 7f7ac2df7700 1 -- 172.21.15.133:0/3396159203 wait complete. 2023-07-14T18:15:05.983 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph tell osd.0 flush_pg_stats 2023-07-14T18:15:05.984 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph tell osd.1 flush_pg_stats 2023-07-14T18:15:05.985 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph tell osd.2 flush_pg_stats 2023-07-14T18:15:05.986 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph tell osd.3 flush_pg_stats 2023-07-14T18:15:05.986 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph tell osd.4 flush_pg_stats 2023-07-14T18:15:05.987 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph tell osd.5 flush_pg_stats 2023-07-14T18:15:05.987 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph tell osd.6 flush_pg_stats 2023-07-14T18:15:05.988 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph tell osd.7 flush_pg_stats 2023-07-14T18:15:05.989 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph tell osd.8 flush_pg_stats 2023-07-14T18:15:05.989 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph tell osd.9 flush_pg_stats 2023-07-14T18:15:05.990 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph tell osd.10 flush_pg_stats 2023-07-14T18:15:05.990 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph tell osd.11 flush_pg_stats 2023-07-14T18:15:06.263 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:06 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3396159203' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:15:06.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:06 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3396159203' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:15:06.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:06 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3396159203' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:15:07.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:07 smithi161 ceph-mon[128993]: pgmap v299: 129 pgs: 129 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2023-07-14T18:15:07.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:07 smithi171 ceph-mon[129241]: pgmap v299: 129 pgs: 129 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2023-07-14T18:15:07.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:07 smithi133 ceph-mon[126384]: pgmap v299: 129 pgs: 129 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2023-07-14T18:15:08.056 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:08.118 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:08.133 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:08.150 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:08.284 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:08.285 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:08.304 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:08.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:08 smithi161 ceph-mon[128993]: pgmap v300: 129 pgs: 129 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2023-07-14T18:15:08.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:08 smithi171 ceph-mon[129241]: pgmap v300: 129 pgs: 129 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2023-07-14T18:15:08.372 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:08 smithi133 ceph-mon[126384]: pgmap v300: 129 pgs: 129 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2023-07-14T18:15:08.457 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:08.861 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:08.937 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:09.047 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:09.109 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:11.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:10 smithi133 ceph-mon[126384]: pgmap v301: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2023-07-14T18:15:11.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:10 smithi161 ceph-mon[128993]: pgmap v301: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2023-07-14T18:15:11.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:10 smithi171 ceph-mon[129241]: pgmap v301: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2023-07-14T18:15:12.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.816+0000 7f62f5336700 1 -- 172.21.15.133:0/51575678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62f00fff30 msgr2=0x7f62f0100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:12.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.816+0000 7f62f5336700 1 --2- 172.21.15.133:0/51575678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62f00fff30 0x7f62f0100300 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f62e00099c0 tx=0x7f62e0009cd0 comp rx=0 tx=0).stop 2023-07-14T18:15:12.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.816+0000 7f62f5336700 1 -- 172.21.15.133:0/51575678 shutdown_connections 2023-07-14T18:15:12.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.816+0000 7f62f5336700 1 --2- 172.21.15.133:0/51575678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62f0104fd0 0x7f62f010cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:12.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.816+0000 7f62f5336700 1 --2- 172.21.15.133:0/51575678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62f00fff30 0x7f62f0100300 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:12.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.816+0000 7f62f5336700 1 --2- 172.21.15.133:0/51575678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62f01008d0 0x7f62f01048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:12.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.816+0000 7f62f5336700 1 -- 172.21.15.133:0/51575678 >> 172.21.15.133:0/51575678 conn(0x7f62f00fb790 msgr2=0x7f62f00fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:12.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.816+0000 7f62f5336700 1 -- 172.21.15.133:0/51575678 shutdown_connections 2023-07-14T18:15:12.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.816+0000 7f62f5336700 1 -- 172.21.15.133:0/51575678 wait complete. 2023-07-14T18:15:12.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.817+0000 7f62f5336700 1 Processor -- start 2023-07-14T18:15:12.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.817+0000 7f62f5336700 1 -- start start 2023-07-14T18:15:12.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.818+0000 7f62f5336700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62f00fff30 0x7f62f0197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:12.863 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.818+0000 7f62f5336700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62f01008d0 0x7f62f0198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:12.863 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.818+0000 7f62f5336700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62f0104fd0 0x7f62f019c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:12.863 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.818+0000 7f62f5336700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f019cfa0 con 0x7f62f0104fd0 2023-07-14T18:15:12.863 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.818+0000 7f62f5336700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f019d110 con 0x7f62f00fff30 2023-07-14T18:15:12.863 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.818+0000 7f62f5336700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f019d2d0 con 0x7f62f01008d0 2023-07-14T18:15:12.863 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.818+0000 7f62e7fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62f01008d0 0x7f62f0198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:12.863 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.818+0000 7f62ef7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62f0104fd0 0x7f62f019c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:12.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.818+0000 7f62e7fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62f01008d0 0x7f62f0198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:33644/0 (socket says 172.21.15.133:33644) 2023-07-14T18:15:12.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.818+0000 7f62e7fff700 1 -- 172.21.15.133:0/3349527972 learned_addr learned my addr 172.21.15.133:0/3349527972 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:12.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.818+0000 7f62eeffd700 1 --2- 172.21.15.133:0/3349527972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62f00fff30 0x7f62f0197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:12.865 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.819+0000 7f62e7fff700 1 -- 172.21.15.133:0/3349527972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62f00fff30 msgr2=0x7f62f0197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:12.865 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.819+0000 7f62e7fff700 1 --2- 172.21.15.133:0/3349527972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62f00fff30 0x7f62f0197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:12.865 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.819+0000 7f62e7fff700 1 -- 172.21.15.133:0/3349527972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62f0104fd0 msgr2=0x7f62f019c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:12.865 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.819+0000 7f62e7fff700 1 --2- 172.21.15.133:0/3349527972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62f0104fd0 0x7f62f019c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:12.865 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.819+0000 7f62e7fff700 1 -- 172.21.15.133:0/3349527972 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e00095f0 con 0x7f62f01008d0 2023-07-14T18:15:12.865 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.819+0000 7f62e7fff700 1 --2- 172.21.15.133:0/3349527972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62f01008d0 0x7f62f0198520 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f62d800e970 tx=0x7f62d800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:12.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.819+0000 7f62ecff9700 1 -- 172.21.15.133:0/3349527972 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d8009790 con 0x7f62f01008d0 2023-07-14T18:15:12.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.819+0000 7f62f5336700 1 -- 172.21.15.133:0/3349527972 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62f019d5b0 con 0x7f62f01008d0 2023-07-14T18:15:12.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.819+0000 7f62ecff9700 1 -- 172.21.15.133:0/3349527972 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62d800c7e0 con 0x7f62f01008d0 2023-07-14T18:15:12.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.819+0000 7f62ecff9700 1 -- 172.21.15.133:0/3349527972 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d80054e0 con 0x7f62f01008d0 2023-07-14T18:15:12.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.819+0000 7f62f5336700 1 -- 172.21.15.133:0/3349527972 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f01a4e10 con 0x7f62f01008d0 2023-07-14T18:15:12.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.820+0000 7f62f5336700 1 -- 172.21.15.133:0/3349527972 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f62f010fdd0 con 0x7f62f01008d0 2023-07-14T18:15:12.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.823+0000 7f62ecff9700 1 -- 172.21.15.133:0/3349527972 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f62d800c950 con 0x7f62f01008d0 2023-07-14T18:15:12.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.825+0000 7f62ecff9700 1 --2- 172.21.15.133:0/3349527972 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f62dc0a11e0 0x7f62dc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:12.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.825+0000 7f62ecff9700 1 -- 172.21.15.133:0/3349527972 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f62d8014070 con 0x7f62f01008d0 2023-07-14T18:15:12.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.825+0000 7f62eeffd700 1 --2- 172.21.15.133:0/3349527972 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f62dc0a11e0 0x7f62dc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:12.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.825+0000 7f62ecff9700 1 --2- 172.21.15.133:0/3349527972 >> [v2:172.21.15.133:6802/3076510801,v1:172.21.15.133:6803/3076510801] conn(0x7f62dc0ac1f0 0x7f62dc0ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:12.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.825+0000 7f62ecff9700 1 -- 172.21.15.133:0/3349527972 --> [v2:172.21.15.133:6802/3076510801,v1:172.21.15.133:6803/3076510801] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f62dc0aed50 con 0x7f62dc0ac1f0 2023-07-14T18:15:12.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.825+0000 7f62ecff9700 1 -- 172.21.15.133:0/3349527972 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62d80cd5f0 con 0x7f62f01008d0 2023-07-14T18:15:12.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.826+0000 7f62ef7fe700 1 --2- 172.21.15.133:0/3349527972 >> [v2:172.21.15.133:6802/3076510801,v1:172.21.15.133:6803/3076510801] conn(0x7f62dc0ac1f0 0x7f62dc0ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:12.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.826+0000 7f62eeffd700 1 --2- 172.21.15.133:0/3349527972 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f62dc0a11e0 0x7f62dc0a3690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f62e0009990 tx=0x7f62e0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:12.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.827+0000 7f62ef7fe700 1 --2- 172.21.15.133:0/3349527972 >> [v2:172.21.15.133:6802/3076510801,v1:172.21.15.133:6803/3076510801] conn(0x7f62dc0ac1f0 0x7f62dc0ae680 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:12.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.827+0000 7f62ecff9700 1 -- 172.21.15.133:0/3349527972 <== osd.0 v2:172.21.15.133:6802/3076510801 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f62dc0aed50 con 0x7f62dc0ac1f0 2023-07-14T18:15:12.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.844+0000 7f62f5336700 1 -- 172.21.15.133:0/3349527972 --> [v2:172.21.15.133:6802/3076510801,v1:172.21.15.133:6803/3076510801] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f62f004e340 con 0x7f62dc0ac1f0 2023-07-14T18:15:12.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.844+0000 7f62ecff9700 1 -- 172.21.15.133:0/3349527972 <== osd.0 v2:172.21.15.133:6802/3076510801 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f62f004e340 con 0x7f62dc0ac1f0 2023-07-14T18:15:12.889 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.844+0000 7f62f5336700 1 -- 172.21.15.133:0/3349527972 >> [v2:172.21.15.133:6802/3076510801,v1:172.21.15.133:6803/3076510801] conn(0x7f62dc0ac1f0 msgr2=0x7f62dc0ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:12.889 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.844+0000 7f62f5336700 1 --2- 172.21.15.133:0/3349527972 >> [v2:172.21.15.133:6802/3076510801,v1:172.21.15.133:6803/3076510801] conn(0x7f62dc0ac1f0 0x7f62dc0ae680 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:12.889 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.844+0000 7f62f5336700 1 -- 172.21.15.133:0/3349527972 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f62dc0a11e0 msgr2=0x7f62dc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:12.889 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.844+0000 7f62f5336700 1 --2- 172.21.15.133:0/3349527972 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f62dc0a11e0 0x7f62dc0a3690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f62e0009990 tx=0x7f62e0005fb0 comp rx=0 tx=0).stop 2023-07-14T18:15:12.889 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.844+0000 7f62f5336700 1 -- 172.21.15.133:0/3349527972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62f01008d0 msgr2=0x7f62f0198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:12.890 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.844+0000 7f62f5336700 1 --2- 172.21.15.133:0/3349527972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62f01008d0 0x7f62f0198520 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f62d800e970 tx=0x7f62d800ec80 comp rx=0 tx=0).stop 2023-07-14T18:15:12.890 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.844+0000 7f62ef7fe700 1 -- 172.21.15.133:0/3349527972 reap_dead start 2023-07-14T18:15:12.890 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.845+0000 7f62f5336700 1 -- 172.21.15.133:0/3349527972 shutdown_connections 2023-07-14T18:15:12.890 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.845+0000 7f62f5336700 1 -- 172.21.15.133:0/3349527972 >> 172.21.15.133:0/3349527972 conn(0x7f62f00fb790 msgr2=0x7f62f00fc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:12.890 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.845+0000 7f62f5336700 1 -- 172.21.15.133:0/3349527972 shutdown_connections 2023-07-14T18:15:12.890 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:12.845+0000 7f62f5336700 1 -- 172.21.15.133:0/3349527972 wait complete. 2023-07-14T18:15:13.192 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:12 smithi133 ceph-mon[126384]: pgmap v302: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2023-07-14T18:15:13.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:12 smithi161 ceph-mon[128993]: pgmap v302: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2023-07-14T18:15:13.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:12 smithi171 ceph-mon[129241]: pgmap v302: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2023-07-14T18:15:14.189 INFO:teuthology.orchestra.run.smithi133.stdout:34359738442 2023-07-14T18:15:14.189 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.0 2023-07-14T18:15:14.238 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.193+0000 7f63fdcd9700 1 -- 172.21.15.133:0/887031376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f8110cc0 msgr2=0x7f63f81130a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:14.238 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.193+0000 7f63fdcd9700 1 --2- 172.21.15.133:0/887031376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f8110cc0 0x7f63f81130a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f63f000b0a0 tx=0x7f63f000b3b0 comp rx=0 tx=0).stop 2023-07-14T18:15:14.238 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.194+0000 7f63fdcd9700 1 -- 172.21.15.133:0/887031376 shutdown_connections 2023-07-14T18:15:14.238 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.194+0000 7f63fdcd9700 1 --2- 172.21.15.133:0/887031376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f8110cc0 0x7f63f81130a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:14.238 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.194+0000 7f63fdcd9700 1 --2- 172.21.15.133:0/887031376 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63f8074b70 0x7f63f8074f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:14.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.194+0000 7f63fdcd9700 1 --2- 172.21.15.133:0/887031376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63f8075480 0x7f63f8110780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:14.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.194+0000 7f63fdcd9700 1 -- 172.21.15.133:0/887031376 >> 172.21.15.133:0/887031376 conn(0x7f63f80fdb40 msgr2=0x7f63f80fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:14.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.194+0000 7f63fdcd9700 1 -- 172.21.15.133:0/887031376 shutdown_connections 2023-07-14T18:15:14.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.194+0000 7f63fdcd9700 1 -- 172.21.15.133:0/887031376 wait complete. 2023-07-14T18:15:14.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.195+0000 7f63fdcd9700 1 Processor -- start 2023-07-14T18:15:14.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.195+0000 7f63fdcd9700 1 -- start start 2023-07-14T18:15:14.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.196+0000 7f63fdcd9700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63f8074b70 0x7f63f819c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:14.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.196+0000 7f63fdcd9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63f8075480 0x7f63f819ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:14.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.196+0000 7f63fdcd9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f8110cc0 0x7f63f81a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:14.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.196+0000 7f63fdcd9700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f81a1490 con 0x7f63f8110cc0 2023-07-14T18:15:14.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.196+0000 7f63fdcd9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f81a1600 con 0x7f63f8075480 2023-07-14T18:15:14.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.196+0000 7f63fdcd9700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f81a17c0 con 0x7f63f8074b70 2023-07-14T18:15:14.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.196+0000 7f63f77fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63f8074b70 0x7f63f819c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:14.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.196+0000 7f63f77fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63f8074b70 0x7f63f819c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:33666/0 (socket says 172.21.15.133:33666) 2023-07-14T18:15:14.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.196+0000 7f63f77fe700 1 -- 172.21.15.133:0/2942796509 learned_addr learned my addr 172.21.15.133:0/2942796509 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:14.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.196+0000 7f63f6ffd700 1 --2- 172.21.15.133:0/2942796509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63f8075480 0x7f63f819ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:14.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.196+0000 7f63f7fff700 1 --2- 172.21.15.133:0/2942796509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f8110cc0 0x7f63f81a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:14.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.197+0000 7f63f77fe700 1 -- 172.21.15.133:0/2942796509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63f8075480 msgr2=0x7f63f819ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:14.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.197+0000 7f63f77fe700 1 --2- 172.21.15.133:0/2942796509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63f8075480 0x7f63f819ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:14.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.197+0000 7f63f77fe700 1 -- 172.21.15.133:0/2942796509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f8110cc0 msgr2=0x7f63f81a0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:14.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.197+0000 7f63f77fe700 1 --2- 172.21.15.133:0/2942796509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f8110cc0 0x7f63f81a0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:14.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.197+0000 7f63f77fe700 1 -- 172.21.15.133:0/2942796509 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f0009d00 con 0x7f63f8074b70 2023-07-14T18:15:14.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.197+0000 7f63f6ffd700 1 --2- 172.21.15.133:0/2942796509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63f8075480 0x7f63f819ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:14.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.197+0000 7f63f7fff700 1 --2- 172.21.15.133:0/2942796509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f8110cc0 0x7f63f81a0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:14.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.197+0000 7f63f77fe700 1 --2- 172.21.15.133:0/2942796509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63f8074b70 0x7f63f819c4d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f63e800e9a0 tx=0x7f63e800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:14.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.197+0000 7f63f4ff9700 1 -- 172.21.15.133:0/2942796509 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8009790 con 0x7f63f8074b70 2023-07-14T18:15:14.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.197+0000 7f63fdcd9700 1 -- 172.21.15.133:0/2942796509 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f81a1aa0 con 0x7f63f8074b70 2023-07-14T18:15:14.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.197+0000 7f63f4ff9700 1 -- 172.21.15.133:0/2942796509 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63e8010430 con 0x7f63f8074b70 2023-07-14T18:15:14.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.197+0000 7f63f4ff9700 1 -- 172.21.15.133:0/2942796509 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8018680 con 0x7f63f8074b70 2023-07-14T18:15:14.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.198+0000 7f63fdcd9700 1 -- 172.21.15.133:0/2942796509 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f81a9300 con 0x7f63f8074b70 2023-07-14T18:15:14.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.199+0000 7f63fdcd9700 1 -- 172.21.15.133:0/2942796509 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f63f8116500 con 0x7f63f8074b70 2023-07-14T18:15:14.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.201+0000 7f63f4ff9700 1 -- 172.21.15.133:0/2942796509 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f63e80105a0 con 0x7f63f8074b70 2023-07-14T18:15:14.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.202+0000 7f63f4ff9700 1 --2- 172.21.15.133:0/2942796509 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f63e00a1260 0x7f63e00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:14.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.202+0000 7f63f4ff9700 1 -- 172.21.15.133:0/2942796509 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f63e8014070 con 0x7f63f8074b70 2023-07-14T18:15:14.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.202+0000 7f63f6ffd700 1 --2- 172.21.15.133:0/2942796509 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f63e00a1260 0x7f63e00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:14.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.203+0000 7f63f4ff9700 1 --2- 172.21.15.133:0/2942796509 >> [v2:172.21.15.171:6800/346103099,v1:172.21.15.171:6801/346103099] conn(0x7f63e00ac270 0x7f63e00ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:14.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.203+0000 7f63f4ff9700 1 -- 172.21.15.133:0/2942796509 --> [v2:172.21.15.171:6800/346103099,v1:172.21.15.171:6801/346103099] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f63e00aedd0 con 0x7f63e00ac270 2023-07-14T18:15:14.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.203+0000 7f63f4ff9700 1 -- 172.21.15.133:0/2942796509 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63e80cd5e0 con 0x7f63f8074b70 2023-07-14T18:15:14.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.203+0000 7f63f7fff700 1 --2- 172.21.15.133:0/2942796509 >> [v2:172.21.15.171:6800/346103099,v1:172.21.15.171:6801/346103099] conn(0x7f63e00ac270 0x7f63e00ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:14.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.203+0000 7f63f6ffd700 1 --2- 172.21.15.133:0/2942796509 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f63e00a1260 0x7f63e00a3710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f63f819daf0 tx=0x7f63ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:14.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.204+0000 7f63f7fff700 1 --2- 172.21.15.133:0/2942796509 >> [v2:172.21.15.171:6800/346103099,v1:172.21.15.171:6801/346103099] conn(0x7f63e00ac270 0x7f63e00ae700 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:14.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.204+0000 7f63f4ff9700 1 -- 172.21.15.133:0/2942796509 <== osd.2 v2:172.21.15.171:6800/346103099 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f63e00aedd0 con 0x7f63e00ac270 2023-07-14T18:15:14.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.222+0000 7f63fdcd9700 1 -- 172.21.15.133:0/2942796509 --> [v2:172.21.15.171:6800/346103099,v1:172.21.15.171:6801/346103099] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f63f804e340 con 0x7f63e00ac270 2023-07-14T18:15:14.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.222+0000 7f63f4ff9700 1 -- 172.21.15.133:0/2942796509 <== osd.2 v2:172.21.15.171:6800/346103099 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f63f804e340 con 0x7f63e00ac270 2023-07-14T18:15:14.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.223+0000 7f63fdcd9700 1 -- 172.21.15.133:0/2942796509 >> [v2:172.21.15.171:6800/346103099,v1:172.21.15.171:6801/346103099] conn(0x7f63e00ac270 msgr2=0x7f63e00ae700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:14.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.223+0000 7f63fdcd9700 1 --2- 172.21.15.133:0/2942796509 >> [v2:172.21.15.171:6800/346103099,v1:172.21.15.171:6801/346103099] conn(0x7f63e00ac270 0x7f63e00ae700 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:14.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.223+0000 7f63fdcd9700 1 -- 172.21.15.133:0/2942796509 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f63e00a1260 msgr2=0x7f63e00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:14.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.223+0000 7f63fdcd9700 1 --2- 172.21.15.133:0/2942796509 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f63e00a1260 0x7f63e00a3710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f63f819daf0 tx=0x7f63ec009260 comp rx=0 tx=0).stop 2023-07-14T18:15:14.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.223+0000 7f63fdcd9700 1 -- 172.21.15.133:0/2942796509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63f8074b70 msgr2=0x7f63f819c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:14.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.223+0000 7f63fdcd9700 1 --2- 172.21.15.133:0/2942796509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63f8074b70 0x7f63f819c4d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f63e800e9a0 tx=0x7f63e800ecb0 comp rx=0 tx=0).stop 2023-07-14T18:15:14.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.223+0000 7f63f7fff700 1 -- 172.21.15.133:0/2942796509 reap_dead start 2023-07-14T18:15:14.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.223+0000 7f63fdcd9700 1 -- 172.21.15.133:0/2942796509 shutdown_connections 2023-07-14T18:15:14.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.223+0000 7f63fdcd9700 1 -- 172.21.15.133:0/2942796509 >> 172.21.15.133:0/2942796509 conn(0x7f63f80fdb40 msgr2=0x7f63f80fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:14.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.223+0000 7f63fdcd9700 1 -- 172.21.15.133:0/2942796509 shutdown_connections 2023-07-14T18:15:14.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:14.223+0000 7f63fdcd9700 1 -- 172.21.15.133:0/2942796509 wait complete. 2023-07-14T18:15:15.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.021+0000 7f47ca10f700 1 -- 172.21.15.133:0/1883374726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47c40ffb30 msgr2=0x7f47c40fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:15.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.021+0000 7f47ca10f700 1 --2- 172.21.15.133:0/1883374726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47c40ffb30 0x7f47c40fffa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009990 tx=0x7f47b4009ca0 comp rx=0 tx=0).stop 2023-07-14T18:15:15.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.021+0000 7f47ca10f700 1 -- 172.21.15.133:0/1883374726 shutdown_connections 2023-07-14T18:15:15.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.021+0000 7f47ca10f700 1 --2- 172.21.15.133:0/1883374726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47c4105b50 0x7f47c4105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:15.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.021+0000 7f47ca10f700 1 --2- 172.21.15.133:0/1883374726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47c41004e0 0x7f47c410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:15.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.021+0000 7f47ca10f700 1 --2- 172.21.15.133:0/1883374726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47c40ffb30 0x7f47c40fffa0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:15.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.021+0000 7f47ca10f700 1 -- 172.21.15.133:0/1883374726 >> 172.21.15.133:0/1883374726 conn(0x7f47c40fb690 msgr2=0x7f47c40fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:15.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.021+0000 7f47ca10f700 1 -- 172.21.15.133:0/1883374726 shutdown_connections 2023-07-14T18:15:15.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.021+0000 7f47ca10f700 1 -- 172.21.15.133:0/1883374726 wait complete. 2023-07-14T18:15:15.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.022+0000 7f47ca10f700 1 Processor -- start 2023-07-14T18:15:15.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.022+0000 7f47ca10f700 1 -- start start 2023-07-14T18:15:15.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.023+0000 7f47ca10f700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47c40ffb30 0x7f47c4197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:15.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.023+0000 7f47ca10f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47c41004e0 0x7f47c4198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:15.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.023+0000 7f47ca10f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47c4105b50 0x7f47c419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:15.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.023+0000 7f47ca10f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c419cd10 con 0x7f47c41004e0 2023-07-14T18:15:15.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.023+0000 7f47ca10f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c419ce80 con 0x7f47c4105b50 2023-07-14T18:15:15.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.023+0000 7f47ca10f700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c419d040 con 0x7f47c40ffb30 2023-07-14T18:15:15.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.023+0000 7f47c910d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47c40ffb30 0x7f47c4197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:15.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.023+0000 7f47c910d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47c40ffb30 0x7f47c4197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:33686/0 (socket says 172.21.15.133:33686) 2023-07-14T18:15:15.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.023+0000 7f47c910d700 1 -- 172.21.15.133:0/2404280804 learned_addr learned my addr 172.21.15.133:0/2404280804 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:15.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.023+0000 7f47c890c700 1 --2- 172.21.15.133:0/2404280804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47c41004e0 0x7f47c4198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:15.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.023+0000 7f47c990e700 1 --2- 172.21.15.133:0/2404280804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47c4105b50 0x7f47c419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:15.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.023+0000 7f47c910d700 1 -- 172.21.15.133:0/2404280804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47c4105b50 msgr2=0x7f47c419c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:15.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.023+0000 7f47c910d700 1 --2- 172.21.15.133:0/2404280804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47c4105b50 0x7f47c419c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:15.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.023+0000 7f47c910d700 1 -- 172.21.15.133:0/2404280804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47c41004e0 msgr2=0x7f47c4198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:15.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.023+0000 7f47c910d700 1 --2- 172.21.15.133:0/2404280804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47c41004e0 0x7f47c4198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:15.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.023+0000 7f47c910d700 1 -- 172.21.15.133:0/2404280804 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b40095f0 con 0x7f47c40ffb30 2023-07-14T18:15:15.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.024+0000 7f47c990e700 1 --2- 172.21.15.133:0/2404280804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47c4105b50 0x7f47c419c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T18:15:15.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.024+0000 7f47c910d700 1 --2- 172.21.15.133:0/2404280804 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47c40ffb30 0x7f47c4197d50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f47c000e9a0 tx=0x7f47c000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:15.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.024+0000 7f47ba7fc700 1 -- 172.21.15.133:0/2404280804 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c0009790 con 0x7f47c40ffb30 2023-07-14T18:15:15.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.024+0000 7f47ba7fc700 1 -- 172.21.15.133:0/2404280804 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47c000cbd0 con 0x7f47c40ffb30 2023-07-14T18:15:15.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.024+0000 7f47ca10f700 1 -- 172.21.15.133:0/2404280804 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c419d320 con 0x7f47c40ffb30 2023-07-14T18:15:15.186 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.024+0000 7f47ba7fc700 1 -- 172.21.15.133:0/2404280804 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c0018680 con 0x7f47c40ffb30 2023-07-14T18:15:15.186 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.024+0000 7f47ca10f700 1 -- 172.21.15.133:0/2404280804 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c41a4b80 con 0x7f47c40ffb30 2023-07-14T18:15:15.186 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.025+0000 7f47ca10f700 1 -- 172.21.15.133:0/2404280804 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f47c404eb90 con 0x7f47c40ffb30 2023-07-14T18:15:15.186 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.029+0000 7f47ba7fc700 1 -- 172.21.15.133:0/2404280804 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f47c000cd40 con 0x7f47c40ffb30 2023-07-14T18:15:15.186 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.030+0000 7f47ba7fc700 1 --2- 172.21.15.133:0/2404280804 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f47b00a11e0 0x7f47b00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:15.186 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.030+0000 7f47ba7fc700 1 -- 172.21.15.133:0/2404280804 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f47c0014070 con 0x7f47c40ffb30 2023-07-14T18:15:15.186 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.030+0000 7f47c890c700 1 --2- 172.21.15.133:0/2404280804 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f47b00a11e0 0x7f47b00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:15.187 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.030+0000 7f47ba7fc700 1 --2- 172.21.15.133:0/2404280804 >> [v2:172.21.15.171:6816/2771494483,v1:172.21.15.171:6817/2771494483] conn(0x7f47b00ac1f0 0x7f47b00ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:15.187 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.030+0000 7f47ba7fc700 1 -- 172.21.15.133:0/2404280804 --> [v2:172.21.15.171:6816/2771494483,v1:172.21.15.171:6817/2771494483] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47b00aed50 con 0x7f47b00ac1f0 2023-07-14T18:15:15.187 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.030+0000 7f47ba7fc700 1 -- 172.21.15.133:0/2404280804 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47c00cd7b0 con 0x7f47c40ffb30 2023-07-14T18:15:15.187 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.031+0000 7f47c890c700 1 --2- 172.21.15.133:0/2404280804 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f47b00a11e0 0x7f47b00a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f47c4199370 tx=0x7f47b4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:15.187 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.031+0000 7f47c990e700 1 --2- 172.21.15.133:0/2404280804 >> [v2:172.21.15.171:6816/2771494483,v1:172.21.15.171:6817/2771494483] conn(0x7f47b00ac1f0 0x7f47b00ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:15.187 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.031+0000 7f47c990e700 1 --2- 172.21.15.133:0/2404280804 >> [v2:172.21.15.171:6816/2771494483,v1:172.21.15.171:6817/2771494483] conn(0x7f47b00ac1f0 0x7f47b00ae680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:15.188 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.032+0000 7f47ba7fc700 1 -- 172.21.15.133:0/2404280804 <== osd.8 v2:172.21.15.171:6816/2771494483 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f47b00aed50 con 0x7f47b00ac1f0 2023-07-14T18:15:15.188 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.055+0000 7f47ca10f700 1 -- 172.21.15.133:0/2404280804 --> [v2:172.21.15.171:6816/2771494483,v1:172.21.15.171:6817/2771494483] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f47c410fd70 con 0x7f47b00ac1f0 2023-07-14T18:15:15.188 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.055+0000 7f47ba7fc700 1 -- 172.21.15.133:0/2404280804 <== osd.8 v2:172.21.15.171:6816/2771494483 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f47bc009050 con 0x7f47b00ac1f0 2023-07-14T18:15:15.188 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.056+0000 7f47ca10f700 1 -- 172.21.15.133:0/2404280804 >> [v2:172.21.15.171:6816/2771494483,v1:172.21.15.171:6817/2771494483] conn(0x7f47b00ac1f0 msgr2=0x7f47b00ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:15.188 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.056+0000 7f47ca10f700 1 --2- 172.21.15.133:0/2404280804 >> [v2:172.21.15.171:6816/2771494483,v1:172.21.15.171:6817/2771494483] conn(0x7f47b00ac1f0 0x7f47b00ae680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:15.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.056+0000 7f47ca10f700 1 -- 172.21.15.133:0/2404280804 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f47b00a11e0 msgr2=0x7f47b00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:15.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.056+0000 7f47ca10f700 1 --2- 172.21.15.133:0/2404280804 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f47b00a11e0 0x7f47b00a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f47c4199370 tx=0x7f47b4005fd0 comp rx=0 tx=0).stop 2023-07-14T18:15:15.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.056+0000 7f47ca10f700 1 -- 172.21.15.133:0/2404280804 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47c40ffb30 msgr2=0x7f47c4197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:15.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.056+0000 7f47ca10f700 1 --2- 172.21.15.133:0/2404280804 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47c40ffb30 0x7f47c4197d50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f47c000e9a0 tx=0x7f47c000ecb0 comp rx=0 tx=0).stop 2023-07-14T18:15:15.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.056+0000 7f47c990e700 1 -- 172.21.15.133:0/2404280804 reap_dead start 2023-07-14T18:15:15.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.056+0000 7f47ca10f700 1 -- 172.21.15.133:0/2404280804 shutdown_connections 2023-07-14T18:15:15.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.056+0000 7f47ca10f700 1 -- 172.21.15.133:0/2404280804 >> 172.21.15.133:0/2404280804 conn(0x7f47c40fb690 msgr2=0x7f47c40fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:15.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.056+0000 7f47ca10f700 1 -- 172.21.15.133:0/2404280804 shutdown_connections 2023-07-14T18:15:15.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.056+0000 7f47ca10f700 1 -- 172.21.15.133:0/2404280804 wait complete. 2023-07-14T18:15:15.257 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:14 smithi133 ceph-mon[126384]: pgmap v303: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-07-14T18:15:15.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:14 smithi161 ceph-mon[128993]: pgmap v303: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-07-14T18:15:15.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:14 smithi171 ceph-mon[129241]: pgmap v303: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-07-14T18:15:16.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.692+0000 7fb9b5e41700 1 -- 172.21.15.133:0/76086038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9b0104fd0 msgr2=0x7fb9b010cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:16.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.692+0000 7fb9b5e41700 1 --2- 172.21.15.133:0/76086038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9b0104fd0 0x7fb9b010cb50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800b0d0 tx=0x7fb9a800b3e0 comp rx=0 tx=0).stop 2023-07-14T18:15:16.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.692+0000 7fb9b5e41700 1 -- 172.21.15.133:0/76086038 shutdown_connections 2023-07-14T18:15:16.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.692+0000 7fb9b5e41700 1 --2- 172.21.15.133:0/76086038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9b00fff30 0x7fb9b0100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:16.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.692+0000 7fb9b5e41700 1 --2- 172.21.15.133:0/76086038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9b0104fd0 0x7fb9b010cb50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:16.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.692+0000 7fb9b5e41700 1 --2- 172.21.15.133:0/76086038 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9b01008d0 0x7fb9b01048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:16.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.692+0000 7fb9b5e41700 1 -- 172.21.15.133:0/76086038 >> 172.21.15.133:0/76086038 conn(0x7fb9b00fb790 msgr2=0x7fb9b00fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:16.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.692+0000 7fb9b5e41700 1 -- 172.21.15.133:0/76086038 shutdown_connections 2023-07-14T18:15:16.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.692+0000 7fb9b5e41700 1 -- 172.21.15.133:0/76086038 wait complete. 2023-07-14T18:15:16.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.693+0000 7fb9b5e41700 1 Processor -- start 2023-07-14T18:15:16.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.694+0000 7fb9b5e41700 1 -- start start 2023-07-14T18:15:16.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.694+0000 7fb9b5e41700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9b00fff30 0x7fb9b0072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:16.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.694+0000 7fb9b5e41700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9b01008d0 0x7fb9b0072e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:16.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.694+0000 7fb9b5e41700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9b0104fd0 0x7fb9b0073400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:16.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.694+0000 7fb9b5e41700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b01a4360 con 0x7fb9b00fff30 2023-07-14T18:15:16.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.694+0000 7fb9b5e41700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b01a44d0 con 0x7fb9b01008d0 2023-07-14T18:15:16.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.694+0000 7fb9b5e41700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b01a4690 con 0x7fb9b0104fd0 2023-07-14T18:15:16.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.694+0000 7fb9af7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9b00fff30 0x7fb9b0072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:16.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.694+0000 7fb9af7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9b00fff30 0x7fb9b0072940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34644/0 (socket says 172.21.15.133:34644) 2023-07-14T18:15:16.251 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.694+0000 7fb9af7fe700 1 -- 172.21.15.133:0/3702050528 learned_addr learned my addr 172.21.15.133:0/3702050528 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:16.251 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.694+0000 7fb9affff700 1 --2- 172.21.15.133:0/3702050528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9b0104fd0 0x7fb9b0073400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:16.251 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.694+0000 7fb9aeffd700 1 --2- 172.21.15.133:0/3702050528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9b01008d0 0x7fb9b0072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:16.251 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.695+0000 7fb9af7fe700 1 -- 172.21.15.133:0/3702050528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9b0104fd0 msgr2=0x7fb9b0073400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:16.252 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.695+0000 7fb9af7fe700 1 --2- 172.21.15.133:0/3702050528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9b0104fd0 0x7fb9b0073400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:16.252 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.695+0000 7fb9af7fe700 1 -- 172.21.15.133:0/3702050528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9b01008d0 msgr2=0x7fb9b0072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:16.252 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.695+0000 7fb9af7fe700 1 --2- 172.21.15.133:0/3702050528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9b01008d0 0x7fb9b0072e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:16.252 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.695+0000 7fb9af7fe700 1 -- 172.21.15.133:0/3702050528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a0009520 con 0x7fb9b00fff30 2023-07-14T18:15:16.252 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.695+0000 7fb9affff700 1 --2- 172.21.15.133:0/3702050528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9b0104fd0 0x7fb9b0073400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:16.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.695+0000 7fb9af7fe700 1 --2- 172.21.15.133:0/3702050528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9b00fff30 0x7fb9b0072940 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000eb30 tx=0x7fb9a000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:16.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.695+0000 7fb9acff9700 1 -- 172.21.15.133:0/3702050528 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0009880 con 0x7fb9b00fff30 2023-07-14T18:15:16.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.695+0000 7fb9b5e41700 1 -- 172.21.15.133:0/3702050528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9a4009520 con 0x7fb9b00fff30 2023-07-14T18:15:16.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.695+0000 7fb9acff9700 1 -- 172.21.15.133:0/3702050528 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9a00044e0 con 0x7fb9b00fff30 2023-07-14T18:15:16.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.695+0000 7fb9acff9700 1 -- 172.21.15.133:0/3702050528 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0003d80 con 0x7fb9b00fff30 2023-07-14T18:15:16.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.695+0000 7fb9b5e41700 1 -- 172.21.15.133:0/3702050528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9a8009d00 con 0x7fb9b00fff30 2023-07-14T18:15:16.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.696+0000 7fb9b5e41700 1 -- 172.21.15.133:0/3702050528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb9b010fe30 con 0x7fb9b00fff30 2023-07-14T18:15:16.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.698+0000 7fb9acff9700 1 -- 172.21.15.133:0/3702050528 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fb9a0003ee0 con 0x7fb9b00fff30 2023-07-14T18:15:16.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.700+0000 7fb9acff9700 1 --2- 172.21.15.133:0/3702050528 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb9980a1260 0x7fb9980a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:16.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.700+0000 7fb9acff9700 1 -- 172.21.15.133:0/3702050528 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb9a0014070 con 0x7fb9b00fff30 2023-07-14T18:15:16.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.700+0000 7fb9aeffd700 1 --2- 172.21.15.133:0/3702050528 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb9980a1260 0x7fb9980a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:16.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.700+0000 7fb9acff9700 1 --2- 172.21.15.133:0/3702050528 >> [v2:172.21.15.133:6810/2152994143,v1:172.21.15.133:6811/2152994143] conn(0x7fb9980ac270 0x7fb9980ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:16.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.700+0000 7fb9acff9700 1 -- 172.21.15.133:0/3702050528 --> [v2:172.21.15.133:6810/2152994143,v1:172.21.15.133:6811/2152994143] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb9980aedd0 con 0x7fb9980ac270 2023-07-14T18:15:16.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.700+0000 7fb9acff9700 1 -- 172.21.15.133:0/3702050528 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9a00cd7d0 con 0x7fb9b00fff30 2023-07-14T18:15:16.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.700+0000 7fb9affff700 1 --2- 172.21.15.133:0/3702050528 >> [v2:172.21.15.133:6810/2152994143,v1:172.21.15.133:6811/2152994143] conn(0x7fb9980ac270 0x7fb9980ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:16.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.700+0000 7fb9aeffd700 1 --2- 172.21.15.133:0/3702050528 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb9980a1260 0x7fb9980a3710 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0074130 tx=0x7fb9a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:16.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.701+0000 7fb9affff700 1 --2- 172.21.15.133:0/3702050528 >> [v2:172.21.15.133:6810/2152994143,v1:172.21.15.133:6811/2152994143] conn(0x7fb9980ac270 0x7fb9980ae700 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:16.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.702+0000 7fb9acff9700 1 -- 172.21.15.133:0/3702050528 <== osd.3 v2:172.21.15.133:6810/2152994143 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb9980aedd0 con 0x7fb9980ac270 2023-07-14T18:15:16.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.719+0000 7fb9b5e41700 1 -- 172.21.15.133:0/3702050528 --> [v2:172.21.15.133:6810/2152994143,v1:172.21.15.133:6811/2152994143] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb9b004e340 con 0x7fb9980ac270 2023-07-14T18:15:16.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.719+0000 7fb9acff9700 1 -- 172.21.15.133:0/3702050528 <== osd.3 v2:172.21.15.133:6810/2152994143 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb9b004e340 con 0x7fb9980ac270 2023-07-14T18:15:16.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.720+0000 7fb9b5e41700 1 -- 172.21.15.133:0/3702050528 >> [v2:172.21.15.133:6810/2152994143,v1:172.21.15.133:6811/2152994143] conn(0x7fb9980ac270 msgr2=0x7fb9980ae700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:16.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.720+0000 7fb9b5e41700 1 --2- 172.21.15.133:0/3702050528 >> [v2:172.21.15.133:6810/2152994143,v1:172.21.15.133:6811/2152994143] conn(0x7fb9980ac270 0x7fb9980ae700 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:16.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.720+0000 7fb9b5e41700 1 -- 172.21.15.133:0/3702050528 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb9980a1260 msgr2=0x7fb9980a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:16.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.720+0000 7fb9b5e41700 1 --2- 172.21.15.133:0/3702050528 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb9980a1260 0x7fb9980a3710 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0074130 tx=0x7fb9a4009260 comp rx=0 tx=0).stop 2023-07-14T18:15:16.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.720+0000 7fb9b5e41700 1 -- 172.21.15.133:0/3702050528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9b00fff30 msgr2=0x7fb9b0072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:16.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.720+0000 7fb9b5e41700 1 --2- 172.21.15.133:0/3702050528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9b00fff30 0x7fb9b0072940 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000eb30 tx=0x7fb9a000ee40 comp rx=0 tx=0).stop 2023-07-14T18:15:16.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.720+0000 7fb9affff700 1 -- 172.21.15.133:0/3702050528 reap_dead start 2023-07-14T18:15:16.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.720+0000 7fb9b5e41700 1 -- 172.21.15.133:0/3702050528 shutdown_connections 2023-07-14T18:15:16.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.720+0000 7fb9b5e41700 1 -- 172.21.15.133:0/3702050528 >> 172.21.15.133:0/3702050528 conn(0x7fb9b00fb790 msgr2=0x7fb9b00fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:16.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.720+0000 7fb9b5e41700 1 -- 172.21.15.133:0/3702050528 shutdown_connections 2023-07-14T18:15:16.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.720+0000 7fb9b5e41700 1 -- 172.21.15.133:0/3702050528 wait complete. 2023-07-14T18:15:16.558 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.513+0000 7f6871ce4700 1 -- 172.21.15.133:0/1555092265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f686c101830 msgr2=0x7f686c109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:16.558 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.513+0000 7f6871ce4700 1 --2- 172.21.15.133:0/1555092265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f686c101830 0x7f686c109d30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6860009990 tx=0x7f6860009ca0 comp rx=0 tx=0).stop 2023-07-14T18:15:16.558 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.513+0000 7f6871ce4700 1 -- 172.21.15.133:0/1555092265 shutdown_connections 2023-07-14T18:15:16.558 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.513+0000 7f6871ce4700 1 --2- 172.21.15.133:0/1555092265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f686c10a270 0x7f686c10c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:16.558 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.513+0000 7f6871ce4700 1 --2- 172.21.15.133:0/1555092265 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f686c100f20 0x7f686c1012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:16.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.513+0000 7f6871ce4700 1 --2- 172.21.15.133:0/1555092265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f686c101830 0x7f686c109d30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:16.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.513+0000 7f6871ce4700 1 -- 172.21.15.133:0/1555092265 >> 172.21.15.133:0/1555092265 conn(0x7f686c075ca0 msgr2=0x7f686c0760a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:16.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.513+0000 7f6871ce4700 1 -- 172.21.15.133:0/1555092265 shutdown_connections 2023-07-14T18:15:16.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.513+0000 7f6871ce4700 1 -- 172.21.15.133:0/1555092265 wait complete. 2023-07-14T18:15:16.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.514+0000 7f6871ce4700 1 Processor -- start 2023-07-14T18:15:16.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.515+0000 7f6871ce4700 1 -- start start 2023-07-14T18:15:16.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.515+0000 7f6871ce4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f686c100f20 0x7f686c197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:16.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.515+0000 7f6871ce4700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f686c101830 0x7f686c1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:16.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.515+0000 7f6871ce4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f686c10a270 0x7f686c19c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:16.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.515+0000 7f6871ce4700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c19cf60 con 0x7f686c100f20 2023-07-14T18:15:16.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.515+0000 7f6871ce4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c19d0d0 con 0x7f686c10a270 2023-07-14T18:15:16.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.515+0000 7f6871ce4700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c19d290 con 0x7f686c101830 2023-07-14T18:15:16.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.515+0000 7f686bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f686c10a270 0x7f686c19c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:16.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.515+0000 7f686b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f686c100f20 0x7f686c197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:16.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.515+0000 7f686bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f686c10a270 0x7f686c19c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:56768/0 (socket says 172.21.15.133:56768) 2023-07-14T18:15:16.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.515+0000 7f686bfff700 1 -- 172.21.15.133:0/3589546901 learned_addr learned my addr 172.21.15.133:0/3589546901 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:16.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.515+0000 7f686affd700 1 --2- 172.21.15.133:0/3589546901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f686c101830 0x7f686c1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:16.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.516+0000 7f686bfff700 1 -- 172.21.15.133:0/3589546901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f686c101830 msgr2=0x7f686c1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:16.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.516+0000 7f686bfff700 1 --2- 172.21.15.133:0/3589546901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f686c101830 0x7f686c1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:16.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.516+0000 7f686bfff700 1 -- 172.21.15.133:0/3589546901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f686c100f20 msgr2=0x7f686c197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:16.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.516+0000 7f686bfff700 1 --2- 172.21.15.133:0/3589546901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f686c100f20 0x7f686c197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:16.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.516+0000 7f686bfff700 1 -- 172.21.15.133:0/3589546901 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6864009c40 con 0x7f686c10a270 2023-07-14T18:15:16.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.516+0000 7f686bfff700 1 --2- 172.21.15.133:0/3589546901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f686c10a270 0x7f686c19c950 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f686400c0f0 tx=0x7f686400c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:16.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.516+0000 7f6868ff9700 1 -- 172.21.15.133:0/3589546901 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6864003a00 con 0x7f686c10a270 2023-07-14T18:15:16.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.516+0000 7f6868ff9700 1 -- 172.21.15.133:0/3589546901 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f686400e030 con 0x7f686c10a270 2023-07-14T18:15:16.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.516+0000 7f6871ce4700 1 -- 172.21.15.133:0/3589546901 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68600095f0 con 0x7f686c10a270 2023-07-14T18:15:16.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.516+0000 7f6868ff9700 1 -- 172.21.15.133:0/3589546901 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6864004650 con 0x7f686c10a270 2023-07-14T18:15:16.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.516+0000 7f6871ce4700 1 -- 172.21.15.133:0/3589546901 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f686c1a4c00 con 0x7f686c10a270 2023-07-14T18:15:16.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.517+0000 7f6871ce4700 1 -- 172.21.15.133:0/3589546901 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f686c04e340 con 0x7f686c10a270 2023-07-14T18:15:16.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.519+0000 7f6868ff9700 1 -- 172.21.15.133:0/3589546901 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f6864003cf0 con 0x7f686c10a270 2023-07-14T18:15:16.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.521+0000 7f6868ff9700 1 --2- 172.21.15.133:0/3589546901 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f68540a11e0 0x7f68540a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:16.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.521+0000 7f6868ff9700 1 -- 172.21.15.133:0/3589546901 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f68640cd200 con 0x7f686c10a270 2023-07-14T18:15:16.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.521+0000 7f686b7fe700 1 --2- 172.21.15.133:0/3589546901 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f68540a11e0 0x7f68540a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:16.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.521+0000 7f6868ff9700 1 --2- 172.21.15.133:0/3589546901 >> [v2:172.21.15.161:6816/4183705749,v1:172.21.15.161:6817/4183705749] conn(0x7f68540ac1f0 0x7f68540ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:16.566 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.521+0000 7f6868ff9700 1 -- 172.21.15.133:0/3589546901 --> [v2:172.21.15.161:6816/4183705749,v1:172.21.15.161:6817/4183705749] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f68540aed50 con 0x7f68540ac1f0 2023-07-14T18:15:16.566 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.521+0000 7f6868ff9700 1 -- 172.21.15.133:0/3589546901 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68640cd570 con 0x7f686c10a270 2023-07-14T18:15:16.566 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.521+0000 7f686b7fe700 1 --2- 172.21.15.133:0/3589546901 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f68540a11e0 0x7f68540a3690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f685c009c60 tx=0x7f685c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:16.566 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.521+0000 7f686affd700 1 --2- 172.21.15.133:0/3589546901 >> [v2:172.21.15.161:6816/4183705749,v1:172.21.15.161:6817/4183705749] conn(0x7f68540ac1f0 0x7f68540ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:16.566 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.522+0000 7f686affd700 1 --2- 172.21.15.133:0/3589546901 >> [v2:172.21.15.161:6816/4183705749,v1:172.21.15.161:6817/4183705749] conn(0x7f68540ac1f0 0x7f68540ae680 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:16.567 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.523+0000 7f6868ff9700 1 -- 172.21.15.133:0/3589546901 <== osd.7 v2:172.21.15.161:6816/4183705749 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f68540aed50 con 0x7f68540ac1f0 2023-07-14T18:15:16.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.539+0000 7f6871ce4700 1 -- 172.21.15.133:0/3589546901 --> [v2:172.21.15.161:6816/4183705749,v1:172.21.15.161:6817/4183705749] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f686c10fe50 con 0x7f68540ac1f0 2023-07-14T18:15:16.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.540+0000 7f6868ff9700 1 -- 172.21.15.133:0/3589546901 <== osd.7 v2:172.21.15.161:6816/4183705749 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f686001e040 con 0x7f68540ac1f0 2023-07-14T18:15:16.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.540+0000 7f6871ce4700 1 -- 172.21.15.133:0/3589546901 >> [v2:172.21.15.161:6816/4183705749,v1:172.21.15.161:6817/4183705749] conn(0x7f68540ac1f0 msgr2=0x7f68540ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:16.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.540+0000 7f6871ce4700 1 --2- 172.21.15.133:0/3589546901 >> [v2:172.21.15.161:6816/4183705749,v1:172.21.15.161:6817/4183705749] conn(0x7f68540ac1f0 0x7f68540ae680 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:16.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.540+0000 7f6871ce4700 1 -- 172.21.15.133:0/3589546901 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f68540a11e0 msgr2=0x7f68540a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:16.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.540+0000 7f6871ce4700 1 --2- 172.21.15.133:0/3589546901 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f68540a11e0 0x7f68540a3690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f685c009c60 tx=0x7f685c009260 comp rx=0 tx=0).stop 2023-07-14T18:15:16.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.540+0000 7f6871ce4700 1 -- 172.21.15.133:0/3589546901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f686c10a270 msgr2=0x7f686c19c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:16.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.540+0000 7f6871ce4700 1 --2- 172.21.15.133:0/3589546901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f686c10a270 0x7f686c19c950 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f686400c0f0 tx=0x7f686400c400 comp rx=0 tx=0).stop 2023-07-14T18:15:16.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.540+0000 7f686bfff700 1 -- 172.21.15.133:0/3589546901 reap_dead start 2023-07-14T18:15:16.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.540+0000 7f6871ce4700 1 -- 172.21.15.133:0/3589546901 shutdown_connections 2023-07-14T18:15:16.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.540+0000 7f6871ce4700 1 -- 172.21.15.133:0/3589546901 >> 172.21.15.133:0/3589546901 conn(0x7f686c075ca0 msgr2=0x7f686c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:16.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.541+0000 7f6871ce4700 1 -- 172.21.15.133:0/3589546901 shutdown_connections 2023-07-14T18:15:16.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:16.541+0000 7f6871ce4700 1 -- 172.21.15.133:0/3589546901 wait complete. 2023-07-14T18:15:16.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.873+0000 7fee2c94a700 1 -- 172.21.15.133:0/2713125714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee28110ce0 msgr2=0x7fee281130c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:16.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.873+0000 7fee2c94a700 1 --2- 172.21.15.133:0/2713125714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee28110ce0 0x7fee281130c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fee2000b0a0 tx=0x7fee2000b3b0 comp rx=0 tx=0).stop 2023-07-14T18:15:16.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.873+0000 7fee2c94a700 1 -- 172.21.15.133:0/2713125714 shutdown_connections 2023-07-14T18:15:16.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.873+0000 7fee2c94a700 1 --2- 172.21.15.133:0/2713125714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee28110ce0 0x7fee281130c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:16.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.873+0000 7fee2c94a700 1 --2- 172.21.15.133:0/2713125714 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee280729e0 0x7fee28072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:16.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.873+0000 7fee2c94a700 1 --2- 172.21.15.133:0/2713125714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee280732f0 0x7fee281107a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:16.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.873+0000 7fee2c94a700 1 -- 172.21.15.133:0/2713125714 >> 172.21.15.133:0/2713125714 conn(0x7fee280fb790 msgr2=0x7fee280fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:16.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.873+0000 7fee2c94a700 1 -- 172.21.15.133:0/2713125714 shutdown_connections 2023-07-14T18:15:16.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.873+0000 7fee2c94a700 1 -- 172.21.15.133:0/2713125714 wait complete. 2023-07-14T18:15:16.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.874+0000 7fee2c94a700 1 Processor -- start 2023-07-14T18:15:16.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.874+0000 7fee2c94a700 1 -- start start 2023-07-14T18:15:16.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.875+0000 7fee2c94a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee280729e0 0x7fee2819c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:16.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.875+0000 7fee2c94a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee280732f0 0x7fee2819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:16.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.875+0000 7fee2c94a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee28110ce0 0x7fee281a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:16.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.875+0000 7fee2c94a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee281a14d0 con 0x7fee28110ce0 2023-07-14T18:15:16.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.875+0000 7fee2c94a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee281a1640 con 0x7fee280729e0 2023-07-14T18:15:16.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.875+0000 7fee2c94a700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee281a1800 con 0x7fee280732f0 2023-07-14T18:15:16.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.875+0000 7fee25d9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee280732f0 0x7fee2819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:16.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.875+0000 7fee25d9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee280732f0 0x7fee2819ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:33736/0 (socket says 172.21.15.133:33736) 2023-07-14T18:15:16.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.875+0000 7fee2659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee280729e0 0x7fee2819c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:16.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.875+0000 7fee25d9b700 1 -- 172.21.15.133:0/876582582 learned_addr learned my addr 172.21.15.133:0/876582582 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:16.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.875+0000 7fee26d9d700 1 --2- 172.21.15.133:0/876582582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee28110ce0 0x7fee281a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:16.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.875+0000 7fee25d9b700 1 -- 172.21.15.133:0/876582582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee280729e0 msgr2=0x7fee2819c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:16.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.875+0000 7fee25d9b700 1 --2- 172.21.15.133:0/876582582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee280729e0 0x7fee2819c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:16.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.875+0000 7fee25d9b700 1 -- 172.21.15.133:0/876582582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee28110ce0 msgr2=0x7fee281a0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:16.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.875+0000 7fee25d9b700 1 --2- 172.21.15.133:0/876582582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee28110ce0 0x7fee281a0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:16.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.876+0000 7fee25d9b700 1 -- 172.21.15.133:0/876582582 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee20009d00 con 0x7fee280732f0 2023-07-14T18:15:16.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.876+0000 7fee25d9b700 1 --2- 172.21.15.133:0/876582582 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee280732f0 0x7fee2819ca50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00e9a0 tx=0x7fee1c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:16.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.876+0000 7fee177fe700 1 -- 172.21.15.133:0/876582582 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c00ca10 con 0x7fee280732f0 2023-07-14T18:15:16.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.876+0000 7fee2c94a700 1 -- 172.21.15.133:0/876582582 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee281a1ae0 con 0x7fee280732f0 2023-07-14T18:15:16.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.876+0000 7fee2c94a700 1 -- 172.21.15.133:0/876582582 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee281a9340 con 0x7fee280732f0 2023-07-14T18:15:16.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.877+0000 7fee177fe700 1 -- 172.21.15.133:0/876582582 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee1c00cb70 con 0x7fee280732f0 2023-07-14T18:15:16.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.877+0000 7fee177fe700 1 -- 172.21.15.133:0/876582582 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c018680 con 0x7fee280732f0 2023-07-14T18:15:16.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.877+0000 7fee2c94a700 1 -- 172.21.15.133:0/876582582 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fee08000ff0 con 0x7fee280732f0 2023-07-14T18:15:16.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.880+0000 7fee177fe700 1 -- 172.21.15.133:0/876582582 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fee1c00cce0 con 0x7fee280732f0 2023-07-14T18:15:16.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.881+0000 7fee177fe700 1 --2- 172.21.15.133:0/876582582 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fee100a1260 0x7fee100a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:16.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.881+0000 7fee177fe700 1 -- 172.21.15.133:0/876582582 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fee1c014070 con 0x7fee280732f0 2023-07-14T18:15:16.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.882+0000 7fee2659c700 1 --2- 172.21.15.133:0/876582582 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fee100a1260 0x7fee100a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:16.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.882+0000 7fee177fe700 1 --2- 172.21.15.133:0/876582582 >> [v2:172.21.15.161:6808/47213291,v1:172.21.15.161:6809/47213291] conn(0x7fee100ac270 0x7fee100ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:16.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.882+0000 7fee177fe700 1 -- 172.21.15.133:0/876582582 --> [v2:172.21.15.161:6808/47213291,v1:172.21.15.161:6809/47213291] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fee100aedd0 con 0x7fee100ac270 2023-07-14T18:15:16.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.882+0000 7fee177fe700 1 -- 172.21.15.133:0/876582582 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee1c0cd510 con 0x7fee280732f0 2023-07-14T18:15:16.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.882+0000 7fee26d9d700 1 --2- 172.21.15.133:0/876582582 >> [v2:172.21.15.161:6808/47213291,v1:172.21.15.161:6809/47213291] conn(0x7fee100ac270 0x7fee100ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:16.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.882+0000 7fee2659c700 1 --2- 172.21.15.133:0/876582582 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fee100a1260 0x7fee100a3710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fee18009c60 tx=0x7fee18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:16.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.883+0000 7fee26d9d700 1 --2- 172.21.15.133:0/876582582 >> [v2:172.21.15.161:6808/47213291,v1:172.21.15.161:6809/47213291] conn(0x7fee100ac270 0x7fee100ae700 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:16.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.883+0000 7fee177fe700 1 -- 172.21.15.133:0/876582582 <== osd.4 v2:172.21.15.161:6808/47213291 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fee100aedd0 con 0x7fee100ac270 2023-07-14T18:15:16.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.901+0000 7fee2c94a700 1 -- 172.21.15.133:0/876582582 --> [v2:172.21.15.161:6808/47213291,v1:172.21.15.161:6809/47213291] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fee08002d70 con 0x7fee100ac270 2023-07-14T18:15:16.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.902+0000 7fee177fe700 1 -- 172.21.15.133:0/876582582 <== osd.4 v2:172.21.15.161:6808/47213291 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fee08002d70 con 0x7fee100ac270 2023-07-14T18:15:16.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.902+0000 7fee2c94a700 1 -- 172.21.15.133:0/876582582 >> [v2:172.21.15.161:6808/47213291,v1:172.21.15.161:6809/47213291] conn(0x7fee100ac270 msgr2=0x7fee100ae700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:16.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.902+0000 7fee2c94a700 1 --2- 172.21.15.133:0/876582582 >> [v2:172.21.15.161:6808/47213291,v1:172.21.15.161:6809/47213291] conn(0x7fee100ac270 0x7fee100ae700 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:16.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.902+0000 7fee2c94a700 1 -- 172.21.15.133:0/876582582 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fee100a1260 msgr2=0x7fee100a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:16.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.902+0000 7fee2c94a700 1 --2- 172.21.15.133:0/876582582 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fee100a1260 0x7fee100a3710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fee18009c60 tx=0x7fee18009260 comp rx=0 tx=0).stop 2023-07-14T18:15:16.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.902+0000 7fee2c94a700 1 -- 172.21.15.133:0/876582582 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee280732f0 msgr2=0x7fee2819ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:16.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.902+0000 7fee2c94a700 1 --2- 172.21.15.133:0/876582582 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee280732f0 0x7fee2819ca50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00e9a0 tx=0x7fee1c00ecb0 comp rx=0 tx=0).stop 2023-07-14T18:15:16.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.903+0000 7fee26d9d700 1 -- 172.21.15.133:0/876582582 reap_dead start 2023-07-14T18:15:16.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.903+0000 7fee2c94a700 1 -- 172.21.15.133:0/876582582 shutdown_connections 2023-07-14T18:15:16.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.903+0000 7fee2c94a700 1 -- 172.21.15.133:0/876582582 >> 172.21.15.133:0/876582582 conn(0x7fee280fb790 msgr2=0x7fee280fc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:16.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.903+0000 7fee2c94a700 1 -- 172.21.15.133:0/876582582 shutdown_connections 2023-07-14T18:15:16.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:15.903+0000 7fee2c94a700 1 -- 172.21.15.133:0/876582582 wait complete. 2023-07-14T18:15:16.792 INFO:teuthology.orchestra.run.smithi133.stdout:253403070502 2023-07-14T18:15:16.793 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.8 2023-07-14T18:15:17.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:16 smithi133 ceph-mon[126384]: pgmap v304: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2023-07-14T18:15:17.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:16 smithi161 ceph-mon[128993]: pgmap v304: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2023-07-14T18:15:17.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:16 smithi171 ceph-mon[129241]: pgmap v304: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2023-07-14T18:15:17.968 INFO:teuthology.orchestra.run.smithi133.stdout:141733920823 2023-07-14T18:15:17.969 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.4 2023-07-14T18:15:18.031 INFO:teuthology.orchestra.run.smithi133.stdout:219043332137 2023-07-14T18:15:18.032 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.7 2023-07-14T18:15:18.537 INFO:teuthology.orchestra.run.smithi133.stdout:111669149756 2023-07-14T18:15:18.538 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.3 2023-07-14T18:15:18.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.546+0000 7f8cd812c700 1 -- 172.21.15.133:0/600997292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cd01008d0 msgr2=0x7f8cd01048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:18.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.546+0000 7f8cd812c700 1 --2- 172.21.15.133:0/600997292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cd01008d0 0x7f8cd01048a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0009990 tx=0x7f8cc0009ca0 comp rx=0 tx=0).stop 2023-07-14T18:15:18.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.546+0000 7f8cd812c700 1 -- 172.21.15.133:0/600997292 shutdown_connections 2023-07-14T18:15:18.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.546+0000 7f8cd812c700 1 --2- 172.21.15.133:0/600997292 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8cd0104fd0 0x7f8cd010cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:18.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.546+0000 7f8cd812c700 1 --2- 172.21.15.133:0/600997292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cd00fff30 0x7f8cd0100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:18.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.546+0000 7f8cd812c700 1 --2- 172.21.15.133:0/600997292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cd01008d0 0x7f8cd01048a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:18.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.546+0000 7f8cd812c700 1 -- 172.21.15.133:0/600997292 >> 172.21.15.133:0/600997292 conn(0x7f8cd00fb790 msgr2=0x7f8cd00fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:18.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.547+0000 7f8cd812c700 1 -- 172.21.15.133:0/600997292 shutdown_connections 2023-07-14T18:15:18.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.547+0000 7f8cd812c700 1 -- 172.21.15.133:0/600997292 wait complete. 2023-07-14T18:15:18.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.548+0000 7f8cd812c700 1 Processor -- start 2023-07-14T18:15:18.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.548+0000 7f8cd812c700 1 -- start start 2023-07-14T18:15:18.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.548+0000 7f8cd812c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cd00fff30 0x7f8cd0195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:18.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.548+0000 7f8cd812c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8cd01008d0 0x7f8cd01963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:18.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.548+0000 7f8cd812c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cd0104fd0 0x7f8cd019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:18.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.548+0000 7f8cd812c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd019ada0 con 0x7f8cd0104fd0 2023-07-14T18:15:18.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.548+0000 7f8cd812c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd019af10 con 0x7f8cd00fff30 2023-07-14T18:15:18.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.548+0000 7f8cd812c700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd019b0d0 con 0x7f8cd01008d0 2023-07-14T18:15:18.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.549+0000 7f8cd56c7700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8cd01008d0 0x7f8cd01963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:18.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.549+0000 7f8cd56c7700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8cd01008d0 0x7f8cd01963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:47848/0 (socket says 172.21.15.133:47848) 2023-07-14T18:15:18.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.549+0000 7f8cd56c7700 1 -- 172.21.15.133:0/1416382659 learned_addr learned my addr 172.21.15.133:0/1416382659 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:18.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.549+0000 7f8cd5ec8700 1 --2- 172.21.15.133:0/1416382659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cd00fff30 0x7f8cd0195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:18.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.549+0000 7f8cd66c9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cd0104fd0 0x7f8cd019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:18.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.549+0000 7f8cd5ec8700 1 -- 172.21.15.133:0/1416382659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8cd01008d0 msgr2=0x7f8cd01963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:18.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.549+0000 7f8cd5ec8700 1 --2- 172.21.15.133:0/1416382659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8cd01008d0 0x7f8cd01963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:18.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.549+0000 7f8cd5ec8700 1 -- 172.21.15.133:0/1416382659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cd0104fd0 msgr2=0x7f8cd019a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:18.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.549+0000 7f8cd5ec8700 1 --2- 172.21.15.133:0/1416382659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cd0104fd0 0x7f8cd019a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:18.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.549+0000 7f8cd5ec8700 1 -- 172.21.15.133:0/1416382659 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc00095f0 con 0x7f8cd00fff30 2023-07-14T18:15:18.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.549+0000 7f8cd66c9700 1 --2- 172.21.15.133:0/1416382659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cd0104fd0 0x7f8cd019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:18.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.550+0000 7f8cd5ec8700 1 --2- 172.21.15.133:0/1416382659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cd00fff30 0x7f8cd0195e80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00e9f0 tx=0x7f8ccc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:18.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.550+0000 7f8cc6ffd700 1 -- 172.21.15.133:0/1416382659 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc00c9c0 con 0x7f8cd00fff30 2023-07-14T18:15:18.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.550+0000 7f8cd812c700 1 -- 172.21.15.133:0/1416382659 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd019b3b0 con 0x7f8cd00fff30 2023-07-14T18:15:18.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.550+0000 7f8cc6ffd700 1 -- 172.21.15.133:0/1416382659 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ccc010430 con 0x7f8cd00fff30 2023-07-14T18:15:18.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.550+0000 7f8cc6ffd700 1 -- 172.21.15.133:0/1416382659 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc018680 con 0x7f8cd00fff30 2023-07-14T18:15:18.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.550+0000 7f8cd812c700 1 -- 172.21.15.133:0/1416382659 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd0072a40 con 0x7f8cd00fff30 2023-07-14T18:15:18.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.551+0000 7f8cd812c700 1 -- 172.21.15.133:0/1416382659 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8cd004e340 con 0x7f8cd00fff30 2023-07-14T18:15:18.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.554+0000 7f8cc6ffd700 1 -- 172.21.15.133:0/1416382659 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f8ccc00cb20 con 0x7f8cd00fff30 2023-07-14T18:15:18.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.555+0000 7f8cc6ffd700 1 --2- 172.21.15.133:0/1416382659 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8cbc0a11e0 0x7f8cbc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:18.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.555+0000 7f8cc6ffd700 1 -- 172.21.15.133:0/1416382659 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8ccc014070 con 0x7f8cd00fff30 2023-07-14T18:15:18.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.556+0000 7f8cd56c7700 1 --2- 172.21.15.133:0/1416382659 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8cbc0a11e0 0x7f8cbc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:18.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.556+0000 7f8cc6ffd700 1 --2- 172.21.15.133:0/1416382659 >> [v2:172.21.15.171:6824/3403894313,v1:172.21.15.171:6825/3403894313] conn(0x7f8cbc0ac1f0 0x7f8cbc0ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:18.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.556+0000 7f8cc6ffd700 1 -- 172.21.15.133:0/1416382659 --> [v2:172.21.15.171:6824/3403894313,v1:172.21.15.171:6825/3403894313] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8cbc0aed50 con 0x7f8cbc0ac1f0 2023-07-14T18:15:18.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.556+0000 7f8cc6ffd700 1 -- 172.21.15.133:0/1416382659 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ccc0cd7c0 con 0x7f8cd00fff30 2023-07-14T18:15:18.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.556+0000 7f8cd56c7700 1 --2- 172.21.15.133:0/1416382659 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8cbc0a11e0 0x7f8cbc0a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000b2e0 tx=0x7f8cc0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:18.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.556+0000 7f8cd66c9700 1 --2- 172.21.15.133:0/1416382659 >> [v2:172.21.15.171:6824/3403894313,v1:172.21.15.171:6825/3403894313] conn(0x7f8cbc0ac1f0 0x7f8cbc0ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:18.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.557+0000 7f8cd66c9700 1 --2- 172.21.15.133:0/1416382659 >> [v2:172.21.15.171:6824/3403894313,v1:172.21.15.171:6825/3403894313] conn(0x7f8cbc0ac1f0 0x7f8cbc0ae680 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:18.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.558+0000 7f8cc6ffd700 1 -- 172.21.15.133:0/1416382659 <== osd.11 v2:172.21.15.171:6824/3403894313 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8cbc0aed50 con 0x7f8cbc0ac1f0 2023-07-14T18:15:18.630 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.586+0000 7f8cd812c700 1 -- 172.21.15.133:0/1416382659 --> [v2:172.21.15.171:6824/3403894313,v1:172.21.15.171:6825/3403894313] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8cd010fed0 con 0x7f8cbc0ac1f0 2023-07-14T18:15:18.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.587+0000 7f8cc6ffd700 1 -- 172.21.15.133:0/1416382659 <== osd.11 v2:172.21.15.171:6824/3403894313 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8cc8016040 con 0x7f8cbc0ac1f0 2023-07-14T18:15:18.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.587+0000 7f8cd812c700 1 -- 172.21.15.133:0/1416382659 >> [v2:172.21.15.171:6824/3403894313,v1:172.21.15.171:6825/3403894313] conn(0x7f8cbc0ac1f0 msgr2=0x7f8cbc0ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:18.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.587+0000 7f8cd812c700 1 --2- 172.21.15.133:0/1416382659 >> [v2:172.21.15.171:6824/3403894313,v1:172.21.15.171:6825/3403894313] conn(0x7f8cbc0ac1f0 0x7f8cbc0ae680 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:18.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.588+0000 7f8cd812c700 1 -- 172.21.15.133:0/1416382659 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8cbc0a11e0 msgr2=0x7f8cbc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:18.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.588+0000 7f8cd812c700 1 --2- 172.21.15.133:0/1416382659 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8cbc0a11e0 0x7f8cbc0a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000b2e0 tx=0x7f8cc0005fb0 comp rx=0 tx=0).stop 2023-07-14T18:15:18.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.588+0000 7f8cd812c700 1 -- 172.21.15.133:0/1416382659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cd00fff30 msgr2=0x7f8cd0195e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:18.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.588+0000 7f8cd812c700 1 --2- 172.21.15.133:0/1416382659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cd00fff30 0x7f8cd0195e80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00e9f0 tx=0x7f8ccc00ed00 comp rx=0 tx=0).stop 2023-07-14T18:15:18.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.588+0000 7f8cd66c9700 1 -- 172.21.15.133:0/1416382659 reap_dead start 2023-07-14T18:15:18.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.588+0000 7f8cd812c700 1 -- 172.21.15.133:0/1416382659 shutdown_connections 2023-07-14T18:15:18.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.588+0000 7f8cd812c700 1 -- 172.21.15.133:0/1416382659 >> 172.21.15.133:0/1416382659 conn(0x7f8cd00fb790 msgr2=0x7f8cd00fc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:18.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.588+0000 7f8cd812c700 1 -- 172.21.15.133:0/1416382659 shutdown_connections 2023-07-14T18:15:18.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.588+0000 7f8cd812c700 1 -- 172.21.15.133:0/1416382659 wait complete. 2023-07-14T18:15:18.803 INFO:teuthology.orchestra.run.smithi133.stdout:77309411392 2023-07-14T18:15:18.804 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.2 2023-07-14T18:15:18.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.887+0000 7ff679ef2700 1 -- 172.21.15.133:0/59955769 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6741008d0 msgr2=0x7ff6741048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:18.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.887+0000 7ff679ef2700 1 --2- 172.21.15.133:0/59955769 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6741008d0 0x7ff6741048a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff6680099c0 tx=0x7ff668009cd0 comp rx=0 tx=0).stop 2023-07-14T18:15:18.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.888+0000 7ff679ef2700 1 -- 172.21.15.133:0/59955769 shutdown_connections 2023-07-14T18:15:18.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.888+0000 7ff679ef2700 1 --2- 172.21.15.133:0/59955769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6740fff30 0x7ff674100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:18.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.888+0000 7ff679ef2700 1 --2- 172.21.15.133:0/59955769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff674104fd0 0x7ff67410cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:18.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.888+0000 7ff679ef2700 1 --2- 172.21.15.133:0/59955769 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6741008d0 0x7ff6741048a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:18.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.888+0000 7ff679ef2700 1 -- 172.21.15.133:0/59955769 >> 172.21.15.133:0/59955769 conn(0x7ff6740fb790 msgr2=0x7ff6740fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:18.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.888+0000 7ff679ef2700 1 -- 172.21.15.133:0/59955769 shutdown_connections 2023-07-14T18:15:18.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.888+0000 7ff679ef2700 1 -- 172.21.15.133:0/59955769 wait complete. 2023-07-14T18:15:18.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.889+0000 7ff679ef2700 1 Processor -- start 2023-07-14T18:15:18.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.889+0000 7ff679ef2700 1 -- start start 2023-07-14T18:15:18.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.890+0000 7ff679ef2700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6740fff30 0x7ff674072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:18.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.890+0000 7ff679ef2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6741008d0 0x7ff674072e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:18.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.890+0000 7ff679ef2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff674104fd0 0x7ff674073400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:18.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.890+0000 7ff679ef2700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6741a4220 con 0x7ff6741008d0 2023-07-14T18:15:18.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.890+0000 7ff679ef2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6741a4390 con 0x7ff674104fd0 2023-07-14T18:15:18.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.890+0000 7ff679ef2700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6741a4550 con 0x7ff6740fff30 2023-07-14T18:15:18.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.890+0000 7ff673fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff674104fd0 0x7ff674073400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:18.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.890+0000 7ff673fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff674104fd0 0x7ff674073400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:38570/0 (socket says 172.21.15.133:38570) 2023-07-14T18:15:18.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.890+0000 7ff673fff700 1 -- 172.21.15.133:0/2952080015 learned_addr learned my addr 172.21.15.133:0/2952080015 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:18.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.890+0000 7ff6737fe700 1 --2- 172.21.15.133:0/2952080015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6740fff30 0x7ff674072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:18.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.890+0000 7ff672ffd700 1 --2- 172.21.15.133:0/2952080015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6741008d0 0x7ff674072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:18.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.890+0000 7ff6737fe700 1 -- 172.21.15.133:0/2952080015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff674104fd0 msgr2=0x7ff674073400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:18.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.890+0000 7ff6737fe700 1 --2- 172.21.15.133:0/2952080015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff674104fd0 0x7ff674073400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:18.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.890+0000 7ff6737fe700 1 -- 172.21.15.133:0/2952080015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6741008d0 msgr2=0x7ff674072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:18.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.890+0000 7ff6737fe700 1 --2- 172.21.15.133:0/2952080015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6741008d0 0x7ff674072e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:18.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.890+0000 7ff6737fe700 1 -- 172.21.15.133:0/2952080015 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6680095f0 con 0x7ff6740fff30 2023-07-14T18:15:18.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.890+0000 7ff673fff700 1 --2- 172.21.15.133:0/2952080015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff674104fd0 0x7ff674073400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:18.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.891+0000 7ff6737fe700 1 --2- 172.21.15.133:0/2952080015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6740fff30 0x7ff674072940 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff66400e9f0 tx=0x7ff66400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:18.938 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.891+0000 7ff670ff9700 1 -- 172.21.15.133:0/2952080015 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66400c9c0 con 0x7ff6740fff30 2023-07-14T18:15:18.938 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.891+0000 7ff679ef2700 1 -- 172.21.15.133:0/2952080015 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6741a4830 con 0x7ff6740fff30 2023-07-14T18:15:18.938 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.891+0000 7ff670ff9700 1 -- 172.21.15.133:0/2952080015 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff664010430 con 0x7ff6740fff30 2023-07-14T18:15:18.938 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.891+0000 7ff670ff9700 1 -- 172.21.15.133:0/2952080015 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff664018680 con 0x7ff6740fff30 2023-07-14T18:15:18.938 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.891+0000 7ff679ef2700 1 -- 172.21.15.133:0/2952080015 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6741a4d50 con 0x7ff6740fff30 2023-07-14T18:15:18.938 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.892+0000 7ff679ef2700 1 -- 172.21.15.133:0/2952080015 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff67404e340 con 0x7ff6740fff30 2023-07-14T18:15:18.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.895+0000 7ff670ff9700 1 -- 172.21.15.133:0/2952080015 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7ff664010860 con 0x7ff6740fff30 2023-07-14T18:15:18.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.896+0000 7ff670ff9700 1 --2- 172.21.15.133:0/2952080015 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff65c0a1260 0x7ff65c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:18.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.896+0000 7ff670ff9700 1 -- 172.21.15.133:0/2952080015 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff664014070 con 0x7ff6740fff30 2023-07-14T18:15:18.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.896+0000 7ff670ff9700 1 --2- 172.21.15.133:0/2952080015 >> [v2:172.21.15.171:6808/777556751,v1:172.21.15.171:6809/777556751] conn(0x7ff65c0ac270 0x7ff65c0ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:18.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.896+0000 7ff670ff9700 1 -- 172.21.15.133:0/2952080015 --> [v2:172.21.15.171:6808/777556751,v1:172.21.15.171:6809/777556751] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff65c0aedd0 con 0x7ff65c0ac270 2023-07-14T18:15:18.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.896+0000 7ff670ff9700 1 -- 172.21.15.133:0/2952080015 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6640cd4f0 con 0x7ff6740fff30 2023-07-14T18:15:18.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.896+0000 7ff672ffd700 1 --2- 172.21.15.133:0/2952080015 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff65c0a1260 0x7ff65c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:18.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.897+0000 7ff673fff700 1 --2- 172.21.15.133:0/2952080015 >> [v2:172.21.15.171:6808/777556751,v1:172.21.15.171:6809/777556751] conn(0x7ff65c0ac270 0x7ff65c0ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:18.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.897+0000 7ff673fff700 1 --2- 172.21.15.133:0/2952080015 >> [v2:172.21.15.171:6808/777556751,v1:172.21.15.171:6809/777556751] conn(0x7ff65c0ac270 0x7ff65c0ae700 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:18.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.897+0000 7ff672ffd700 1 --2- 172.21.15.133:0/2952080015 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff65c0a1260 0x7ff65c0a3710 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff668009fd0 tx=0x7ff668005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:18.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.898+0000 7ff670ff9700 1 -- 172.21.15.133:0/2952080015 <== osd.5 v2:172.21.15.171:6808/777556751 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff65c0aedd0 con 0x7ff65c0ac270 2023-07-14T18:15:18.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.920+0000 7ff679ef2700 1 -- 172.21.15.133:0/2952080015 --> [v2:172.21.15.171:6808/777556751,v1:172.21.15.171:6809/777556751] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff67410fe70 con 0x7ff65c0ac270 2023-07-14T18:15:18.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.921+0000 7ff670ff9700 1 -- 172.21.15.133:0/2952080015 <== osd.5 v2:172.21.15.171:6808/777556751 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff66c00f040 con 0x7ff65c0ac270 2023-07-14T18:15:18.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.922+0000 7ff679ef2700 1 -- 172.21.15.133:0/2952080015 >> [v2:172.21.15.171:6808/777556751,v1:172.21.15.171:6809/777556751] conn(0x7ff65c0ac270 msgr2=0x7ff65c0ae700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:18.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.922+0000 7ff679ef2700 1 --2- 172.21.15.133:0/2952080015 >> [v2:172.21.15.171:6808/777556751,v1:172.21.15.171:6809/777556751] conn(0x7ff65c0ac270 0x7ff65c0ae700 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:18.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.922+0000 7ff679ef2700 1 -- 172.21.15.133:0/2952080015 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff65c0a1260 msgr2=0x7ff65c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:18.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.922+0000 7ff679ef2700 1 --2- 172.21.15.133:0/2952080015 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff65c0a1260 0x7ff65c0a3710 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff668009fd0 tx=0x7ff668005fb0 comp rx=0 tx=0).stop 2023-07-14T18:15:18.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.922+0000 7ff679ef2700 1 -- 172.21.15.133:0/2952080015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6740fff30 msgr2=0x7ff674072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:18.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.922+0000 7ff679ef2700 1 --2- 172.21.15.133:0/2952080015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6740fff30 0x7ff674072940 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff66400e9f0 tx=0x7ff66400ed00 comp rx=0 tx=0).stop 2023-07-14T18:15:18.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.922+0000 7ff673fff700 1 -- 172.21.15.133:0/2952080015 reap_dead start 2023-07-14T18:15:18.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.922+0000 7ff679ef2700 1 -- 172.21.15.133:0/2952080015 shutdown_connections 2023-07-14T18:15:18.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.922+0000 7ff679ef2700 1 -- 172.21.15.133:0/2952080015 >> 172.21.15.133:0/2952080015 conn(0x7ff6740fb790 msgr2=0x7ff6740fc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:18.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.923+0000 7ff679ef2700 1 -- 172.21.15.133:0/2952080015 shutdown_connections 2023-07-14T18:15:18.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:18.923+0000 7ff679ef2700 1 -- 172.21.15.133:0/2952080015 wait complete. 2023-07-14T18:15:19.334 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:18 smithi133 ceph-mon[126384]: pgmap v305: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-14T18:15:19.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:18 smithi161 ceph-mon[128993]: pgmap v305: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-14T18:15:19.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:18 smithi171 ceph-mon[129241]: pgmap v305: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-14T18:15:19.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.235+0000 7f9c64555700 1 -- 172.21.15.133:0/4166129628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c5c1008d0 msgr2=0x7f9c5c1048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:19.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.235+0000 7f9c64555700 1 --2- 172.21.15.133:0/4166129628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c5c1008d0 0x7f9c5c1048a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009990 tx=0x7f9c4c009ca0 comp rx=0 tx=0).stop 2023-07-14T18:15:19.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.235+0000 7f9c64555700 1 -- 172.21.15.133:0/4166129628 shutdown_connections 2023-07-14T18:15:19.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.235+0000 7f9c64555700 1 --2- 172.21.15.133:0/4166129628 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c5c104fd0 0x7f9c5c10cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:19.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.235+0000 7f9c64555700 1 --2- 172.21.15.133:0/4166129628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c5c0fff30 0x7f9c5c100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:19.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.235+0000 7f9c64555700 1 --2- 172.21.15.133:0/4166129628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c5c1008d0 0x7f9c5c1048a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:19.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.235+0000 7f9c64555700 1 -- 172.21.15.133:0/4166129628 >> 172.21.15.133:0/4166129628 conn(0x7f9c5c0fb790 msgr2=0x7f9c5c0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:19.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.235+0000 7f9c64555700 1 -- 172.21.15.133:0/4166129628 shutdown_connections 2023-07-14T18:15:19.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.235+0000 7f9c64555700 1 -- 172.21.15.133:0/4166129628 wait complete. 2023-07-14T18:15:19.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.236+0000 7f9c64555700 1 Processor -- start 2023-07-14T18:15:19.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.237+0000 7f9c64555700 1 -- start start 2023-07-14T18:15:19.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.237+0000 7f9c64555700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c5c0fff30 0x7f9c5c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:19.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.237+0000 7f9c64555700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c5c1008d0 0x7f9c5c198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:19.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.237+0000 7f9c64555700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c5c104fd0 0x7f9c5c19c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:19.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.237+0000 7f9c64555700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c19cff0 con 0x7f9c5c0fff30 2023-07-14T18:15:19.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.237+0000 7f9c64555700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c19d160 con 0x7f9c5c104fd0 2023-07-14T18:15:19.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.237+0000 7f9c64555700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c19d320 con 0x7f9c5c1008d0 2023-07-14T18:15:19.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.237+0000 7f9c622f1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c5c0fff30 0x7f9c5c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:19.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.237+0000 7f9c622f1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c5c0fff30 0x7f9c5c198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33918/0 (socket says 172.21.15.133:33918) 2023-07-14T18:15:19.476 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.237+0000 7f9c622f1700 1 -- 172.21.15.133:0/1678320252 learned_addr learned my addr 172.21.15.133:0/1678320252 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:19.476 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.237+0000 7f9c61af0700 1 --2- 172.21.15.133:0/1678320252 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c5c1008d0 0x7f9c5c198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:19.476 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.237+0000 7f9c62af2700 1 --2- 172.21.15.133:0/1678320252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c5c104fd0 0x7f9c5c19c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:19.476 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.238+0000 7f9c61af0700 1 -- 172.21.15.133:0/1678320252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c5c104fd0 msgr2=0x7f9c5c19c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:19.476 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.238+0000 7f9c61af0700 1 --2- 172.21.15.133:0/1678320252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c5c104fd0 0x7f9c5c19c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:19.477 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.238+0000 7f9c61af0700 1 -- 172.21.15.133:0/1678320252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c5c0fff30 msgr2=0x7f9c5c198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:19.477 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.238+0000 7f9c61af0700 1 --2- 172.21.15.133:0/1678320252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c5c0fff30 0x7f9c5c198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:19.477 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.238+0000 7f9c61af0700 1 -- 172.21.15.133:0/1678320252 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4c0095f0 con 0x7f9c5c1008d0 2023-07-14T18:15:19.477 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.238+0000 7f9c622f1700 1 --2- 172.21.15.133:0/1678320252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c5c0fff30 0x7f9c5c198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:19.477 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.238+0000 7f9c61af0700 1 --2- 172.21.15.133:0/1678320252 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c5c1008d0 0x7f9c5c198570 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c000c00 tx=0x7f9c4c003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:19.477 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.238+0000 7f9c537fe700 1 -- 172.21.15.133:0/1678320252 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c01d070 con 0x7f9c5c1008d0 2023-07-14T18:15:19.477 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.238+0000 7f9c64555700 1 -- 172.21.15.133:0/1678320252 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5c19d5a0 con 0x7f9c5c1008d0 2023-07-14T18:15:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.238+0000 7f9c537fe700 1 -- 172.21.15.133:0/1678320252 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c4c005580 con 0x7f9c5c1008d0 2023-07-14T18:15:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.238+0000 7f9c537fe700 1 -- 172.21.15.133:0/1678320252 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c00fc90 con 0x7f9c5c1008d0 2023-07-14T18:15:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.238+0000 7f9c64555700 1 -- 172.21.15.133:0/1678320252 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5c1a4e60 con 0x7f9c5c1008d0 2023-07-14T18:15:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.239+0000 7f9c64555700 1 -- 172.21.15.133:0/1678320252 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9c5c10fe30 con 0x7f9c5c1008d0 2023-07-14T18:15:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.241+0000 7f9c537fe700 1 -- 172.21.15.133:0/1678320252 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f9c4c00f460 con 0x7f9c5c1008d0 2023-07-14T18:15:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.243+0000 7f9c537fe700 1 --2- 172.21.15.133:0/1678320252 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f9c480a11e0 0x7f9c480a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.243+0000 7f9c537fe700 1 -- 172.21.15.133:0/1678320252 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9c4c0cdfa0 con 0x7f9c5c1008d0 2023-07-14T18:15:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.243+0000 7f9c622f1700 1 --2- 172.21.15.133:0/1678320252 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f9c480a11e0 0x7f9c480a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:19.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.243+0000 7f9c537fe700 1 --2- 172.21.15.133:0/1678320252 >> [v2:172.21.15.133:6826/2066164149,v1:172.21.15.133:6827/2066164149] conn(0x7f9c480ac1f0 0x7f9c480ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:19.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.243+0000 7f9c537fe700 1 -- 172.21.15.133:0/1678320252 --> [v2:172.21.15.133:6826/2066164149,v1:172.21.15.133:6827/2066164149] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9c480aed50 con 0x7f9c480ac1f0 2023-07-14T18:15:19.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.243+0000 7f9c537fe700 1 -- 172.21.15.133:0/1678320252 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c4c0ce310 con 0x7f9c5c1008d0 2023-07-14T18:15:19.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.243+0000 7f9c62af2700 1 --2- 172.21.15.133:0/1678320252 >> [v2:172.21.15.133:6826/2066164149,v1:172.21.15.133:6827/2066164149] conn(0x7f9c480ac1f0 0x7f9c480ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:19.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.244+0000 7f9c622f1700 1 --2- 172.21.15.133:0/1678320252 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f9c480a11e0 0x7f9c480a3690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9c58005950 tx=0x7f9c5800b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:19.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.244+0000 7f9c62af2700 1 --2- 172.21.15.133:0/1678320252 >> [v2:172.21.15.133:6826/2066164149,v1:172.21.15.133:6827/2066164149] conn(0x7f9c480ac1f0 0x7f9c480ae680 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:19.480 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.245+0000 7f9c537fe700 1 -- 172.21.15.133:0/1678320252 <== osd.9 v2:172.21.15.133:6826/2066164149 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9c480aed50 con 0x7f9c480ac1f0 2023-07-14T18:15:19.480 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.264+0000 7f9c64555700 1 -- 172.21.15.133:0/1678320252 --> [v2:172.21.15.133:6826/2066164149,v1:172.21.15.133:6827/2066164149] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9c5c19d870 con 0x7f9c480ac1f0 2023-07-14T18:15:19.480 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.264+0000 7f9c537fe700 1 -- 172.21.15.133:0/1678320252 <== osd.9 v2:172.21.15.133:6826/2066164149 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9c5400f040 con 0x7f9c480ac1f0 2023-07-14T18:15:19.481 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.265+0000 7f9c64555700 1 -- 172.21.15.133:0/1678320252 >> [v2:172.21.15.133:6826/2066164149,v1:172.21.15.133:6827/2066164149] conn(0x7f9c480ac1f0 msgr2=0x7f9c480ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:19.481 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.265+0000 7f9c64555700 1 --2- 172.21.15.133:0/1678320252 >> [v2:172.21.15.133:6826/2066164149,v1:172.21.15.133:6827/2066164149] conn(0x7f9c480ac1f0 0x7f9c480ae680 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:19.481 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.265+0000 7f9c64555700 1 -- 172.21.15.133:0/1678320252 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f9c480a11e0 msgr2=0x7f9c480a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:19.481 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.265+0000 7f9c64555700 1 --2- 172.21.15.133:0/1678320252 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f9c480a11e0 0x7f9c480a3690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9c58005950 tx=0x7f9c5800b500 comp rx=0 tx=0).stop 2023-07-14T18:15:19.481 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.265+0000 7f9c64555700 1 -- 172.21.15.133:0/1678320252 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c5c1008d0 msgr2=0x7f9c5c198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:19.481 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.265+0000 7f9c64555700 1 --2- 172.21.15.133:0/1678320252 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c5c1008d0 0x7f9c5c198570 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c000c00 tx=0x7f9c4c003b90 comp rx=0 tx=0).stop 2023-07-14T18:15:19.481 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.265+0000 7f9c62af2700 1 -- 172.21.15.133:0/1678320252 reap_dead start 2023-07-14T18:15:19.481 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.265+0000 7f9c64555700 1 -- 172.21.15.133:0/1678320252 shutdown_connections 2023-07-14T18:15:19.482 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.265+0000 7f9c64555700 1 -- 172.21.15.133:0/1678320252 >> 172.21.15.133:0/1678320252 conn(0x7f9c5c0fb790 msgr2=0x7f9c5c0fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:19.482 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.265+0000 7f9c64555700 1 -- 172.21.15.133:0/1678320252 shutdown_connections 2023-07-14T18:15:19.482 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.265+0000 7f9c64555700 1 -- 172.21.15.133:0/1678320252 wait complete. 2023-07-14T18:15:19.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.446+0000 7fcdf4ea5700 1 -- 172.21.15.133:0/804698575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdf01016a0 msgr2=0x7fcdf0109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:19.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.446+0000 7fcdf4ea5700 1 --2- 172.21.15.133:0/804698575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdf01016a0 0x7fcdf0109d70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcde4009990 tx=0x7fcde4009ca0 comp rx=0 tx=0).stop 2023-07-14T18:15:19.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.447+0000 7fcdf4ea5700 1 -- 172.21.15.133:0/804698575 shutdown_connections 2023-07-14T18:15:19.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.447+0000 7fcdf4ea5700 1 --2- 172.21.15.133:0/804698575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdf010a2b0 0x7fcdf010c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:19.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.447+0000 7fcdf4ea5700 1 --2- 172.21.15.133:0/804698575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdf0100d90 0x7fcdf0101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:19.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.447+0000 7fcdf4ea5700 1 --2- 172.21.15.133:0/804698575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdf01016a0 0x7fcdf0109d70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:19.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.447+0000 7fcdf4ea5700 1 -- 172.21.15.133:0/804698575 >> 172.21.15.133:0/804698575 conn(0x7fcdf0075ca0 msgr2=0x7fcdf00760a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:19.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.447+0000 7fcdf4ea5700 1 -- 172.21.15.133:0/804698575 shutdown_connections 2023-07-14T18:15:19.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.447+0000 7fcdf4ea5700 1 -- 172.21.15.133:0/804698575 wait complete. 2023-07-14T18:15:19.493 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.448+0000 7fcdf4ea5700 1 Processor -- start 2023-07-14T18:15:19.493 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.449+0000 7fcdf4ea5700 1 -- start start 2023-07-14T18:15:19.493 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.449+0000 7fcdf4ea5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdf0100d90 0x7fcdf0197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:19.494 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.449+0000 7fcdf4ea5700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdf01016a0 0x7fcdf01984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:19.494 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.449+0000 7fcdf4ea5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdf010a2b0 0x7fcdf019c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:19.494 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.449+0000 7fcdf4ea5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf019cf50 con 0x7fcdf0100d90 2023-07-14T18:15:19.494 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.449+0000 7fcdf4ea5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf019d0c0 con 0x7fcdf010a2b0 2023-07-14T18:15:19.494 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.449+0000 7fcdf4ea5700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf019d280 con 0x7fcdf01016a0 2023-07-14T18:15:19.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.450+0000 7fcdedd9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdf01016a0 0x7fcdf01984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:19.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.450+0000 7fcdedd9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdf01016a0 0x7fcdf01984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:47904/0 (socket says 172.21.15.133:47904) 2023-07-14T18:15:19.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.450+0000 7fcdedd9b700 1 -- 172.21.15.133:0/975568751 learned_addr learned my addr 172.21.15.133:0/975568751 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:19.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.450+0000 7fcdee59c700 1 --2- 172.21.15.133:0/975568751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdf0100d90 0x7fcdf0197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:19.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.451+0000 7fcdee59c700 1 -- 172.21.15.133:0/975568751 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdf01016a0 msgr2=0x7fcdf01984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:19.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.451+0000 7fcdee59c700 1 --2- 172.21.15.133:0/975568751 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdf01016a0 0x7fcdf01984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:19.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.451+0000 7fcdee59c700 1 -- 172.21.15.133:0/975568751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdf010a2b0 msgr2=0x7fcdf019c940 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-14T18:15:19.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.451+0000 7fcdee59c700 1 --2- 172.21.15.133:0/975568751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdf010a2b0 0x7fcdf019c940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:19.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.451+0000 7fcdee59c700 1 -- 172.21.15.133:0/975568751 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde40095f0 con 0x7fcdf0100d90 2023-07-14T18:15:19.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.451+0000 7fcdee59c700 1 --2- 172.21.15.133:0/975568751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdf0100d90 0x7fcdf0197f90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcde000e9f0 tx=0x7fcde000edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:19.497 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.452+0000 7fcddf7fe700 1 -- 172.21.15.133:0/975568751 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde000cbd0 con 0x7fcdf0100d90 2023-07-14T18:15:19.497 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.453+0000 7fcddf7fe700 1 -- 172.21.15.133:0/975568751 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcde000cd30 con 0x7fcdf0100d90 2023-07-14T18:15:19.497 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.453+0000 7fcddf7fe700 1 -- 172.21.15.133:0/975568751 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde0018710 con 0x7fcdf0100d90 2023-07-14T18:15:19.497 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.453+0000 7fcdf4ea5700 1 -- 172.21.15.133:0/975568751 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf019d560 con 0x7fcdf0100d90 2023-07-14T18:15:19.498 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.453+0000 7fcdf4ea5700 1 -- 172.21.15.133:0/975568751 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf01a4dc0 con 0x7fcdf0100d90 2023-07-14T18:15:19.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.454+0000 7fcdf4ea5700 1 -- 172.21.15.133:0/975568751 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcdf010fe20 con 0x7fcdf0100d90 2023-07-14T18:15:19.501 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.457+0000 7fcddf7fe700 1 -- 172.21.15.133:0/975568751 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fcde0018870 con 0x7fcdf0100d90 2023-07-14T18:15:19.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.458+0000 7fcddf7fe700 1 --2- 172.21.15.133:0/975568751 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fcdd80a1260 0x7fcdd80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:19.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.458+0000 7fcddf7fe700 1 -- 172.21.15.133:0/975568751 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fcde0014070 con 0x7fcdf0100d90 2023-07-14T18:15:19.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.458+0000 7fcddf7fe700 1 --2- 172.21.15.133:0/975568751 >> [v2:172.21.15.133:6818/2884152358,v1:172.21.15.133:6819/2884152358] conn(0x7fcdd80ac270 0x7fcdd80ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:19.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.458+0000 7fcddf7fe700 1 -- 172.21.15.133:0/975568751 --> [v2:172.21.15.133:6818/2884152358,v1:172.21.15.133:6819/2884152358] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcdd80aedd0 con 0x7fcdd80ac270 2023-07-14T18:15:19.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.458+0000 7fcddf7fe700 1 -- 172.21.15.133:0/975568751 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcde00d55a0 con 0x7fcdf0100d90 2023-07-14T18:15:19.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.459+0000 7fcdedd9b700 1 --2- 172.21.15.133:0/975568751 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fcdd80a1260 0x7fcdd80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:19.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.459+0000 7fcdeed9d700 1 --2- 172.21.15.133:0/975568751 >> [v2:172.21.15.133:6818/2884152358,v1:172.21.15.133:6819/2884152358] conn(0x7fcdd80ac270 0x7fcdd80ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:19.504 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.460+0000 7fcdeed9d700 1 --2- 172.21.15.133:0/975568751 >> [v2:172.21.15.133:6818/2884152358,v1:172.21.15.133:6819/2884152358] conn(0x7fcdd80ac270 0x7fcdd80ae700 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:19.504 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.460+0000 7fcdedd9b700 1 --2- 172.21.15.133:0/975568751 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fcdd80a1260 0x7fcdd80a3710 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcdf01995b0 tx=0x7fcde400b2a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:19.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.461+0000 7fcddf7fe700 1 -- 172.21.15.133:0/975568751 <== osd.6 v2:172.21.15.133:6818/2884152358 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fcdd80aedd0 con 0x7fcdd80ac270 2023-07-14T18:15:19.522 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.478+0000 7fcdf4ea5700 1 -- 172.21.15.133:0/975568751 --> [v2:172.21.15.133:6818/2884152358,v1:172.21.15.133:6819/2884152358] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcdf004e340 con 0x7fcdd80ac270 2023-07-14T18:15:19.522 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.478+0000 7fcddf7fe700 1 -- 172.21.15.133:0/975568751 <== osd.6 v2:172.21.15.133:6818/2884152358 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fcdf004e340 con 0x7fcdd80ac270 2023-07-14T18:15:19.523 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.478+0000 7fcdf4ea5700 1 -- 172.21.15.133:0/975568751 >> [v2:172.21.15.133:6818/2884152358,v1:172.21.15.133:6819/2884152358] conn(0x7fcdd80ac270 msgr2=0x7fcdd80ae700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:19.523 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.478+0000 7fcdf4ea5700 1 --2- 172.21.15.133:0/975568751 >> [v2:172.21.15.133:6818/2884152358,v1:172.21.15.133:6819/2884152358] conn(0x7fcdd80ac270 0x7fcdd80ae700 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:19.523 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.479+0000 7fcdf4ea5700 1 -- 172.21.15.133:0/975568751 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fcdd80a1260 msgr2=0x7fcdd80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:19.523 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.479+0000 7fcdf4ea5700 1 --2- 172.21.15.133:0/975568751 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fcdd80a1260 0x7fcdd80a3710 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcdf01995b0 tx=0x7fcde400b2a0 comp rx=0 tx=0).stop 2023-07-14T18:15:19.523 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.479+0000 7fcdf4ea5700 1 -- 172.21.15.133:0/975568751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdf0100d90 msgr2=0x7fcdf0197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:19.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.479+0000 7fcdf4ea5700 1 --2- 172.21.15.133:0/975568751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdf0100d90 0x7fcdf0197f90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcde000e9f0 tx=0x7fcde000edb0 comp rx=0 tx=0).stop 2023-07-14T18:15:19.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.479+0000 7fcdeed9d700 1 -- 172.21.15.133:0/975568751 reap_dead start 2023-07-14T18:15:19.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.479+0000 7fcdf4ea5700 1 -- 172.21.15.133:0/975568751 shutdown_connections 2023-07-14T18:15:19.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.479+0000 7fcdf4ea5700 1 -- 172.21.15.133:0/975568751 >> 172.21.15.133:0/975568751 conn(0x7fcdf0075ca0 msgr2=0x7fcdf00ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:19.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.479+0000 7fcdf4ea5700 1 -- 172.21.15.133:0/975568751 shutdown_connections 2023-07-14T18:15:19.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:19.479+0000 7fcdf4ea5700 1 -- 172.21.15.133:0/975568751 wait complete. 2023-07-14T18:15:19.760 INFO:teuthology.orchestra.run.smithi133.stdout:339302416408 2023-07-14T18:15:19.761 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.11 2023-07-14T18:15:20.066 INFO:teuthology.orchestra.run.smithi133.stdout:163208757300 2023-07-14T18:15:20.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.021+0000 7ff65bdf5700 1 -- 172.21.15.133:0/1042663061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff654104fd0 msgr2=0x7ff65410cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:20.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.021+0000 7ff65bdf5700 1 --2- 172.21.15.133:0/1042663061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff654104fd0 0x7ff65410cb50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff64c00b0d0 tx=0x7ff64c00b3e0 comp rx=0 tx=0).stop 2023-07-14T18:15:20.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.021+0000 7ff65bdf5700 1 -- 172.21.15.133:0/1042663061 shutdown_connections 2023-07-14T18:15:20.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.021+0000 7ff65bdf5700 1 --2- 172.21.15.133:0/1042663061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff654104fd0 0x7ff65410cb50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:20.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.021+0000 7ff65bdf5700 1 --2- 172.21.15.133:0/1042663061 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6540fff30 0x7ff654100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:20.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.021+0000 7ff65bdf5700 1 --2- 172.21.15.133:0/1042663061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6541008d0 0x7ff6541048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:20.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.021+0000 7ff65bdf5700 1 -- 172.21.15.133:0/1042663061 >> 172.21.15.133:0/1042663061 conn(0x7ff6540fb790 msgr2=0x7ff6540fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:20.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.021+0000 7ff65bdf5700 1 -- 172.21.15.133:0/1042663061 shutdown_connections 2023-07-14T18:15:20.068 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.5 2023-07-14T18:15:20.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.021+0000 7ff65bdf5700 1 -- 172.21.15.133:0/1042663061 wait complete. 2023-07-14T18:15:20.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.022+0000 7ff65bdf5700 1 Processor -- start 2023-07-14T18:15:20.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.022+0000 7ff65bdf5700 1 -- start start 2023-07-14T18:15:20.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.023+0000 7ff65bdf5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6540fff30 0x7ff654072870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:20.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.023+0000 7ff65bdf5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6541008d0 0x7ff654072db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:20.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.023+0000 7ff65bdf5700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff654104fd0 0x7ff654073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:20.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.023+0000 7ff65bdf5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65410b720 con 0x7ff6541008d0 2023-07-14T18:15:20.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.023+0000 7ff65bdf5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65410b890 con 0x7ff6540fff30 2023-07-14T18:15:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.023+0000 7ff65bdf5700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65410ba50 con 0x7ff654104fd0 2023-07-14T18:15:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.023+0000 7ff659b91700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6540fff30 0x7ff654072870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.023+0000 7ff659b91700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6540fff30 0x7ff654072870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:38618/0 (socket says 172.21.15.133:38618) 2023-07-14T18:15:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.023+0000 7ff659b91700 1 -- 172.21.15.133:0/745856909 learned_addr learned my addr 172.21.15.133:0/745856909 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.023+0000 7ff659b91700 1 -- 172.21.15.133:0/745856909 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff654104fd0 msgr2=0x7ff654073330 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-14T18:15:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.023+0000 7ff659b91700 1 --2- 172.21.15.133:0/745856909 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff654104fd0 0x7ff654073330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.023+0000 7ff659b91700 1 -- 172.21.15.133:0/745856909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6541008d0 msgr2=0x7ff654072db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.023+0000 7ff659b91700 1 --2- 172.21.15.133:0/745856909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6541008d0 0x7ff654072db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.023+0000 7ff659b91700 1 -- 172.21.15.133:0/745856909 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64c009d00 con 0x7ff6540fff30 2023-07-14T18:15:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.024+0000 7ff659b91700 1 --2- 172.21.15.133:0/745856909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6540fff30 0x7ff654072870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff65000d950 tx=0x7ff65000dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.024+0000 7ff64affd700 1 -- 172.21.15.133:0/745856909 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff650009b40 con 0x7ff6540fff30 2023-07-14T18:15:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.024+0000 7ff64affd700 1 -- 172.21.15.133:0/745856909 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff650009ca0 con 0x7ff6540fff30 2023-07-14T18:15:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.024+0000 7ff65bdf5700 1 -- 172.21.15.133:0/745856909 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff65410bd30 con 0x7ff6540fff30 2023-07-14T18:15:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.024+0000 7ff64affd700 1 -- 172.21.15.133:0/745856909 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65000f620 con 0x7ff6540fff30 2023-07-14T18:15:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.024+0000 7ff65bdf5700 1 -- 172.21.15.133:0/745856909 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff65410c230 con 0x7ff6540fff30 2023-07-14T18:15:20.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.025+0000 7ff65bdf5700 1 -- 172.21.15.133:0/745856909 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff65406bcd0 con 0x7ff6540fff30 2023-07-14T18:15:20.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.028+0000 7ff64affd700 1 -- 172.21.15.133:0/745856909 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7ff650010460 con 0x7ff6540fff30 2023-07-14T18:15:20.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.030+0000 7ff64affd700 1 --2- 172.21.15.133:0/745856909 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff6400a11e0 0x7ff6400a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:20.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.031+0000 7ff64affd700 1 -- 172.21.15.133:0/745856909 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff6500ccde0 con 0x7ff6540fff30 2023-07-14T18:15:20.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.031+0000 7ff64affd700 1 --2- 172.21.15.133:0/745856909 >> [v2:172.21.15.161:6824/2162333626,v1:172.21.15.161:6825/2162333626] conn(0x7ff6400ac1f0 0x7ff6400ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:20.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.031+0000 7ff659390700 1 --2- 172.21.15.133:0/745856909 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff6400a11e0 0x7ff6400a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:20.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.031+0000 7ff64affd700 1 -- 172.21.15.133:0/745856909 --> [v2:172.21.15.161:6824/2162333626,v1:172.21.15.161:6825/2162333626] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff6400aed50 con 0x7ff6400ac1f0 2023-07-14T18:15:20.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.031+0000 7ff64affd700 1 -- 172.21.15.133:0/745856909 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6500cf3c0 con 0x7ff6540fff30 2023-07-14T18:15:20.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.032+0000 7ff659390700 1 --2- 172.21.15.133:0/745856909 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff6400a11e0 0x7ff6400a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff654074010 tx=0x7ff644006cd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:20.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.035+0000 7ff65a392700 1 --2- 172.21.15.133:0/745856909 >> [v2:172.21.15.161:6824/2162333626,v1:172.21.15.161:6825/2162333626] conn(0x7ff6400ac1f0 0x7ff6400ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:20.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.036+0000 7ff65a392700 1 --2- 172.21.15.133:0/745856909 >> [v2:172.21.15.161:6824/2162333626,v1:172.21.15.161:6825/2162333626] conn(0x7ff6400ac1f0 0x7ff6400ae680 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:20.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.037+0000 7ff64affd700 1 -- 172.21.15.133:0/745856909 <== osd.10 v2:172.21.15.161:6824/2162333626 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff6400aed50 con 0x7ff6400ac1f0 2023-07-14T18:15:20.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.054+0000 7ff65bdf5700 1 -- 172.21.15.133:0/745856909 --> [v2:172.21.15.161:6824/2162333626,v1:172.21.15.161:6825/2162333626] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff65404e340 con 0x7ff6400ac1f0 2023-07-14T18:15:20.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.055+0000 7ff64affd700 1 -- 172.21.15.133:0/745856909 <== osd.10 v2:172.21.15.161:6824/2162333626 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff65404e340 con 0x7ff6400ac1f0 2023-07-14T18:15:20.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.055+0000 7ff65bdf5700 1 -- 172.21.15.133:0/745856909 >> [v2:172.21.15.161:6824/2162333626,v1:172.21.15.161:6825/2162333626] conn(0x7ff6400ac1f0 msgr2=0x7ff6400ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:20.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.055+0000 7ff65bdf5700 1 --2- 172.21.15.133:0/745856909 >> [v2:172.21.15.161:6824/2162333626,v1:172.21.15.161:6825/2162333626] conn(0x7ff6400ac1f0 0x7ff6400ae680 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:20.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.056+0000 7ff65bdf5700 1 -- 172.21.15.133:0/745856909 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff6400a11e0 msgr2=0x7ff6400a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:20.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.056+0000 7ff65bdf5700 1 --2- 172.21.15.133:0/745856909 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff6400a11e0 0x7ff6400a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff654074010 tx=0x7ff644006cd0 comp rx=0 tx=0).stop 2023-07-14T18:15:20.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.056+0000 7ff65bdf5700 1 -- 172.21.15.133:0/745856909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6540fff30 msgr2=0x7ff654072870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:20.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.056+0000 7ff65bdf5700 1 --2- 172.21.15.133:0/745856909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6540fff30 0x7ff654072870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff65000d950 tx=0x7ff65000dd10 comp rx=0 tx=0).stop 2023-07-14T18:15:20.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.056+0000 7ff65a392700 1 -- 172.21.15.133:0/745856909 reap_dead start 2023-07-14T18:15:20.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.056+0000 7ff65bdf5700 1 -- 172.21.15.133:0/745856909 shutdown_connections 2023-07-14T18:15:20.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.056+0000 7ff65bdf5700 1 -- 172.21.15.133:0/745856909 >> 172.21.15.133:0/745856909 conn(0x7ff6540fb790 msgr2=0x7ff6540fc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:20.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.056+0000 7ff65bdf5700 1 -- 172.21.15.133:0/745856909 shutdown_connections 2023-07-14T18:15:20.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.056+0000 7ff65bdf5700 1 -- 172.21.15.133:0/745856909 wait complete. 2023-07-14T18:15:20.247 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:19 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:15:20.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:19 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:15:20.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:19 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:15:20.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.460+0000 7f078a641700 1 -- 172.21.15.133:0/539308883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0784104fd0 msgr2=0x7f078410cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:20.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.460+0000 7f078a641700 1 --2- 172.21.15.133:0/539308883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0784104fd0 0x7f078410cb50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f077c00b0d0 tx=0x7f077c00b3e0 comp rx=0 tx=0).stop 2023-07-14T18:15:20.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.461+0000 7f078a641700 1 -- 172.21.15.133:0/539308883 shutdown_connections 2023-07-14T18:15:20.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.461+0000 7f078a641700 1 --2- 172.21.15.133:0/539308883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07840fff30 0x7f0784100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:20.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.461+0000 7f078a641700 1 --2- 172.21.15.133:0/539308883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0784104fd0 0x7f078410cb50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:20.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.461+0000 7f078a641700 1 --2- 172.21.15.133:0/539308883 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07841008d0 0x7f07841048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:20.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.461+0000 7f078a641700 1 -- 172.21.15.133:0/539308883 >> 172.21.15.133:0/539308883 conn(0x7f07840fb790 msgr2=0x7f07840fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:20.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.461+0000 7f078a641700 1 -- 172.21.15.133:0/539308883 shutdown_connections 2023-07-14T18:15:20.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.461+0000 7f078a641700 1 -- 172.21.15.133:0/539308883 wait complete. 2023-07-14T18:15:20.510 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.462+0000 7f078a641700 1 Processor -- start 2023-07-14T18:15:20.510 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.462+0000 7f078a641700 1 -- start start 2023-07-14T18:15:20.510 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.462+0000 7f078a641700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07840fff30 0x7f0784195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:20.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.462+0000 7f078a641700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07841008d0 0x7f0784196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:20.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.462+0000 7f078a641700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0784104fd0 0x7f078419a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:20.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.462+0000 7f078a641700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078419aeb0 con 0x7f07840fff30 2023-07-14T18:15:20.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.462+0000 7f078a641700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078419b020 con 0x7f0784104fd0 2023-07-14T18:15:20.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.462+0000 7f078a641700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078419b1e0 con 0x7f07841008d0 2023-07-14T18:15:20.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.463+0000 7f0788bde700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0784104fd0 0x7f078419a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:20.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.463+0000 7f07837fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07841008d0 0x7f0784196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:20.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.463+0000 7f0783fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07840fff30 0x7f0784195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:20.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.463+0000 7f07837fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07841008d0 0x7f0784196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:47940/0 (socket says 172.21.15.133:47940) 2023-07-14T18:15:20.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.463+0000 7f0788bde700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0784104fd0 0x7f078419a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:38626/0 (socket says 172.21.15.133:38626) 2023-07-14T18:15:20.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.463+0000 7f07837fe700 1 -- 172.21.15.133:0/75253221 learned_addr learned my addr 172.21.15.133:0/75253221 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:20.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.463+0000 7f0788bde700 1 -- 172.21.15.133:0/75253221 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07841008d0 msgr2=0x7f0784196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:20.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.463+0000 7f0788bde700 1 --2- 172.21.15.133:0/75253221 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07841008d0 0x7f0784196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:20.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.463+0000 7f0788bde700 1 -- 172.21.15.133:0/75253221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07840fff30 msgr2=0x7f0784195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:20.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.463+0000 7f0788bde700 1 --2- 172.21.15.133:0/75253221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07840fff30 0x7f0784195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:20.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.463+0000 7f0788bde700 1 -- 172.21.15.133:0/75253221 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c009d00 con 0x7f0784104fd0 2023-07-14T18:15:20.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.463+0000 7f07837fe700 1 --2- 172.21.15.133:0/75253221 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07841008d0 0x7f0784196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:20.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.463+0000 7f0783fff700 1 --2- 172.21.15.133:0/75253221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07840fff30 0x7f0784195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:20.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.463+0000 7f0788bde700 1 --2- 172.21.15.133:0/75253221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0784104fd0 0x7f078419a8a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f077c00b770 tx=0x7f077c007870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:20.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.463+0000 7f07817fa700 1 -- 172.21.15.133:0/75253221 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c00e040 con 0x7f0784104fd0 2023-07-14T18:15:20.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.464+0000 7f078a641700 1 -- 172.21.15.133:0/75253221 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078419b460 con 0x7f0784104fd0 2023-07-14T18:15:20.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.464+0000 7f07817fa700 1 -- 172.21.15.133:0/75253221 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f077c01d930 con 0x7f0784104fd0 2023-07-14T18:15:20.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.464+0000 7f07817fa700 1 -- 172.21.15.133:0/75253221 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c01b820 con 0x7f0784104fd0 2023-07-14T18:15:20.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.464+0000 7f078a641700 1 -- 172.21.15.133:0/75253221 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07841a2d20 con 0x7f0784104fd0 2023-07-14T18:15:20.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.464+0000 7f078a641700 1 -- 172.21.15.133:0/75253221 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f078410fe30 con 0x7f0784104fd0 2023-07-14T18:15:20.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.467+0000 7f07817fa700 1 -- 172.21.15.133:0/75253221 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f077c01d470 con 0x7f0784104fd0 2023-07-14T18:15:20.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.468+0000 7f07817fa700 1 --2- 172.21.15.133:0/75253221 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f076c0a11e0 0x7f076c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:20.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.468+0000 7f07817fa700 1 -- 172.21.15.133:0/75253221 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f077c0ce830 con 0x7f0784104fd0 2023-07-14T18:15:20.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.468+0000 7f0783fff700 1 --2- 172.21.15.133:0/75253221 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f076c0a11e0 0x7f076c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:20.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.468+0000 7f07817fa700 1 --2- 172.21.15.133:0/75253221 >> [v2:172.21.15.161:6800/1841901852,v1:172.21.15.161:6801/1841901852] conn(0x7f076c0ac1f0 0x7f076c0ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:20.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.468+0000 7f07817fa700 1 -- 172.21.15.133:0/75253221 --> [v2:172.21.15.161:6800/1841901852,v1:172.21.15.161:6801/1841901852] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f076c0aed50 con 0x7f076c0ac1f0 2023-07-14T18:15:20.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.469+0000 7f07817fa700 1 -- 172.21.15.133:0/75253221 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f077c0ceba0 con 0x7f0784104fd0 2023-07-14T18:15:20.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.469+0000 7f07837fe700 1 --2- 172.21.15.133:0/75253221 >> [v2:172.21.15.161:6800/1841901852,v1:172.21.15.161:6801/1841901852] conn(0x7f076c0ac1f0 0x7f076c0ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:20.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.469+0000 7f0783fff700 1 --2- 172.21.15.133:0/75253221 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f076c0a11e0 0x7f076c0a3690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0774011490 tx=0x7f0774009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:20.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.469+0000 7f07837fe700 1 --2- 172.21.15.133:0/75253221 >> [v2:172.21.15.161:6800/1841901852,v1:172.21.15.161:6801/1841901852] conn(0x7f076c0ac1f0 0x7f076c0ae680 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:20.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.470+0000 7f07817fa700 1 -- 172.21.15.133:0/75253221 <== osd.1 v2:172.21.15.161:6800/1841901852 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f076c0aed50 con 0x7f076c0ac1f0 2023-07-14T18:15:20.530 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.486+0000 7f078a641700 1 -- 172.21.15.133:0/75253221 --> [v2:172.21.15.161:6800/1841901852,v1:172.21.15.161:6801/1841901852] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f078404e340 con 0x7f076c0ac1f0 2023-07-14T18:15:20.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.487+0000 7f07817fa700 1 -- 172.21.15.133:0/75253221 <== osd.1 v2:172.21.15.161:6800/1841901852 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f078404e340 con 0x7f076c0ac1f0 2023-07-14T18:15:20.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.487+0000 7f078a641700 1 -- 172.21.15.133:0/75253221 >> [v2:172.21.15.161:6800/1841901852,v1:172.21.15.161:6801/1841901852] conn(0x7f076c0ac1f0 msgr2=0x7f076c0ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:20.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.487+0000 7f078a641700 1 --2- 172.21.15.133:0/75253221 >> [v2:172.21.15.161:6800/1841901852,v1:172.21.15.161:6801/1841901852] conn(0x7f076c0ac1f0 0x7f076c0ae680 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:20.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.487+0000 7f078a641700 1 -- 172.21.15.133:0/75253221 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f076c0a11e0 msgr2=0x7f076c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:20.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.487+0000 7f078a641700 1 --2- 172.21.15.133:0/75253221 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f076c0a11e0 0x7f076c0a3690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0774011490 tx=0x7f0774009260 comp rx=0 tx=0).stop 2023-07-14T18:15:20.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.488+0000 7f078a641700 1 -- 172.21.15.133:0/75253221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0784104fd0 msgr2=0x7f078419a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:20.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.488+0000 7f078a641700 1 --2- 172.21.15.133:0/75253221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0784104fd0 0x7f078419a8a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f077c00b770 tx=0x7f077c007870 comp rx=0 tx=0).stop 2023-07-14T18:15:20.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.488+0000 7f0788bde700 1 -- 172.21.15.133:0/75253221 reap_dead start 2023-07-14T18:15:20.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.488+0000 7f078a641700 1 -- 172.21.15.133:0/75253221 shutdown_connections 2023-07-14T18:15:20.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.488+0000 7f078a641700 1 -- 172.21.15.133:0/75253221 >> 172.21.15.133:0/75253221 conn(0x7f07840fb790 msgr2=0x7f07840fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:20.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.488+0000 7f078a641700 1 -- 172.21.15.133:0/75253221 shutdown_connections 2023-07-14T18:15:20.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:20.488+0000 7f078a641700 1 -- 172.21.15.133:0/75253221 wait complete. 2023-07-14T18:15:21.019 INFO:teuthology.orchestra.run.smithi133.stdout:287762808866 2023-07-14T18:15:21.019 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.9 2023-07-14T18:15:21.044 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:21.046 INFO:teuthology.orchestra.run.smithi133.stdout:197568495662 2023-07-14T18:15:21.046 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.6 2023-07-14T18:15:21.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:20 smithi161 ceph-mon[128993]: pgmap v306: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2023-07-14T18:15:21.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:20 smithi171 ceph-mon[129241]: pgmap v306: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2023-07-14T18:15:21.386 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:20 smithi133 ceph-mon[126384]: pgmap v306: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2023-07-14T18:15:21.454 INFO:teuthology.orchestra.run.smithi133.stdout:55834574917 2023-07-14T18:15:21.454 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.1 2023-07-14T18:15:21.769 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:21.801 INFO:teuthology.orchestra.run.smithi133.stdout:309237645341 2023-07-14T18:15:21.801 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.10 2023-07-14T18:15:22.213 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:22.421 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:22.725 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:23.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:22 smithi133 ceph-mon[126384]: pgmap v307: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-14T18:15:23.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:22 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:15:23.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:22 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:15:23.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:22 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:15:23.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:22 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:15:23.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:22 smithi161 ceph-mon[128993]: pgmap v307: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-14T18:15:23.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:22 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:15:23.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:22 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:15:23.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:22 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:15:23.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:22 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:15:23.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:22 smithi171 ceph-mon[129241]: pgmap v307: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-14T18:15:23.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:22 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:15:23.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:22 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:15:23.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:22 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:15:23.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:22 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:15:23.512 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:23.798 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:24.036 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:24.854 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:25.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:24 smithi133 ceph-mon[126384]: pgmap v308: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2023-07-14T18:15:25.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:24 smithi161 ceph-mon[128993]: pgmap v308: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2023-07-14T18:15:25.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:24 smithi171 ceph-mon[129241]: pgmap v308: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2023-07-14T18:15:26.202 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:26.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.692+0000 7fd7ab849700 1 -- 172.21.15.133:0/2179283238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7a40ff3c0 msgr2=0x7fd7a410c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:26.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.692+0000 7fd7ab849700 1 --2- 172.21.15.133:0/2179283238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7a40ff3c0 0x7fd7a410c120 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd794009990 tx=0x7fd794009ca0 comp rx=0 tx=0).stop 2023-07-14T18:15:26.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.692+0000 7fd7ab849700 1 -- 172.21.15.133:0/2179283238 shutdown_connections 2023-07-14T18:15:26.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.692+0000 7fd7ab849700 1 --2- 172.21.15.133:0/2179283238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7a410c660 0x7fd7a410ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:26.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.692+0000 7fd7ab849700 1 --2- 172.21.15.133:0/2179283238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7a40feab0 0x7fd7a40fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:26.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.692+0000 7fd7ab849700 1 --2- 172.21.15.133:0/2179283238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7a40ff3c0 0x7fd7a410c120 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:26.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.692+0000 7fd7ab849700 1 -- 172.21.15.133:0/2179283238 >> 172.21.15.133:0/2179283238 conn(0x7fd7a4075ca0 msgr2=0x7fd7a40760a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:26.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.692+0000 7fd7ab849700 1 -- 172.21.15.133:0/2179283238 shutdown_connections 2023-07-14T18:15:26.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.693+0000 7fd7ab849700 1 -- 172.21.15.133:0/2179283238 wait complete. 2023-07-14T18:15:26.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.693+0000 7fd7ab849700 1 Processor -- start 2023-07-14T18:15:26.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.694+0000 7fd7ab849700 1 -- start start 2023-07-14T18:15:26.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.694+0000 7fd7ab849700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7a40feab0 0x7fd7a4198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:26.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.694+0000 7fd7ab849700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7a40ff3c0 0x7fd7a4198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:26.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.694+0000 7fd7ab849700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7a410c660 0x7fd7a419c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:26.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.694+0000 7fd7ab849700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a419cfe0 con 0x7fd7a410c660 2023-07-14T18:15:26.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.694+0000 7fd7ab849700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a419d150 con 0x7fd7a40feab0 2023-07-14T18:15:26.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.694+0000 7fd7ab849700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a419d310 con 0x7fd7a40ff3c0 2023-07-14T18:15:26.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.694+0000 7fd7a8de4700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7a40ff3c0 0x7fd7a4198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:26.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.694+0000 7fd7a95e5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7a40feab0 0x7fd7a4198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:26.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.694+0000 7fd7a8de4700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7a40ff3c0 0x7fd7a4198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:47966/0 (socket says 172.21.15.133:47966) 2023-07-14T18:15:26.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.694+0000 7fd7a8de4700 1 -- 172.21.15.133:0/1389791415 learned_addr learned my addr 172.21.15.133:0/1389791415 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:26.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.694+0000 7fd7a9de6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7a410c660 0x7fd7a419c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:26.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.695+0000 7fd7a8de4700 1 -- 172.21.15.133:0/1389791415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7a40feab0 msgr2=0x7fd7a4198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:26.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.695+0000 7fd7a8de4700 1 --2- 172.21.15.133:0/1389791415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7a40feab0 0x7fd7a4198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:26.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.695+0000 7fd7a8de4700 1 -- 172.21.15.133:0/1389791415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7a410c660 msgr2=0x7fd7a419c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:26.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.695+0000 7fd7a8de4700 1 --2- 172.21.15.133:0/1389791415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7a410c660 0x7fd7a419c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:26.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.695+0000 7fd7a8de4700 1 -- 172.21.15.133:0/1389791415 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7940095f0 con 0x7fd7a40ff3c0 2023-07-14T18:15:26.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.695+0000 7fd7a9de6700 1 --2- 172.21.15.133:0/1389791415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7a410c660 0x7fd7a419c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:26.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.695+0000 7fd7a8de4700 1 --2- 172.21.15.133:0/1389791415 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7a40ff3c0 0x7fd7a4198560 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd794000c00 tx=0x7fd79400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:26.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.695+0000 7fd79a7fc700 1 -- 172.21.15.133:0/1389791415 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79401d070 con 0x7fd7a40ff3c0 2023-07-14T18:15:26.743 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.695+0000 7fd7ab849700 1 -- 172.21.15.133:0/1389791415 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a419d590 con 0x7fd7a40ff3c0 2023-07-14T18:15:26.743 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.695+0000 7fd79a7fc700 1 -- 172.21.15.133:0/1389791415 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd79400be40 con 0x7fd7a40ff3c0 2023-07-14T18:15:26.743 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.695+0000 7fd79a7fc700 1 -- 172.21.15.133:0/1389791415 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7940173f0 con 0x7fd7a40ff3c0 2023-07-14T18:15:26.743 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.696+0000 7fd7ab849700 1 -- 172.21.15.133:0/1389791415 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a41a4e50 con 0x7fd7a40ff3c0 2023-07-14T18:15:26.743 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.696+0000 7fd7ab849700 1 -- 172.21.15.133:0/1389791415 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7a404e340 con 0x7fd7a40ff3c0 2023-07-14T18:15:26.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.699+0000 7fd79a7fc700 1 -- 172.21.15.133:0/1389791415 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fd794022470 con 0x7fd7a40ff3c0 2023-07-14T18:15:26.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.700+0000 7fd79a7fc700 1 --2- 172.21.15.133:0/1389791415 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd7900a11e0 0x7fd7900a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:26.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.700+0000 7fd79a7fc700 1 -- 172.21.15.133:0/1389791415 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd7940ce550 con 0x7fd7a40ff3c0 2023-07-14T18:15:26.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.701+0000 7fd7a95e5700 1 --2- 172.21.15.133:0/1389791415 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd7900a11e0 0x7fd7900a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:26.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.701+0000 7fd7a95e5700 1 --2- 172.21.15.133:0/1389791415 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd7900a11e0 0x7fd7900a3690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0009c60 tx=0x7fd7a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:26.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.702+0000 7fd79a7fc700 1 -- 172.21.15.133:0/1389791415 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fd794097860 con 0x7fd7a40ff3c0 2023-07-14T18:15:26.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.908+0000 7fd7ab849700 1 -- 172.21.15.133:0/1389791415 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd7a41a4fe0 con 0x7fd7a40ff3c0 2023-07-14T18:15:26.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.908+0000 7fd79a7fc700 1 -- 172.21.15.133:0/1389791415 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd79409ae80 con 0x7fd7a40ff3c0 2023-07-14T18:15:26.953 INFO:teuthology.orchestra.run.smithi133.stdout:77309411394 2023-07-14T18:15:26.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.911+0000 7fd7ab849700 1 -- 172.21.15.133:0/1389791415 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd7900a11e0 msgr2=0x7fd7900a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:26.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.911+0000 7fd7ab849700 1 --2- 172.21.15.133:0/1389791415 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd7900a11e0 0x7fd7900a3690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0009c60 tx=0x7fd7a0009260 comp rx=0 tx=0).stop 2023-07-14T18:15:26.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.911+0000 7fd7ab849700 1 -- 172.21.15.133:0/1389791415 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7a40ff3c0 msgr2=0x7fd7a4198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:26.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.911+0000 7fd7ab849700 1 --2- 172.21.15.133:0/1389791415 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7a40ff3c0 0x7fd7a4198560 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd794000c00 tx=0x7fd79400f690 comp rx=0 tx=0).stop 2023-07-14T18:15:26.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.911+0000 7fd7ab849700 1 -- 172.21.15.133:0/1389791415 shutdown_connections 2023-07-14T18:15:26.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.911+0000 7fd7ab849700 1 --2- 172.21.15.133:0/1389791415 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd7900a11e0 0x7fd7900a3690 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:26.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.911+0000 7fd7ab849700 1 --2- 172.21.15.133:0/1389791415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7a40feab0 0x7fd7a4198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:26.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.911+0000 7fd7ab849700 1 --2- 172.21.15.133:0/1389791415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7a410c660 0x7fd7a419c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:26.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.911+0000 7fd7ab849700 1 --2- 172.21.15.133:0/1389791415 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7a40ff3c0 0x7fd7a4198560 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:26.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.911+0000 7fd7ab849700 1 -- 172.21.15.133:0/1389791415 >> 172.21.15.133:0/1389791415 conn(0x7fd7a4075ca0 msgr2=0x7fd7a40fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:26.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.911+0000 7fd7ab849700 1 -- 172.21.15.133:0/1389791415 shutdown_connections 2023-07-14T18:15:26.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:26.911+0000 7fd7ab849700 1 -- 172.21.15.133:0/1389791415 wait complete. 2023-07-14T18:15:27.200 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:26 smithi133 ceph-mon[126384]: pgmap v309: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-14T18:15:27.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:26 smithi161 ceph-mon[128993]: pgmap v309: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-14T18:15:27.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:26 smithi171 ceph-mon[129241]: pgmap v309: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-14T18:15:27.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.537+0000 7f3fb9ab6700 1 -- 172.21.15.133:0/4256644542 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fb40fff30 msgr2=0x7f3fb4100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:27.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.537+0000 7f3fb9ab6700 1 --2- 172.21.15.133:0/4256644542 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fb40fff30 0x7f3fb4100300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40099c0 tx=0x7f3fa4009cd0 comp rx=0 tx=0).stop 2023-07-14T18:15:27.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.537+0000 7f3fb9ab6700 1 -- 172.21.15.133:0/4256644542 shutdown_connections 2023-07-14T18:15:27.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.537+0000 7f3fb9ab6700 1 --2- 172.21.15.133:0/4256644542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fb4104fd0 0x7f3fb410cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:27.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.537+0000 7f3fb9ab6700 1 --2- 172.21.15.133:0/4256644542 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fb40fff30 0x7f3fb4100300 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:27.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.537+0000 7f3fb9ab6700 1 --2- 172.21.15.133:0/4256644542 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3fb41008d0 0x7f3fb41048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:27.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.537+0000 7f3fb9ab6700 1 -- 172.21.15.133:0/4256644542 >> 172.21.15.133:0/4256644542 conn(0x7f3fb40fb790 msgr2=0x7f3fb40fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:27.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.537+0000 7f3fb9ab6700 1 -- 172.21.15.133:0/4256644542 shutdown_connections 2023-07-14T18:15:27.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.537+0000 7f3fb9ab6700 1 -- 172.21.15.133:0/4256644542 wait complete. 2023-07-14T18:15:27.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.538+0000 7f3fb9ab6700 1 Processor -- start 2023-07-14T18:15:27.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.538+0000 7f3fb9ab6700 1 -- start start 2023-07-14T18:15:27.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.539+0000 7f3fb9ab6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fb40fff30 0x7f3fb4195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:27.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.539+0000 7f3fb9ab6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fb41008d0 0x7f3fb4196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:27.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.539+0000 7f3fb9ab6700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3fb4104fd0 0x7f3fb419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:27.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.539+0000 7f3fb9ab6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb419adc0 con 0x7f3fb41008d0 2023-07-14T18:15:27.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.539+0000 7f3fb9ab6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb419af30 con 0x7f3fb40fff30 2023-07-14T18:15:27.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.539+0000 7f3fb9ab6700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb419b0f0 con 0x7f3fb4104fd0 2023-07-14T18:15:27.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.539+0000 7f3fb2ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fb41008d0 0x7f3fb4196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:27.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.539+0000 7f3fb37fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fb40fff30 0x7f3fb4195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:27.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.539+0000 7f3fb2ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fb41008d0 0x7f3fb4196340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51770/0 (socket says 172.21.15.133:51770) 2023-07-14T18:15:27.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.539+0000 7f3fb2ffd700 1 -- 172.21.15.133:0/2777898555 learned_addr learned my addr 172.21.15.133:0/2777898555 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:27.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.539+0000 7f3fb3fff700 1 --2- 172.21.15.133:0/2777898555 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3fb4104fd0 0x7f3fb419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:27.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.539+0000 7f3fb37fe700 1 -- 172.21.15.133:0/2777898555 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3fb4104fd0 msgr2=0x7f3fb419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:27.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.539+0000 7f3fb37fe700 1 --2- 172.21.15.133:0/2777898555 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3fb4104fd0 0x7f3fb419a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:27.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.540+0000 7f3fb37fe700 1 -- 172.21.15.133:0/2777898555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fb41008d0 msgr2=0x7f3fb4196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:27.587 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.540+0000 7f3fb37fe700 1 --2- 172.21.15.133:0/2777898555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fb41008d0 0x7f3fb4196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:27.587 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.540+0000 7f3fb37fe700 1 -- 172.21.15.133:0/2777898555 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa40095f0 con 0x7f3fb40fff30 2023-07-14T18:15:27.587 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.540+0000 7f3fb37fe700 1 --2- 172.21.15.133:0/2777898555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fb40fff30 0x7f3fb4195e00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400b580 tx=0x7f3fa400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:27.587 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.540+0000 7f3fb0ff9700 1 -- 172.21.15.133:0/2777898555 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa401d070 con 0x7f3fb40fff30 2023-07-14T18:15:27.587 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.540+0000 7f3fb0ff9700 1 -- 172.21.15.133:0/2777898555 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fa4005020 con 0x7f3fb40fff30 2023-07-14T18:15:27.587 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.540+0000 7f3fb9ab6700 1 -- 172.21.15.133:0/2777898555 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fb419b370 con 0x7f3fb40fff30 2023-07-14T18:15:27.587 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.540+0000 7f3fb0ff9700 1 -- 172.21.15.133:0/2777898555 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4004a90 con 0x7f3fb40fff30 2023-07-14T18:15:27.588 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.540+0000 7f3fb9ab6700 1 -- 172.21.15.133:0/2777898555 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb41a2a60 con 0x7f3fb40fff30 2023-07-14T18:15:27.588 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.541+0000 7f3fb9ab6700 1 -- 172.21.15.133:0/2777898555 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fb404e340 con 0x7f3fb40fff30 2023-07-14T18:15:27.588 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.543+0000 7f3fb0ff9700 1 -- 172.21.15.133:0/2777898555 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f3fa4005400 con 0x7f3fb40fff30 2023-07-14T18:15:27.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.545+0000 7f3fb0ff9700 1 --2- 172.21.15.133:0/2777898555 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3f9c0a11e0 0x7f3f9c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:27.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.545+0000 7f3fb0ff9700 1 -- 172.21.15.133:0/2777898555 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3fa40cdce0 con 0x7f3fb40fff30 2023-07-14T18:15:27.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.545+0000 7f3fb2ffd700 1 --2- 172.21.15.133:0/2777898555 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3f9c0a11e0 0x7f3f9c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:27.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.545+0000 7f3fb2ffd700 1 --2- 172.21.15.133:0/2777898555 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3f9c0a11e0 0x7f3f9c0a3690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4197420 tx=0x7f3fa800b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:27.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.547+0000 7f3fb0ff9700 1 -- 172.21.15.133:0/2777898555 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f3fa4096ff0 con 0x7f3fb40fff30 2023-07-14T18:15:27.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.788+0000 7f3fb9ab6700 1 -- 172.21.15.133:0/2777898555 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f3fb4197160 con 0x7f3fb40fff30 2023-07-14T18:15:27.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.789+0000 7f3fb0ff9700 1 -- 172.21.15.133:0/2777898555 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3fa4005190 con 0x7f3fb40fff30 2023-07-14T18:15:27.834 INFO:teuthology.orchestra.run.smithi133.stdout:219043332139 2023-07-14T18:15:27.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.791+0000 7f3fb9ab6700 1 -- 172.21.15.133:0/2777898555 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3f9c0a11e0 msgr2=0x7f3f9c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:27.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.791+0000 7f3fb9ab6700 1 --2- 172.21.15.133:0/2777898555 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3f9c0a11e0 0x7f3f9c0a3690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4197420 tx=0x7f3fa800b480 comp rx=0 tx=0).stop 2023-07-14T18:15:27.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.791+0000 7f3fb9ab6700 1 -- 172.21.15.133:0/2777898555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fb40fff30 msgr2=0x7f3fb4195e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:27.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.791+0000 7f3fb9ab6700 1 --2- 172.21.15.133:0/2777898555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fb40fff30 0x7f3fb4195e00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400b580 tx=0x7f3fa400f690 comp rx=0 tx=0).stop 2023-07-14T18:15:27.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.792+0000 7f3fb9ab6700 1 -- 172.21.15.133:0/2777898555 shutdown_connections 2023-07-14T18:15:27.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.792+0000 7f3fb9ab6700 1 --2- 172.21.15.133:0/2777898555 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3f9c0a11e0 0x7f3f9c0a3690 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:27.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.792+0000 7f3fb9ab6700 1 --2- 172.21.15.133:0/2777898555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fb40fff30 0x7f3fb4195e00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:27.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.792+0000 7f3fb9ab6700 1 --2- 172.21.15.133:0/2777898555 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3fb4104fd0 0x7f3fb419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:27.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.792+0000 7f3fb9ab6700 1 --2- 172.21.15.133:0/2777898555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fb41008d0 0x7f3fb4196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:27.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.792+0000 7f3fb9ab6700 1 -- 172.21.15.133:0/2777898555 >> 172.21.15.133:0/2777898555 conn(0x7f3fb40fb790 msgr2=0x7f3fb40fc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:27.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.792+0000 7f3fb9ab6700 1 -- 172.21.15.133:0/2777898555 shutdown_connections 2023-07-14T18:15:27.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:27.792+0000 7f3fb9ab6700 1 -- 172.21.15.133:0/2777898555 wait complete. 2023-07-14T18:15:28.083 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:27 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1389791415' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-14T18:15:28.083 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:27 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2777898555' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-14T18:15:28.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.017+0000 7fce2bed1700 1 -- 172.21.15.133:0/1159461530 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce2410e8d0 msgr2=0x7fce2406d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:28.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.017+0000 7fce2bed1700 1 --2- 172.21.15.133:0/1159461530 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce2410e8d0 0x7fce2406d320 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fce14009990 tx=0x7fce14009ca0 comp rx=0 tx=0).stop 2023-07-14T18:15:28.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.017+0000 7fce2bed1700 1 -- 172.21.15.133:0/1159461530 shutdown_connections 2023-07-14T18:15:28.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.017+0000 7fce2bed1700 1 --2- 172.21.15.133:0/1159461530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce2410e470 0x7fce2406cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:28.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.017+0000 7fce2bed1700 1 --2- 172.21.15.133:0/1159461530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce2406d9b0 0x7fce2411b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:28.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.017+0000 7fce2bed1700 1 --2- 172.21.15.133:0/1159461530 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce2410e8d0 0x7fce2406d320 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:28.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.017+0000 7fce2bed1700 1 -- 172.21.15.133:0/1159461530 >> 172.21.15.133:0/1159461530 conn(0x7fce2406bf70 msgr2=0x7fce24071150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:28.122 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.017+0000 7fce2bed1700 1 -- 172.21.15.133:0/1159461530 shutdown_connections 2023-07-14T18:15:28.122 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.017+0000 7fce2bed1700 1 -- 172.21.15.133:0/1159461530 wait complete. 2023-07-14T18:15:28.122 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.018+0000 7fce2bed1700 1 Processor -- start 2023-07-14T18:15:28.122 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.018+0000 7fce2bed1700 1 -- start start 2023-07-14T18:15:28.122 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.019+0000 7fce2bed1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce2406d9b0 0x7fce241aa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:28.122 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.019+0000 7fce2bed1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce2410e470 0x7fce241aaef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:28.122 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.019+0000 7fce2bed1700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce2410e8d0 0x7fce241a4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:28.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.019+0000 7fce2bed1700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce241a50a0 con 0x7fce2406d9b0 2023-07-14T18:15:28.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.019+0000 7fce2bed1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce241a5210 con 0x7fce2410e470 2023-07-14T18:15:28.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.019+0000 7fce2bed1700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce241a53d0 con 0x7fce2410e8d0 2023-07-14T18:15:28.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.019+0000 7fce2a46e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce2410e8d0 0x7fce241a4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:28.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.020+0000 7fce2a46e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce2410e8d0 0x7fce241a4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:43942/0 (socket says 172.21.15.133:43942) 2023-07-14T18:15:28.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.020+0000 7fce2a46e700 1 -- 172.21.15.133:0/842550101 learned_addr learned my addr 172.21.15.133:0/842550101 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:28.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.020+0000 7fce2946c700 1 --2- 172.21.15.133:0/842550101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce2410e470 0x7fce241aaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:28.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.020+0000 7fce2a46e700 1 -- 172.21.15.133:0/842550101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce2410e470 msgr2=0x7fce241aaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:28.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.020+0000 7fce2a46e700 1 --2- 172.21.15.133:0/842550101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce2410e470 0x7fce241aaef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:28.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.020+0000 7fce2a46e700 1 -- 172.21.15.133:0/842550101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce2406d9b0 msgr2=0x7fce241aa9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:28.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.020+0000 7fce2a46e700 1 --2- 172.21.15.133:0/842550101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce2406d9b0 0x7fce241aa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:28.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.020+0000 7fce2a46e700 1 -- 172.21.15.133:0/842550101 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce140095f0 con 0x7fce2410e8d0 2023-07-14T18:15:28.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.021+0000 7fce2a46e700 1 --2- 172.21.15.133:0/842550101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce2410e8d0 0x7fce241a4a30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fce1c00b9e0 tx=0x7fce1c00bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:28.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.023+0000 7fce1affd700 1 -- 172.21.15.133:0/842550101 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c00c9b0 con 0x7fce2410e8d0 2023-07-14T18:15:28.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.023+0000 7fce1affd700 1 -- 172.21.15.133:0/842550101 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce1c014440 con 0x7fce2410e8d0 2023-07-14T18:15:28.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.023+0000 7fce1affd700 1 -- 172.21.15.133:0/842550101 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c0125a0 con 0x7fce2410e8d0 2023-07-14T18:15:28.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.023+0000 7fce2bed1700 1 -- 172.21.15.133:0/842550101 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce241a56b0 con 0x7fce2410e8d0 2023-07-14T18:15:28.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.023+0000 7fce2bed1700 1 -- 172.21.15.133:0/842550101 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce241b19a0 con 0x7fce2410e8d0 2023-07-14T18:15:28.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.024+0000 7fce2bed1700 1 -- 172.21.15.133:0/842550101 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce2404e340 con 0x7fce2410e8d0 2023-07-14T18:15:28.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.029+0000 7fce1affd700 1 -- 172.21.15.133:0/842550101 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fce1c0145b0 con 0x7fce2410e8d0 2023-07-14T18:15:28.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.031+0000 7fce1affd700 1 --2- 172.21.15.133:0/842550101 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fce100a1330 0x7fce100a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:28.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.031+0000 7fce1affd700 1 -- 172.21.15.133:0/842550101 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fce1c006e40 con 0x7fce2410e8d0 2023-07-14T18:15:28.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.031+0000 7fce29c6d700 1 --2- 172.21.15.133:0/842550101 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fce100a1330 0x7fce100a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:28.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.031+0000 7fce1affd700 1 -- 172.21.15.133:0/842550101 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fce1c0f8840 con 0x7fce2410e8d0 2023-07-14T18:15:28.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.032+0000 7fce29c6d700 1 --2- 172.21.15.133:0/842550101 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fce100a1330 0x7fce100a37e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fce2406dfa0 tx=0x7fce20005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:28.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.289+0000 7fce2bed1700 1 -- 172.21.15.133:0/842550101 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fce241a5f40 con 0x7fce2410e8d0 2023-07-14T18:15:28.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.290+0000 7fce1affd700 1 -- 172.21.15.133:0/842550101 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fce1c019020 con 0x7fce2410e8d0 2023-07-14T18:15:28.334 INFO:teuthology.orchestra.run.smithi133.stdout:111669149758 2023-07-14T18:15:28.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.292+0000 7fce2bed1700 1 -- 172.21.15.133:0/842550101 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fce100a1330 msgr2=0x7fce100a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:28.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.292+0000 7fce2bed1700 1 --2- 172.21.15.133:0/842550101 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fce100a1330 0x7fce100a37e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fce2406dfa0 tx=0x7fce20005d20 comp rx=0 tx=0).stop 2023-07-14T18:15:28.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.292+0000 7fce2bed1700 1 -- 172.21.15.133:0/842550101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce2410e8d0 msgr2=0x7fce241a4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:28.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.292+0000 7fce2bed1700 1 --2- 172.21.15.133:0/842550101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce2410e8d0 0x7fce241a4a30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fce1c00b9e0 tx=0x7fce1c00bda0 comp rx=0 tx=0).stop 2023-07-14T18:15:28.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.292+0000 7fce2bed1700 1 -- 172.21.15.133:0/842550101 shutdown_connections 2023-07-14T18:15:28.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.292+0000 7fce2bed1700 1 --2- 172.21.15.133:0/842550101 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fce100a1330 0x7fce100a37e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:28.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.292+0000 7fce2bed1700 1 --2- 172.21.15.133:0/842550101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce2410e8d0 0x7fce241a4a30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:28.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.292+0000 7fce2bed1700 1 --2- 172.21.15.133:0/842550101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce2406d9b0 0x7fce241aa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:28.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.292+0000 7fce2bed1700 1 --2- 172.21.15.133:0/842550101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce2410e470 0x7fce241aaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:28.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.292+0000 7fce2bed1700 1 -- 172.21.15.133:0/842550101 >> 172.21.15.133:0/842550101 conn(0x7fce2406bf70 msgr2=0x7fce24109fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:28.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.292+0000 7fce2bed1700 1 -- 172.21.15.133:0/842550101 shutdown_connections 2023-07-14T18:15:28.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.292+0000 7fce2bed1700 1 -- 172.21.15.133:0/842550101 wait complete. 2023-07-14T18:15:28.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:27 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1389791415' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-14T18:15:28.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:27 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2777898555' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-14T18:15:28.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:27 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1389791415' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-14T18:15:28.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:27 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2777898555' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-14T18:15:28.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.594+0000 7f59b4d8a700 1 -- 172.21.15.133:0/3961132682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b00fff30 msgr2=0x7f59b0100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:28.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.594+0000 7f59b4d8a700 1 --2- 172.21.15.133:0/3961132682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b00fff30 0x7f59b0100300 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f59a00099c0 tx=0x7f59a0009cd0 comp rx=0 tx=0).stop 2023-07-14T18:15:28.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.594+0000 7f59b4d8a700 1 -- 172.21.15.133:0/3961132682 shutdown_connections 2023-07-14T18:15:28.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.594+0000 7f59b4d8a700 1 --2- 172.21.15.133:0/3961132682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59b0104fd0 0x7f59b010cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:28.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.594+0000 7f59b4d8a700 1 --2- 172.21.15.133:0/3961132682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b00fff30 0x7f59b0100300 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:28.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.594+0000 7f59b4d8a700 1 --2- 172.21.15.133:0/3961132682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f59b01008d0 0x7f59b01048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:28.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.594+0000 7f59b4d8a700 1 -- 172.21.15.133:0/3961132682 >> 172.21.15.133:0/3961132682 conn(0x7f59b00fb790 msgr2=0x7f59b00fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:28.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.594+0000 7f59b4d8a700 1 -- 172.21.15.133:0/3961132682 shutdown_connections 2023-07-14T18:15:28.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.594+0000 7f59b4d8a700 1 -- 172.21.15.133:0/3961132682 wait complete. 2023-07-14T18:15:28.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.595+0000 7f59b4d8a700 1 Processor -- start 2023-07-14T18:15:28.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.595+0000 7f59b4d8a700 1 -- start start 2023-07-14T18:15:28.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.596+0000 7f59b4d8a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b00fff30 0x7f59b0072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:28.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.596+0000 7f59b4d8a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59b01008d0 0x7f59b0072e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:28.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.596+0000 7f59b4d8a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f59b0104fd0 0x7f59b0073400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:28.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.596+0000 7f59b4d8a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b01a4210 con 0x7f59b00fff30 2023-07-14T18:15:28.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.596+0000 7f59b4d8a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b01a4380 con 0x7f59b01008d0 2023-07-14T18:15:28.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.596+0000 7f59b4d8a700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b01a4540 con 0x7f59b0104fd0 2023-07-14T18:15:28.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.596+0000 7f59ae59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b00fff30 0x7f59b0072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:28.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.596+0000 7f59ae59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b00fff30 0x7f59b0072940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51798/0 (socket says 172.21.15.133:51798) 2023-07-14T18:15:28.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.596+0000 7f59ae59c700 1 -- 172.21.15.133:0/896498854 learned_addr learned my addr 172.21.15.133:0/896498854 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:28.743 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.596+0000 7f59aed9d700 1 --2- 172.21.15.133:0/896498854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f59b0104fd0 0x7f59b0073400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:28.743 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.596+0000 7f59add9b700 1 --2- 172.21.15.133:0/896498854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59b01008d0 0x7f59b0072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:28.743 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.597+0000 7f59ae59c700 1 -- 172.21.15.133:0/896498854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f59b0104fd0 msgr2=0x7f59b0073400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:28.743 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.597+0000 7f59ae59c700 1 --2- 172.21.15.133:0/896498854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f59b0104fd0 0x7f59b0073400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:28.743 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.597+0000 7f59ae59c700 1 -- 172.21.15.133:0/896498854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59b01008d0 msgr2=0x7f59b0072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:28.743 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.597+0000 7f59ae59c700 1 --2- 172.21.15.133:0/896498854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59b01008d0 0x7f59b0072e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:28.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.597+0000 7f59ae59c700 1 -- 172.21.15.133:0/896498854 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a00095f0 con 0x7f59b00fff30 2023-07-14T18:15:28.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.597+0000 7f59aed9d700 1 --2- 172.21.15.133:0/896498854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f59b0104fd0 0x7f59b0073400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:28.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.597+0000 7f59ae59c700 1 --2- 172.21.15.133:0/896498854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b00fff30 0x7f59b0072940 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f59a0000c00 tx=0x7f59a000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:28.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.597+0000 7f599f7fe700 1 -- 172.21.15.133:0/896498854 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a001d070 con 0x7f59b00fff30 2023-07-14T18:15:28.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.597+0000 7f599f7fe700 1 -- 172.21.15.133:0/896498854 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59a0005050 con 0x7f59b00fff30 2023-07-14T18:15:28.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.597+0000 7f59b4d8a700 1 -- 172.21.15.133:0/896498854 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b01a47c0 con 0x7f59b00fff30 2023-07-14T18:15:28.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.597+0000 7f599f7fe700 1 -- 172.21.15.133:0/896498854 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a0004ab0 con 0x7f59b00fff30 2023-07-14T18:15:28.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.597+0000 7f59b4d8a700 1 -- 172.21.15.133:0/896498854 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b01a4c30 con 0x7f59b00fff30 2023-07-14T18:15:28.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.598+0000 7f59b4d8a700 1 -- 172.21.15.133:0/896498854 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b004e340 con 0x7f59b00fff30 2023-07-14T18:15:28.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.600+0000 7f599f7fe700 1 -- 172.21.15.133:0/896498854 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f59a00051c0 con 0x7f59b00fff30 2023-07-14T18:15:28.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.601+0000 7f599f7fe700 1 --2- 172.21.15.133:0/896498854 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f59980a1260 0x7f59980a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:28.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.602+0000 7f599f7fe700 1 -- 172.21.15.133:0/896498854 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f59a00cda30 con 0x7f59b00fff30 2023-07-14T18:15:28.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.602+0000 7f59add9b700 1 --2- 172.21.15.133:0/896498854 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f59980a1260 0x7f59980a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:28.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.602+0000 7f59add9b700 1 --2- 172.21.15.133:0/896498854 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f59980a1260 0x7f59980a3710 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f59b0074130 tx=0x7f59a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:28.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.605+0000 7f599f7fe700 1 -- 172.21.15.133:0/896498854 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f59a0096d40 con 0x7f59b00fff30 2023-07-14T18:15:28.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.820+0000 7f59b4d8a700 1 -- 172.21.15.133:0/896498854 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f59b01a4e50 con 0x7f59b00fff30 2023-07-14T18:15:28.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.820+0000 7f599f7fe700 1 -- 172.21.15.133:0/896498854 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f59a009a360 con 0x7f59b00fff30 2023-07-14T18:15:28.869 INFO:teuthology.orchestra.run.smithi133.stdout:197568495663 2023-07-14T18:15:28.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.822+0000 7f59b4d8a700 1 -- 172.21.15.133:0/896498854 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f59980a1260 msgr2=0x7f59980a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:28.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.822+0000 7f59b4d8a700 1 --2- 172.21.15.133:0/896498854 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f59980a1260 0x7f59980a3710 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f59b0074130 tx=0x7f59a4009260 comp rx=0 tx=0).stop 2023-07-14T18:15:28.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.822+0000 7f59b4d8a700 1 -- 172.21.15.133:0/896498854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b00fff30 msgr2=0x7f59b0072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:28.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.822+0000 7f59b4d8a700 1 --2- 172.21.15.133:0/896498854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b00fff30 0x7f59b0072940 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f59a0000c00 tx=0x7f59a000bfd0 comp rx=0 tx=0).stop 2023-07-14T18:15:28.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.823+0000 7f59b4d8a700 1 -- 172.21.15.133:0/896498854 shutdown_connections 2023-07-14T18:15:28.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.823+0000 7f59b4d8a700 1 --2- 172.21.15.133:0/896498854 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f59980a1260 0x7f59980a3710 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:28.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.823+0000 7f59b4d8a700 1 --2- 172.21.15.133:0/896498854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f59b0104fd0 0x7f59b0073400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:28.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.823+0000 7f59b4d8a700 1 --2- 172.21.15.133:0/896498854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b00fff30 0x7f59b0072940 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:28.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.823+0000 7f59b4d8a700 1 --2- 172.21.15.133:0/896498854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59b01008d0 0x7f59b0072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:28.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.823+0000 7f59b4d8a700 1 -- 172.21.15.133:0/896498854 >> 172.21.15.133:0/896498854 conn(0x7f59b00fb790 msgr2=0x7f59b0103880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:28.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.823+0000 7f59b4d8a700 1 -- 172.21.15.133:0/896498854 shutdown_connections 2023-07-14T18:15:28.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:28.823+0000 7f59b4d8a700 1 -- 172.21.15.133:0/896498854 wait complete. 2023-07-14T18:15:29.054 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:28 smithi133 ceph-mon[126384]: pgmap v310: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-14T18:15:29.054 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:28 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/842550101' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-14T18:15:29.054 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:28 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/896498854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-14T18:15:29.062 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411392 got 77309411394 for osd.2 2023-07-14T18:15:29.062 DEBUG:teuthology.parallel:result is None 2023-07-14T18:15:29.289 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:29.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:28 smithi161 ceph-mon[128993]: pgmap v310: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-14T18:15:29.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:28 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/842550101' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-14T18:15:29.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:28 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/896498854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-14T18:15:29.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:28 smithi171 ceph-mon[129241]: pgmap v310: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-14T18:15:29.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:28 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/842550101' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-14T18:15:29.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:28 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/896498854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-14T18:15:29.391 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332137 got 219043332139 for osd.7 2023-07-14T18:15:29.391 DEBUG:teuthology.parallel:result is None 2023-07-14T18:15:29.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.750+0000 7f23e2ebc700 1 -- 172.21.15.133:0/427561213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23dc1004e0 msgr2=0x7f23dc10c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:29.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.750+0000 7f23e2ebc700 1 --2- 172.21.15.133:0/427561213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23dc1004e0 0x7f23dc10c920 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f23d400b0a0 tx=0x7f23d400b3b0 comp rx=0 tx=0).stop 2023-07-14T18:15:29.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.750+0000 7f23e2ebc700 1 -- 172.21.15.133:0/427561213 shutdown_connections 2023-07-14T18:15:29.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.750+0000 7f23e2ebc700 1 --2- 172.21.15.133:0/427561213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23dc1004e0 0x7f23dc10c920 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:29.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.750+0000 7f23e2ebc700 1 --2- 172.21.15.133:0/427561213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23dc105b50 0x7f23dc105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:29.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.750+0000 7f23e2ebc700 1 --2- 172.21.15.133:0/427561213 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f23dc0ffb30 0x7f23dc0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:29.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.750+0000 7f23e2ebc700 1 -- 172.21.15.133:0/427561213 >> 172.21.15.133:0/427561213 conn(0x7f23dc0fb690 msgr2=0x7f23dc0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:29.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.750+0000 7f23e2ebc700 1 -- 172.21.15.133:0/427561213 shutdown_connections 2023-07-14T18:15:29.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.750+0000 7f23e2ebc700 1 -- 172.21.15.133:0/427561213 wait complete. 2023-07-14T18:15:29.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.752+0000 7f23e2ebc700 1 Processor -- start 2023-07-14T18:15:29.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.752+0000 7f23e2ebc700 1 -- start start 2023-07-14T18:15:29.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.752+0000 7f23e2ebc700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f23dc0ffb30 0x7f23dc195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:29.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.753+0000 7f23e2ebc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23dc1004e0 0x7f23dc196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:29.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.753+0000 7f23e2ebc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23dc105b50 0x7f23dc1995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:29.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.753+0000 7f23e2ebc700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc199bd0 con 0x7f23dc105b50 2023-07-14T18:15:29.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.753+0000 7f23e2ebc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc199d40 con 0x7f23dc1004e0 2023-07-14T18:15:29.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.753+0000 7f23e2ebc700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc199f00 con 0x7f23dc0ffb30 2023-07-14T18:15:29.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.753+0000 7f23e1eba700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f23dc0ffb30 0x7f23dc195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:29.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.753+0000 7f23e26bb700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23dc105b50 0x7f23dc1995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:29.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.753+0000 7f23e1eba700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f23dc0ffb30 0x7f23dc195cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:43970/0 (socket says 172.21.15.133:43970) 2023-07-14T18:15:29.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.753+0000 7f23e1eba700 1 -- 172.21.15.133:0/754537516 learned_addr learned my addr 172.21.15.133:0/754537516 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:29.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.753+0000 7f23e1eba700 1 -- 172.21.15.133:0/754537516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23dc1004e0 msgr2=0x7f23dc196210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T18:15:29.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.753+0000 7f23e1eba700 1 --2- 172.21.15.133:0/754537516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23dc1004e0 0x7f23dc196210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:29.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.753+0000 7f23e1eba700 1 -- 172.21.15.133:0/754537516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23dc105b50 msgr2=0x7f23dc1995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:29.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.753+0000 7f23e1eba700 1 --2- 172.21.15.133:0/754537516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23dc105b50 0x7f23dc1995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:29.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.753+0000 7f23e1eba700 1 -- 172.21.15.133:0/754537516 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d4009d00 con 0x7f23dc0ffb30 2023-07-14T18:15:29.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.754+0000 7f23e1eba700 1 --2- 172.21.15.133:0/754537516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f23dc0ffb30 0x7f23dc195cd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f23d800e9a0 tx=0x7f23d800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:29.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.754+0000 7f23d2ffd700 1 -- 172.21.15.133:0/754537516 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d800cbd0 con 0x7f23dc0ffb30 2023-07-14T18:15:29.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.754+0000 7f23e2ebc700 1 -- 172.21.15.133:0/754537516 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23dc19a1e0 con 0x7f23dc0ffb30 2023-07-14T18:15:29.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.754+0000 7f23d2ffd700 1 -- 172.21.15.133:0/754537516 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23d800cd30 con 0x7f23dc0ffb30 2023-07-14T18:15:29.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.754+0000 7f23d2ffd700 1 -- 172.21.15.133:0/754537516 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d8018680 con 0x7f23dc0ffb30 2023-07-14T18:15:29.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.754+0000 7f23e2ebc700 1 -- 172.21.15.133:0/754537516 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23dc072a40 con 0x7f23dc0ffb30 2023-07-14T18:15:29.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.755+0000 7f23e2ebc700 1 -- 172.21.15.133:0/754537516 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23dc04eb90 con 0x7f23dc0ffb30 2023-07-14T18:15:29.803 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.759+0000 7f23d2ffd700 1 -- 172.21.15.133:0/754537516 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f23d8010430 con 0x7f23dc0ffb30 2023-07-14T18:15:29.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.760+0000 7f23d2ffd700 1 --2- 172.21.15.133:0/754537516 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f23c80a1260 0x7f23c80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:29.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.760+0000 7f23d2ffd700 1 -- 172.21.15.133:0/754537516 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f23d8014070 con 0x7f23dc0ffb30 2023-07-14T18:15:29.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.760+0000 7f23e16b9700 1 --2- 172.21.15.133:0/754537516 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f23c80a1260 0x7f23c80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:29.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.761+0000 7f23d2ffd700 1 -- 172.21.15.133:0/754537516 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f23d8096300 con 0x7f23dc0ffb30 2023-07-14T18:15:29.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:29.761+0000 7f23e16b9700 1 --2- 172.21.15.133:0/754537516 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f23c80a1260 0x7f23c80a3710 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f23dc1972b0 tx=0x7f23cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:29.883 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:30.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.031+0000 7f23e2ebc700 1 -- 172.21.15.133:0/754537516 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f23dc196ff0 con 0x7f23dc0ffb30 2023-07-14T18:15:30.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.041+0000 7f23d2ffd700 1 -- 172.21.15.133:0/754537516 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f23d8099920 con 0x7f23dc0ffb30 2023-07-14T18:15:30.086 INFO:teuthology.orchestra.run.smithi133.stdout:34359738445 2023-07-14T18:15:30.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.045+0000 7f23e2ebc700 1 -- 172.21.15.133:0/754537516 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f23c80a1260 msgr2=0x7f23c80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:30.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.045+0000 7f23e2ebc700 1 --2- 172.21.15.133:0/754537516 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f23c80a1260 0x7f23c80a3710 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f23dc1972b0 tx=0x7f23cc009260 comp rx=0 tx=0).stop 2023-07-14T18:15:30.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.045+0000 7f23e2ebc700 1 -- 172.21.15.133:0/754537516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f23dc0ffb30 msgr2=0x7f23dc195cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:30.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.045+0000 7f23e2ebc700 1 --2- 172.21.15.133:0/754537516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f23dc0ffb30 0x7f23dc195cd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f23d800e9a0 tx=0x7f23d800ecb0 comp rx=0 tx=0).stop 2023-07-14T18:15:30.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.045+0000 7f23e2ebc700 1 -- 172.21.15.133:0/754537516 shutdown_connections 2023-07-14T18:15:30.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.045+0000 7f23e2ebc700 1 --2- 172.21.15.133:0/754537516 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f23c80a1260 0x7f23c80a3710 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:30.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.045+0000 7f23e2ebc700 1 --2- 172.21.15.133:0/754537516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23dc105b50 0x7f23dc1995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:30.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.045+0000 7f23e2ebc700 1 --2- 172.21.15.133:0/754537516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f23dc0ffb30 0x7f23dc195cd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:30.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.045+0000 7f23e2ebc700 1 --2- 172.21.15.133:0/754537516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23dc1004e0 0x7f23dc196210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:30.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.045+0000 7f23e2ebc700 1 -- 172.21.15.133:0/754537516 >> 172.21.15.133:0/754537516 conn(0x7f23dc0fb690 msgr2=0x7f23dc0fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:30.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.045+0000 7f23e2ebc700 1 -- 172.21.15.133:0/754537516 shutdown_connections 2023-07-14T18:15:30.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.045+0000 7f23e2ebc700 1 -- 172.21.15.133:0/754537516 wait complete. 2023-07-14T18:15:30.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.386+0000 7f88c2549700 1 -- 172.21.15.133:0/1440520043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88bc1008d0 msgr2=0x7f88bc1048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:30.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.386+0000 7f88c2549700 1 --2- 172.21.15.133:0/1440520043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88bc1008d0 0x7f88bc1048a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f88a40099c0 tx=0x7f88a4009cd0 comp rx=0 tx=0).stop 2023-07-14T18:15:30.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.386+0000 7f88c2549700 1 -- 172.21.15.133:0/1440520043 shutdown_connections 2023-07-14T18:15:30.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.386+0000 7f88c2549700 1 --2- 172.21.15.133:0/1440520043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88bc104fd0 0x7f88bc10cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:30.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.386+0000 7f88c2549700 1 --2- 172.21.15.133:0/1440520043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f88bc0fff30 0x7f88bc100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:30.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.386+0000 7f88c2549700 1 --2- 172.21.15.133:0/1440520043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88bc1008d0 0x7f88bc1048a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:30.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.386+0000 7f88c2549700 1 -- 172.21.15.133:0/1440520043 >> 172.21.15.133:0/1440520043 conn(0x7f88bc0fb790 msgr2=0x7f88bc0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:30.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.386+0000 7f88c2549700 1 -- 172.21.15.133:0/1440520043 shutdown_connections 2023-07-14T18:15:30.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.386+0000 7f88c2549700 1 -- 172.21.15.133:0/1440520043 wait complete. 2023-07-14T18:15:30.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.387+0000 7f88c2549700 1 Processor -- start 2023-07-14T18:15:30.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.388+0000 7f88c2549700 1 -- start start 2023-07-14T18:15:30.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.388+0000 7f88c2549700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88bc0fff30 0x7f88bc195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:30.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.388+0000 7f88c2549700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88bc1008d0 0x7f88bc1963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.388+0000 7f88c2549700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f88bc104fd0 0x7f88bc19a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.388+0000 7f88c2549700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc19ae50 con 0x7f88bc0fff30 2023-07-14T18:15:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.388+0000 7f88c2549700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc19afc0 con 0x7f88bc1008d0 2023-07-14T18:15:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.388+0000 7f88c2549700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc19b180 con 0x7f88bc104fd0 2023-07-14T18:15:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.389+0000 7f88bbfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88bc0fff30 0x7f88bc195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.389+0000 7f88b37fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88bc1008d0 0x7f88bc1963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.389+0000 7f88bbfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88bc0fff30 0x7f88bc195e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51852/0 (socket says 172.21.15.133:51852) 2023-07-14T18:15:30.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.389+0000 7f88bbfff700 1 -- 172.21.15.133:0/1997548203 learned_addr learned my addr 172.21.15.133:0/1997548203 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:30.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.389+0000 7f88c0ae6700 1 --2- 172.21.15.133:0/1997548203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f88bc104fd0 0x7f88bc19a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:30.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.389+0000 7f88bbfff700 1 -- 172.21.15.133:0/1997548203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f88bc104fd0 msgr2=0x7f88bc19a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:30.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.389+0000 7f88bbfff700 1 --2- 172.21.15.133:0/1997548203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f88bc104fd0 0x7f88bc19a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:30.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.389+0000 7f88bbfff700 1 -- 172.21.15.133:0/1997548203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88bc1008d0 msgr2=0x7f88bc1963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:30.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.389+0000 7f88bbfff700 1 --2- 172.21.15.133:0/1997548203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88bc1008d0 0x7f88bc1963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:30.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.389+0000 7f88bbfff700 1 -- 172.21.15.133:0/1997548203 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a40095f0 con 0x7f88bc0fff30 2023-07-14T18:15:30.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.389+0000 7f88bbfff700 1 --2- 172.21.15.133:0/1997548203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88bc0fff30 0x7f88bc195e90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00ef10 tx=0x7f88ac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:30.566 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.390+0000 7f88b9ffb700 1 -- 172.21.15.133:0/1997548203 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac00c9c0 con 0x7f88bc0fff30 2023-07-14T18:15:30.566 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.390+0000 7f88c2549700 1 -- 172.21.15.133:0/1997548203 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88bc19b400 con 0x7f88bc0fff30 2023-07-14T18:15:30.566 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.390+0000 7f88b9ffb700 1 -- 172.21.15.133:0/1997548203 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88ac00f040 con 0x7f88bc0fff30 2023-07-14T18:15:30.566 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.390+0000 7f88b9ffb700 1 -- 172.21.15.133:0/1997548203 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac014480 con 0x7f88bc0fff30 2023-07-14T18:15:30.566 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.390+0000 7f88c2549700 1 -- 172.21.15.133:0/1997548203 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88bc1a2cc0 con 0x7f88bc0fff30 2023-07-14T18:15:30.566 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.391+0000 7f88c2549700 1 -- 172.21.15.133:0/1997548203 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88bc04e340 con 0x7f88bc0fff30 2023-07-14T18:15:30.566 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.393+0000 7f88b9ffb700 1 -- 172.21.15.133:0/1997548203 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f88ac007690 con 0x7f88bc0fff30 2023-07-14T18:15:30.566 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.394+0000 7f88b9ffb700 1 --2- 172.21.15.133:0/1997548203 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f88a80a11e0 0x7f88a80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:30.567 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.394+0000 7f88b9ffb700 1 -- 172.21.15.133:0/1997548203 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f88ac0cd0d0 con 0x7f88bc0fff30 2023-07-14T18:15:30.567 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.395+0000 7f88b37fe700 1 --2- 172.21.15.133:0/1997548203 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f88a80a11e0 0x7f88a80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:30.567 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.395+0000 7f88b37fe700 1 --2- 172.21.15.133:0/1997548203 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f88a80a11e0 0x7f88a80a3690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f88a4009fd0 tx=0x7f88a4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:30.567 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.398+0000 7f88b9ffb700 1 -- 172.21.15.133:0/1997548203 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f88ac096350 con 0x7f88bc0fff30 2023-07-14T18:15:30.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.634+0000 7f88c2549700 1 -- 172.21.15.133:0/1997548203 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f88bc1a2e50 con 0x7f88bc0fff30 2023-07-14T18:15:30.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.634+0000 7f88b9ffb700 1 -- 172.21.15.133:0/1997548203 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f88ac099970 con 0x7f88bc0fff30 2023-07-14T18:15:30.679 INFO:teuthology.orchestra.run.smithi133.stdout:339302416410 2023-07-14T18:15:30.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.637+0000 7f88c2549700 1 -- 172.21.15.133:0/1997548203 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f88a80a11e0 msgr2=0x7f88a80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:30.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.637+0000 7f88c2549700 1 --2- 172.21.15.133:0/1997548203 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f88a80a11e0 0x7f88a80a3690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f88a4009fd0 tx=0x7f88a4005fb0 comp rx=0 tx=0).stop 2023-07-14T18:15:30.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.637+0000 7f88c2549700 1 -- 172.21.15.133:0/1997548203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88bc0fff30 msgr2=0x7f88bc195e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:30.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.637+0000 7f88c2549700 1 --2- 172.21.15.133:0/1997548203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88bc0fff30 0x7f88bc195e90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00ef10 tx=0x7f88ac00c3a0 comp rx=0 tx=0).stop 2023-07-14T18:15:30.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.637+0000 7f88c2549700 1 -- 172.21.15.133:0/1997548203 shutdown_connections 2023-07-14T18:15:30.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.637+0000 7f88c2549700 1 --2- 172.21.15.133:0/1997548203 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f88a80a11e0 0x7f88a80a3690 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:30.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.637+0000 7f88c2549700 1 --2- 172.21.15.133:0/1997548203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f88bc104fd0 0x7f88bc19a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:30.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.637+0000 7f88c2549700 1 --2- 172.21.15.133:0/1997548203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88bc0fff30 0x7f88bc195e90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:30.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.637+0000 7f88c2549700 1 --2- 172.21.15.133:0/1997548203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88bc1008d0 0x7f88bc1963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:30.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.637+0000 7f88c2549700 1 -- 172.21.15.133:0/1997548203 >> 172.21.15.133:0/1997548203 conn(0x7f88bc0fb790 msgr2=0x7f88bc0fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:30.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.637+0000 7f88c2549700 1 -- 172.21.15.133:0/1997548203 shutdown_connections 2023-07-14T18:15:30.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.637+0000 7f88c2549700 1 -- 172.21.15.133:0/1997548203 wait complete. 2023-07-14T18:15:31.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.616+0000 7f42136cb700 1 -- 172.21.15.133:0/1232819994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f420c102ae0 msgr2=0x7f420c106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:31.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.616+0000 7f42136cb700 1 --2- 172.21.15.133:0/1232819994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f420c102ae0 0x7f420c106ab0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4200009920 tx=0x7f4200009c30 comp rx=0 tx=0).stop 2023-07-14T18:15:31.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.616+0000 7f42136cb700 1 -- 172.21.15.133:0/1232819994 shutdown_connections 2023-07-14T18:15:31.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.616+0000 7f42136cb700 1 --2- 172.21.15.133:0/1232819994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f420c1071e0 0x7f420c10ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:31.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.616+0000 7f42136cb700 1 --2- 172.21.15.133:0/1232819994 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f420c102140 0x7f420c102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:31.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.616+0000 7f42136cb700 1 --2- 172.21.15.133:0/1232819994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f420c102ae0 0x7f420c106ab0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:31.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.616+0000 7f42136cb700 1 -- 172.21.15.133:0/1232819994 >> 172.21.15.133:0/1232819994 conn(0x7f420c0fd9e0 msgr2=0x7f420c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:31.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.616+0000 7f42136cb700 1 -- 172.21.15.133:0/1232819994 shutdown_connections 2023-07-14T18:15:31.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.616+0000 7f42136cb700 1 -- 172.21.15.133:0/1232819994 wait complete. 2023-07-14T18:15:31.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.618+0000 7f42136cb700 1 Processor -- start 2023-07-14T18:15:31.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.618+0000 7f42136cb700 1 -- start start 2023-07-14T18:15:31.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.618+0000 7f42136cb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f420c102140 0x7f420c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:31.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.618+0000 7f42136cb700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f420c102ae0 0x7f420c198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:31.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.619+0000 7f42136cb700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f420c1071e0 0x7f420c19c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:31.230 INFO:teuthology.orchestra.run.smithi133.stdout:141733920826 2023-07-14T18:15:31.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.619+0000 7f42136cb700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420c19cff0 con 0x7f420c1071e0 2023-07-14T18:15:31.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.619+0000 7f42136cb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420c19d160 con 0x7f420c102140 2023-07-14T18:15:31.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.619+0000 7f42136cb700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420c19d320 con 0x7f420c102ae0 2023-07-14T18:15:31.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.619+0000 7f4211c68700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f420c1071e0 0x7f420c19c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:31.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.619+0000 7f4210c66700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f420c102ae0 0x7f420c198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:31.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.619+0000 7f4211467700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f420c102140 0x7f420c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:31.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.619+0000 7f4210c66700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f420c102ae0 0x7f420c198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:44006/0 (socket says 172.21.15.133:44006) 2023-07-14T18:15:31.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.619+0000 7f4211467700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f420c102140 0x7f420c198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:40974/0 (socket says 172.21.15.133:40974) 2023-07-14T18:15:31.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.619+0000 7f4210c66700 1 -- 172.21.15.133:0/3238476140 learned_addr learned my addr 172.21.15.133:0/3238476140 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:31.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.619+0000 7f4211467700 1 -- 172.21.15.133:0/3238476140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f420c102ae0 msgr2=0x7f420c198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:31.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.619+0000 7f4211467700 1 --2- 172.21.15.133:0/3238476140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f420c102ae0 0x7f420c198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:31.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.619+0000 7f4211467700 1 -- 172.21.15.133:0/3238476140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f420c1071e0 msgr2=0x7f420c19c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:31.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.619+0000 7f4211467700 1 --2- 172.21.15.133:0/3238476140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f420c1071e0 0x7f420c19c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:31.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.619+0000 7f4211467700 1 -- 172.21.15.133:0/3238476140 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4208009520 con 0x7f420c102140 2023-07-14T18:15:31.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.620+0000 7f4211467700 1 --2- 172.21.15.133:0/3238476140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f420c102140 0x7f420c198030 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f420800e9f0 tx=0x7f420800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:31.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.620+0000 7f41fe7fc700 1 -- 172.21.15.133:0/3238476140 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420800cb70 con 0x7f420c102140 2023-07-14T18:15:31.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.620+0000 7f42136cb700 1 -- 172.21.15.133:0/3238476140 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4204009d00 con 0x7f420c102140 2023-07-14T18:15:31.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.620+0000 7f41fe7fc700 1 -- 172.21.15.133:0/3238476140 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f420800ee90 con 0x7f420c102140 2023-07-14T18:15:31.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.620+0000 7f41fe7fc700 1 -- 172.21.15.133:0/3238476140 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4208003cf0 con 0x7f420c102140 2023-07-14T18:15:31.234 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.620+0000 7f42136cb700 1 -- 172.21.15.133:0/3238476140 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4200009520 con 0x7f420c102140 2023-07-14T18:15:31.234 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.621+0000 7f42136cb700 1 -- 172.21.15.133:0/3238476140 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f420c04e340 con 0x7f420c102140 2023-07-14T18:15:31.234 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.623+0000 7f41fe7fc700 1 -- 172.21.15.133:0/3238476140 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f420800ccd0 con 0x7f420c102140 2023-07-14T18:15:31.234 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.624+0000 7f41fe7fc700 1 --2- 172.21.15.133:0/3238476140 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f41f80a11e0 0x7f41f80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:31.234 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.624+0000 7f41fe7fc700 1 -- 172.21.15.133:0/3238476140 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4208014070 con 0x7f420c102140 2023-07-14T18:15:31.234 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.624+0000 7f4210c66700 1 --2- 172.21.15.133:0/3238476140 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f41f80a11e0 0x7f41f80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:31.234 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.625+0000 7f4210c66700 1 --2- 172.21.15.133:0/3238476140 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f41f80a11e0 0x7f41f80a3690 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f420000c010 tx=0x7f4200019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.627+0000 7f41fe7fc700 1 -- 172.21.15.133:0/3238476140 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f4208096490 con 0x7f420c102140 2023-07-14T18:15:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.838+0000 7f42136cb700 1 -- 172.21.15.133:0/3238476140 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f420c1a4df0 con 0x7f420c102140 2023-07-14T18:15:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.839+0000 7f41fe7fc700 1 -- 172.21.15.133:0/3238476140 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4208099ab0 con 0x7f420c102140 2023-07-14T18:15:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.841+0000 7f42136cb700 1 -- 172.21.15.133:0/3238476140 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f41f80a11e0 msgr2=0x7f41f80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.841+0000 7f42136cb700 1 --2- 172.21.15.133:0/3238476140 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f41f80a11e0 0x7f41f80a3690 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f420000c010 tx=0x7f4200019040 comp rx=0 tx=0).stop 2023-07-14T18:15:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.841+0000 7f42136cb700 1 -- 172.21.15.133:0/3238476140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f420c102140 msgr2=0x7f420c198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.841+0000 7f42136cb700 1 --2- 172.21.15.133:0/3238476140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f420c102140 0x7f420c198030 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f420800e9f0 tx=0x7f420800ed00 comp rx=0 tx=0).stop 2023-07-14T18:15:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.841+0000 7f42136cb700 1 -- 172.21.15.133:0/3238476140 shutdown_connections 2023-07-14T18:15:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.841+0000 7f42136cb700 1 --2- 172.21.15.133:0/3238476140 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f41f80a11e0 0x7f41f80a3690 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.841+0000 7f42136cb700 1 --2- 172.21.15.133:0/3238476140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f420c102140 0x7f420c198030 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.841+0000 7f42136cb700 1 --2- 172.21.15.133:0/3238476140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f420c1071e0 0x7f420c19c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.841+0000 7f42136cb700 1 --2- 172.21.15.133:0/3238476140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f420c102ae0 0x7f420c198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.841+0000 7f42136cb700 1 -- 172.21.15.133:0/3238476140 >> 172.21.15.133:0/3238476140 conn(0x7f420c0fd9e0 msgr2=0x7f420c0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.842+0000 7f42136cb700 1 -- 172.21.15.133:0/3238476140 shutdown_connections 2023-07-14T18:15:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:30.842+0000 7f42136cb700 1 -- 172.21.15.133:0/3238476140 wait complete. 2023-07-14T18:15:31.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:30 smithi161 ceph-mon[128993]: pgmap v311: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-14T18:15:31.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:30 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/754537516' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-14T18:15:31.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:30 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1997548203' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-14T18:15:31.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:30 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3238476140' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-14T18:15:31.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:30 smithi171 ceph-mon[129241]: pgmap v311: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-14T18:15:31.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:30 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/754537516' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-14T18:15:31.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:30 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1997548203' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-14T18:15:31.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:30 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3238476140' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-14T18:15:31.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:30 smithi133 ceph-mon[126384]: pgmap v311: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-14T18:15:31.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:30 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/754537516' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-14T18:15:31.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:30 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1997548203' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-14T18:15:31.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:30 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3238476140' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-14T18:15:31.425 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495662 got 197568495663 for osd.6 2023-07-14T18:15:31.425 DEBUG:teuthology.parallel:result is None 2023-07-14T18:15:31.701 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149756 got 111669149758 for osd.3 2023-07-14T18:15:31.702 DEBUG:teuthology.parallel:result is None 2023-07-14T18:15:32.190 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920823 got 141733920826 for osd.4 2023-07-14T18:15:32.190 DEBUG:teuthology.parallel:result is None 2023-07-14T18:15:32.241 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738442 got 34359738445 for osd.0 2023-07-14T18:15:32.241 DEBUG:teuthology.parallel:result is None 2023-07-14T18:15:32.387 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.340+0000 7ff9f9117700 1 -- 172.21.15.133:0/2370456158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9f40fff30 msgr2=0x7ff9f4100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:32.387 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.340+0000 7ff9f9117700 1 --2- 172.21.15.133:0/2370456158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9f40fff30 0x7ff9f4100300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80098f0 tx=0x7ff9e8009c00 comp rx=0 tx=0).stop 2023-07-14T18:15:32.388 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.342+0000 7ff9f9117700 1 -- 172.21.15.133:0/2370456158 shutdown_connections 2023-07-14T18:15:32.388 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.342+0000 7ff9f9117700 1 --2- 172.21.15.133:0/2370456158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9f40fff30 0x7ff9f4100300 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:32.388 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.342+0000 7ff9f9117700 1 --2- 172.21.15.133:0/2370456158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f4104fd0 0x7ff9f410cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:32.388 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.342+0000 7ff9f9117700 1 --2- 172.21.15.133:0/2370456158 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9f41008d0 0x7ff9f41048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:32.388 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.342+0000 7ff9f9117700 1 -- 172.21.15.133:0/2370456158 >> 172.21.15.133:0/2370456158 conn(0x7ff9f40fb790 msgr2=0x7ff9f40fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:32.388 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.342+0000 7ff9f9117700 1 -- 172.21.15.133:0/2370456158 shutdown_connections 2023-07-14T18:15:32.388 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.342+0000 7ff9f9117700 1 -- 172.21.15.133:0/2370456158 wait complete. 2023-07-14T18:15:32.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.344+0000 7ff9f9117700 1 Processor -- start 2023-07-14T18:15:32.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.344+0000 7ff9f9117700 1 -- start start 2023-07-14T18:15:32.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.344+0000 7ff9f9117700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f40fff30 0x7ff9f4112e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:32.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.344+0000 7ff9f9117700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9f41008d0 0x7ff9f4113380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:32.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.344+0000 7ff9f9117700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9f4104fd0 0x7ff9f410ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:32.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.344+0000 7ff9f9117700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f410e6a0 con 0x7ff9f40fff30 2023-07-14T18:15:32.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.344+0000 7ff9f9117700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f410e810 con 0x7ff9f41008d0 2023-07-14T18:15:32.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.344+0000 7ff9f9117700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f410e9d0 con 0x7ff9f4104fd0 2023-07-14T18:15:32.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.345+0000 7ff9f2d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f40fff30 0x7ff9f4112e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:32.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.345+0000 7ff9f2d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f40fff30 0x7ff9f4112e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51884/0 (socket says 172.21.15.133:51884) 2023-07-14T18:15:32.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.345+0000 7ff9f2d9d700 1 -- 172.21.15.133:0/1033280355 learned_addr learned my addr 172.21.15.133:0/1033280355 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:32.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.345+0000 7ff9f259c700 1 --2- 172.21.15.133:0/1033280355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9f41008d0 0x7ff9f4113380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:32.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.345+0000 7ff9f359e700 1 --2- 172.21.15.133:0/1033280355 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9f4104fd0 0x7ff9f410ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:32.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.345+0000 7ff9f2d9d700 1 -- 172.21.15.133:0/1033280355 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9f4104fd0 msgr2=0x7ff9f410ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:32.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.345+0000 7ff9f2d9d700 1 --2- 172.21.15.133:0/1033280355 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9f4104fd0 0x7ff9f410ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:32.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.345+0000 7ff9f2d9d700 1 -- 172.21.15.133:0/1033280355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9f41008d0 msgr2=0x7ff9f4113380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:32.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.345+0000 7ff9f2d9d700 1 --2- 172.21.15.133:0/1033280355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9f41008d0 0x7ff9f4113380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:32.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.345+0000 7ff9f2d9d700 1 -- 172.21.15.133:0/1033280355 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e8009520 con 0x7ff9f40fff30 2023-07-14T18:15:32.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.345+0000 7ff9f259c700 1 --2- 172.21.15.133:0/1033280355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9f41008d0 0x7ff9f4113380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:32.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.346+0000 7ff9f2d9d700 1 --2- 172.21.15.133:0/1033280355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f40fff30 0x7ff9f4112e20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009810 tx=0x7ff9e800b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:32.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.346+0000 7ff9dbfff700 1 -- 172.21.15.133:0/1033280355 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e801c070 con 0x7ff9f40fff30 2023-07-14T18:15:32.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.346+0000 7ff9f9117700 1 -- 172.21.15.133:0/1033280355 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f410ec50 con 0x7ff9f40fff30 2023-07-14T18:15:32.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.346+0000 7ff9dbfff700 1 -- 172.21.15.133:0/1033280355 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9e800fa20 con 0x7ff9f40fff30 2023-07-14T18:15:32.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.346+0000 7ff9dbfff700 1 -- 172.21.15.133:0/1033280355 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8003970 con 0x7ff9f40fff30 2023-07-14T18:15:32.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.346+0000 7ff9f9117700 1 -- 172.21.15.133:0/1033280355 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f411ade0 con 0x7ff9f40fff30 2023-07-14T18:15:32.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.347+0000 7ff9d9ffb700 1 -- 172.21.15.133:0/1033280355 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9d40052f0 con 0x7ff9f40fff30 2023-07-14T18:15:32.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.349+0000 7ff9dbfff700 1 -- 172.21.15.133:0/1033280355 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7ff9e8020030 con 0x7ff9f40fff30 2023-07-14T18:15:32.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.351+0000 7ff9dbfff700 1 --2- 172.21.15.133:0/1033280355 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff9dc0a1260 0x7ff9dc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:32.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.351+0000 7ff9dbfff700 1 -- 172.21.15.133:0/1033280355 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff9e80cde60 con 0x7ff9f40fff30 2023-07-14T18:15:32.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.351+0000 7ff9f259c700 1 --2- 172.21.15.133:0/1033280355 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff9dc0a1260 0x7ff9dc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:32.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.352+0000 7ff9f259c700 1 --2- 172.21.15.133:0/1033280355 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff9dc0a1260 0x7ff9dc0a3710 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff9e400d5c0 tx=0x7ff9e400f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:32.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.356+0000 7ff9dbfff700 1 -- 172.21.15.133:0/1033280355 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7ff9e8097170 con 0x7ff9f40fff30 2023-07-14T18:15:32.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.589+0000 7ff9d9ffb700 1 -- 172.21.15.133:0/1033280355 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff9d4005160 con 0x7ff9f40fff30 2023-07-14T18:15:32.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.589+0000 7ff9dbfff700 1 -- 172.21.15.133:0/1033280355 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff9e809a790 con 0x7ff9f40fff30 2023-07-14T18:15:32.634 INFO:teuthology.orchestra.run.smithi133.stdout:55834574920 2023-07-14T18:15:32.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.592+0000 7ff9d9ffb700 1 -- 172.21.15.133:0/1033280355 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff9dc0a1260 msgr2=0x7ff9dc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:32.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.592+0000 7ff9d9ffb700 1 --2- 172.21.15.133:0/1033280355 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff9dc0a1260 0x7ff9dc0a3710 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff9e400d5c0 tx=0x7ff9e400f040 comp rx=0 tx=0).stop 2023-07-14T18:15:32.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.592+0000 7ff9d9ffb700 1 -- 172.21.15.133:0/1033280355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f40fff30 msgr2=0x7ff9f4112e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:32.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.592+0000 7ff9d9ffb700 1 --2- 172.21.15.133:0/1033280355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f40fff30 0x7ff9f4112e20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009810 tx=0x7ff9e800b770 comp rx=0 tx=0).stop 2023-07-14T18:15:32.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.592+0000 7ff9d9ffb700 1 -- 172.21.15.133:0/1033280355 shutdown_connections 2023-07-14T18:15:32.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.592+0000 7ff9d9ffb700 1 --2- 172.21.15.133:0/1033280355 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff9dc0a1260 0x7ff9dc0a3710 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:32.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.592+0000 7ff9d9ffb700 1 --2- 172.21.15.133:0/1033280355 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9f4104fd0 0x7ff9f410ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:32.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.592+0000 7ff9d9ffb700 1 --2- 172.21.15.133:0/1033280355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f40fff30 0x7ff9f4112e20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:32.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.592+0000 7ff9d9ffb700 1 --2- 172.21.15.133:0/1033280355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9f41008d0 0x7ff9f4113380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:32.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.592+0000 7ff9d9ffb700 1 -- 172.21.15.133:0/1033280355 >> 172.21.15.133:0/1033280355 conn(0x7ff9f40fb790 msgr2=0x7ff9f40fc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:32.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.592+0000 7ff9d9ffb700 1 -- 172.21.15.133:0/1033280355 shutdown_connections 2023-07-14T18:15:32.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:32.592+0000 7ff9d9ffb700 1 -- 172.21.15.133:0/1033280355 wait complete. 2023-07-14T18:15:33.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:32 smithi161 ceph-mon[128993]: pgmap v312: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T18:15:33.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:32 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1033280355' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-14T18:15:33.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:32 smithi171 ceph-mon[129241]: pgmap v312: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T18:15:33.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:32 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1033280355' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-14T18:15:33.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:32 smithi133 ceph-mon[126384]: pgmap v312: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T18:15:33.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:32 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1033280355' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-14T18:15:33.581 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574917 got 55834574920 for osd.1 2023-07-14T18:15:33.581 DEBUG:teuthology.parallel:result is None 2023-07-14T18:15:33.653 INFO:tasks.cephadm.ceph_manager.ceph:need seq 339302416408 got 339302416410 for osd.11 2023-07-14T18:15:33.653 DEBUG:teuthology.parallel:result is None 2023-07-14T18:15:33.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.794+0000 7fb892f61700 1 -- 172.21.15.133:0/1309806297 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb88c10c510 msgr2=0x7fb88c10e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:33.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.794+0000 7fb892f61700 1 --2- 172.21.15.133:0/1309806297 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb88c10c510 0x7fb88c10e8f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb884009f90 tx=0x7fb88400b330 comp rx=0 tx=0).stop 2023-07-14T18:15:33.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.794+0000 7fb892f61700 1 -- 172.21.15.133:0/1309806297 shutdown_connections 2023-07-14T18:15:33.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.794+0000 7fb892f61700 1 --2- 172.21.15.133:0/1309806297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb88c0729e0 0x7fb88c072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:33.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.794+0000 7fb892f61700 1 --2- 172.21.15.133:0/1309806297 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb88c10c510 0x7fb88c10e8f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:33.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.794+0000 7fb892f61700 1 --2- 172.21.15.133:0/1309806297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb88c0732f0 0x7fb88c10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:33.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.794+0000 7fb892f61700 1 -- 172.21.15.133:0/1309806297 >> 172.21.15.133:0/1309806297 conn(0x7fb88c0fb790 msgr2=0x7fb88c0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:33.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.794+0000 7fb892f61700 1 -- 172.21.15.133:0/1309806297 shutdown_connections 2023-07-14T18:15:33.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.794+0000 7fb892f61700 1 -- 172.21.15.133:0/1309806297 wait complete. 2023-07-14T18:15:33.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.795+0000 7fb892f61700 1 Processor -- start 2023-07-14T18:15:33.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.796+0000 7fb892f61700 1 -- start start 2023-07-14T18:15:33.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.796+0000 7fb892f61700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb88c0729e0 0x7fb88c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:33.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.796+0000 7fb892f61700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb88c0732f0 0x7fb88c198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:33.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.796+0000 7fb892f61700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb88c10c510 0x7fb88c19c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:33.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.796+0000 7fb892f61700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c19d060 con 0x7fb88c10c510 2023-07-14T18:15:33.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.796+0000 7fb892f61700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c19d1d0 con 0x7fb88c0732f0 2023-07-14T18:15:33.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.796+0000 7fb892f61700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c19d390 con 0x7fb88c0729e0 2023-07-14T18:15:33.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.796+0000 7fb88bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb88c0732f0 0x7fb88c198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:33.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.797+0000 7fb88bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb88c0732f0 0x7fb88c198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:41024/0 (socket says 172.21.15.133:41024) 2023-07-14T18:15:33.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.797+0000 7fb88bfff700 1 -- 172.21.15.133:0/2297981140 learned_addr learned my addr 172.21.15.133:0/2297981140 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:33.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.797+0000 7fb890cfd700 1 --2- 172.21.15.133:0/2297981140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb88c0729e0 0x7fb88c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:33.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.797+0000 7fb8914fe700 1 --2- 172.21.15.133:0/2297981140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb88c10c510 0x7fb88c19c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:33.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.797+0000 7fb88bfff700 1 -- 172.21.15.133:0/2297981140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb88c0729e0 msgr2=0x7fb88c198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:33.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.797+0000 7fb88bfff700 1 --2- 172.21.15.133:0/2297981140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb88c0729e0 0x7fb88c198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:33.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.797+0000 7fb88bfff700 1 -- 172.21.15.133:0/2297981140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb88c10c510 msgr2=0x7fb88c19c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:33.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.797+0000 7fb88bfff700 1 --2- 172.21.15.133:0/2297981140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb88c10c510 0x7fb88c19c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:33.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.797+0000 7fb88bfff700 1 -- 172.21.15.133:0/2297981140 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb880009520 con 0x7fb88c0732f0 2023-07-14T18:15:33.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.797+0000 7fb890cfd700 1 --2- 172.21.15.133:0/2297981140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb88c0729e0 0x7fb88c198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:33.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.797+0000 7fb88bfff700 1 --2- 172.21.15.133:0/2297981140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb88c0732f0 0x7fb88c198550 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb87800e9c0 tx=0x7fb87800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:33.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.797+0000 7fb889ffb700 1 -- 172.21.15.133:0/2297981140 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb878004a30 con 0x7fb88c0732f0 2023-07-14T18:15:33.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.798+0000 7fb889ffb700 1 -- 172.21.15.133:0/2297981140 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb87800cc80 con 0x7fb88c0732f0 2023-07-14T18:15:33.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.798+0000 7fb892f61700 1 -- 172.21.15.133:0/2297981140 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb884009c40 con 0x7fb88c0732f0 2023-07-14T18:15:33.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.798+0000 7fb889ffb700 1 -- 172.21.15.133:0/2297981140 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8780054e0 con 0x7fb88c0732f0 2023-07-14T18:15:33.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.798+0000 7fb892f61700 1 -- 172.21.15.133:0/2297981140 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb88c1a4e40 con 0x7fb88c0732f0 2023-07-14T18:15:33.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.799+0000 7fb892f61700 1 -- 172.21.15.133:0/2297981140 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb88c106e30 con 0x7fb88c0732f0 2023-07-14T18:15:33.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.802+0000 7fb889ffb700 1 -- 172.21.15.133:0/2297981140 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fb87800c7c0 con 0x7fb88c0732f0 2023-07-14T18:15:33.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.804+0000 7fb889ffb700 1 --2- 172.21.15.133:0/2297981140 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb87c0a1260 0x7fb87c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:33.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.804+0000 7fb889ffb700 1 -- 172.21.15.133:0/2297981140 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb878014070 con 0x7fb88c0732f0 2023-07-14T18:15:33.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.804+0000 7fb890cfd700 1 --2- 172.21.15.133:0/2297981140 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb87c0a1260 0x7fb87c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:33.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.805+0000 7fb890cfd700 1 --2- 172.21.15.133:0/2297981140 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb87c0a1260 0x7fb87c0a3710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb880009fd0 tx=0x7fb880009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:33.852 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.808+0000 7fb889ffb700 1 -- 172.21.15.133:0/2297981140 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fb878096070 con 0x7fb88c0732f0 2023-07-14T18:15:33.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.896+0000 7fe048214700 1 -- 172.21.15.133:0/52813339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0400732f0 msgr2=0x7fe0401107a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:33.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.896+0000 7fe048214700 1 --2- 172.21.15.133:0/52813339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0400732f0 0x7fe0401107a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe030009990 tx=0x7fe030009ca0 comp rx=0 tx=0).stop 2023-07-14T18:15:33.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.896+0000 7fe048214700 1 -- 172.21.15.133:0/52813339 shutdown_connections 2023-07-14T18:15:33.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.896+0000 7fe048214700 1 --2- 172.21.15.133:0/52813339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0400729e0 0x7fe040072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:33.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.896+0000 7fe048214700 1 --2- 172.21.15.133:0/52813339 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe040110ce0 0x7fe0401130c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:33.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.896+0000 7fe048214700 1 --2- 172.21.15.133:0/52813339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0400732f0 0x7fe0401107a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:33.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.896+0000 7fe048214700 1 -- 172.21.15.133:0/52813339 >> 172.21.15.133:0/52813339 conn(0x7fe0400fb790 msgr2=0x7fe0400fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:33.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.896+0000 7fe048214700 1 -- 172.21.15.133:0/52813339 shutdown_connections 2023-07-14T18:15:33.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.897+0000 7fe048214700 1 -- 172.21.15.133:0/52813339 wait complete. 2023-07-14T18:15:33.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.898+0000 7fe048214700 1 Processor -- start 2023-07-14T18:15:33.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.898+0000 7fe048214700 1 -- start start 2023-07-14T18:15:33.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.898+0000 7fe048214700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0400729e0 0x7fe04019c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:33.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.898+0000 7fe048214700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0400732f0 0x7fe04019c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:33.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.898+0000 7fe048214700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe040110ce0 0x7fe0401a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:33.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.898+0000 7fe048214700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0401a13d0 con 0x7fe0400729e0 2023-07-14T18:15:33.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.898+0000 7fe048214700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0401a1540 con 0x7fe0400732f0 2023-07-14T18:15:33.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.898+0000 7fe048214700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0401a1700 con 0x7fe040110ce0 2023-07-14T18:15:33.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.899+0000 7fe045fb0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0400729e0 0x7fe04019c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:33.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.899+0000 7fe045fb0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0400729e0 0x7fe04019c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51916/0 (socket says 172.21.15.133:51916) 2023-07-14T18:15:33.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.899+0000 7fe045fb0700 1 -- 172.21.15.133:0/1116863020 learned_addr learned my addr 172.21.15.133:0/1116863020 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:33.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.899+0000 7fe0457af700 1 --2- 172.21.15.133:0/1116863020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0400732f0 0x7fe04019c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:33.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.899+0000 7fe0467b1700 1 --2- 172.21.15.133:0/1116863020 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe040110ce0 0x7fe0401a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:33.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.899+0000 7fe045fb0700 1 -- 172.21.15.133:0/1116863020 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe040110ce0 msgr2=0x7fe0401a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:33.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.899+0000 7fe045fb0700 1 --2- 172.21.15.133:0/1116863020 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe040110ce0 0x7fe0401a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:33.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.899+0000 7fe045fb0700 1 -- 172.21.15.133:0/1116863020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0400732f0 msgr2=0x7fe04019c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:33.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.899+0000 7fe045fb0700 1 --2- 172.21.15.133:0/1116863020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0400732f0 0x7fe04019c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:33.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.899+0000 7fe045fb0700 1 -- 172.21.15.133:0/1116863020 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe038009c40 con 0x7fe0400729e0 2023-07-14T18:15:33.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.899+0000 7fe0467b1700 1 --2- 172.21.15.133:0/1116863020 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe040110ce0 0x7fe0401a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:33.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.900+0000 7fe045fb0700 1 --2- 172.21.15.133:0/1116863020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0400729e0 0x7fe04019c410 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe03c00e9a0 tx=0x7fe03c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:33.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.900+0000 7fe036ffd700 1 -- 172.21.15.133:0/1116863020 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c009790 con 0x7fe0400729e0 2023-07-14T18:15:33.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.900+0000 7fe036ffd700 1 -- 172.21.15.133:0/1116863020 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe03c00ce80 con 0x7fe0400729e0 2023-07-14T18:15:33.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.900+0000 7fe048214700 1 -- 172.21.15.133:0/1116863020 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0300095f0 con 0x7fe0400729e0 2023-07-14T18:15:33.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.900+0000 7fe036ffd700 1 -- 172.21.15.133:0/1116863020 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c018680 con 0x7fe0400729e0 2023-07-14T18:15:33.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.900+0000 7fe048214700 1 -- 172.21.15.133:0/1116863020 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0401a9070 con 0x7fe0400729e0 2023-07-14T18:15:33.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.902+0000 7fe048214700 1 -- 172.21.15.133:0/1116863020 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0401031c0 con 0x7fe0400729e0 2023-07-14T18:15:33.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.904+0000 7fe036ffd700 1 -- 172.21.15.133:0/1116863020 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fe03c010430 con 0x7fe0400729e0 2023-07-14T18:15:33.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.906+0000 7fe036ffd700 1 --2- 172.21.15.133:0/1116863020 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe02c0a1260 0x7fe02c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:33.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.906+0000 7fe036ffd700 1 -- 172.21.15.133:0/1116863020 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe03c014070 con 0x7fe0400729e0 2023-07-14T18:15:33.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.906+0000 7fe0457af700 1 --2- 172.21.15.133:0/1116863020 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe02c0a1260 0x7fe02c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:33.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.907+0000 7fe0457af700 1 --2- 172.21.15.133:0/1116863020 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe02c0a1260 0x7fe02c0a3710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe04019da30 tx=0x7fe030005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:33.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:33.912+0000 7fe036ffd700 1 -- 172.21.15.133:0/1116863020 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fe03c051780 con 0x7fe0400729e0 2023-07-14T18:15:34.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.056+0000 7fb892f61700 1 -- 172.21.15.133:0/2297981140 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fb88c04e340 con 0x7fb88c0732f0 2023-07-14T18:15:34.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.057+0000 7fb889ffb700 1 -- 172.21.15.133:0/2297981140 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fb878099690 con 0x7fb88c0732f0 2023-07-14T18:15:34.101 INFO:teuthology.orchestra.run.smithi133.stdout:309237645344 2023-07-14T18:15:34.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.060+0000 7fb892f61700 1 -- 172.21.15.133:0/2297981140 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb87c0a1260 msgr2=0x7fb87c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:34.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.060+0000 7fb892f61700 1 --2- 172.21.15.133:0/2297981140 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb87c0a1260 0x7fb87c0a3710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb880009fd0 tx=0x7fb880009260 comp rx=0 tx=0).stop 2023-07-14T18:15:34.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.060+0000 7fb892f61700 1 -- 172.21.15.133:0/2297981140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb88c0732f0 msgr2=0x7fb88c198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:34.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.060+0000 7fb892f61700 1 --2- 172.21.15.133:0/2297981140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb88c0732f0 0x7fb88c198550 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb87800e9c0 tx=0x7fb87800ecd0 comp rx=0 tx=0).stop 2023-07-14T18:15:34.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.061+0000 7fb892f61700 1 -- 172.21.15.133:0/2297981140 shutdown_connections 2023-07-14T18:15:34.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.061+0000 7fb892f61700 1 --2- 172.21.15.133:0/2297981140 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb87c0a1260 0x7fb87c0a3710 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:34.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.061+0000 7fb892f61700 1 --2- 172.21.15.133:0/2297981140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb88c10c510 0x7fb88c19c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:34.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.061+0000 7fb892f61700 1 --2- 172.21.15.133:0/2297981140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb88c0729e0 0x7fb88c198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:34.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.061+0000 7fb892f61700 1 --2- 172.21.15.133:0/2297981140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb88c0732f0 0x7fb88c198550 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:34.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.061+0000 7fb892f61700 1 -- 172.21.15.133:0/2297981140 >> 172.21.15.133:0/2297981140 conn(0x7fb88c0fb790 msgr2=0x7fb88c103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:34.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.061+0000 7fb892f61700 1 -- 172.21.15.133:0/2297981140 shutdown_connections 2023-07-14T18:15:34.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.061+0000 7fb892f61700 1 -- 172.21.15.133:0/2297981140 wait complete. 2023-07-14T18:15:34.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.171+0000 7fe048214700 1 -- 172.21.15.133:0/1116863020 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe040102ba0 con 0x7fe0400729e0 2023-07-14T18:15:34.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.171+0000 7fe036ffd700 1 -- 172.21.15.133:0/1116863020 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe03c0d1070 con 0x7fe0400729e0 2023-07-14T18:15:34.216 INFO:teuthology.orchestra.run.smithi133.stdout:163208757303 2023-07-14T18:15:34.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.175+0000 7fe048214700 1 -- 172.21.15.133:0/1116863020 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe02c0a1260 msgr2=0x7fe02c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:34.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.175+0000 7fe048214700 1 --2- 172.21.15.133:0/1116863020 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe02c0a1260 0x7fe02c0a3710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe04019da30 tx=0x7fe030005fd0 comp rx=0 tx=0).stop 2023-07-14T18:15:34.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.175+0000 7fe048214700 1 -- 172.21.15.133:0/1116863020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0400729e0 msgr2=0x7fe04019c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:34.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.175+0000 7fe048214700 1 --2- 172.21.15.133:0/1116863020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0400729e0 0x7fe04019c410 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe03c00e9a0 tx=0x7fe03c00ecb0 comp rx=0 tx=0).stop 2023-07-14T18:15:34.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.175+0000 7fe048214700 1 -- 172.21.15.133:0/1116863020 shutdown_connections 2023-07-14T18:15:34.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.175+0000 7fe048214700 1 --2- 172.21.15.133:0/1116863020 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe02c0a1260 0x7fe02c0a3710 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:34.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.175+0000 7fe048214700 1 --2- 172.21.15.133:0/1116863020 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe040110ce0 0x7fe0401a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:34.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.175+0000 7fe048214700 1 --2- 172.21.15.133:0/1116863020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0400729e0 0x7fe04019c410 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:34.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.175+0000 7fe048214700 1 --2- 172.21.15.133:0/1116863020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0400732f0 0x7fe04019c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:34.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.175+0000 7fe048214700 1 -- 172.21.15.133:0/1116863020 >> 172.21.15.133:0/1116863020 conn(0x7fe0400fb790 msgr2=0x7fe0400fc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:34.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.176+0000 7fe048214700 1 -- 172.21.15.133:0/1116863020 shutdown_connections 2023-07-14T18:15:34.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.176+0000 7fe048214700 1 -- 172.21.15.133:0/1116863020 wait complete. 2023-07-14T18:15:34.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.552+0000 7ff7a30ae700 1 -- 172.21.15.133:0/4150095429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff79c0729e0 msgr2=0x7ff79c072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:34.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.552+0000 7ff7a30ae700 1 --2- 172.21.15.133:0/4150095429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff79c0729e0 0x7ff79c072db0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff790009990 tx=0x7ff790009ca0 comp rx=0 tx=0).stop 2023-07-14T18:15:34.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.552+0000 7ff7a30ae700 1 -- 172.21.15.133:0/4150095429 shutdown_connections 2023-07-14T18:15:34.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.552+0000 7ff7a30ae700 1 --2- 172.21.15.133:0/4150095429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff79c0729e0 0x7ff79c072db0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:34.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.552+0000 7ff7a30ae700 1 --2- 172.21.15.133:0/4150095429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff79c10c510 0x7ff79c10e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:34.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.552+0000 7ff7a30ae700 1 --2- 172.21.15.133:0/4150095429 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff79c0732f0 0x7ff79c10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:34.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.552+0000 7ff7a30ae700 1 -- 172.21.15.133:0/4150095429 >> 172.21.15.133:0/4150095429 conn(0x7ff79c0fb790 msgr2=0x7ff79c0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:34.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.552+0000 7ff7a30ae700 1 -- 172.21.15.133:0/4150095429 shutdown_connections 2023-07-14T18:15:34.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.552+0000 7ff7a30ae700 1 -- 172.21.15.133:0/4150095429 wait complete. 2023-07-14T18:15:34.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.553+0000 7ff7a30ae700 1 Processor -- start 2023-07-14T18:15:34.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.554+0000 7ff7a30ae700 1 -- start start 2023-07-14T18:15:34.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.554+0000 7ff7a30ae700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff79c0729e0 0x7ff79c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:34.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.554+0000 7ff7a30ae700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff79c0732f0 0x7ff79c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:34.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.554+0000 7ff7a30ae700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff79c10c510 0x7ff79c19c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:34.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.554+0000 7ff7a30ae700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c19cf70 con 0x7ff79c10c510 2023-07-14T18:15:34.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.554+0000 7ff7a30ae700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c19d0e0 con 0x7ff79c0729e0 2023-07-14T18:15:34.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.554+0000 7ff7a30ae700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c19d2a0 con 0x7ff79c0732f0 2023-07-14T18:15:34.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.554+0000 7ff7a0e4a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff79c0729e0 0x7ff79c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:34.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.554+0000 7ff79bfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff79c0732f0 0x7ff79c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:34.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.554+0000 7ff7a0e4a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff79c0729e0 0x7ff79c197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:41074/0 (socket says 172.21.15.133:41074) 2023-07-14T18:15:34.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.554+0000 7ff7a0e4a700 1 -- 172.21.15.133:0/1556575751 learned_addr learned my addr 172.21.15.133:0/1556575751 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:34.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.554+0000 7ff7a164b700 1 --2- 172.21.15.133:0/1556575751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff79c10c510 0x7ff79c19c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:34.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.555+0000 7ff7a0e4a700 1 -- 172.21.15.133:0/1556575751 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff79c0732f0 msgr2=0x7ff79c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:34.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.555+0000 7ff7a0e4a700 1 --2- 172.21.15.133:0/1556575751 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff79c0732f0 0x7ff79c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:34.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.555+0000 7ff7a0e4a700 1 -- 172.21.15.133:0/1556575751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff79c10c510 msgr2=0x7ff79c19c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:34.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.555+0000 7ff7a0e4a700 1 --2- 172.21.15.133:0/1556575751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff79c10c510 0x7ff79c19c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:34.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.555+0000 7ff7a0e4a700 1 -- 172.21.15.133:0/1556575751 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7900095f0 con 0x7ff79c0729e0 2023-07-14T18:15:34.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.555+0000 7ff79bfff700 1 --2- 172.21.15.133:0/1556575751 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff79c0732f0 0x7ff79c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:34.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.555+0000 7ff7a0e4a700 1 --2- 172.21.15.133:0/1556575751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff79c0729e0 0x7ff79c197f20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff790006010 tx=0x7ff790005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:34.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.555+0000 7ff799ffb700 1 -- 172.21.15.133:0/1556575751 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79001d070 con 0x7ff79c0729e0 2023-07-14T18:15:34.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.555+0000 7ff799ffb700 1 -- 172.21.15.133:0/1556575751 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff790022470 con 0x7ff79c0729e0 2023-07-14T18:15:34.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.555+0000 7ff7a30ae700 1 -- 172.21.15.133:0/1556575751 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff79c19d520 con 0x7ff79c0729e0 2023-07-14T18:15:34.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.555+0000 7ff799ffb700 1 -- 172.21.15.133:0/1556575751 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff790017650 con 0x7ff79c0729e0 2023-07-14T18:15:34.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.555+0000 7ff7a30ae700 1 -- 172.21.15.133:0/1556575751 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff79c1a4d50 con 0x7ff79c0729e0 2023-07-14T18:15:34.802 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.557+0000 7ff7877fe700 1 -- 172.21.15.133:0/1556575751 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff77c0052f0 con 0x7ff79c0729e0 2023-07-14T18:15:34.802 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.559+0000 7ff799ffb700 1 -- 172.21.15.133:0/1556575751 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7ff790022850 con 0x7ff79c0729e0 2023-07-14T18:15:34.802 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.560+0000 7ff799ffb700 1 --2- 172.21.15.133:0/1556575751 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff78c0a1260 0x7ff78c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:34.802 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.560+0000 7ff799ffb700 1 -- 172.21.15.133:0/1556575751 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff7900ce1b0 con 0x7ff79c0729e0 2023-07-14T18:15:34.802 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.561+0000 7ff79bfff700 1 --2- 172.21.15.133:0/1556575751 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff78c0a1260 0x7ff78c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:34.802 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.561+0000 7ff79bfff700 1 --2- 172.21.15.133:0/1556575751 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff78c0a1260 0x7ff78c0a3710 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff79c199540 tx=0x7ff788009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:34.802 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.564+0000 7ff799ffb700 1 -- 172.21.15.133:0/1556575751 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7ff79005b780 con 0x7ff79c0729e0 2023-07-14T18:15:34.821 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.777+0000 7ff7877fe700 1 -- 172.21.15.133:0/1556575751 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7ff77c005c90 con 0x7ff79c0729e0 2023-07-14T18:15:34.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.778+0000 7ff799ffb700 1 -- 172.21.15.133:0/1556575751 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff7900974c0 con 0x7ff79c0729e0 2023-07-14T18:15:34.822 INFO:teuthology.orchestra.run.smithi133.stdout:253403070506 2023-07-14T18:15:34.824 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.780+0000 7ff7877fe700 1 -- 172.21.15.133:0/1556575751 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff78c0a1260 msgr2=0x7ff78c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:34.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.780+0000 7ff7877fe700 1 --2- 172.21.15.133:0/1556575751 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff78c0a1260 0x7ff78c0a3710 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff79c199540 tx=0x7ff788009260 comp rx=0 tx=0).stop 2023-07-14T18:15:34.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.780+0000 7ff7877fe700 1 -- 172.21.15.133:0/1556575751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff79c0729e0 msgr2=0x7ff79c197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:34.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.780+0000 7ff7877fe700 1 --2- 172.21.15.133:0/1556575751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff79c0729e0 0x7ff79c197f20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff790006010 tx=0x7ff790005720 comp rx=0 tx=0).stop 2023-07-14T18:15:34.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.780+0000 7ff7877fe700 1 -- 172.21.15.133:0/1556575751 shutdown_connections 2023-07-14T18:15:34.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.780+0000 7ff7877fe700 1 --2- 172.21.15.133:0/1556575751 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff78c0a1260 0x7ff78c0a3710 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:34.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.780+0000 7ff7877fe700 1 --2- 172.21.15.133:0/1556575751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff79c0729e0 0x7ff79c197f20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:34.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.780+0000 7ff7877fe700 1 --2- 172.21.15.133:0/1556575751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff79c10c510 0x7ff79c19c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:34.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.780+0000 7ff7877fe700 1 --2- 172.21.15.133:0/1556575751 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff79c0732f0 0x7ff79c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:34.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.780+0000 7ff7877fe700 1 -- 172.21.15.133:0/1556575751 >> 172.21.15.133:0/1556575751 conn(0x7ff79c0fb790 msgr2=0x7ff79c10e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:34.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.781+0000 7ff7877fe700 1 -- 172.21.15.133:0/1556575751 shutdown_connections 2023-07-14T18:15:34.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:34.781+0000 7ff7877fe700 1 -- 172.21.15.133:0/1556575751 wait complete. 2023-07-14T18:15:35.089 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:34 smithi133 ceph-mon[126384]: pgmap v313: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2023-07-14T18:15:35.089 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:34 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:15:35.089 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:34 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:15:35.090 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:34 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2297981140' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-14T18:15:35.090 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:34 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1116863020' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-14T18:15:35.095 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757300 got 163208757303 for osd.5 2023-07-14T18:15:35.095 DEBUG:teuthology.parallel:result is None 2023-07-14T18:15:35.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:34 smithi161 ceph-mon[128993]: pgmap v313: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2023-07-14T18:15:35.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:34 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:15:35.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:34 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:15:35.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:34 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2297981140' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-14T18:15:35.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:34 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1116863020' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-14T18:15:35.116 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:34 smithi171 ceph-mon[129241]: pgmap v313: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2023-07-14T18:15:35.116 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:34 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:15:35.116 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:34 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:15:35.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:34 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2297981140' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-14T18:15:35.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:34 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1116863020' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-14T18:15:35.247 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645341 got 309237645344 for osd.10 2023-07-14T18:15:35.248 DEBUG:teuthology.parallel:result is None 2023-07-14T18:15:35.511 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070502 got 253403070506 for osd.8 2023-07-14T18:15:35.511 DEBUG:teuthology.parallel:result is None 2023-07-14T18:15:35.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.805+0000 7fb7b98be700 1 -- 172.21.15.133:0/1611853591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7b40fff30 msgr2=0x7fb7b4100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:35.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.805+0000 7fb7b98be700 1 --2- 172.21.15.133:0/1611853591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7b40fff30 0x7fb7b4100300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb7a40099c0 tx=0x7fb7a4009cd0 comp rx=0 tx=0).stop 2023-07-14T18:15:35.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.805+0000 7fb7b98be700 1 -- 172.21.15.133:0/1611853591 shutdown_connections 2023-07-14T18:15:35.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.805+0000 7fb7b98be700 1 --2- 172.21.15.133:0/1611853591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7b4104fd0 0x7fb7b410cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:35.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.805+0000 7fb7b98be700 1 --2- 172.21.15.133:0/1611853591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7b40fff30 0x7fb7b4100300 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:35.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.805+0000 7fb7b98be700 1 --2- 172.21.15.133:0/1611853591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7b41008d0 0x7fb7b41048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:35.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.805+0000 7fb7b98be700 1 -- 172.21.15.133:0/1611853591 >> 172.21.15.133:0/1611853591 conn(0x7fb7b40fb790 msgr2=0x7fb7b40fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:35.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.805+0000 7fb7b98be700 1 -- 172.21.15.133:0/1611853591 shutdown_connections 2023-07-14T18:15:35.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.805+0000 7fb7b98be700 1 -- 172.21.15.133:0/1611853591 wait complete. 2023-07-14T18:15:35.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.806+0000 7fb7b98be700 1 Processor -- start 2023-07-14T18:15:35.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.806+0000 7fb7b98be700 1 -- start start 2023-07-14T18:15:35.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.807+0000 7fb7b98be700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7b40fff30 0x7fb7b4198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:35.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.807+0000 7fb7b98be700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7b41008d0 0x7fb7b4198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:35.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.807+0000 7fb7b98be700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7b4104fd0 0x7fb7b419c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:35.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.807+0000 7fb7b98be700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b419cf70 con 0x7fb7b4104fd0 2023-07-14T18:15:35.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.807+0000 7fb7b98be700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b419d0e0 con 0x7fb7b40fff30 2023-07-14T18:15:35.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.807+0000 7fb7b98be700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b419d2a0 con 0x7fb7b41008d0 2023-07-14T18:15:35.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.807+0000 7fb7b37fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7b4104fd0 0x7fb7b419c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:35.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.807+0000 7fb7b37fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7b4104fd0 0x7fb7b419c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51962/0 (socket says 172.21.15.133:51962) 2023-07-14T18:15:35.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.807+0000 7fb7b2ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7b40fff30 0x7fb7b4198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:35.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.807+0000 7fb7b37fe700 1 -- 172.21.15.133:0/3537158473 learned_addr learned my addr 172.21.15.133:0/3537158473 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:35.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.807+0000 7fb7b27fc700 1 --2- 172.21.15.133:0/3537158473 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7b41008d0 0x7fb7b4198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:35.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.807+0000 7fb7b37fe700 1 -- 172.21.15.133:0/3537158473 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7b41008d0 msgr2=0x7fb7b4198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:35.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.807+0000 7fb7b37fe700 1 --2- 172.21.15.133:0/3537158473 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7b41008d0 0x7fb7b4198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:35.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.807+0000 7fb7b37fe700 1 -- 172.21.15.133:0/3537158473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7b40fff30 msgr2=0x7fb7b4198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:35.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.807+0000 7fb7b37fe700 1 --2- 172.21.15.133:0/3537158473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7b40fff30 0x7fb7b4198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:35.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.807+0000 7fb7b37fe700 1 -- 172.21.15.133:0/3537158473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a40095f0 con 0x7fb7b4104fd0 2023-07-14T18:15:35.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.808+0000 7fb7b2ffd700 1 --2- 172.21.15.133:0/3537158473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7b40fff30 0x7fb7b4198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:35.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.808+0000 7fb7b37fe700 1 --2- 172.21.15.133:0/3537158473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7b4104fd0 0x7fb7b419c960 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac007ee0 tx=0x7fb7ac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:35.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.808+0000 7fb7b88bc700 1 -- 172.21.15.133:0/3537158473 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac00dd20 con 0x7fb7b4104fd0 2023-07-14T18:15:35.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.808+0000 7fb7b88bc700 1 -- 172.21.15.133:0/3537158473 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7ac00f040 con 0x7fb7b4104fd0 2023-07-14T18:15:35.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.808+0000 7fb7b98be700 1 -- 172.21.15.133:0/3537158473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b419d580 con 0x7fb7b4104fd0 2023-07-14T18:15:35.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.808+0000 7fb7b88bc700 1 -- 172.21.15.133:0/3537158473 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac0048a0 con 0x7fb7b4104fd0 2023-07-14T18:15:35.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.808+0000 7fb7b98be700 1 -- 172.21.15.133:0/3537158473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b41a4de0 con 0x7fb7b4104fd0 2023-07-14T18:15:35.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.809+0000 7fb7b98be700 1 -- 172.21.15.133:0/3537158473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b404e340 con 0x7fb7b4104fd0 2023-07-14T18:15:35.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.811+0000 7fb7b88bc700 1 -- 172.21.15.133:0/3537158473 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fb7ac003a00 con 0x7fb7b4104fd0 2023-07-14T18:15:35.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.812+0000 7fb7b88bc700 1 --2- 172.21.15.133:0/3537158473 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb79c0a1260 0x7fb79c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:35.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.813+0000 7fb7b88bc700 1 -- 172.21.15.133:0/3537158473 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb7ac0cc5a0 con 0x7fb7b4104fd0 2023-07-14T18:15:35.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.813+0000 7fb7b2ffd700 1 --2- 172.21.15.133:0/3537158473 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb79c0a1260 0x7fb79c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:35.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.813+0000 7fb7b2ffd700 1 --2- 172.21.15.133:0/3537158473 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb79c0a1260 0x7fb79c0a3710 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4009990 tx=0x7fb7a401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:35.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:35.816+0000 7fb7b88bc700 1 -- 172.21.15.133:0/3537158473 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fb7ac04e780 con 0x7fb7b4104fd0 2023-07-14T18:15:36.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:36.014+0000 7fb7b98be700 1 -- 172.21.15.133:0/3537158473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fb7b41993b0 con 0x7fb7b4104fd0 2023-07-14T18:15:36.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:36.014+0000 7fb7b88bc700 1 -- 172.21.15.133:0/3537158473 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb7ac095820 con 0x7fb7b4104fd0 2023-07-14T18:15:36.059 INFO:teuthology.orchestra.run.smithi133.stdout:287762808869 2023-07-14T18:15:36.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:36.016+0000 7fb7b98be700 1 -- 172.21.15.133:0/3537158473 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb79c0a1260 msgr2=0x7fb79c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:36.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:36.016+0000 7fb7b98be700 1 --2- 172.21.15.133:0/3537158473 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb79c0a1260 0x7fb79c0a3710 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4009990 tx=0x7fb7a401a040 comp rx=0 tx=0).stop 2023-07-14T18:15:36.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:36.016+0000 7fb7b98be700 1 -- 172.21.15.133:0/3537158473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7b4104fd0 msgr2=0x7fb7b419c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:36.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:36.016+0000 7fb7b98be700 1 --2- 172.21.15.133:0/3537158473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7b4104fd0 0x7fb7b419c960 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac007ee0 tx=0x7fb7ac00d3b0 comp rx=0 tx=0).stop 2023-07-14T18:15:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:36.016+0000 7fb7b98be700 1 -- 172.21.15.133:0/3537158473 shutdown_connections 2023-07-14T18:15:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:36.016+0000 7fb7b98be700 1 --2- 172.21.15.133:0/3537158473 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb79c0a1260 0x7fb79c0a3710 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:36.017+0000 7fb7b98be700 1 --2- 172.21.15.133:0/3537158473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7b40fff30 0x7fb7b4198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:36.017+0000 7fb7b98be700 1 --2- 172.21.15.133:0/3537158473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7b4104fd0 0x7fb7b419c960 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:36.017+0000 7fb7b98be700 1 --2- 172.21.15.133:0/3537158473 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7b41008d0 0x7fb7b4198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:36.017+0000 7fb7b98be700 1 -- 172.21.15.133:0/3537158473 >> 172.21.15.133:0/3537158473 conn(0x7fb7b40fb790 msgr2=0x7fb7b40fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:36.017+0000 7fb7b98be700 1 -- 172.21.15.133:0/3537158473 shutdown_connections 2023-07-14T18:15:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:36.017+0000 7fb7b98be700 1 -- 172.21.15.133:0/3537158473 wait complete. 2023-07-14T18:15:36.083 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:35 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1556575751' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-14T18:15:36.083 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:35 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:15:36.084 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:35 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:15:36.084 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:35 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:15:36.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:35 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1556575751' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-14T18:15:36.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:35 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:15:36.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:35 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:15:36.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:35 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:15:36.116 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:35 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1556575751' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-14T18:15:36.116 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:35 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:15:36.116 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:35 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:15:36.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:35 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:15:36.577 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808866 got 287762808869 for osd.9 2023-07-14T18:15:36.577 DEBUG:teuthology.parallel:result is None 2023-07-14T18:15:36.578 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-07-14T18:15:36.578 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph pg dump --format=json 2023-07-14T18:15:37.299 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:37.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:36 smithi161 ceph-mon[128993]: pgmap v314: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T18:15:37.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:36 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3537158473' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-14T18:15:37.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:36 smithi171 ceph-mon[129241]: pgmap v314: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T18:15:37.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:36 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3537158473' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-14T18:15:37.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:36 smithi133 ceph-mon[126384]: pgmap v314: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T18:15:37.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:36 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3537158473' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-14T18:15:38.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.752+0000 7f6313205700 1 -- 172.21.15.133:0/1703646530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630c104fd0 msgr2=0x7f630c10cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:38.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.752+0000 7f6313205700 1 --2- 172.21.15.133:0/1703646530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630c104fd0 0x7f630c10cb50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6304009fc0 tx=0x7f630400b330 comp rx=0 tx=0).stop 2023-07-14T18:15:38.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.752+0000 7f6313205700 1 -- 172.21.15.133:0/1703646530 shutdown_connections 2023-07-14T18:15:38.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.752+0000 7f6313205700 1 --2- 172.21.15.133:0/1703646530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630c104fd0 0x7f630c10cb50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:38.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.752+0000 7f6313205700 1 --2- 172.21.15.133:0/1703646530 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f630c0fff30 0x7f630c100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:38.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.752+0000 7f6313205700 1 --2- 172.21.15.133:0/1703646530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f630c1008d0 0x7f630c1048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:38.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.752+0000 7f6313205700 1 -- 172.21.15.133:0/1703646530 >> 172.21.15.133:0/1703646530 conn(0x7f630c0fb790 msgr2=0x7f630c0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:38.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.752+0000 7f6313205700 1 -- 172.21.15.133:0/1703646530 shutdown_connections 2023-07-14T18:15:38.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.752+0000 7f6313205700 1 -- 172.21.15.133:0/1703646530 wait complete. 2023-07-14T18:15:38.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.753+0000 7f6313205700 1 Processor -- start 2023-07-14T18:15:38.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.753+0000 7f6313205700 1 -- start start 2023-07-14T18:15:38.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.753+0000 7f6313205700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630c0fff30 0x7f630c195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:38.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.753+0000 7f6313205700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f630c1008d0 0x7f630c1963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:38.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.753+0000 7f6313205700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f630c104fd0 0x7f630c19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:38.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.753+0000 7f6313205700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630c19ae20 con 0x7f630c0fff30 2023-07-14T18:15:38.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.753+0000 7f6313205700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630c19af90 con 0x7f630c104fd0 2023-07-14T18:15:38.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.753+0000 7f6313205700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630c19b150 con 0x7f630c1008d0 2023-07-14T18:15:38.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.754+0000 7f6310fa1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630c0fff30 0x7f630c195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:38.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.754+0000 7f6310fa1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630c0fff30 0x7f630c195e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50408/0 (socket says 172.21.15.133:50408) 2023-07-14T18:15:38.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.754+0000 7f6310fa1700 1 -- 172.21.15.133:0/1874797274 learned_addr learned my addr 172.21.15.133:0/1874797274 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:38.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.754+0000 7f630bfff700 1 --2- 172.21.15.133:0/1874797274 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f630c1008d0 0x7f630c1963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:38.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.754+0000 7f63117a2700 1 --2- 172.21.15.133:0/1874797274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f630c104fd0 0x7f630c19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:38.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.754+0000 7f6310fa1700 1 -- 172.21.15.133:0/1874797274 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f630c1008d0 msgr2=0x7f630c1963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:38.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.754+0000 7f6310fa1700 1 --2- 172.21.15.133:0/1874797274 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f630c1008d0 0x7f630c1963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:38.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.754+0000 7f6310fa1700 1 -- 172.21.15.133:0/1874797274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f630c104fd0 msgr2=0x7f630c19a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:38.802 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.754+0000 7f6310fa1700 1 --2- 172.21.15.133:0/1874797274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f630c104fd0 0x7f630c19a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:38.802 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.754+0000 7f6310fa1700 1 -- 172.21.15.133:0/1874797274 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6304009c40 con 0x7f630c0fff30 2023-07-14T18:15:38.802 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.754+0000 7f63117a2700 1 --2- 172.21.15.133:0/1874797274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f630c104fd0 0x7f630c19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:15:38.802 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.754+0000 7f6310fa1700 1 --2- 172.21.15.133:0/1874797274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630c0fff30 0x7f630c195e60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f630000e9a0 tx=0x7f630000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:38.802 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.755+0000 7f6309ffb700 1 -- 172.21.15.133:0/1874797274 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6300009790 con 0x7f630c0fff30 2023-07-14T18:15:38.802 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.755+0000 7f6313205700 1 -- 172.21.15.133:0/1874797274 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f630c19b430 con 0x7f630c0fff30 2023-07-14T18:15:38.802 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.755+0000 7f6309ffb700 1 -- 172.21.15.133:0/1874797274 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f630000cbd0 con 0x7f630c0fff30 2023-07-14T18:15:38.802 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.755+0000 7f6309ffb700 1 -- 172.21.15.133:0/1874797274 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630000eee0 con 0x7f630c0fff30 2023-07-14T18:15:38.803 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.755+0000 7f6313205700 1 -- 172.21.15.133:0/1874797274 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f630c072a40 con 0x7f630c0fff30 2023-07-14T18:15:38.803 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.756+0000 7f6313205700 1 -- 172.21.15.133:0/1874797274 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f630c04e340 con 0x7f630c0fff30 2023-07-14T18:15:38.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.758+0000 7f6309ffb700 1 -- 172.21.15.133:0/1874797274 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f630000cd40 con 0x7f630c0fff30 2023-07-14T18:15:38.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.759+0000 7f6309ffb700 1 --2- 172.21.15.133:0/1874797274 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f62fc0a11e0 0x7f62fc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:38.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.759+0000 7f6309ffb700 1 -- 172.21.15.133:0/1874797274 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6300014070 con 0x7f630c0fff30 2023-07-14T18:15:38.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.759+0000 7f630bfff700 1 --2- 172.21.15.133:0/1874797274 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f62fc0a11e0 0x7f62fc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:38.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.760+0000 7f630bfff700 1 --2- 172.21.15.133:0/1874797274 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f62fc0a11e0 0x7f62fc0a3690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f630c197480 tx=0x7f62f8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:38.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.763+0000 7f6309ffb700 1 -- 172.21.15.133:0/1874797274 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f6300051780 con 0x7f630c0fff30 2023-07-14T18:15:39.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.957+0000 7f6313205700 1 -- 172.21.15.133:0/1874797274 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f630c0fc870 con 0x7f62fc0a11e0 2023-07-14T18:15:39.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.971+0000 7f6309ffb700 1 -- 172.21.15.133:0/1874797274 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427161 (secure 0 0 0) 0x7f630c0fc870 con 0x7f62fc0a11e0 2023-07-14T18:15:39.020 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:15:39.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.976+0000 7f6313205700 1 -- 172.21.15.133:0/1874797274 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f62fc0a11e0 msgr2=0x7f62fc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:39.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.976+0000 7f6313205700 1 --2- 172.21.15.133:0/1874797274 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f62fc0a11e0 0x7f62fc0a3690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f630c197480 tx=0x7f62f8009310 comp rx=0 tx=0).stop 2023-07-14T18:15:39.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.976+0000 7f6313205700 1 -- 172.21.15.133:0/1874797274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630c0fff30 msgr2=0x7f630c195e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:39.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.976+0000 7f6313205700 1 --2- 172.21.15.133:0/1874797274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630c0fff30 0x7f630c195e60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f630000e9a0 tx=0x7f630000ed60 comp rx=0 tx=0).stop 2023-07-14T18:15:39.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.976+0000 7f6313205700 1 -- 172.21.15.133:0/1874797274 shutdown_connections 2023-07-14T18:15:39.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.976+0000 7f6313205700 1 --2- 172.21.15.133:0/1874797274 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f62fc0a11e0 0x7f62fc0a3690 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:39.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.976+0000 7f6313205700 1 --2- 172.21.15.133:0/1874797274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f630c104fd0 0x7f630c19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:39.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.976+0000 7f6313205700 1 --2- 172.21.15.133:0/1874797274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630c0fff30 0x7f630c195e60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:39.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.976+0000 7f6313205700 1 --2- 172.21.15.133:0/1874797274 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f630c1008d0 0x7f630c1963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:39.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.976+0000 7f6313205700 1 -- 172.21.15.133:0/1874797274 >> 172.21.15.133:0/1874797274 conn(0x7f630c0fb790 msgr2=0x7f630c0fc160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:39.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.976+0000 7f6313205700 1 -- 172.21.15.133:0/1874797274 shutdown_connections 2023-07-14T18:15:39.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:38.976+0000 7f6313205700 1 -- 172.21.15.133:0/1874797274 wait complete. 2023-07-14T18:15:39.024 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2023-07-14T18:15:39.214 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:38 smithi133 ceph-mon[126384]: pgmap v315: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T18:15:39.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:38 smithi161 ceph-mon[128993]: pgmap v315: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T18:15:39.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:38 smithi171 ceph-mon[129241]: pgmap v315: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T18:15:39.485 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2023-07-14T18:15:37.450286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654326,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":417,"num_read_kb":319,"num_write":340,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":277,"ondisk_log_size":277,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336208,"kb_used_data":12000,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124605616,"statfs":{"total":1151940427776,"available":1151596150784,"internally_reserved":0,"allocated":12288000,"data_stored":7849470,"data_compressed":318684,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003338"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945666+0000","last_change":"2023-07-14T18:14:17.927179+0000","last_active":"2023-07-14T18:14:48.945666+0000","last_peered":"2023-07-14T18:14:48.945666+0000","last_clean":"2023-07-14T18:14:48.945666+0000","last_became_active":"2023-07-14T18:14:17.926880+0000","last_became_peered":"2023-07-14T18:14:17.926880+0000","last_unstale":"2023-07-14T18:14:48.945666+0000","last_undegraded":"2023-07-14T18:14:48.945666+0000","last_fullsized":"2023-07-14T18:14:48.945666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:26:26.468709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541115+0000","last_change":"2023-07-14T18:14:17.933834+0000","last_active":"2023-07-14T18:14:49.541115+0000","last_peered":"2023-07-14T18:14:49.541115+0000","last_clean":"2023-07-14T18:14:49.541115+0000","last_became_active":"2023-07-14T18:14:17.933529+0000","last_became_peered":"2023-07-14T18:14:17.933529+0000","last_unstale":"2023-07-14T18:14:49.541115+0000","last_undegraded":"2023-07-14T18:14:49.541115+0000","last_fullsized":"2023-07-14T18:14:49.541115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:34:13.388837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540772+0000","last_change":"2023-07-14T18:14:17.928400+0000","last_active":"2023-07-14T18:14:49.540772+0000","last_peered":"2023-07-14T18:14:49.540772+0000","last_clean":"2023-07-14T18:14:49.540772+0000","last_became_active":"2023-07-14T18:14:17.928077+0000","last_became_peered":"2023-07-14T18:14:17.928077+0000","last_unstale":"2023-07-14T18:14:49.540772+0000","last_undegraded":"2023-07-14T18:14:49.540772+0000","last_fullsized":"2023-07-14T18:14:49.540772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:24.522613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.947528+0000","last_change":"2023-07-14T18:14:17.932623+0000","last_active":"2023-07-14T18:14:48.947528+0000","last_peered":"2023-07-14T18:14:48.947528+0000","last_clean":"2023-07-14T18:14:48.947528+0000","last_became_active":"2023-07-14T18:14:17.932298+0000","last_became_peered":"2023-07-14T18:14:17.932298+0000","last_unstale":"2023-07-14T18:14:48.947528+0000","last_undegraded":"2023-07-14T18:14:48.947528+0000","last_fullsized":"2023-07-14T18:14:48.947528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:17:46.564398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945288+0000","last_change":"2023-07-14T18:14:17.927109+0000","last_active":"2023-07-14T18:14:48.945288+0000","last_peered":"2023-07-14T18:14:48.945288+0000","last_clean":"2023-07-14T18:14:48.945288+0000","last_became_active":"2023-07-14T18:14:17.926800+0000","last_became_peered":"2023-07-14T18:14:17.926800+0000","last_unstale":"2023-07-14T18:14:48.945288+0000","last_undegraded":"2023-07-14T18:14:48.945288+0000","last_fullsized":"2023-07-14T18:14:48.945288+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:43:24.990024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541350+0000","last_change":"2023-07-14T18:14:17.935380+0000","last_active":"2023-07-14T18:14:49.541350+0000","last_peered":"2023-07-14T18:14:49.541350+0000","last_clean":"2023-07-14T18:14:49.541350+0000","last_became_active":"2023-07-14T18:14:17.935099+0000","last_became_peered":"2023-07-14T18:14:17.935099+0000","last_unstale":"2023-07-14T18:14:49.541350+0000","last_undegraded":"2023-07-14T18:14:49.541350+0000","last_fullsized":"2023-07-14T18:14:49.541350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:04:06.339952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.943679+0000","last_change":"2023-07-14T18:14:17.930207+0000","last_active":"2023-07-14T18:14:48.943679+0000","last_peered":"2023-07-14T18:14:48.943679+0000","last_clean":"2023-07-14T18:14:48.943679+0000","last_became_active":"2023-07-14T18:14:17.929898+0000","last_became_peered":"2023-07-14T18:14:17.929898+0000","last_unstale":"2023-07-14T18:14:48.943679+0000","last_undegraded":"2023-07-14T18:14:48.943679+0000","last_fullsized":"2023-07-14T18:14:48.943679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:33:33.679907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541324+0000","last_change":"2023-07-14T18:14:17.928645+0000","last_active":"2023-07-14T18:14:49.541324+0000","last_peered":"2023-07-14T18:14:49.541324+0000","last_clean":"2023-07-14T18:14:49.541324+0000","last_became_active":"2023-07-14T18:14:17.928288+0000","last_became_peered":"2023-07-14T18:14:17.928288+0000","last_unstale":"2023-07-14T18:14:49.541324+0000","last_undegraded":"2023-07-14T18:14:49.541324+0000","last_fullsized":"2023-07-14T18:14:49.541324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:31:13.535796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.948678+0000","last_change":"2023-07-14T18:14:17.925888+0000","last_active":"2023-07-14T18:14:48.948678+0000","last_peered":"2023-07-14T18:14:48.948678+0000","last_clean":"2023-07-14T18:14:48.948678+0000","last_became_active":"2023-07-14T18:14:17.925580+0000","last_became_peered":"2023-07-14T18:14:17.925580+0000","last_unstale":"2023-07-14T18:14:48.948678+0000","last_undegraded":"2023-07-14T18:14:48.948678+0000","last_fullsized":"2023-07-14T18:14:48.948678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:28:42.610213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541751+0000","last_change":"2023-07-14T18:14:17.924440+0000","last_active":"2023-07-14T18:14:49.541751+0000","last_peered":"2023-07-14T18:14:49.541751+0000","last_clean":"2023-07-14T18:14:49.541751+0000","last_became_active":"2023-07-14T18:14:17.923839+0000","last_became_peered":"2023-07-14T18:14:17.923839+0000","last_unstale":"2023-07-14T18:14:49.541751+0000","last_undegraded":"2023-07-14T18:14:49.541751+0000","last_fullsized":"2023-07-14T18:14:49.541751+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:11:20.578050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944134+0000","last_change":"2023-07-14T18:14:17.931067+0000","last_active":"2023-07-14T18:14:48.944134+0000","last_peered":"2023-07-14T18:14:48.944134+0000","last_clean":"2023-07-14T18:14:48.944134+0000","last_became_active":"2023-07-14T18:14:17.930633+0000","last_became_peered":"2023-07-14T18:14:17.930633+0000","last_unstale":"2023-07-14T18:14:48.944134+0000","last_undegraded":"2023-07-14T18:14:48.944134+0000","last_fullsized":"2023-07-14T18:14:48.944134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:35:03.441365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.947103+0000","last_change":"2023-07-14T18:14:17.932770+0000","last_active":"2023-07-14T18:14:48.947103+0000","last_peered":"2023-07-14T18:14:48.947103+0000","last_clean":"2023-07-14T18:14:48.947103+0000","last_became_active":"2023-07-14T18:14:17.932438+0000","last_became_peered":"2023-07-14T18:14:17.932438+0000","last_unstale":"2023-07-14T18:14:48.947103+0000","last_undegraded":"2023-07-14T18:14:48.947103+0000","last_fullsized":"2023-07-14T18:14:48.947103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:45:58.878384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942853+0000","last_change":"2023-07-14T18:14:17.938213+0000","last_active":"2023-07-14T18:14:48.942853+0000","last_peered":"2023-07-14T18:14:48.942853+0000","last_clean":"2023-07-14T18:14:48.942853+0000","last_became_active":"2023-07-14T18:14:17.937565+0000","last_became_peered":"2023-07-14T18:14:17.937565+0000","last_unstale":"2023-07-14T18:14:48.942853+0000","last_undegraded":"2023-07-14T18:14:48.942853+0000","last_fullsized":"2023-07-14T18:14:48.942853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:18:33.559796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541023+0000","last_change":"2023-07-14T18:14:17.934677+0000","last_active":"2023-07-14T18:14:49.541023+0000","last_peered":"2023-07-14T18:14:49.541023+0000","last_clean":"2023-07-14T18:14:49.541023+0000","last_became_active":"2023-07-14T18:14:17.933622+0000","last_became_peered":"2023-07-14T18:14:17.933622+0000","last_unstale":"2023-07-14T18:14:49.541023+0000","last_undegraded":"2023-07-14T18:14:49.541023+0000","last_fullsized":"2023-07-14T18:14:49.541023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:44:58.693553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540682+0000","last_change":"2023-07-14T18:14:17.935317+0000","last_active":"2023-07-14T18:14:49.540682+0000","last_peered":"2023-07-14T18:14:49.540682+0000","last_clean":"2023-07-14T18:14:49.540682+0000","last_became_active":"2023-07-14T18:14:17.935034+0000","last_became_peered":"2023-07-14T18:14:17.935034+0000","last_unstale":"2023-07-14T18:14:49.540682+0000","last_undegraded":"2023-07-14T18:14:49.540682+0000","last_fullsized":"2023-07-14T18:14:49.540682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:45:23.409187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946950+0000","last_change":"2023-07-14T18:14:17.928222+0000","last_active":"2023-07-14T18:14:48.946950+0000","last_peered":"2023-07-14T18:14:48.946950+0000","last_clean":"2023-07-14T18:14:48.946950+0000","last_became_active":"2023-07-14T18:14:17.927964+0000","last_became_peered":"2023-07-14T18:14:17.927964+0000","last_unstale":"2023-07-14T18:14:48.946950+0000","last_undegraded":"2023-07-14T18:14:48.946950+0000","last_fullsized":"2023-07-14T18:14:48.946950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:44:57.639864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944940+0000","last_change":"2023-07-14T18:14:17.927276+0000","last_active":"2023-07-14T18:14:48.944940+0000","last_peered":"2023-07-14T18:14:48.944940+0000","last_clean":"2023-07-14T18:14:48.944940+0000","last_became_active":"2023-07-14T18:14:17.926952+0000","last_became_peered":"2023-07-14T18:14:17.926952+0000","last_unstale":"2023-07-14T18:14:48.944940+0000","last_undegraded":"2023-07-14T18:14:48.944940+0000","last_fullsized":"2023-07-14T18:14:48.944940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:48:53.419369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945947+0000","last_change":"2023-07-14T18:14:17.937364+0000","last_active":"2023-07-14T18:14:48.945947+0000","last_peered":"2023-07-14T18:14:48.945947+0000","last_clean":"2023-07-14T18:14:48.945947+0000","last_became_active":"2023-07-14T18:14:17.937014+0000","last_became_peered":"2023-07-14T18:14:17.937014+0000","last_unstale":"2023-07-14T18:14:48.945947+0000","last_undegraded":"2023-07-14T18:14:48.945947+0000","last_fullsized":"2023-07-14T18:14:48.945947+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:57:26.423684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541653+0000","last_change":"2023-07-14T18:14:17.920358+0000","last_active":"2023-07-14T18:14:49.541653+0000","last_peered":"2023-07-14T18:14:49.541653+0000","last_clean":"2023-07-14T18:14:49.541653+0000","last_became_active":"2023-07-14T18:14:17.920042+0000","last_became_peered":"2023-07-14T18:14:17.920042+0000","last_unstale":"2023-07-14T18:14:49.541653+0000","last_undegraded":"2023-07-14T18:14:49.541653+0000","last_fullsized":"2023-07-14T18:14:49.541653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:53:36.975517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541158+0000","last_change":"2023-07-14T18:14:22.045376+0000","last_active":"2023-07-14T18:14:49.541158+0000","last_peered":"2023-07-14T18:14:49.541158+0000","last_clean":"2023-07-14T18:14:49.541158+0000","last_became_active":"2023-07-14T18:14:22.044984+0000","last_became_peered":"2023-07-14T18:14:22.044984+0000","last_unstale":"2023-07-14T18:14:49.541158+0000","last_undegraded":"2023-07-14T18:14:49.541158+0000","last_fullsized":"2023-07-14T18:14:49.541158+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:47:15.523409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.894895+0000","last_change":"2023-07-14T18:14:48.954270+0000","last_active":"2023-07-14T18:14:49.894895+0000","last_peered":"2023-07-14T18:14:49.894895+0000","last_clean":"2023-07-14T18:14:49.894895+0000","last_became_active":"2023-07-14T18:14:48.947868+0000","last_became_peered":"2023-07-14T18:14:48.947868+0000","last_unstale":"2023-07-14T18:14:49.894895+0000","last_undegraded":"2023-07-14T18:14:49.894895+0000","last_fullsized":"2023-07-14T18:14:49.894895+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:13:41.862407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540866+0000","last_change":"2023-07-14T18:14:17.925822+0000","last_active":"2023-07-14T18:14:49.540866+0000","last_peered":"2023-07-14T18:14:49.540866+0000","last_clean":"2023-07-14T18:14:49.540866+0000","last_became_active":"2023-07-14T18:14:17.925508+0000","last_became_peered":"2023-07-14T18:14:17.925508+0000","last_unstale":"2023-07-14T18:14:49.540866+0000","last_undegraded":"2023-07-14T18:14:49.540866+0000","last_fullsized":"2023-07-14T18:14:49.540866+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:19:54.558854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946499+0000","last_change":"2023-07-14T18:14:22.043098+0000","last_active":"2023-07-14T18:14:48.946499+0000","last_peered":"2023-07-14T18:14:48.946499+0000","last_clean":"2023-07-14T18:14:48.946499+0000","last_became_active":"2023-07-14T18:14:22.042517+0000","last_became_peered":"2023-07-14T18:14:22.042517+0000","last_unstale":"2023-07-14T18:14:48.946499+0000","last_undegraded":"2023-07-14T18:14:48.946499+0000","last_fullsized":"2023-07-14T18:14:48.946499+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:33:54.491558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541428+0000","last_change":"2023-07-14T18:14:16.927612+0000","last_active":"2023-07-14T18:14:49.541428+0000","last_peered":"2023-07-14T18:14:49.541428+0000","last_clean":"2023-07-14T18:14:49.541428+0000","last_became_active":"2023-07-14T18:14:16.927284+0000","last_became_peered":"2023-07-14T18:14:16.927284+0000","last_unstale":"2023-07-14T18:14:49.541428+0000","last_undegraded":"2023-07-14T18:14:49.541428+0000","last_fullsized":"2023-07-14T18:14:49.541428+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:56:37.293734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540962+0000","last_change":"2023-07-14T18:14:17.930624+0000","last_active":"2023-07-14T18:14:49.540962+0000","last_peered":"2023-07-14T18:14:49.540962+0000","last_clean":"2023-07-14T18:14:49.540962+0000","last_became_active":"2023-07-14T18:14:17.930273+0000","last_became_peered":"2023-07-14T18:14:17.930273+0000","last_unstale":"2023-07-14T18:14:49.540962+0000","last_undegraded":"2023-07-14T18:14:49.540962+0000","last_fullsized":"2023-07-14T18:14:49.540962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:19:40.573682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.543208+0000","last_change":"2023-07-14T18:14:49.543208+0000","last_active":"2023-07-14T18:14:49.543208+0000","last_peered":"2023-07-14T18:14:49.543208+0000","last_clean":"2023-07-14T18:14:49.543208+0000","last_became_active":"2023-07-14T18:14:49.542685+0000","last_became_peered":"2023-07-14T18:14:49.542685+0000","last_unstale":"2023-07-14T18:14:49.543208+0000","last_undegraded":"2023-07-14T18:14:49.543208+0000","last_fullsized":"2023-07-14T18:14:49.543208+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:32:12.710336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.894625+0000","last_change":"2023-07-14T18:14:16.926185+0000","last_active":"2023-07-14T18:14:49.894625+0000","last_peered":"2023-07-14T18:14:49.894625+0000","last_clean":"2023-07-14T18:14:49.894625+0000","last_became_active":"2023-07-14T18:14:16.925874+0000","last_became_peered":"2023-07-14T18:14:16.925874+0000","last_unstale":"2023-07-14T18:14:49.894625+0000","last_undegraded":"2023-07-14T18:14:49.894625+0000","last_fullsized":"2023-07-14T18:14:49.894625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:26:07.280801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.947867+0000","last_change":"2023-07-14T18:14:17.936601+0000","last_active":"2023-07-14T18:14:48.947867+0000","last_peered":"2023-07-14T18:14:48.947867+0000","last_clean":"2023-07-14T18:14:48.947867+0000","last_became_active":"2023-07-14T18:14:17.936291+0000","last_became_peered":"2023-07-14T18:14:17.936291+0000","last_unstale":"2023-07-14T18:14:48.947867+0000","last_undegraded":"2023-07-14T18:14:48.947867+0000","last_fullsized":"2023-07-14T18:14:48.947867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:50:25.290580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946635+0000","last_change":"2023-07-14T18:14:22.042952+0000","last_active":"2023-07-14T18:14:48.946635+0000","last_peered":"2023-07-14T18:14:48.946635+0000","last_clean":"2023-07-14T18:14:48.946635+0000","last_became_active":"2023-07-14T18:14:22.042256+0000","last_became_peered":"2023-07-14T18:14:22.042256+0000","last_unstale":"2023-07-14T18:14:48.946635+0000","last_undegraded":"2023-07-14T18:14:48.946635+0000","last_fullsized":"2023-07-14T18:14:48.946635+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:20:29.914375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945839+0000","last_change":"2023-07-14T18:14:16.920902+0000","last_active":"2023-07-14T18:14:48.945839+0000","last_peered":"2023-07-14T18:14:48.945839+0000","last_clean":"2023-07-14T18:14:48.945839+0000","last_became_active":"2023-07-14T18:14:16.920554+0000","last_became_peered":"2023-07-14T18:14:16.920554+0000","last_unstale":"2023-07-14T18:14:48.945839+0000","last_undegraded":"2023-07-14T18:14:48.945839+0000","last_fullsized":"2023-07-14T18:14:48.945839+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:05:37.194027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945093+0000","last_change":"2023-07-14T18:14:17.925005+0000","last_active":"2023-07-14T18:14:48.945093+0000","last_peered":"2023-07-14T18:14:48.945093+0000","last_clean":"2023-07-14T18:14:48.945093+0000","last_became_active":"2023-07-14T18:14:17.924723+0000","last_became_peered":"2023-07-14T18:14:17.924723+0000","last_unstale":"2023-07-14T18:14:48.945093+0000","last_undegraded":"2023-07-14T18:14:48.945093+0000","last_fullsized":"2023-07-14T18:14:48.945093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:34:45.456102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541394+0000","last_change":"2023-07-14T18:14:22.038482+0000","last_active":"2023-07-14T18:14:49.541394+0000","last_peered":"2023-07-14T18:14:49.541394+0000","last_clean":"2023-07-14T18:14:49.541394+0000","last_became_active":"2023-07-14T18:14:22.038059+0000","last_became_peered":"2023-07-14T18:14:22.038059+0000","last_unstale":"2023-07-14T18:14:49.541394+0000","last_undegraded":"2023-07-14T18:14:49.541394+0000","last_fullsized":"2023-07-14T18:14:49.541394+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:40:04.711668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":88,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:15:34.953612+0000","last_change":"2023-07-14T18:14:48.949585+0000","last_active":"2023-07-14T18:15:34.953612+0000","last_peered":"2023-07-14T18:15:34.953612+0000","last_clean":"2023-07-14T18:15:34.953612+0000","last_became_active":"2023-07-14T18:14:48.949106+0000","last_became_peered":"2023-07-14T18:14:48.949106+0000","last_unstale":"2023-07-14T18:15:34.953612+0000","last_undegraded":"2023-07-14T18:15:34.953612+0000","last_fullsized":"2023-07-14T18:15:34.953612+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:27:44.232817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945805+0000","last_change":"2023-07-14T18:14:17.934894+0000","last_active":"2023-07-14T18:14:48.945805+0000","last_peered":"2023-07-14T18:14:48.945805+0000","last_clean":"2023-07-14T18:14:48.945805+0000","last_became_active":"2023-07-14T18:14:17.934651+0000","last_became_peered":"2023-07-14T18:14:17.934651+0000","last_unstale":"2023-07-14T18:14:48.945805+0000","last_undegraded":"2023-07-14T18:14:48.945805+0000","last_fullsized":"2023-07-14T18:14:48.945805+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:07:18.984143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541406+0000","last_change":"2023-07-14T18:14:22.043611+0000","last_active":"2023-07-14T18:14:49.541406+0000","last_peered":"2023-07-14T18:14:49.541406+0000","last_clean":"2023-07-14T18:14:49.541406+0000","last_became_active":"2023-07-14T18:14:22.043101+0000","last_became_peered":"2023-07-14T18:14:22.043101+0000","last_unstale":"2023-07-14T18:14:49.541406+0000","last_undegraded":"2023-07-14T18:14:49.541406+0000","last_fullsized":"2023-07-14T18:14:49.541406+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:09:27.645400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.894491+0000","last_change":"2023-07-14T18:14:16.925321+0000","last_active":"2023-07-14T18:14:49.894491+0000","last_peered":"2023-07-14T18:14:49.894491+0000","last_clean":"2023-07-14T18:14:49.894491+0000","last_became_active":"2023-07-14T18:14:16.925016+0000","last_became_peered":"2023-07-14T18:14:16.925016+0000","last_unstale":"2023-07-14T18:14:49.894491+0000","last_undegraded":"2023-07-14T18:14:49.894491+0000","last_fullsized":"2023-07-14T18:14:49.894491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:51:09.710945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944916+0000","last_change":"2023-07-14T18:14:17.928838+0000","last_active":"2023-07-14T18:14:48.944916+0000","last_peered":"2023-07-14T18:14:48.944916+0000","last_clean":"2023-07-14T18:14:48.944916+0000","last_became_active":"2023-07-14T18:14:17.928511+0000","last_became_peered":"2023-07-14T18:14:17.928511+0000","last_unstale":"2023-07-14T18:14:48.944916+0000","last_undegraded":"2023-07-14T18:14:48.944916+0000","last_fullsized":"2023-07-14T18:14:48.944916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:25:59.076594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944791+0000","last_change":"2023-07-14T18:14:22.043182+0000","last_active":"2023-07-14T18:14:48.944791+0000","last_peered":"2023-07-14T18:14:48.944791+0000","last_clean":"2023-07-14T18:14:48.944791+0000","last_became_active":"2023-07-14T18:14:22.042779+0000","last_became_peered":"2023-07-14T18:14:22.042779+0000","last_unstale":"2023-07-14T18:14:48.944791+0000","last_undegraded":"2023-07-14T18:14:48.944791+0000","last_fullsized":"2023-07-14T18:14:48.944791+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:15:03.016184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.895882+0000","last_change":"2023-07-14T18:14:16.935321+0000","last_active":"2023-07-14T18:14:49.895882+0000","last_peered":"2023-07-14T18:14:49.895882+0000","last_clean":"2023-07-14T18:14:49.895882+0000","last_became_active":"2023-07-14T18:14:16.934494+0000","last_became_peered":"2023-07-14T18:14:16.934494+0000","last_unstale":"2023-07-14T18:14:49.895882+0000","last_undegraded":"2023-07-14T18:14:49.895882+0000","last_fullsized":"2023-07-14T18:14:49.895882+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:00:50.926893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541135+0000","last_change":"2023-07-14T18:14:17.922139+0000","last_active":"2023-07-14T18:14:49.541135+0000","last_peered":"2023-07-14T18:14:49.541135+0000","last_clean":"2023-07-14T18:14:49.541135+0000","last_became_active":"2023-07-14T18:14:17.921829+0000","last_became_peered":"2023-07-14T18:14:17.921829+0000","last_unstale":"2023-07-14T18:14:49.541135+0000","last_undegraded":"2023-07-14T18:14:49.541135+0000","last_fullsized":"2023-07-14T18:14:49.541135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:16:28.099919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540988+0000","last_change":"2023-07-14T18:14:22.033625+0000","last_active":"2023-07-14T18:14:49.540988+0000","last_peered":"2023-07-14T18:14:49.540988+0000","last_clean":"2023-07-14T18:14:49.540988+0000","last_became_active":"2023-07-14T18:14:22.033253+0000","last_became_peered":"2023-07-14T18:14:22.033253+0000","last_unstale":"2023-07-14T18:14:49.540988+0000","last_undegraded":"2023-07-14T18:14:49.540988+0000","last_fullsized":"2023-07-14T18:14:49.540988+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:16:24.641370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942449+0000","last_change":"2023-07-14T18:14:16.923864+0000","last_active":"2023-07-14T18:14:48.942449+0000","last_peered":"2023-07-14T18:14:48.942449+0000","last_clean":"2023-07-14T18:14:48.942449+0000","last_became_active":"2023-07-14T18:14:16.923546+0000","last_became_peered":"2023-07-14T18:14:16.923546+0000","last_unstale":"2023-07-14T18:14:48.942449+0000","last_undegraded":"2023-07-14T18:14:48.942449+0000","last_fullsized":"2023-07-14T18:14:48.942449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:31:33.868703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945453+0000","last_change":"2023-07-14T18:14:17.936473+0000","last_active":"2023-07-14T18:14:48.945453+0000","last_peered":"2023-07-14T18:14:48.945453+0000","last_clean":"2023-07-14T18:14:48.945453+0000","last_became_active":"2023-07-14T18:14:17.935387+0000","last_became_peered":"2023-07-14T18:14:17.935387+0000","last_unstale":"2023-07-14T18:14:48.945453+0000","last_undegraded":"2023-07-14T18:14:48.945453+0000","last_fullsized":"2023-07-14T18:14:48.945453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:09:31.140915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541512+0000","last_change":"2023-07-14T18:14:22.041544+0000","last_active":"2023-07-14T18:14:49.541512+0000","last_peered":"2023-07-14T18:14:49.541512+0000","last_clean":"2023-07-14T18:14:49.541512+0000","last_became_active":"2023-07-14T18:14:22.041183+0000","last_became_peered":"2023-07-14T18:14:22.041183+0000","last_unstale":"2023-07-14T18:14:49.541512+0000","last_undegraded":"2023-07-14T18:14:49.541512+0000","last_fullsized":"2023-07-14T18:14:49.541512+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:24:18.995001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":70,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:15:30.946657+0000","last_change":"2023-07-14T18:14:16.928860+0000","last_active":"2023-07-14T18:15:30.946657+0000","last_peered":"2023-07-14T18:15:30.946657+0000","last_clean":"2023-07-14T18:15:30.946657+0000","last_became_active":"2023-07-14T18:14:16.928567+0000","last_became_peered":"2023-07-14T18:14:16.928567+0000","last_unstale":"2023-07-14T18:15:30.946657+0000","last_undegraded":"2023-07-14T18:15:30.946657+0000","last_fullsized":"2023-07-14T18:15:30.946657+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:44:11.774963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.948806+0000","last_change":"2023-07-14T18:14:17.938456+0000","last_active":"2023-07-14T18:14:48.948806+0000","last_peered":"2023-07-14T18:14:48.948806+0000","last_clean":"2023-07-14T18:14:48.948806+0000","last_became_active":"2023-07-14T18:14:17.938086+0000","last_became_peered":"2023-07-14T18:14:17.938086+0000","last_unstale":"2023-07-14T18:14:48.948806+0000","last_undegraded":"2023-07-14T18:14:48.948806+0000","last_fullsized":"2023-07-14T18:14:48.948806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:16:01.795875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945337+0000","last_change":"2023-07-14T18:14:22.042785+0000","last_active":"2023-07-14T18:14:48.945337+0000","last_peered":"2023-07-14T18:14:48.945337+0000","last_clean":"2023-07-14T18:14:48.945337+0000","last_became_active":"2023-07-14T18:14:22.042346+0000","last_became_peered":"2023-07-14T18:14:22.042346+0000","last_unstale":"2023-07-14T18:14:48.945337+0000","last_undegraded":"2023-07-14T18:14:48.945337+0000","last_fullsized":"2023-07-14T18:14:48.945337+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:26:17.301312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944005+0000","last_change":"2023-07-14T18:14:16.921168+0000","last_active":"2023-07-14T18:14:48.944005+0000","last_peered":"2023-07-14T18:14:48.944005+0000","last_clean":"2023-07-14T18:14:48.944005+0000","last_became_active":"2023-07-14T18:14:16.920808+0000","last_became_peered":"2023-07-14T18:14:16.920808+0000","last_unstale":"2023-07-14T18:14:48.944005+0000","last_undegraded":"2023-07-14T18:14:48.944005+0000","last_fullsized":"2023-07-14T18:14:48.944005+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:41:57.870102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541090+0000","last_change":"2023-07-14T18:14:17.935100+0000","last_active":"2023-07-14T18:14:49.541090+0000","last_peered":"2023-07-14T18:14:49.541090+0000","last_clean":"2023-07-14T18:14:49.541090+0000","last_became_active":"2023-07-14T18:14:17.934794+0000","last_became_peered":"2023-07-14T18:14:17.934794+0000","last_unstale":"2023-07-14T18:14:49.541090+0000","last_undegraded":"2023-07-14T18:14:49.541090+0000","last_fullsized":"2023-07-14T18:14:49.541090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:37:28.349127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.948310+0000","last_change":"2023-07-14T18:14:22.043495+0000","last_active":"2023-07-14T18:14:48.948310+0000","last_peered":"2023-07-14T18:14:48.948310+0000","last_clean":"2023-07-14T18:14:48.948310+0000","last_became_active":"2023-07-14T18:14:22.042686+0000","last_became_peered":"2023-07-14T18:14:22.042686+0000","last_unstale":"2023-07-14T18:14:48.948310+0000","last_undegraded":"2023-07-14T18:14:48.948310+0000","last_fullsized":"2023-07-14T18:14:48.948310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:16:19.904966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944994+0000","last_change":"2023-07-14T18:14:16.926006+0000","last_active":"2023-07-14T18:14:48.944994+0000","last_peered":"2023-07-14T18:14:48.944994+0000","last_clean":"2023-07-14T18:14:48.944994+0000","last_became_active":"2023-07-14T18:14:16.925719+0000","last_became_peered":"2023-07-14T18:14:16.925719+0000","last_unstale":"2023-07-14T18:14:48.944994+0000","last_undegraded":"2023-07-14T18:14:48.944994+0000","last_fullsized":"2023-07-14T18:14:48.944994+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:03:27.517349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.948103+0000","last_change":"2023-07-14T18:14:17.934155+0000","last_active":"2023-07-14T18:14:48.948103+0000","last_peered":"2023-07-14T18:14:48.948103+0000","last_clean":"2023-07-14T18:14:48.948103+0000","last_became_active":"2023-07-14T18:14:17.933756+0000","last_became_peered":"2023-07-14T18:14:17.933756+0000","last_unstale":"2023-07-14T18:14:48.948103+0000","last_undegraded":"2023-07-14T18:14:48.948103+0000","last_fullsized":"2023-07-14T18:14:48.948103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:04:42.915865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945422+0000","last_change":"2023-07-14T18:14:22.039198+0000","last_active":"2023-07-14T18:14:48.945422+0000","last_peered":"2023-07-14T18:14:48.945422+0000","last_clean":"2023-07-14T18:14:48.945422+0000","last_became_active":"2023-07-14T18:14:22.038842+0000","last_became_peered":"2023-07-14T18:14:22.038842+0000","last_unstale":"2023-07-14T18:14:48.945422+0000","last_undegraded":"2023-07-14T18:14:48.945422+0000","last_fullsized":"2023-07-14T18:14:48.945422+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:32:54.838956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'33","reported_seq":151,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:15:31.953558+0000","last_change":"2023-07-14T18:14:16.920918+0000","last_active":"2023-07-14T18:15:31.953558+0000","last_peered":"2023-07-14T18:15:31.953558+0000","last_clean":"2023-07-14T18:15:31.953558+0000","last_became_active":"2023-07-14T18:14:16.920572+0000","last_became_peered":"2023-07-14T18:14:16.920572+0000","last_unstale":"2023-07-14T18:15:31.953558+0000","last_undegraded":"2023-07-14T18:15:31.953558+0000","last_fullsized":"2023-07-14T18:15:31.953558+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:55:35.900625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59808,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":94,"num_read_kb":105,"num_write":34,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945963+0000","last_change":"2023-07-14T18:14:17.929413+0000","last_active":"2023-07-14T18:14:48.945963+0000","last_peered":"2023-07-14T18:14:48.945963+0000","last_clean":"2023-07-14T18:14:48.945963+0000","last_became_active":"2023-07-14T18:14:17.929064+0000","last_became_peered":"2023-07-14T18:14:17.929064+0000","last_unstale":"2023-07-14T18:14:48.945963+0000","last_undegraded":"2023-07-14T18:14:48.945963+0000","last_fullsized":"2023-07-14T18:14:48.945963+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:29:27.786039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945372+0000","last_change":"2023-07-14T18:14:22.041983+0000","last_active":"2023-07-14T18:14:48.945372+0000","last_peered":"2023-07-14T18:14:48.945372+0000","last_clean":"2023-07-14T18:14:48.945372+0000","last_became_active":"2023-07-14T18:14:22.041553+0000","last_became_peered":"2023-07-14T18:14:22.041553+0000","last_unstale":"2023-07-14T18:14:48.945372+0000","last_undegraded":"2023-07-14T18:14:48.945372+0000","last_fullsized":"2023-07-14T18:14:48.945372+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:59:09.096024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.892826+0000","last_change":"2023-07-14T18:14:16.925444+0000","last_active":"2023-07-14T18:14:49.892826+0000","last_peered":"2023-07-14T18:14:49.892826+0000","last_clean":"2023-07-14T18:14:49.892826+0000","last_became_active":"2023-07-14T18:14:16.925138+0000","last_became_peered":"2023-07-14T18:14:16.925138+0000","last_unstale":"2023-07-14T18:14:49.892826+0000","last_undegraded":"2023-07-14T18:14:49.892826+0000","last_fullsized":"2023-07-14T18:14:49.892826+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:10:33.163171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942332+0000","last_change":"2023-07-14T18:14:17.922351+0000","last_active":"2023-07-14T18:14:48.942332+0000","last_peered":"2023-07-14T18:14:48.942332+0000","last_clean":"2023-07-14T18:14:48.942332+0000","last_became_active":"2023-07-14T18:14:17.921973+0000","last_became_peered":"2023-07-14T18:14:17.921973+0000","last_unstale":"2023-07-14T18:14:48.942332+0000","last_undegraded":"2023-07-14T18:14:48.942332+0000","last_fullsized":"2023-07-14T18:14:48.942332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:16:09.934284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945943+0000","last_change":"2023-07-14T18:14:22.048274+0000","last_active":"2023-07-14T18:14:48.945943+0000","last_peered":"2023-07-14T18:14:48.945943+0000","last_clean":"2023-07-14T18:14:48.945943+0000","last_became_active":"2023-07-14T18:14:22.047946+0000","last_became_peered":"2023-07-14T18:14:22.047946+0000","last_unstale":"2023-07-14T18:14:48.945943+0000","last_undegraded":"2023-07-14T18:14:48.945943+0000","last_fullsized":"2023-07-14T18:14:48.945943+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:29:47.580140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944169+0000","last_change":"2023-07-14T18:14:16.933566+0000","last_active":"2023-07-14T18:14:48.944169+0000","last_peered":"2023-07-14T18:14:48.944169+0000","last_clean":"2023-07-14T18:14:48.944169+0000","last_became_active":"2023-07-14T18:14:16.933300+0000","last_became_peered":"2023-07-14T18:14:16.933300+0000","last_unstale":"2023-07-14T18:14:48.944169+0000","last_undegraded":"2023-07-14T18:14:48.944169+0000","last_fullsized":"2023-07-14T18:14:48.944169+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:05:57.086039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944233+0000","last_change":"2023-07-14T18:14:17.928521+0000","last_active":"2023-07-14T18:14:48.944233+0000","last_peered":"2023-07-14T18:14:48.944233+0000","last_clean":"2023-07-14T18:14:48.944233+0000","last_became_active":"2023-07-14T18:14:17.928025+0000","last_became_peered":"2023-07-14T18:14:17.928025+0000","last_unstale":"2023-07-14T18:14:48.944233+0000","last_undegraded":"2023-07-14T18:14:48.944233+0000","last_fullsized":"2023-07-14T18:14:48.944233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:21:23.554363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945666+0000","last_change":"2023-07-14T18:14:22.046820+0000","last_active":"2023-07-14T18:14:48.945666+0000","last_peered":"2023-07-14T18:14:48.945666+0000","last_clean":"2023-07-14T18:14:48.945666+0000","last_became_active":"2023-07-14T18:14:22.046440+0000","last_became_peered":"2023-07-14T18:14:22.046440+0000","last_unstale":"2023-07-14T18:14:48.945666+0000","last_undegraded":"2023-07-14T18:14:48.945666+0000","last_fullsized":"2023-07-14T18:14:48.945666+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:46:22.890726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":76,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:15:33.960627+0000","last_change":"2023-07-14T18:14:16.922075+0000","last_active":"2023-07-14T18:15:33.960627+0000","last_peered":"2023-07-14T18:15:33.960627+0000","last_clean":"2023-07-14T18:15:33.960627+0000","last_became_active":"2023-07-14T18:14:16.921570+0000","last_became_peered":"2023-07-14T18:14:16.921570+0000","last_unstale":"2023-07-14T18:15:33.960627+0000","last_undegraded":"2023-07-14T18:15:33.960627+0000","last_fullsized":"2023-07-14T18:15:33.960627+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:08:37.059574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.943804+0000","last_change":"2023-07-14T18:14:17.936439+0000","last_active":"2023-07-14T18:14:48.943804+0000","last_peered":"2023-07-14T18:14:48.943804+0000","last_clean":"2023-07-14T18:14:48.943804+0000","last_became_active":"2023-07-14T18:14:17.936138+0000","last_became_peered":"2023-07-14T18:14:17.936138+0000","last_unstale":"2023-07-14T18:14:48.943804+0000","last_undegraded":"2023-07-14T18:14:48.943804+0000","last_fullsized":"2023-07-14T18:14:48.943804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:58:02.987288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.543302+0000","last_change":"2023-07-14T18:14:49.543302+0000","last_active":"2023-07-14T18:14:49.543302+0000","last_peered":"2023-07-14T18:14:49.543302+0000","last_clean":"2023-07-14T18:14:49.543302+0000","last_became_active":"2023-07-14T18:14:49.542768+0000","last_became_peered":"2023-07-14T18:14:49.542768+0000","last_unstale":"2023-07-14T18:14:49.543302+0000","last_undegraded":"2023-07-14T18:14:49.543302+0000","last_fullsized":"2023-07-14T18:14:49.543302+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:14:41.906790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.896057+0000","last_change":"2023-07-14T18:14:16.926417+0000","last_active":"2023-07-14T18:14:49.896057+0000","last_peered":"2023-07-14T18:14:49.896057+0000","last_clean":"2023-07-14T18:14:49.896057+0000","last_became_active":"2023-07-14T18:14:16.926101+0000","last_became_peered":"2023-07-14T18:14:16.926101+0000","last_unstale":"2023-07-14T18:14:49.896057+0000","last_undegraded":"2023-07-14T18:14:49.896057+0000","last_fullsized":"2023-07-14T18:14:49.896057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:09:38.329843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.950979+0000","last_change":"2023-07-14T18:14:48.950649+0000","last_active":"2023-07-14T18:14:48.950979+0000","last_peered":"2023-07-14T18:14:48.950979+0000","last_clean":"2023-07-14T18:14:48.950979+0000","last_became_active":"2023-07-14T18:14:48.950303+0000","last_became_peered":"2023-07-14T18:14:48.950303+0000","last_unstale":"2023-07-14T18:14:48.950979+0000","last_undegraded":"2023-07-14T18:14:48.950979+0000","last_fullsized":"2023-07-14T18:14:48.950979+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:40:15.252731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942609+0000","last_change":"2023-07-14T18:14:22.042171+0000","last_active":"2023-07-14T18:14:48.942609+0000","last_peered":"2023-07-14T18:14:48.942609+0000","last_clean":"2023-07-14T18:14:48.942609+0000","last_became_active":"2023-07-14T18:14:22.041692+0000","last_became_peered":"2023-07-14T18:14:22.041692+0000","last_unstale":"2023-07-14T18:14:48.942609+0000","last_undegraded":"2023-07-14T18:14:48.942609+0000","last_fullsized":"2023-07-14T18:14:48.942609+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:03:01.650548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.881565+0000","last_change":"2023-07-14T18:14:16.922883+0000","last_active":"2023-07-14T18:14:49.881565+0000","last_peered":"2023-07-14T18:14:49.881565+0000","last_clean":"2023-07-14T18:14:49.881565+0000","last_became_active":"2023-07-14T18:14:16.922501+0000","last_became_peered":"2023-07-14T18:14:16.922501+0000","last_unstale":"2023-07-14T18:14:49.881565+0000","last_undegraded":"2023-07-14T18:14:49.881565+0000","last_fullsized":"2023-07-14T18:14:49.881565+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:51:12.741737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946509+0000","last_change":"2023-07-14T18:14:17.922899+0000","last_active":"2023-07-14T18:14:48.946509+0000","last_peered":"2023-07-14T18:14:48.946509+0000","last_clean":"2023-07-14T18:14:48.946509+0000","last_became_active":"2023-07-14T18:14:17.922608+0000","last_became_peered":"2023-07-14T18:14:17.922608+0000","last_unstale":"2023-07-14T18:14:48.946509+0000","last_undegraded":"2023-07-14T18:14:48.946509+0000","last_fullsized":"2023-07-14T18:14:48.946509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:53:19.741997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945671+0000","last_change":"2023-07-14T18:14:22.327484+0000","last_active":"2023-07-14T18:14:48.945671+0000","last_peered":"2023-07-14T18:14:48.945671+0000","last_clean":"2023-07-14T18:14:48.945671+0000","last_became_active":"2023-07-14T18:14:22.327010+0000","last_became_peered":"2023-07-14T18:14:22.327010+0000","last_unstale":"2023-07-14T18:14:48.945671+0000","last_undegraded":"2023-07-14T18:14:48.945671+0000","last_fullsized":"2023-07-14T18:14:48.945671+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:32:16.310729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945788+0000","last_change":"2023-07-14T18:14:16.930606+0000","last_active":"2023-07-14T18:14:48.945788+0000","last_peered":"2023-07-14T18:14:48.945788+0000","last_clean":"2023-07-14T18:14:48.945788+0000","last_became_active":"2023-07-14T18:14:16.930175+0000","last_became_peered":"2023-07-14T18:14:16.930175+0000","last_unstale":"2023-07-14T18:14:48.945788+0000","last_undegraded":"2023-07-14T18:14:48.945788+0000","last_fullsized":"2023-07-14T18:14:48.945788+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:39:16.156926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541700+0000","last_change":"2023-07-14T18:14:17.931962+0000","last_active":"2023-07-14T18:14:49.541700+0000","last_peered":"2023-07-14T18:14:49.541700+0000","last_clean":"2023-07-14T18:14:49.541700+0000","last_became_active":"2023-07-14T18:14:17.931621+0000","last_became_peered":"2023-07-14T18:14:17.931621+0000","last_unstale":"2023-07-14T18:14:49.541700+0000","last_undegraded":"2023-07-14T18:14:49.541700+0000","last_fullsized":"2023-07-14T18:14:49.541700+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:32:19.781881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942854+0000","last_change":"2023-07-14T18:14:22.038977+0000","last_active":"2023-07-14T18:14:48.942854+0000","last_peered":"2023-07-14T18:14:48.942854+0000","last_clean":"2023-07-14T18:14:48.942854+0000","last_became_active":"2023-07-14T18:14:22.038515+0000","last_became_peered":"2023-07-14T18:14:22.038515+0000","last_unstale":"2023-07-14T18:14:48.942854+0000","last_undegraded":"2023-07-14T18:14:48.942854+0000","last_fullsized":"2023-07-14T18:14:48.942854+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:40:29.285675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.943866+0000","last_change":"2023-07-14T18:14:16.925457+0000","last_active":"2023-07-14T18:14:48.943866+0000","last_peered":"2023-07-14T18:14:48.943866+0000","last_clean":"2023-07-14T18:14:48.943866+0000","last_became_active":"2023-07-14T18:14:16.925150+0000","last_became_peered":"2023-07-14T18:14:16.925150+0000","last_unstale":"2023-07-14T18:14:48.943866+0000","last_undegraded":"2023-07-14T18:14:48.943866+0000","last_fullsized":"2023-07-14T18:14:48.943866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:46:38.768309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541290+0000","last_change":"2023-07-14T18:14:17.926492+0000","last_active":"2023-07-14T18:14:49.541290+0000","last_peered":"2023-07-14T18:14:49.541290+0000","last_clean":"2023-07-14T18:14:49.541290+0000","last_became_active":"2023-07-14T18:14:17.926218+0000","last_became_peered":"2023-07-14T18:14:17.926218+0000","last_unstale":"2023-07-14T18:14:49.541290+0000","last_undegraded":"2023-07-14T18:14:49.541290+0000","last_fullsized":"2023-07-14T18:14:49.541290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:08:20.317990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942594+0000","last_change":"2023-07-14T18:14:22.327242+0000","last_active":"2023-07-14T18:14:48.942594+0000","last_peered":"2023-07-14T18:14:48.942594+0000","last_clean":"2023-07-14T18:14:48.942594+0000","last_became_active":"2023-07-14T18:14:22.326787+0000","last_became_peered":"2023-07-14T18:14:22.326787+0000","last_unstale":"2023-07-14T18:14:48.942594+0000","last_undegraded":"2023-07-14T18:14:48.942594+0000","last_fullsized":"2023-07-14T18:14:48.942594+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:08:03.165732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942605+0000","last_change":"2023-07-14T18:14:17.925035+0000","last_active":"2023-07-14T18:14:48.942605+0000","last_peered":"2023-07-14T18:14:48.942605+0000","last_clean":"2023-07-14T18:14:48.942605+0000","last_became_active":"2023-07-14T18:14:17.924713+0000","last_became_peered":"2023-07-14T18:14:17.924713+0000","last_unstale":"2023-07-14T18:14:48.942605+0000","last_undegraded":"2023-07-14T18:14:48.942605+0000","last_fullsized":"2023-07-14T18:14:48.942605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:40:51.683089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":73,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:15:31.955472+0000","last_change":"2023-07-14T18:14:16.935561+0000","last_active":"2023-07-14T18:15:31.955472+0000","last_peered":"2023-07-14T18:15:31.955472+0000","last_clean":"2023-07-14T18:15:31.955472+0000","last_became_active":"2023-07-14T18:14:16.935022+0000","last_became_peered":"2023-07-14T18:14:16.935022+0000","last_unstale":"2023-07-14T18:15:31.955472+0000","last_undegraded":"2023-07-14T18:15:31.955472+0000","last_fullsized":"2023-07-14T18:15:31.955472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:31:09.667735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.943916+0000","last_change":"2023-07-14T18:14:22.043818+0000","last_active":"2023-07-14T18:14:48.943916+0000","last_peered":"2023-07-14T18:14:48.943916+0000","last_clean":"2023-07-14T18:14:48.943916+0000","last_became_active":"2023-07-14T18:14:22.043449+0000","last_became_peered":"2023-07-14T18:14:22.043449+0000","last_unstale":"2023-07-14T18:14:48.943916+0000","last_undegraded":"2023-07-14T18:14:48.943916+0000","last_fullsized":"2023-07-14T18:14:48.943916+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:25:52.282417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540949+0000","last_change":"2023-07-14T18:14:17.930211+0000","last_active":"2023-07-14T18:14:49.540949+0000","last_peered":"2023-07-14T18:14:49.540949+0000","last_clean":"2023-07-14T18:14:49.540949+0000","last_became_active":"2023-07-14T18:14:17.929851+0000","last_became_peered":"2023-07-14T18:14:17.929851+0000","last_unstale":"2023-07-14T18:14:49.540949+0000","last_undegraded":"2023-07-14T18:14:49.540949+0000","last_fullsized":"2023-07-14T18:14:49.540949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:49:16.236513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.894113+0000","last_change":"2023-07-14T18:14:16.928995+0000","last_active":"2023-07-14T18:14:49.894113+0000","last_peered":"2023-07-14T18:14:49.894113+0000","last_clean":"2023-07-14T18:14:49.894113+0000","last_became_active":"2023-07-14T18:14:16.928612+0000","last_became_peered":"2023-07-14T18:14:16.928612+0000","last_unstale":"2023-07-14T18:14:49.894113+0000","last_undegraded":"2023-07-14T18:14:49.894113+0000","last_fullsized":"2023-07-14T18:14:49.894113+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:33:55.215553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":340,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.943619+0000","last_change":"2023-07-14T18:13:11.614507+0000","last_active":"2023-07-14T18:14:48.943619+0000","last_peered":"2023-07-14T18:14:48.943619+0000","last_clean":"2023-07-14T18:14:48.943619+0000","last_became_active":"2023-07-14T18:13:11.614063+0000","last_became_peered":"2023-07-14T18:13:11.614063+0000","last_unstale":"2023-07-14T18:14:48.943619+0000","last_undegraded":"2023-07-14T18:14:48.943619+0000","last_fullsized":"2023-07-14T18:14:48.943619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:10:03.995193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:10:03.995193+0000","last_clean_scrub_stamp":"2023-07-14T18:10:03.995193+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:35:25.729988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.543558+0000","last_change":"2023-07-14T18:14:49.542604+0000","last_active":"2023-07-14T18:14:49.543558+0000","last_peered":"2023-07-14T18:14:49.543558+0000","last_clean":"2023-07-14T18:14:49.543558+0000","last_became_active":"2023-07-14T18:14:49.542111+0000","last_became_peered":"2023-07-14T18:14:49.542111+0000","last_unstale":"2023-07-14T18:14:49.543558+0000","last_undegraded":"2023-07-14T18:14:49.543558+0000","last_fullsized":"2023-07-14T18:14:49.543558+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:40:56.970642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.543511+0000","last_change":"2023-07-14T18:14:49.543511+0000","last_active":"2023-07-14T18:14:49.543511+0000","last_peered":"2023-07-14T18:14:49.543511+0000","last_clean":"2023-07-14T18:14:49.543511+0000","last_became_active":"2023-07-14T18:14:49.542407+0000","last_became_peered":"2023-07-14T18:14:49.542407+0000","last_unstale":"2023-07-14T18:14:49.543511+0000","last_undegraded":"2023-07-14T18:14:49.543511+0000","last_fullsized":"2023-07-14T18:14:49.543511+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:55:54.330638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.894434+0000","last_change":"2023-07-14T18:14:48.950022+0000","last_active":"2023-07-14T18:14:49.894434+0000","last_peered":"2023-07-14T18:14:49.894434+0000","last_clean":"2023-07-14T18:14:49.894434+0000","last_became_active":"2023-07-14T18:14:48.949149+0000","last_became_peered":"2023-07-14T18:14:48.949149+0000","last_unstale":"2023-07-14T18:14:49.894434+0000","last_undegraded":"2023-07-14T18:14:49.894434+0000","last_fullsized":"2023-07-14T18:14:49.894434+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:10:25.993299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945005+0000","last_change":"2023-07-14T18:14:22.038428+0000","last_active":"2023-07-14T18:14:48.945005+0000","last_peered":"2023-07-14T18:14:48.945005+0000","last_clean":"2023-07-14T18:14:48.945005+0000","last_became_active":"2023-07-14T18:14:22.038038+0000","last_became_peered":"2023-07-14T18:14:22.038038+0000","last_unstale":"2023-07-14T18:14:48.945005+0000","last_undegraded":"2023-07-14T18:14:48.945005+0000","last_fullsized":"2023-07-14T18:14:48.945005+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:58:40.233180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946196+0000","last_change":"2023-07-14T18:14:16.933110+0000","last_active":"2023-07-14T18:14:48.946196+0000","last_peered":"2023-07-14T18:14:48.946196+0000","last_clean":"2023-07-14T18:14:48.946196+0000","last_became_active":"2023-07-14T18:14:16.932844+0000","last_became_peered":"2023-07-14T18:14:16.932844+0000","last_unstale":"2023-07-14T18:14:48.946196+0000","last_undegraded":"2023-07-14T18:14:48.946196+0000","last_fullsized":"2023-07-14T18:14:48.946196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:38:45.588165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942980+0000","last_change":"2023-07-14T18:14:17.920168+0000","last_active":"2023-07-14T18:14:48.942980+0000","last_peered":"2023-07-14T18:14:48.942980+0000","last_clean":"2023-07-14T18:14:48.942980+0000","last_became_active":"2023-07-14T18:14:17.919576+0000","last_became_peered":"2023-07-14T18:14:17.919576+0000","last_unstale":"2023-07-14T18:14:48.942980+0000","last_undegraded":"2023-07-14T18:14:48.942980+0000","last_fullsized":"2023-07-14T18:14:48.942980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:22:46.766040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540761+0000","last_change":"2023-07-14T18:14:22.327206+0000","last_active":"2023-07-14T18:14:49.540761+0000","last_peered":"2023-07-14T18:14:49.540761+0000","last_clean":"2023-07-14T18:14:49.540761+0000","last_became_active":"2023-07-14T18:14:22.326743+0000","last_became_peered":"2023-07-14T18:14:22.326743+0000","last_unstale":"2023-07-14T18:14:49.540761+0000","last_undegraded":"2023-07-14T18:14:49.540761+0000","last_fullsized":"2023-07-14T18:14:49.540761+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:19:42.552871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'6","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:55.918214+0000","last_change":"2023-07-14T18:14:16.928994+0000","last_active":"2023-07-14T18:14:55.918214+0000","last_peered":"2023-07-14T18:14:55.918214+0000","last_clean":"2023-07-14T18:14:55.918214+0000","last_became_active":"2023-07-14T18:14:16.928468+0000","last_became_peered":"2023-07-14T18:14:16.928468+0000","last_unstale":"2023-07-14T18:14:55.918214+0000","last_undegraded":"2023-07-14T18:14:55.918214+0000","last_fullsized":"2023-07-14T18:14:55.918214+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:19:19.175018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541197+0000","last_change":"2023-07-14T18:14:17.919147+0000","last_active":"2023-07-14T18:14:49.541197+0000","last_peered":"2023-07-14T18:14:49.541197+0000","last_clean":"2023-07-14T18:14:49.541197+0000","last_became_active":"2023-07-14T18:14:17.918811+0000","last_became_peered":"2023-07-14T18:14:17.918811+0000","last_unstale":"2023-07-14T18:14:49.541197+0000","last_undegraded":"2023-07-14T18:14:49.541197+0000","last_fullsized":"2023-07-14T18:14:49.541197+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:55:38.239373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540750+0000","last_change":"2023-07-14T18:14:22.032632+0000","last_active":"2023-07-14T18:14:49.540750+0000","last_peered":"2023-07-14T18:14:49.540750+0000","last_clean":"2023-07-14T18:14:49.540750+0000","last_became_active":"2023-07-14T18:14:22.032233+0000","last_became_peered":"2023-07-14T18:14:22.032233+0000","last_unstale":"2023-07-14T18:14:49.540750+0000","last_undegraded":"2023-07-14T18:14:49.540750+0000","last_fullsized":"2023-07-14T18:14:49.540750+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:16:49.157181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.947795+0000","last_change":"2023-07-14T18:14:16.924993+0000","last_active":"2023-07-14T18:14:48.947795+0000","last_peered":"2023-07-14T18:14:48.947795+0000","last_clean":"2023-07-14T18:14:48.947795+0000","last_became_active":"2023-07-14T18:14:16.924702+0000","last_became_peered":"2023-07-14T18:14:16.924702+0000","last_unstale":"2023-07-14T18:14:48.947795+0000","last_undegraded":"2023-07-14T18:14:48.947795+0000","last_fullsized":"2023-07-14T18:14:48.947795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:21:25.188427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541234+0000","last_change":"2023-07-14T18:14:17.943397+0000","last_active":"2023-07-14T18:14:49.541234+0000","last_peered":"2023-07-14T18:14:49.541234+0000","last_clean":"2023-07-14T18:14:49.541234+0000","last_became_active":"2023-07-14T18:14:17.942114+0000","last_became_peered":"2023-07-14T18:14:17.942114+0000","last_unstale":"2023-07-14T18:14:49.541234+0000","last_undegraded":"2023-07-14T18:14:49.541234+0000","last_fullsized":"2023-07-14T18:14:49.541234+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:46:40.372375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.947250+0000","last_change":"2023-07-14T18:14:22.328273+0000","last_active":"2023-07-14T18:14:48.947250+0000","last_peered":"2023-07-14T18:14:48.947250+0000","last_clean":"2023-07-14T18:14:48.947250+0000","last_became_active":"2023-07-14T18:14:22.327843+0000","last_became_peered":"2023-07-14T18:14:22.327843+0000","last_unstale":"2023-07-14T18:14:48.947250+0000","last_undegraded":"2023-07-14T18:14:48.947250+0000","last_fullsized":"2023-07-14T18:14:48.947250+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:23:58.886753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.545792+0000","last_change":"2023-07-14T18:14:49.543525+0000","last_active":"2023-07-14T18:14:49.545792+0000","last_peered":"2023-07-14T18:14:49.545792+0000","last_clean":"2023-07-14T18:14:49.545792+0000","last_became_active":"2023-07-14T18:14:49.542962+0000","last_became_peered":"2023-07-14T18:14:49.542962+0000","last_unstale":"2023-07-14T18:14:49.545792+0000","last_undegraded":"2023-07-14T18:14:49.545792+0000","last_fullsized":"2023-07-14T18:14:49.545792+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:11:59.955363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945273+0000","last_change":"2023-07-14T18:14:17.932181+0000","last_active":"2023-07-14T18:14:48.945273+0000","last_peered":"2023-07-14T18:14:48.945273+0000","last_clean":"2023-07-14T18:14:48.945273+0000","last_became_active":"2023-07-14T18:14:17.931811+0000","last_became_peered":"2023-07-14T18:14:17.931811+0000","last_unstale":"2023-07-14T18:14:48.945273+0000","last_undegraded":"2023-07-14T18:14:48.945273+0000","last_fullsized":"2023-07-14T18:14:48.945273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:24:11.968765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945853+0000","last_change":"2023-07-14T18:14:22.047281+0000","last_active":"2023-07-14T18:14:48.945853+0000","last_peered":"2023-07-14T18:14:48.945853+0000","last_clean":"2023-07-14T18:14:48.945853+0000","last_became_active":"2023-07-14T18:14:22.045531+0000","last_became_peered":"2023-07-14T18:14:22.045531+0000","last_unstale":"2023-07-14T18:14:48.945853+0000","last_undegraded":"2023-07-14T18:14:48.945853+0000","last_fullsized":"2023-07-14T18:14:48.945853+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:59:25.065086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541181+0000","last_change":"2023-07-14T18:14:16.926904+0000","last_active":"2023-07-14T18:14:49.541181+0000","last_peered":"2023-07-14T18:14:49.541181+0000","last_clean":"2023-07-14T18:14:49.541181+0000","last_became_active":"2023-07-14T18:14:16.926579+0000","last_became_peered":"2023-07-14T18:14:16.926579+0000","last_unstale":"2023-07-14T18:14:49.541181+0000","last_undegraded":"2023-07-14T18:14:49.541181+0000","last_fullsized":"2023-07-14T18:14:49.541181+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:34:03.520810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946719+0000","last_change":"2023-07-14T18:14:17.926005+0000","last_active":"2023-07-14T18:14:48.946719+0000","last_peered":"2023-07-14T18:14:48.946719+0000","last_clean":"2023-07-14T18:14:48.946719+0000","last_became_active":"2023-07-14T18:14:17.925266+0000","last_became_peered":"2023-07-14T18:14:17.925266+0000","last_unstale":"2023-07-14T18:14:48.946719+0000","last_undegraded":"2023-07-14T18:14:48.946719+0000","last_fullsized":"2023-07-14T18:14:48.946719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:03:29.763528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946978+0000","last_change":"2023-07-14T18:14:22.043219+0000","last_active":"2023-07-14T18:14:48.946978+0000","last_peered":"2023-07-14T18:14:48.946978+0000","last_clean":"2023-07-14T18:14:48.946978+0000","last_became_active":"2023-07-14T18:14:22.042654+0000","last_became_peered":"2023-07-14T18:14:22.042654+0000","last_unstale":"2023-07-14T18:14:48.946978+0000","last_undegraded":"2023-07-14T18:14:48.946978+0000","last_fullsized":"2023-07-14T18:14:48.946978+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:46:41.786917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540895+0000","last_change":"2023-07-14T18:14:16.923130+0000","last_active":"2023-07-14T18:14:49.540895+0000","last_peered":"2023-07-14T18:14:49.540895+0000","last_clean":"2023-07-14T18:14:49.540895+0000","last_became_active":"2023-07-14T18:14:16.922835+0000","last_became_peered":"2023-07-14T18:14:16.922835+0000","last_unstale":"2023-07-14T18:14:49.540895+0000","last_undegraded":"2023-07-14T18:14:49.540895+0000","last_fullsized":"2023-07-14T18:14:49.540895+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:48:50.705976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540779+0000","last_change":"2023-07-14T18:14:17.931024+0000","last_active":"2023-07-14T18:14:49.540779+0000","last_peered":"2023-07-14T18:14:49.540779+0000","last_clean":"2023-07-14T18:14:49.540779+0000","last_became_active":"2023-07-14T18:14:17.929529+0000","last_became_peered":"2023-07-14T18:14:17.929529+0000","last_unstale":"2023-07-14T18:14:49.540779+0000","last_undegraded":"2023-07-14T18:14:49.540779+0000","last_fullsized":"2023-07-14T18:14:49.540779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:39.524476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946795+0000","last_change":"2023-07-14T18:14:22.327313+0000","last_active":"2023-07-14T18:14:48.946795+0000","last_peered":"2023-07-14T18:14:48.946795+0000","last_clean":"2023-07-14T18:14:48.946795+0000","last_became_active":"2023-07-14T18:14:22.326922+0000","last_became_peered":"2023-07-14T18:14:22.326922+0000","last_unstale":"2023-07-14T18:14:48.946795+0000","last_undegraded":"2023-07-14T18:14:48.946795+0000","last_fullsized":"2023-07-14T18:14:48.946795+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:48:34.325995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945190+0000","last_change":"2023-07-14T18:14:16.926632+0000","last_active":"2023-07-14T18:14:48.945190+0000","last_peered":"2023-07-14T18:14:48.945190+0000","last_clean":"2023-07-14T18:14:48.945190+0000","last_became_active":"2023-07-14T18:14:16.926279+0000","last_became_peered":"2023-07-14T18:14:16.926279+0000","last_unstale":"2023-07-14T18:14:48.945190+0000","last_undegraded":"2023-07-14T18:14:48.945190+0000","last_fullsized":"2023-07-14T18:14:48.945190+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:25:51.630494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944432+0000","last_change":"2023-07-14T18:14:17.924416+0000","last_active":"2023-07-14T18:14:48.944432+0000","last_peered":"2023-07-14T18:14:48.944432+0000","last_clean":"2023-07-14T18:14:48.944432+0000","last_became_active":"2023-07-14T18:14:17.924140+0000","last_became_peered":"2023-07-14T18:14:17.924140+0000","last_unstale":"2023-07-14T18:14:48.944432+0000","last_undegraded":"2023-07-14T18:14:48.944432+0000","last_fullsized":"2023-07-14T18:14:48.944432+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:36:11.267149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946249+0000","last_change":"2023-07-14T18:14:22.037127+0000","last_active":"2023-07-14T18:14:48.946249+0000","last_peered":"2023-07-14T18:14:48.946249+0000","last_clean":"2023-07-14T18:14:48.946249+0000","last_became_active":"2023-07-14T18:14:22.036647+0000","last_became_peered":"2023-07-14T18:14:22.036647+0000","last_unstale":"2023-07-14T18:14:48.946249+0000","last_undegraded":"2023-07-14T18:14:48.946249+0000","last_fullsized":"2023-07-14T18:14:48.946249+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:45:53.709735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.895800+0000","last_change":"2023-07-14T18:14:16.921494+0000","last_active":"2023-07-14T18:14:49.895800+0000","last_peered":"2023-07-14T18:14:49.895800+0000","last_clean":"2023-07-14T18:14:49.895800+0000","last_became_active":"2023-07-14T18:14:16.919622+0000","last_became_peered":"2023-07-14T18:14:16.919622+0000","last_unstale":"2023-07-14T18:14:49.895800+0000","last_undegraded":"2023-07-14T18:14:49.895800+0000","last_fullsized":"2023-07-14T18:14:49.895800+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:49:14.678276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944208+0000","last_change":"2023-07-14T18:14:17.922266+0000","last_active":"2023-07-14T18:14:48.944208+0000","last_peered":"2023-07-14T18:14:48.944208+0000","last_clean":"2023-07-14T18:14:48.944208+0000","last_became_active":"2023-07-14T18:14:17.921461+0000","last_became_peered":"2023-07-14T18:14:17.921461+0000","last_unstale":"2023-07-14T18:14:48.944208+0000","last_undegraded":"2023-07-14T18:14:48.944208+0000","last_fullsized":"2023-07-14T18:14:48.944208+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:27:06.427711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541171+0000","last_change":"2023-07-14T18:14:22.044080+0000","last_active":"2023-07-14T18:14:49.541171+0000","last_peered":"2023-07-14T18:14:49.541171+0000","last_clean":"2023-07-14T18:14:49.541171+0000","last_became_active":"2023-07-14T18:14:22.043612+0000","last_became_peered":"2023-07-14T18:14:22.043612+0000","last_unstale":"2023-07-14T18:14:49.541171+0000","last_undegraded":"2023-07-14T18:14:49.541171+0000","last_fullsized":"2023-07-14T18:14:49.541171+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:41:00.904598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945287+0000","last_change":"2023-07-14T18:14:16.931871+0000","last_active":"2023-07-14T18:14:48.945287+0000","last_peered":"2023-07-14T18:14:48.945287+0000","last_clean":"2023-07-14T18:14:48.945287+0000","last_became_active":"2023-07-14T18:14:16.931549+0000","last_became_peered":"2023-07-14T18:14:16.931549+0000","last_unstale":"2023-07-14T18:14:48.945287+0000","last_undegraded":"2023-07-14T18:14:48.945287+0000","last_fullsized":"2023-07-14T18:14:48.945287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:39:50.087515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944103+0000","last_change":"2023-07-14T18:14:17.929550+0000","last_active":"2023-07-14T18:14:48.944103+0000","last_peered":"2023-07-14T18:14:48.944103+0000","last_clean":"2023-07-14T18:14:48.944103+0000","last_became_active":"2023-07-14T18:14:17.929204+0000","last_became_peered":"2023-07-14T18:14:17.929204+0000","last_unstale":"2023-07-14T18:14:48.944103+0000","last_undegraded":"2023-07-14T18:14:48.944103+0000","last_fullsized":"2023-07-14T18:14:48.944103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:46:07.635189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942348+0000","last_change":"2023-07-14T18:14:22.034899+0000","last_active":"2023-07-14T18:14:48.942348+0000","last_peered":"2023-07-14T18:14:48.942348+0000","last_clean":"2023-07-14T18:14:48.942348+0000","last_became_active":"2023-07-14T18:14:22.034524+0000","last_became_peered":"2023-07-14T18:14:22.034524+0000","last_unstale":"2023-07-14T18:14:48.942348+0000","last_undegraded":"2023-07-14T18:14:48.942348+0000","last_fullsized":"2023-07-14T18:14:48.942348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:44:56.300609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942705+0000","last_change":"2023-07-14T18:14:16.935285+0000","last_active":"2023-07-14T18:14:48.942705+0000","last_peered":"2023-07-14T18:14:48.942705+0000","last_clean":"2023-07-14T18:14:48.942705+0000","last_became_active":"2023-07-14T18:14:16.934888+0000","last_became_peered":"2023-07-14T18:14:16.934888+0000","last_unstale":"2023-07-14T18:14:48.942705+0000","last_undegraded":"2023-07-14T18:14:48.942705+0000","last_fullsized":"2023-07-14T18:14:48.942705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:46:09.021281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944372+0000","last_change":"2023-07-14T18:14:17.930982+0000","last_active":"2023-07-14T18:14:48.944372+0000","last_peered":"2023-07-14T18:14:48.944372+0000","last_clean":"2023-07-14T18:14:48.944372+0000","last_became_active":"2023-07-14T18:14:17.930661+0000","last_became_peered":"2023-07-14T18:14:17.930661+0000","last_unstale":"2023-07-14T18:14:48.944372+0000","last_undegraded":"2023-07-14T18:14:48.944372+0000","last_fullsized":"2023-07-14T18:14:48.944372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:42:10.943678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946615+0000","last_change":"2023-07-14T18:14:17.926204+0000","last_active":"2023-07-14T18:14:48.946615+0000","last_peered":"2023-07-14T18:14:48.946615+0000","last_clean":"2023-07-14T18:14:48.946615+0000","last_became_active":"2023-07-14T18:14:17.925689+0000","last_became_peered":"2023-07-14T18:14:17.925689+0000","last_unstale":"2023-07-14T18:14:48.946615+0000","last_undegraded":"2023-07-14T18:14:48.946615+0000","last_fullsized":"2023-07-14T18:14:48.946615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:32:10.441762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541397+0000","last_change":"2023-07-14T18:14:17.923146+0000","last_active":"2023-07-14T18:14:49.541397+0000","last_peered":"2023-07-14T18:14:49.541397+0000","last_clean":"2023-07-14T18:14:49.541397+0000","last_became_active":"2023-07-14T18:14:17.922734+0000","last_became_peered":"2023-07-14T18:14:17.922734+0000","last_unstale":"2023-07-14T18:14:49.541397+0000","last_undegraded":"2023-07-14T18:14:49.541397+0000","last_fullsized":"2023-07-14T18:14:49.541397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:01.011677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540904+0000","last_change":"2023-07-14T18:14:17.927397+0000","last_active":"2023-07-14T18:14:49.540904+0000","last_peered":"2023-07-14T18:14:49.540904+0000","last_clean":"2023-07-14T18:14:49.540904+0000","last_became_active":"2023-07-14T18:14:17.927000+0000","last_became_peered":"2023-07-14T18:14:17.927000+0000","last_unstale":"2023-07-14T18:14:49.540904+0000","last_undegraded":"2023-07-14T18:14:49.540904+0000","last_fullsized":"2023-07-14T18:14:49.540904+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:57:52.235148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541451+0000","last_change":"2023-07-14T18:14:17.935266+0000","last_active":"2023-07-14T18:14:49.541451+0000","last_peered":"2023-07-14T18:14:49.541451+0000","last_clean":"2023-07-14T18:14:49.541451+0000","last_became_active":"2023-07-14T18:14:17.934944+0000","last_became_peered":"2023-07-14T18:14:17.934944+0000","last_unstale":"2023-07-14T18:14:49.541451+0000","last_undegraded":"2023-07-14T18:14:49.541451+0000","last_fullsized":"2023-07-14T18:14:49.541451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:36:06.516397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.542810+0000","last_change":"2023-07-14T18:14:49.542810+0000","last_active":"2023-07-14T18:14:49.542810+0000","last_peered":"2023-07-14T18:14:49.542810+0000","last_clean":"2023-07-14T18:14:49.542810+0000","last_became_active":"2023-07-14T18:14:49.542421+0000","last_became_peered":"2023-07-14T18:14:49.542421+0000","last_unstale":"2023-07-14T18:14:49.542810+0000","last_undegraded":"2023-07-14T18:14:49.542810+0000","last_fullsized":"2023-07-14T18:14:49.542810+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:16:57.268229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946187+0000","last_change":"2023-07-14T18:14:17.930757+0000","last_active":"2023-07-14T18:14:48.946187+0000","last_peered":"2023-07-14T18:14:48.946187+0000","last_clean":"2023-07-14T18:14:48.946187+0000","last_became_active":"2023-07-14T18:14:17.930302+0000","last_became_peered":"2023-07-14T18:14:17.930302+0000","last_unstale":"2023-07-14T18:14:48.946187+0000","last_undegraded":"2023-07-14T18:14:48.946187+0000","last_fullsized":"2023-07-14T18:14:48.946187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:44:33.862208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541685+0000","last_change":"2023-07-14T18:14:17.930337+0000","last_active":"2023-07-14T18:14:49.541685+0000","last_peered":"2023-07-14T18:14:49.541685+0000","last_clean":"2023-07-14T18:14:49.541685+0000","last_became_active":"2023-07-14T18:14:17.930037+0000","last_became_peered":"2023-07-14T18:14:17.930037+0000","last_unstale":"2023-07-14T18:14:49.541685+0000","last_undegraded":"2023-07-14T18:14:49.541685+0000","last_fullsized":"2023-07-14T18:14:49.541685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:56:10.479187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541417+0000","last_change":"2023-07-14T18:14:17.912750+0000","last_active":"2023-07-14T18:14:49.541417+0000","last_peered":"2023-07-14T18:14:49.541417+0000","last_clean":"2023-07-14T18:14:49.541417+0000","last_became_active":"2023-07-14T18:14:17.912402+0000","last_became_peered":"2023-07-14T18:14:17.912402+0000","last_unstale":"2023-07-14T18:14:49.541417+0000","last_undegraded":"2023-07-14T18:14:49.541417+0000","last_fullsized":"2023-07-14T18:14:49.541417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:16:35.639559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942521+0000","last_change":"2023-07-14T18:14:17.930004+0000","last_active":"2023-07-14T18:14:48.942521+0000","last_peered":"2023-07-14T18:14:48.942521+0000","last_clean":"2023-07-14T18:14:48.942521+0000","last_became_active":"2023-07-14T18:14:17.929719+0000","last_became_peered":"2023-07-14T18:14:17.929719+0000","last_unstale":"2023-07-14T18:14:48.942521+0000","last_undegraded":"2023-07-14T18:14:48.942521+0000","last_fullsized":"2023-07-14T18:14:48.942521+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:51:50.359987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541319+0000","last_change":"2023-07-14T18:14:17.919044+0000","last_active":"2023-07-14T18:14:49.541319+0000","last_peered":"2023-07-14T18:14:49.541319+0000","last_clean":"2023-07-14T18:14:49.541319+0000","last_became_active":"2023-07-14T18:14:17.918079+0000","last_became_peered":"2023-07-14T18:14:17.918079+0000","last_unstale":"2023-07-14T18:14:49.541319+0000","last_undegraded":"2023-07-14T18:14:49.541319+0000","last_fullsized":"2023-07-14T18:14:49.541319+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:12:44.589756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946548+0000","last_change":"2023-07-14T18:14:17.921560+0000","last_active":"2023-07-14T18:14:48.946548+0000","last_peered":"2023-07-14T18:14:48.946548+0000","last_clean":"2023-07-14T18:14:48.946548+0000","last_became_active":"2023-07-14T18:14:17.921243+0000","last_became_peered":"2023-07-14T18:14:17.921243+0000","last_unstale":"2023-07-14T18:14:48.946548+0000","last_undegraded":"2023-07-14T18:14:48.946548+0000","last_fullsized":"2023-07-14T18:14:48.946548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:03:15.907251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942971+0000","last_change":"2023-07-14T18:14:17.927504+0000","last_active":"2023-07-14T18:14:48.942971+0000","last_peered":"2023-07-14T18:14:48.942971+0000","last_clean":"2023-07-14T18:14:48.942971+0000","last_became_active":"2023-07-14T18:14:17.927233+0000","last_became_peered":"2023-07-14T18:14:17.927233+0000","last_unstale":"2023-07-14T18:14:48.942971+0000","last_undegraded":"2023-07-14T18:14:48.942971+0000","last_fullsized":"2023-07-14T18:14:48.942971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:37:40.310852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946334+0000","last_change":"2023-07-14T18:14:17.932317+0000","last_active":"2023-07-14T18:14:48.946334+0000","last_peered":"2023-07-14T18:14:48.946334+0000","last_clean":"2023-07-14T18:14:48.946334+0000","last_became_active":"2023-07-14T18:14:17.931973+0000","last_became_peered":"2023-07-14T18:14:17.931973+0000","last_unstale":"2023-07-14T18:14:48.946334+0000","last_undegraded":"2023-07-14T18:14:48.946334+0000","last_fullsized":"2023-07-14T18:14:48.946334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:01:51.245001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281026,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63712,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":291,"num_read_kb":210,"num_write":157,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":191136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416412,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":492605,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.94199999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.59899999999999998}]},{"osd":1,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63900000000000001,"15min":0.63800000000000001},"min":{"1min":0.44500000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.41899999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77700000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.70299999999999996}]},{"osd":2,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72299999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.502,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72699999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.64300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":4,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68600000000000005,"15min":0.68000000000000005},"min":{"1min":0.48099999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89900000000000002,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.66400000000000003,"15min":0.67000000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86799999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.83799999999999997}]},{"osd":5,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.61099999999999999,"15min":0.625},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68300000000000005,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.38700000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96399999999999997,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.81599999999999995}]},{"osd":6,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.749,"15min":0.75800000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.95999999999999996,"5min":1.351,"15min":1.351},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.79000000000000004,"15min":0.79400000000000004},"min":{"1min":0.48299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":3.2410000000000001,"5min":3.2410000000000001,"15min":3.2410000000000001},"last":0.76500000000000001}]},{"osd":7,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77800000000000002,"15min":0.77900000000000003},"min":{"1min":0.58099999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.026,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.74399999999999999,"15min":0.749},"min":{"1min":0.51600000000000001,"5min":0.505,"15min":0.505},"max":{"1min":0.93100000000000005,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.92800000000000005}]},{"osd":8,"last update":"Fri Jul 14 18:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1,"5min":1,"15min":1},"last":0.95299999999999996}]},{"osd":9,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80200000000000005,"15min":0.80600000000000005},"min":{"1min":0.58099999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1279999999999999,"5min":1.635,"15min":1.635},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.84899999999999998,"15min":0.85099999999999998},"min":{"1min":0.59899999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.05,"5min":1.147,"15min":1.147},"last":0.98899999999999999}]},{"osd":10,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80500000000000005,"15min":0.80900000000000005},"min":{"1min":0.57599999999999996,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.97299999999999998,"5min":1.272,"15min":1.272},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.83999999999999997,"15min":0.84999999999999998},"min":{"1min":0.59699999999999998,"5min":0.501,"15min":0.501},"max":{"1min":0.97099999999999997,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.91300000000000003}]}]},{"osd":10,"up_from":72,"seq":309237645344,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":1139302,"data_compressed":53365,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.76100000000000001}]},{"osd":1,"last update":"Fri Jul 14 18:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.443,"15min":0.44500000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.59599999999999997,"5min":0.626,"15min":0.626},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.48099999999999998,"15min":0.48499999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.53900000000000003}]},{"osd":2,"last update":"Fri Jul 14 18:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59999999999999998,"15min":0.59699999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.79400000000000004,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.622},{"interface":"front","average":{"1min":0.624,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.63600000000000001}]},{"osd":3,"last update":"Fri Jul 14 18:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.64600000000000002,"15min":0.63800000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":3.4129999999999998,"5min":3.4129999999999998,"15min":3.4129999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.70699999999999996,"15min":0.68200000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":5.9939999999999998,"5min":5.9939999999999998,"15min":5.9939999999999998},"last":0.71499999999999997}]},{"osd":4,"last update":"Fri Jul 14 18:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.56200000000000006,"15min":0.56799999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.75,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.57199999999999995}]},{"osd":5,"last update":"Fri Jul 14 18:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65800000000000003,"15min":0.65500000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88400000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.46200000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.68200000000000005}]},{"osd":6,"last update":"Fri Jul 14 18:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.71499999999999997,"15min":0.72599999999999998},"min":{"1min":0.51200000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.91800000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.752,"15min":0.76700000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.81399999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.92600000000000005}]},{"osd":7,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.42199999999999999}]},{"osd":8,"last update":"Fri Jul 14 18:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.74099999999999999,"15min":0.747},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.90300000000000002,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.75700000000000001,"15min":0.76100000000000001},"min":{"1min":0.51100000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.92000000000000004,"5min":1.026,"15min":1.026},"last":0.81399999999999995}]},{"osd":9,"last update":"Fri Jul 14 18:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81699999999999995,"15min":0.81799999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.3359999999999999,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80100000000000005,"15min":0.80400000000000005},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":0.86799999999999999}]},{"osd":11,"last update":"Fri Jul 14 18:14:32 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.79100000000000004}]}]},{"osd":9,"up_from":67,"seq":287762808870,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":496937,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.53000000000000003,"15min":0.54200000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.59999999999999998,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.55600000000000005,"15min":0.56999999999999995},"min":{"1min":0.35399999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85299999999999998,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.35799999999999998}]},{"osd":1,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.65500000000000003,"15min":0.66000000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.72499999999999998,"15min":0.73699999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.879,"5min":1.085,"15min":1.085},"last":0.77000000000000002}]},{"osd":2,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.51300000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93200000000000005,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.745,"15min":0.75900000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.91300000000000003,"5min":1.147,"15min":1.147},"last":0.59299999999999997}]},{"osd":3,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.65200000000000002,"15min":0.67400000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.81100000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.497},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.65100000000000002,"15min":0.66800000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77700000000000002,"5min":1.018,"15min":1.018},"last":0.42999999999999999}]},{"osd":4,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.77800000000000002,"15min":0.79200000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.89600000000000002,"5min":1.198,"15min":1.198},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.77500000000000002,"15min":0.78900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.96299999999999997,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.78400000000000003}]},{"osd":5,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.79600000000000004,"15min":0.81200000000000006},"min":{"1min":0.54800000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.84199999999999997,"5min":1.224,"15min":1.224},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.75,"15min":0.75800000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95799999999999996,"5min":1.113,"15min":1.113},"last":0.47999999999999998}]},{"osd":6,"last update":"Fri Jul 14 18:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.75900000000000001}]},{"osd":7,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.85799999999999998,"15min":0.88200000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.92800000000000005,"5min":1.323,"15min":1.323},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.85099999999999998,"15min":0.86899999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.89500000000000002,"5min":1.411,"15min":1.411},"last":0.72399999999999998}]},{"osd":8,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.90100000000000002,"15min":0.92100000000000004},"min":{"1min":0.63,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0620000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.628},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.90200000000000002,"15min":0.92100000000000004},"min":{"1min":0.60799999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0109999999999999,"5min":1.371,"15min":1.371},"last":0.61399999999999999}]},{"osd":10,"last update":"Fri Jul 14 18:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.86299999999999999,"15min":0.879},"min":{"1min":0.66600000000000004,"5min":0.626,"15min":0.626},"max":{"1min":0.97799999999999998,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.751},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.85699999999999998,"15min":0.871},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.95499999999999996,"5min":1.095,"15min":1.095},"last":0.84799999999999998}]},{"osd":11,"last update":"Fri Jul 14 18:15:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.56299999999999994}]}]},{"osd":8,"up_from":59,"seq":253403070507,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":492179,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.43099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.212,"5min":1.238,"15min":1.238},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.60699999999999998,"15min":0.57499999999999996},"min":{"1min":0.47399999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.5950000000000002,"5min":2.5950000000000002,"15min":2.5950000000000002},"last":0.57799999999999996}]},{"osd":1,"last update":"Fri Jul 14 18:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.58099999999999996,"15min":0.56499999999999995},"min":{"1min":0.46600000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.77500000000000002,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60199999999999998,"15min":0.58699999999999997},"min":{"1min":0.45400000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.624}]},{"osd":2,"last update":"Fri Jul 14 18:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.47399999999999998,"15min":0.47199999999999998},"min":{"1min":0.34799999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.69699999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.47499999999999998,"15min":0.46600000000000003},"min":{"1min":0.34699999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.75800000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.46200000000000002}]},{"osd":3,"last update":"Fri Jul 14 18:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.69299999999999995,"15min":0.64500000000000002},"min":{"1min":0.48199999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":5.9749999999999996,"5min":5.9749999999999996,"15min":5.9749999999999996},"last":3.8090000000000002},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.66000000000000003,"15min":0.61399999999999999},"min":{"1min":0.42899999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":4.5229999999999997,"5min":4.5229999999999997,"15min":4.5229999999999997},"last":1.772}]},{"osd":4,"last update":"Fri Jul 14 18:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.439},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.66500000000000004,"15min":0.65700000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.66200000000000003}]},{"osd":5,"last update":"Fri Jul 14 18:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.39700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.94299999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81999999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.55100000000000005}]},{"osd":6,"last update":"Fri Jul 14 18:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.75600000000000001,"15min":0.73399999999999999},"min":{"1min":0.52800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":4.6900000000000004,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.998,"5min":0.77300000000000002,"15min":0.72699999999999998},"min":{"1min":0.57699999999999996,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":4.9589999999999996,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":3.6339999999999999}]},{"osd":7,"last update":"Fri Jul 14 18:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.73499999999999999,"15min":0.72899999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.71399999999999997,"15min":0.70299999999999996},"min":{"1min":0.54100000000000004,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95099999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.59799999999999998}]},{"osd":9,"last update":"Fri Jul 14 18:14:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.78900000000000003,"15min":0.78800000000000003},"min":{"1min":0.58699999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.78900000000000003,"15min":0.79400000000000004},"min":{"1min":0.54300000000000004,"5min":0.502,"15min":0.502},"max":{"1min":0.91900000000000004,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.67600000000000005}]},{"osd":10,"last update":"Fri Jul 14 18:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.77000000000000002,"15min":0.76300000000000001},"min":{"1min":0.56999999999999995,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77800000000000002,"15min":0.77400000000000002},"min":{"1min":0.57799999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.0089999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.76000000000000001}]},{"osd":11,"last update":"Fri Jul 14 18:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.70999999999999996}]}]},{"osd":7,"up_from":51,"seq":219043332141,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":1088573,"data_compressed":53365,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.755,"15min":0.77300000000000002},"min":{"1min":0.38200000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.024,"5min":5.843,"15min":5.843},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.51100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.81299999999999994,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.40899999999999997}]},{"osd":1,"last update":"Fri Jul 14 18:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.33400000000000002,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.54100000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46200000000000002,"15min":0.46100000000000002},"min":{"1min":0.29499999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66100000000000003,"5min":0.752,"15min":0.752},"last":0.35599999999999998}]},{"osd":2,"last update":"Fri Jul 14 18:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.625,"15min":0.59899999999999998},"min":{"1min":0.45100000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":3.4889999999999999,"5min":3.4889999999999999,"15min":3.4889999999999999},"last":0.498},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.48799999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80900000000000005,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.45200000000000001}]},{"osd":3,"last update":"Fri Jul 14 18:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.65100000000000002,"15min":0.64700000000000002},"min":{"1min":0.52500000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.94199999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.443,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77900000000000003,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.434}]},{"osd":4,"last update":"Fri Jul 14 18:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52600000000000002,"15min":0.53200000000000003},"min":{"1min":0.35299999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.64500000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.501,"15min":0.498},"min":{"1min":0.34699999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72399999999999998,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.379}]},{"osd":5,"last update":"Fri Jul 14 18:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61799999999999999,"15min":0.61399999999999999},"min":{"1min":0.5,"5min":0.375,"15min":0.375},"max":{"1min":0.71399999999999997,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68700000000000006,"15min":0.68500000000000005},"min":{"1min":0.56599999999999995,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.878,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.56599999999999995}]},{"osd":6,"last update":"Fri Jul 14 18:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":1.036,"15min":1.1499999999999999},"min":{"1min":0.56200000000000006,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.5019999999999998,"5min":13.189,"15min":13.189},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.51100000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88300000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.54900000000000004}]},{"osd":8,"last update":"Fri Jul 14 18:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.71599999999999997,"15min":0.69799999999999995},"min":{"1min":0.59899999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86899999999999999,"5min":1.04,"15min":1.04},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69799999999999995,"15min":0.69199999999999995},"min":{"1min":0.54700000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85799999999999998,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.61799999999999999}]},{"osd":9,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.94599999999999995,"15min":0.97599999999999998},"min":{"1min":0.58099999999999996,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.91000000000000003,"5min":5.8250000000000002,"15min":5.8250000000000002},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.77500000000000002,"15min":0.77100000000000002},"min":{"1min":0.61799999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.64200000000000002}]},{"osd":10,"last update":"Fri Jul 14 18:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72099999999999997,"15min":0.71799999999999997},"min":{"1min":0.48499999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92000000000000004,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.63300000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.90300000000000002,"5min":0.995,"15min":0.995},"last":0.60499999999999998}]},{"osd":11,"last update":"Fri Jul 14 18:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.70199999999999996}]}]},{"osd":6,"up_from":46,"seq":197568495665,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":488017,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.46500000000000002,"15min":0.45500000000000002},"min":{"1min":0.32700000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.61299999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.49299999999999999,"15min":0.48199999999999998},"min":{"1min":0.35399999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65200000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.96599999999999997}]},{"osd":1,"last update":"Fri Jul 14 18:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.64700000000000002,"15min":0.64000000000000001},"min":{"1min":0.46000000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83499999999999996,"5min":0.87,"15min":0.87},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67700000000000005,"15min":0.67400000000000004},"min":{"1min":0.438,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91400000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":1.0229999999999999}]},{"osd":2,"last update":"Fri Jul 14 18:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63600000000000001,"15min":0.626},"min":{"1min":0.46100000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0309999999999999,"5min":1.087,"15min":1.087},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68400000000000005,"15min":0.67800000000000005},"min":{"1min":0.52200000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.86099999999999999,"5min":1.097,"15min":1.097},"last":0.63100000000000001}]},{"osd":3,"last update":"Fri Jul 14 18:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.39200000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84499999999999997,"5min":1.03,"15min":1.03},"last":1.2769999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.64100000000000001,"15min":0.64900000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83699999999999997,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":1.351}]},{"osd":4,"last update":"Fri Jul 14 18:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86199999999999999,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71799999999999997,"15min":0.72099999999999997},"min":{"1min":0.48099999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.80200000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.59799999999999998}]},{"osd":5,"last update":"Fri Jul 14 18:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.75700000000000001,"15min":0.76300000000000001},"min":{"1min":0.52300000000000002,"5min":0.435,"15min":0.435},"max":{"1min":1.0429999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":1.4330000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.52900000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94899999999999995,"5min":1.163,"15min":1.163},"last":0.93999999999999995}]},{"osd":7,"last update":"Fri Jul 14 18:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73699999999999999,"15min":0.72699999999999998},"min":{"1min":0.65900000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.877,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":1.3899999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.72999999999999998,"15min":0.72199999999999998},"min":{"1min":0.499,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90800000000000003,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.76500000000000001}]},{"osd":8,"last update":"Fri Jul 14 18:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79900000000000004,"15min":0.80600000000000005},"min":{"1min":0.59199999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0149999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":1.456},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.76500000000000001,"15min":0.75600000000000001},"min":{"1min":0.59599999999999997,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.91200000000000003}]},{"osd":9,"last update":"Fri Jul 14 18:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.80500000000000005,"15min":0.81899999999999995},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.99199999999999999,"5min":1.147,"15min":1.147},"last":0.879},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.83499999999999996,"15min":0.84599999999999997},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":0.93500000000000005,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.83399999999999996}]},{"osd":10,"last update":"Fri Jul 14 18:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.86599999999999999,"15min":0.86899999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.0349999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.411},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.85599999999999998,"15min":0.85399999999999998},"min":{"1min":0.68300000000000005,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.1539999999999999,"5min":1.248,"15min":1.248},"last":0.84799999999999998}]},{"osd":11,"last update":"Fri Jul 14 18:15:21 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.79700000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":548317,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:14:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54500000000000004,"15min":0.51200000000000001},"min":{"1min":0.441,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76800000000000002,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57599999999999996,"15min":0.53200000000000003},"min":{"1min":0.41499999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81799999999999995,"5min":1.272,"15min":1.272},"last":0.59399999999999997}]},{"osd":2,"last update":"Fri Jul 14 18:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.59399999999999997,"15min":0.54400000000000004},"min":{"1min":0.45100000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84599999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.57299999999999995,"15min":0.52600000000000002},"min":{"1min":0.42699999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89900000000000002,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.61799999999999999}]},{"osd":3,"last update":"Fri Jul 14 18:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.56499999999999995,"15min":0.52400000000000002},"min":{"1min":0.39700000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.93500000000000005,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.60199999999999998,"15min":0.58199999999999996},"min":{"1min":0.45600000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91300000000000003,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.75800000000000001}]},{"osd":4,"last update":"Fri Jul 14 18:14:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.48399999999999999,"15min":0.45500000000000002},"min":{"1min":0.34399999999999997,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.877},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.47499999999999998,"15min":0.45000000000000001},"min":{"1min":0.36599999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.96899999999999997}]},{"osd":5,"last update":"Fri Jul 14 18:15:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.629,"15min":0.60499999999999998},"min":{"1min":0.53200000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.63100000000000001,"15min":0.60699999999999998},"min":{"1min":0.57599999999999996,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.82699999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.90900000000000003}]},{"osd":6,"last update":"Fri Jul 14 18:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.67200000000000004,"15min":0.66900000000000004},"min":{"1min":0.46000000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90200000000000002,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67600000000000005,"15min":0.66800000000000004},"min":{"1min":0.47499999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.0029999999999999,"5min":1.23,"15min":1.23},"last":0.83399999999999996}]},{"osd":7,"last update":"Fri Jul 14 18:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.57799999999999996,"15min":0.55600000000000005},"min":{"1min":0.35699999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.98799999999999999,"5min":1.04,"15min":1.04},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.60699999999999998,"15min":0.58299999999999996},"min":{"1min":0.38700000000000001,"5min":0.307,"15min":0.307},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.93799999999999994}]},{"osd":8,"last update":"Fri Jul 14 18:15:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.66500000000000004,"15min":0.64300000000000002},"min":{"1min":0.51000000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":1.008},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.72399999999999998,"15min":0.68999999999999995},"min":{"1min":0.629,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.026,"5min":1.052,"15min":1.052},"last":0.95699999999999996}]},{"osd":9,"last update":"Fri Jul 14 18:14:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.76900000000000002,"15min":0.76300000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.82799999999999996,"15min":0.82399999999999995},"min":{"1min":0.65300000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":1.018}]},{"osd":10,"last update":"Fri Jul 14 18:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.81100000000000005,"15min":0.80500000000000005},"min":{"1min":0.59799999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.121,"5min":1.167,"15min":1.167},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76200000000000001,"15min":0.755},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.044,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":1.0329999999999999}]},{"osd":11,"last update":"Fri Jul 14 18:15:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.996},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":1.073}]}]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":489405,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 18:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59299999999999997,"15min":0.55000000000000004},"min":{"1min":0.372,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.93300000000000005,"5min":1.698,"15min":1.698},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60299999999999998,"15min":0.57799999999999996},"min":{"1min":0.49199999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.91500000000000004,"5min":1.179,"15min":1.179},"last":0.50600000000000001}]},{"osd":2,"last update":"Fri Jul 14 18:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.622,"15min":0.56399999999999995},"min":{"1min":0.46400000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89700000000000002,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.64700000000000002,"15min":0.58399999999999996},"min":{"1min":0.498,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.075,"5min":1.258,"15min":1.258},"last":0.56699999999999995}]},{"osd":3,"last update":"Fri Jul 14 18:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.83699999999999997}]},{"osd":4,"last update":"Fri Jul 14 18:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.59699999999999998,"15min":0.56299999999999994},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86399999999999999,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.65700000000000003,"15min":0.623},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.92300000000000004,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.80400000000000005}]},{"osd":5,"last update":"Fri Jul 14 18:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.72499999999999998,"15min":0.68700000000000006},"min":{"1min":0.50700000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.89800000000000002,"5min":3.3740000000000001,"15min":3.3740000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.70099999999999996,"15min":0.68300000000000005},"min":{"1min":0.57499999999999996,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.99299999999999999,"5min":1.228,"15min":1.228},"last":0.92000000000000004}]},{"osd":6,"last update":"Fri Jul 14 18:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64100000000000001,"15min":0.61699999999999999},"min":{"1min":0.44700000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.66400000000000003,"15min":0.64000000000000001},"min":{"1min":0.433,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.88600000000000001}]},{"osd":7,"last update":"Fri Jul 14 18:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.626},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.97799999999999998}]},{"osd":8,"last update":"Fri Jul 14 18:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80900000000000005,"15min":0.79600000000000004},"min":{"1min":0.59599999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0649999999999999,"5min":1.839,"15min":1.839},"last":1.028},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.79100000000000004,"15min":0.76800000000000002},"min":{"1min":0.68300000000000005,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.018,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.96899999999999997}]},{"osd":9,"last update":"Fri Jul 14 18:14:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.77000000000000002,"15min":0.77200000000000002},"min":{"1min":0.57099999999999995,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":1.1080000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":1.157,"15min":1.2150000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0389999999999999,"5min":7.8689999999999998,"15min":7.8689999999999998},"last":1.1259999999999999}]},{"osd":10,"last update":"Fri Jul 14 18:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.85799999999999998,"15min":0.86199999999999999},"min":{"1min":0.626,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.139,"5min":1.319,"15min":1.319},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.014,"5min":1.081,"15min":1.081},"last":1.0569999999999999}]},{"osd":11,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.90300000000000002,"15min":0.90700000000000003},"min":{"1min":0.65300000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.093,"5min":1.869,"15min":1.869},"last":1.087},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.92900000000000005,"15min":0.93500000000000005},"min":{"1min":0.74399999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.0629999999999999,"5min":1.758,"15min":1.758},"last":1.01}]}]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":548463,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.55900000000000005,"15min":0.53300000000000003},"min":{"1min":0.41199999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.57699999999999996,"15min":0.53100000000000003},"min":{"1min":0.55600000000000005,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84099999999999997,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.56999999999999995}]},{"osd":1,"last update":"Fri Jul 14 18:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.58499999999999996,"15min":0.52300000000000002},"min":{"1min":0.40799999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.58099999999999996,"15min":0.52200000000000002},"min":{"1min":0.36899999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.55000000000000004}]},{"osd":3,"last update":"Fri Jul 14 18:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60699999999999998,"15min":0.57299999999999995},"min":{"1min":0.441,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.877,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65100000000000002,"15min":0.65700000000000003},"min":{"1min":0.53200000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81299999999999994,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":0.58499999999999996}]},{"osd":4,"last update":"Fri Jul 14 18:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67200000000000004}]},{"osd":5,"last update":"Fri Jul 14 18:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.46899999999999997,"15min":0.45700000000000002},"min":{"1min":0.38200000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.83199999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.434},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.50700000000000001,"15min":0.48599999999999999},"min":{"1min":0.44,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.752,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.47699999999999998}]},{"osd":6,"last update":"Fri Jul 14 18:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64400000000000002,"15min":0.63400000000000001},"min":{"1min":0.52100000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84199999999999997,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.65600000000000003,"15min":0.63100000000000001},"min":{"1min":0.54600000000000004,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84799999999999998,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.73299999999999998}]},{"osd":7,"last update":"Fri Jul 14 18:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.60199999999999998}]},{"osd":8,"last update":"Fri Jul 14 18:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.65400000000000003,"15min":0.64600000000000002},"min":{"1min":0.50900000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.64800000000000002,"15min":0.63800000000000001},"min":{"1min":0.40500000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.94499999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.751}]},{"osd":9,"last update":"Fri Jul 14 18:14:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.79100000000000004,"15min":0.78700000000000003},"min":{"1min":0.58699999999999997,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.627},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.81499999999999995,"15min":0.81100000000000005},"min":{"1min":0.62,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.083,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.77300000000000002}]},{"osd":10,"last update":"Fri Jul 14 18:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.77600000000000002,"15min":0.75800000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.78000000000000003,"15min":0.76900000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.83599999999999997}]},{"osd":11,"last update":"Fri Jul 14 18:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.79200000000000004}]}]},{"osd":3,"up_from":26,"seq":111669149760,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":488211,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.43099999999999999,"15min":0.39100000000000001},"min":{"1min":0.32700000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45000000000000001,"15min":0.41399999999999998},"min":{"1min":0.34000000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.92400000000000004,"5min":0.998,"15min":0.998},"last":0.51900000000000002}]},{"osd":1,"last update":"Fri Jul 14 18:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.59799999999999998,"15min":0.57599999999999996},"min":{"1min":0.44500000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.85299999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.503},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.625,"15min":0.58999999999999997},"min":{"1min":0.54500000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.032,"5min":1.29,"15min":1.29},"last":0.64600000000000002}]},{"osd":2,"last update":"Fri Jul 14 18:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.61399999999999999,"15min":0.57899999999999996},"min":{"1min":0.52100000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.81599999999999995,"5min":1.087,"15min":1.087},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61799999999999999,"15min":0.59999999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95499999999999996,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.55400000000000005}]},{"osd":4,"last update":"Fri Jul 14 18:14:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.629,"15min":0.58899999999999997},"min":{"1min":0.47399999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.83599999999999997,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67100000000000004,"15min":0.63700000000000001},"min":{"1min":0.51300000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.875,"5min":1.153,"15min":1.153},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Jul 14 18:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.66700000000000004,"15min":0.63700000000000001},"min":{"1min":0.51200000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.66200000000000003,"15min":0.62},"min":{"1min":0.55900000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.66500000000000004}]},{"osd":6,"last update":"Fri Jul 14 18:14:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.621,"15min":0.59899999999999998},"min":{"1min":0.40699999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65400000000000003,"15min":0.622},"min":{"1min":0.47999999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.627}]},{"osd":7,"last update":"Fri Jul 14 18:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71299999999999997,"15min":0.68899999999999995},"min":{"1min":0.54100000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.73399999999999999,"15min":0.70699999999999996},"min":{"1min":0.61299999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.60299999999999998}]},{"osd":8,"last update":"Fri Jul 14 18:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.72799999999999998}]},{"osd":9,"last update":"Fri Jul 14 18:14:53 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.81599999999999995,"15min":0.82099999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.89300000000000002,"15min":0.91100000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.76400000000000001}]},{"osd":10,"last update":"Fri Jul 14 18:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.88300000000000001,"15min":0.88900000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.84399999999999997,"15min":0.85599999999999998},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.0589999999999999,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.74199999999999999}]},{"osd":11,"last update":"Fri Jul 14 18:15:19 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.754},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.77800000000000002}]}]},{"osd":4,"up_from":33,"seq":141733920828,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":493583,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.48999999999999999,"15min":0.46000000000000002},"min":{"1min":0.39700000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.83799999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.50800000000000001,"15min":0.47799999999999998},"min":{"1min":0.46700000000000003,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.747,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.39600000000000002}]},{"osd":1,"last update":"Fri Jul 14 18:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.43099999999999999,"15min":0.376},"min":{"1min":0.34200000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":3.7890000000000001,"5min":3.7890000000000001,"15min":3.7890000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.41999999999999998,"15min":0.38800000000000001},"min":{"1min":0.379,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.57699999999999996}]},{"osd":2,"last update":"Fri Jul 14 18:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.58999999999999997,"15min":0.55300000000000005},"min":{"1min":0.434,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.628,"15min":0.55800000000000005},"min":{"1min":0.48199999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.6989999999999998,"5min":3.6989999999999998,"15min":3.6989999999999998},"last":0.80500000000000005}]},{"osd":3,"last update":"Fri Jul 14 18:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.57999999999999996,"15min":0.52800000000000002},"min":{"1min":0.53600000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.79600000000000004,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.58399999999999996,"15min":0.53600000000000003},"min":{"1min":0.45100000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.5640000000000001,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":0.66200000000000003}]},{"osd":5,"last update":"Fri Jul 14 18:15:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.64600000000000002,"15min":0.57799999999999996},"min":{"1min":0.50600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":3.7490000000000001,"5min":3.7490000000000001,"15min":3.7490000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.63200000000000001,"15min":0.57699999999999996},"min":{"1min":0.52300000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86899999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.79400000000000004}]},{"osd":6,"last update":"Fri Jul 14 18:14:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67000000000000004,"15min":0.65800000000000003},"min":{"1min":0.50800000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.98799999999999999,"5min":1.075,"15min":1.075},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.68300000000000005,"15min":0.66600000000000004},"min":{"1min":0.498,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.006,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.70099999999999996}]},{"osd":7,"last update":"Fri Jul 14 18:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.65200000000000002,"15min":0.59199999999999997},"min":{"1min":0.54500000000000004,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":3.8999999999999999,"5min":3.8999999999999999,"15min":3.8999999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.875,"5min":0.65800000000000003,"15min":0.60399999999999998},"min":{"1min":0.47599999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.2250000000000001,"5min":3.2250000000000001,"15min":3.2250000000000001},"last":0.71999999999999997}]},{"osd":8,"last update":"Fri Jul 14 18:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.67800000000000005,"15min":0.65000000000000002},"min":{"1min":0.51100000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.79500000000000004,"15min":0.76300000000000001},"min":{"1min":0.58099999999999996,"5min":0.5,"15min":0.5},"max":{"1min":3.1739999999999999,"5min":3.1739999999999999,"15min":3.1739999999999999},"last":0.89100000000000001}]},{"osd":9,"last update":"Fri Jul 14 18:14:53 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.80400000000000005,"15min":0.79500000000000004},"min":{"1min":0.74399999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0700000000000001,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.877,"5min":0.73099999999999998,"15min":0.70699999999999996},"min":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.3889999999999998,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.91200000000000003}]},{"osd":10,"last update":"Fri Jul 14 18:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.81799999999999995,"15min":0.79400000000000004},"min":{"1min":0.68200000000000005,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":3.8239999999999998,"5min":3.8239999999999998,"15min":3.8239999999999998},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.754,"15min":0.73999999999999999},"min":{"1min":0.497,"5min":0.433,"15min":0.433},"max":{"1min":2.52,"5min":2.52,"15min":2.52},"last":0.82299999999999995}]},{"osd":11,"last update":"Fri Jul 14 18:15:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":3.548,"5min":3.548,"15min":3.548},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":0.97999999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757304,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":1083878,"data_compressed":53365,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:15:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56200000000000006,"15min":0.55300000000000005},"min":{"1min":0.39800000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71799999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56899999999999995,"15min":0.57899999999999996},"min":{"1min":0.46999999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76100000000000001,"5min":1.645,"15min":1.645},"last":0.621}]},{"osd":1,"last update":"Fri Jul 14 18:15:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.58999999999999997,"15min":0.55800000000000005},"min":{"1min":0.503,"5min":0.374,"15min":0.374},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.503},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58599999999999997,"15min":0.57099999999999995},"min":{"1min":0.48799999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.75600000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.73399999999999999}]},{"osd":2,"last update":"Fri Jul 14 18:15:19 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.42099999999999999,"15min":0.42399999999999999},"min":{"1min":0.32800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.622,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.46600000000000003,"15min":0.48899999999999999},"min":{"1min":0.314,"5min":0.252,"15min":0.252},"max":{"1min":0.61099999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.372}]},{"osd":3,"last update":"Fri Jul 14 18:15:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59599999999999997,"15min":0.61599999999999999},"min":{"1min":0.38,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.95699999999999996,"5min":1.591,"15min":1.591},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60699999999999998,"15min":0.60099999999999998},"min":{"1min":0.42599999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.79200000000000004,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.69799999999999995}]},{"osd":4,"last update":"Fri Jul 14 18:15:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.46899999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80900000000000005,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.623,"15min":0.61899999999999999},"min":{"1min":0.5,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79800000000000004,"5min":1.504,"15min":1.504},"last":0.53300000000000003}]},{"osd":6,"last update":"Fri Jul 14 18:14:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.65400000000000003,"15min":0.66000000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.79300000000000004,"5min":1.627,"15min":1.627},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65100000000000002,"15min":0.63600000000000001},"min":{"1min":0.51000000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.83099999999999996,"5min":1.036,"15min":1.036},"last":0.63500000000000001}]},{"osd":7,"last update":"Fri Jul 14 18:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.68500000000000005},"min":{"1min":0.439,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.84899999999999998,"5min":1.554,"15min":1.554},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.67100000000000004,"15min":0.65500000000000003},"min":{"1min":0.51000000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.56999999999999995}]},{"osd":8,"last update":"Fri Jul 14 18:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59399999999999997,"15min":0.57999999999999996},"min":{"1min":0.36599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80600000000000005,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59499999999999997,"15min":0.58299999999999996},"min":{"1min":0.41199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.443}]},{"osd":9,"last update":"Fri Jul 14 18:14:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72899999999999998,"15min":0.72999999999999998},"min":{"1min":0.51800000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92600000000000005,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.76500000000000001,"15min":0.76800000000000002},"min":{"1min":0.46700000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.98199999999999998,"5min":1.149,"15min":1.149},"last":0.748}]},{"osd":10,"last update":"Fri Jul 14 18:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.73799999999999999,"15min":0.72799999999999998},"min":{"1min":0.67100000000000004,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.91300000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.76200000000000001,"15min":0.752},"min":{"1min":0.58199999999999996,"5min":0.505,"15min":0.505},"max":{"1min":1.0349999999999999,"5min":1.107,"15min":1.107},"last":0.66600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34102,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34102,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34102,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":60390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":60536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T18:15:39.496 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph pg dump --format=json 2023-07-14T18:15:40.217 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:40.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:39 smithi161 ceph-mon[128993]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T18:15:40.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:39 smithi171 ceph-mon[129241]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T18:15:40.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:39 smithi133 ceph-mon[126384]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T18:15:41.053 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:40 smithi133 ceph-mon[126384]: pgmap v316: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-14T18:15:41.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:40 smithi161 ceph-mon[128993]: pgmap v316: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-14T18:15:41.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:40 smithi171 ceph-mon[129241]: pgmap v316: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-14T18:15:41.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.577+0000 7fd6d4ba2700 1 -- 172.21.15.133:0/2454770261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6d01008d0 msgr2=0x7fd6d01048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:41.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.577+0000 7fd6d4ba2700 1 --2- 172.21.15.133:0/2454770261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6d01008d0 0x7fd6d01048a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd6c40099c0 tx=0x7fd6c4009cd0 comp rx=0 tx=0).stop 2023-07-14T18:15:41.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.577+0000 7fd6d4ba2700 1 -- 172.21.15.133:0/2454770261 shutdown_connections 2023-07-14T18:15:41.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.577+0000 7fd6d4ba2700 1 --2- 172.21.15.133:0/2454770261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d00fff30 0x7fd6d0100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:41.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.577+0000 7fd6d4ba2700 1 --2- 172.21.15.133:0/2454770261 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6d0104fd0 0x7fd6d010cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:41.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.577+0000 7fd6d4ba2700 1 --2- 172.21.15.133:0/2454770261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6d01008d0 0x7fd6d01048a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:41.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.577+0000 7fd6d4ba2700 1 -- 172.21.15.133:0/2454770261 >> 172.21.15.133:0/2454770261 conn(0x7fd6d00fb790 msgr2=0x7fd6d00fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:41.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.577+0000 7fd6d4ba2700 1 -- 172.21.15.133:0/2454770261 shutdown_connections 2023-07-14T18:15:41.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.577+0000 7fd6d4ba2700 1 -- 172.21.15.133:0/2454770261 wait complete. 2023-07-14T18:15:41.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.579+0000 7fd6d4ba2700 1 Processor -- start 2023-07-14T18:15:41.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.579+0000 7fd6d4ba2700 1 -- start start 2023-07-14T18:15:41.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.579+0000 7fd6d4ba2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d00fff30 0x7fd6d0195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:41.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.579+0000 7fd6d4ba2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6d01008d0 0x7fd6d01963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:41.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.579+0000 7fd6d4ba2700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6d0104fd0 0x7fd6d019a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:41.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.579+0000 7fd6d4ba2700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d019ae50 con 0x7fd6d01008d0 2023-07-14T18:15:41.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.579+0000 7fd6d4ba2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d019afc0 con 0x7fd6d00fff30 2023-07-14T18:15:41.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.579+0000 7fd6d4ba2700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d019b180 con 0x7fd6d0104fd0 2023-07-14T18:15:41.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.579+0000 7fd6ce59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d00fff30 0x7fd6d0195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:41.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.579+0000 7fd6ce59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d00fff30 0x7fd6d0195e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:38062/0 (socket says 172.21.15.133:38062) 2023-07-14T18:15:41.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.579+0000 7fd6ce59c700 1 -- 172.21.15.133:0/1430293409 learned_addr learned my addr 172.21.15.133:0/1430293409 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:41.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.580+0000 7fd6ced9d700 1 --2- 172.21.15.133:0/1430293409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6d0104fd0 0x7fd6d019a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:41.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.580+0000 7fd6cdd9b700 1 --2- 172.21.15.133:0/1430293409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6d01008d0 0x7fd6d01963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:41.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.580+0000 7fd6ce59c700 1 -- 172.21.15.133:0/1430293409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6d0104fd0 msgr2=0x7fd6d019a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:41.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.580+0000 7fd6ce59c700 1 --2- 172.21.15.133:0/1430293409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6d0104fd0 0x7fd6d019a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:41.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.580+0000 7fd6ce59c700 1 -- 172.21.15.133:0/1430293409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6d01008d0 msgr2=0x7fd6d01963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:41.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.580+0000 7fd6ce59c700 1 --2- 172.21.15.133:0/1430293409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6d01008d0 0x7fd6d01963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:41.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.580+0000 7fd6ce59c700 1 -- 172.21.15.133:0/1430293409 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c40095f0 con 0x7fd6d00fff30 2023-07-14T18:15:41.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.580+0000 7fd6cdd9b700 1 --2- 172.21.15.133:0/1430293409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6d01008d0 0x7fd6d01963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:15:41.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.580+0000 7fd6ce59c700 1 --2- 172.21.15.133:0/1430293409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d00fff30 0x7fd6d0195e90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000e9a0 tx=0x7fd6c000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:41.629 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.580+0000 7fd6bf7fe700 1 -- 172.21.15.133:0/1430293409 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0009790 con 0x7fd6d00fff30 2023-07-14T18:15:41.629 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.580+0000 7fd6bf7fe700 1 -- 172.21.15.133:0/1430293409 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6c000cbd0 con 0x7fd6d00fff30 2023-07-14T18:15:41.629 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.580+0000 7fd6d4ba2700 1 -- 172.21.15.133:0/1430293409 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d019b460 con 0x7fd6d00fff30 2023-07-14T18:15:41.629 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.580+0000 7fd6bf7fe700 1 -- 172.21.15.133:0/1430293409 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0018680 con 0x7fd6d00fff30 2023-07-14T18:15:41.629 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.581+0000 7fd6d4ba2700 1 -- 172.21.15.133:0/1430293409 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d01a2cc0 con 0x7fd6d00fff30 2023-07-14T18:15:41.630 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.581+0000 7fd6d4ba2700 1 -- 172.21.15.133:0/1430293409 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d004e340 con 0x7fd6d00fff30 2023-07-14T18:15:41.630 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.584+0000 7fd6bf7fe700 1 -- 172.21.15.133:0/1430293409 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fd6c000cd40 con 0x7fd6d00fff30 2023-07-14T18:15:41.630 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.585+0000 7fd6bf7fe700 1 --2- 172.21.15.133:0/1430293409 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd6b80a1260 0x7fd6b80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:41.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.585+0000 7fd6bf7fe700 1 -- 172.21.15.133:0/1430293409 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd6c0014070 con 0x7fd6d00fff30 2023-07-14T18:15:41.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.585+0000 7fd6cdd9b700 1 --2- 172.21.15.133:0/1430293409 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd6b80a1260 0x7fd6b80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:41.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.586+0000 7fd6cdd9b700 1 --2- 172.21.15.133:0/1430293409 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd6b80a1260 0x7fd6b80a3710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4009fd0 tx=0x7fd6c401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:41.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.587+0000 7fd6bf7fe700 1 -- 172.21.15.133:0/1430293409 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fd6c0096520 con 0x7fd6d00fff30 2023-07-14T18:15:41.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.777+0000 7fd6d4ba2700 1 -- 172.21.15.133:0/1430293409 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd6d00fc970 con 0x7fd6b80a1260 2023-07-14T18:15:41.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.789+0000 7fd6bf7fe700 1 -- 172.21.15.133:0/1430293409 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427137 (secure 0 0 0) 0x7fd6d00fc970 con 0x7fd6b80a1260 2023-07-14T18:15:41.841 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:15:41.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.795+0000 7fd6d4ba2700 1 -- 172.21.15.133:0/1430293409 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd6b80a1260 msgr2=0x7fd6b80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:41.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.795+0000 7fd6d4ba2700 1 --2- 172.21.15.133:0/1430293409 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd6b80a1260 0x7fd6b80a3710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4009fd0 tx=0x7fd6c401a040 comp rx=0 tx=0).stop 2023-07-14T18:15:41.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.795+0000 7fd6d4ba2700 1 -- 172.21.15.133:0/1430293409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d00fff30 msgr2=0x7fd6d0195e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:41.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.795+0000 7fd6d4ba2700 1 --2- 172.21.15.133:0/1430293409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d00fff30 0x7fd6d0195e90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000e9a0 tx=0x7fd6c000ecb0 comp rx=0 tx=0).stop 2023-07-14T18:15:41.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.795+0000 7fd6d4ba2700 1 -- 172.21.15.133:0/1430293409 shutdown_connections 2023-07-14T18:15:41.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.795+0000 7fd6d4ba2700 1 --2- 172.21.15.133:0/1430293409 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd6b80a1260 0x7fd6b80a3710 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:41.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.795+0000 7fd6d4ba2700 1 --2- 172.21.15.133:0/1430293409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d00fff30 0x7fd6d0195e90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:41.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.795+0000 7fd6d4ba2700 1 --2- 172.21.15.133:0/1430293409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6d0104fd0 0x7fd6d019a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:41.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.795+0000 7fd6d4ba2700 1 --2- 172.21.15.133:0/1430293409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6d01008d0 0x7fd6d01963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:41.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.795+0000 7fd6d4ba2700 1 -- 172.21.15.133:0/1430293409 >> 172.21.15.133:0/1430293409 conn(0x7fd6d00fb790 msgr2=0x7fd6d00fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:41.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.796+0000 7fd6d4ba2700 1 -- 172.21.15.133:0/1430293409 shutdown_connections 2023-07-14T18:15:41.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:41.796+0000 7fd6d4ba2700 1 -- 172.21.15.133:0/1430293409 wait complete. 2023-07-14T18:15:41.845 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2023-07-14T18:15:42.306 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2023-07-14T18:15:41.451401+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654326,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":442,"num_read_kb":334,"num_write":340,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":277,"ondisk_log_size":277,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336208,"kb_used_data":12000,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124605616,"statfs":{"total":1151940427776,"available":1151596150784,"internally_reserved":0,"allocated":12288000,"data_stored":7849470,"data_compressed":318684,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003206"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945666+0000","last_change":"2023-07-14T18:14:17.927179+0000","last_active":"2023-07-14T18:14:48.945666+0000","last_peered":"2023-07-14T18:14:48.945666+0000","last_clean":"2023-07-14T18:14:48.945666+0000","last_became_active":"2023-07-14T18:14:17.926880+0000","last_became_peered":"2023-07-14T18:14:17.926880+0000","last_unstale":"2023-07-14T18:14:48.945666+0000","last_undegraded":"2023-07-14T18:14:48.945666+0000","last_fullsized":"2023-07-14T18:14:48.945666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:26:26.468709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541115+0000","last_change":"2023-07-14T18:14:17.933834+0000","last_active":"2023-07-14T18:14:49.541115+0000","last_peered":"2023-07-14T18:14:49.541115+0000","last_clean":"2023-07-14T18:14:49.541115+0000","last_became_active":"2023-07-14T18:14:17.933529+0000","last_became_peered":"2023-07-14T18:14:17.933529+0000","last_unstale":"2023-07-14T18:14:49.541115+0000","last_undegraded":"2023-07-14T18:14:49.541115+0000","last_fullsized":"2023-07-14T18:14:49.541115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:34:13.388837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540772+0000","last_change":"2023-07-14T18:14:17.928400+0000","last_active":"2023-07-14T18:14:49.540772+0000","last_peered":"2023-07-14T18:14:49.540772+0000","last_clean":"2023-07-14T18:14:49.540772+0000","last_became_active":"2023-07-14T18:14:17.928077+0000","last_became_peered":"2023-07-14T18:14:17.928077+0000","last_unstale":"2023-07-14T18:14:49.540772+0000","last_undegraded":"2023-07-14T18:14:49.540772+0000","last_fullsized":"2023-07-14T18:14:49.540772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:24.522613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.947528+0000","last_change":"2023-07-14T18:14:17.932623+0000","last_active":"2023-07-14T18:14:48.947528+0000","last_peered":"2023-07-14T18:14:48.947528+0000","last_clean":"2023-07-14T18:14:48.947528+0000","last_became_active":"2023-07-14T18:14:17.932298+0000","last_became_peered":"2023-07-14T18:14:17.932298+0000","last_unstale":"2023-07-14T18:14:48.947528+0000","last_undegraded":"2023-07-14T18:14:48.947528+0000","last_fullsized":"2023-07-14T18:14:48.947528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:17:46.564398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945288+0000","last_change":"2023-07-14T18:14:17.927109+0000","last_active":"2023-07-14T18:14:48.945288+0000","last_peered":"2023-07-14T18:14:48.945288+0000","last_clean":"2023-07-14T18:14:48.945288+0000","last_became_active":"2023-07-14T18:14:17.926800+0000","last_became_peered":"2023-07-14T18:14:17.926800+0000","last_unstale":"2023-07-14T18:14:48.945288+0000","last_undegraded":"2023-07-14T18:14:48.945288+0000","last_fullsized":"2023-07-14T18:14:48.945288+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:43:24.990024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541350+0000","last_change":"2023-07-14T18:14:17.935380+0000","last_active":"2023-07-14T18:14:49.541350+0000","last_peered":"2023-07-14T18:14:49.541350+0000","last_clean":"2023-07-14T18:14:49.541350+0000","last_became_active":"2023-07-14T18:14:17.935099+0000","last_became_peered":"2023-07-14T18:14:17.935099+0000","last_unstale":"2023-07-14T18:14:49.541350+0000","last_undegraded":"2023-07-14T18:14:49.541350+0000","last_fullsized":"2023-07-14T18:14:49.541350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:04:06.339952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.943679+0000","last_change":"2023-07-14T18:14:17.930207+0000","last_active":"2023-07-14T18:14:48.943679+0000","last_peered":"2023-07-14T18:14:48.943679+0000","last_clean":"2023-07-14T18:14:48.943679+0000","last_became_active":"2023-07-14T18:14:17.929898+0000","last_became_peered":"2023-07-14T18:14:17.929898+0000","last_unstale":"2023-07-14T18:14:48.943679+0000","last_undegraded":"2023-07-14T18:14:48.943679+0000","last_fullsized":"2023-07-14T18:14:48.943679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:33:33.679907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541324+0000","last_change":"2023-07-14T18:14:17.928645+0000","last_active":"2023-07-14T18:14:49.541324+0000","last_peered":"2023-07-14T18:14:49.541324+0000","last_clean":"2023-07-14T18:14:49.541324+0000","last_became_active":"2023-07-14T18:14:17.928288+0000","last_became_peered":"2023-07-14T18:14:17.928288+0000","last_unstale":"2023-07-14T18:14:49.541324+0000","last_undegraded":"2023-07-14T18:14:49.541324+0000","last_fullsized":"2023-07-14T18:14:49.541324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:31:13.535796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.948678+0000","last_change":"2023-07-14T18:14:17.925888+0000","last_active":"2023-07-14T18:14:48.948678+0000","last_peered":"2023-07-14T18:14:48.948678+0000","last_clean":"2023-07-14T18:14:48.948678+0000","last_became_active":"2023-07-14T18:14:17.925580+0000","last_became_peered":"2023-07-14T18:14:17.925580+0000","last_unstale":"2023-07-14T18:14:48.948678+0000","last_undegraded":"2023-07-14T18:14:48.948678+0000","last_fullsized":"2023-07-14T18:14:48.948678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:28:42.610213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541751+0000","last_change":"2023-07-14T18:14:17.924440+0000","last_active":"2023-07-14T18:14:49.541751+0000","last_peered":"2023-07-14T18:14:49.541751+0000","last_clean":"2023-07-14T18:14:49.541751+0000","last_became_active":"2023-07-14T18:14:17.923839+0000","last_became_peered":"2023-07-14T18:14:17.923839+0000","last_unstale":"2023-07-14T18:14:49.541751+0000","last_undegraded":"2023-07-14T18:14:49.541751+0000","last_fullsized":"2023-07-14T18:14:49.541751+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:11:20.578050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944134+0000","last_change":"2023-07-14T18:14:17.931067+0000","last_active":"2023-07-14T18:14:48.944134+0000","last_peered":"2023-07-14T18:14:48.944134+0000","last_clean":"2023-07-14T18:14:48.944134+0000","last_became_active":"2023-07-14T18:14:17.930633+0000","last_became_peered":"2023-07-14T18:14:17.930633+0000","last_unstale":"2023-07-14T18:14:48.944134+0000","last_undegraded":"2023-07-14T18:14:48.944134+0000","last_fullsized":"2023-07-14T18:14:48.944134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:35:03.441365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.947103+0000","last_change":"2023-07-14T18:14:17.932770+0000","last_active":"2023-07-14T18:14:48.947103+0000","last_peered":"2023-07-14T18:14:48.947103+0000","last_clean":"2023-07-14T18:14:48.947103+0000","last_became_active":"2023-07-14T18:14:17.932438+0000","last_became_peered":"2023-07-14T18:14:17.932438+0000","last_unstale":"2023-07-14T18:14:48.947103+0000","last_undegraded":"2023-07-14T18:14:48.947103+0000","last_fullsized":"2023-07-14T18:14:48.947103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:45:58.878384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942853+0000","last_change":"2023-07-14T18:14:17.938213+0000","last_active":"2023-07-14T18:14:48.942853+0000","last_peered":"2023-07-14T18:14:48.942853+0000","last_clean":"2023-07-14T18:14:48.942853+0000","last_became_active":"2023-07-14T18:14:17.937565+0000","last_became_peered":"2023-07-14T18:14:17.937565+0000","last_unstale":"2023-07-14T18:14:48.942853+0000","last_undegraded":"2023-07-14T18:14:48.942853+0000","last_fullsized":"2023-07-14T18:14:48.942853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:18:33.559796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541023+0000","last_change":"2023-07-14T18:14:17.934677+0000","last_active":"2023-07-14T18:14:49.541023+0000","last_peered":"2023-07-14T18:14:49.541023+0000","last_clean":"2023-07-14T18:14:49.541023+0000","last_became_active":"2023-07-14T18:14:17.933622+0000","last_became_peered":"2023-07-14T18:14:17.933622+0000","last_unstale":"2023-07-14T18:14:49.541023+0000","last_undegraded":"2023-07-14T18:14:49.541023+0000","last_fullsized":"2023-07-14T18:14:49.541023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:44:58.693553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540682+0000","last_change":"2023-07-14T18:14:17.935317+0000","last_active":"2023-07-14T18:14:49.540682+0000","last_peered":"2023-07-14T18:14:49.540682+0000","last_clean":"2023-07-14T18:14:49.540682+0000","last_became_active":"2023-07-14T18:14:17.935034+0000","last_became_peered":"2023-07-14T18:14:17.935034+0000","last_unstale":"2023-07-14T18:14:49.540682+0000","last_undegraded":"2023-07-14T18:14:49.540682+0000","last_fullsized":"2023-07-14T18:14:49.540682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:45:23.409187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946950+0000","last_change":"2023-07-14T18:14:17.928222+0000","last_active":"2023-07-14T18:14:48.946950+0000","last_peered":"2023-07-14T18:14:48.946950+0000","last_clean":"2023-07-14T18:14:48.946950+0000","last_became_active":"2023-07-14T18:14:17.927964+0000","last_became_peered":"2023-07-14T18:14:17.927964+0000","last_unstale":"2023-07-14T18:14:48.946950+0000","last_undegraded":"2023-07-14T18:14:48.946950+0000","last_fullsized":"2023-07-14T18:14:48.946950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:44:57.639864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944940+0000","last_change":"2023-07-14T18:14:17.927276+0000","last_active":"2023-07-14T18:14:48.944940+0000","last_peered":"2023-07-14T18:14:48.944940+0000","last_clean":"2023-07-14T18:14:48.944940+0000","last_became_active":"2023-07-14T18:14:17.926952+0000","last_became_peered":"2023-07-14T18:14:17.926952+0000","last_unstale":"2023-07-14T18:14:48.944940+0000","last_undegraded":"2023-07-14T18:14:48.944940+0000","last_fullsized":"2023-07-14T18:14:48.944940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:48:53.419369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945947+0000","last_change":"2023-07-14T18:14:17.937364+0000","last_active":"2023-07-14T18:14:48.945947+0000","last_peered":"2023-07-14T18:14:48.945947+0000","last_clean":"2023-07-14T18:14:48.945947+0000","last_became_active":"2023-07-14T18:14:17.937014+0000","last_became_peered":"2023-07-14T18:14:17.937014+0000","last_unstale":"2023-07-14T18:14:48.945947+0000","last_undegraded":"2023-07-14T18:14:48.945947+0000","last_fullsized":"2023-07-14T18:14:48.945947+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:57:26.423684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541653+0000","last_change":"2023-07-14T18:14:17.920358+0000","last_active":"2023-07-14T18:14:49.541653+0000","last_peered":"2023-07-14T18:14:49.541653+0000","last_clean":"2023-07-14T18:14:49.541653+0000","last_became_active":"2023-07-14T18:14:17.920042+0000","last_became_peered":"2023-07-14T18:14:17.920042+0000","last_unstale":"2023-07-14T18:14:49.541653+0000","last_undegraded":"2023-07-14T18:14:49.541653+0000","last_fullsized":"2023-07-14T18:14:49.541653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:53:36.975517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541158+0000","last_change":"2023-07-14T18:14:22.045376+0000","last_active":"2023-07-14T18:14:49.541158+0000","last_peered":"2023-07-14T18:14:49.541158+0000","last_clean":"2023-07-14T18:14:49.541158+0000","last_became_active":"2023-07-14T18:14:22.044984+0000","last_became_peered":"2023-07-14T18:14:22.044984+0000","last_unstale":"2023-07-14T18:14:49.541158+0000","last_undegraded":"2023-07-14T18:14:49.541158+0000","last_fullsized":"2023-07-14T18:14:49.541158+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:47:15.523409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.894895+0000","last_change":"2023-07-14T18:14:48.954270+0000","last_active":"2023-07-14T18:14:49.894895+0000","last_peered":"2023-07-14T18:14:49.894895+0000","last_clean":"2023-07-14T18:14:49.894895+0000","last_became_active":"2023-07-14T18:14:48.947868+0000","last_became_peered":"2023-07-14T18:14:48.947868+0000","last_unstale":"2023-07-14T18:14:49.894895+0000","last_undegraded":"2023-07-14T18:14:49.894895+0000","last_fullsized":"2023-07-14T18:14:49.894895+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:13:41.862407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540866+0000","last_change":"2023-07-14T18:14:17.925822+0000","last_active":"2023-07-14T18:14:49.540866+0000","last_peered":"2023-07-14T18:14:49.540866+0000","last_clean":"2023-07-14T18:14:49.540866+0000","last_became_active":"2023-07-14T18:14:17.925508+0000","last_became_peered":"2023-07-14T18:14:17.925508+0000","last_unstale":"2023-07-14T18:14:49.540866+0000","last_undegraded":"2023-07-14T18:14:49.540866+0000","last_fullsized":"2023-07-14T18:14:49.540866+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:19:54.558854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946499+0000","last_change":"2023-07-14T18:14:22.043098+0000","last_active":"2023-07-14T18:14:48.946499+0000","last_peered":"2023-07-14T18:14:48.946499+0000","last_clean":"2023-07-14T18:14:48.946499+0000","last_became_active":"2023-07-14T18:14:22.042517+0000","last_became_peered":"2023-07-14T18:14:22.042517+0000","last_unstale":"2023-07-14T18:14:48.946499+0000","last_undegraded":"2023-07-14T18:14:48.946499+0000","last_fullsized":"2023-07-14T18:14:48.946499+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:33:54.491558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541428+0000","last_change":"2023-07-14T18:14:16.927612+0000","last_active":"2023-07-14T18:14:49.541428+0000","last_peered":"2023-07-14T18:14:49.541428+0000","last_clean":"2023-07-14T18:14:49.541428+0000","last_became_active":"2023-07-14T18:14:16.927284+0000","last_became_peered":"2023-07-14T18:14:16.927284+0000","last_unstale":"2023-07-14T18:14:49.541428+0000","last_undegraded":"2023-07-14T18:14:49.541428+0000","last_fullsized":"2023-07-14T18:14:49.541428+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:56:37.293734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540962+0000","last_change":"2023-07-14T18:14:17.930624+0000","last_active":"2023-07-14T18:14:49.540962+0000","last_peered":"2023-07-14T18:14:49.540962+0000","last_clean":"2023-07-14T18:14:49.540962+0000","last_became_active":"2023-07-14T18:14:17.930273+0000","last_became_peered":"2023-07-14T18:14:17.930273+0000","last_unstale":"2023-07-14T18:14:49.540962+0000","last_undegraded":"2023-07-14T18:14:49.540962+0000","last_fullsized":"2023-07-14T18:14:49.540962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:19:40.573682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.543208+0000","last_change":"2023-07-14T18:14:49.543208+0000","last_active":"2023-07-14T18:14:49.543208+0000","last_peered":"2023-07-14T18:14:49.543208+0000","last_clean":"2023-07-14T18:14:49.543208+0000","last_became_active":"2023-07-14T18:14:49.542685+0000","last_became_peered":"2023-07-14T18:14:49.542685+0000","last_unstale":"2023-07-14T18:14:49.543208+0000","last_undegraded":"2023-07-14T18:14:49.543208+0000","last_fullsized":"2023-07-14T18:14:49.543208+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:32:12.710336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.894625+0000","last_change":"2023-07-14T18:14:16.926185+0000","last_active":"2023-07-14T18:14:49.894625+0000","last_peered":"2023-07-14T18:14:49.894625+0000","last_clean":"2023-07-14T18:14:49.894625+0000","last_became_active":"2023-07-14T18:14:16.925874+0000","last_became_peered":"2023-07-14T18:14:16.925874+0000","last_unstale":"2023-07-14T18:14:49.894625+0000","last_undegraded":"2023-07-14T18:14:49.894625+0000","last_fullsized":"2023-07-14T18:14:49.894625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:26:07.280801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.947867+0000","last_change":"2023-07-14T18:14:17.936601+0000","last_active":"2023-07-14T18:14:48.947867+0000","last_peered":"2023-07-14T18:14:48.947867+0000","last_clean":"2023-07-14T18:14:48.947867+0000","last_became_active":"2023-07-14T18:14:17.936291+0000","last_became_peered":"2023-07-14T18:14:17.936291+0000","last_unstale":"2023-07-14T18:14:48.947867+0000","last_undegraded":"2023-07-14T18:14:48.947867+0000","last_fullsized":"2023-07-14T18:14:48.947867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:50:25.290580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946635+0000","last_change":"2023-07-14T18:14:22.042952+0000","last_active":"2023-07-14T18:14:48.946635+0000","last_peered":"2023-07-14T18:14:48.946635+0000","last_clean":"2023-07-14T18:14:48.946635+0000","last_became_active":"2023-07-14T18:14:22.042256+0000","last_became_peered":"2023-07-14T18:14:22.042256+0000","last_unstale":"2023-07-14T18:14:48.946635+0000","last_undegraded":"2023-07-14T18:14:48.946635+0000","last_fullsized":"2023-07-14T18:14:48.946635+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:20:29.914375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945839+0000","last_change":"2023-07-14T18:14:16.920902+0000","last_active":"2023-07-14T18:14:48.945839+0000","last_peered":"2023-07-14T18:14:48.945839+0000","last_clean":"2023-07-14T18:14:48.945839+0000","last_became_active":"2023-07-14T18:14:16.920554+0000","last_became_peered":"2023-07-14T18:14:16.920554+0000","last_unstale":"2023-07-14T18:14:48.945839+0000","last_undegraded":"2023-07-14T18:14:48.945839+0000","last_fullsized":"2023-07-14T18:14:48.945839+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:05:37.194027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945093+0000","last_change":"2023-07-14T18:14:17.925005+0000","last_active":"2023-07-14T18:14:48.945093+0000","last_peered":"2023-07-14T18:14:48.945093+0000","last_clean":"2023-07-14T18:14:48.945093+0000","last_became_active":"2023-07-14T18:14:17.924723+0000","last_became_peered":"2023-07-14T18:14:17.924723+0000","last_unstale":"2023-07-14T18:14:48.945093+0000","last_undegraded":"2023-07-14T18:14:48.945093+0000","last_fullsized":"2023-07-14T18:14:48.945093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:34:45.456102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541394+0000","last_change":"2023-07-14T18:14:22.038482+0000","last_active":"2023-07-14T18:14:49.541394+0000","last_peered":"2023-07-14T18:14:49.541394+0000","last_clean":"2023-07-14T18:14:49.541394+0000","last_became_active":"2023-07-14T18:14:22.038059+0000","last_became_peered":"2023-07-14T18:14:22.038059+0000","last_unstale":"2023-07-14T18:14:49.541394+0000","last_undegraded":"2023-07-14T18:14:49.541394+0000","last_fullsized":"2023-07-14T18:14:49.541394+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:40:04.711668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":88,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:15:34.953612+0000","last_change":"2023-07-14T18:14:48.949585+0000","last_active":"2023-07-14T18:15:34.953612+0000","last_peered":"2023-07-14T18:15:34.953612+0000","last_clean":"2023-07-14T18:15:34.953612+0000","last_became_active":"2023-07-14T18:14:48.949106+0000","last_became_peered":"2023-07-14T18:14:48.949106+0000","last_unstale":"2023-07-14T18:15:34.953612+0000","last_undegraded":"2023-07-14T18:15:34.953612+0000","last_fullsized":"2023-07-14T18:15:34.953612+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:27:44.232817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945805+0000","last_change":"2023-07-14T18:14:17.934894+0000","last_active":"2023-07-14T18:14:48.945805+0000","last_peered":"2023-07-14T18:14:48.945805+0000","last_clean":"2023-07-14T18:14:48.945805+0000","last_became_active":"2023-07-14T18:14:17.934651+0000","last_became_peered":"2023-07-14T18:14:17.934651+0000","last_unstale":"2023-07-14T18:14:48.945805+0000","last_undegraded":"2023-07-14T18:14:48.945805+0000","last_fullsized":"2023-07-14T18:14:48.945805+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:07:18.984143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541406+0000","last_change":"2023-07-14T18:14:22.043611+0000","last_active":"2023-07-14T18:14:49.541406+0000","last_peered":"2023-07-14T18:14:49.541406+0000","last_clean":"2023-07-14T18:14:49.541406+0000","last_became_active":"2023-07-14T18:14:22.043101+0000","last_became_peered":"2023-07-14T18:14:22.043101+0000","last_unstale":"2023-07-14T18:14:49.541406+0000","last_undegraded":"2023-07-14T18:14:49.541406+0000","last_fullsized":"2023-07-14T18:14:49.541406+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:09:27.645400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.894491+0000","last_change":"2023-07-14T18:14:16.925321+0000","last_active":"2023-07-14T18:14:49.894491+0000","last_peered":"2023-07-14T18:14:49.894491+0000","last_clean":"2023-07-14T18:14:49.894491+0000","last_became_active":"2023-07-14T18:14:16.925016+0000","last_became_peered":"2023-07-14T18:14:16.925016+0000","last_unstale":"2023-07-14T18:14:49.894491+0000","last_undegraded":"2023-07-14T18:14:49.894491+0000","last_fullsized":"2023-07-14T18:14:49.894491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:51:09.710945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944916+0000","last_change":"2023-07-14T18:14:17.928838+0000","last_active":"2023-07-14T18:14:48.944916+0000","last_peered":"2023-07-14T18:14:48.944916+0000","last_clean":"2023-07-14T18:14:48.944916+0000","last_became_active":"2023-07-14T18:14:17.928511+0000","last_became_peered":"2023-07-14T18:14:17.928511+0000","last_unstale":"2023-07-14T18:14:48.944916+0000","last_undegraded":"2023-07-14T18:14:48.944916+0000","last_fullsized":"2023-07-14T18:14:48.944916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:25:59.076594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944791+0000","last_change":"2023-07-14T18:14:22.043182+0000","last_active":"2023-07-14T18:14:48.944791+0000","last_peered":"2023-07-14T18:14:48.944791+0000","last_clean":"2023-07-14T18:14:48.944791+0000","last_became_active":"2023-07-14T18:14:22.042779+0000","last_became_peered":"2023-07-14T18:14:22.042779+0000","last_unstale":"2023-07-14T18:14:48.944791+0000","last_undegraded":"2023-07-14T18:14:48.944791+0000","last_fullsized":"2023-07-14T18:14:48.944791+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:15:03.016184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.895882+0000","last_change":"2023-07-14T18:14:16.935321+0000","last_active":"2023-07-14T18:14:49.895882+0000","last_peered":"2023-07-14T18:14:49.895882+0000","last_clean":"2023-07-14T18:14:49.895882+0000","last_became_active":"2023-07-14T18:14:16.934494+0000","last_became_peered":"2023-07-14T18:14:16.934494+0000","last_unstale":"2023-07-14T18:14:49.895882+0000","last_undegraded":"2023-07-14T18:14:49.895882+0000","last_fullsized":"2023-07-14T18:14:49.895882+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:00:50.926893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541135+0000","last_change":"2023-07-14T18:14:17.922139+0000","last_active":"2023-07-14T18:14:49.541135+0000","last_peered":"2023-07-14T18:14:49.541135+0000","last_clean":"2023-07-14T18:14:49.541135+0000","last_became_active":"2023-07-14T18:14:17.921829+0000","last_became_peered":"2023-07-14T18:14:17.921829+0000","last_unstale":"2023-07-14T18:14:49.541135+0000","last_undegraded":"2023-07-14T18:14:49.541135+0000","last_fullsized":"2023-07-14T18:14:49.541135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:16:28.099919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540988+0000","last_change":"2023-07-14T18:14:22.033625+0000","last_active":"2023-07-14T18:14:49.540988+0000","last_peered":"2023-07-14T18:14:49.540988+0000","last_clean":"2023-07-14T18:14:49.540988+0000","last_became_active":"2023-07-14T18:14:22.033253+0000","last_became_peered":"2023-07-14T18:14:22.033253+0000","last_unstale":"2023-07-14T18:14:49.540988+0000","last_undegraded":"2023-07-14T18:14:49.540988+0000","last_fullsized":"2023-07-14T18:14:49.540988+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:16:24.641370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942449+0000","last_change":"2023-07-14T18:14:16.923864+0000","last_active":"2023-07-14T18:14:48.942449+0000","last_peered":"2023-07-14T18:14:48.942449+0000","last_clean":"2023-07-14T18:14:48.942449+0000","last_became_active":"2023-07-14T18:14:16.923546+0000","last_became_peered":"2023-07-14T18:14:16.923546+0000","last_unstale":"2023-07-14T18:14:48.942449+0000","last_undegraded":"2023-07-14T18:14:48.942449+0000","last_fullsized":"2023-07-14T18:14:48.942449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:31:33.868703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945453+0000","last_change":"2023-07-14T18:14:17.936473+0000","last_active":"2023-07-14T18:14:48.945453+0000","last_peered":"2023-07-14T18:14:48.945453+0000","last_clean":"2023-07-14T18:14:48.945453+0000","last_became_active":"2023-07-14T18:14:17.935387+0000","last_became_peered":"2023-07-14T18:14:17.935387+0000","last_unstale":"2023-07-14T18:14:48.945453+0000","last_undegraded":"2023-07-14T18:14:48.945453+0000","last_fullsized":"2023-07-14T18:14:48.945453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:09:31.140915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541512+0000","last_change":"2023-07-14T18:14:22.041544+0000","last_active":"2023-07-14T18:14:49.541512+0000","last_peered":"2023-07-14T18:14:49.541512+0000","last_clean":"2023-07-14T18:14:49.541512+0000","last_became_active":"2023-07-14T18:14:22.041183+0000","last_became_peered":"2023-07-14T18:14:22.041183+0000","last_unstale":"2023-07-14T18:14:49.541512+0000","last_undegraded":"2023-07-14T18:14:49.541512+0000","last_fullsized":"2023-07-14T18:14:49.541512+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:24:18.995001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":75,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:15:35.955078+0000","last_change":"2023-07-14T18:14:16.928860+0000","last_active":"2023-07-14T18:15:35.955078+0000","last_peered":"2023-07-14T18:15:35.955078+0000","last_clean":"2023-07-14T18:15:35.955078+0000","last_became_active":"2023-07-14T18:14:16.928567+0000","last_became_peered":"2023-07-14T18:14:16.928567+0000","last_unstale":"2023-07-14T18:15:35.955078+0000","last_undegraded":"2023-07-14T18:15:35.955078+0000","last_fullsized":"2023-07-14T18:15:35.955078+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:44:11.774963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.948806+0000","last_change":"2023-07-14T18:14:17.938456+0000","last_active":"2023-07-14T18:14:48.948806+0000","last_peered":"2023-07-14T18:14:48.948806+0000","last_clean":"2023-07-14T18:14:48.948806+0000","last_became_active":"2023-07-14T18:14:17.938086+0000","last_became_peered":"2023-07-14T18:14:17.938086+0000","last_unstale":"2023-07-14T18:14:48.948806+0000","last_undegraded":"2023-07-14T18:14:48.948806+0000","last_fullsized":"2023-07-14T18:14:48.948806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:16:01.795875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945337+0000","last_change":"2023-07-14T18:14:22.042785+0000","last_active":"2023-07-14T18:14:48.945337+0000","last_peered":"2023-07-14T18:14:48.945337+0000","last_clean":"2023-07-14T18:14:48.945337+0000","last_became_active":"2023-07-14T18:14:22.042346+0000","last_became_peered":"2023-07-14T18:14:22.042346+0000","last_unstale":"2023-07-14T18:14:48.945337+0000","last_undegraded":"2023-07-14T18:14:48.945337+0000","last_fullsized":"2023-07-14T18:14:48.945337+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:26:17.301312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944005+0000","last_change":"2023-07-14T18:14:16.921168+0000","last_active":"2023-07-14T18:14:48.944005+0000","last_peered":"2023-07-14T18:14:48.944005+0000","last_clean":"2023-07-14T18:14:48.944005+0000","last_became_active":"2023-07-14T18:14:16.920808+0000","last_became_peered":"2023-07-14T18:14:16.920808+0000","last_unstale":"2023-07-14T18:14:48.944005+0000","last_undegraded":"2023-07-14T18:14:48.944005+0000","last_fullsized":"2023-07-14T18:14:48.944005+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:41:57.870102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541090+0000","last_change":"2023-07-14T18:14:17.935100+0000","last_active":"2023-07-14T18:14:49.541090+0000","last_peered":"2023-07-14T18:14:49.541090+0000","last_clean":"2023-07-14T18:14:49.541090+0000","last_became_active":"2023-07-14T18:14:17.934794+0000","last_became_peered":"2023-07-14T18:14:17.934794+0000","last_unstale":"2023-07-14T18:14:49.541090+0000","last_undegraded":"2023-07-14T18:14:49.541090+0000","last_fullsized":"2023-07-14T18:14:49.541090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:37:28.349127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.948310+0000","last_change":"2023-07-14T18:14:22.043495+0000","last_active":"2023-07-14T18:14:48.948310+0000","last_peered":"2023-07-14T18:14:48.948310+0000","last_clean":"2023-07-14T18:14:48.948310+0000","last_became_active":"2023-07-14T18:14:22.042686+0000","last_became_peered":"2023-07-14T18:14:22.042686+0000","last_unstale":"2023-07-14T18:14:48.948310+0000","last_undegraded":"2023-07-14T18:14:48.948310+0000","last_fullsized":"2023-07-14T18:14:48.948310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:16:19.904966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944994+0000","last_change":"2023-07-14T18:14:16.926006+0000","last_active":"2023-07-14T18:14:48.944994+0000","last_peered":"2023-07-14T18:14:48.944994+0000","last_clean":"2023-07-14T18:14:48.944994+0000","last_became_active":"2023-07-14T18:14:16.925719+0000","last_became_peered":"2023-07-14T18:14:16.925719+0000","last_unstale":"2023-07-14T18:14:48.944994+0000","last_undegraded":"2023-07-14T18:14:48.944994+0000","last_fullsized":"2023-07-14T18:14:48.944994+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:03:27.517349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.948103+0000","last_change":"2023-07-14T18:14:17.934155+0000","last_active":"2023-07-14T18:14:48.948103+0000","last_peered":"2023-07-14T18:14:48.948103+0000","last_clean":"2023-07-14T18:14:48.948103+0000","last_became_active":"2023-07-14T18:14:17.933756+0000","last_became_peered":"2023-07-14T18:14:17.933756+0000","last_unstale":"2023-07-14T18:14:48.948103+0000","last_undegraded":"2023-07-14T18:14:48.948103+0000","last_fullsized":"2023-07-14T18:14:48.948103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:04:42.915865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945422+0000","last_change":"2023-07-14T18:14:22.039198+0000","last_active":"2023-07-14T18:14:48.945422+0000","last_peered":"2023-07-14T18:14:48.945422+0000","last_clean":"2023-07-14T18:14:48.945422+0000","last_became_active":"2023-07-14T18:14:22.038842+0000","last_became_peered":"2023-07-14T18:14:22.038842+0000","last_unstale":"2023-07-14T18:14:48.945422+0000","last_undegraded":"2023-07-14T18:14:48.945422+0000","last_fullsized":"2023-07-14T18:14:48.945422+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:32:54.838956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'33","reported_seq":161,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:15:36.961067+0000","last_change":"2023-07-14T18:14:16.920918+0000","last_active":"2023-07-14T18:15:36.961067+0000","last_peered":"2023-07-14T18:15:36.961067+0000","last_clean":"2023-07-14T18:15:36.961067+0000","last_became_active":"2023-07-14T18:14:16.920572+0000","last_became_peered":"2023-07-14T18:14:16.920572+0000","last_unstale":"2023-07-14T18:15:36.961067+0000","last_undegraded":"2023-07-14T18:15:36.961067+0000","last_fullsized":"2023-07-14T18:15:36.961067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:55:35.900625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59808,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":104,"num_read_kb":110,"num_write":34,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945963+0000","last_change":"2023-07-14T18:14:17.929413+0000","last_active":"2023-07-14T18:14:48.945963+0000","last_peered":"2023-07-14T18:14:48.945963+0000","last_clean":"2023-07-14T18:14:48.945963+0000","last_became_active":"2023-07-14T18:14:17.929064+0000","last_became_peered":"2023-07-14T18:14:17.929064+0000","last_unstale":"2023-07-14T18:14:48.945963+0000","last_undegraded":"2023-07-14T18:14:48.945963+0000","last_fullsized":"2023-07-14T18:14:48.945963+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:29:27.786039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945372+0000","last_change":"2023-07-14T18:14:22.041983+0000","last_active":"2023-07-14T18:14:48.945372+0000","last_peered":"2023-07-14T18:14:48.945372+0000","last_clean":"2023-07-14T18:14:48.945372+0000","last_became_active":"2023-07-14T18:14:22.041553+0000","last_became_peered":"2023-07-14T18:14:22.041553+0000","last_unstale":"2023-07-14T18:14:48.945372+0000","last_undegraded":"2023-07-14T18:14:48.945372+0000","last_fullsized":"2023-07-14T18:14:48.945372+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:59:09.096024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.892826+0000","last_change":"2023-07-14T18:14:16.925444+0000","last_active":"2023-07-14T18:14:49.892826+0000","last_peered":"2023-07-14T18:14:49.892826+0000","last_clean":"2023-07-14T18:14:49.892826+0000","last_became_active":"2023-07-14T18:14:16.925138+0000","last_became_peered":"2023-07-14T18:14:16.925138+0000","last_unstale":"2023-07-14T18:14:49.892826+0000","last_undegraded":"2023-07-14T18:14:49.892826+0000","last_fullsized":"2023-07-14T18:14:49.892826+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:10:33.163171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942332+0000","last_change":"2023-07-14T18:14:17.922351+0000","last_active":"2023-07-14T18:14:48.942332+0000","last_peered":"2023-07-14T18:14:48.942332+0000","last_clean":"2023-07-14T18:14:48.942332+0000","last_became_active":"2023-07-14T18:14:17.921973+0000","last_became_peered":"2023-07-14T18:14:17.921973+0000","last_unstale":"2023-07-14T18:14:48.942332+0000","last_undegraded":"2023-07-14T18:14:48.942332+0000","last_fullsized":"2023-07-14T18:14:48.942332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:16:09.934284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945943+0000","last_change":"2023-07-14T18:14:22.048274+0000","last_active":"2023-07-14T18:14:48.945943+0000","last_peered":"2023-07-14T18:14:48.945943+0000","last_clean":"2023-07-14T18:14:48.945943+0000","last_became_active":"2023-07-14T18:14:22.047946+0000","last_became_peered":"2023-07-14T18:14:22.047946+0000","last_unstale":"2023-07-14T18:14:48.945943+0000","last_undegraded":"2023-07-14T18:14:48.945943+0000","last_fullsized":"2023-07-14T18:14:48.945943+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:29:47.580140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944169+0000","last_change":"2023-07-14T18:14:16.933566+0000","last_active":"2023-07-14T18:14:48.944169+0000","last_peered":"2023-07-14T18:14:48.944169+0000","last_clean":"2023-07-14T18:14:48.944169+0000","last_became_active":"2023-07-14T18:14:16.933300+0000","last_became_peered":"2023-07-14T18:14:16.933300+0000","last_unstale":"2023-07-14T18:14:48.944169+0000","last_undegraded":"2023-07-14T18:14:48.944169+0000","last_fullsized":"2023-07-14T18:14:48.944169+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:05:57.086039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944233+0000","last_change":"2023-07-14T18:14:17.928521+0000","last_active":"2023-07-14T18:14:48.944233+0000","last_peered":"2023-07-14T18:14:48.944233+0000","last_clean":"2023-07-14T18:14:48.944233+0000","last_became_active":"2023-07-14T18:14:17.928025+0000","last_became_peered":"2023-07-14T18:14:17.928025+0000","last_unstale":"2023-07-14T18:14:48.944233+0000","last_undegraded":"2023-07-14T18:14:48.944233+0000","last_fullsized":"2023-07-14T18:14:48.944233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:21:23.554363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945666+0000","last_change":"2023-07-14T18:14:22.046820+0000","last_active":"2023-07-14T18:14:48.945666+0000","last_peered":"2023-07-14T18:14:48.945666+0000","last_clean":"2023-07-14T18:14:48.945666+0000","last_became_active":"2023-07-14T18:14:22.046440+0000","last_became_peered":"2023-07-14T18:14:22.046440+0000","last_unstale":"2023-07-14T18:14:48.945666+0000","last_undegraded":"2023-07-14T18:14:48.945666+0000","last_fullsized":"2023-07-14T18:14:48.945666+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:46:22.890726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":81,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:15:38.966998+0000","last_change":"2023-07-14T18:14:16.922075+0000","last_active":"2023-07-14T18:15:38.966998+0000","last_peered":"2023-07-14T18:15:38.966998+0000","last_clean":"2023-07-14T18:15:38.966998+0000","last_became_active":"2023-07-14T18:14:16.921570+0000","last_became_peered":"2023-07-14T18:14:16.921570+0000","last_unstale":"2023-07-14T18:15:38.966998+0000","last_undegraded":"2023-07-14T18:15:38.966998+0000","last_fullsized":"2023-07-14T18:15:38.966998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:08:37.059574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.943804+0000","last_change":"2023-07-14T18:14:17.936439+0000","last_active":"2023-07-14T18:14:48.943804+0000","last_peered":"2023-07-14T18:14:48.943804+0000","last_clean":"2023-07-14T18:14:48.943804+0000","last_became_active":"2023-07-14T18:14:17.936138+0000","last_became_peered":"2023-07-14T18:14:17.936138+0000","last_unstale":"2023-07-14T18:14:48.943804+0000","last_undegraded":"2023-07-14T18:14:48.943804+0000","last_fullsized":"2023-07-14T18:14:48.943804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:58:02.987288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.543302+0000","last_change":"2023-07-14T18:14:49.543302+0000","last_active":"2023-07-14T18:14:49.543302+0000","last_peered":"2023-07-14T18:14:49.543302+0000","last_clean":"2023-07-14T18:14:49.543302+0000","last_became_active":"2023-07-14T18:14:49.542768+0000","last_became_peered":"2023-07-14T18:14:49.542768+0000","last_unstale":"2023-07-14T18:14:49.543302+0000","last_undegraded":"2023-07-14T18:14:49.543302+0000","last_fullsized":"2023-07-14T18:14:49.543302+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:14:41.906790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.896057+0000","last_change":"2023-07-14T18:14:16.926417+0000","last_active":"2023-07-14T18:14:49.896057+0000","last_peered":"2023-07-14T18:14:49.896057+0000","last_clean":"2023-07-14T18:14:49.896057+0000","last_became_active":"2023-07-14T18:14:16.926101+0000","last_became_peered":"2023-07-14T18:14:16.926101+0000","last_unstale":"2023-07-14T18:14:49.896057+0000","last_undegraded":"2023-07-14T18:14:49.896057+0000","last_fullsized":"2023-07-14T18:14:49.896057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:09:38.329843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.950979+0000","last_change":"2023-07-14T18:14:48.950649+0000","last_active":"2023-07-14T18:14:48.950979+0000","last_peered":"2023-07-14T18:14:48.950979+0000","last_clean":"2023-07-14T18:14:48.950979+0000","last_became_active":"2023-07-14T18:14:48.950303+0000","last_became_peered":"2023-07-14T18:14:48.950303+0000","last_unstale":"2023-07-14T18:14:48.950979+0000","last_undegraded":"2023-07-14T18:14:48.950979+0000","last_fullsized":"2023-07-14T18:14:48.950979+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:40:15.252731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942609+0000","last_change":"2023-07-14T18:14:22.042171+0000","last_active":"2023-07-14T18:14:48.942609+0000","last_peered":"2023-07-14T18:14:48.942609+0000","last_clean":"2023-07-14T18:14:48.942609+0000","last_became_active":"2023-07-14T18:14:22.041692+0000","last_became_peered":"2023-07-14T18:14:22.041692+0000","last_unstale":"2023-07-14T18:14:48.942609+0000","last_undegraded":"2023-07-14T18:14:48.942609+0000","last_fullsized":"2023-07-14T18:14:48.942609+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:03:01.650548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.881565+0000","last_change":"2023-07-14T18:14:16.922883+0000","last_active":"2023-07-14T18:14:49.881565+0000","last_peered":"2023-07-14T18:14:49.881565+0000","last_clean":"2023-07-14T18:14:49.881565+0000","last_became_active":"2023-07-14T18:14:16.922501+0000","last_became_peered":"2023-07-14T18:14:16.922501+0000","last_unstale":"2023-07-14T18:14:49.881565+0000","last_undegraded":"2023-07-14T18:14:49.881565+0000","last_fullsized":"2023-07-14T18:14:49.881565+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:51:12.741737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946509+0000","last_change":"2023-07-14T18:14:17.922899+0000","last_active":"2023-07-14T18:14:48.946509+0000","last_peered":"2023-07-14T18:14:48.946509+0000","last_clean":"2023-07-14T18:14:48.946509+0000","last_became_active":"2023-07-14T18:14:17.922608+0000","last_became_peered":"2023-07-14T18:14:17.922608+0000","last_unstale":"2023-07-14T18:14:48.946509+0000","last_undegraded":"2023-07-14T18:14:48.946509+0000","last_fullsized":"2023-07-14T18:14:48.946509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:53:19.741997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945671+0000","last_change":"2023-07-14T18:14:22.327484+0000","last_active":"2023-07-14T18:14:48.945671+0000","last_peered":"2023-07-14T18:14:48.945671+0000","last_clean":"2023-07-14T18:14:48.945671+0000","last_became_active":"2023-07-14T18:14:22.327010+0000","last_became_peered":"2023-07-14T18:14:22.327010+0000","last_unstale":"2023-07-14T18:14:48.945671+0000","last_undegraded":"2023-07-14T18:14:48.945671+0000","last_fullsized":"2023-07-14T18:14:48.945671+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:32:16.310729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945788+0000","last_change":"2023-07-14T18:14:16.930606+0000","last_active":"2023-07-14T18:14:48.945788+0000","last_peered":"2023-07-14T18:14:48.945788+0000","last_clean":"2023-07-14T18:14:48.945788+0000","last_became_active":"2023-07-14T18:14:16.930175+0000","last_became_peered":"2023-07-14T18:14:16.930175+0000","last_unstale":"2023-07-14T18:14:48.945788+0000","last_undegraded":"2023-07-14T18:14:48.945788+0000","last_fullsized":"2023-07-14T18:14:48.945788+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:39:16.156926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541700+0000","last_change":"2023-07-14T18:14:17.931962+0000","last_active":"2023-07-14T18:14:49.541700+0000","last_peered":"2023-07-14T18:14:49.541700+0000","last_clean":"2023-07-14T18:14:49.541700+0000","last_became_active":"2023-07-14T18:14:17.931621+0000","last_became_peered":"2023-07-14T18:14:17.931621+0000","last_unstale":"2023-07-14T18:14:49.541700+0000","last_undegraded":"2023-07-14T18:14:49.541700+0000","last_fullsized":"2023-07-14T18:14:49.541700+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:32:19.781881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942854+0000","last_change":"2023-07-14T18:14:22.038977+0000","last_active":"2023-07-14T18:14:48.942854+0000","last_peered":"2023-07-14T18:14:48.942854+0000","last_clean":"2023-07-14T18:14:48.942854+0000","last_became_active":"2023-07-14T18:14:22.038515+0000","last_became_peered":"2023-07-14T18:14:22.038515+0000","last_unstale":"2023-07-14T18:14:48.942854+0000","last_undegraded":"2023-07-14T18:14:48.942854+0000","last_fullsized":"2023-07-14T18:14:48.942854+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:40:29.285675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.943866+0000","last_change":"2023-07-14T18:14:16.925457+0000","last_active":"2023-07-14T18:14:48.943866+0000","last_peered":"2023-07-14T18:14:48.943866+0000","last_clean":"2023-07-14T18:14:48.943866+0000","last_became_active":"2023-07-14T18:14:16.925150+0000","last_became_peered":"2023-07-14T18:14:16.925150+0000","last_unstale":"2023-07-14T18:14:48.943866+0000","last_undegraded":"2023-07-14T18:14:48.943866+0000","last_fullsized":"2023-07-14T18:14:48.943866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:46:38.768309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541290+0000","last_change":"2023-07-14T18:14:17.926492+0000","last_active":"2023-07-14T18:14:49.541290+0000","last_peered":"2023-07-14T18:14:49.541290+0000","last_clean":"2023-07-14T18:14:49.541290+0000","last_became_active":"2023-07-14T18:14:17.926218+0000","last_became_peered":"2023-07-14T18:14:17.926218+0000","last_unstale":"2023-07-14T18:14:49.541290+0000","last_undegraded":"2023-07-14T18:14:49.541290+0000","last_fullsized":"2023-07-14T18:14:49.541290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:08:20.317990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942594+0000","last_change":"2023-07-14T18:14:22.327242+0000","last_active":"2023-07-14T18:14:48.942594+0000","last_peered":"2023-07-14T18:14:48.942594+0000","last_clean":"2023-07-14T18:14:48.942594+0000","last_became_active":"2023-07-14T18:14:22.326787+0000","last_became_peered":"2023-07-14T18:14:22.326787+0000","last_unstale":"2023-07-14T18:14:48.942594+0000","last_undegraded":"2023-07-14T18:14:48.942594+0000","last_fullsized":"2023-07-14T18:14:48.942594+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:08:03.165732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942605+0000","last_change":"2023-07-14T18:14:17.925035+0000","last_active":"2023-07-14T18:14:48.942605+0000","last_peered":"2023-07-14T18:14:48.942605+0000","last_clean":"2023-07-14T18:14:48.942605+0000","last_became_active":"2023-07-14T18:14:17.924713+0000","last_became_peered":"2023-07-14T18:14:17.924713+0000","last_unstale":"2023-07-14T18:14:48.942605+0000","last_undegraded":"2023-07-14T18:14:48.942605+0000","last_fullsized":"2023-07-14T18:14:48.942605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:40:51.683089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":78,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:15:36.963807+0000","last_change":"2023-07-14T18:14:16.935561+0000","last_active":"2023-07-14T18:15:36.963807+0000","last_peered":"2023-07-14T18:15:36.963807+0000","last_clean":"2023-07-14T18:15:36.963807+0000","last_became_active":"2023-07-14T18:14:16.935022+0000","last_became_peered":"2023-07-14T18:14:16.935022+0000","last_unstale":"2023-07-14T18:15:36.963807+0000","last_undegraded":"2023-07-14T18:15:36.963807+0000","last_fullsized":"2023-07-14T18:15:36.963807+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:31:09.667735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.943916+0000","last_change":"2023-07-14T18:14:22.043818+0000","last_active":"2023-07-14T18:14:48.943916+0000","last_peered":"2023-07-14T18:14:48.943916+0000","last_clean":"2023-07-14T18:14:48.943916+0000","last_became_active":"2023-07-14T18:14:22.043449+0000","last_became_peered":"2023-07-14T18:14:22.043449+0000","last_unstale":"2023-07-14T18:14:48.943916+0000","last_undegraded":"2023-07-14T18:14:48.943916+0000","last_fullsized":"2023-07-14T18:14:48.943916+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:25:52.282417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540949+0000","last_change":"2023-07-14T18:14:17.930211+0000","last_active":"2023-07-14T18:14:49.540949+0000","last_peered":"2023-07-14T18:14:49.540949+0000","last_clean":"2023-07-14T18:14:49.540949+0000","last_became_active":"2023-07-14T18:14:17.929851+0000","last_became_peered":"2023-07-14T18:14:17.929851+0000","last_unstale":"2023-07-14T18:14:49.540949+0000","last_undegraded":"2023-07-14T18:14:49.540949+0000","last_fullsized":"2023-07-14T18:14:49.540949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:49:16.236513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.894113+0000","last_change":"2023-07-14T18:14:16.928995+0000","last_active":"2023-07-14T18:14:49.894113+0000","last_peered":"2023-07-14T18:14:49.894113+0000","last_clean":"2023-07-14T18:14:49.894113+0000","last_became_active":"2023-07-14T18:14:16.928612+0000","last_became_peered":"2023-07-14T18:14:16.928612+0000","last_unstale":"2023-07-14T18:14:49.894113+0000","last_undegraded":"2023-07-14T18:14:49.894113+0000","last_fullsized":"2023-07-14T18:14:49.894113+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:33:55.215553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":340,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.943619+0000","last_change":"2023-07-14T18:13:11.614507+0000","last_active":"2023-07-14T18:14:48.943619+0000","last_peered":"2023-07-14T18:14:48.943619+0000","last_clean":"2023-07-14T18:14:48.943619+0000","last_became_active":"2023-07-14T18:13:11.614063+0000","last_became_peered":"2023-07-14T18:13:11.614063+0000","last_unstale":"2023-07-14T18:14:48.943619+0000","last_undegraded":"2023-07-14T18:14:48.943619+0000","last_fullsized":"2023-07-14T18:14:48.943619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:10:03.995193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:10:03.995193+0000","last_clean_scrub_stamp":"2023-07-14T18:10:03.995193+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:35:25.729988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.543558+0000","last_change":"2023-07-14T18:14:49.542604+0000","last_active":"2023-07-14T18:14:49.543558+0000","last_peered":"2023-07-14T18:14:49.543558+0000","last_clean":"2023-07-14T18:14:49.543558+0000","last_became_active":"2023-07-14T18:14:49.542111+0000","last_became_peered":"2023-07-14T18:14:49.542111+0000","last_unstale":"2023-07-14T18:14:49.543558+0000","last_undegraded":"2023-07-14T18:14:49.543558+0000","last_fullsized":"2023-07-14T18:14:49.543558+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:40:56.970642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.543511+0000","last_change":"2023-07-14T18:14:49.543511+0000","last_active":"2023-07-14T18:14:49.543511+0000","last_peered":"2023-07-14T18:14:49.543511+0000","last_clean":"2023-07-14T18:14:49.543511+0000","last_became_active":"2023-07-14T18:14:49.542407+0000","last_became_peered":"2023-07-14T18:14:49.542407+0000","last_unstale":"2023-07-14T18:14:49.543511+0000","last_undegraded":"2023-07-14T18:14:49.543511+0000","last_fullsized":"2023-07-14T18:14:49.543511+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:55:54.330638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.894434+0000","last_change":"2023-07-14T18:14:48.950022+0000","last_active":"2023-07-14T18:14:49.894434+0000","last_peered":"2023-07-14T18:14:49.894434+0000","last_clean":"2023-07-14T18:14:49.894434+0000","last_became_active":"2023-07-14T18:14:48.949149+0000","last_became_peered":"2023-07-14T18:14:48.949149+0000","last_unstale":"2023-07-14T18:14:49.894434+0000","last_undegraded":"2023-07-14T18:14:49.894434+0000","last_fullsized":"2023-07-14T18:14:49.894434+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:10:25.993299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945005+0000","last_change":"2023-07-14T18:14:22.038428+0000","last_active":"2023-07-14T18:14:48.945005+0000","last_peered":"2023-07-14T18:14:48.945005+0000","last_clean":"2023-07-14T18:14:48.945005+0000","last_became_active":"2023-07-14T18:14:22.038038+0000","last_became_peered":"2023-07-14T18:14:22.038038+0000","last_unstale":"2023-07-14T18:14:48.945005+0000","last_undegraded":"2023-07-14T18:14:48.945005+0000","last_fullsized":"2023-07-14T18:14:48.945005+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:58:40.233180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946196+0000","last_change":"2023-07-14T18:14:16.933110+0000","last_active":"2023-07-14T18:14:48.946196+0000","last_peered":"2023-07-14T18:14:48.946196+0000","last_clean":"2023-07-14T18:14:48.946196+0000","last_became_active":"2023-07-14T18:14:16.932844+0000","last_became_peered":"2023-07-14T18:14:16.932844+0000","last_unstale":"2023-07-14T18:14:48.946196+0000","last_undegraded":"2023-07-14T18:14:48.946196+0000","last_fullsized":"2023-07-14T18:14:48.946196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:38:45.588165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942980+0000","last_change":"2023-07-14T18:14:17.920168+0000","last_active":"2023-07-14T18:14:48.942980+0000","last_peered":"2023-07-14T18:14:48.942980+0000","last_clean":"2023-07-14T18:14:48.942980+0000","last_became_active":"2023-07-14T18:14:17.919576+0000","last_became_peered":"2023-07-14T18:14:17.919576+0000","last_unstale":"2023-07-14T18:14:48.942980+0000","last_undegraded":"2023-07-14T18:14:48.942980+0000","last_fullsized":"2023-07-14T18:14:48.942980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:22:46.766040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540761+0000","last_change":"2023-07-14T18:14:22.327206+0000","last_active":"2023-07-14T18:14:49.540761+0000","last_peered":"2023-07-14T18:14:49.540761+0000","last_clean":"2023-07-14T18:14:49.540761+0000","last_became_active":"2023-07-14T18:14:22.326743+0000","last_became_peered":"2023-07-14T18:14:22.326743+0000","last_unstale":"2023-07-14T18:14:49.540761+0000","last_undegraded":"2023-07-14T18:14:49.540761+0000","last_fullsized":"2023-07-14T18:14:49.540761+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:19:42.552871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'6","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:55.918214+0000","last_change":"2023-07-14T18:14:16.928994+0000","last_active":"2023-07-14T18:14:55.918214+0000","last_peered":"2023-07-14T18:14:55.918214+0000","last_clean":"2023-07-14T18:14:55.918214+0000","last_became_active":"2023-07-14T18:14:16.928468+0000","last_became_peered":"2023-07-14T18:14:16.928468+0000","last_unstale":"2023-07-14T18:14:55.918214+0000","last_undegraded":"2023-07-14T18:14:55.918214+0000","last_fullsized":"2023-07-14T18:14:55.918214+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:19:19.175018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541197+0000","last_change":"2023-07-14T18:14:17.919147+0000","last_active":"2023-07-14T18:14:49.541197+0000","last_peered":"2023-07-14T18:14:49.541197+0000","last_clean":"2023-07-14T18:14:49.541197+0000","last_became_active":"2023-07-14T18:14:17.918811+0000","last_became_peered":"2023-07-14T18:14:17.918811+0000","last_unstale":"2023-07-14T18:14:49.541197+0000","last_undegraded":"2023-07-14T18:14:49.541197+0000","last_fullsized":"2023-07-14T18:14:49.541197+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:55:38.239373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540750+0000","last_change":"2023-07-14T18:14:22.032632+0000","last_active":"2023-07-14T18:14:49.540750+0000","last_peered":"2023-07-14T18:14:49.540750+0000","last_clean":"2023-07-14T18:14:49.540750+0000","last_became_active":"2023-07-14T18:14:22.032233+0000","last_became_peered":"2023-07-14T18:14:22.032233+0000","last_unstale":"2023-07-14T18:14:49.540750+0000","last_undegraded":"2023-07-14T18:14:49.540750+0000","last_fullsized":"2023-07-14T18:14:49.540750+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:16:49.157181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.947795+0000","last_change":"2023-07-14T18:14:16.924993+0000","last_active":"2023-07-14T18:14:48.947795+0000","last_peered":"2023-07-14T18:14:48.947795+0000","last_clean":"2023-07-14T18:14:48.947795+0000","last_became_active":"2023-07-14T18:14:16.924702+0000","last_became_peered":"2023-07-14T18:14:16.924702+0000","last_unstale":"2023-07-14T18:14:48.947795+0000","last_undegraded":"2023-07-14T18:14:48.947795+0000","last_fullsized":"2023-07-14T18:14:48.947795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:21:25.188427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541234+0000","last_change":"2023-07-14T18:14:17.943397+0000","last_active":"2023-07-14T18:14:49.541234+0000","last_peered":"2023-07-14T18:14:49.541234+0000","last_clean":"2023-07-14T18:14:49.541234+0000","last_became_active":"2023-07-14T18:14:17.942114+0000","last_became_peered":"2023-07-14T18:14:17.942114+0000","last_unstale":"2023-07-14T18:14:49.541234+0000","last_undegraded":"2023-07-14T18:14:49.541234+0000","last_fullsized":"2023-07-14T18:14:49.541234+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:46:40.372375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.947250+0000","last_change":"2023-07-14T18:14:22.328273+0000","last_active":"2023-07-14T18:14:48.947250+0000","last_peered":"2023-07-14T18:14:48.947250+0000","last_clean":"2023-07-14T18:14:48.947250+0000","last_became_active":"2023-07-14T18:14:22.327843+0000","last_became_peered":"2023-07-14T18:14:22.327843+0000","last_unstale":"2023-07-14T18:14:48.947250+0000","last_undegraded":"2023-07-14T18:14:48.947250+0000","last_fullsized":"2023-07-14T18:14:48.947250+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:23:58.886753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.545792+0000","last_change":"2023-07-14T18:14:49.543525+0000","last_active":"2023-07-14T18:14:49.545792+0000","last_peered":"2023-07-14T18:14:49.545792+0000","last_clean":"2023-07-14T18:14:49.545792+0000","last_became_active":"2023-07-14T18:14:49.542962+0000","last_became_peered":"2023-07-14T18:14:49.542962+0000","last_unstale":"2023-07-14T18:14:49.545792+0000","last_undegraded":"2023-07-14T18:14:49.545792+0000","last_fullsized":"2023-07-14T18:14:49.545792+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:11:59.955363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945273+0000","last_change":"2023-07-14T18:14:17.932181+0000","last_active":"2023-07-14T18:14:48.945273+0000","last_peered":"2023-07-14T18:14:48.945273+0000","last_clean":"2023-07-14T18:14:48.945273+0000","last_became_active":"2023-07-14T18:14:17.931811+0000","last_became_peered":"2023-07-14T18:14:17.931811+0000","last_unstale":"2023-07-14T18:14:48.945273+0000","last_undegraded":"2023-07-14T18:14:48.945273+0000","last_fullsized":"2023-07-14T18:14:48.945273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:24:11.968765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945853+0000","last_change":"2023-07-14T18:14:22.047281+0000","last_active":"2023-07-14T18:14:48.945853+0000","last_peered":"2023-07-14T18:14:48.945853+0000","last_clean":"2023-07-14T18:14:48.945853+0000","last_became_active":"2023-07-14T18:14:22.045531+0000","last_became_peered":"2023-07-14T18:14:22.045531+0000","last_unstale":"2023-07-14T18:14:48.945853+0000","last_undegraded":"2023-07-14T18:14:48.945853+0000","last_fullsized":"2023-07-14T18:14:48.945853+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:59:25.065086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541181+0000","last_change":"2023-07-14T18:14:16.926904+0000","last_active":"2023-07-14T18:14:49.541181+0000","last_peered":"2023-07-14T18:14:49.541181+0000","last_clean":"2023-07-14T18:14:49.541181+0000","last_became_active":"2023-07-14T18:14:16.926579+0000","last_became_peered":"2023-07-14T18:14:16.926579+0000","last_unstale":"2023-07-14T18:14:49.541181+0000","last_undegraded":"2023-07-14T18:14:49.541181+0000","last_fullsized":"2023-07-14T18:14:49.541181+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:34:03.520810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946719+0000","last_change":"2023-07-14T18:14:17.926005+0000","last_active":"2023-07-14T18:14:48.946719+0000","last_peered":"2023-07-14T18:14:48.946719+0000","last_clean":"2023-07-14T18:14:48.946719+0000","last_became_active":"2023-07-14T18:14:17.925266+0000","last_became_peered":"2023-07-14T18:14:17.925266+0000","last_unstale":"2023-07-14T18:14:48.946719+0000","last_undegraded":"2023-07-14T18:14:48.946719+0000","last_fullsized":"2023-07-14T18:14:48.946719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:03:29.763528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946978+0000","last_change":"2023-07-14T18:14:22.043219+0000","last_active":"2023-07-14T18:14:48.946978+0000","last_peered":"2023-07-14T18:14:48.946978+0000","last_clean":"2023-07-14T18:14:48.946978+0000","last_became_active":"2023-07-14T18:14:22.042654+0000","last_became_peered":"2023-07-14T18:14:22.042654+0000","last_unstale":"2023-07-14T18:14:48.946978+0000","last_undegraded":"2023-07-14T18:14:48.946978+0000","last_fullsized":"2023-07-14T18:14:48.946978+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:46:41.786917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540895+0000","last_change":"2023-07-14T18:14:16.923130+0000","last_active":"2023-07-14T18:14:49.540895+0000","last_peered":"2023-07-14T18:14:49.540895+0000","last_clean":"2023-07-14T18:14:49.540895+0000","last_became_active":"2023-07-14T18:14:16.922835+0000","last_became_peered":"2023-07-14T18:14:16.922835+0000","last_unstale":"2023-07-14T18:14:49.540895+0000","last_undegraded":"2023-07-14T18:14:49.540895+0000","last_fullsized":"2023-07-14T18:14:49.540895+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:48:50.705976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540779+0000","last_change":"2023-07-14T18:14:17.931024+0000","last_active":"2023-07-14T18:14:49.540779+0000","last_peered":"2023-07-14T18:14:49.540779+0000","last_clean":"2023-07-14T18:14:49.540779+0000","last_became_active":"2023-07-14T18:14:17.929529+0000","last_became_peered":"2023-07-14T18:14:17.929529+0000","last_unstale":"2023-07-14T18:14:49.540779+0000","last_undegraded":"2023-07-14T18:14:49.540779+0000","last_fullsized":"2023-07-14T18:14:49.540779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:39.524476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946795+0000","last_change":"2023-07-14T18:14:22.327313+0000","last_active":"2023-07-14T18:14:48.946795+0000","last_peered":"2023-07-14T18:14:48.946795+0000","last_clean":"2023-07-14T18:14:48.946795+0000","last_became_active":"2023-07-14T18:14:22.326922+0000","last_became_peered":"2023-07-14T18:14:22.326922+0000","last_unstale":"2023-07-14T18:14:48.946795+0000","last_undegraded":"2023-07-14T18:14:48.946795+0000","last_fullsized":"2023-07-14T18:14:48.946795+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:48:34.325995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945190+0000","last_change":"2023-07-14T18:14:16.926632+0000","last_active":"2023-07-14T18:14:48.945190+0000","last_peered":"2023-07-14T18:14:48.945190+0000","last_clean":"2023-07-14T18:14:48.945190+0000","last_became_active":"2023-07-14T18:14:16.926279+0000","last_became_peered":"2023-07-14T18:14:16.926279+0000","last_unstale":"2023-07-14T18:14:48.945190+0000","last_undegraded":"2023-07-14T18:14:48.945190+0000","last_fullsized":"2023-07-14T18:14:48.945190+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:25:51.630494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944432+0000","last_change":"2023-07-14T18:14:17.924416+0000","last_active":"2023-07-14T18:14:48.944432+0000","last_peered":"2023-07-14T18:14:48.944432+0000","last_clean":"2023-07-14T18:14:48.944432+0000","last_became_active":"2023-07-14T18:14:17.924140+0000","last_became_peered":"2023-07-14T18:14:17.924140+0000","last_unstale":"2023-07-14T18:14:48.944432+0000","last_undegraded":"2023-07-14T18:14:48.944432+0000","last_fullsized":"2023-07-14T18:14:48.944432+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:36:11.267149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946249+0000","last_change":"2023-07-14T18:14:22.037127+0000","last_active":"2023-07-14T18:14:48.946249+0000","last_peered":"2023-07-14T18:14:48.946249+0000","last_clean":"2023-07-14T18:14:48.946249+0000","last_became_active":"2023-07-14T18:14:22.036647+0000","last_became_peered":"2023-07-14T18:14:22.036647+0000","last_unstale":"2023-07-14T18:14:48.946249+0000","last_undegraded":"2023-07-14T18:14:48.946249+0000","last_fullsized":"2023-07-14T18:14:48.946249+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:45:53.709735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.895800+0000","last_change":"2023-07-14T18:14:16.921494+0000","last_active":"2023-07-14T18:14:49.895800+0000","last_peered":"2023-07-14T18:14:49.895800+0000","last_clean":"2023-07-14T18:14:49.895800+0000","last_became_active":"2023-07-14T18:14:16.919622+0000","last_became_peered":"2023-07-14T18:14:16.919622+0000","last_unstale":"2023-07-14T18:14:49.895800+0000","last_undegraded":"2023-07-14T18:14:49.895800+0000","last_fullsized":"2023-07-14T18:14:49.895800+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:49:14.678276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944208+0000","last_change":"2023-07-14T18:14:17.922266+0000","last_active":"2023-07-14T18:14:48.944208+0000","last_peered":"2023-07-14T18:14:48.944208+0000","last_clean":"2023-07-14T18:14:48.944208+0000","last_became_active":"2023-07-14T18:14:17.921461+0000","last_became_peered":"2023-07-14T18:14:17.921461+0000","last_unstale":"2023-07-14T18:14:48.944208+0000","last_undegraded":"2023-07-14T18:14:48.944208+0000","last_fullsized":"2023-07-14T18:14:48.944208+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:27:06.427711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541171+0000","last_change":"2023-07-14T18:14:22.044080+0000","last_active":"2023-07-14T18:14:49.541171+0000","last_peered":"2023-07-14T18:14:49.541171+0000","last_clean":"2023-07-14T18:14:49.541171+0000","last_became_active":"2023-07-14T18:14:22.043612+0000","last_became_peered":"2023-07-14T18:14:22.043612+0000","last_unstale":"2023-07-14T18:14:49.541171+0000","last_undegraded":"2023-07-14T18:14:49.541171+0000","last_fullsized":"2023-07-14T18:14:49.541171+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:41:00.904598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.945287+0000","last_change":"2023-07-14T18:14:16.931871+0000","last_active":"2023-07-14T18:14:48.945287+0000","last_peered":"2023-07-14T18:14:48.945287+0000","last_clean":"2023-07-14T18:14:48.945287+0000","last_became_active":"2023-07-14T18:14:16.931549+0000","last_became_peered":"2023-07-14T18:14:16.931549+0000","last_unstale":"2023-07-14T18:14:48.945287+0000","last_undegraded":"2023-07-14T18:14:48.945287+0000","last_fullsized":"2023-07-14T18:14:48.945287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:39:50.087515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944103+0000","last_change":"2023-07-14T18:14:17.929550+0000","last_active":"2023-07-14T18:14:48.944103+0000","last_peered":"2023-07-14T18:14:48.944103+0000","last_clean":"2023-07-14T18:14:48.944103+0000","last_became_active":"2023-07-14T18:14:17.929204+0000","last_became_peered":"2023-07-14T18:14:17.929204+0000","last_unstale":"2023-07-14T18:14:48.944103+0000","last_undegraded":"2023-07-14T18:14:48.944103+0000","last_fullsized":"2023-07-14T18:14:48.944103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:46:07.635189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942348+0000","last_change":"2023-07-14T18:14:22.034899+0000","last_active":"2023-07-14T18:14:48.942348+0000","last_peered":"2023-07-14T18:14:48.942348+0000","last_clean":"2023-07-14T18:14:48.942348+0000","last_became_active":"2023-07-14T18:14:22.034524+0000","last_became_peered":"2023-07-14T18:14:22.034524+0000","last_unstale":"2023-07-14T18:14:48.942348+0000","last_undegraded":"2023-07-14T18:14:48.942348+0000","last_fullsized":"2023-07-14T18:14:48.942348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:44:56.300609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942705+0000","last_change":"2023-07-14T18:14:16.935285+0000","last_active":"2023-07-14T18:14:48.942705+0000","last_peered":"2023-07-14T18:14:48.942705+0000","last_clean":"2023-07-14T18:14:48.942705+0000","last_became_active":"2023-07-14T18:14:16.934888+0000","last_became_peered":"2023-07-14T18:14:16.934888+0000","last_unstale":"2023-07-14T18:14:48.942705+0000","last_undegraded":"2023-07-14T18:14:48.942705+0000","last_fullsized":"2023-07-14T18:14:48.942705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:46:09.021281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.944372+0000","last_change":"2023-07-14T18:14:17.930982+0000","last_active":"2023-07-14T18:14:48.944372+0000","last_peered":"2023-07-14T18:14:48.944372+0000","last_clean":"2023-07-14T18:14:48.944372+0000","last_became_active":"2023-07-14T18:14:17.930661+0000","last_became_peered":"2023-07-14T18:14:17.930661+0000","last_unstale":"2023-07-14T18:14:48.944372+0000","last_undegraded":"2023-07-14T18:14:48.944372+0000","last_fullsized":"2023-07-14T18:14:48.944372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:42:10.943678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946615+0000","last_change":"2023-07-14T18:14:17.926204+0000","last_active":"2023-07-14T18:14:48.946615+0000","last_peered":"2023-07-14T18:14:48.946615+0000","last_clean":"2023-07-14T18:14:48.946615+0000","last_became_active":"2023-07-14T18:14:17.925689+0000","last_became_peered":"2023-07-14T18:14:17.925689+0000","last_unstale":"2023-07-14T18:14:48.946615+0000","last_undegraded":"2023-07-14T18:14:48.946615+0000","last_fullsized":"2023-07-14T18:14:48.946615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:32:10.441762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541397+0000","last_change":"2023-07-14T18:14:17.923146+0000","last_active":"2023-07-14T18:14:49.541397+0000","last_peered":"2023-07-14T18:14:49.541397+0000","last_clean":"2023-07-14T18:14:49.541397+0000","last_became_active":"2023-07-14T18:14:17.922734+0000","last_became_peered":"2023-07-14T18:14:17.922734+0000","last_unstale":"2023-07-14T18:14:49.541397+0000","last_undegraded":"2023-07-14T18:14:49.541397+0000","last_fullsized":"2023-07-14T18:14:49.541397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:39:01.011677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.540904+0000","last_change":"2023-07-14T18:14:17.927397+0000","last_active":"2023-07-14T18:14:49.540904+0000","last_peered":"2023-07-14T18:14:49.540904+0000","last_clean":"2023-07-14T18:14:49.540904+0000","last_became_active":"2023-07-14T18:14:17.927000+0000","last_became_peered":"2023-07-14T18:14:17.927000+0000","last_unstale":"2023-07-14T18:14:49.540904+0000","last_undegraded":"2023-07-14T18:14:49.540904+0000","last_fullsized":"2023-07-14T18:14:49.540904+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:57:52.235148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541451+0000","last_change":"2023-07-14T18:14:17.935266+0000","last_active":"2023-07-14T18:14:49.541451+0000","last_peered":"2023-07-14T18:14:49.541451+0000","last_clean":"2023-07-14T18:14:49.541451+0000","last_became_active":"2023-07-14T18:14:17.934944+0000","last_became_peered":"2023-07-14T18:14:17.934944+0000","last_unstale":"2023-07-14T18:14:49.541451+0000","last_undegraded":"2023-07-14T18:14:49.541451+0000","last_fullsized":"2023-07-14T18:14:49.541451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:36:06.516397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.542810+0000","last_change":"2023-07-14T18:14:49.542810+0000","last_active":"2023-07-14T18:14:49.542810+0000","last_peered":"2023-07-14T18:14:49.542810+0000","last_clean":"2023-07-14T18:14:49.542810+0000","last_became_active":"2023-07-14T18:14:49.542421+0000","last_became_peered":"2023-07-14T18:14:49.542421+0000","last_unstale":"2023-07-14T18:14:49.542810+0000","last_undegraded":"2023-07-14T18:14:49.542810+0000","last_fullsized":"2023-07-14T18:14:49.542810+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:16:57.268229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946187+0000","last_change":"2023-07-14T18:14:17.930757+0000","last_active":"2023-07-14T18:14:48.946187+0000","last_peered":"2023-07-14T18:14:48.946187+0000","last_clean":"2023-07-14T18:14:48.946187+0000","last_became_active":"2023-07-14T18:14:17.930302+0000","last_became_peered":"2023-07-14T18:14:17.930302+0000","last_unstale":"2023-07-14T18:14:48.946187+0000","last_undegraded":"2023-07-14T18:14:48.946187+0000","last_fullsized":"2023-07-14T18:14:48.946187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:44:33.862208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541685+0000","last_change":"2023-07-14T18:14:17.930337+0000","last_active":"2023-07-14T18:14:49.541685+0000","last_peered":"2023-07-14T18:14:49.541685+0000","last_clean":"2023-07-14T18:14:49.541685+0000","last_became_active":"2023-07-14T18:14:17.930037+0000","last_became_peered":"2023-07-14T18:14:17.930037+0000","last_unstale":"2023-07-14T18:14:49.541685+0000","last_undegraded":"2023-07-14T18:14:49.541685+0000","last_fullsized":"2023-07-14T18:14:49.541685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:56:10.479187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541417+0000","last_change":"2023-07-14T18:14:17.912750+0000","last_active":"2023-07-14T18:14:49.541417+0000","last_peered":"2023-07-14T18:14:49.541417+0000","last_clean":"2023-07-14T18:14:49.541417+0000","last_became_active":"2023-07-14T18:14:17.912402+0000","last_became_peered":"2023-07-14T18:14:17.912402+0000","last_unstale":"2023-07-14T18:14:49.541417+0000","last_undegraded":"2023-07-14T18:14:49.541417+0000","last_fullsized":"2023-07-14T18:14:49.541417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:16:35.639559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942521+0000","last_change":"2023-07-14T18:14:17.930004+0000","last_active":"2023-07-14T18:14:48.942521+0000","last_peered":"2023-07-14T18:14:48.942521+0000","last_clean":"2023-07-14T18:14:48.942521+0000","last_became_active":"2023-07-14T18:14:17.929719+0000","last_became_peered":"2023-07-14T18:14:17.929719+0000","last_unstale":"2023-07-14T18:14:48.942521+0000","last_undegraded":"2023-07-14T18:14:48.942521+0000","last_fullsized":"2023-07-14T18:14:48.942521+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:51:50.359987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:49.541319+0000","last_change":"2023-07-14T18:14:17.919044+0000","last_active":"2023-07-14T18:14:49.541319+0000","last_peered":"2023-07-14T18:14:49.541319+0000","last_clean":"2023-07-14T18:14:49.541319+0000","last_became_active":"2023-07-14T18:14:17.918079+0000","last_became_peered":"2023-07-14T18:14:17.918079+0000","last_unstale":"2023-07-14T18:14:49.541319+0000","last_undegraded":"2023-07-14T18:14:49.541319+0000","last_fullsized":"2023-07-14T18:14:49.541319+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:12:44.589756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946548+0000","last_change":"2023-07-14T18:14:17.921560+0000","last_active":"2023-07-14T18:14:48.946548+0000","last_peered":"2023-07-14T18:14:48.946548+0000","last_clean":"2023-07-14T18:14:48.946548+0000","last_became_active":"2023-07-14T18:14:17.921243+0000","last_became_peered":"2023-07-14T18:14:17.921243+0000","last_unstale":"2023-07-14T18:14:48.946548+0000","last_undegraded":"2023-07-14T18:14:48.946548+0000","last_fullsized":"2023-07-14T18:14:48.946548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:03:15.907251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.942971+0000","last_change":"2023-07-14T18:14:17.927504+0000","last_active":"2023-07-14T18:14:48.942971+0000","last_peered":"2023-07-14T18:14:48.942971+0000","last_clean":"2023-07-14T18:14:48.942971+0000","last_became_active":"2023-07-14T18:14:17.927233+0000","last_became_peered":"2023-07-14T18:14:17.927233+0000","last_unstale":"2023-07-14T18:14:48.942971+0000","last_undegraded":"2023-07-14T18:14:48.942971+0000","last_fullsized":"2023-07-14T18:14:48.942971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:37:40.310852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T18:14:48.946334+0000","last_change":"2023-07-14T18:14:17.932317+0000","last_active":"2023-07-14T18:14:48.946334+0000","last_peered":"2023-07-14T18:14:48.946334+0000","last_clean":"2023-07-14T18:14:48.946334+0000","last_became_active":"2023-07-14T18:14:17.931973+0000","last_became_peered":"2023-07-14T18:14:17.931973+0000","last_unstale":"2023-07-14T18:14:48.946334+0000","last_undegraded":"2023-07-14T18:14:48.946334+0000","last_fullsized":"2023-07-14T18:14:48.946334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:01:51.245001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281026,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63712,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":316,"num_read_kb":225,"num_write":157,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":191136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416413,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":492605,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.94199999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.496},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.44800000000000001}]},{"osd":1,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63900000000000001,"15min":0.63800000000000001},"min":{"1min":0.44500000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.753},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.41899999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77700000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.61699999999999999}]},{"osd":2,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72299999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72699999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.40500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":4,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68600000000000005,"15min":0.68000000000000005},"min":{"1min":0.48099999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89900000000000002,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.66400000000000003,"15min":0.67000000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86799999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.71099999999999997}]},{"osd":5,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.61099999999999999,"15min":0.625},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68300000000000005,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.38700000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96399999999999997,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.56699999999999995}]},{"osd":6,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.749,"15min":0.75800000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.95999999999999996,"5min":1.351,"15min":1.351},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.79000000000000004,"15min":0.79400000000000004},"min":{"1min":0.48299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":3.2410000000000001,"5min":3.2410000000000001,"15min":3.2410000000000001},"last":0.59599999999999997}]},{"osd":7,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77800000000000002,"15min":0.77900000000000003},"min":{"1min":0.58099999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.026,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.74399999999999999,"15min":0.749},"min":{"1min":0.51600000000000001,"5min":0.505,"15min":0.505},"max":{"1min":0.93100000000000005,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.77100000000000002}]},{"osd":8,"last update":"Fri Jul 14 18:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1,"5min":1,"15min":1},"last":0.69299999999999995}]},{"osd":9,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80200000000000005,"15min":0.80600000000000005},"min":{"1min":0.58099999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1279999999999999,"5min":1.635,"15min":1.635},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.84899999999999998,"15min":0.85099999999999998},"min":{"1min":0.59899999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.05,"5min":1.147,"15min":1.147},"last":0.81200000000000006}]},{"osd":10,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80500000000000005,"15min":0.80900000000000005},"min":{"1min":0.57599999999999996,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.97299999999999998,"5min":1.272,"15min":1.272},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.83999999999999997,"15min":0.84999999999999998},"min":{"1min":0.59699999999999998,"5min":0.501,"15min":0.501},"max":{"1min":0.97099999999999997,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.78600000000000003}]}]},{"osd":10,"up_from":72,"seq":309237645345,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":1139302,"data_compressed":53365,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.77300000000000002}]},{"osd":1,"last update":"Fri Jul 14 18:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.443,"15min":0.44500000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.59599999999999997,"5min":0.626,"15min":0.626},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.48099999999999998,"15min":0.48499999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Jul 14 18:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59999999999999998,"15min":0.59699999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.79400000000000004,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.59299999999999997}]},{"osd":3,"last update":"Fri Jul 14 18:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.64600000000000002,"15min":0.63800000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":3.4129999999999998,"5min":3.4129999999999998,"15min":3.4129999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.70699999999999996,"15min":0.68200000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":5.9939999999999998,"5min":5.9939999999999998,"15min":5.9939999999999998},"last":0.61399999999999999}]},{"osd":4,"last update":"Fri Jul 14 18:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.56200000000000006,"15min":0.56799999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.75,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.47399999999999998}]},{"osd":5,"last update":"Fri Jul 14 18:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65800000000000003,"15min":0.65500000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88400000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.46200000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.64700000000000002}]},{"osd":6,"last update":"Fri Jul 14 18:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.71499999999999997,"15min":0.72599999999999998},"min":{"1min":0.51200000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.91800000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.752,"15min":0.76700000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.81399999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.66400000000000003}]},{"osd":7,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.42199999999999999}]},{"osd":8,"last update":"Fri Jul 14 18:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.74099999999999999,"15min":0.747},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.90300000000000002,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.75700000000000001,"15min":0.76100000000000001},"min":{"1min":0.51100000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.92000000000000004,"5min":1.026,"15min":1.026},"last":0.628}]},{"osd":9,"last update":"Fri Jul 14 18:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81699999999999995,"15min":0.81799999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.3359999999999999,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80100000000000005,"15min":0.80400000000000005},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":0.79800000000000004}]},{"osd":11,"last update":"Fri Jul 14 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.752,"15min":0.751},"min":{"1min":0.61499999999999999,"5min":0.499,"15min":0.499},"max":{"1min":0.89300000000000002,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.76500000000000001,"15min":0.76200000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.96699999999999997,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.75600000000000001}]}]},{"osd":9,"up_from":67,"seq":287762808870,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":496937,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.53000000000000003,"15min":0.54200000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.59999999999999998,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.55600000000000005,"15min":0.56999999999999995},"min":{"1min":0.35399999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85299999999999998,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.35799999999999998}]},{"osd":1,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.65500000000000003,"15min":0.66000000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.72499999999999998,"15min":0.73699999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.879,"5min":1.085,"15min":1.085},"last":0.77000000000000002}]},{"osd":2,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.51300000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93200000000000005,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.745,"15min":0.75900000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.91300000000000003,"5min":1.147,"15min":1.147},"last":0.59299999999999997}]},{"osd":3,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.65200000000000002,"15min":0.67400000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.81100000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.497},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.65100000000000002,"15min":0.66800000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77700000000000002,"5min":1.018,"15min":1.018},"last":0.42999999999999999}]},{"osd":4,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.77800000000000002,"15min":0.79200000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.89600000000000002,"5min":1.198,"15min":1.198},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.77500000000000002,"15min":0.78900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.96299999999999997,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.78400000000000003}]},{"osd":5,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.79600000000000004,"15min":0.81200000000000006},"min":{"1min":0.54800000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.84199999999999997,"5min":1.224,"15min":1.224},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.75,"15min":0.75800000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95799999999999996,"5min":1.113,"15min":1.113},"last":0.47999999999999998}]},{"osd":6,"last update":"Fri Jul 14 18:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.75900000000000001}]},{"osd":7,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.85799999999999998,"15min":0.88200000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.92800000000000005,"5min":1.323,"15min":1.323},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.85099999999999998,"15min":0.86899999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.89500000000000002,"5min":1.411,"15min":1.411},"last":0.72399999999999998}]},{"osd":8,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.90100000000000002,"15min":0.92100000000000004},"min":{"1min":0.63,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0620000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.628},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.90200000000000002,"15min":0.92100000000000004},"min":{"1min":0.60799999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0109999999999999,"5min":1.371,"15min":1.371},"last":0.61399999999999999}]},{"osd":10,"last update":"Fri Jul 14 18:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.86299999999999999,"15min":0.879},"min":{"1min":0.66600000000000004,"5min":0.626,"15min":0.626},"max":{"1min":0.97799999999999998,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.751},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.85699999999999998,"15min":0.871},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.95499999999999996,"5min":1.095,"15min":1.095},"last":0.84799999999999998}]},{"osd":11,"last update":"Fri Jul 14 18:15:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.56299999999999994}]}]},{"osd":8,"up_from":59,"seq":253403070507,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":492179,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.43099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.212,"5min":1.238,"15min":1.238},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.60699999999999998,"15min":0.57499999999999996},"min":{"1min":0.47399999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.5950000000000002,"5min":2.5950000000000002,"15min":2.5950000000000002},"last":0.57799999999999996}]},{"osd":1,"last update":"Fri Jul 14 18:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.58099999999999996,"15min":0.56499999999999995},"min":{"1min":0.46600000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.77500000000000002,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60199999999999998,"15min":0.58699999999999997},"min":{"1min":0.45400000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.624}]},{"osd":2,"last update":"Fri Jul 14 18:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.47399999999999998,"15min":0.47199999999999998},"min":{"1min":0.34799999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.69699999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.47499999999999998,"15min":0.46600000000000003},"min":{"1min":0.34699999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.75800000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.46200000000000002}]},{"osd":3,"last update":"Fri Jul 14 18:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.69299999999999995,"15min":0.64500000000000002},"min":{"1min":0.48199999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":5.9749999999999996,"5min":5.9749999999999996,"15min":5.9749999999999996},"last":3.8090000000000002},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.66000000000000003,"15min":0.61399999999999999},"min":{"1min":0.42899999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":4.5229999999999997,"5min":4.5229999999999997,"15min":4.5229999999999997},"last":1.772}]},{"osd":4,"last update":"Fri Jul 14 18:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.439},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.66500000000000004,"15min":0.65700000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.66200000000000003}]},{"osd":5,"last update":"Fri Jul 14 18:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.39700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.94299999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81999999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.55100000000000005}]},{"osd":6,"last update":"Fri Jul 14 18:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.75600000000000001,"15min":0.73399999999999999},"min":{"1min":0.52800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":4.6900000000000004,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.998,"5min":0.77300000000000002,"15min":0.72699999999999998},"min":{"1min":0.57699999999999996,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":4.9589999999999996,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":3.6339999999999999}]},{"osd":7,"last update":"Fri Jul 14 18:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.73499999999999999,"15min":0.72899999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.71399999999999997,"15min":0.70299999999999996},"min":{"1min":0.54100000000000004,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95099999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.59799999999999998}]},{"osd":9,"last update":"Fri Jul 14 18:14:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.78900000000000003,"15min":0.78800000000000003},"min":{"1min":0.58699999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.78900000000000003,"15min":0.79400000000000004},"min":{"1min":0.54300000000000004,"5min":0.502,"15min":0.502},"max":{"1min":0.91900000000000004,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.67600000000000005}]},{"osd":10,"last update":"Fri Jul 14 18:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.77000000000000002,"15min":0.76300000000000001},"min":{"1min":0.56999999999999995,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77800000000000002,"15min":0.77400000000000002},"min":{"1min":0.57799999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.0089999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.76000000000000001}]},{"osd":11,"last update":"Fri Jul 14 18:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.70999999999999996}]}]},{"osd":7,"up_from":51,"seq":219043332142,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":1088573,"data_compressed":53365,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.755,"15min":0.77300000000000002},"min":{"1min":0.38200000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.024,"5min":5.843,"15min":5.843},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.51100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.81299999999999994,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.40899999999999997}]},{"osd":1,"last update":"Fri Jul 14 18:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.33400000000000002,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.54100000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46200000000000002,"15min":0.46100000000000002},"min":{"1min":0.29499999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66100000000000003,"5min":0.752,"15min":0.752},"last":0.36399999999999999}]},{"osd":2,"last update":"Fri Jul 14 18:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.625,"15min":0.59899999999999998},"min":{"1min":0.45100000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":3.4889999999999999,"5min":3.4889999999999999,"15min":3.4889999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.48799999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80900000000000005,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.71599999999999997}]},{"osd":3,"last update":"Fri Jul 14 18:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.65100000000000002,"15min":0.64700000000000002},"min":{"1min":0.52500000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.94199999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.443,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77900000000000003,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.505}]},{"osd":4,"last update":"Fri Jul 14 18:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52600000000000002,"15min":0.53200000000000003},"min":{"1min":0.35299999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.64500000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.443},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.501,"15min":0.498},"min":{"1min":0.34699999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72399999999999998,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.42199999999999999}]},{"osd":5,"last update":"Fri Jul 14 18:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61799999999999999,"15min":0.61399999999999999},"min":{"1min":0.5,"5min":0.375,"15min":0.375},"max":{"1min":0.71399999999999997,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68700000000000006,"15min":0.68500000000000005},"min":{"1min":0.56599999999999995,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.878,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.79400000000000004}]},{"osd":6,"last update":"Fri Jul 14 18:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":1.036,"15min":1.1499999999999999},"min":{"1min":0.56200000000000006,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.5019999999999998,"5min":13.189,"15min":13.189},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.51100000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88300000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.73599999999999999}]},{"osd":8,"last update":"Fri Jul 14 18:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.71599999999999997,"15min":0.69799999999999995},"min":{"1min":0.59899999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86899999999999999,"5min":1.04,"15min":1.04},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69799999999999995,"15min":0.69199999999999995},"min":{"1min":0.54700000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85799999999999998,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.81499999999999995}]},{"osd":9,"last update":"Fri Jul 14 18:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.94599999999999995,"15min":0.97599999999999998},"min":{"1min":0.58099999999999996,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.91000000000000003,"5min":5.8250000000000002,"15min":5.8250000000000002},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.77500000000000002,"15min":0.77100000000000002},"min":{"1min":0.61799999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.83399999999999996}]},{"osd":10,"last update":"Fri Jul 14 18:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72099999999999997,"15min":0.71799999999999997},"min":{"1min":0.48499999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92000000000000004,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.63300000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.90300000000000002,"5min":0.995,"15min":0.995},"last":0.755}]},{"osd":11,"last update":"Fri Jul 14 18:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.86399999999999999}]}]},{"osd":6,"up_from":46,"seq":197568495666,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":488017,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.46500000000000002,"15min":0.45500000000000002},"min":{"1min":0.32700000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.61299999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.49299999999999999,"15min":0.48199999999999998},"min":{"1min":0.35399999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65200000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.42299999999999999}]},{"osd":1,"last update":"Fri Jul 14 18:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.64700000000000002,"15min":0.64000000000000001},"min":{"1min":0.46000000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83499999999999996,"5min":0.87,"15min":0.87},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67700000000000005,"15min":0.67400000000000004},"min":{"1min":0.438,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91400000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.69399999999999995}]},{"osd":2,"last update":"Fri Jul 14 18:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63600000000000001,"15min":0.626},"min":{"1min":0.46100000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0309999999999999,"5min":1.087,"15min":1.087},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68400000000000005,"15min":0.67800000000000005},"min":{"1min":0.52200000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.86099999999999999,"5min":1.097,"15min":1.097},"last":0.60499999999999998}]},{"osd":3,"last update":"Fri Jul 14 18:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.39200000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84499999999999997,"5min":1.03,"15min":1.03},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.64100000000000001,"15min":0.64900000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83699999999999997,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.504}]},{"osd":4,"last update":"Fri Jul 14 18:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86199999999999999,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71799999999999997,"15min":0.72099999999999997},"min":{"1min":0.48099999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.80200000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.70899999999999996}]},{"osd":5,"last update":"Fri Jul 14 18:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.75700000000000001,"15min":0.76300000000000001},"min":{"1min":0.52300000000000002,"5min":0.435,"15min":0.435},"max":{"1min":1.0429999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.52900000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94899999999999995,"5min":1.163,"15min":1.163},"last":0.59099999999999997}]},{"osd":7,"last update":"Fri Jul 14 18:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73699999999999999,"15min":0.72699999999999998},"min":{"1min":0.65900000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.877,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.72999999999999998,"15min":0.72199999999999998},"min":{"1min":0.499,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90800000000000003,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.68200000000000005}]},{"osd":8,"last update":"Fri Jul 14 18:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79900000000000004,"15min":0.80600000000000005},"min":{"1min":0.59199999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0149999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.76500000000000001,"15min":0.75600000000000001},"min":{"1min":0.59599999999999997,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.623}]},{"osd":9,"last update":"Fri Jul 14 18:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.80500000000000005,"15min":0.81899999999999995},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.99199999999999999,"5min":1.147,"15min":1.147},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.83499999999999996,"15min":0.84599999999999997},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":0.93500000000000005,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.57299999999999995}]},{"osd":10,"last update":"Fri Jul 14 18:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.86599999999999999,"15min":0.86899999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.0349999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.85599999999999998,"15min":0.85399999999999998},"min":{"1min":0.68300000000000005,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.1539999999999999,"5min":1.248,"15min":1.248},"last":0.82899999999999996}]},{"osd":11,"last update":"Fri Jul 14 18:15:21 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.752}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":548317,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:14:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54500000000000004,"15min":0.51200000000000001},"min":{"1min":0.441,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76800000000000002,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57599999999999996,"15min":0.53200000000000003},"min":{"1min":0.41499999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81799999999999995,"5min":1.272,"15min":1.272},"last":0.498}]},{"osd":2,"last update":"Fri Jul 14 18:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.59399999999999997,"15min":0.54400000000000004},"min":{"1min":0.45100000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84599999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.57299999999999995,"15min":0.52600000000000002},"min":{"1min":0.42699999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89900000000000002,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.66400000000000003}]},{"osd":3,"last update":"Fri Jul 14 18:15:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.59299999999999997,"15min":0.53300000000000003},"min":{"1min":0.42799999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.125,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.65600000000000003,"15min":0.59999999999999998},"min":{"1min":0.501,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.5009999999999999,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":0.62}]},{"osd":4,"last update":"Fri Jul 14 18:14:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.48399999999999999,"15min":0.45500000000000002},"min":{"1min":0.34399999999999997,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.47499999999999998,"15min":0.45000000000000001},"min":{"1min":0.36599999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.54000000000000004}]},{"osd":5,"last update":"Fri Jul 14 18:15:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.629,"15min":0.60499999999999998},"min":{"1min":0.53200000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.63100000000000001,"15min":0.60699999999999998},"min":{"1min":0.57599999999999996,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.82699999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.88600000000000001}]},{"osd":6,"last update":"Fri Jul 14 18:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.67200000000000004,"15min":0.66900000000000004},"min":{"1min":0.46000000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90200000000000002,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67600000000000005,"15min":0.66800000000000004},"min":{"1min":0.47499999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.0029999999999999,"5min":1.23,"15min":1.23},"last":0.70299999999999996}]},{"osd":7,"last update":"Fri Jul 14 18:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.57799999999999996,"15min":0.55600000000000005},"min":{"1min":0.35699999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.98799999999999999,"5min":1.04,"15min":1.04},"last":0.746},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.60699999999999998,"15min":0.58299999999999996},"min":{"1min":0.38700000000000001,"5min":0.307,"15min":0.307},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.83299999999999996}]},{"osd":8,"last update":"Fri Jul 14 18:15:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.66500000000000004,"15min":0.64300000000000002},"min":{"1min":0.51000000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.72399999999999998,"15min":0.68999999999999995},"min":{"1min":0.629,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.026,"5min":1.052,"15min":1.052},"last":0.86699999999999999}]},{"osd":9,"last update":"Fri Jul 14 18:14:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.76900000000000002,"15min":0.76300000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.82799999999999996,"15min":0.82399999999999995},"min":{"1min":0.65300000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.90100000000000002}]},{"osd":10,"last update":"Fri Jul 14 18:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.81100000000000005,"15min":0.80500000000000005},"min":{"1min":0.59799999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.121,"5min":1.167,"15min":1.167},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76200000000000001,"15min":0.755},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.044,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.92900000000000005}]},{"osd":11,"last update":"Fri Jul 14 18:15:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.98299999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":489405,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 18:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59299999999999997,"15min":0.55000000000000004},"min":{"1min":0.372,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.93300000000000005,"5min":1.698,"15min":1.698},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60299999999999998,"15min":0.57799999999999996},"min":{"1min":0.49199999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.91500000000000004,"5min":1.179,"15min":1.179},"last":0.40899999999999997}]},{"osd":2,"last update":"Fri Jul 14 18:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.622,"15min":0.56399999999999995},"min":{"1min":0.46400000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89700000000000002,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.64700000000000002,"15min":0.58399999999999996},"min":{"1min":0.498,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.075,"5min":1.258,"15min":1.258},"last":0.73999999999999999}]},{"osd":3,"last update":"Fri Jul 14 18:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.68400000000000005}]},{"osd":4,"last update":"Fri Jul 14 18:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.59699999999999998,"15min":0.56299999999999994},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86399999999999999,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.65700000000000003,"15min":0.623},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.92300000000000004,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.44}]},{"osd":5,"last update":"Fri Jul 14 18:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.72499999999999998,"15min":0.68700000000000006},"min":{"1min":0.50700000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.89800000000000002,"5min":3.3740000000000001,"15min":3.3740000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.70099999999999996,"15min":0.68300000000000005},"min":{"1min":0.57499999999999996,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.99299999999999999,"5min":1.228,"15min":1.228},"last":0.83199999999999996}]},{"osd":6,"last update":"Fri Jul 14 18:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64100000000000001,"15min":0.61699999999999999},"min":{"1min":0.44700000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.66400000000000003,"15min":0.64000000000000001},"min":{"1min":0.433,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.625}]},{"osd":7,"last update":"Fri Jul 14 18:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.64800000000000002}]},{"osd":8,"last update":"Fri Jul 14 18:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80900000000000005,"15min":0.79600000000000004},"min":{"1min":0.59599999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0649999999999999,"5min":1.839,"15min":1.839},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.79100000000000004,"15min":0.76800000000000002},"min":{"1min":0.68300000000000005,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.018,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.81200000000000006}]},{"osd":9,"last update":"Fri Jul 14 18:14:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.77000000000000002,"15min":0.77200000000000002},"min":{"1min":0.57099999999999995,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":1.157,"15min":1.2150000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0389999999999999,"5min":7.8689999999999998,"15min":7.8689999999999998},"last":0.77900000000000003}]},{"osd":10,"last update":"Fri Jul 14 18:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.85799999999999998,"15min":0.86199999999999999},"min":{"1min":0.626,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.139,"5min":1.319,"15min":1.319},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.014,"5min":1.081,"15min":1.081},"last":0.86799999999999999}]},{"osd":11,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.90300000000000002,"15min":0.90700000000000003},"min":{"1min":0.65300000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.093,"5min":1.869,"15min":1.869},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.92900000000000005,"15min":0.93500000000000005},"min":{"1min":0.74399999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.0629999999999999,"5min":1.758,"15min":1.758},"last":0.85299999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":548463,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.55900000000000005,"15min":0.53300000000000003},"min":{"1min":0.41199999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.496},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.57699999999999996,"15min":0.53100000000000003},"min":{"1min":0.55600000000000005,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84099999999999997,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.57899999999999996}]},{"osd":1,"last update":"Fri Jul 14 18:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.58499999999999996,"15min":0.52300000000000002},"min":{"1min":0.40799999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.58099999999999996,"15min":0.52200000000000002},"min":{"1min":0.36899999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.44900000000000001}]},{"osd":3,"last update":"Fri Jul 14 18:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60699999999999998,"15min":0.57299999999999995},"min":{"1min":0.441,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.877,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.621},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65100000000000002,"15min":0.65700000000000003},"min":{"1min":0.53200000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81299999999999994,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":0.65100000000000002}]},{"osd":4,"last update":"Fri Jul 14 18:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.376}]},{"osd":5,"last update":"Fri Jul 14 18:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.46899999999999997,"15min":0.45700000000000002},"min":{"1min":0.38200000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.83199999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.50700000000000001,"15min":0.48599999999999999},"min":{"1min":0.44,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.752,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.55500000000000005}]},{"osd":6,"last update":"Fri Jul 14 18:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64400000000000002,"15min":0.63400000000000001},"min":{"1min":0.52100000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84199999999999997,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.65600000000000003,"15min":0.63100000000000001},"min":{"1min":0.54600000000000004,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84799999999999998,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.52100000000000002}]},{"osd":7,"last update":"Fri Jul 14 18:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.53500000000000003}]},{"osd":8,"last update":"Fri Jul 14 18:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.65400000000000003,"15min":0.64600000000000002},"min":{"1min":0.50900000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.64800000000000002,"15min":0.63800000000000001},"min":{"1min":0.40500000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.94499999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.69399999999999995}]},{"osd":9,"last update":"Fri Jul 14 18:14:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.79100000000000004,"15min":0.78700000000000003},"min":{"1min":0.58699999999999997,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.81499999999999995,"15min":0.81100000000000005},"min":{"1min":0.62,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.083,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.751}]},{"osd":10,"last update":"Fri Jul 14 18:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.77600000000000002,"15min":0.75800000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.78000000000000003,"15min":0.76900000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.77100000000000002}]},{"osd":11,"last update":"Fri Jul 14 18:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.73599999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149761,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":488211,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.44900000000000001,"15min":0.39700000000000002},"min":{"1min":0.34300000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.74099999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.47099999999999997,"15min":0.42099999999999999},"min":{"1min":0.311,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.96199999999999997,"5min":0.998,"15min":0.998},"last":0.39900000000000002}]},{"osd":1,"last update":"Fri Jul 14 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.61899999999999999,"15min":0.58299999999999996},"min":{"1min":0.48799999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.64100000000000001,"15min":0.59499999999999997},"min":{"1min":0.51400000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.874,"5min":1.29,"15min":1.29},"last":0.64100000000000001}]},{"osd":2,"last update":"Fri Jul 14 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.64900000000000002,"15min":0.59099999999999997},"min":{"1min":0.55200000000000005,"5min":0.44,"15min":0.44},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.64000000000000001,"15min":0.60699999999999998},"min":{"1min":0.46999999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.58099999999999996}]},{"osd":4,"last update":"Fri Jul 14 18:14:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.629,"15min":0.58899999999999997},"min":{"1min":0.47399999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.83599999999999997,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67100000000000004,"15min":0.63700000000000001},"min":{"1min":0.51300000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.875,"5min":1.153,"15min":1.153},"last":0.59599999999999997}]},{"osd":5,"last update":"Fri Jul 14 18:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.66700000000000004,"15min":0.63700000000000001},"min":{"1min":0.51200000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.66200000000000003,"15min":0.62},"min":{"1min":0.55900000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.54200000000000004}]},{"osd":6,"last update":"Fri Jul 14 18:14:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.621,"15min":0.59899999999999998},"min":{"1min":0.40699999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65400000000000003,"15min":0.622},"min":{"1min":0.47999999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.61299999999999999}]},{"osd":7,"last update":"Fri Jul 14 18:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71299999999999997,"15min":0.68899999999999995},"min":{"1min":0.54100000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.73399999999999999,"15min":0.70699999999999996},"min":{"1min":0.61299999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.69799999999999995}]},{"osd":8,"last update":"Fri Jul 14 18:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.72799999999999998}]},{"osd":9,"last update":"Fri Jul 14 18:14:53 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.81599999999999995,"15min":0.82099999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.628},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.89300000000000002,"15min":0.91100000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.71899999999999997}]},{"osd":10,"last update":"Fri Jul 14 18:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.88300000000000001,"15min":0.88900000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.84399999999999997,"15min":0.85599999999999998},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.0589999999999999,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.70599999999999996}]},{"osd":11,"last update":"Fri Jul 14 18:15:19 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.747}]}]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":493583,"data_compressed":17621,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.48999999999999999,"15min":0.46000000000000002},"min":{"1min":0.39700000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.83799999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.50800000000000001,"15min":0.47799999999999998},"min":{"1min":0.46700000000000003,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.747,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.39600000000000002}]},{"osd":1,"last update":"Fri Jul 14 18:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.43099999999999999,"15min":0.376},"min":{"1min":0.34200000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":3.7890000000000001,"5min":3.7890000000000001,"15min":3.7890000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.41999999999999998,"15min":0.38800000000000001},"min":{"1min":0.379,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.50700000000000001}]},{"osd":2,"last update":"Fri Jul 14 18:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.58999999999999997,"15min":0.55300000000000005},"min":{"1min":0.434,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.628,"15min":0.55800000000000005},"min":{"1min":0.48199999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.6989999999999998,"5min":3.6989999999999998,"15min":3.6989999999999998},"last":0.67200000000000004}]},{"osd":3,"last update":"Fri Jul 14 18:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.57999999999999996,"15min":0.52800000000000002},"min":{"1min":0.53600000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.79600000000000004,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.58399999999999996,"15min":0.53600000000000003},"min":{"1min":0.45100000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.5640000000000001,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":0.78400000000000003}]},{"osd":5,"last update":"Fri Jul 14 18:15:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.64600000000000002,"15min":0.57799999999999996},"min":{"1min":0.50600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":3.7490000000000001,"5min":3.7490000000000001,"15min":3.7490000000000001},"last":0.745},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.63200000000000001,"15min":0.57699999999999996},"min":{"1min":0.52300000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86899999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.625}]},{"osd":6,"last update":"Fri Jul 14 18:14:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67000000000000004,"15min":0.65800000000000003},"min":{"1min":0.50800000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.98799999999999999,"5min":1.075,"15min":1.075},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.68300000000000005,"15min":0.66600000000000004},"min":{"1min":0.498,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.006,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.94499999999999995}]},{"osd":7,"last update":"Fri Jul 14 18:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.65200000000000002,"15min":0.59199999999999997},"min":{"1min":0.54500000000000004,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":3.8999999999999999,"5min":3.8999999999999999,"15min":3.8999999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.875,"5min":0.65800000000000003,"15min":0.60399999999999998},"min":{"1min":0.47599999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.2250000000000001,"5min":3.2250000000000001,"15min":3.2250000000000001},"last":0.91300000000000003}]},{"osd":8,"last update":"Fri Jul 14 18:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.67800000000000005,"15min":0.65000000000000002},"min":{"1min":0.51100000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.79500000000000004,"15min":0.76300000000000001},"min":{"1min":0.58099999999999996,"5min":0.5,"15min":0.5},"max":{"1min":3.1739999999999999,"5min":3.1739999999999999,"15min":3.1739999999999999},"last":0.96499999999999997}]},{"osd":9,"last update":"Fri Jul 14 18:14:53 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.80400000000000005,"15min":0.79500000000000004},"min":{"1min":0.74399999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0700000000000001,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.998},{"interface":"front","average":{"1min":0.877,"5min":0.73099999999999998,"15min":0.70699999999999996},"min":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.3889999999999998,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.81899999999999995}]},{"osd":10,"last update":"Fri Jul 14 18:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.81799999999999995,"15min":0.79400000000000004},"min":{"1min":0.68200000000000005,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":3.8239999999999998,"5min":3.8239999999999998,"15min":3.8239999999999998},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.754,"15min":0.73999999999999999},"min":{"1min":0.497,"5min":0.433,"15min":0.433},"max":{"1min":2.52,"5min":2.52,"15min":2.52},"last":0.85399999999999998}]},{"osd":11,"last update":"Fri Jul 14 18:15:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":3.548,"5min":3.548,"15min":3.548},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":0.98099999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757304,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":1083878,"data_compressed":53365,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:15:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56200000000000006,"15min":0.55300000000000005},"min":{"1min":0.39800000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71799999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56899999999999995,"15min":0.57899999999999996},"min":{"1min":0.46999999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76100000000000001,"5min":1.645,"15min":1.645},"last":0.621}]},{"osd":1,"last update":"Fri Jul 14 18:15:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.58999999999999997,"15min":0.55800000000000005},"min":{"1min":0.503,"5min":0.374,"15min":0.374},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.503},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58599999999999997,"15min":0.57099999999999995},"min":{"1min":0.48799999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.75600000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.73399999999999999}]},{"osd":2,"last update":"Fri Jul 14 18:15:19 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.42099999999999999,"15min":0.42399999999999999},"min":{"1min":0.32800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.622,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.46600000000000003,"15min":0.48899999999999999},"min":{"1min":0.314,"5min":0.252,"15min":0.252},"max":{"1min":0.61099999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.372}]},{"osd":3,"last update":"Fri Jul 14 18:15:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59599999999999997,"15min":0.61599999999999999},"min":{"1min":0.38,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.95699999999999996,"5min":1.591,"15min":1.591},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60699999999999998,"15min":0.60099999999999998},"min":{"1min":0.42599999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.79200000000000004,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.69799999999999995}]},{"osd":4,"last update":"Fri Jul 14 18:15:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.46899999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80900000000000005,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.623,"15min":0.61899999999999999},"min":{"1min":0.5,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79800000000000004,"5min":1.504,"15min":1.504},"last":0.53300000000000003}]},{"osd":6,"last update":"Fri Jul 14 18:14:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.65400000000000003,"15min":0.66000000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.79300000000000004,"5min":1.627,"15min":1.627},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65100000000000002,"15min":0.63600000000000001},"min":{"1min":0.51000000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.83099999999999996,"5min":1.036,"15min":1.036},"last":0.63500000000000001}]},{"osd":7,"last update":"Fri Jul 14 18:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.68500000000000005},"min":{"1min":0.439,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.84899999999999998,"5min":1.554,"15min":1.554},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.67100000000000004,"15min":0.65500000000000003},"min":{"1min":0.51000000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.56999999999999995}]},{"osd":8,"last update":"Fri Jul 14 18:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59399999999999997,"15min":0.57999999999999996},"min":{"1min":0.36599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80600000000000005,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59499999999999997,"15min":0.58299999999999996},"min":{"1min":0.41199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.443}]},{"osd":9,"last update":"Fri Jul 14 18:14:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72899999999999998,"15min":0.72999999999999998},"min":{"1min":0.51800000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92600000000000005,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.76500000000000001,"15min":0.76800000000000002},"min":{"1min":0.46700000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.98199999999999998,"5min":1.149,"15min":1.149},"last":0.748}]},{"osd":10,"last update":"Fri Jul 14 18:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.73799999999999999,"15min":0.72799999999999998},"min":{"1min":0.67100000000000004,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.91300000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.76200000000000001,"15min":0.752},"min":{"1min":0.58199999999999996,"5min":0.505,"15min":0.505},"max":{"1min":1.0349999999999999,"5min":1.107,"15min":1.107},"last":0.66600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34102,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34102,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34102,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":60390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":60536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T18:15:42.315 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-07-14T18:15:42.316 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-07-14T18:15:42.316 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-07-14T18:15:42.316 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph health --format=json 2023-07-14T18:15:43.039 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:43.327 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:42 smithi133 ceph-mon[126384]: pgmap v317: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T18:15:43.327 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:42 smithi133 ceph-mon[126384]: from='client.25007 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T18:15:43.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:42 smithi161 ceph-mon[128993]: pgmap v317: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T18:15:43.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:42 smithi161 ceph-mon[128993]: from='client.25007 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T18:15:43.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:42 smithi171 ceph-mon[129241]: pgmap v317: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T18:15:43.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:42 smithi171 ceph-mon[129241]: from='client.25007 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T18:15:44.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.534+0000 7fc34aad8700 1 -- 172.21.15.133:0/5577621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3441008d0 msgr2=0x7fc3441048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:44.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.534+0000 7fc34aad8700 1 --2- 172.21.15.133:0/5577621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3441008d0 0x7fc3441048a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc3300099c0 tx=0x7fc330009cd0 comp rx=0 tx=0).stop 2023-07-14T18:15:44.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.534+0000 7fc34aad8700 1 -- 172.21.15.133:0/5577621 shutdown_connections 2023-07-14T18:15:44.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.534+0000 7fc34aad8700 1 --2- 172.21.15.133:0/5577621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc344104fd0 0x7fc34410cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:44.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.534+0000 7fc34aad8700 1 --2- 172.21.15.133:0/5577621 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3440fff30 0x7fc344100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:44.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.534+0000 7fc34aad8700 1 --2- 172.21.15.133:0/5577621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3441008d0 0x7fc3441048a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:44.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.534+0000 7fc34aad8700 1 -- 172.21.15.133:0/5577621 >> 172.21.15.133:0/5577621 conn(0x7fc3440fb790 msgr2=0x7fc3440fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:44.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.534+0000 7fc34aad8700 1 -- 172.21.15.133:0/5577621 shutdown_connections 2023-07-14T18:15:44.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.534+0000 7fc34aad8700 1 -- 172.21.15.133:0/5577621 wait complete. 2023-07-14T18:15:44.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.535+0000 7fc34aad8700 1 Processor -- start 2023-07-14T18:15:44.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.535+0000 7fc34aad8700 1 -- start start 2023-07-14T18:15:44.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.536+0000 7fc34aad8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3440fff30 0x7fc344072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:44.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.536+0000 7fc34aad8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3441008d0 0x7fc344072e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:44.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.536+0000 7fc34aad8700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc344104fd0 0x7fc3440733c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:44.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.536+0000 7fc34aad8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3441a41f0 con 0x7fc3441008d0 2023-07-14T18:15:44.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.536+0000 7fc34aad8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3441a4360 con 0x7fc3440fff30 2023-07-14T18:15:44.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.536+0000 7fc34aad8700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3441a4520 con 0x7fc344104fd0 2023-07-14T18:15:44.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.536+0000 7fc343fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3441008d0 0x7fc344072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:44.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.536+0000 7fc343fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3441008d0 0x7fc344072e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50434/0 (socket says 172.21.15.133:50434) 2023-07-14T18:15:44.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.536+0000 7fc343fff700 1 -- 172.21.15.133:0/2292579531 learned_addr learned my addr 172.21.15.133:0/2292579531 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:44.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.536+0000 7fc348874700 1 --2- 172.21.15.133:0/2292579531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3440fff30 0x7fc344072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:44.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.536+0000 7fc349075700 1 --2- 172.21.15.133:0/2292579531 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc344104fd0 0x7fc3440733c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:44.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.536+0000 7fc343fff700 1 -- 172.21.15.133:0/2292579531 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc344104fd0 msgr2=0x7fc3440733c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:44.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.536+0000 7fc343fff700 1 --2- 172.21.15.133:0/2292579531 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc344104fd0 0x7fc3440733c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:44.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.536+0000 7fc343fff700 1 -- 172.21.15.133:0/2292579531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3440fff30 msgr2=0x7fc344072940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:44.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.536+0000 7fc343fff700 1 --2- 172.21.15.133:0/2292579531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3440fff30 0x7fc344072940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:44.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.536+0000 7fc343fff700 1 -- 172.21.15.133:0/2292579531 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc338009c40 con 0x7fc3441008d0 2023-07-14T18:15:44.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.537+0000 7fc343fff700 1 --2- 172.21.15.133:0/2292579531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3441008d0 0x7fc344072e80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc33000b580 tx=0x7fc33000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:44.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.537+0000 7fc341ffb700 1 -- 172.21.15.133:0/2292579531 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33001d070 con 0x7fc3441008d0 2023-07-14T18:15:44.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.537+0000 7fc34aad8700 1 -- 172.21.15.133:0/2292579531 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3300095f0 con 0x7fc3441008d0 2023-07-14T18:15:44.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.537+0000 7fc341ffb700 1 -- 172.21.15.133:0/2292579531 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc330005050 con 0x7fc3441008d0 2023-07-14T18:15:44.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.537+0000 7fc341ffb700 1 -- 172.21.15.133:0/2292579531 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc330004950 con 0x7fc3441008d0 2023-07-14T18:15:44.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.537+0000 7fc34aad8700 1 -- 172.21.15.133:0/2292579531 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3441a4a20 con 0x7fc3441008d0 2023-07-14T18:15:44.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.538+0000 7fc34aad8700 1 -- 172.21.15.133:0/2292579531 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc34404e340 con 0x7fc3441008d0 2023-07-14T18:15:44.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.540+0000 7fc341ffb700 1 -- 172.21.15.133:0/2292579531 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fc3300051c0 con 0x7fc3441008d0 2023-07-14T18:15:44.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.542+0000 7fc341ffb700 1 --2- 172.21.15.133:0/2292579531 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc3340a11e0 0x7fc3340a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:44.587 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.542+0000 7fc341ffb700 1 -- 172.21.15.133:0/2292579531 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc3300cdf30 con 0x7fc3441008d0 2023-07-14T18:15:44.588 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.542+0000 7fc348874700 1 --2- 172.21.15.133:0/2292579531 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc3340a11e0 0x7fc3340a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:44.588 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.543+0000 7fc348874700 1 --2- 172.21.15.133:0/2292579531 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc3340a11e0 0x7fc3340a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc33800f6c0 tx=0x7fc33800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:44.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.545+0000 7fc341ffb700 1 -- 172.21.15.133:0/2292579531 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fc330097240 con 0x7fc3441008d0 2023-07-14T18:15:44.827 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.782+0000 7fc34aad8700 1 -- 172.21.15.133:0/2292579531 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc344073e70 con 0x7fc3441008d0 2023-07-14T18:15:44.827 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.782+0000 7fc341ffb700 1 -- 172.21.15.133:0/2292579531 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fc33009a860 con 0x7fc3441008d0 2023-07-14T18:15:44.827 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:15:44.828 INFO:teuthology.orchestra.run.smithi133.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-14T18:15:44.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.785+0000 7fc34aad8700 1 -- 172.21.15.133:0/2292579531 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc3340a11e0 msgr2=0x7fc3340a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:44.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.785+0000 7fc34aad8700 1 --2- 172.21.15.133:0/2292579531 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc3340a11e0 0x7fc3340a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc33800f6c0 tx=0x7fc33800b040 comp rx=0 tx=0).stop 2023-07-14T18:15:44.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.785+0000 7fc34aad8700 1 -- 172.21.15.133:0/2292579531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3441008d0 msgr2=0x7fc344072e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:44.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.785+0000 7fc34aad8700 1 --2- 172.21.15.133:0/2292579531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3441008d0 0x7fc344072e80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc33000b580 tx=0x7fc33000bfd0 comp rx=0 tx=0).stop 2023-07-14T18:15:44.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.785+0000 7fc34aad8700 1 -- 172.21.15.133:0/2292579531 shutdown_connections 2023-07-14T18:15:44.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.785+0000 7fc34aad8700 1 --2- 172.21.15.133:0/2292579531 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc3340a11e0 0x7fc3340a3690 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:44.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.785+0000 7fc34aad8700 1 --2- 172.21.15.133:0/2292579531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3440fff30 0x7fc344072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:44.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.785+0000 7fc34aad8700 1 --2- 172.21.15.133:0/2292579531 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc344104fd0 0x7fc3440733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:44.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.785+0000 7fc34aad8700 1 --2- 172.21.15.133:0/2292579531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3441008d0 0x7fc344072e80 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:44.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.785+0000 7fc34aad8700 1 -- 172.21.15.133:0/2292579531 >> 172.21.15.133:0/2292579531 conn(0x7fc3440fb790 msgr2=0x7fc3440fc200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:44.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.785+0000 7fc34aad8700 1 -- 172.21.15.133:0/2292579531 shutdown_connections 2023-07-14T18:15:44.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:44.785+0000 7fc34aad8700 1 -- 172.21.15.133:0/2292579531 wait complete. 2023-07-14T18:15:45.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:44 smithi133 ceph-mon[126384]: pgmap v318: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-14T18:15:45.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:44 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2292579531' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-14T18:15:45.337 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-07-14T18:15:45.337 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T18:15:45.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:44 smithi161 ceph-mon[128993]: pgmap v318: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-14T18:15:45.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:44 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2292579531' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-14T18:15:45.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:44 smithi171 ceph-mon[129241]: pgmap v318: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-14T18:15:45.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:44 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2292579531' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-14T18:15:45.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.493+0000 7f2ab93ff700 1 -- 172.21.15.133:0/526141419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ab4100e20 msgr2=0x7f2ab4101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:45.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.493+0000 7f2ab93ff700 1 --2- 172.21.15.133:0/526141419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ab4100e20 0x7f2ab4101270 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8009990 tx=0x7f2aa8009ca0 comp rx=0 tx=0).stop 2023-07-14T18:15:45.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.493+0000 7f2ab93ff700 1 -- 172.21.15.133:0/526141419 shutdown_connections 2023-07-14T18:15:45.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.493+0000 7f2ab93ff700 1 --2- 172.21.15.133:0/526141419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ab41017b0 0x7f2ab4108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:45.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.493+0000 7f2ab93ff700 1 --2- 172.21.15.133:0/526141419 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ab40ffc30 0x7f2ab4100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:45.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.493+0000 7f2ab93ff700 1 --2- 172.21.15.133:0/526141419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ab4100e20 0x7f2ab4101270 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:45.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.493+0000 7f2ab93ff700 1 -- 172.21.15.133:0/526141419 >> 172.21.15.133:0/526141419 conn(0x7f2ab40fb210 msgr2=0x7f2ab40fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:45.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.493+0000 7f2ab93ff700 1 -- 172.21.15.133:0/526141419 shutdown_connections 2023-07-14T18:15:45.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.494+0000 7f2ab93ff700 1 -- 172.21.15.133:0/526141419 wait complete. 2023-07-14T18:15:45.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.495+0000 7f2ab93ff700 1 Processor -- start 2023-07-14T18:15:45.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.495+0000 7f2ab93ff700 1 -- start start 2023-07-14T18:15:45.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.495+0000 7f2ab93ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ab40ffc30 0x7f2ab4197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:45.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.495+0000 7f2ab93ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ab4100e20 0x7f2ab4198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:45.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.495+0000 7f2ab93ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ab41017b0 0x7f2ab419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:45.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.495+0000 7f2ab93ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab419f650 con 0x7f2ab4100e20 2023-07-14T18:15:45.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.495+0000 7f2ab93ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab419f790 con 0x7f2ab40ffc30 2023-07-14T18:15:45.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.495+0000 7f2ab93ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab419f920 con 0x7f2ab41017b0 2023-07-14T18:15:45.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.495+0000 7f2ab27fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ab4100e20 0x7f2ab4198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:45.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.495+0000 7f2ab27fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ab4100e20 0x7f2ab4198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50452/0 (socket says 172.21.15.133:50452) 2023-07-14T18:15:45.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.495+0000 7f2ab27fc700 1 -- 172.21.15.133:0/638318128 learned_addr learned my addr 172.21.15.133:0/638318128 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:45.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.495+0000 7f2ab2ffd700 1 --2- 172.21.15.133:0/638318128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ab40ffc30 0x7f2ab4197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:45.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.496+0000 7f2ab37fe700 1 --2- 172.21.15.133:0/638318128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ab41017b0 0x7f2ab419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:45.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.496+0000 7f2ab2ffd700 1 -- 172.21.15.133:0/638318128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ab41017b0 msgr2=0x7f2ab419f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:45.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.496+0000 7f2ab2ffd700 1 --2- 172.21.15.133:0/638318128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ab41017b0 0x7f2ab419f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:45.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.496+0000 7f2ab2ffd700 1 -- 172.21.15.133:0/638318128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ab4100e20 msgr2=0x7f2ab4198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:45.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.496+0000 7f2ab2ffd700 1 --2- 172.21.15.133:0/638318128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ab4100e20 0x7f2ab4198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:45.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.496+0000 7f2ab2ffd700 1 -- 172.21.15.133:0/638318128 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa80095f0 con 0x7f2ab40ffc30 2023-07-14T18:15:45.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.496+0000 7f2ab37fe700 1 --2- 172.21.15.133:0/638318128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ab41017b0 0x7f2ab419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:15:45.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.496+0000 7f2ab27fc700 1 --2- 172.21.15.133:0/638318128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ab4100e20 0x7f2ab4198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:45.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.496+0000 7f2ab2ffd700 1 --2- 172.21.15.133:0/638318128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ab40ffc30 0x7f2ab4197ad0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2aa400e9c0 tx=0x7f2aa400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:45.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.496+0000 7f2a9bfff700 1 -- 172.21.15.133:0/638318128 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa400cbd0 con 0x7f2ab40ffc30 2023-07-14T18:15:45.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.496+0000 7f2ab93ff700 1 -- 172.21.15.133:0/638318128 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab419fc00 con 0x7f2ab40ffc30 2023-07-14T18:15:45.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.496+0000 7f2a9bfff700 1 -- 172.21.15.133:0/638318128 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2aa400eeb0 con 0x7f2ab40ffc30 2023-07-14T18:15:45.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.496+0000 7f2a9bfff700 1 -- 172.21.15.133:0/638318128 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4003c70 con 0x7f2ab40ffc30 2023-07-14T18:15:45.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.496+0000 7f2ab93ff700 1 -- 172.21.15.133:0/638318128 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab41a0160 con 0x7f2ab40ffc30 2023-07-14T18:15:45.545 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.497+0000 7f2ab93ff700 1 -- 172.21.15.133:0/638318128 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab410b450 con 0x7f2ab40ffc30 2023-07-14T18:15:45.545 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.500+0000 7f2a9bfff700 1 -- 172.21.15.133:0/638318128 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f2aa4004870 con 0x7f2ab40ffc30 2023-07-14T18:15:45.546 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.501+0000 7f2a9bfff700 1 --2- 172.21.15.133:0/638318128 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f2a9c0a1220 0x7f2a9c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:45.546 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.501+0000 7f2a9bfff700 1 -- 172.21.15.133:0/638318128 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2aa4014070 con 0x7f2ab40ffc30 2023-07-14T18:15:45.546 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.501+0000 7f2ab27fc700 1 --2- 172.21.15.133:0/638318128 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f2a9c0a1220 0x7f2a9c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:45.546 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.502+0000 7f2ab27fc700 1 --2- 172.21.15.133:0/638318128 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f2a9c0a1220 0x7f2a9c0a36d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8006010 tx=0x7f2aa800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:45.547 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.503+0000 7f2a9bfff700 1 -- 172.21.15.133:0/638318128 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f2aa4096980 con 0x7f2ab40ffc30 2023-07-14T18:15:45.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.706+0000 7f2ab93ff700 1 -- 172.21.15.133:0/638318128 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ab410b5e0 con 0x7f2ab40ffc30 2023-07-14T18:15:45.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.709+0000 7f2a9bfff700 1 -- 172.21.15.133:0/638318128 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24473 (secure 0 0 0) 0x7f2aa4099fa0 con 0x7f2ab40ffc30 2023-07-14T18:15:45.754 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:15:45.754 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":90,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","created":"2023-07-14T18:06:49.561582+0000","modified":"2023-07-14T18:14:48.934538+0000","last_up_change":"2023-07-14T18:13:29.836099+0000","last_in_change":"2023-07-14T18:13:11.606710+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T18:10:03.945039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T18:14:15.451340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T18:14:16.892215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T18:14:20.493175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"85a29de7-d712-422a-b647-d9e90091aaf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6803","nonce":3076510801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6805","nonce":3076510801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6809","nonce":3076510801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6807","nonce":3076510801}]},"public_addr":"172.21.15.133:6803/3076510801","cluster_addr":"172.21.15.133:6805/3076510801","heartbeat_back_addr":"172.21.15.133:6809/3076510801","heartbeat_front_addr":"172.21.15.133:6807/3076510801","state":["exists","up"]},{"osd":1,"uuid":"80ad4e78-7917-495e-b119-99f7c7c36243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6801","nonce":1841901852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6803","nonce":1841901852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6807","nonce":1841901852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6805","nonce":1841901852}]},"public_addr":"172.21.15.161:6801/1841901852","cluster_addr":"172.21.15.161:6803/1841901852","heartbeat_back_addr":"172.21.15.161:6807/1841901852","heartbeat_front_addr":"172.21.15.161:6805/1841901852","state":["exists","up"]},{"osd":2,"uuid":"fc4bfd37-1fdd-47c6-a48e-6d910daf9e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6801","nonce":346103099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6803","nonce":346103099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6807","nonce":346103099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6805","nonce":346103099}]},"public_addr":"172.21.15.171:6801/346103099","cluster_addr":"172.21.15.171:6803/346103099","heartbeat_back_addr":"172.21.15.171:6807/346103099","heartbeat_front_addr":"172.21.15.171:6805/346103099","state":["exists","up"]},{"osd":3,"uuid":"4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6811","nonce":2152994143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6813","nonce":2152994143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6817","nonce":2152994143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6815","nonce":2152994143}]},"public_addr":"172.21.15.133:6811/2152994143","cluster_addr":"172.21.15.133:6813/2152994143","heartbeat_back_addr":"172.21.15.133:6817/2152994143","heartbeat_front_addr":"172.21.15.133:6815/2152994143","state":["exists","up"]},{"osd":4,"uuid":"4601a1c2-254b-4c4d-9b15-8fc952b3d2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6809","nonce":47213291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6811","nonce":47213291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6815","nonce":47213291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6813","nonce":47213291}]},"public_addr":"172.21.15.161:6809/47213291","cluster_addr":"172.21.15.161:6811/47213291","heartbeat_back_addr":"172.21.15.161:6815/47213291","heartbeat_front_addr":"172.21.15.161:6813/47213291","state":["exists","up"]},{"osd":5,"uuid":"6cca180a-9140-4e49-835b-86842dae0574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6809","nonce":777556751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6811","nonce":777556751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6815","nonce":777556751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6813","nonce":777556751}]},"public_addr":"172.21.15.171:6809/777556751","cluster_addr":"172.21.15.171:6811/777556751","heartbeat_back_addr":"172.21.15.171:6815/777556751","heartbeat_front_addr":"172.21.15.171:6813/777556751","state":["exists","up"]},{"osd":6,"uuid":"ea0299ff-7740-4494-a3a9-1503b3fc0ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6819","nonce":2884152358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6821","nonce":2884152358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6825","nonce":2884152358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6823","nonce":2884152358}]},"public_addr":"172.21.15.133:6819/2884152358","cluster_addr":"172.21.15.133:6821/2884152358","heartbeat_back_addr":"172.21.15.133:6825/2884152358","heartbeat_front_addr":"172.21.15.133:6823/2884152358","state":["exists","up"]},{"osd":7,"uuid":"b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6817","nonce":4183705749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6819","nonce":4183705749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6823","nonce":4183705749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6821","nonce":4183705749}]},"public_addr":"172.21.15.161:6817/4183705749","cluster_addr":"172.21.15.161:6819/4183705749","heartbeat_back_addr":"172.21.15.161:6823/4183705749","heartbeat_front_addr":"172.21.15.161:6821/4183705749","state":["exists","up"]},{"osd":8,"uuid":"87dffc07-bf48-48c8-b418-5c4b509686d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6817","nonce":2771494483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6819","nonce":2771494483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6823","nonce":2771494483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6821","nonce":2771494483}]},"public_addr":"172.21.15.171:6817/2771494483","cluster_addr":"172.21.15.171:6819/2771494483","heartbeat_back_addr":"172.21.15.171:6823/2771494483","heartbeat_front_addr":"172.21.15.171:6821/2771494483","state":["exists","up"]},{"osd":9,"uuid":"e0832870-79ce-4bce-8045-58b7a7f6dc69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6827","nonce":2066164149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6829","nonce":2066164149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6833","nonce":2066164149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6831","nonce":2066164149}]},"public_addr":"172.21.15.133:6827/2066164149","cluster_addr":"172.21.15.133:6829/2066164149","heartbeat_back_addr":"172.21.15.133:6833/2066164149","heartbeat_front_addr":"172.21.15.133:6831/2066164149","state":["exists","up"]},{"osd":10,"uuid":"081f0484-d134-4b2e-9c48-ef8e6a646422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6825","nonce":2162333626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6827","nonce":2162333626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6831","nonce":2162333626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6829","nonce":2162333626}]},"public_addr":"172.21.15.161:6825/2162333626","cluster_addr":"172.21.15.161:6827/2162333626","heartbeat_back_addr":"172.21.15.161:6831/2162333626","heartbeat_front_addr":"172.21.15.161:6829/2162333626","state":["exists","up"]},{"osd":11,"uuid":"4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6825","nonce":3403894313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6827","nonce":3403894313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6831","nonce":3403894313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6829","nonce":3403894313}]},"public_addr":"172.21.15.171:6825/3403894313","cluster_addr":"172.21.15.171:6827/3403894313","heartbeat_back_addr":"172.21.15.171:6831/3403894313","heartbeat_front_addr":"172.21.15.171:6829/3403894313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:17.816229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:39.638543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:00.543317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:26.988921+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:48.644694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:10.049860+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:35.986508+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:57.753846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:19.018245+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:44.460450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:06.279960+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:27.955006+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/1832569241":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6801/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:0/4219815304":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:6801/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/432253615":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1840700480":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1807775439":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1089030734":"2023-07-15T18:07:47.346704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T18:15:45.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.711+0000 7f2ab93ff700 1 -- 172.21.15.133:0/638318128 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f2a9c0a1220 msgr2=0x7f2a9c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:45.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.711+0000 7f2ab93ff700 1 --2- 172.21.15.133:0/638318128 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f2a9c0a1220 0x7f2a9c0a36d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8006010 tx=0x7f2aa800b260 comp rx=0 tx=0).stop 2023-07-14T18:15:45.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.711+0000 7f2ab93ff700 1 -- 172.21.15.133:0/638318128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ab40ffc30 msgr2=0x7f2ab4197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:45.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.711+0000 7f2ab93ff700 1 --2- 172.21.15.133:0/638318128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ab40ffc30 0x7f2ab4197ad0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2aa400e9c0 tx=0x7f2aa400ecd0 comp rx=0 tx=0).stop 2023-07-14T18:15:45.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.711+0000 7f2ab93ff700 1 -- 172.21.15.133:0/638318128 shutdown_connections 2023-07-14T18:15:45.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.711+0000 7f2ab93ff700 1 --2- 172.21.15.133:0/638318128 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f2a9c0a1220 0x7f2a9c0a36d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:45.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.711+0000 7f2ab93ff700 1 --2- 172.21.15.133:0/638318128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ab40ffc30 0x7f2ab4197ad0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:45.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.711+0000 7f2ab93ff700 1 --2- 172.21.15.133:0/638318128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ab41017b0 0x7f2ab419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:45.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.711+0000 7f2ab93ff700 1 --2- 172.21.15.133:0/638318128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ab4100e20 0x7f2ab4198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:45.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.711+0000 7f2ab93ff700 1 -- 172.21.15.133:0/638318128 >> 172.21.15.133:0/638318128 conn(0x7f2ab40fb210 msgr2=0x7f2ab41066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:45.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.711+0000 7f2ab93ff700 1 -- 172.21.15.133:0/638318128 shutdown_connections 2023-07-14T18:15:45.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.711+0000 7f2ab93ff700 1 -- 172.21.15.133:0/638318128 wait complete. 2023-07-14T18:15:45.775 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T18:10:03.945039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T18:14:15.451340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T18:14:16.892215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T18:14:20.493175+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T18:15:45.776 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T18:15:45.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.937+0000 7f8b715ff700 1 -- 172.21.15.133:0/1151086731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b6c103940 msgr2=0x7f8b6c10a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:45.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.937+0000 7f8b715ff700 1 --2- 172.21.15.133:0/1151086731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b6c103940 0x7f8b6c10a270 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400b0a0 tx=0x7f8b6400b3b0 comp rx=0 tx=0).stop 2023-07-14T18:15:45.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.937+0000 7f8b715ff700 1 -- 172.21.15.133:0/1151086731 shutdown_connections 2023-07-14T18:15:45.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.937+0000 7f8b715ff700 1 --2- 172.21.15.133:0/1151086731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b6c101e40 0x7f8b6c102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:45.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.937+0000 7f8b715ff700 1 --2- 172.21.15.133:0/1151086731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b6c103940 0x7f8b6c10a270 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:45.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.937+0000 7f8b715ff700 1 --2- 172.21.15.133:0/1151086731 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b6c102f90 0x7f8b6c103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:45.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.937+0000 7f8b715ff700 1 -- 172.21.15.133:0/1151086731 >> 172.21.15.133:0/1151086731 conn(0x7f8b6c0fd480 msgr2=0x7f8b6c0ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:45.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.937+0000 7f8b715ff700 1 -- 172.21.15.133:0/1151086731 shutdown_connections 2023-07-14T18:15:45.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.937+0000 7f8b715ff700 1 -- 172.21.15.133:0/1151086731 wait complete. 2023-07-14T18:15:45.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.938+0000 7f8b715ff700 1 Processor -- start 2023-07-14T18:15:45.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.938+0000 7f8b715ff700 1 -- start start 2023-07-14T18:15:45.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.939+0000 7f8b715ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b6c101e40 0x7f8b6c197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:45.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.939+0000 7f8b715ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b6c102f90 0x7f8b6c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:45.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.939+0000 7f8b715ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b6c103940 0x7f8b6c19f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:45.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.939+0000 7f8b715ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c19f9b0 con 0x7f8b6c102f90 2023-07-14T18:15:45.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.939+0000 7f8b715ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c19faf0 con 0x7f8b6c103940 2023-07-14T18:15:45.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.939+0000 7f8b715ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c19fc80 con 0x7f8b6c101e40 2023-07-14T18:15:45.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.939+0000 7f8b6a7fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b6c102f90 0x7f8b6c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:45.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.939+0000 7f8b6a7fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b6c102f90 0x7f8b6c198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50478/0 (socket says 172.21.15.133:50478) 2023-07-14T18:15:45.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.939+0000 7f8b6a7fc700 1 -- 172.21.15.133:0/4262468523 learned_addr learned my addr 172.21.15.133:0/4262468523 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:45.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.939+0000 7f8b6b7fe700 1 --2- 172.21.15.133:0/4262468523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b6c103940 0x7f8b6c19f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:45.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.939+0000 7f8b6affd700 1 --2- 172.21.15.133:0/4262468523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b6c101e40 0x7f8b6c197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:45.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.939+0000 7f8b6a7fc700 1 -- 172.21.15.133:0/4262468523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b6c101e40 msgr2=0x7f8b6c197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:45.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.939+0000 7f8b6a7fc700 1 --2- 172.21.15.133:0/4262468523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b6c101e40 0x7f8b6c197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:45.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.939+0000 7f8b6a7fc700 1 -- 172.21.15.133:0/4262468523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b6c103940 msgr2=0x7f8b6c19f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:45.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.939+0000 7f8b6a7fc700 1 --2- 172.21.15.133:0/4262468523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b6c103940 0x7f8b6c19f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:45.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.940+0000 7f8b6a7fc700 1 -- 172.21.15.133:0/4262468523 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b64009d00 con 0x7f8b6c102f90 2023-07-14T18:15:45.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.940+0000 7f8b6b7fe700 1 --2- 172.21.15.133:0/4262468523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b6c103940 0x7f8b6c19f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:45.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.940+0000 7f8b6a7fc700 1 --2- 172.21.15.133:0/4262468523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b6c102f90 0x7f8b6c198370 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000e9a0 tx=0x7f8b6000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:45.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.940+0000 7f8b53fff700 1 -- 172.21.15.133:0/4262468523 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b60009790 con 0x7f8b6c102f90 2023-07-14T18:15:45.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.940+0000 7f8b53fff700 1 -- 172.21.15.133:0/4262468523 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b60010430 con 0x7f8b6c102f90 2023-07-14T18:15:45.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.940+0000 7f8b715ff700 1 -- 172.21.15.133:0/4262468523 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6c19ff60 con 0x7f8b6c102f90 2023-07-14T18:15:45.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.940+0000 7f8b53fff700 1 -- 172.21.15.133:0/4262468523 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b60018680 con 0x7f8b6c102f90 2023-07-14T18:15:45.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.940+0000 7f8b715ff700 1 -- 172.21.15.133:0/4262468523 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6c1a0430 con 0x7f8b6c102f90 2023-07-14T18:15:45.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.941+0000 7f8b715ff700 1 -- 172.21.15.133:0/4262468523 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b6c109110 con 0x7f8b6c102f90 2023-07-14T18:15:45.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.943+0000 7f8b53fff700 1 -- 172.21.15.133:0/4262468523 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f8b600105a0 con 0x7f8b6c102f90 2023-07-14T18:15:45.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.945+0000 7f8b53fff700 1 --2- 172.21.15.133:0/4262468523 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8b540a1320 0x7f8b540a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:45.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.945+0000 7f8b53fff700 1 -- 172.21.15.133:0/4262468523 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8b60014070 con 0x7f8b6c102f90 2023-07-14T18:15:45.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.945+0000 7f8b6affd700 1 --2- 172.21.15.133:0/4262468523 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8b540a1320 0x7f8b540a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:45.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.946+0000 7f8b6affd700 1 --2- 172.21.15.133:0/4262468523 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8b540a1320 0x7f8b540a37d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c009c60 tx=0x7f8b5c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:45.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:45.949+0000 7f8b53fff700 1 -- 172.21.15.133:0/4262468523 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f8b60051780 con 0x7f8b6c102f90 2023-07-14T18:15:46.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:45 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/638318128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:15:46.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.146+0000 7f8b715ff700 1 -- 172.21.15.133:0/4262468523 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8b6c061250 con 0x7f8b6c102f90 2023-07-14T18:15:46.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.146+0000 7f8b53fff700 1 -- 172.21.15.133:0/4262468523 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8b600d1070 con 0x7f8b6c102f90 2023-07-14T18:15:46.191 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-07-14T18:15:46.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.149+0000 7f8b715ff700 1 -- 172.21.15.133:0/4262468523 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8b540a1320 msgr2=0x7f8b540a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:46.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.149+0000 7f8b715ff700 1 --2- 172.21.15.133:0/4262468523 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8b540a1320 0x7f8b540a37d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c009c60 tx=0x7f8b5c009260 comp rx=0 tx=0).stop 2023-07-14T18:15:46.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.149+0000 7f8b715ff700 1 -- 172.21.15.133:0/4262468523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b6c102f90 msgr2=0x7f8b6c198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:46.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.149+0000 7f8b715ff700 1 --2- 172.21.15.133:0/4262468523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b6c102f90 0x7f8b6c198370 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000e9a0 tx=0x7f8b6000ecb0 comp rx=0 tx=0).stop 2023-07-14T18:15:46.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.149+0000 7f8b715ff700 1 -- 172.21.15.133:0/4262468523 shutdown_connections 2023-07-14T18:15:46.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.149+0000 7f8b715ff700 1 --2- 172.21.15.133:0/4262468523 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8b540a1320 0x7f8b540a37d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:46.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.149+0000 7f8b715ff700 1 --2- 172.21.15.133:0/4262468523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b6c103940 0x7f8b6c19f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:46.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.149+0000 7f8b715ff700 1 --2- 172.21.15.133:0/4262468523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b6c101e40 0x7f8b6c197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:46.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.149+0000 7f8b715ff700 1 --2- 172.21.15.133:0/4262468523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b6c102f90 0x7f8b6c198370 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:46.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.149+0000 7f8b715ff700 1 -- 172.21.15.133:0/4262468523 >> 172.21.15.133:0/4262468523 conn(0x7f8b6c0fd480 msgr2=0x7f8b6c106250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:46.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.149+0000 7f8b715ff700 1 -- 172.21.15.133:0/4262468523 shutdown_connections 2023-07-14T18:15:46.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.149+0000 7f8b715ff700 1 -- 172.21.15.133:0/4262468523 wait complete. 2023-07-14T18:15:46.212 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T18:15:46.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:45 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/638318128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:15:46.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:45 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/638318128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:15:46.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.368+0000 7ff6ce7ff700 1 -- 172.21.15.133:0/1242689519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c8100e00 msgr2=0x7ff6c8101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:46.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.368+0000 7ff6ce7ff700 1 --2- 172.21.15.133:0/1242689519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c8100e00 0x7ff6c8101250 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc009990 tx=0x7ff6bc009ca0 comp rx=0 tx=0).stop 2023-07-14T18:15:46.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.368+0000 7ff6ce7ff700 1 -- 172.21.15.133:0/1242689519 shutdown_connections 2023-07-14T18:15:46.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.368+0000 7ff6ce7ff700 1 --2- 172.21.15.133:0/1242689519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6c80ffc10 0x7ff6c8100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:46.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.368+0000 7ff6ce7ff700 1 --2- 172.21.15.133:0/1242689519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6c8101790 0x7ff6c8108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:46.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.368+0000 7ff6ce7ff700 1 --2- 172.21.15.133:0/1242689519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c8100e00 0x7ff6c8101250 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:46.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.368+0000 7ff6ce7ff700 1 -- 172.21.15.133:0/1242689519 >> 172.21.15.133:0/1242689519 conn(0x7ff6c80fb210 msgr2=0x7ff6c80fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:46.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.368+0000 7ff6ce7ff700 1 -- 172.21.15.133:0/1242689519 shutdown_connections 2023-07-14T18:15:46.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.368+0000 7ff6ce7ff700 1 -- 172.21.15.133:0/1242689519 wait complete. 2023-07-14T18:15:46.416 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.369+0000 7ff6ce7ff700 1 Processor -- start 2023-07-14T18:15:46.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.369+0000 7ff6ce7ff700 1 -- start start 2023-07-14T18:15:46.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.370+0000 7ff6ce7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6c80ffc10 0x7ff6c81959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:46.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.370+0000 7ff6ce7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c8100e00 0x7ff6c8195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:46.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.370+0000 7ff6ce7ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6c8101790 0x7ff6c819cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:46.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.370+0000 7ff6ce7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c819d530 con 0x7ff6c8100e00 2023-07-14T18:15:46.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.370+0000 7ff6ce7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c819d670 con 0x7ff6c80ffc10 2023-07-14T18:15:46.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.370+0000 7ff6ce7ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c819d800 con 0x7ff6c8101790 2023-07-14T18:15:46.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.370+0000 7ff6c77fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c8100e00 0x7ff6c8195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:46.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.370+0000 7ff6c7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6c80ffc10 0x7ff6c81959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:46.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.370+0000 7ff6c77fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c8100e00 0x7ff6c8195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50492/0 (socket says 172.21.15.133:50492) 2023-07-14T18:15:46.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.370+0000 7ff6c77fe700 1 -- 172.21.15.133:0/3588477388 learned_addr learned my addr 172.21.15.133:0/3588477388 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:46.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.370+0000 7ff6ccd9c700 1 --2- 172.21.15.133:0/3588477388 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6c8101790 0x7ff6c819cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:46.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.370+0000 7ff6c7fff700 1 -- 172.21.15.133:0/3588477388 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6c8101790 msgr2=0x7ff6c819cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:46.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.370+0000 7ff6c7fff700 1 --2- 172.21.15.133:0/3588477388 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6c8101790 0x7ff6c819cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:46.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.370+0000 7ff6c7fff700 1 -- 172.21.15.133:0/3588477388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c8100e00 msgr2=0x7ff6c8195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:46.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.370+0000 7ff6c7fff700 1 --2- 172.21.15.133:0/3588477388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c8100e00 0x7ff6c8195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:46.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.370+0000 7ff6c7fff700 1 -- 172.21.15.133:0/3588477388 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6bc0095f0 con 0x7ff6c80ffc10 2023-07-14T18:15:46.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.370+0000 7ff6c77fe700 1 --2- 172.21.15.133:0/3588477388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c8100e00 0x7ff6c8195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:46.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.371+0000 7ff6c7fff700 1 --2- 172.21.15.133:0/3588477388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6c80ffc10 0x7ff6c81959b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff6b800e9c0 tx=0x7ff6b800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:46.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.371+0000 7ff6c57fa700 1 -- 172.21.15.133:0/3588477388 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8004220 con 0x7ff6c80ffc10 2023-07-14T18:15:46.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.371+0000 7ff6ce7ff700 1 -- 172.21.15.133:0/3588477388 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c819dae0 con 0x7ff6c80ffc10 2023-07-14T18:15:46.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.371+0000 7ff6c57fa700 1 -- 172.21.15.133:0/3588477388 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6b8004380 con 0x7ff6c80ffc10 2023-07-14T18:15:46.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.371+0000 7ff6c57fa700 1 -- 172.21.15.133:0/3588477388 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8003be0 con 0x7ff6c80ffc10 2023-07-14T18:15:46.421 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.371+0000 7ff6ce7ff700 1 -- 172.21.15.133:0/3588477388 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c819dff0 con 0x7ff6c80ffc10 2023-07-14T18:15:46.421 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.372+0000 7ff6ce7ff700 1 -- 172.21.15.133:0/3588477388 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c810b430 con 0x7ff6c80ffc10 2023-07-14T18:15:46.421 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.374+0000 7ff6c57fa700 1 -- 172.21.15.133:0/3588477388 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7ff6b8003d40 con 0x7ff6c80ffc10 2023-07-14T18:15:46.422 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.376+0000 7ff6c57fa700 1 --2- 172.21.15.133:0/3588477388 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff6b00a11e0 0x7ff6b00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:46.422 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.376+0000 7ff6c57fa700 1 -- 172.21.15.133:0/3588477388 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff6b8014070 con 0x7ff6c80ffc10 2023-07-14T18:15:46.422 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.376+0000 7ff6c77fe700 1 --2- 172.21.15.133:0/3588477388 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff6b00a11e0 0x7ff6b00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:46.422 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.376+0000 7ff6c77fe700 1 --2- 172.21.15.133:0/3588477388 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff6b00a11e0 0x7ff6b00a3690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00b2e0 tx=0x7ff6bc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:46.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.378+0000 7ff6c57fa700 1 -- 172.21.15.133:0/3588477388 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7ff6b80964c0 con 0x7ff6c80ffc10 2023-07-14T18:15:46.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.577+0000 7ff6ce7ff700 1 -- 172.21.15.133:0/3588477388 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff6c8106b10 con 0x7ff6c80ffc10 2023-07-14T18:15:46.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.578+0000 7ff6c57fa700 1 -- 172.21.15.133:0/3588477388 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff6b801f340 con 0x7ff6c80ffc10 2023-07-14T18:15:46.622 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-07-14T18:15:46.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.580+0000 7ff6ce7ff700 1 -- 172.21.15.133:0/3588477388 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff6b00a11e0 msgr2=0x7ff6b00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:46.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.580+0000 7ff6ce7ff700 1 --2- 172.21.15.133:0/3588477388 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff6b00a11e0 0x7ff6b00a3690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00b2e0 tx=0x7ff6bc005fb0 comp rx=0 tx=0).stop 2023-07-14T18:15:46.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.580+0000 7ff6ce7ff700 1 -- 172.21.15.133:0/3588477388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6c80ffc10 msgr2=0x7ff6c81959b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:46.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.580+0000 7ff6ce7ff700 1 --2- 172.21.15.133:0/3588477388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6c80ffc10 0x7ff6c81959b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff6b800e9c0 tx=0x7ff6b800ecd0 comp rx=0 tx=0).stop 2023-07-14T18:15:46.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.580+0000 7ff6ce7ff700 1 -- 172.21.15.133:0/3588477388 shutdown_connections 2023-07-14T18:15:46.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.580+0000 7ff6ce7ff700 1 --2- 172.21.15.133:0/3588477388 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff6b00a11e0 0x7ff6b00a3690 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:46.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.580+0000 7ff6ce7ff700 1 --2- 172.21.15.133:0/3588477388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6c80ffc10 0x7ff6c81959b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:46.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.580+0000 7ff6ce7ff700 1 --2- 172.21.15.133:0/3588477388 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6c8101790 0x7ff6c819cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:46.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.580+0000 7ff6ce7ff700 1 --2- 172.21.15.133:0/3588477388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c8100e00 0x7ff6c8195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:46.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.580+0000 7ff6ce7ff700 1 -- 172.21.15.133:0/3588477388 >> 172.21.15.133:0/3588477388 conn(0x7ff6c80fb210 msgr2=0x7ff6c80fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:46.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.580+0000 7ff6ce7ff700 1 -- 172.21.15.133:0/3588477388 shutdown_connections 2023-07-14T18:15:46.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.580+0000 7ff6ce7ff700 1 -- 172.21.15.133:0/3588477388 wait complete. 2023-07-14T18:15:46.643 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T18:15:46.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.798+0000 7f815dbff700 1 -- 172.21.15.133:0/1953530709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8158102f90 msgr2=0x7f8158103400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:46.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.798+0000 7f815dbff700 1 --2- 172.21.15.133:0/1953530709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8158102f90 0x7f8158103400 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8140009990 tx=0x7f8140009ca0 comp rx=0 tx=0).stop 2023-07-14T18:15:46.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.798+0000 7f815dbff700 1 -- 172.21.15.133:0/1953530709 shutdown_connections 2023-07-14T18:15:46.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.798+0000 7f815dbff700 1 --2- 172.21.15.133:0/1953530709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8158101e40 0x7f8158102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:46.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.798+0000 7f815dbff700 1 --2- 172.21.15.133:0/1953530709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8158103940 0x7f815810a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:46.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.798+0000 7f815dbff700 1 --2- 172.21.15.133:0/1953530709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8158102f90 0x7f8158103400 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:46.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.798+0000 7f815dbff700 1 -- 172.21.15.133:0/1953530709 >> 172.21.15.133:0/1953530709 conn(0x7f81580fd480 msgr2=0x7f81580ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:46.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.798+0000 7f815dbff700 1 -- 172.21.15.133:0/1953530709 shutdown_connections 2023-07-14T18:15:46.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.798+0000 7f815dbff700 1 -- 172.21.15.133:0/1953530709 wait complete. 2023-07-14T18:15:46.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.799+0000 7f815dbff700 1 Processor -- start 2023-07-14T18:15:46.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.799+0000 7f815dbff700 1 -- start start 2023-07-14T18:15:46.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.800+0000 7f815dbff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8158101e40 0x7f8158197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:46.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.800+0000 7f815dbff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8158102f90 0x7f81581982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:46.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.800+0000 7f815dbff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8158103940 0x7f815819f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:46.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.800+0000 7f815dbff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815819f930 con 0x7f8158101e40 2023-07-14T18:15:46.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.800+0000 7f815dbff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815819fa70 con 0x7f8158103940 2023-07-14T18:15:46.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.800+0000 7f815dbff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815819fc00 con 0x7f8158102f90 2023-07-14T18:15:46.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.800+0000 7f81577fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8158101e40 0x7f8158197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:46.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.800+0000 7f81577fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8158101e40 0x7f8158197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60762/0 (socket says 172.21.15.133:60762) 2023-07-14T18:15:46.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.800+0000 7f81577fe700 1 -- 172.21.15.133:0/458191818 learned_addr learned my addr 172.21.15.133:0/458191818 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:46.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.800+0000 7f814ffff700 1 --2- 172.21.15.133:0/458191818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8158102f90 0x7f81581982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:46.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.800+0000 7f8157fff700 1 --2- 172.21.15.133:0/458191818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8158103940 0x7f815819f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:46.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.800+0000 7f81577fe700 1 -- 172.21.15.133:0/458191818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8158102f90 msgr2=0x7f81581982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:46.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.800+0000 7f81577fe700 1 --2- 172.21.15.133:0/458191818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8158102f90 0x7f81581982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:46.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.800+0000 7f81577fe700 1 -- 172.21.15.133:0/458191818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8158103940 msgr2=0x7f815819f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:46.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.800+0000 7f81577fe700 1 --2- 172.21.15.133:0/458191818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8158103940 0x7f815819f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:46.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.800+0000 7f81577fe700 1 -- 172.21.15.133:0/458191818 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81400095f0 con 0x7f8158101e40 2023-07-14T18:15:46.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.801+0000 7f81577fe700 1 --2- 172.21.15.133:0/458191818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8158101e40 0x7f8158197db0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f814800e9a0 tx=0x7f814800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:46.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.801+0000 7f81557fa700 1 -- 172.21.15.133:0/458191818 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8148009790 con 0x7f8158101e40 2023-07-14T18:15:46.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.801+0000 7f81557fa700 1 -- 172.21.15.133:0/458191818 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8148010430 con 0x7f8158101e40 2023-07-14T18:15:46.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.801+0000 7f81557fa700 1 -- 172.21.15.133:0/458191818 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8148018680 con 0x7f8158101e40 2023-07-14T18:15:46.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.801+0000 7f815dbff700 1 -- 172.21.15.133:0/458191818 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f815819fee0 con 0x7f8158101e40 2023-07-14T18:15:46.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.801+0000 7f815dbff700 1 -- 172.21.15.133:0/458191818 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81581a0250 con 0x7f8158101e40 2023-07-14T18:15:46.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.802+0000 7f815dbff700 1 -- 172.21.15.133:0/458191818 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8158109110 con 0x7f8158101e40 2023-07-14T18:15:46.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.804+0000 7f81557fa700 1 -- 172.21.15.133:0/458191818 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f81480105a0 con 0x7f8158101e40 2023-07-14T18:15:46.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.805+0000 7f81557fa700 1 --2- 172.21.15.133:0/458191818 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f81440a11e0 0x7f81440a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:46.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.805+0000 7f81557fa700 1 -- 172.21.15.133:0/458191818 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8148014070 con 0x7f8158101e40 2023-07-14T18:15:46.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.805+0000 7f814ffff700 1 --2- 172.21.15.133:0/458191818 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f81440a11e0 0x7f81440a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:46.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.806+0000 7f814ffff700 1 --2- 172.21.15.133:0/458191818 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f81440a11e0 0x7f81440a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f814000b2e0 tx=0x7f8140005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:46.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:46.809+0000 7f81557fa700 1 -- 172.21.15.133:0/458191818 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f8148051780 con 0x7f8158101e40 2023-07-14T18:15:47.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.010+0000 7f815dbff700 1 -- 172.21.15.133:0/458191818 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f815802cd40 con 0x7f8158101e40 2023-07-14T18:15:47.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.010+0000 7f81557fa700 1 -- 172.21.15.133:0/458191818 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8148096240 con 0x7f8158101e40 2023-07-14T18:15:47.055 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2023-07-14T18:15:47.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.012+0000 7f815dbff700 1 -- 172.21.15.133:0/458191818 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f81440a11e0 msgr2=0x7f81440a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:47.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.012+0000 7f815dbff700 1 --2- 172.21.15.133:0/458191818 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f81440a11e0 0x7f81440a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f814000b2e0 tx=0x7f8140005fb0 comp rx=0 tx=0).stop 2023-07-14T18:15:47.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.012+0000 7f815dbff700 1 -- 172.21.15.133:0/458191818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8158101e40 msgr2=0x7f8158197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:47.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.012+0000 7f815dbff700 1 --2- 172.21.15.133:0/458191818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8158101e40 0x7f8158197db0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f814800e9a0 tx=0x7f814800ecb0 comp rx=0 tx=0).stop 2023-07-14T18:15:47.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.013+0000 7f815dbff700 1 -- 172.21.15.133:0/458191818 shutdown_connections 2023-07-14T18:15:47.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.013+0000 7f815dbff700 1 --2- 172.21.15.133:0/458191818 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f81440a11e0 0x7f81440a3690 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:47.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.013+0000 7f815dbff700 1 --2- 172.21.15.133:0/458191818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8158103940 0x7f815819f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:47.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.013+0000 7f815dbff700 1 --2- 172.21.15.133:0/458191818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8158101e40 0x7f8158197db0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:47.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.013+0000 7f815dbff700 1 --2- 172.21.15.133:0/458191818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8158102f90 0x7f81581982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:47.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.013+0000 7f815dbff700 1 -- 172.21.15.133:0/458191818 >> 172.21.15.133:0/458191818 conn(0x7f81580fd480 msgr2=0x7f8158106250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:47.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.013+0000 7f815dbff700 1 -- 172.21.15.133:0/458191818 shutdown_connections 2023-07-14T18:15:47.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.013+0000 7f815dbff700 1 -- 172.21.15.133:0/458191818 wait complete. 2023-07-14T18:15:47.076 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T18:15:47.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.229+0000 7fd4c21ff700 1 -- 172.21.15.133:0/1435924512 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4bc100e10 msgr2=0x7fd4bc101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:47.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.229+0000 7fd4c21ff700 1 --2- 172.21.15.133:0/1435924512 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4bc100e10 0x7fd4bc101260 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0009990 tx=0x7fd4b0009ca0 comp rx=0 tx=0).stop 2023-07-14T18:15:47.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.230+0000 7fd4c21ff700 1 -- 172.21.15.133:0/1435924512 shutdown_connections 2023-07-14T18:15:47.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.230+0000 7fd4c21ff700 1 --2- 172.21.15.133:0/1435924512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4bc0ffc20 0x7fd4bc100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:47.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.230+0000 7fd4c21ff700 1 --2- 172.21.15.133:0/1435924512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4bc1017a0 0x7fd4bc108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:47.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.230+0000 7fd4c21ff700 1 --2- 172.21.15.133:0/1435924512 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4bc100e10 0x7fd4bc101260 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:47.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.230+0000 7fd4c21ff700 1 -- 172.21.15.133:0/1435924512 >> 172.21.15.133:0/1435924512 conn(0x7fd4bc0fb260 msgr2=0x7fd4bc0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:47.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.230+0000 7fd4c21ff700 1 -- 172.21.15.133:0/1435924512 shutdown_connections 2023-07-14T18:15:47.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.230+0000 7fd4c21ff700 1 -- 172.21.15.133:0/1435924512 wait complete. 2023-07-14T18:15:47.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.231+0000 7fd4c21ff700 1 Processor -- start 2023-07-14T18:15:47.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.231+0000 7fd4c21ff700 1 -- start start 2023-07-14T18:15:47.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.231+0000 7fd4c21ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4bc0ffc20 0x7fd4bc0743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:47.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.231+0000 7fd4c21ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4bc100e10 0x7fd4bc072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:47.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.231+0000 7fd4c21ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4bc1017a0 0x7fd4bc072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:47.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.231+0000 7fd4c21ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc073710 con 0x7fd4bc100e10 2023-07-14T18:15:47.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.231+0000 7fd4c21ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc073880 con 0x7fd4bc0ffc20 2023-07-14T18:15:47.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.231+0000 7fd4c21ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc19f900 con 0x7fd4bc1017a0 2023-07-14T18:15:47.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.231+0000 7fd4baffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4bc100e10 0x7fd4bc072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:47.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.231+0000 7fd4bb7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4bc0ffc20 0x7fd4bc0743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:47.279 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.231+0000 7fd4baffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4bc100e10 0x7fd4bc072a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60768/0 (socket says 172.21.15.133:60768) 2023-07-14T18:15:47.279 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.231+0000 7fd4baffd700 1 -- 172.21.15.133:0/436854071 learned_addr learned my addr 172.21.15.133:0/436854071 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:47.279 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.231+0000 7fd4bbfff700 1 --2- 172.21.15.133:0/436854071 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4bc1017a0 0x7fd4bc072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:47.279 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.232+0000 7fd4baffd700 1 -- 172.21.15.133:0/436854071 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4bc1017a0 msgr2=0x7fd4bc072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:47.279 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.232+0000 7fd4baffd700 1 --2- 172.21.15.133:0/436854071 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4bc1017a0 0x7fd4bc072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:47.279 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.232+0000 7fd4baffd700 1 -- 172.21.15.133:0/436854071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4bc0ffc20 msgr2=0x7fd4bc0743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:47.279 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.232+0000 7fd4baffd700 1 --2- 172.21.15.133:0/436854071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4bc0ffc20 0x7fd4bc0743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:47.279 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.232+0000 7fd4baffd700 1 -- 172.21.15.133:0/436854071 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b00095f0 con 0x7fd4bc100e10 2023-07-14T18:15:47.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.232+0000 7fd4bb7fe700 1 --2- 172.21.15.133:0/436854071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4bc0ffc20 0x7fd4bc0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:47.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.232+0000 7fd4baffd700 1 --2- 172.21.15.133:0/436854071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4bc100e10 0x7fd4bc072a30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000b680 tx=0x7fd4b0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:47.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.232+0000 7fd4b8ff9700 1 -- 172.21.15.133:0/436854071 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b001d070 con 0x7fd4bc100e10 2023-07-14T18:15:47.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.232+0000 7fd4b8ff9700 1 -- 172.21.15.133:0/436854071 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4b0022470 con 0x7fd4bc100e10 2023-07-14T18:15:47.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.232+0000 7fd4c21ff700 1 -- 172.21.15.133:0/436854071 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4bc19fb80 con 0x7fd4bc100e10 2023-07-14T18:15:47.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.232+0000 7fd4b8ff9700 1 -- 172.21.15.133:0/436854071 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0017650 con 0x7fd4bc100e10 2023-07-14T18:15:47.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.232+0000 7fd4c21ff700 1 -- 172.21.15.133:0/436854071 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4bc1a0050 con 0x7fd4bc100e10 2023-07-14T18:15:47.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.233+0000 7fd4c21ff700 1 -- 172.21.15.133:0/436854071 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4bc10b3e0 con 0x7fd4bc100e10 2023-07-14T18:15:47.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.235+0000 7fd4b8ff9700 1 -- 172.21.15.133:0/436854071 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fd4b00225e0 con 0x7fd4bc100e10 2023-07-14T18:15:47.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.237+0000 7fd4b8ff9700 1 --2- 172.21.15.133:0/436854071 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd4a40a1260 0x7fd4a40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:47.282 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.237+0000 7fd4b8ff9700 1 -- 172.21.15.133:0/436854071 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd4b00ce8b0 con 0x7fd4bc100e10 2023-07-14T18:15:47.282 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.237+0000 7fd4bb7fe700 1 --2- 172.21.15.133:0/436854071 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd4a40a1260 0x7fd4a40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:47.282 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.237+0000 7fd4bb7fe700 1 --2- 172.21.15.133:0/436854071 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd4a40a1260 0x7fd4a40a3710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc100c70 tx=0x7fd4ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:47.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.239+0000 7fd4b8ff9700 1 -- 172.21.15.133:0/436854071 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fd4b0097c40 con 0x7fd4bc100e10 2023-07-14T18:15:47.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:46 smithi161 ceph-mon[128993]: pgmap v319: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T18:15:47.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:46 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/4262468523' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:15:47.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:46 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3588477388' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:15:47.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:46 smithi171 ceph-mon[129241]: pgmap v319: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T18:15:47.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:46 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/4262468523' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:15:47.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:46 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3588477388' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:15:47.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:46 smithi133 ceph-mon[126384]: pgmap v319: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T18:15:47.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:46 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/4262468523' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:15:47.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:46 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3588477388' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:15:47.484 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.439+0000 7fd4c21ff700 1 -- 172.21.15.133:0/436854071 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd4bc106ad0 con 0x7fd4bc100e10 2023-07-14T18:15:47.484 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.439+0000 7fd4b8ff9700 1 -- 172.21.15.133:0/436854071 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd4b0027020 con 0x7fd4bc100e10 2023-07-14T18:15:47.485 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-07-14T18:15:47.487 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.442+0000 7fd4c21ff700 1 -- 172.21.15.133:0/436854071 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd4a40a1260 msgr2=0x7fd4a40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:47.487 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.442+0000 7fd4c21ff700 1 --2- 172.21.15.133:0/436854071 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd4a40a1260 0x7fd4a40a3710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc100c70 tx=0x7fd4ac009260 comp rx=0 tx=0).stop 2023-07-14T18:15:47.487 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.442+0000 7fd4c21ff700 1 -- 172.21.15.133:0/436854071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4bc100e10 msgr2=0x7fd4bc072a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:47.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.442+0000 7fd4c21ff700 1 --2- 172.21.15.133:0/436854071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4bc100e10 0x7fd4bc072a30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000b680 tx=0x7fd4b0005740 comp rx=0 tx=0).stop 2023-07-14T18:15:47.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.442+0000 7fd4c21ff700 1 -- 172.21.15.133:0/436854071 shutdown_connections 2023-07-14T18:15:47.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.442+0000 7fd4c21ff700 1 --2- 172.21.15.133:0/436854071 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd4a40a1260 0x7fd4a40a3710 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:47.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.442+0000 7fd4c21ff700 1 --2- 172.21.15.133:0/436854071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4bc0ffc20 0x7fd4bc0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:47.489 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.442+0000 7fd4c21ff700 1 --2- 172.21.15.133:0/436854071 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4bc1017a0 0x7fd4bc072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:47.489 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.442+0000 7fd4c21ff700 1 --2- 172.21.15.133:0/436854071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4bc100e10 0x7fd4bc072a30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:47.489 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.442+0000 7fd4c21ff700 1 -- 172.21.15.133:0/436854071 >> 172.21.15.133:0/436854071 conn(0x7fd4bc0fb260 msgr2=0x7fd4bc0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:47.489 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.442+0000 7fd4c21ff700 1 -- 172.21.15.133:0/436854071 shutdown_connections 2023-07-14T18:15:47.490 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.442+0000 7fd4c21ff700 1 -- 172.21.15.133:0/436854071 wait complete. 2023-07-14T18:15:47.506 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T18:15:47.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.659+0000 7f3e371ff700 1 -- 172.21.15.133:0/2332546470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e30100e20 msgr2=0x7f3e30101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:47.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.659+0000 7f3e371ff700 1 --2- 172.21.15.133:0/2332546470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e30100e20 0x7f3e30101270 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009990 tx=0x7f3e1c009ca0 comp rx=0 tx=0).stop 2023-07-14T18:15:47.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.659+0000 7f3e371ff700 1 -- 172.21.15.133:0/2332546470 shutdown_connections 2023-07-14T18:15:47.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.659+0000 7f3e371ff700 1 --2- 172.21.15.133:0/2332546470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e301017b0 0x7f3e30108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:47.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.659+0000 7f3e371ff700 1 --2- 172.21.15.133:0/2332546470 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e300ffc30 0x7f3e30100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:47.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.659+0000 7f3e371ff700 1 --2- 172.21.15.133:0/2332546470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e30100e20 0x7f3e30101270 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:47.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.659+0000 7f3e371ff700 1 -- 172.21.15.133:0/2332546470 >> 172.21.15.133:0/2332546470 conn(0x7f3e300fb210 msgr2=0x7f3e300fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:47.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.659+0000 7f3e371ff700 1 -- 172.21.15.133:0/2332546470 shutdown_connections 2023-07-14T18:15:47.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.659+0000 7f3e371ff700 1 -- 172.21.15.133:0/2332546470 wait complete. 2023-07-14T18:15:47.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.660+0000 7f3e371ff700 1 Processor -- start 2023-07-14T18:15:47.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.660+0000 7f3e371ff700 1 -- start start 2023-07-14T18:15:47.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.661+0000 7f3e371ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e300ffc30 0x7f3e300743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:47.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.661+0000 7f3e371ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e30100e20 0x7f3e30072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:47.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.661+0000 7f3e371ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e301017b0 0x7f3e30072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:47.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.661+0000 7f3e371ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e30073710 con 0x7f3e300ffc30 2023-07-14T18:15:47.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.661+0000 7f3e371ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e30073880 con 0x7f3e301017b0 2023-07-14T18:15:47.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.661+0000 7f3e371ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3019f960 con 0x7f3e30100e20 2023-07-14T18:15:47.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.661+0000 7f3e34f9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e300ffc30 0x7f3e300743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:47.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.661+0000 7f3e34f9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e300ffc30 0x7f3e300743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60782/0 (socket says 172.21.15.133:60782) 2023-07-14T18:15:47.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.661+0000 7f3e34f9b700 1 -- 172.21.15.133:0/1915282211 learned_addr learned my addr 172.21.15.133:0/1915282211 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:47.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.661+0000 7f3e2ffff700 1 --2- 172.21.15.133:0/1915282211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e30100e20 0x7f3e30072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:47.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.661+0000 7f3e3579c700 1 --2- 172.21.15.133:0/1915282211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e301017b0 0x7f3e30072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:47.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.661+0000 7f3e34f9b700 1 -- 172.21.15.133:0/1915282211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e30100e20 msgr2=0x7f3e30072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:47.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.661+0000 7f3e34f9b700 1 --2- 172.21.15.133:0/1915282211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e30100e20 0x7f3e30072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:47.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.662+0000 7f3e34f9b700 1 -- 172.21.15.133:0/1915282211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e301017b0 msgr2=0x7f3e30072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:47.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.662+0000 7f3e34f9b700 1 --2- 172.21.15.133:0/1915282211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e301017b0 0x7f3e30072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:47.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.662+0000 7f3e34f9b700 1 -- 172.21.15.133:0/1915282211 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e1c0095f0 con 0x7f3e300ffc30 2023-07-14T18:15:47.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.662+0000 7f3e2ffff700 1 --2- 172.21.15.133:0/1915282211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e30100e20 0x7f3e30072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:47.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.662+0000 7f3e3579c700 1 --2- 172.21.15.133:0/1915282211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e301017b0 0x7f3e30072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:15:47.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.662+0000 7f3e34f9b700 1 --2- 172.21.15.133:0/1915282211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e300ffc30 0x7f3e300743f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3e2400d980 tx=0x7f3e2400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:47.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.662+0000 7f3e2dffb700 1 -- 172.21.15.133:0/1915282211 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2400b950 con 0x7f3e300ffc30 2023-07-14T18:15:47.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.662+0000 7f3e2dffb700 1 -- 172.21.15.133:0/1915282211 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e24007620 con 0x7f3e300ffc30 2023-07-14T18:15:47.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.662+0000 7f3e371ff700 1 -- 172.21.15.133:0/1915282211 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e3019fc40 con 0x7f3e300ffc30 2023-07-14T18:15:47.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.662+0000 7f3e2dffb700 1 -- 172.21.15.133:0/1915282211 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2400f3c0 con 0x7f3e300ffc30 2023-07-14T18:15:47.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.662+0000 7f3e371ff700 1 -- 172.21.15.133:0/1915282211 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e301a0150 con 0x7f3e300ffc30 2023-07-14T18:15:47.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.663+0000 7f3e371ff700 1 -- 172.21.15.133:0/1915282211 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e30107150 con 0x7f3e300ffc30 2023-07-14T18:15:47.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.665+0000 7f3e2dffb700 1 -- 172.21.15.133:0/1915282211 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f3e24010640 con 0x7f3e300ffc30 2023-07-14T18:15:47.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.666+0000 7f3e2dffb700 1 --2- 172.21.15.133:0/1915282211 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3e200a1220 0x7f3e200a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:47.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.666+0000 7f3e2dffb700 1 -- 172.21.15.133:0/1915282211 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3e240cccf0 con 0x7f3e300ffc30 2023-07-14T18:15:47.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.667+0000 7f3e2ffff700 1 --2- 172.21.15.133:0/1915282211 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3e200a1220 0x7f3e200a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:47.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.667+0000 7f3e2ffff700 1 --2- 172.21.15.133:0/1915282211 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3e200a1220 0x7f3e200a36d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c00b2e0 tx=0x7f3e1c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:47.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.670+0000 7f3e2dffb700 1 -- 172.21.15.133:0/1915282211 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f3e24095f70 con 0x7f3e300ffc30 2023-07-14T18:15:47.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.912+0000 7f3e371ff700 1 -- 172.21.15.133:0/1915282211 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e3002c8e0 con 0x7f3e300ffc30 2023-07-14T18:15:47.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.913+0000 7f3e2dffb700 1 -- 172.21.15.133:0/1915282211 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10851 (secure 0 0 0) 0x7f3e24099590 con 0x7f3e300ffc30 2023-07-14T18:15:47.958 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:15:47.958 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3168151271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3168151271},{"type":"v1","addr":"172.21.15.133:6837","nonce":3168151271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14544,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/3093144261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3093144261},{"type":"v1","addr":"172.21.15.133:6839","nonce":3093144261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14556,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/3282902068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":3282902068},{"type":"v1","addr":"172.21.15.133:6841","nonce":3282902068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/4023127259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4023127259},{"type":"v1","addr":"172.21.15.161:6833","nonce":4023127259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2722373740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2722373740},{"type":"v1","addr":"172.21.15.171:6833","nonce":2722373740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2399005074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2399005074},{"type":"v1","addr":"172.21.15.133:6835","nonce":2399005074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T18:14:17.440873+0000","modified":"2023-07-14T18:14:50.886364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24557,"mds_1":24541,"mds_2":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3725502564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3725502564},{"type":"v1","addr":"172.21.15.161:6835","nonce":3725502564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6835/1877362116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1877362116},{"type":"v1","addr":"172.21.15.171:6835","nonce":1877362116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/3458064798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3458064798},{"type":"v1","addr":"172.21.15.161:6837","nonce":3458064798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/106205332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":106205332},{"type":"v1","addr":"172.21.15.171:6837","nonce":106205332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.171:6839/816201198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":816201198},{"type":"v1","addr":"172.21.15.171:6839","nonce":816201198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/2503865871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2503865871},{"type":"v1","addr":"172.21.15.161:6839","nonce":2503865871}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T18:15:47.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.915+0000 7f3e371ff700 1 -- 172.21.15.133:0/1915282211 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3e200a1220 msgr2=0x7f3e200a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:47.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.915+0000 7f3e371ff700 1 --2- 172.21.15.133:0/1915282211 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3e200a1220 0x7f3e200a36d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c00b2e0 tx=0x7f3e1c005fb0 comp rx=0 tx=0).stop 2023-07-14T18:15:47.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.915+0000 7f3e371ff700 1 -- 172.21.15.133:0/1915282211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e300ffc30 msgr2=0x7f3e300743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:47.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.915+0000 7f3e371ff700 1 --2- 172.21.15.133:0/1915282211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e300ffc30 0x7f3e300743f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3e2400d980 tx=0x7f3e2400dc90 comp rx=0 tx=0).stop 2023-07-14T18:15:47.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.915+0000 7f3e371ff700 1 -- 172.21.15.133:0/1915282211 shutdown_connections 2023-07-14T18:15:47.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.915+0000 7f3e371ff700 1 --2- 172.21.15.133:0/1915282211 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3e200a1220 0x7f3e200a36d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:47.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.915+0000 7f3e371ff700 1 --2- 172.21.15.133:0/1915282211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e301017b0 0x7f3e30072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:47.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.915+0000 7f3e371ff700 1 --2- 172.21.15.133:0/1915282211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e300ffc30 0x7f3e300743f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:47.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.915+0000 7f3e371ff700 1 --2- 172.21.15.133:0/1915282211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e30100e20 0x7f3e30072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:47.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.915+0000 7f3e371ff700 1 -- 172.21.15.133:0/1915282211 >> 172.21.15.133:0/1915282211 conn(0x7f3e300fb210 msgr2=0x7f3e30104290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:47.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.915+0000 7f3e371ff700 1 -- 172.21.15.133:0/1915282211 shutdown_connections 2023-07-14T18:15:47.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:47.916+0000 7f3e371ff700 1 -- 172.21.15.133:0/1915282211 wait complete. 2023-07-14T18:15:47.962 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 25 2023-07-14T18:15:47.979 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T18:15:48.175 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.130+0000 7f77c8dff700 1 -- 172.21.15.133:0/2244519607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77c40ffc30 msgr2=0x7f77c4100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:48.175 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.130+0000 7f77c8dff700 1 --2- 172.21.15.133:0/2244519607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77c40ffc30 0x7f77c4100040 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009990 tx=0x7f77b4009ca0 comp rx=0 tx=0).stop 2023-07-14T18:15:48.175 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.130+0000 7f77c8dff700 1 -- 172.21.15.133:0/2244519607 shutdown_connections 2023-07-14T18:15:48.175 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.130+0000 7f77c8dff700 1 --2- 172.21.15.133:0/2244519607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77c41017b0 0x7f77c4108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:48.175 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.130+0000 7f77c8dff700 1 --2- 172.21.15.133:0/2244519607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77c40ffc30 0x7f77c4100040 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:48.176 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.130+0000 7f77c8dff700 1 --2- 172.21.15.133:0/2244519607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77c4100e20 0x7f77c4101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:48.176 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.130+0000 7f77c8dff700 1 -- 172.21.15.133:0/2244519607 >> 172.21.15.133:0/2244519607 conn(0x7f77c40fb210 msgr2=0x7f77c40fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:48.176 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.130+0000 7f77c8dff700 1 -- 172.21.15.133:0/2244519607 shutdown_connections 2023-07-14T18:15:48.176 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.130+0000 7f77c8dff700 1 -- 172.21.15.133:0/2244519607 wait complete. 2023-07-14T18:15:48.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.131+0000 7f77c8dff700 1 Processor -- start 2023-07-14T18:15:48.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.131+0000 7f77c8dff700 1 -- start start 2023-07-14T18:15:48.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.132+0000 7f77c8dff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77c40ffc30 0x7f77c4197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:48.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.132+0000 7f77c8dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77c4100e20 0x7f77c4198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:48.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.132+0000 7f77c8dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77c41017b0 0x7f77c419f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:48.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.132+0000 7f77c8dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c419f690 con 0x7f77c41017b0 2023-07-14T18:15:48.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.132+0000 7f77c8dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c419f7d0 con 0x7f77c4100e20 2023-07-14T18:15:48.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.132+0000 7f77c8dff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c419f960 con 0x7f77c40ffc30 2023-07-14T18:15:48.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.132+0000 7f77c2d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77c41017b0 0x7f77c419f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:48.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.132+0000 7f77c2d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77c41017b0 0x7f77c419f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60810/0 (socket says 172.21.15.133:60810) 2023-07-14T18:15:48.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.132+0000 7f77c2d9d700 1 -- 172.21.15.133:0/3191642871 learned_addr learned my addr 172.21.15.133:0/3191642871 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:48.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.132+0000 7f77c1d9b700 1 --2- 172.21.15.133:0/3191642871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77c4100e20 0x7f77c4198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:48.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.132+0000 7f77c259c700 1 --2- 172.21.15.133:0/3191642871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77c40ffc30 0x7f77c4197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:48.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.132+0000 7f77c2d9d700 1 -- 172.21.15.133:0/3191642871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77c40ffc30 msgr2=0x7f77c4197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:48.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.132+0000 7f77c2d9d700 1 --2- 172.21.15.133:0/3191642871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77c40ffc30 0x7f77c4197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:48.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.132+0000 7f77c2d9d700 1 -- 172.21.15.133:0/3191642871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77c4100e20 msgr2=0x7f77c4198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:48.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.132+0000 7f77c2d9d700 1 --2- 172.21.15.133:0/3191642871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77c4100e20 0x7f77c4198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:48.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.132+0000 7f77c2d9d700 1 -- 172.21.15.133:0/3191642871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b40095f0 con 0x7f77c41017b0 2023-07-14T18:15:48.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.132+0000 7f77c2d9d700 1 --2- 172.21.15.133:0/3191642871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77c41017b0 0x7f77c419f0c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f77bc00ef30 tx=0x7f77bc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:48.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.133+0000 7f77b37fe700 1 -- 172.21.15.133:0/3191642871 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc018070 con 0x7f77c41017b0 2023-07-14T18:15:48.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.133+0000 7f77c8dff700 1 -- 172.21.15.133:0/3191642871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77c419fbe0 con 0x7f77c41017b0 2023-07-14T18:15:48.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.133+0000 7f77b37fe700 1 -- 172.21.15.133:0/3191642871 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77bc00f040 con 0x7f77c41017b0 2023-07-14T18:15:48.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.133+0000 7f77b37fe700 1 -- 172.21.15.133:0/3191642871 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc00cd00 con 0x7f77c41017b0 2023-07-14T18:15:48.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.133+0000 7f77c8dff700 1 -- 172.21.15.133:0/3191642871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77c41a00a0 con 0x7f77c41017b0 2023-07-14T18:15:48.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.134+0000 7f77c8dff700 1 -- 172.21.15.133:0/3191642871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77c41922c0 con 0x7f77c41017b0 2023-07-14T18:15:48.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.136+0000 7f77b37fe700 1 -- 172.21.15.133:0/3191642871 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f77bc003a00 con 0x7f77c41017b0 2023-07-14T18:15:48.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.137+0000 7f77b37fe700 1 --2- 172.21.15.133:0/3191642871 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f77ac0a1220 0x7f77ac0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:48.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.137+0000 7f77b37fe700 1 -- 172.21.15.133:0/3191642871 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f77bc0cc960 con 0x7f77c41017b0 2023-07-14T18:15:48.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.137+0000 7f77c259c700 1 --2- 172.21.15.133:0/3191642871 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f77ac0a1220 0x7f77ac0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:48.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.138+0000 7f77c259c700 1 --2- 172.21.15.133:0/3191642871 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f77ac0a1220 0x7f77ac0a36d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f77b400b2e0 tx=0x7f77b4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:48.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.141+0000 7f77b37fe700 1 -- 172.21.15.133:0/3191642871 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f77bc095c70 con 0x7f77c41017b0 2023-07-14T18:15:48.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:47 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/458191818' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:15:48.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:47 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/436854071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:15:48.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:47 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1915282211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:15:48.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:47 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/458191818' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:15:48.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:47 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/436854071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:15:48.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:47 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1915282211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:15:48.384 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.339+0000 7f77c8dff700 1 -- 172.21.15.133:0/3191642871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77c402c8e0 con 0x7f77c41017b0 2023-07-14T18:15:48.385 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.341+0000 7f77b37fe700 1 -- 172.21.15.133:0/3191642871 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24473 (secure 0 0 0) 0x7f77bc099290 con 0x7f77c41017b0 2023-07-14T18:15:48.386 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:15:48.386 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":90,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","created":"2023-07-14T18:06:49.561582+0000","modified":"2023-07-14T18:14:48.934538+0000","last_up_change":"2023-07-14T18:13:29.836099+0000","last_in_change":"2023-07-14T18:13:11.606710+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T18:10:03.945039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T18:14:15.451340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T18:14:16.892215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T18:14:20.493175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"85a29de7-d712-422a-b647-d9e90091aaf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6803","nonce":3076510801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6805","nonce":3076510801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6809","nonce":3076510801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6807","nonce":3076510801}]},"public_addr":"172.21.15.133:6803/3076510801","cluster_addr":"172.21.15.133:6805/3076510801","heartbeat_back_addr":"172.21.15.133:6809/3076510801","heartbeat_front_addr":"172.21.15.133:6807/3076510801","state":["exists","up"]},{"osd":1,"uuid":"80ad4e78-7917-495e-b119-99f7c7c36243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6801","nonce":1841901852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6803","nonce":1841901852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6807","nonce":1841901852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6805","nonce":1841901852}]},"public_addr":"172.21.15.161:6801/1841901852","cluster_addr":"172.21.15.161:6803/1841901852","heartbeat_back_addr":"172.21.15.161:6807/1841901852","heartbeat_front_addr":"172.21.15.161:6805/1841901852","state":["exists","up"]},{"osd":2,"uuid":"fc4bfd37-1fdd-47c6-a48e-6d910daf9e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6801","nonce":346103099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6803","nonce":346103099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6807","nonce":346103099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6805","nonce":346103099}]},"public_addr":"172.21.15.171:6801/346103099","cluster_addr":"172.21.15.171:6803/346103099","heartbeat_back_addr":"172.21.15.171:6807/346103099","heartbeat_front_addr":"172.21.15.171:6805/346103099","state":["exists","up"]},{"osd":3,"uuid":"4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6811","nonce":2152994143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6813","nonce":2152994143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6817","nonce":2152994143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6815","nonce":2152994143}]},"public_addr":"172.21.15.133:6811/2152994143","cluster_addr":"172.21.15.133:6813/2152994143","heartbeat_back_addr":"172.21.15.133:6817/2152994143","heartbeat_front_addr":"172.21.15.133:6815/2152994143","state":["exists","up"]},{"osd":4,"uuid":"4601a1c2-254b-4c4d-9b15-8fc952b3d2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6809","nonce":47213291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6811","nonce":47213291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6815","nonce":47213291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6813","nonce":47213291}]},"public_addr":"172.21.15.161:6809/47213291","cluster_addr":"172.21.15.161:6811/47213291","heartbeat_back_addr":"172.21.15.161:6815/47213291","heartbeat_front_addr":"172.21.15.161:6813/47213291","state":["exists","up"]},{"osd":5,"uuid":"6cca180a-9140-4e49-835b-86842dae0574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6809","nonce":777556751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6811","nonce":777556751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6815","nonce":777556751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6813","nonce":777556751}]},"public_addr":"172.21.15.171:6809/777556751","cluster_addr":"172.21.15.171:6811/777556751","heartbeat_back_addr":"172.21.15.171:6815/777556751","heartbeat_front_addr":"172.21.15.171:6813/777556751","state":["exists","up"]},{"osd":6,"uuid":"ea0299ff-7740-4494-a3a9-1503b3fc0ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6819","nonce":2884152358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6821","nonce":2884152358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6825","nonce":2884152358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6823","nonce":2884152358}]},"public_addr":"172.21.15.133:6819/2884152358","cluster_addr":"172.21.15.133:6821/2884152358","heartbeat_back_addr":"172.21.15.133:6825/2884152358","heartbeat_front_addr":"172.21.15.133:6823/2884152358","state":["exists","up"]},{"osd":7,"uuid":"b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6817","nonce":4183705749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6819","nonce":4183705749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6823","nonce":4183705749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6821","nonce":4183705749}]},"public_addr":"172.21.15.161:6817/4183705749","cluster_addr":"172.21.15.161:6819/4183705749","heartbeat_back_addr":"172.21.15.161:6823/4183705749","heartbeat_front_addr":"172.21.15.161:6821/4183705749","state":["exists","up"]},{"osd":8,"uuid":"87dffc07-bf48-48c8-b418-5c4b509686d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6817","nonce":2771494483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6819","nonce":2771494483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6823","nonce":2771494483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6821","nonce":2771494483}]},"public_addr":"172.21.15.171:6817/2771494483","cluster_addr":"172.21.15.171:6819/2771494483","heartbeat_back_addr":"172.21.15.171:6823/2771494483","heartbeat_front_addr":"172.21.15.171:6821/2771494483","state":["exists","up"]},{"osd":9,"uuid":"e0832870-79ce-4bce-8045-58b7a7f6dc69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6827","nonce":2066164149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6829","nonce":2066164149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6833","nonce":2066164149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6831","nonce":2066164149}]},"public_addr":"172.21.15.133:6827/2066164149","cluster_addr":"172.21.15.133:6829/2066164149","heartbeat_back_addr":"172.21.15.133:6833/2066164149","heartbeat_front_addr":"172.21.15.133:6831/2066164149","state":["exists","up"]},{"osd":10,"uuid":"081f0484-d134-4b2e-9c48-ef8e6a646422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6825","nonce":2162333626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6827","nonce":2162333626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6831","nonce":2162333626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6829","nonce":2162333626}]},"public_addr":"172.21.15.161:6825/2162333626","cluster_addr":"172.21.15.161:6827/2162333626","heartbeat_back_addr":"172.21.15.161:6831/2162333626","heartbeat_front_addr":"172.21.15.161:6829/2162333626","state":["exists","up"]},{"osd":11,"uuid":"4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6825","nonce":3403894313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6827","nonce":3403894313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6831","nonce":3403894313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6829","nonce":3403894313}]},"public_addr":"172.21.15.171:6825/3403894313","cluster_addr":"172.21.15.171:6827/3403894313","heartbeat_back_addr":"172.21.15.171:6831/3403894313","heartbeat_front_addr":"172.21.15.171:6829/3403894313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:17.816229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:39.638543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:00.543317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:26.988921+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:48.644694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:10.049860+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:35.986508+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:57.753846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:19.018245+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:44.460450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:06.279960+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:27.955006+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/1832569241":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6801/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:0/4219815304":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:6801/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/432253615":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1840700480":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1807775439":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1089030734":"2023-07-15T18:07:47.346704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T18:15:48.388 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.344+0000 7f77c8dff700 1 -- 172.21.15.133:0/3191642871 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f77ac0a1220 msgr2=0x7f77ac0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:48.388 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.344+0000 7f77c8dff700 1 --2- 172.21.15.133:0/3191642871 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f77ac0a1220 0x7f77ac0a36d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f77b400b2e0 tx=0x7f77b4005960 comp rx=0 tx=0).stop 2023-07-14T18:15:48.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.344+0000 7f77c8dff700 1 -- 172.21.15.133:0/3191642871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77c41017b0 msgr2=0x7f77c419f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:48.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.344+0000 7f77c8dff700 1 --2- 172.21.15.133:0/3191642871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77c41017b0 0x7f77c419f0c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f77bc00ef30 tx=0x7f77bc00c3a0 comp rx=0 tx=0).stop 2023-07-14T18:15:48.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.344+0000 7f77c8dff700 1 -- 172.21.15.133:0/3191642871 shutdown_connections 2023-07-14T18:15:48.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.344+0000 7f77c8dff700 1 --2- 172.21.15.133:0/3191642871 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f77ac0a1220 0x7f77ac0a36d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:48.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.344+0000 7f77c8dff700 1 --2- 172.21.15.133:0/3191642871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77c41017b0 0x7f77c419f0c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:48.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.344+0000 7f77c8dff700 1 --2- 172.21.15.133:0/3191642871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77c40ffc30 0x7f77c4197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:48.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.344+0000 7f77c8dff700 1 --2- 172.21.15.133:0/3191642871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77c4100e20 0x7f77c4198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:48.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.344+0000 7f77c8dff700 1 -- 172.21.15.133:0/3191642871 >> 172.21.15.133:0/3191642871 conn(0x7f77c40fb210 msgr2=0x7f77c4106700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:48.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.344+0000 7f77c8dff700 1 -- 172.21.15.133:0/3191642871 shutdown_connections 2023-07-14T18:15:48.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.344+0000 7f77c8dff700 1 -- 172.21.15.133:0/3191642871 wait complete. 2023-07-14T18:15:48.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:47 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/458191818' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:15:48.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:47 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/436854071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:15:48.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:47 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1915282211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:15:48.408 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T18:10:03.945039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T18:14:15.451340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T18:14:16.892215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T18:14:20.493175+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T18:15:48.409 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T18:15:48.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.565+0000 7f84e55ff700 1 -- 172.21.15.133:0/3093464456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84e00ffbd0 msgr2=0x7f84e00fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:48.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.565+0000 7f84e55ff700 1 --2- 172.21.15.133:0/3093464456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84e00ffbd0 0x7f84e00fffe0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f84d0009990 tx=0x7f84d0009ca0 comp rx=0 tx=0).stop 2023-07-14T18:15:48.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.565+0000 7f84e55ff700 1 -- 172.21.15.133:0/3093464456 shutdown_connections 2023-07-14T18:15:48.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.565+0000 7f84e55ff700 1 --2- 172.21.15.133:0/3093464456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84e0101810 0x7f84e01080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:48.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.565+0000 7f84e55ff700 1 --2- 172.21.15.133:0/3093464456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84e00ffbd0 0x7f84e00fffe0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:48.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.565+0000 7f84e55ff700 1 --2- 172.21.15.133:0/3093464456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84e0100e60 0x7f84e01012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:48.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.565+0000 7f84e55ff700 1 -- 172.21.15.133:0/3093464456 >> 172.21.15.133:0/3093464456 conn(0x7f84e00fb1f0 msgr2=0x7f84e00fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:48.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.565+0000 7f84e55ff700 1 -- 172.21.15.133:0/3093464456 shutdown_connections 2023-07-14T18:15:48.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.565+0000 7f84e55ff700 1 -- 172.21.15.133:0/3093464456 wait complete. 2023-07-14T18:15:48.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.566+0000 7f84e55ff700 1 Processor -- start 2023-07-14T18:15:48.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.567+0000 7f84e55ff700 1 -- start start 2023-07-14T18:15:48.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.567+0000 7f84e55ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84e00ffbd0 0x7f84e0197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:48.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.567+0000 7f84e55ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84e0100e60 0x7f84e0198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:48.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.567+0000 7f84e55ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84e0101810 0x7f84e019f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:48.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.567+0000 7f84e55ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e019f6a0 con 0x7f84e00ffbd0 2023-07-14T18:15:48.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.567+0000 7f84e55ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e019f7e0 con 0x7f84e0100e60 2023-07-14T18:15:48.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.567+0000 7f84e55ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e019f970 con 0x7f84e0101810 2023-07-14T18:15:48.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.567+0000 7f84deffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84e00ffbd0 0x7f84e0197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:48.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.567+0000 7f84deffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84e00ffbd0 0x7f84e0197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60828/0 (socket says 172.21.15.133:60828) 2023-07-14T18:15:48.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.567+0000 7f84deffd700 1 -- 172.21.15.133:0/1520305092 learned_addr learned my addr 172.21.15.133:0/1520305092 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:48.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.567+0000 7f84de7fc700 1 --2- 172.21.15.133:0/1520305092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84e0100e60 0x7f84e0198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:48.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.567+0000 7f84df7fe700 1 --2- 172.21.15.133:0/1520305092 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84e0101810 0x7f84e019f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:48.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.568+0000 7f84deffd700 1 -- 172.21.15.133:0/1520305092 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84e0101810 msgr2=0x7f84e019f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:48.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.568+0000 7f84deffd700 1 --2- 172.21.15.133:0/1520305092 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84e0101810 0x7f84e019f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:48.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.568+0000 7f84deffd700 1 -- 172.21.15.133:0/1520305092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84e0100e60 msgr2=0x7f84e0198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:48.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.568+0000 7f84deffd700 1 --2- 172.21.15.133:0/1520305092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84e0100e60 0x7f84e0198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:48.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.568+0000 7f84deffd700 1 -- 172.21.15.133:0/1520305092 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d00095f0 con 0x7f84e00ffbd0 2023-07-14T18:15:48.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.568+0000 7f84deffd700 1 --2- 172.21.15.133:0/1520305092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84e00ffbd0 0x7f84e0197b20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f84d0000c00 tx=0x7f84d000f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:48.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.568+0000 7f84c7fff700 1 -- 172.21.15.133:0/1520305092 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d001d070 con 0x7f84e00ffbd0 2023-07-14T18:15:48.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.568+0000 7f84e55ff700 1 -- 172.21.15.133:0/1520305092 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e019fbf0 con 0x7f84e00ffbd0 2023-07-14T18:15:48.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.568+0000 7f84c7fff700 1 -- 172.21.15.133:0/1520305092 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84d000fc00 con 0x7f84e00ffbd0 2023-07-14T18:15:48.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.568+0000 7f84c7fff700 1 -- 172.21.15.133:0/1520305092 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d0017650 con 0x7f84e00ffbd0 2023-07-14T18:15:48.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.568+0000 7f84e55ff700 1 -- 172.21.15.133:0/1520305092 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e01a00a0 con 0x7f84e00ffbd0 2023-07-14T18:15:48.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.569+0000 7f84e55ff700 1 -- 172.21.15.133:0/1520305092 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e010b3a0 con 0x7f84e00ffbd0 2023-07-14T18:15:48.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.571+0000 7f84c7fff700 1 -- 172.21.15.133:0/1520305092 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f84d0021410 con 0x7f84e00ffbd0 2023-07-14T18:15:48.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.573+0000 7f84c7fff700 1 --2- 172.21.15.133:0/1520305092 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f84c80a11e0 0x7f84c80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:48.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.573+0000 7f84c7fff700 1 -- 172.21.15.133:0/1520305092 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f84d00ce350 con 0x7f84e00ffbd0 2023-07-14T18:15:48.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.573+0000 7f84de7fc700 1 --2- 172.21.15.133:0/1520305092 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f84c80a11e0 0x7f84c80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:48.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.573+0000 7f84de7fc700 1 --2- 172.21.15.133:0/1520305092 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f84c80a11e0 0x7f84c80a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009c60 tx=0x7f84d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:48.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.576+0000 7f84c7fff700 1 -- 172.21.15.133:0/1520305092 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f84d0097660 con 0x7f84e00ffbd0 2023-07-14T18:15:48.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.786+0000 7f84e55ff700 1 -- 172.21.15.133:0/1520305092 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f84e01a0650 con 0x7f84e00ffbd0 2023-07-14T18:15:48.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.786+0000 7f84c7fff700 1 -- 172.21.15.133:0/1520305092 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f84d009ac80 con 0x7f84e00ffbd0 2023-07-14T18:15:48.831 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-07-14T18:15:48.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.789+0000 7f84e55ff700 1 -- 172.21.15.133:0/1520305092 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f84c80a11e0 msgr2=0x7f84c80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:48.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.789+0000 7f84e55ff700 1 --2- 172.21.15.133:0/1520305092 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f84c80a11e0 0x7f84c80a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009c60 tx=0x7f84d4009260 comp rx=0 tx=0).stop 2023-07-14T18:15:48.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.789+0000 7f84e55ff700 1 -- 172.21.15.133:0/1520305092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84e00ffbd0 msgr2=0x7f84e0197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:48.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.789+0000 7f84e55ff700 1 --2- 172.21.15.133:0/1520305092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84e00ffbd0 0x7f84e0197b20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f84d0000c00 tx=0x7f84d000f460 comp rx=0 tx=0).stop 2023-07-14T18:15:48.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.789+0000 7f84e55ff700 1 -- 172.21.15.133:0/1520305092 shutdown_connections 2023-07-14T18:15:48.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.789+0000 7f84e55ff700 1 --2- 172.21.15.133:0/1520305092 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f84c80a11e0 0x7f84c80a3690 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:48.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.789+0000 7f84e55ff700 1 --2- 172.21.15.133:0/1520305092 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84e0101810 0x7f84e019f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:48.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.789+0000 7f84e55ff700 1 --2- 172.21.15.133:0/1520305092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84e00ffbd0 0x7f84e0197b20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:48.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.789+0000 7f84e55ff700 1 --2- 172.21.15.133:0/1520305092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84e0100e60 0x7f84e0198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:48.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.789+0000 7f84e55ff700 1 -- 172.21.15.133:0/1520305092 >> 172.21.15.133:0/1520305092 conn(0x7f84e00fb1f0 msgr2=0x7f84e00fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:48.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.789+0000 7f84e55ff700 1 -- 172.21.15.133:0/1520305092 shutdown_connections 2023-07-14T18:15:48.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:48.789+0000 7f84e55ff700 1 -- 172.21.15.133:0/1520305092 wait complete. 2023-07-14T18:15:48.852 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T18:15:49.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.013+0000 7fd83dbff700 1 -- 172.21.15.133:0/1603889234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8381006f0 msgr2=0x7fd838102ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:49.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.013+0000 7fd83dbff700 1 --2- 172.21.15.133:0/1603889234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8381006f0 0x7fd838102ad0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd828009990 tx=0x7fd828009ca0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.013+0000 7fd83dbff700 1 -- 172.21.15.133:0/1603889234 shutdown_connections 2023-07-14T18:15:49.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.013+0000 7fd83dbff700 1 --2- 172.21.15.133:0/1603889234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd838105930 0x7fd838107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.013+0000 7fd83dbff700 1 --2- 172.21.15.133:0/1603889234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8381006f0 0x7fd838102ad0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.013+0000 7fd83dbff700 1 --2- 172.21.15.133:0/1603889234 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd838103010 0x7fd8381053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.013+0000 7fd83dbff700 1 -- 172.21.15.133:0/1603889234 >> 172.21.15.133:0/1603889234 conn(0x7fd8380fa130 msgr2=0x7fd8380fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:49.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.013+0000 7fd83dbff700 1 -- 172.21.15.133:0/1603889234 shutdown_connections 2023-07-14T18:15:49.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.013+0000 7fd83dbff700 1 -- 172.21.15.133:0/1603889234 wait complete. 2023-07-14T18:15:49.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.014+0000 7fd83dbff700 1 Processor -- start 2023-07-14T18:15:49.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.014+0000 7fd83dbff700 1 -- start start 2023-07-14T18:15:49.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.014+0000 7fd83dbff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8381006f0 0x7fd838197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:49.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.014+0000 7fd83dbff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd838103010 0x7fd838198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:49.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.014+0000 7fd83dbff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd838105930 0x7fd83819f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:49.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.014+0000 7fd83dbff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83819f750 con 0x7fd838105930 2023-07-14T18:15:49.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.014+0000 7fd83dbff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83819f890 con 0x7fd8381006f0 2023-07-14T18:15:49.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.014+0000 7fd83dbff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83819fa20 con 0x7fd838103010 2023-07-14T18:15:49.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.015+0000 7fd837fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd838105930 0x7fd83819f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:49.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.015+0000 7fd8377fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8381006f0 0x7fd838197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:49.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.015+0000 7fd82ffff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd838103010 0x7fd838198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:49.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.015+0000 7fd8377fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8381006f0 0x7fd838197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:48626/0 (socket says 172.21.15.133:48626) 2023-07-14T18:15:49.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.015+0000 7fd837fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd838105930 0x7fd83819f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60846/0 (socket says 172.21.15.133:60846) 2023-07-14T18:15:49.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.015+0000 7fd82ffff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd838103010 0x7fd838198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:39520/0 (socket says 172.21.15.133:39520) 2023-07-14T18:15:49.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.015+0000 7fd8377fe700 1 -- 172.21.15.133:0/2838526003 learned_addr learned my addr 172.21.15.133:0/2838526003 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:49.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.015+0000 7fd837fff700 1 -- 172.21.15.133:0/2838526003 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd838103010 msgr2=0x7fd838198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:49.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.015+0000 7fd837fff700 1 --2- 172.21.15.133:0/2838526003 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd838103010 0x7fd838198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.015+0000 7fd837fff700 1 -- 172.21.15.133:0/2838526003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8381006f0 msgr2=0x7fd838197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:49.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.015+0000 7fd837fff700 1 --2- 172.21.15.133:0/2838526003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8381006f0 0x7fd838197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.015+0000 7fd837fff700 1 -- 172.21.15.133:0/2838526003 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8280095f0 con 0x7fd838105930 2023-07-14T18:15:49.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.015+0000 7fd82ffff700 1 --2- 172.21.15.133:0/2838526003 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd838103010 0x7fd838198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:49.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.015+0000 7fd837fff700 1 --2- 172.21.15.133:0/2838526003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd838105930 0x7fd83819f180 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd830007f10 tx=0x7fd83000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:49.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.016+0000 7fd8357fa700 1 -- 172.21.15.133:0/2838526003 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd830004db0 con 0x7fd838105930 2023-07-14T18:15:49.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.016+0000 7fd83dbff700 1 -- 172.21.15.133:0/2838526003 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd83819fd00 con 0x7fd838105930 2023-07-14T18:15:49.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.016+0000 7fd8357fa700 1 -- 172.21.15.133:0/2838526003 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd83000f040 con 0x7fd838105930 2023-07-14T18:15:49.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.016+0000 7fd8357fa700 1 -- 172.21.15.133:0/2838526003 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd830005990 con 0x7fd838105930 2023-07-14T18:15:49.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.016+0000 7fd83dbff700 1 -- 172.21.15.133:0/2838526003 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8381a01c0 con 0x7fd838105930 2023-07-14T18:15:49.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.017+0000 7fd83dbff700 1 -- 172.21.15.133:0/2838526003 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8381074a0 con 0x7fd838105930 2023-07-14T18:15:49.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.019+0000 7fd8357fa700 1 -- 172.21.15.133:0/2838526003 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fd83000db20 con 0x7fd838105930 2023-07-14T18:15:49.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.020+0000 7fd8357fa700 1 --2- 172.21.15.133:0/2838526003 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd8240a11e0 0x7fd8240a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:49.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.020+0000 7fd8357fa700 1 -- 172.21.15.133:0/2838526003 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd8300cc750 con 0x7fd838105930 2023-07-14T18:15:49.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.020+0000 7fd8377fe700 1 --2- 172.21.15.133:0/2838526003 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd8240a11e0 0x7fd8240a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:49.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.021+0000 7fd8377fe700 1 --2- 172.21.15.133:0/2838526003 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd8240a11e0 0x7fd8240a3690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd828006010 tx=0x7fd828005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:49.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.022+0000 7fd8357fa700 1 -- 172.21.15.133:0/2838526003 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fd8300959b0 con 0x7fd838105930 2023-07-14T18:15:49.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:48 smithi133 ceph-mon[126384]: pgmap v320: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T18:15:49.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:48 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2023-07-14T18:15:49.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:48 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3191642871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:15:49.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:48 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1520305092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:15:49.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.219+0000 7fd83dbff700 1 -- 172.21.15.133:0/2838526003 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd83802ca70 con 0x7fd838105930 2023-07-14T18:15:49.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.219+0000 7fd8357fa700 1 -- 172.21.15.133:0/2838526003 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd830098fd0 con 0x7fd838105930 2023-07-14T18:15:49.264 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-07-14T18:15:49.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.221+0000 7fd83dbff700 1 -- 172.21.15.133:0/2838526003 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd8240a11e0 msgr2=0x7fd8240a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:49.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.221+0000 7fd83dbff700 1 --2- 172.21.15.133:0/2838526003 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd8240a11e0 0x7fd8240a3690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd828006010 tx=0x7fd828005960 comp rx=0 tx=0).stop 2023-07-14T18:15:49.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.222+0000 7fd83dbff700 1 -- 172.21.15.133:0/2838526003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd838105930 msgr2=0x7fd83819f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:49.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.222+0000 7fd83dbff700 1 --2- 172.21.15.133:0/2838526003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd838105930 0x7fd83819f180 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd830007f10 tx=0x7fd83000d3b0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.222+0000 7fd83dbff700 1 -- 172.21.15.133:0/2838526003 shutdown_connections 2023-07-14T18:15:49.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.222+0000 7fd83dbff700 1 --2- 172.21.15.133:0/2838526003 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd8240a11e0 0x7fd8240a3690 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.222+0000 7fd83dbff700 1 --2- 172.21.15.133:0/2838526003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8381006f0 0x7fd838197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.222+0000 7fd83dbff700 1 --2- 172.21.15.133:0/2838526003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd838105930 0x7fd83819f180 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.222+0000 7fd83dbff700 1 --2- 172.21.15.133:0/2838526003 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd838103010 0x7fd838198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.222+0000 7fd83dbff700 1 -- 172.21.15.133:0/2838526003 >> 172.21.15.133:0/2838526003 conn(0x7fd8380fa130 msgr2=0x7fd8380fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:49.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.222+0000 7fd83dbff700 1 -- 172.21.15.133:0/2838526003 shutdown_connections 2023-07-14T18:15:49.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.222+0000 7fd83dbff700 1 -- 172.21.15.133:0/2838526003 wait complete. 2023-07-14T18:15:49.285 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T18:15:49.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:48 smithi161 ceph-mon[128993]: pgmap v320: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T18:15:49.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:48 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2023-07-14T18:15:49.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:48 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3191642871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:15:49.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:48 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1520305092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:15:49.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:48 smithi171 ceph-mon[129241]: pgmap v320: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T18:15:49.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:48 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2023-07-14T18:15:49.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:48 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3191642871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:15:49.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:48 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1520305092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:15:49.487 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.442+0000 7f190c5ff700 1 -- 172.21.15.133:0/2259035561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19040ffc20 msgr2=0x7f1904100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:49.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.442+0000 7f190c5ff700 1 --2- 172.21.15.133:0/2259035561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19040ffc20 0x7f1904100030 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1900009990 tx=0x7f1900009ca0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.442+0000 7f190c5ff700 1 -- 172.21.15.133:0/2259035561 shutdown_connections 2023-07-14T18:15:49.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.442+0000 7f190c5ff700 1 --2- 172.21.15.133:0/2259035561 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f19041017a0 0x7f1904108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.442+0000 7f190c5ff700 1 --2- 172.21.15.133:0/2259035561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19040ffc20 0x7f1904100030 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.442+0000 7f190c5ff700 1 --2- 172.21.15.133:0/2259035561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1904100e10 0x7f1904101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.489 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.442+0000 7f190c5ff700 1 -- 172.21.15.133:0/2259035561 >> 172.21.15.133:0/2259035561 conn(0x7f19040fb260 msgr2=0x7f19040fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:49.489 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.442+0000 7f190c5ff700 1 -- 172.21.15.133:0/2259035561 shutdown_connections 2023-07-14T18:15:49.489 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.442+0000 7f190c5ff700 1 -- 172.21.15.133:0/2259035561 wait complete. 2023-07-14T18:15:49.490 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.443+0000 7f190c5ff700 1 Processor -- start 2023-07-14T18:15:49.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.444+0000 7f190c5ff700 1 -- start start 2023-07-14T18:15:49.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.444+0000 7f190c5ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f19040ffc20 0x7f1904197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:49.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.444+0000 7f190c5ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1904100e10 0x7f19041980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:49.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.444+0000 7f190c5ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19041017a0 0x7f190419f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:49.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.444+0000 7f190c5ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190419f710 con 0x7f19041017a0 2023-07-14T18:15:49.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.444+0000 7f190c5ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190419f850 con 0x7f1904100e10 2023-07-14T18:15:49.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.444+0000 7f190c5ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190419f9e0 con 0x7f19040ffc20 2023-07-14T18:15:49.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.444+0000 7f190ab9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19041017a0 0x7f190419f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:49.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.444+0000 7f190ab9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19041017a0 0x7f190419f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60860/0 (socket says 172.21.15.133:60860) 2023-07-14T18:15:49.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.444+0000 7f190ab9c700 1 -- 172.21.15.133:0/1305206412 learned_addr learned my addr 172.21.15.133:0/1305206412 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:49.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.444+0000 7f190a39b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f19040ffc20 0x7f1904197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:49.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.444+0000 7f1909b9a700 1 --2- 172.21.15.133:0/1305206412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1904100e10 0x7f19041980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:49.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.445+0000 7f190ab9c700 1 -- 172.21.15.133:0/1305206412 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f19040ffc20 msgr2=0x7f1904197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:49.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.445+0000 7f190ab9c700 1 --2- 172.21.15.133:0/1305206412 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f19040ffc20 0x7f1904197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.493 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.445+0000 7f190ab9c700 1 -- 172.21.15.133:0/1305206412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1904100e10 msgr2=0x7f19041980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:49.493 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.445+0000 7f190ab9c700 1 --2- 172.21.15.133:0/1305206412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1904100e10 0x7f19041980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.493 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.445+0000 7f190ab9c700 1 -- 172.21.15.133:0/1305206412 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19000095f0 con 0x7f19041017a0 2023-07-14T18:15:49.493 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.445+0000 7f190ab9c700 1 --2- 172.21.15.133:0/1305206412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19041017a0 0x7f190419f140 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f18fc007ee0 tx=0x7f18fc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:49.493 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.445+0000 7f18fb7fe700 1 -- 172.21.15.133:0/1305206412 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc017070 con 0x7f19041017a0 2023-07-14T18:15:49.493 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.445+0000 7f18fb7fe700 1 -- 172.21.15.133:0/1305206412 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18fc00f040 con 0x7f19041017a0 2023-07-14T18:15:49.493 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.445+0000 7f190c5ff700 1 -- 172.21.15.133:0/1305206412 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190419fcc0 con 0x7f19041017a0 2023-07-14T18:15:49.494 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.445+0000 7f18fb7fe700 1 -- 172.21.15.133:0/1305206412 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc005580 con 0x7f19041017a0 2023-07-14T18:15:49.494 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.445+0000 7f190c5ff700 1 -- 172.21.15.133:0/1305206412 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19041a01d0 con 0x7f19041017a0 2023-07-14T18:15:49.494 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.446+0000 7f190c5ff700 1 -- 172.21.15.133:0/1305206412 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1904192300 con 0x7f19041017a0 2023-07-14T18:15:49.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.448+0000 7f18fb7fe700 1 -- 172.21.15.133:0/1305206412 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f18fc00db20 con 0x7f19041017a0 2023-07-14T18:15:49.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.449+0000 7f18fb7fe700 1 --2- 172.21.15.133:0/1305206412 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f18f00a1260 0x7f18f00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:49.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.450+0000 7f18fb7fe700 1 -- 172.21.15.133:0/1305206412 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f18fc0cc4e0 con 0x7f19041017a0 2023-07-14T18:15:49.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.450+0000 7f190a39b700 1 --2- 172.21.15.133:0/1305206412 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f18f00a1260 0x7f18f00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:49.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.450+0000 7f190a39b700 1 --2- 172.21.15.133:0/1305206412 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f18f00a1260 0x7f18f00a3710 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f190000b2e0 tx=0x7f1900005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:49.498 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.453+0000 7f18fb7fe700 1 -- 172.21.15.133:0/1305206412 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f18fc095740 con 0x7f19041017a0 2023-07-14T18:15:49.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.653+0000 7f190c5ff700 1 -- 172.21.15.133:0/1305206412 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f190402cb00 con 0x7f19041017a0 2023-07-14T18:15:49.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.653+0000 7f18fb7fe700 1 -- 172.21.15.133:0/1305206412 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f18fc098d60 con 0x7f19041017a0 2023-07-14T18:15:49.698 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2023-07-14T18:15:49.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.655+0000 7f190c5ff700 1 -- 172.21.15.133:0/1305206412 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f18f00a1260 msgr2=0x7f18f00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:49.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.655+0000 7f190c5ff700 1 --2- 172.21.15.133:0/1305206412 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f18f00a1260 0x7f18f00a3710 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f190000b2e0 tx=0x7f1900005fb0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.656+0000 7f190c5ff700 1 -- 172.21.15.133:0/1305206412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19041017a0 msgr2=0x7f190419f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:49.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.656+0000 7f190c5ff700 1 --2- 172.21.15.133:0/1305206412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19041017a0 0x7f190419f140 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f18fc007ee0 tx=0x7f18fc00d3b0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.656+0000 7f190c5ff700 1 -- 172.21.15.133:0/1305206412 shutdown_connections 2023-07-14T18:15:49.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.656+0000 7f190c5ff700 1 --2- 172.21.15.133:0/1305206412 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f18f00a1260 0x7f18f00a3710 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.656+0000 7f190c5ff700 1 --2- 172.21.15.133:0/1305206412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19041017a0 0x7f190419f140 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.656+0000 7f190c5ff700 1 --2- 172.21.15.133:0/1305206412 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f19040ffc20 0x7f1904197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.656+0000 7f190c5ff700 1 --2- 172.21.15.133:0/1305206412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1904100e10 0x7f19041980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.656+0000 7f190c5ff700 1 -- 172.21.15.133:0/1305206412 >> 172.21.15.133:0/1305206412 conn(0x7f19040fb260 msgr2=0x7f1904106710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:49.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.656+0000 7f190c5ff700 1 -- 172.21.15.133:0/1305206412 shutdown_connections 2023-07-14T18:15:49.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.656+0000 7f190c5ff700 1 -- 172.21.15.133:0/1305206412 wait complete. 2023-07-14T18:15:49.720 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T18:15:49.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.876+0000 7f68109ff700 1 -- 172.21.15.133:0/3435749651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f680c0ffc20 msgr2=0x7f680c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:49.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.876+0000 7f68109ff700 1 --2- 172.21.15.133:0/3435749651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f680c0ffc20 0x7f680c100030 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0098f0 tx=0x7f67fc009c00 comp rx=0 tx=0).stop 2023-07-14T18:15:49.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.876+0000 7f68109ff700 1 -- 172.21.15.133:0/3435749651 shutdown_connections 2023-07-14T18:15:49.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.876+0000 7f68109ff700 1 --2- 172.21.15.133:0/3435749651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f680c1017a0 0x7f680c108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.876+0000 7f68109ff700 1 --2- 172.21.15.133:0/3435749651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f680c0ffc20 0x7f680c100030 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.876+0000 7f68109ff700 1 --2- 172.21.15.133:0/3435749651 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f680c100e10 0x7f680c101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.876+0000 7f68109ff700 1 -- 172.21.15.133:0/3435749651 >> 172.21.15.133:0/3435749651 conn(0x7f680c0fb260 msgr2=0x7f680c0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:49.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.877+0000 7f68109ff700 1 -- 172.21.15.133:0/3435749651 shutdown_connections 2023-07-14T18:15:49.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.877+0000 7f68109ff700 1 -- 172.21.15.133:0/3435749651 wait complete. 2023-07-14T18:15:49.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.878+0000 7f68109ff700 1 Processor -- start 2023-07-14T18:15:49.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.878+0000 7f68109ff700 1 -- start start 2023-07-14T18:15:49.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.879+0000 7f68109ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f680c0ffc20 0x7f680c197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:49.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.879+0000 7f68109ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f680c100e10 0x7f680c1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:49.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.879+0000 7f68109ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f680c1017a0 0x7f680c19f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:49.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.879+0000 7f68109ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c19f7f0 con 0x7f680c1017a0 2023-07-14T18:15:49.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.879+0000 7f68109ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c19f960 con 0x7f680c100e10 2023-07-14T18:15:49.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.879+0000 7f68109ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c19fb20 con 0x7f680c0ffc20 2023-07-14T18:15:49.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.879+0000 7f680ad9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f680c1017a0 0x7f680c19f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:49.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.879+0000 7f680a59c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f680c0ffc20 0x7f680c197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:49.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.879+0000 7f680a59c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f680c0ffc20 0x7f680c197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:39550/0 (socket says 172.21.15.133:39550) 2023-07-14T18:15:49.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.879+0000 7f680a59c700 1 -- 172.21.15.133:0/1104058552 learned_addr learned my addr 172.21.15.133:0/1104058552 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:49.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.879+0000 7f6809d9b700 1 --2- 172.21.15.133:0/1104058552 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f680c100e10 0x7f680c1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:49.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.879+0000 7f680ad9d700 1 -- 172.21.15.133:0/1104058552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f680c0ffc20 msgr2=0x7f680c197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:49.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.879+0000 7f680ad9d700 1 --2- 172.21.15.133:0/1104058552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f680c0ffc20 0x7f680c197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.879+0000 7f680ad9d700 1 -- 172.21.15.133:0/1104058552 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f680c100e10 msgr2=0x7f680c1981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:49.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.879+0000 7f680ad9d700 1 --2- 172.21.15.133:0/1104058552 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f680c100e10 0x7f680c1981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:49.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.879+0000 7f680ad9d700 1 -- 172.21.15.133:0/1104058552 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6804009d00 con 0x7f680c1017a0 2023-07-14T18:15:49.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.879+0000 7f680a59c700 1 --2- 172.21.15.133:0/1104058552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f680c0ffc20 0x7f680c197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:49.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.880+0000 7f680ad9d700 1 --2- 172.21.15.133:0/1104058552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f680c1017a0 0x7f680c19f220 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6804009cd0 tx=0x7f680400c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:49.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.880+0000 7f67fb7fe700 1 -- 172.21.15.133:0/1104058552 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6804017070 con 0x7f680c1017a0 2023-07-14T18:15:49.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.880+0000 7f68109ff700 1 -- 172.21.15.133:0/1104058552 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67fc009520 con 0x7f680c1017a0 2023-07-14T18:15:49.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.880+0000 7f67fb7fe700 1 -- 172.21.15.133:0/1104058552 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f680400f040 con 0x7f680c1017a0 2023-07-14T18:15:49.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.880+0000 7f67fb7fe700 1 -- 172.21.15.133:0/1104058552 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680400e050 con 0x7f680c1017a0 2023-07-14T18:15:49.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.880+0000 7f68109ff700 1 -- 172.21.15.133:0/1104058552 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680c1a0120 con 0x7f680c1017a0 2023-07-14T18:15:49.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.881+0000 7f68109ff700 1 -- 172.21.15.133:0/1104058552 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f680c192300 con 0x7f680c1017a0 2023-07-14T18:15:49.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.883+0000 7f67fb7fe700 1 -- 172.21.15.133:0/1104058552 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f680400cd10 con 0x7f680c1017a0 2023-07-14T18:15:49.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.884+0000 7f67fb7fe700 1 --2- 172.21.15.133:0/1104058552 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f67f40a11e0 0x7f67f40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:49.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.884+0000 7f67fb7fe700 1 -- 172.21.15.133:0/1104058552 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f68040cc6a0 con 0x7f680c1017a0 2023-07-14T18:15:49.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.885+0000 7f680a59c700 1 --2- 172.21.15.133:0/1104058552 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f67f40a11e0 0x7f67f40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:49.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.885+0000 7f680a59c700 1 --2- 172.21.15.133:0/1104058552 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f67f40a11e0 0x7f67f40a3690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00f9e0 tx=0x7f67fc019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:49.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:49.888+0000 7f67fb7fe700 1 -- 172.21.15.133:0/1104058552 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f6804095900 con 0x7f680c1017a0 2023-07-14T18:15:50.139 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.095+0000 7f68109ff700 1 -- 172.21.15.133:0/1104058552 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f680c02cb30 con 0x7f680c1017a0 2023-07-14T18:15:50.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.095+0000 7f67fb7fe700 1 -- 172.21.15.133:0/1104058552 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6804098f20 con 0x7f680c1017a0 2023-07-14T18:15:50.141 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-07-14T18:15:50.143 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.098+0000 7f68109ff700 1 -- 172.21.15.133:0/1104058552 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f67f40a11e0 msgr2=0x7f67f40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:50.143 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.098+0000 7f68109ff700 1 --2- 172.21.15.133:0/1104058552 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f67f40a11e0 0x7f67f40a3690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00f9e0 tx=0x7f67fc019040 comp rx=0 tx=0).stop 2023-07-14T18:15:50.143 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.098+0000 7f68109ff700 1 -- 172.21.15.133:0/1104058552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f680c1017a0 msgr2=0x7f680c19f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:50.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.098+0000 7f68109ff700 1 --2- 172.21.15.133:0/1104058552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f680c1017a0 0x7f680c19f220 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6804009cd0 tx=0x7f680400c620 comp rx=0 tx=0).stop 2023-07-14T18:15:50.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.098+0000 7f68109ff700 1 -- 172.21.15.133:0/1104058552 shutdown_connections 2023-07-14T18:15:50.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.098+0000 7f68109ff700 1 --2- 172.21.15.133:0/1104058552 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f67f40a11e0 0x7f67f40a3690 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:50.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.098+0000 7f68109ff700 1 --2- 172.21.15.133:0/1104058552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f680c1017a0 0x7f680c19f220 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:50.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.098+0000 7f68109ff700 1 --2- 172.21.15.133:0/1104058552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f680c0ffc20 0x7f680c197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:50.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.098+0000 7f68109ff700 1 --2- 172.21.15.133:0/1104058552 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f680c100e10 0x7f680c1981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:50.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.098+0000 7f68109ff700 1 -- 172.21.15.133:0/1104058552 >> 172.21.15.133:0/1104058552 conn(0x7f680c0fb260 msgr2=0x7f680c1067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:50.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.098+0000 7f68109ff700 1 -- 172.21.15.133:0/1104058552 shutdown_connections 2023-07-14T18:15:50.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.098+0000 7f68109ff700 1 -- 172.21.15.133:0/1104058552 wait complete. 2023-07-14T18:15:50.161 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T18:15:50.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:49 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2023-07-14T18:15:50.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:49 smithi161 ceph-mon[128993]: osdmap e91: 12 total, 12 up, 12 in 2023-07-14T18:15:50.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:49 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2838526003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:15:50.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:49 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1305206412' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:15:50.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.315+0000 7f5152fff700 1 -- 172.21.15.133:0/2587215734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f514c100e20 msgr2=0x7f514c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:50.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.315+0000 7f5152fff700 1 --2- 172.21.15.133:0/2587215734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f514c100e20 0x7f514c101270 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5138009990 tx=0x7f5138009ca0 comp rx=0 tx=0).stop 2023-07-14T18:15:50.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.315+0000 7f5152fff700 1 -- 172.21.15.133:0/2587215734 shutdown_connections 2023-07-14T18:15:50.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.315+0000 7f5152fff700 1 --2- 172.21.15.133:0/2587215734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f514c1017b0 0x7f514c108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:50.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.315+0000 7f5152fff700 1 --2- 172.21.15.133:0/2587215734 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f514c0ffc30 0x7f514c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:50.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.315+0000 7f5152fff700 1 --2- 172.21.15.133:0/2587215734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f514c100e20 0x7f514c101270 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:50.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.315+0000 7f5152fff700 1 -- 172.21.15.133:0/2587215734 >> 172.21.15.133:0/2587215734 conn(0x7f514c0fb210 msgr2=0x7f514c0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:50.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.315+0000 7f5152fff700 1 -- 172.21.15.133:0/2587215734 shutdown_connections 2023-07-14T18:15:50.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.315+0000 7f5152fff700 1 -- 172.21.15.133:0/2587215734 wait complete. 2023-07-14T18:15:50.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.316+0000 7f5152fff700 1 Processor -- start 2023-07-14T18:15:50.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.316+0000 7f5152fff700 1 -- start start 2023-07-14T18:15:50.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.316+0000 7f5152fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f514c0ffc30 0x7f514c0743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:50.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.316+0000 7f5152fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f514c100e20 0x7f514c072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:50.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.316+0000 7f5152fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f514c1017b0 0x7f514c072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:50.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.316+0000 7f5152fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c073710 con 0x7f514c1017b0 2023-07-14T18:15:50.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.316+0000 7f5152fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c073880 con 0x7f514c100e20 2023-07-14T18:15:50.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.316+0000 7f5152fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c19f9d0 con 0x7f514c0ffc30 2023-07-14T18:15:50.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.317+0000 7f5150d9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f514c0ffc30 0x7f514c0743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:50.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.317+0000 7f5150d9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f514c0ffc30 0x7f514c0743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:39572/0 (socket says 172.21.15.133:39572) 2023-07-14T18:15:50.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.317+0000 7f515159c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f514c1017b0 0x7f514c072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:50.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.317+0000 7f5150d9b700 1 -- 172.21.15.133:0/2579061484 learned_addr learned my addr 172.21.15.133:0/2579061484 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:50.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.317+0000 7f514bfff700 1 --2- 172.21.15.133:0/2579061484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f514c100e20 0x7f514c072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:50.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.317+0000 7f515159c700 1 -- 172.21.15.133:0/2579061484 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f514c0ffc30 msgr2=0x7f514c0743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:50.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.317+0000 7f515159c700 1 --2- 172.21.15.133:0/2579061484 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f514c0ffc30 0x7f514c0743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:50.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.317+0000 7f515159c700 1 -- 172.21.15.133:0/2579061484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f514c100e20 msgr2=0x7f514c072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:50.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.317+0000 7f515159c700 1 --2- 172.21.15.133:0/2579061484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f514c100e20 0x7f514c072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:50.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.317+0000 7f515159c700 1 -- 172.21.15.133:0/2579061484 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51380095f0 con 0x7f514c1017b0 2023-07-14T18:15:50.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.317+0000 7f5150d9b700 1 --2- 172.21.15.133:0/2579061484 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f514c0ffc30 0x7f514c0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:50.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.317+0000 7f515159c700 1 --2- 172.21.15.133:0/2579061484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f514c1017b0 0x7f514c072f70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f514400efc0 tx=0x7f514400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:50.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.317+0000 7f5149ffb700 1 -- 172.21.15.133:0/2579061484 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5144018070 con 0x7f514c1017b0 2023-07-14T18:15:50.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.317+0000 7f5149ffb700 1 -- 172.21.15.133:0/2579061484 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f514400f040 con 0x7f514c1017b0 2023-07-14T18:15:50.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.318+0000 7f5152fff700 1 -- 172.21.15.133:0/2579061484 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514c19fc50 con 0x7f514c1017b0 2023-07-14T18:15:50.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.318+0000 7f5149ffb700 1 -- 172.21.15.133:0/2579061484 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514400cd40 con 0x7f514c1017b0 2023-07-14T18:15:50.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.318+0000 7f5152fff700 1 -- 172.21.15.133:0/2579061484 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514c1a0110 con 0x7f514c1017b0 2023-07-14T18:15:50.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.319+0000 7f5152fff700 1 -- 172.21.15.133:0/2579061484 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514c107120 con 0x7f514c1017b0 2023-07-14T18:15:50.372 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:49 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2023-07-14T18:15:50.372 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:49 smithi171 ceph-mon[129241]: osdmap e91: 12 total, 12 up, 12 in 2023-07-14T18:15:50.372 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:49 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2838526003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:15:50.372 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:49 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1305206412' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:15:50.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.321+0000 7f5149ffb700 1 -- 172.21.15.133:0/2579061484 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f5144003a00 con 0x7f514c1017b0 2023-07-14T18:15:50.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.322+0000 7f5149ffb700 1 --2- 172.21.15.133:0/2579061484 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f513c0a1220 0x7f513c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:50.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.322+0000 7f5149ffb700 1 -- 172.21.15.133:0/2579061484 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f51440cc930 con 0x7f514c1017b0 2023-07-14T18:15:50.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.322+0000 7f5150d9b700 1 --2- 172.21.15.133:0/2579061484 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f513c0a1220 0x7f513c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:50.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.323+0000 7f5150d9b700 1 --2- 172.21.15.133:0/2579061484 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f513c0a1220 0x7f513c0a36d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5140005950 tx=0x7f5140009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:50.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.326+0000 7f5149ffb700 1 -- 172.21.15.133:0/2579061484 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f5144095c20 con 0x7f514c1017b0 2023-07-14T18:15:50.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:49 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2023-07-14T18:15:50.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:49 smithi133 ceph-mon[126384]: osdmap e91: 12 total, 12 up, 12 in 2023-07-14T18:15:50.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:49 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2838526003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:15:50.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:49 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1305206412' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:15:50.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.567+0000 7f5152fff700 1 -- 172.21.15.133:0/2579061484 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f514c02c8e0 con 0x7f514c1017b0 2023-07-14T18:15:50.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.568+0000 7f5149ffb700 1 -- 172.21.15.133:0/2579061484 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10851 (secure 0 0 0) 0x7f5144099240 con 0x7f514c1017b0 2023-07-14T18:15:50.613 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:15:50.613 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3168151271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3168151271},{"type":"v1","addr":"172.21.15.133:6837","nonce":3168151271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14544,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/3093144261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3093144261},{"type":"v1","addr":"172.21.15.133:6839","nonce":3093144261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14556,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/3282902068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":3282902068},{"type":"v1","addr":"172.21.15.133:6841","nonce":3282902068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/4023127259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4023127259},{"type":"v1","addr":"172.21.15.161:6833","nonce":4023127259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2722373740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2722373740},{"type":"v1","addr":"172.21.15.171:6833","nonce":2722373740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2399005074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2399005074},{"type":"v1","addr":"172.21.15.133:6835","nonce":2399005074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T18:14:17.440873+0000","modified":"2023-07-14T18:14:50.886364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24557,"mds_1":24541,"mds_2":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3725502564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3725502564},{"type":"v1","addr":"172.21.15.161:6835","nonce":3725502564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6835/1877362116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1877362116},{"type":"v1","addr":"172.21.15.171:6835","nonce":1877362116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/3458064798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3458064798},{"type":"v1","addr":"172.21.15.161:6837","nonce":3458064798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/106205332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":106205332},{"type":"v1","addr":"172.21.15.171:6837","nonce":106205332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.171:6839/816201198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":816201198},{"type":"v1","addr":"172.21.15.171:6839","nonce":816201198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/2503865871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2503865871},{"type":"v1","addr":"172.21.15.161:6839","nonce":2503865871}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T18:15:50.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.570+0000 7f5152fff700 1 -- 172.21.15.133:0/2579061484 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f513c0a1220 msgr2=0x7f513c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:50.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.570+0000 7f5152fff700 1 --2- 172.21.15.133:0/2579061484 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f513c0a1220 0x7f513c0a36d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5140005950 tx=0x7f5140009260 comp rx=0 tx=0).stop 2023-07-14T18:15:50.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.570+0000 7f5152fff700 1 -- 172.21.15.133:0/2579061484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f514c1017b0 msgr2=0x7f514c072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:50.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.570+0000 7f5152fff700 1 --2- 172.21.15.133:0/2579061484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f514c1017b0 0x7f514c072f70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f514400efc0 tx=0x7f514400c3a0 comp rx=0 tx=0).stop 2023-07-14T18:15:50.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.570+0000 7f5152fff700 1 -- 172.21.15.133:0/2579061484 shutdown_connections 2023-07-14T18:15:50.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.570+0000 7f5152fff700 1 --2- 172.21.15.133:0/2579061484 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f513c0a1220 0x7f513c0a36d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:50.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.570+0000 7f5152fff700 1 --2- 172.21.15.133:0/2579061484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f514c1017b0 0x7f514c072f70 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:50.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.570+0000 7f5152fff700 1 --2- 172.21.15.133:0/2579061484 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f514c0ffc30 0x7f514c0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:50.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.570+0000 7f5152fff700 1 --2- 172.21.15.133:0/2579061484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f514c100e20 0x7f514c072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:50.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.570+0000 7f5152fff700 1 -- 172.21.15.133:0/2579061484 >> 172.21.15.133:0/2579061484 conn(0x7f514c0fb210 msgr2=0x7f514c104260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:50.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.570+0000 7f5152fff700 1 -- 172.21.15.133:0/2579061484 shutdown_connections 2023-07-14T18:15:50.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.570+0000 7f5152fff700 1 -- 172.21.15.133:0/2579061484 wait complete. 2023-07-14T18:15:50.617 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 25 2023-07-14T18:15:50.634 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T18:15:50.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.787+0000 7fbe9c5ff700 1 -- 172.21.15.133:0/3217842648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe941017b0 msgr2=0x7fbe94108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:50.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.787+0000 7fbe9c5ff700 1 --2- 172.21.15.133:0/3217842648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe941017b0 0x7fbe94108070 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009f90 tx=0x7fbe8c00b330 comp rx=0 tx=0).stop 2023-07-14T18:15:50.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.787+0000 7fbe9c5ff700 1 -- 172.21.15.133:0/3217842648 shutdown_connections 2023-07-14T18:15:50.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.787+0000 7fbe9c5ff700 1 --2- 172.21.15.133:0/3217842648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe940ffc30 0x7fbe94100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:50.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.787+0000 7fbe9c5ff700 1 --2- 172.21.15.133:0/3217842648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe941017b0 0x7fbe94108070 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:50.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.787+0000 7fbe9c5ff700 1 --2- 172.21.15.133:0/3217842648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe94100e20 0x7fbe94101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:50.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.787+0000 7fbe9c5ff700 1 -- 172.21.15.133:0/3217842648 >> 172.21.15.133:0/3217842648 conn(0x7fbe940fb210 msgr2=0x7fbe940fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:50.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.787+0000 7fbe9c5ff700 1 -- 172.21.15.133:0/3217842648 shutdown_connections 2023-07-14T18:15:50.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.787+0000 7fbe9c5ff700 1 -- 172.21.15.133:0/3217842648 wait complete. 2023-07-14T18:15:50.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.788+0000 7fbe9c5ff700 1 Processor -- start 2023-07-14T18:15:50.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.788+0000 7fbe9c5ff700 1 -- start start 2023-07-14T18:15:50.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.788+0000 7fbe9c5ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe940ffc30 0x7fbe94197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:50.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.788+0000 7fbe9c5ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe94100e20 0x7fbe941981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:50.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.788+0000 7fbe9c5ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe941017b0 0x7fbe9419f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:50.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.788+0000 7fbe9c5ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9419f7b0 con 0x7fbe94100e20 2023-07-14T18:15:50.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.788+0000 7fbe9c5ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9419f920 con 0x7fbe940ffc30 2023-07-14T18:15:50.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.788+0000 7fbe9c5ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9419fae0 con 0x7fbe941017b0 2023-07-14T18:15:50.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.789+0000 7fbe99b9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe94100e20 0x7fbe941981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:50.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.789+0000 7fbe99b9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe94100e20 0x7fbe941981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60908/0 (socket says 172.21.15.133:60908) 2023-07-14T18:15:50.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.789+0000 7fbe99b9a700 1 -- 172.21.15.133:0/1653812202 learned_addr learned my addr 172.21.15.133:0/1653812202 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:50.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.789+0000 7fbe9a39b700 1 --2- 172.21.15.133:0/1653812202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe940ffc30 0x7fbe94197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:50.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.789+0000 7fbe9ab9c700 1 --2- 172.21.15.133:0/1653812202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe941017b0 0x7fbe9419f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:50.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.789+0000 7fbe99b9a700 1 -- 172.21.15.133:0/1653812202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe941017b0 msgr2=0x7fbe9419f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:50.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.789+0000 7fbe99b9a700 1 --2- 172.21.15.133:0/1653812202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe941017b0 0x7fbe9419f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:50.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.789+0000 7fbe99b9a700 1 -- 172.21.15.133:0/1653812202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe940ffc30 msgr2=0x7fbe94197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:50.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.789+0000 7fbe99b9a700 1 --2- 172.21.15.133:0/1653812202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe940ffc30 0x7fbe94197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:50.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.789+0000 7fbe99b9a700 1 -- 172.21.15.133:0/1653812202 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8c009c40 con 0x7fbe94100e20 2023-07-14T18:15:50.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.789+0000 7fbe9ab9c700 1 --2- 172.21.15.133:0/1653812202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe941017b0 0x7fbe9419f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:15:50.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.789+0000 7fbe99b9a700 1 --2- 172.21.15.133:0/1653812202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe94100e20 0x7fbe941981b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400e9c0 tx=0x7fbe8400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:50.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.789+0000 7fbe8b7fe700 1 -- 172.21.15.133:0/1653812202 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8400cbd0 con 0x7fbe94100e20 2023-07-14T18:15:50.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.790+0000 7fbe9c5ff700 1 -- 172.21.15.133:0/1653812202 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9419fdc0 con 0x7fbe94100e20 2023-07-14T18:15:50.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.790+0000 7fbe8b7fe700 1 -- 172.21.15.133:0/1653812202 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe8400eeb0 con 0x7fbe94100e20 2023-07-14T18:15:50.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.790+0000 7fbe8b7fe700 1 -- 172.21.15.133:0/1653812202 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84005580 con 0x7fbe94100e20 2023-07-14T18:15:50.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.790+0000 7fbe9c5ff700 1 -- 172.21.15.133:0/1653812202 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe941a0210 con 0x7fbe94100e20 2023-07-14T18:15:50.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.791+0000 7fbe9c5ff700 1 -- 172.21.15.133:0/1653812202 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe9410b460 con 0x7fbe94100e20 2023-07-14T18:15:50.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.793+0000 7fbe8b7fe700 1 -- 172.21.15.133:0/1653812202 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fbe84004750 con 0x7fbe94100e20 2023-07-14T18:15:50.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.794+0000 7fbe8b7fe700 1 --2- 172.21.15.133:0/1653812202 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fbe800a1220 0x7fbe800a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:50.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.794+0000 7fbe8b7fe700 1 -- 172.21.15.133:0/1653812202 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbe84014070 con 0x7fbe94100e20 2023-07-14T18:15:50.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.794+0000 7fbe9a39b700 1 --2- 172.21.15.133:0/1653812202 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fbe800a1220 0x7fbe800a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:50.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.795+0000 7fbe9a39b700 1 --2- 172.21.15.133:0/1653812202 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fbe800a1220 0x7fbe800a36d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbe94199140 tx=0x7fbe90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:50.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.796+0000 7fbe8b7fe700 1 -- 172.21.15.133:0/1653812202 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fbe84096720 con 0x7fbe94100e20 2023-07-14T18:15:51.040 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.996+0000 7fbe9c5ff700 1 -- 172.21.15.133:0/1653812202 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe9410b650 con 0x7fbe94100e20 2023-07-14T18:15:51.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:50.997+0000 7fbe8b7fe700 1 -- 172.21.15.133:0/1653812202 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24520 (secure 0 0 0) 0x7fbe84099d40 con 0x7fbe94100e20 2023-07-14T18:15:51.042 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:15:51.043 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":92,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","created":"2023-07-14T18:06:49.561582+0000","modified":"2023-07-14T18:15:49.951050+0000","last_up_change":"2023-07-14T18:13:29.836099+0000","last_in_change":"2023-07-14T18:13:11.606710+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T18:10:03.945039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T18:14:15.451340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T18:14:16.892215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T18:14:20.493175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"85a29de7-d712-422a-b647-d9e90091aaf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6803","nonce":3076510801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6805","nonce":3076510801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6809","nonce":3076510801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6807","nonce":3076510801}]},"public_addr":"172.21.15.133:6803/3076510801","cluster_addr":"172.21.15.133:6805/3076510801","heartbeat_back_addr":"172.21.15.133:6809/3076510801","heartbeat_front_addr":"172.21.15.133:6807/3076510801","state":["exists","up"]},{"osd":1,"uuid":"80ad4e78-7917-495e-b119-99f7c7c36243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6801","nonce":1841901852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6803","nonce":1841901852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6807","nonce":1841901852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6805","nonce":1841901852}]},"public_addr":"172.21.15.161:6801/1841901852","cluster_addr":"172.21.15.161:6803/1841901852","heartbeat_back_addr":"172.21.15.161:6807/1841901852","heartbeat_front_addr":"172.21.15.161:6805/1841901852","state":["exists","up"]},{"osd":2,"uuid":"fc4bfd37-1fdd-47c6-a48e-6d910daf9e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6801","nonce":346103099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6803","nonce":346103099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6807","nonce":346103099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6805","nonce":346103099}]},"public_addr":"172.21.15.171:6801/346103099","cluster_addr":"172.21.15.171:6803/346103099","heartbeat_back_addr":"172.21.15.171:6807/346103099","heartbeat_front_addr":"172.21.15.171:6805/346103099","state":["exists","up"]},{"osd":3,"uuid":"4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6811","nonce":2152994143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6813","nonce":2152994143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6817","nonce":2152994143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6815","nonce":2152994143}]},"public_addr":"172.21.15.133:6811/2152994143","cluster_addr":"172.21.15.133:6813/2152994143","heartbeat_back_addr":"172.21.15.133:6817/2152994143","heartbeat_front_addr":"172.21.15.133:6815/2152994143","state":["exists","up"]},{"osd":4,"uuid":"4601a1c2-254b-4c4d-9b15-8fc952b3d2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6809","nonce":47213291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6811","nonce":47213291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6815","nonce":47213291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6813","nonce":47213291}]},"public_addr":"172.21.15.161:6809/47213291","cluster_addr":"172.21.15.161:6811/47213291","heartbeat_back_addr":"172.21.15.161:6815/47213291","heartbeat_front_addr":"172.21.15.161:6813/47213291","state":["exists","up"]},{"osd":5,"uuid":"6cca180a-9140-4e49-835b-86842dae0574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6809","nonce":777556751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6811","nonce":777556751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6815","nonce":777556751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6813","nonce":777556751}]},"public_addr":"172.21.15.171:6809/777556751","cluster_addr":"172.21.15.171:6811/777556751","heartbeat_back_addr":"172.21.15.171:6815/777556751","heartbeat_front_addr":"172.21.15.171:6813/777556751","state":["exists","up"]},{"osd":6,"uuid":"ea0299ff-7740-4494-a3a9-1503b3fc0ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6819","nonce":2884152358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6821","nonce":2884152358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6825","nonce":2884152358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6823","nonce":2884152358}]},"public_addr":"172.21.15.133:6819/2884152358","cluster_addr":"172.21.15.133:6821/2884152358","heartbeat_back_addr":"172.21.15.133:6825/2884152358","heartbeat_front_addr":"172.21.15.133:6823/2884152358","state":["exists","up"]},{"osd":7,"uuid":"b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6817","nonce":4183705749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6819","nonce":4183705749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6823","nonce":4183705749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6821","nonce":4183705749}]},"public_addr":"172.21.15.161:6817/4183705749","cluster_addr":"172.21.15.161:6819/4183705749","heartbeat_back_addr":"172.21.15.161:6823/4183705749","heartbeat_front_addr":"172.21.15.161:6821/4183705749","state":["exists","up"]},{"osd":8,"uuid":"87dffc07-bf48-48c8-b418-5c4b509686d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6817","nonce":2771494483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6819","nonce":2771494483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6823","nonce":2771494483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6821","nonce":2771494483}]},"public_addr":"172.21.15.171:6817/2771494483","cluster_addr":"172.21.15.171:6819/2771494483","heartbeat_back_addr":"172.21.15.171:6823/2771494483","heartbeat_front_addr":"172.21.15.171:6821/2771494483","state":["exists","up"]},{"osd":9,"uuid":"e0832870-79ce-4bce-8045-58b7a7f6dc69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6827","nonce":2066164149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6829","nonce":2066164149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6833","nonce":2066164149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6831","nonce":2066164149}]},"public_addr":"172.21.15.133:6827/2066164149","cluster_addr":"172.21.15.133:6829/2066164149","heartbeat_back_addr":"172.21.15.133:6833/2066164149","heartbeat_front_addr":"172.21.15.133:6831/2066164149","state":["exists","up"]},{"osd":10,"uuid":"081f0484-d134-4b2e-9c48-ef8e6a646422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6825","nonce":2162333626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6827","nonce":2162333626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6831","nonce":2162333626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6829","nonce":2162333626}]},"public_addr":"172.21.15.161:6825/2162333626","cluster_addr":"172.21.15.161:6827/2162333626","heartbeat_back_addr":"172.21.15.161:6831/2162333626","heartbeat_front_addr":"172.21.15.161:6829/2162333626","state":["exists","up"]},{"osd":11,"uuid":"4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6825","nonce":3403894313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6827","nonce":3403894313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6831","nonce":3403894313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6829","nonce":3403894313}]},"public_addr":"172.21.15.171:6825/3403894313","cluster_addr":"172.21.15.171:6827/3403894313","heartbeat_back_addr":"172.21.15.171:6831/3403894313","heartbeat_front_addr":"172.21.15.171:6829/3403894313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:17.816229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:39.638543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:00.543317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:26.988921+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:48.644694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:10.049860+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:35.986508+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:57.753846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:19.018245+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:44.460450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:06.279960+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:27.955006+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/1832569241":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6801/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:0/4219815304":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:6801/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/432253615":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1840700480":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1807775439":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1089030734":"2023-07-15T18:07:47.346704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T18:15:51.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.000+0000 7fbe9c5ff700 1 -- 172.21.15.133:0/1653812202 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fbe800a1220 msgr2=0x7fbe800a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:51.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.000+0000 7fbe9c5ff700 1 --2- 172.21.15.133:0/1653812202 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fbe800a1220 0x7fbe800a36d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbe94199140 tx=0x7fbe90009260 comp rx=0 tx=0).stop 2023-07-14T18:15:51.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.000+0000 7fbe9c5ff700 1 -- 172.21.15.133:0/1653812202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe94100e20 msgr2=0x7fbe941981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:51.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.000+0000 7fbe9c5ff700 1 --2- 172.21.15.133:0/1653812202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe94100e20 0x7fbe941981b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400e9c0 tx=0x7fbe8400ecd0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.000+0000 7fbe9c5ff700 1 -- 172.21.15.133:0/1653812202 shutdown_connections 2023-07-14T18:15:51.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.000+0000 7fbe9c5ff700 1 --2- 172.21.15.133:0/1653812202 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fbe800a1220 0x7fbe800a36d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.000+0000 7fbe9c5ff700 1 --2- 172.21.15.133:0/1653812202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe940ffc30 0x7fbe94197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.000+0000 7fbe9c5ff700 1 --2- 172.21.15.133:0/1653812202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe941017b0 0x7fbe9419f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.000+0000 7fbe9c5ff700 1 --2- 172.21.15.133:0/1653812202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe94100e20 0x7fbe941981b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.000+0000 7fbe9c5ff700 1 -- 172.21.15.133:0/1653812202 >> 172.21.15.133:0/1653812202 conn(0x7fbe940fb210 msgr2=0x7fbe940fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:51.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.000+0000 7fbe9c5ff700 1 -- 172.21.15.133:0/1653812202 shutdown_connections 2023-07-14T18:15:51.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.000+0000 7fbe9c5ff700 1 -- 172.21.15.133:0/1653812202 wait complete. 2023-07-14T18:15:51.064 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T18:15:51.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.217+0000 7f66f01ff700 1 -- 172.21.15.133:0/3987421014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66e80ffc30 msgr2=0x7f66e8100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:51.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.217+0000 7f66f01ff700 1 --2- 172.21.15.133:0/3987421014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66e80ffc30 0x7f66e8100040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f66e4009990 tx=0x7f66e4009ca0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.217+0000 7f66f01ff700 1 -- 172.21.15.133:0/3987421014 shutdown_connections 2023-07-14T18:15:51.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.217+0000 7f66f01ff700 1 --2- 172.21.15.133:0/3987421014 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66e81017b0 0x7f66e8108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.217+0000 7f66f01ff700 1 --2- 172.21.15.133:0/3987421014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66e80ffc30 0x7f66e8100040 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.217+0000 7f66f01ff700 1 --2- 172.21.15.133:0/3987421014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66e8100e20 0x7f66e8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.217+0000 7f66f01ff700 1 -- 172.21.15.133:0/3987421014 >> 172.21.15.133:0/3987421014 conn(0x7f66e80fb210 msgr2=0x7f66e80fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:51.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.217+0000 7f66f01ff700 1 -- 172.21.15.133:0/3987421014 shutdown_connections 2023-07-14T18:15:51.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.217+0000 7f66f01ff700 1 -- 172.21.15.133:0/3987421014 wait complete. 2023-07-14T18:15:51.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.219+0000 7f66f01ff700 1 Processor -- start 2023-07-14T18:15:51.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.219+0000 7f66f01ff700 1 -- start start 2023-07-14T18:15:51.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.219+0000 7f66f01ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66e80ffc30 0x7f66e81958f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:51.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.219+0000 7f66f01ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66e8100e20 0x7f66e8195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:51.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.219+0000 7f66f01ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66e81017b0 0x7f66e819cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:51.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.219+0000 7f66f01ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e819d470 con 0x7f66e8100e20 2023-07-14T18:15:51.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.219+0000 7f66f01ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e819d5b0 con 0x7f66e80ffc30 2023-07-14T18:15:51.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.219+0000 7f66f01ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e819d740 con 0x7f66e81017b0 2023-07-14T18:15:51.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.220+0000 7f66ed79a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66e8100e20 0x7f66e8195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:51.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.220+0000 7f66edf9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66e80ffc30 0x7f66e81958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:51.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.220+0000 7f66ed79a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66e8100e20 0x7f66e8195e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60934/0 (socket says 172.21.15.133:60934) 2023-07-14T18:15:51.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.220+0000 7f66ed79a700 1 -- 172.21.15.133:0/3070276312 learned_addr learned my addr 172.21.15.133:0/3070276312 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:51.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.220+0000 7f66ee79c700 1 --2- 172.21.15.133:0/3070276312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66e81017b0 0x7f66e819cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:51.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.220+0000 7f66ed79a700 1 -- 172.21.15.133:0/3070276312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66e81017b0 msgr2=0x7f66e819cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:51.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.220+0000 7f66ed79a700 1 --2- 172.21.15.133:0/3070276312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66e81017b0 0x7f66e819cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.220+0000 7f66ed79a700 1 -- 172.21.15.133:0/3070276312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66e80ffc30 msgr2=0x7f66e81958f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:51.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.220+0000 7f66ed79a700 1 --2- 172.21.15.133:0/3070276312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66e80ffc30 0x7f66e81958f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.220+0000 7f66ed79a700 1 -- 172.21.15.133:0/3070276312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e40095f0 con 0x7f66e8100e20 2023-07-14T18:15:51.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.220+0000 7f66ed79a700 1 --2- 172.21.15.133:0/3070276312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66e8100e20 0x7f66e8195e30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f66d800e9a0 tx=0x7f66d800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:51.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.220+0000 7f66deffd700 1 -- 172.21.15.133:0/3070276312 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d8009790 con 0x7f66e8100e20 2023-07-14T18:15:51.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.220+0000 7f66deffd700 1 -- 172.21.15.133:0/3070276312 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66d8010430 con 0x7f66e8100e20 2023-07-14T18:15:51.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.220+0000 7f66f01ff700 1 -- 172.21.15.133:0/3070276312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e819da20 con 0x7f66e8100e20 2023-07-14T18:15:51.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.220+0000 7f66deffd700 1 -- 172.21.15.133:0/3070276312 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d8018680 con 0x7f66e8100e20 2023-07-14T18:15:51.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.221+0000 7f66f01ff700 1 -- 172.21.15.133:0/3070276312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e819df30 con 0x7f66e8100e20 2023-07-14T18:15:51.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.222+0000 7f66f01ff700 1 -- 172.21.15.133:0/3070276312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e810b400 con 0x7f66e8100e20 2023-07-14T18:15:51.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.223+0000 7f66deffd700 1 -- 172.21.15.133:0/3070276312 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f66d80105a0 con 0x7f66e8100e20 2023-07-14T18:15:51.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.225+0000 7f66deffd700 1 --2- 172.21.15.133:0/3070276312 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f66d40a1220 0x7f66d40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:51.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.225+0000 7f66deffd700 1 -- 172.21.15.133:0/3070276312 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f66d8014070 con 0x7f66e8100e20 2023-07-14T18:15:51.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.225+0000 7f66edf9b700 1 --2- 172.21.15.133:0/3070276312 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f66d40a1220 0x7f66d40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:51.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.225+0000 7f66edf9b700 1 --2- 172.21.15.133:0/3070276312 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f66d40a1220 0x7f66d40a36d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f66e400b2e0 tx=0x7f66e4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:51.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.228+0000 7f66deffd700 1 -- 172.21.15.133:0/3070276312 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f66d8051780 con 0x7f66e8100e20 2023-07-14T18:15:51.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:50 smithi161 ceph-mon[128993]: pgmap v322: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-07-14T18:15:51.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:50 smithi161 ceph-mon[128993]: osdmap e92: 12 total, 12 up, 12 in 2023-07-14T18:15:51.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:50 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1104058552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:15:51.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:50 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2579061484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:15:51.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:50 smithi171 ceph-mon[129241]: pgmap v322: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-07-14T18:15:51.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:50 smithi171 ceph-mon[129241]: osdmap e92: 12 total, 12 up, 12 in 2023-07-14T18:15:51.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:50 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1104058552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:15:51.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:50 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2579061484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:15:51.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:50 smithi133 ceph-mon[126384]: pgmap v322: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-07-14T18:15:51.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:50 smithi133 ceph-mon[126384]: osdmap e92: 12 total, 12 up, 12 in 2023-07-14T18:15:51.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:50 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1104058552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:15:51.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:50 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2579061484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:15:51.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.470+0000 7f66f01ff700 1 -- 172.21.15.133:0/3070276312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66e810b5c0 con 0x7f66e8100e20 2023-07-14T18:15:51.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.471+0000 7f66deffd700 1 -- 172.21.15.133:0/3070276312 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10851 (secure 0 0 0) 0x7f66d8096250 con 0x7f66e8100e20 2023-07-14T18:15:51.516 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:15:51.516 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3168151271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3168151271},{"type":"v1","addr":"172.21.15.133:6837","nonce":3168151271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14544,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/3093144261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3093144261},{"type":"v1","addr":"172.21.15.133:6839","nonce":3093144261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14556,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/3282902068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":3282902068},{"type":"v1","addr":"172.21.15.133:6841","nonce":3282902068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/4023127259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4023127259},{"type":"v1","addr":"172.21.15.161:6833","nonce":4023127259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2722373740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2722373740},{"type":"v1","addr":"172.21.15.171:6833","nonce":2722373740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2399005074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2399005074},{"type":"v1","addr":"172.21.15.133:6835","nonce":2399005074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T18:14:17.440873+0000","modified":"2023-07-14T18:14:50.886364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24557,"mds_1":24541,"mds_2":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3725502564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3725502564},{"type":"v1","addr":"172.21.15.161:6835","nonce":3725502564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6835/1877362116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1877362116},{"type":"v1","addr":"172.21.15.171:6835","nonce":1877362116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/3458064798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3458064798},{"type":"v1","addr":"172.21.15.161:6837","nonce":3458064798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/106205332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":106205332},{"type":"v1","addr":"172.21.15.171:6837","nonce":106205332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.171:6839/816201198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":816201198},{"type":"v1","addr":"172.21.15.171:6839","nonce":816201198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/2503865871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2503865871},{"type":"v1","addr":"172.21.15.161:6839","nonce":2503865871}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T18:15:51.518 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.473+0000 7f66f01ff700 1 -- 172.21.15.133:0/3070276312 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f66d40a1220 msgr2=0x7f66d40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:51.518 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.473+0000 7f66f01ff700 1 --2- 172.21.15.133:0/3070276312 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f66d40a1220 0x7f66d40a36d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f66e400b2e0 tx=0x7f66e4005960 comp rx=0 tx=0).stop 2023-07-14T18:15:51.518 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.473+0000 7f66f01ff700 1 -- 172.21.15.133:0/3070276312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66e8100e20 msgr2=0x7f66e8195e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:51.518 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.473+0000 7f66f01ff700 1 --2- 172.21.15.133:0/3070276312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66e8100e20 0x7f66e8195e30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f66d800e9a0 tx=0x7f66d800ecb0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.519 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.473+0000 7f66f01ff700 1 -- 172.21.15.133:0/3070276312 shutdown_connections 2023-07-14T18:15:51.519 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.473+0000 7f66f01ff700 1 --2- 172.21.15.133:0/3070276312 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f66d40a1220 0x7f66d40a36d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.519 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.473+0000 7f66f01ff700 1 --2- 172.21.15.133:0/3070276312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66e80ffc30 0x7f66e81958f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.519 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.473+0000 7f66f01ff700 1 --2- 172.21.15.133:0/3070276312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66e81017b0 0x7f66e819cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.519 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.473+0000 7f66f01ff700 1 --2- 172.21.15.133:0/3070276312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66e8100e20 0x7f66e8195e30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.519 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.473+0000 7f66f01ff700 1 -- 172.21.15.133:0/3070276312 >> 172.21.15.133:0/3070276312 conn(0x7f66e80fb210 msgr2=0x7f66e80fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:51.519 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.474+0000 7f66f01ff700 1 -- 172.21.15.133:0/3070276312 shutdown_connections 2023-07-14T18:15:51.520 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.474+0000 7f66f01ff700 1 -- 172.21.15.133:0/3070276312 wait complete. 2023-07-14T18:15:51.520 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 25 2023-07-14T18:15:51.537 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-07-14T18:14:17.440873+0000', 'modified': '2023-07-14T18:14:50.886364+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24557, 'mds_1': 24541, 'mds_2': 24517}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24494': {'gid': 24494, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6835/3725502564', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 3725502564}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 3725502564}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24517': {'gid': 24517, 'name': 'f', 'rank': 2, 'incarnation': 23, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.171:6835/1877362116', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6834', 'nonce': 1877362116}, {'type': 'v1', 'addr': '172.21.15.171:6835', 'nonce': 1877362116}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24521': {'gid': 24521, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6837/3458064798', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 3458064798}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 3458064798}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24529': {'gid': 24529, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6837/106205332', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6836', 'nonce': 106205332}, {'type': 'v1', 'addr': '172.21.15.171:6837', 'nonce': 106205332}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24541': {'gid': 24541, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.171:6839/816201198', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6838', 'nonce': 816201198}, {'type': 'v1', 'addr': '172.21.15.171:6839', 'nonce': 816201198}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24557': {'gid': 24557, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6839/2503865871', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 2503865871}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 2503865871}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-14T18:15:51.538 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-07-14T18:15:51.539 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-07-14T18:15:51.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.692+0000 7fea63bff700 1 -- 172.21.15.133:0/1665215012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea5c0ffc10 msgr2=0x7fea5c100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:51.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.692+0000 7fea63bff700 1 --2- 172.21.15.133:0/1665215012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea5c0ffc10 0x7fea5c100020 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fea58009990 tx=0x7fea58009ca0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.692+0000 7fea63bff700 1 -- 172.21.15.133:0/1665215012 shutdown_connections 2023-07-14T18:15:51.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.692+0000 7fea63bff700 1 --2- 172.21.15.133:0/1665215012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea5c101790 0x7fea5c108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.692+0000 7fea63bff700 1 --2- 172.21.15.133:0/1665215012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea5c0ffc10 0x7fea5c100020 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.692+0000 7fea63bff700 1 --2- 172.21.15.133:0/1665215012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea5c100e00 0x7fea5c101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.692+0000 7fea63bff700 1 -- 172.21.15.133:0/1665215012 >> 172.21.15.133:0/1665215012 conn(0x7fea5c0fb210 msgr2=0x7fea5c0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:51.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.692+0000 7fea63bff700 1 -- 172.21.15.133:0/1665215012 shutdown_connections 2023-07-14T18:15:51.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.692+0000 7fea63bff700 1 -- 172.21.15.133:0/1665215012 wait complete. 2023-07-14T18:15:51.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.693+0000 7fea63bff700 1 Processor -- start 2023-07-14T18:15:51.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.693+0000 7fea63bff700 1 -- start start 2023-07-14T18:15:51.743 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.693+0000 7fea63bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea5c0ffc10 0x7fea5c197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:51.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.693+0000 7fea63bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea5c100e00 0x7fea5c1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:51.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.694+0000 7fea63bff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea5c101790 0x7fea5c19f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:51.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.694+0000 7fea63bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c19f730 con 0x7fea5c100e00 2023-07-14T18:15:51.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.694+0000 7fea63bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c19f870 con 0x7fea5c0ffc10 2023-07-14T18:15:51.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.694+0000 7fea63bff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c19fa00 con 0x7fea5c101790 2023-07-14T18:15:51.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.694+0000 7fea6119a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea5c100e00 0x7fea5c1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:51.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.694+0000 7fea6119a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea5c100e00 0x7fea5c1980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60946/0 (socket says 172.21.15.133:60946) 2023-07-14T18:15:51.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.694+0000 7fea6119a700 1 -- 172.21.15.133:0/188531062 learned_addr learned my addr 172.21.15.133:0/188531062 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:51.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.694+0000 7fea6199b700 1 --2- 172.21.15.133:0/188531062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea5c0ffc10 0x7fea5c197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:51.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.694+0000 7fea6219c700 1 --2- 172.21.15.133:0/188531062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea5c101790 0x7fea5c19f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:51.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.694+0000 7fea6119a700 1 -- 172.21.15.133:0/188531062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea5c101790 msgr2=0x7fea5c19f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:51.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.694+0000 7fea6119a700 1 --2- 172.21.15.133:0/188531062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea5c101790 0x7fea5c19f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.694+0000 7fea6119a700 1 -- 172.21.15.133:0/188531062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea5c0ffc10 msgr2=0x7fea5c197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:51.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.694+0000 7fea6119a700 1 --2- 172.21.15.133:0/188531062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea5c0ffc10 0x7fea5c197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.694+0000 7fea6119a700 1 -- 172.21.15.133:0/188531062 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea580095f0 con 0x7fea5c100e00 2023-07-14T18:15:51.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.694+0000 7fea6119a700 1 --2- 172.21.15.133:0/188531062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea5c100e00 0x7fea5c1980f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fea4c00e9a0 tx=0x7fea4c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:51.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.695+0000 7fea52ffd700 1 -- 172.21.15.133:0/188531062 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c009790 con 0x7fea5c100e00 2023-07-14T18:15:51.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.695+0000 7fea52ffd700 1 -- 172.21.15.133:0/188531062 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea4c00cbd0 con 0x7fea5c100e00 2023-07-14T18:15:51.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.695+0000 7fea63bff700 1 -- 172.21.15.133:0/188531062 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea5c19fce0 con 0x7fea5c100e00 2023-07-14T18:15:51.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.695+0000 7fea52ffd700 1 -- 172.21.15.133:0/188531062 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c018680 con 0x7fea5c100e00 2023-07-14T18:15:51.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.695+0000 7fea63bff700 1 -- 172.21.15.133:0/188531062 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea5c1a01e0 con 0x7fea5c100e00 2023-07-14T18:15:51.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.699+0000 7fea52ffd700 1 -- 172.21.15.133:0/188531062 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fea4c00cd40 con 0x7fea5c100e00 2023-07-14T18:15:51.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.700+0000 7fea7760bb80 1 Processor -- start 2023-07-14T18:15:51.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.700+0000 7fea7760bb80 1 -- start start 2023-07-14T18:15:51.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.701+0000 7fea52ffd700 1 --2- 172.21.15.133:0/188531062 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fea480a1260 0x7fea480a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:51.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.701+0000 7fea7760bb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fde62d1f10 0x55fde62d2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:51.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.701+0000 7fea7760bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fde62d2860 0x55fde62d2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:51.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.701+0000 7fea7760bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde62d3460 0x55fde62d38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:51.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.701+0000 7fea7760bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde61d6c60 con 0x55fde62d3460 2023-07-14T18:15:51.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.701+0000 7fea7760bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde62299f0 con 0x55fde62d2860 2023-07-14T18:15:51.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.701+0000 7fea7760bb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde6222e80 con 0x55fde62d1f10 2023-07-14T18:15:51.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.701+0000 7fea52ffd700 1 -- 172.21.15.133:0/188531062 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fea4c014070 con 0x7fea5c100e00 2023-07-14T18:15:51.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.701+0000 7fea6199b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fde62d1f10 0x55fde62d2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:51.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.701+0000 7fea6219c700 1 --2- 172.21.15.133:0/188531062 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fea480a1260 0x7fea480a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:51.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.701+0000 7fea6119a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fde62d2860 0x55fde62d2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:51.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.701+0000 7fea6199b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fde62d1f10 0x55fde62d2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:39634/0 (socket says 172.21.15.133:39634) 2023-07-14T18:15:51.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.701+0000 7fea6199b700 1 -- 172.21.15.133:0/26922773 learned_addr learned my addr 172.21.15.133:0/26922773 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:51.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.701+0000 7fea6219c700 1 --2- 172.21.15.133:0/26922773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde62d3460 0x55fde62d38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:51.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.702+0000 7fea6199b700 1 -- 172.21.15.133:0/26922773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fde62d2860 msgr2=0x55fde62d2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:15:51.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.702+0000 7fea6199b700 1 --2- 172.21.15.133:0/26922773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fde62d2860 0x55fde62d2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.702+0000 7fea6199b700 1 -- 172.21.15.133:0/26922773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde62d3460 msgr2=0x55fde62d38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:15:51.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.702+0000 7fea6199b700 1 --2- 172.21.15.133:0/26922773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde62d3460 0x55fde62d38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.702+0000 7fea6199b700 1 -- 172.21.15.133:0/26922773 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fde6050d90 con 0x55fde62d1f10 2023-07-14T18:15:51.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.702+0000 7fea6119a700 1 --2- 172.21.15.133:0/26922773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fde62d2860 0x55fde62d2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:51.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.702+0000 7fea6219c700 1 --2- 172.21.15.133:0/188531062 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fea480a1260 0x7fea480a3710 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fea54015da0 tx=0x7fea54015420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:51.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.702+0000 7fea6199b700 1 --2- 172.21.15.133:0/26922773 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fde62d1f10 0x55fde62d2320 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fea5c100c60 tx=0x7fea58005ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68e7be94e98509ec server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:51.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.702+0000 7fea47fff700 1 -- 172.21.15.133:0/26922773 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5801d070 con 0x55fde62d1f10 2023-07-14T18:15:51.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.702+0000 7fea47fff700 1 -- 172.21.15.133:0/26922773 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea5801e040 con 0x55fde62d1f10 2023-07-14T18:15:51.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.702+0000 7fea47fff700 1 -- 172.21.15.133:0/26922773 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea58004880 con 0x55fde62d1f10 2023-07-14T18:15:51.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.703+0000 7fea7760bb80 1 -- 172.21.15.133:0/26922773 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fde62d1f10 msgr2=0x55fde62d2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:51.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.703+0000 7fea7760bb80 1 --2- 172.21.15.133:0/26922773 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fde62d1f10 0x55fde62d2320 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fea5c100c60 tx=0x7fea58005ea0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.703+0000 7fea7760bb80 1 -- 172.21.15.133:0/26922773 shutdown_connections 2023-07-14T18:15:51.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.703+0000 7fea7760bb80 1 --2- 172.21.15.133:0/26922773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde62d3460 0x55fde62d38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.703+0000 7fea7760bb80 1 --2- 172.21.15.133:0/26922773 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fde62d1f10 0x55fde62d2320 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.703+0000 7fea7760bb80 1 --2- 172.21.15.133:0/26922773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fde62d2860 0x55fde62d2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.703+0000 7fea7760bb80 1 -- 172.21.15.133:0/26922773 >> 172.21.15.133:0/26922773 conn(0x55fde62c7630 msgr2=0x55fde62cc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:51.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.703+0000 7fea7760bb80 1 -- 172.21.15.133:0/26922773 shutdown_connections 2023-07-14T18:15:51.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.703+0000 7fea7760bb80 1 -- 172.21.15.133:0/26922773 wait complete. 2023-07-14T18:15:51.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.704+0000 7fea7760bb80 1 -- start start 2023-07-14T18:15:51.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.704+0000 7fea7760bb80 1 Processor -- start 2023-07-14T18:15:51.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.704+0000 7fea7760bb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fde62d1f10 0x55fde62db860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:51.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.704+0000 7fea7760bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fde62d2860 0x55fde62dbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:51.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.704+0000 7fea7760bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde62d3460 0x55fde62e2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:51.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.704+0000 7fea7760bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde61c2a70 con 0x55fde62d3460 2023-07-14T18:15:51.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.704+0000 7fea7760bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde61c2770 con 0x55fde62d2860 2023-07-14T18:15:51.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.704+0000 7fea7760bb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde60a68d0 con 0x55fde62d1f10 2023-07-14T18:15:51.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.704+0000 7fea7760bb80 10 client.0 _ll_register_callbacks cb 0x55fde61c0710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:15:51.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.705+0000 7fea6219c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde62d3460 0x55fde62e2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:51.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.705+0000 7fea6119a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fde62d2860 0x55fde62dbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:51.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.705+0000 7fea6219c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde62d3460 0x55fde62e2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60950/0 (socket says 172.21.15.133:60950) 2023-07-14T18:15:51.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.705+0000 7fea6119a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fde62d2860 0x55fde62dbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:48760/0 (socket says 172.21.15.133:48760) 2023-07-14T18:15:51.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.705+0000 7fea6219c700 1 -- 172.21.15.133:0/3745597071 learned_addr learned my addr 172.21.15.133:0/3745597071 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:51.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.705+0000 7fea6199b700 1 --2- 172.21.15.133:0/3745597071 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fde62d1f10 0x55fde62db860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:51.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.705+0000 7fea6119a700 1 -- 172.21.15.133:0/3745597071 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fde62d1f10 msgr2=0x55fde62db860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:15:51.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.705+0000 7fea6119a700 1 --2- 172.21.15.133:0/3745597071 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fde62d1f10 0x55fde62db860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.705+0000 7fea6119a700 1 -- 172.21.15.133:0/3745597071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde62d3460 msgr2=0x55fde62e2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:15:51.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.705+0000 7fea6119a700 1 --2- 172.21.15.133:0/3745597071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde62d3460 0x55fde62e2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.705+0000 7fea6119a700 1 -- 172.21.15.133:0/3745597071 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea5c1a01e0 con 0x55fde62d2860 2023-07-14T18:15:51.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.705+0000 7fea6219c700 1 --2- 172.21.15.133:0/3745597071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde62d3460 0x55fde62e2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:51.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.705+0000 7fea6119a700 1 --2- 172.21.15.133:0/3745597071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fde62d2860 0x55fde62dbda0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fea4c0ca670 tx=0x7fea4c099ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dbbafaee3291ecd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:51.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.705+0000 7fea457fa700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:15:51.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.705+0000 7fea457fa700 1 -- 172.21.15.133:0/3745597071 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c0a23c0 con 0x55fde62d2860 2023-07-14T18:15:51.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.705+0000 7fea7760bb80 1 -- 172.21.15.133:0/3745597071 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fde617c4a0 con 0x55fde62d2860 2023-07-14T18:15:51.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.706+0000 7fea457fa700 1 -- 172.21.15.133:0/3745597071 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fea4c09a1d0 con 0x55fde62d2860 2023-07-14T18:15:51.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.706+0000 7fea457fa700 1 -- 172.21.15.133:0/3745597071 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea4c09a390 con 0x55fde62d2860 2023-07-14T18:15:51.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.706+0000 7fea7760bb80 10 client.25109 fetch_fsmap learned FSMap version 25 2023-07-14T18:15:51.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.706+0000 7fea7760bb80 1 -- 172.21.15.133:0/3745597071 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fea5c108980 con 0x55fde62d2860 2023-07-14T18:15:51.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.706+0000 7fea457fa700 1 -- 172.21.15.133:0/3745597071 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fde617c4a0 con 0x55fde62d2860 2023-07-14T18:15:51.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.706+0000 7fea457fa700 1 -- 172.21.15.133:0/3745597071 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fea4c0a1fc0 con 0x55fde62d2860 2023-07-14T18:15:51.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.706+0000 7fea7760bb80 10 client.25109 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T18:15:51.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.706+0000 7fea7760bb80 10 client.25109 resolve_mds: resolved name 'f' to daemon mds.f 2023-07-14T18:15:51.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.706+0000 7fea7760bb80 20 client.25109 populate_metadata read hostname 'smithi133' 2023-07-14T18:15:51.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.707+0000 7fea7760bb80 1 --2- 172.21.15.133:0/3745597071 >> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] conn(0x55fde62e6b00 0x55fde62e5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:51.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.707+0000 7fea7760bb80 4 client.25109 mds_command: new command op to 24517 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:15:51.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.707+0000 7fea7760bb80 1 -- 172.21.15.133:0/3745597071 --> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fde62a0d50 con 0x55fde62e6b00 2023-07-14T18:15:51.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.707+0000 7fea6199b700 1 --2- 172.21.15.133:0/3745597071 >> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] conn(0x55fde62e6b00 0x55fde62e5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:51.764 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.707+0000 7fea6199b700 1 --2- 172.21.15.133:0/3745597071 >> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] conn(0x55fde62e6b00 0x55fde62e5ef0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=addf39337ccea2ed server_cookie=78c0af34015c0523 in_seq=0 out_seq=0 2023-07-14T18:15:51.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.707+0000 7fea457fa700 10 client.25109 ms_handle_connect on v2:172.21.15.171:6834/1877362116 2023-07-14T18:15:51.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.708+0000 7fea457fa700 1 -- 172.21.15.133:0/3745597071 <== mds.2 v2:172.21.15.171:6834/1877362116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55fde62a0d50 con 0x55fde62e6b00 2023-07-14T18:15:51.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.708+0000 7fea457fa700 10 client.25109 handle_command_reply: tid=0 2023-07-14T18:15:51.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.708+0000 7fea7760bb80 1 client.25109 shutdown 2023-07-14T18:15:51.766 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.708+0000 7fea7760bb80 1 -- 172.21.15.133:0/3745597071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fde62d2860 msgr2=0x55fde62dbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:51.766 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.708+0000 7fea7760bb80 1 --2- 172.21.15.133:0/3745597071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fde62d2860 0x55fde62dbda0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fea4c0ca670 tx=0x7fea4c099ff0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.766 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.708+0000 7fea7760bb80 1 -- 172.21.15.133:0/3745597071 shutdown_connections 2023-07-14T18:15:51.766 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.708+0000 7fea7760bb80 1 --2- 172.21.15.133:0/3745597071 >> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] conn(0x55fde62e6b00 0x55fde62e5ef0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.767 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.708+0000 7fea7760bb80 1 --2- 172.21.15.133:0/3745597071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde62d3460 0x55fde62e2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.767 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.708+0000 7fea7760bb80 1 --2- 172.21.15.133:0/3745597071 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fde62d1f10 0x55fde62db860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.767 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.708+0000 7fea457fa700 0 client.25109 ms_handle_reset on v2:172.21.15.171:6834/1877362116 2023-07-14T18:15:51.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.708+0000 7fea7760bb80 1 --2- 172.21.15.133:0/3745597071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fde62d2860 0x55fde62dbda0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.709+0000 7fea7760bb80 1 -- 172.21.15.133:0/3745597071 >> 172.21.15.133:0/3745597071 conn(0x55fde62c7630 msgr2=0x55fde62cb600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:51.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.709+0000 7fea7760bb80 1 -- 172.21.15.133:0/3745597071 shutdown_connections 2023-07-14T18:15:51.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.709+0000 7fea7760bb80 1 -- 172.21.15.133:0/3745597071 wait complete. 2023-07-14T18:15:51.769 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.709+0000 7fea7760bb80 20 client.25109 trim_cache size 0 max 16384 2023-07-14T18:15:51.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.726+0000 7fea7760bb80 1 Processor -- start 2023-07-14T18:15:51.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.726+0000 7fea7760bb80 1 -- start start 2023-07-14T18:15:51.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.727+0000 7fea7760bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fde62d1f10 0x55fde62cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:51.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.727+0000 7fea7760bb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fde62d2860 0x55fde62e2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:51.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.727+0000 7fea7760bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde62d3460 0x55fde62e5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:51.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.727+0000 7fea7760bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde61e1480 con 0x55fde62d3460 2023-07-14T18:15:51.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.727+0000 7fea7760bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde61bed50 con 0x55fde62d1f10 2023-07-14T18:15:51.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.727+0000 7fea7760bb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde62e4ad0 con 0x55fde62d2860 2023-07-14T18:15:51.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.727+0000 7fea6219c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde62d3460 0x55fde62e5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:51.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.727+0000 7fea6219c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde62d3460 0x55fde62e5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60952/0 (socket says 172.21.15.133:60952) 2023-07-14T18:15:51.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.727+0000 7fea6219c700 1 -- 172.21.15.133:0/3486024083 learned_addr learned my addr 172.21.15.133:0/3486024083 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:51.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.727+0000 7fea6119a700 1 --2- 172.21.15.133:0/3486024083 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fde62d2860 0x55fde62e2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:51.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.727+0000 7fea6199b700 1 --2- 172.21.15.133:0/3486024083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fde62d1f10 0x55fde62cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:51.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.727+0000 7fea6219c700 1 -- 172.21.15.133:0/3486024083 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fde62d2860 msgr2=0x55fde62e2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:15:51.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.727+0000 7fea6219c700 1 --2- 172.21.15.133:0/3486024083 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fde62d2860 0x55fde62e2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.727+0000 7fea6219c700 1 -- 172.21.15.133:0/3486024083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fde62d1f10 msgr2=0x55fde62cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:15:51.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.727+0000 7fea6219c700 1 --2- 172.21.15.133:0/3486024083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fde62d1f10 0x55fde62cc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.727+0000 7fea6219c700 1 -- 172.21.15.133:0/3486024083 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fde6039c10 con 0x55fde62d3460 2023-07-14T18:15:51.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.727+0000 7fea6119a700 1 --2- 172.21.15.133:0/3486024083 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fde62d2860 0x55fde62e2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:51.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.727+0000 7fea6199b700 1 --2- 172.21.15.133:0/3486024083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fde62d1f10 0x55fde62cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:51.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.728+0000 7fea6219c700 1 --2- 172.21.15.133:0/3486024083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde62d3460 0x55fde62e5ef0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fea54015ee0 tx=0x7fea5401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a75c8218b75ad09 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:51.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.728+0000 7fea50bff700 1 -- 172.21.15.133:0/3486024083 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5400e040 con 0x55fde62d3460 2023-07-14T18:15:51.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.728+0000 7fea50bff700 1 -- 172.21.15.133:0/3486024083 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea54007dd0 con 0x55fde62d3460 2023-07-14T18:15:51.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.728+0000 7fea50bff700 1 -- 172.21.15.133:0/3486024083 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea54027e70 con 0x55fde62d3460 2023-07-14T18:15:51.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.728+0000 7fea7760bb80 1 -- 172.21.15.133:0/3486024083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde62d3460 msgr2=0x55fde62e5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:51.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.728+0000 7fea7760bb80 1 --2- 172.21.15.133:0/3486024083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde62d3460 0x55fde62e5ef0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fea54015ee0 tx=0x7fea5401c000 comp rx=0 tx=0).stop 2023-07-14T18:15:51.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.729+0000 7fea7760bb80 1 -- 172.21.15.133:0/3486024083 shutdown_connections 2023-07-14T18:15:51.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.729+0000 7fea7760bb80 1 --2- 172.21.15.133:0/3486024083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fde62d1f10 0x55fde62cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.729+0000 7fea7760bb80 1 --2- 172.21.15.133:0/3486024083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde62d3460 0x55fde62e5ef0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.729+0000 7fea7760bb80 1 --2- 172.21.15.133:0/3486024083 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fde62d2860 0x55fde62e2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.729+0000 7fea7760bb80 1 -- 172.21.15.133:0/3486024083 >> 172.21.15.133:0/3486024083 conn(0x55fde62c7630 msgr2=0x55fde62c8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:51.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.729+0000 7fea7760bb80 1 -- 172.21.15.133:0/3486024083 shutdown_connections 2023-07-14T18:15:51.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.729+0000 7fea7760bb80 1 -- 172.21.15.133:0/3486024083 wait complete. 2023-07-14T18:15:51.778 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.729+0000 7fea7760bb80 1 -- start start 2023-07-14T18:15:51.778 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.729+0000 7fea7760bb80 1 Processor -- start 2023-07-14T18:15:51.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.730+0000 7fea7760bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde62d1f10 0x55fde62e0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:51.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.730+0000 7fea7760bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fde62d2860 0x55fde6338790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:51.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.730+0000 7fea7760bb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fde62d3460 0x55fde633ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:51.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.730+0000 7fea7760bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde62d9000 con 0x55fde62d1f10 2023-07-14T18:15:51.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.730+0000 7fea7760bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde633b220 con 0x55fde62d2860 2023-07-14T18:15:51.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.730+0000 7fea7760bb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde633b360 con 0x55fde62d3460 2023-07-14T18:15:51.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.730+0000 7fea7760bb80 10 client.0 _ll_register_callbacks cb 0x7fea4c099ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:15:51.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.730+0000 7fea6199b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde62d1f10 0x55fde62e0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:51.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.730+0000 7fea6199b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde62d1f10 0x55fde62e0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60964/0 (socket says 172.21.15.133:60964) 2023-07-14T18:15:51.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.730+0000 7fea6199b700 1 -- 172.21.15.133:0/221623995 learned_addr learned my addr 172.21.15.133:0/221623995 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:51.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.730+0000 7fea6119a700 1 --2- 172.21.15.133:0/221623995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fde62d2860 0x55fde6338790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:51.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.730+0000 7fea6219c700 1 --2- 172.21.15.133:0/221623995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fde62d3460 0x55fde633ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:51.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.730+0000 7fea6199b700 1 -- 172.21.15.133:0/221623995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fde62d3460 msgr2=0x55fde633ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:15:51.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.730+0000 7fea6199b700 1 --2- 172.21.15.133:0/221623995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fde62d3460 0x55fde633ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.730+0000 7fea6199b700 1 -- 172.21.15.133:0/221623995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fde62d2860 msgr2=0x55fde6338790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:15:51.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.730+0000 7fea6199b700 1 --2- 172.21.15.133:0/221623995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fde62d2860 0x55fde6338790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.730+0000 7fea6199b700 1 -- 172.21.15.133:0/221623995 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fde6050d90 con 0x55fde62d1f10 2023-07-14T18:15:51.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.731+0000 7fea6199b700 1 --2- 172.21.15.133:0/221623995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde62d1f10 0x55fde62e0d90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fea58005ac0 tx=0x7fea580058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b7fa6d2056e3bc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:51.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.731+0000 7fea457fa700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T18:15:51.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.731+0000 7fea457fa700 1 -- 172.21.15.133:0/221623995 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5803a070 con 0x55fde62d1f10 2023-07-14T18:15:51.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.731+0000 7fea457fa700 1 -- 172.21.15.133:0/221623995 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fea580343e0 con 0x55fde62d1f10 2023-07-14T18:15:51.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.731+0000 7fea7760bb80 1 -- 172.21.15.133:0/221623995 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea5801e040 con 0x55fde62d1f10 2023-07-14T18:15:51.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.731+0000 7fea457fa700 1 -- 172.21.15.133:0/221623995 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea58034070 con 0x55fde62d1f10 2023-07-14T18:15:51.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.731+0000 7fea7760bb80 10 client.15132 fetch_fsmap learned FSMap version 25 2023-07-14T18:15:51.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.731+0000 7fea7760bb80 1 -- 172.21.15.133:0/221623995 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fea5c108980 con 0x55fde62d1f10 2023-07-14T18:15:51.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.731+0000 7fea457fa700 1 -- 172.21.15.133:0/221623995 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea5801e040 con 0x55fde62d1f10 2023-07-14T18:15:51.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.731+0000 7fea457fa700 1 -- 172.21.15.133:0/221623995 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fea5803d030 con 0x55fde62d1f10 2023-07-14T18:15:51.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.731+0000 7fea7760bb80 10 client.15132 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T18:15:51.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.732+0000 7fea7760bb80 10 client.15132 resolve_mds: resolved name 'f' to daemon mds.f 2023-07-14T18:15:51.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.732+0000 7fea7760bb80 20 client.15132 populate_metadata read hostname 'smithi133' 2023-07-14T18:15:51.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.732+0000 7fea7760bb80 1 --2- 172.21.15.133:0/221623995 >> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] conn(0x55fde633faf0 0x55fde633c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:51.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.732+0000 7fea7760bb80 4 client.15132 mds_command: new command op to 24517 tid=0[{"prefix": "status"}] 2023-07-14T18:15:51.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.732+0000 7fea7760bb80 1 -- 172.21.15.133:0/221623995 --> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fde621be60 con 0x55fde633faf0 2023-07-14T18:15:51.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.732+0000 7fea6119a700 1 --2- 172.21.15.133:0/221623995 >> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] conn(0x55fde633faf0 0x55fde633c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:51.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.732+0000 7fea6119a700 1 --2- 172.21.15.133:0/221623995 >> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] conn(0x55fde633faf0 0x55fde633c500 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1abac0cbcaa47c0a server_cookie=1c406d25227a1bf9 in_seq=0 out_seq=0 2023-07-14T18:15:51.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.732+0000 7fea457fa700 10 client.15132 ms_handle_connect on v2:172.21.15.171:6834/1877362116 2023-07-14T18:15:51.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.733+0000 7fea457fa700 1 -- 172.21.15.133:0/221623995 <== mds.2 v2:172.21.15.171:6834/1877362116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55fde621be60 con 0x55fde633faf0 2023-07-14T18:15:51.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.733+0000 7fea457fa700 10 client.15132 handle_command_reply: tid=0 2023-07-14T18:15:51.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.733+0000 7fea7760bb80 1 client.15132 shutdown 2023-07-14T18:15:51.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.733+0000 7fea7760bb80 1 -- 172.21.15.133:0/221623995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde62d1f10 msgr2=0x55fde62e0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:51.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.733+0000 7fea7760bb80 1 --2- 172.21.15.133:0/221623995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde62d1f10 0x55fde62e0d90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fea58005ac0 tx=0x7fea580058e0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.733+0000 7fea7760bb80 1 -- 172.21.15.133:0/221623995 shutdown_connections 2023-07-14T18:15:51.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.733+0000 7fea7760bb80 1 --2- 172.21.15.133:0/221623995 >> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] conn(0x55fde633faf0 0x55fde633c500 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.733+0000 7fea7760bb80 1 --2- 172.21.15.133:0/221623995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fde62d3460 0x55fde633ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.733+0000 7fea7760bb80 1 --2- 172.21.15.133:0/221623995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde62d1f10 0x55fde62e0d90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.733+0000 7fea457fa700 0 client.15132 ms_handle_reset on v2:172.21.15.171:6834/1877362116 2023-07-14T18:15:51.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.733+0000 7fea7760bb80 1 --2- 172.21.15.133:0/221623995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fde62d2860 0x55fde6338790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.733+0000 7fea7760bb80 1 -- 172.21.15.133:0/221623995 >> 172.21.15.133:0/221623995 conn(0x55fde62c7630 msgr2=0x55fde62cc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:51.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.733+0000 7fea7760bb80 1 -- 172.21.15.133:0/221623995 shutdown_connections 2023-07-14T18:15:51.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.733+0000 7fea7760bb80 1 -- 172.21.15.133:0/221623995 wait complete. 2023-07-14T18:15:51.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.733+0000 7fea7760bb80 20 client.15132 trim_cache size 0 max 16384 2023-07-14T18:15:51.788 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:15:51.788 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "06d0f5fa-2271-11ee-9b33-001a4aab830c", 2023-07-14T18:15:51.788 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 2, 2023-07-14T18:15:51.788 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24517, 2023-07-14T18:15:51.789 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2023-07-14T18:15:51.789 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2023-07-14T18:15:51.789 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2023-07-14T18:15:51.789 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 61.850496229999997, 2023-07-14T18:15:51.789 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 25, 2023-07-14T18:15:51.789 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 92, 2023-07-14T18:15:51.789 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 90, 2023-07-14T18:15:51.790 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 115.672122307 2023-07-14T18:15:51.790 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:15:51.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.734+0000 7fea63bff700 1 -- 172.21.15.133:0/188531062 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fea480a1260 msgr2=0x7fea480a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:51.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.734+0000 7fea63bff700 1 --2- 172.21.15.133:0/188531062 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fea480a1260 0x7fea480a3710 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fea54015da0 tx=0x7fea54015420 comp rx=0 tx=0).stop 2023-07-14T18:15:51.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.734+0000 7fea63bff700 1 -- 172.21.15.133:0/188531062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea5c100e00 msgr2=0x7fea5c1980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:51.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.734+0000 7fea63bff700 1 --2- 172.21.15.133:0/188531062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea5c100e00 0x7fea5c1980f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fea4c00e9a0 tx=0x7fea4c00ecb0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.734+0000 7fea63bff700 1 -- 172.21.15.133:0/188531062 shutdown_connections 2023-07-14T18:15:51.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.734+0000 7fea63bff700 1 --2- 172.21.15.133:0/188531062 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fea480a1260 0x7fea480a3710 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.734+0000 7fea63bff700 1 --2- 172.21.15.133:0/188531062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea5c0ffc10 0x7fea5c197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.734+0000 7fea63bff700 1 --2- 172.21.15.133:0/188531062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea5c101790 0x7fea5c19f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.734+0000 7fea63bff700 1 --2- 172.21.15.133:0/188531062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea5c100e00 0x7fea5c1980f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.734+0000 7fea63bff700 1 -- 172.21.15.133:0/188531062 >> 172.21.15.133:0/188531062 conn(0x7fea5c0fb210 msgr2=0x7fea5c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:51.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.734+0000 7fea63bff700 1 -- 172.21.15.133:0/188531062 shutdown_connections 2023-07-14T18:15:51.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.734+0000 7fea63bff700 1 -- 172.21.15.133:0/188531062 wait complete. 2023-07-14T18:15:51.797 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-07-14T18:15:51.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.951+0000 7f4f6ddff700 1 -- 172.21.15.133:0/1103646393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f68101790 msgr2=0x7f4f68108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:51.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.951+0000 7f4f6ddff700 1 --2- 172.21.15.133:0/1103646393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f68101790 0x7f4f68108050 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000b0a0 tx=0x7f4f6000b3b0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.951+0000 7f4f6ddff700 1 -- 172.21.15.133:0/1103646393 shutdown_connections 2023-07-14T18:15:51.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.951+0000 7f4f6ddff700 1 --2- 172.21.15.133:0/1103646393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f68101790 0x7f4f68108050 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.951+0000 7f4f6ddff700 1 --2- 172.21.15.133:0/1103646393 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f680ffc10 0x7f4f68100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.951+0000 7f4f6ddff700 1 --2- 172.21.15.133:0/1103646393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f68100e00 0x7f4f68101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:51.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.951+0000 7f4f6ddff700 1 -- 172.21.15.133:0/1103646393 >> 172.21.15.133:0/1103646393 conn(0x7f4f680fb210 msgr2=0x7f4f680fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:51.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.951+0000 7f4f6ddff700 1 -- 172.21.15.133:0/1103646393 shutdown_connections 2023-07-14T18:15:51.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.951+0000 7f4f6ddff700 1 -- 172.21.15.133:0/1103646393 wait complete. 2023-07-14T18:15:51.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.952+0000 7f4f6ddff700 1 Processor -- start 2023-07-14T18:15:51.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.953+0000 7f4f6ddff700 1 -- start start 2023-07-14T18:15:51.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.953+0000 7f4f6ddff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f680ffc10 0x7f4f68197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.953+0000 7f4f6ddff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f68100e00 0x7f4f681981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.953+0000 7f4f6ddff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f68101790 0x7f4f6819f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.953+0000 7f4f6ddff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6819f7f0 con 0x7f4f680ffc10 2023-07-14T18:15:52.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.953+0000 7f4f6ddff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6819f960 con 0x7f4f68100e00 2023-07-14T18:15:52.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.953+0000 7f4f6ddff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6819fb20 con 0x7f4f68101790 2023-07-14T18:15:52.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.953+0000 7f4f677fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f680ffc10 0x7f4f68197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.953+0000 7f4f677fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f680ffc10 0x7f4f68197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60974/0 (socket says 172.21.15.133:60974) 2023-07-14T18:15:52.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.953+0000 7f4f677fe700 1 -- 172.21.15.133:0/178056486 learned_addr learned my addr 172.21.15.133:0/178056486 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:52.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.953+0000 7f4f67fff700 1 --2- 172.21.15.133:0/178056486 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f68101790 0x7f4f6819f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.953+0000 7f4f66ffd700 1 --2- 172.21.15.133:0/178056486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f68100e00 0x7f4f681981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.954+0000 7f4f677fe700 1 -- 172.21.15.133:0/178056486 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f68101790 msgr2=0x7f4f6819f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:52.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.954+0000 7f4f677fe700 1 --2- 172.21.15.133:0/178056486 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f68101790 0x7f4f6819f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.954+0000 7f4f677fe700 1 -- 172.21.15.133:0/178056486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f68100e00 msgr2=0x7f4f681981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:52.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.954+0000 7f4f677fe700 1 --2- 172.21.15.133:0/178056486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f68100e00 0x7f4f681981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.954+0000 7f4f677fe700 1 -- 172.21.15.133:0/178056486 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f60009d00 con 0x7f4f680ffc10 2023-07-14T18:15:52.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.954+0000 7f4f677fe700 1 --2- 172.21.15.133:0/178056486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f680ffc10 0x7f4f68197c80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4f5800e970 tx=0x7f4f5800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:52.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.954+0000 7f4f64ff9700 1 -- 172.21.15.133:0/178056486 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f58009790 con 0x7f4f680ffc10 2023-07-14T18:15:52.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.954+0000 7f4f6ddff700 1 -- 172.21.15.133:0/178056486 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6819fe00 con 0x7f4f680ffc10 2023-07-14T18:15:52.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.954+0000 7f4f64ff9700 1 -- 172.21.15.133:0/178056486 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f5800c7e0 con 0x7f4f680ffc10 2023-07-14T18:15:52.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.954+0000 7f4f64ff9700 1 -- 172.21.15.133:0/178056486 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f58003c70 con 0x7f4f680ffc10 2023-07-14T18:15:52.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.954+0000 7f4f6ddff700 1 -- 172.21.15.133:0/178056486 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f681a0230 con 0x7f4f680ffc10 2023-07-14T18:15:52.011 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.958+0000 7f4f64ff9700 1 -- 172.21.15.133:0/178056486 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f4f5800cc50 con 0x7f4f680ffc10 2023-07-14T18:15:52.011 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.959+0000 7f4f8174cb80 1 Processor -- start 2023-07-14T18:15:52.011 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.959+0000 7f4f8174cb80 1 -- start start 2023-07-14T18:15:52.011 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.959+0000 7f4f8174cb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56026c0d9f10 0x56026c0da320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.959+0000 7f4f8174cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56026c0da860 0x56026c0dae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.959+0000 7f4f8174cb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56026c0db460 0x56026c0db8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.959+0000 7f4f8174cb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56026bfdec60 con 0x56026c0db460 2023-07-14T18:15:52.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.959+0000 7f4f8174cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56026c0319f0 con 0x56026c0da860 2023-07-14T18:15:52.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.959+0000 7f4f8174cb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x56026c02ae80 con 0x56026c0d9f10 2023-07-14T18:15:52.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.960+0000 7f4f67fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56026c0d9f10 0x56026c0da320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.960+0000 7f4f677fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56026c0da860 0x56026c0dae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.960+0000 7f4f67fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56026c0d9f10 0x56026c0da320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:39696/0 (socket says 172.21.15.133:39696) 2023-07-14T18:15:52.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.960+0000 7f4f67fff700 1 -- 172.21.15.133:0/319015518 learned_addr learned my addr 172.21.15.133:0/319015518 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:52.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.960+0000 7f4f66ffd700 1 --2- 172.21.15.133:0/319015518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56026c0db460 0x56026c0db8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.960+0000 7f4f67fff700 1 -- 172.21.15.133:0/319015518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56026c0da860 msgr2=0x56026c0dae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:15:52.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.960+0000 7f4f67fff700 1 --2- 172.21.15.133:0/319015518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56026c0da860 0x56026c0dae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.960+0000 7f4f67fff700 1 -- 172.21.15.133:0/319015518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56026c0db460 msgr2=0x56026c0db8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:15:52.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.960+0000 7f4f67fff700 1 --2- 172.21.15.133:0/319015518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56026c0db460 0x56026c0db8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.960+0000 7f4f67fff700 1 -- 172.21.15.133:0/319015518 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56026be58d90 con 0x56026c0d9f10 2023-07-14T18:15:52.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.960+0000 7f4f66ffd700 1 --2- 172.21.15.133:0/319015518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56026c0db460 0x56026c0db8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:15:52.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.960+0000 7f4f67fff700 1 --2- 172.21.15.133:0/319015518 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56026c0d9f10 0x56026c0da320 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4f600079f0 tx=0x7f4f60009c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b11187b32784958f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:52.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.960+0000 7f4f64ff9700 1 --2- 172.21.15.133:0/178056486 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4f500a11e0 0x7f4f500a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.961+0000 7f4f4dffb700 1 -- 172.21.15.133:0/319015518 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6000e040 con 0x56026c0d9f10 2023-07-14T18:15:52.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.961+0000 7f4f4dffb700 1 -- 172.21.15.133:0/319015518 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f600094a0 con 0x56026c0d9f10 2023-07-14T18:15:52.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.961+0000 7f4f677fe700 1 --2- 172.21.15.133:0/178056486 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4f500a11e0 0x7f4f500a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.961+0000 7f4f4dffb700 1 -- 172.21.15.133:0/319015518 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f60003d80 con 0x56026c0d9f10 2023-07-14T18:15:52.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.961+0000 7f4f64ff9700 1 -- 172.21.15.133:0/178056486 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4f58014070 con 0x7f4f680ffc10 2023-07-14T18:15:52.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.962+0000 7f4f8174cb80 1 -- 172.21.15.133:0/319015518 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56026c0d9f10 msgr2=0x56026c0da320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:52.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.962+0000 7f4f8174cb80 1 --2- 172.21.15.133:0/319015518 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56026c0d9f10 0x56026c0da320 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4f600079f0 tx=0x7f4f60009c60 comp rx=0 tx=0).stop 2023-07-14T18:15:52.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.962+0000 7f4f8174cb80 1 -- 172.21.15.133:0/319015518 shutdown_connections 2023-07-14T18:15:52.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.962+0000 7f4f8174cb80 1 --2- 172.21.15.133:0/319015518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56026c0db460 0x56026c0db8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.962+0000 7f4f8174cb80 1 --2- 172.21.15.133:0/319015518 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56026c0d9f10 0x56026c0da320 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.962+0000 7f4f8174cb80 1 --2- 172.21.15.133:0/319015518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56026c0da860 0x56026c0dae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.962+0000 7f4f8174cb80 1 -- 172.21.15.133:0/319015518 >> 172.21.15.133:0/319015518 conn(0x56026c0cf630 msgr2=0x56026c0d4980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:52.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.962+0000 7f4f8174cb80 1 -- 172.21.15.133:0/319015518 shutdown_connections 2023-07-14T18:15:52.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.962+0000 7f4f8174cb80 1 -- 172.21.15.133:0/319015518 wait complete. 2023-07-14T18:15:52.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.962+0000 7f4f677fe700 1 --2- 172.21.15.133:0/178056486 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4f500a11e0 0x7f4f500a3690 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x56026c0dac70 tx=0x7f4f580d08e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:52.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.962+0000 7f4f8174cb80 1 -- start start 2023-07-14T18:15:52.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.963+0000 7f4f8174cb80 1 Processor -- start 2023-07-14T18:15:52.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.963+0000 7f4f8174cb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56026c0d9f10 0x56026c0e38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.963+0000 7f4f8174cb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56026c0da860 0x56026c0e3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.963+0000 7f4f8174cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56026c0db460 0x56026c0eae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.963+0000 7f4f8174cb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56026bfcaa70 con 0x56026c0da860 2023-07-14T18:15:52.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.963+0000 7f4f8174cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56026bfca770 con 0x56026c0db460 2023-07-14T18:15:52.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.963+0000 7f4f8174cb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x56026beae8d0 con 0x56026c0d9f10 2023-07-14T18:15:52.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.963+0000 7f4f8174cb80 10 client.0 _ll_register_callbacks cb 0x56026bfc8710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:15:52.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.963+0000 7f4f66ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56026c0da860 0x56026c0e3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.963+0000 7f4f66ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56026c0da860 0x56026c0e3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60990/0 (socket says 172.21.15.133:60990) 2023-07-14T18:15:52.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.963+0000 7f4f66ffd700 1 -- 172.21.15.133:0/946473865 learned_addr learned my addr 172.21.15.133:0/946473865 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:52.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.963+0000 7f4f67fff700 1 --2- 172.21.15.133:0/946473865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56026c0db460 0x56026c0eae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.964+0000 7f4f67fff700 1 --2- 172.21.15.133:0/946473865 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56026c0d9f10 0x56026c0e38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.964+0000 7f4f66ffd700 1 -- 172.21.15.133:0/946473865 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56026c0d9f10 msgr2=0x56026c0e38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:15:52.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.964+0000 7f4f66ffd700 1 --2- 172.21.15.133:0/946473865 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56026c0d9f10 0x56026c0e38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.964+0000 7f4f66ffd700 1 -- 172.21.15.133:0/946473865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56026c0db460 msgr2=0x56026c0eae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:15:52.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.964+0000 7f4f66ffd700 1 --2- 172.21.15.133:0/946473865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56026c0db460 0x56026c0eae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.964+0000 7f4f66ffd700 1 -- 172.21.15.133:0/946473865 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f68108980 con 0x56026c0da860 2023-07-14T18:15:52.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.964+0000 7f4f67fff700 1 --2- 172.21.15.133:0/946473865 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56026c0d9f10 0x56026c0e38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:15:52.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.964+0000 7f4f66ffd700 1 --2- 172.21.15.133:0/946473865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56026c0da860 0x56026c0e3e30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c012740 tx=0x7f4f5c012a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52488932e72fa6b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:52.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.964+0000 7f4f46ffd700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T18:15:52.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.965+0000 7f4f46ffd700 1 -- 172.21.15.133:0/946473865 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c017070 con 0x56026c0da860 2023-07-14T18:15:52.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.965+0000 7f4f8174cb80 1 -- 172.21.15.133:0/946473865 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56026bf844a0 con 0x56026c0da860 2023-07-14T18:15:52.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.965+0000 7f4f46ffd700 1 -- 172.21.15.133:0/946473865 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4f5c004050 con 0x56026c0da860 2023-07-14T18:15:52.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.965+0000 7f4f46ffd700 1 -- 172.21.15.133:0/946473865 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f5c004250 con 0x56026c0da860 2023-07-14T18:15:52.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.965+0000 7f4f8174cb80 10 client.15144 fetch_fsmap learned FSMap version 25 2023-07-14T18:15:52.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.965+0000 7f4f8174cb80 1 -- 172.21.15.133:0/946473865 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56026be41c10 con 0x56026c0da860 2023-07-14T18:15:52.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.965+0000 7f4f46ffd700 1 -- 172.21.15.133:0/946473865 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56026bf844a0 con 0x56026c0da860 2023-07-14T18:15:52.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.966+0000 7f4f46ffd700 1 -- 172.21.15.133:0/946473865 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4f5c00f5d0 con 0x56026c0da860 2023-07-14T18:15:52.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.966+0000 7f4f8174cb80 10 client.15144 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T18:15:52.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.966+0000 7f4f8174cb80 10 client.15144 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-14T18:15:52.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.966+0000 7f4f8174cb80 20 client.15144 populate_metadata read hostname 'smithi133' 2023-07-14T18:15:52.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.966+0000 7f4f8174cb80 1 --2- 172.21.15.133:0/946473865 >> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] conn(0x56026c0eebd0 0x56026c0edfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.966+0000 7f4f8174cb80 4 client.15144 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:15:52.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.966+0000 7f4f8174cb80 1 -- 172.21.15.133:0/946473865 --> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56026c0a8d50 con 0x56026c0eebd0 2023-07-14T18:15:52.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.966+0000 7f4f67fff700 1 --2- 172.21.15.133:0/946473865 >> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] conn(0x56026c0eebd0 0x56026c0edfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.966+0000 7f4f67fff700 1 --2- 172.21.15.133:0/946473865 >> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] conn(0x56026c0eebd0 0x56026c0edfd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dc532e1a40a655a6 server_cookie=56910a2b3e229278 in_seq=0 out_seq=0 2023-07-14T18:15:52.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.966+0000 7f4f46ffd700 10 client.15144 ms_handle_connect on v2:172.21.15.171:6838/816201198 2023-07-14T18:15:52.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.967+0000 7f4f46ffd700 1 -- 172.21.15.133:0/946473865 <== mds.1 v2:172.21.15.171:6838/816201198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x56026c0a8d50 con 0x56026c0eebd0 2023-07-14T18:15:52.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.967+0000 7f4f46ffd700 10 client.15144 handle_command_reply: tid=0 2023-07-14T18:15:52.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.967+0000 7f4f8174cb80 1 client.15144 shutdown 2023-07-14T18:15:52.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.968+0000 7f4f8174cb80 1 -- 172.21.15.133:0/946473865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56026c0da860 msgr2=0x56026c0e3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:52.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.968+0000 7f4f8174cb80 1 --2- 172.21.15.133:0/946473865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56026c0da860 0x56026c0e3e30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c012740 tx=0x7f4f5c012a50 comp rx=0 tx=0).stop 2023-07-14T18:15:52.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.968+0000 7f4f8174cb80 1 -- 172.21.15.133:0/946473865 shutdown_connections 2023-07-14T18:15:52.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.968+0000 7f4f8174cb80 1 --2- 172.21.15.133:0/946473865 >> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] conn(0x56026c0eebd0 0x56026c0edfd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.968+0000 7f4f8174cb80 1 --2- 172.21.15.133:0/946473865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56026c0db460 0x56026c0eae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.968+0000 7f4f8174cb80 1 --2- 172.21.15.133:0/946473865 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56026c0d9f10 0x56026c0e38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.968+0000 7f4f46ffd700 0 client.15144 ms_handle_reset on v2:172.21.15.171:6838/816201198 2023-07-14T18:15:52.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.968+0000 7f4f8174cb80 1 --2- 172.21.15.133:0/946473865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56026c0da860 0x56026c0e3e30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.968+0000 7f4f8174cb80 1 -- 172.21.15.133:0/946473865 >> 172.21.15.133:0/946473865 conn(0x56026c0cf630 msgr2=0x56026c0d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:52.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.968+0000 7f4f8174cb80 1 -- 172.21.15.133:0/946473865 shutdown_connections 2023-07-14T18:15:52.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.968+0000 7f4f8174cb80 1 -- 172.21.15.133:0/946473865 wait complete. 2023-07-14T18:15:52.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.968+0000 7f4f8174cb80 20 client.15144 trim_cache size 0 max 16384 2023-07-14T18:15:52.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.985+0000 7f4f8174cb80 1 Processor -- start 2023-07-14T18:15:52.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.985+0000 7f4f8174cb80 1 -- start start 2023-07-14T18:15:52.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.986+0000 7f4f8174cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56026c0d9f10 0x56026c0d40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.986+0000 7f4f8174cb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56026c0da860 0x56026c0eae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.986+0000 7f4f8174cb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56026c0db460 0x56026c0e1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.986+0000 7f4f8174cb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56026c0a8d80 con 0x56026c0da860 2023-07-14T18:15:52.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.986+0000 7f4f8174cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56026bf844d0 con 0x56026c0d9f10 2023-07-14T18:15:52.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.986+0000 7f4f8174cb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x56026bebade0 con 0x56026c0db460 2023-07-14T18:15:52.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.986+0000 7f4f67fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56026c0d9f10 0x56026c0d40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.986+0000 7f4f66ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56026c0da860 0x56026c0eae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.986+0000 7f4f67fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56026c0d9f10 0x56026c0d40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:48806/0 (socket says 172.21.15.133:48806) 2023-07-14T18:15:52.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.986+0000 7f4f67fff700 1 -- 172.21.15.133:0/641740987 learned_addr learned my addr 172.21.15.133:0/641740987 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:52.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.986+0000 7f4f67fff700 1 --2- 172.21.15.133:0/641740987 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56026c0db460 0x56026c0e1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.986+0000 7f4f66ffd700 1 -- 172.21.15.133:0/641740987 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56026c0db460 msgr2=0x56026c0e1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:15:52.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.986+0000 7f4f66ffd700 1 --2- 172.21.15.133:0/641740987 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56026c0db460 0x56026c0e1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.986+0000 7f4f66ffd700 1 -- 172.21.15.133:0/641740987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56026c0d9f10 msgr2=0x56026c0d40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:15:52.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.986+0000 7f4f66ffd700 1 --2- 172.21.15.133:0/641740987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56026c0d9f10 0x56026c0d40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.986+0000 7f4f66ffd700 1 -- 172.21.15.133:0/641740987 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56026c0d4f00 con 0x56026c0da860 2023-07-14T18:15:52.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.987+0000 7f4f66ffd700 1 --2- 172.21.15.133:0/641740987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56026c0da860 0x56026c0eae30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00f9e0 tx=0x7f4f5c00fb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776d929997b2af9e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:52.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.987+0000 7f4f4e7fc700 1 -- 172.21.15.133:0/641740987 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c017070 con 0x56026c0da860 2023-07-14T18:15:52.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.987+0000 7f4f4e7fc700 1 -- 172.21.15.133:0/641740987 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f5c0092d0 con 0x56026c0da860 2023-07-14T18:15:52.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.987+0000 7f4f4e7fc700 1 -- 172.21.15.133:0/641740987 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c005820 con 0x56026c0da860 2023-07-14T18:15:52.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.988+0000 7f4f8174cb80 1 -- 172.21.15.133:0/641740987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56026c0da860 msgr2=0x56026c0eae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:52.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.988+0000 7f4f8174cb80 1 --2- 172.21.15.133:0/641740987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56026c0da860 0x56026c0eae30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00f9e0 tx=0x7f4f5c00fb80 comp rx=0 tx=0).stop 2023-07-14T18:15:52.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.988+0000 7f4f8174cb80 1 -- 172.21.15.133:0/641740987 shutdown_connections 2023-07-14T18:15:52.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.988+0000 7f4f8174cb80 1 --2- 172.21.15.133:0/641740987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56026c0d9f10 0x56026c0d40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.988+0000 7f4f8174cb80 1 --2- 172.21.15.133:0/641740987 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56026c0db460 0x56026c0e1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.988+0000 7f4f8174cb80 1 --2- 172.21.15.133:0/641740987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56026c0da860 0x56026c0eae30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.988+0000 7f4f8174cb80 1 -- 172.21.15.133:0/641740987 >> 172.21.15.133:0/641740987 conn(0x56026c0cf630 msgr2=0x56026c0e2620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:52.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.988+0000 7f4f8174cb80 1 -- 172.21.15.133:0/641740987 shutdown_connections 2023-07-14T18:15:52.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.988+0000 7f4f8174cb80 1 -- 172.21.15.133:0/641740987 wait complete. 2023-07-14T18:15:52.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.988+0000 7f4f8174cb80 1 -- start start 2023-07-14T18:15:52.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.988+0000 7f4f8174cb80 1 Processor -- start 2023-07-14T18:15:52.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.989+0000 7f4f8174cb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56026c0d9f10 0x56026c0e9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.989+0000 7f4f8174cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56026c0da860 0x56026c0e95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.989+0000 7f4f8174cb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56026c0db460 0x56026c0e9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.989+0000 7f4f8174cb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56026c0ea200 con 0x56026c0db460 2023-07-14T18:15:52.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.989+0000 7f4f8174cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56026c0ea370 con 0x56026c0da860 2023-07-14T18:15:52.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.989+0000 7f4f8174cb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x56026c0ea530 con 0x56026c0d9f10 2023-07-14T18:15:52.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.989+0000 7f4f8174cb80 10 client.0 _ll_register_callbacks cb 0x7f4f5c009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:15:52.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.989+0000 7f4f67fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56026c0d9f10 0x56026c0e9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.989+0000 7f4f66ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56026c0da860 0x56026c0e95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.989+0000 7f4f66ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56026c0da860 0x56026c0e95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:48810/0 (socket says 172.21.15.133:48810) 2023-07-14T18:15:52.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.989+0000 7f4f67fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56026c0d9f10 0x56026c0e9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:39712/0 (socket says 172.21.15.133:39712) 2023-07-14T18:15:52.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.989+0000 7f4f66ffd700 1 -- 172.21.15.133:0/1485715975 learned_addr learned my addr 172.21.15.133:0/1485715975 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:52.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.989+0000 7f4f67fff700 1 --2- 172.21.15.133:0/1485715975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56026c0db460 0x56026c0e9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.989+0000 7f4f66ffd700 1 -- 172.21.15.133:0/1485715975 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56026c0d9f10 msgr2=0x56026c0e9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:15:52.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.989+0000 7f4f66ffd700 1 --2- 172.21.15.133:0/1485715975 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56026c0d9f10 0x56026c0e9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.989+0000 7f4f66ffd700 1 -- 172.21.15.133:0/1485715975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56026c0db460 msgr2=0x56026c0e9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:15:52.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.989+0000 7f4f66ffd700 1 --2- 172.21.15.133:0/1485715975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56026c0db460 0x56026c0e9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.989+0000 7f4f66ffd700 1 -- 172.21.15.133:0/1485715975 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f5c021040 con 0x56026c0da860 2023-07-14T18:15:52.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.989+0000 7f4f67fff700 1 --2- 172.21.15.133:0/1485715975 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56026c0d9f10 0x56026c0e9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:52.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.990+0000 7f4f66ffd700 1 --2- 172.21.15.133:0/1485715975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56026c0da860 0x56026c0e95b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00f960 tx=0x7f4f5c012450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=777e44a0a79ffc5b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:52.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.990+0000 7f4f46ffd700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:15:52.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.990+0000 7f4f46ffd700 1 -- 172.21.15.133:0/1485715975 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c017070 con 0x56026c0da860 2023-07-14T18:15:52.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.990+0000 7f4f8174cb80 1 -- 172.21.15.133:0/1485715975 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f600094a0 con 0x56026c0da860 2023-07-14T18:15:52.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.990+0000 7f4f46ffd700 1 -- 172.21.15.133:0/1485715975 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4f5c000e40 con 0x56026c0da860 2023-07-14T18:15:52.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.990+0000 7f4f46ffd700 1 -- 172.21.15.133:0/1485715975 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f5c00fc50 con 0x56026c0da860 2023-07-14T18:15:52.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.990+0000 7f4f8174cb80 10 client.25133 fetch_fsmap learned FSMap version 25 2023-07-14T18:15:52.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.990+0000 7f4f8174cb80 1 -- 172.21.15.133:0/1485715975 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56026be7f200 con 0x56026c0da860 2023-07-14T18:15:52.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.991+0000 7f4f46ffd700 1 -- 172.21.15.133:0/1485715975 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f600094a0 con 0x56026c0da860 2023-07-14T18:15:52.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.991+0000 7f4f46ffd700 1 -- 172.21.15.133:0/1485715975 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4f5c010430 con 0x56026c0da860 2023-07-14T18:15:52.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.991+0000 7f4f8174cb80 10 client.25133 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T18:15:52.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.991+0000 7f4f8174cb80 10 client.25133 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-14T18:15:52.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.991+0000 7f4f8174cb80 20 client.25133 populate_metadata read hostname 'smithi133' 2023-07-14T18:15:52.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.991+0000 7f4f8174cb80 1 --2- 172.21.15.133:0/1485715975 >> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] conn(0x56026c148310 0x56026c144c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.991+0000 7f4f8174cb80 4 client.25133 mds_command: new command op to 24541 tid=0[{"prefix": "status"}] 2023-07-14T18:15:52.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.991+0000 7f4f8174cb80 1 -- 172.21.15.133:0/1485715975 --> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56026c023e60 con 0x56026c148310 2023-07-14T18:15:52.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.991+0000 7f4f67fff700 1 --2- 172.21.15.133:0/1485715975 >> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] conn(0x56026c148310 0x56026c144c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.992+0000 7f4f67fff700 1 --2- 172.21.15.133:0/1485715975 >> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] conn(0x56026c148310 0x56026c144c50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b4d2aa2411f86113 server_cookie=2bb7b275236c24f6 in_seq=0 out_seq=0 2023-07-14T18:15:52.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.992+0000 7f4f46ffd700 10 client.25133 ms_handle_connect on v2:172.21.15.171:6838/816201198 2023-07-14T18:15:52.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.992+0000 7f4f46ffd700 1 -- 172.21.15.133:0/1485715975 <== mds.1 v2:172.21.15.171:6838/816201198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x56026c023e60 con 0x56026c148310 2023-07-14T18:15:52.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.992+0000 7f4f46ffd700 10 client.25133 handle_command_reply: tid=0 2023-07-14T18:15:52.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.992+0000 7f4f8174cb80 1 client.25133 shutdown 2023-07-14T18:15:52.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.992+0000 7f4f8174cb80 1 -- 172.21.15.133:0/1485715975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56026c0da860 msgr2=0x56026c0e95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:52.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.992+0000 7f4f8174cb80 1 --2- 172.21.15.133:0/1485715975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56026c0da860 0x56026c0e95b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00f960 tx=0x7f4f5c012450 comp rx=0 tx=0).stop 2023-07-14T18:15:52.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.992+0000 7f4f8174cb80 1 -- 172.21.15.133:0/1485715975 shutdown_connections 2023-07-14T18:15:52.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.992+0000 7f4f8174cb80 1 --2- 172.21.15.133:0/1485715975 >> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] conn(0x56026c148310 0x56026c144c50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.992+0000 7f4f8174cb80 1 --2- 172.21.15.133:0/1485715975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56026c0db460 0x56026c0e9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.992+0000 7f4f46ffd700 0 client.25133 ms_handle_reset on v2:172.21.15.171:6838/816201198 2023-07-14T18:15:52.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.992+0000 7f4f8174cb80 1 --2- 172.21.15.133:0/1485715975 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56026c0d9f10 0x56026c0e9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.992+0000 7f4f8174cb80 1 --2- 172.21.15.133:0/1485715975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56026c0da860 0x56026c0e95b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.992+0000 7f4f8174cb80 1 -- 172.21.15.133:0/1485715975 >> 172.21.15.133:0/1485715975 conn(0x56026c0cf630 msgr2=0x56026c0e2620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:52.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.992+0000 7f4f8174cb80 1 -- 172.21.15.133:0/1485715975 shutdown_connections 2023-07-14T18:15:52.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.992+0000 7f4f8174cb80 1 -- 172.21.15.133:0/1485715975 wait complete. 2023-07-14T18:15:52.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.992+0000 7f4f8174cb80 20 client.25133 trim_cache size 0 max 16384 2023-07-14T18:15:52.050 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:15:52.051 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "06d0f5fa-2271-11ee-9b33-001a4aab830c", 2023-07-14T18:15:52.051 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 1, 2023-07-14T18:15:52.051 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24541, 2023-07-14T18:15:52.051 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2023-07-14T18:15:52.051 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2023-07-14T18:15:52.051 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2023-07-14T18:15:52.051 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 64.044538207000002, 2023-07-14T18:15:52.051 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 25, 2023-07-14T18:15:52.052 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 92, 2023-07-14T18:15:52.052 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 90, 2023-07-14T18:15:52.052 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 109.422952258 2023-07-14T18:15:52.052 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:15:52.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.993+0000 7f4f6ddff700 1 -- 172.21.15.133:0/178056486 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4f500a11e0 msgr2=0x7f4f500a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:52.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.993+0000 7f4f6ddff700 1 --2- 172.21.15.133:0/178056486 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4f500a11e0 0x7f4f500a3690 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x56026c0dac70 tx=0x7f4f580d08e0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.993+0000 7f4f6ddff700 1 -- 172.21.15.133:0/178056486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f680ffc10 msgr2=0x7f4f68197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:52.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.993+0000 7f4f6ddff700 1 --2- 172.21.15.133:0/178056486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f680ffc10 0x7f4f68197c80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4f5800e970 tx=0x7f4f5800ec80 comp rx=0 tx=0).stop 2023-07-14T18:15:52.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.993+0000 7f4f6ddff700 1 -- 172.21.15.133:0/178056486 shutdown_connections 2023-07-14T18:15:52.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.993+0000 7f4f6ddff700 1 --2- 172.21.15.133:0/178056486 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4f500a11e0 0x7f4f500a3690 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.993+0000 7f4f6ddff700 1 --2- 172.21.15.133:0/178056486 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f68101790 0x7f4f6819f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.993+0000 7f4f6ddff700 1 --2- 172.21.15.133:0/178056486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f680ffc10 0x7f4f68197c80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.993+0000 7f4f6ddff700 1 --2- 172.21.15.133:0/178056486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f68100e00 0x7f4f681981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.993+0000 7f4f6ddff700 1 -- 172.21.15.133:0/178056486 >> 172.21.15.133:0/178056486 conn(0x7f4f680fb210 msgr2=0x7f4f680fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:52.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.993+0000 7f4f6ddff700 1 -- 172.21.15.133:0/178056486 shutdown_connections 2023-07-14T18:15:52.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:51.993+0000 7f4f6ddff700 1 -- 172.21.15.133:0/178056486 wait complete. 2023-07-14T18:15:52.055 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:51 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1653812202' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:15:52.055 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:51 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3070276312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:15:52.056 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-14T18:15:52.252 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.207+0000 7fcbac5ff700 1 -- 172.21.15.133:0/174102126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcba40ffc20 msgr2=0x7fcba4100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:52.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.207+0000 7fcbac5ff700 1 --2- 172.21.15.133:0/174102126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcba40ffc20 0x7fcba4100030 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009990 tx=0x7fcba0009ca0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.208+0000 7fcbac5ff700 1 -- 172.21.15.133:0/174102126 shutdown_connections 2023-07-14T18:15:52.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.208+0000 7fcbac5ff700 1 --2- 172.21.15.133:0/174102126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcba40ffc20 0x7fcba4100030 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.208+0000 7fcbac5ff700 1 --2- 172.21.15.133:0/174102126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcba41017a0 0x7fcba4108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.208+0000 7fcbac5ff700 1 --2- 172.21.15.133:0/174102126 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcba4100e10 0x7fcba4101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.208+0000 7fcbac5ff700 1 -- 172.21.15.133:0/174102126 >> 172.21.15.133:0/174102126 conn(0x7fcba40fb260 msgr2=0x7fcba40fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:52.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.208+0000 7fcbac5ff700 1 -- 172.21.15.133:0/174102126 shutdown_connections 2023-07-14T18:15:52.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.208+0000 7fcbac5ff700 1 -- 172.21.15.133:0/174102126 wait complete. 2023-07-14T18:15:52.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.209+0000 7fcbac5ff700 1 Processor -- start 2023-07-14T18:15:52.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.209+0000 7fcbac5ff700 1 -- start start 2023-07-14T18:15:52.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.209+0000 7fcbac5ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcba40ffc20 0x7fcba4197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.209+0000 7fcbac5ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcba4100e10 0x7fcba4198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.209+0000 7fcbac5ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcba41017a0 0x7fcba419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.209+0000 7fcbac5ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba419f770 con 0x7fcba40ffc20 2023-07-14T18:15:52.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.209+0000 7fcbac5ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba419f8b0 con 0x7fcba41017a0 2023-07-14T18:15:52.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.209+0000 7fcbac5ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba419fa40 con 0x7fcba4100e10 2023-07-14T18:15:52.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.209+0000 7fcbaa39b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcba40ffc20 0x7fcba4197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.209+0000 7fcbaa39b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcba40ffc20 0x7fcba4197bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32790/0 (socket says 172.21.15.133:32790) 2023-07-14T18:15:52.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.209+0000 7fcba9b9a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcba4100e10 0x7fcba4198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.209+0000 7fcbaa39b700 1 -- 172.21.15.133:0/928992333 learned_addr learned my addr 172.21.15.133:0/928992333 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:52.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.209+0000 7fcbaab9c700 1 --2- 172.21.15.133:0/928992333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcba41017a0 0x7fcba419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.210+0000 7fcbaa39b700 1 -- 172.21.15.133:0/928992333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcba4100e10 msgr2=0x7fcba4198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:52.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.210+0000 7fcbaa39b700 1 --2- 172.21.15.133:0/928992333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcba4100e10 0x7fcba4198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.210+0000 7fcbaa39b700 1 -- 172.21.15.133:0/928992333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcba41017a0 msgr2=0x7fcba419f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:52.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.210+0000 7fcbaa39b700 1 --2- 172.21.15.133:0/928992333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcba41017a0 0x7fcba419f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.210+0000 7fcbaa39b700 1 -- 172.21.15.133:0/928992333 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb9c009c40 con 0x7fcba40ffc20 2023-07-14T18:15:52.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.210+0000 7fcbaa39b700 1 --2- 172.21.15.133:0/928992333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcba40ffc20 0x7fcba4197bf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcba0005740 tx=0x7fcba0005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:52.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.210+0000 7fcb9b7fe700 1 -- 172.21.15.133:0/928992333 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba001d070 con 0x7fcba40ffc20 2023-07-14T18:15:52.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.210+0000 7fcbac5ff700 1 -- 172.21.15.133:0/928992333 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba00095f0 con 0x7fcba40ffc20 2023-07-14T18:15:52.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.210+0000 7fcb9b7fe700 1 -- 172.21.15.133:0/928992333 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcba0021410 con 0x7fcba40ffc20 2023-07-14T18:15:52.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.210+0000 7fcb9b7fe700 1 -- 172.21.15.133:0/928992333 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba0017650 con 0x7fcba40ffc20 2023-07-14T18:15:52.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.210+0000 7fcbac5ff700 1 -- 172.21.15.133:0/928992333 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba419ffb0 con 0x7fcba40ffc20 2023-07-14T18:15:52.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.213+0000 7fcb9b7fe700 1 -- 172.21.15.133:0/928992333 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fcba0021580 con 0x7fcba40ffc20 2023-07-14T18:15:52.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.215+0000 7fcb9b7fe700 1 --2- 172.21.15.133:0/928992333 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fcb900a11e0 0x7fcb900a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.215+0000 7fcbbffb3b80 1 Processor -- start 2023-07-14T18:15:52.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.215+0000 7fcb9b7fe700 1 -- 172.21.15.133:0/928992333 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fcba00ce100 con 0x7fcba40ffc20 2023-07-14T18:15:52.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.215+0000 7fcba9b9a700 1 --2- 172.21.15.133:0/928992333 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fcb900a11e0 0x7fcb900a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.215+0000 7fcbbffb3b80 1 -- start start 2023-07-14T18:15:52.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.215+0000 7fcba9b9a700 1 --2- 172.21.15.133:0/928992333 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fcb900a11e0 0x7fcb900a3690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcb94009c60 tx=0x7fcb94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:52.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.215+0000 7fcbbffb3b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 0x5576a8500320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.215+0000 7fcbbffb3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576a8500860 0x5576a8500e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.215+0000 7fcbbffb3b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5576a8501460 0x5576a85018d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.216+0000 7fcbbffb3b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a8404c60 con 0x5576a84fff10 2023-07-14T18:15:52.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.216+0000 7fcbbffb3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a84579f0 con 0x5576a8500860 2023-07-14T18:15:52.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.216+0000 7fcbbffb3b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a8450e80 con 0x5576a8501460 2023-07-14T18:15:52.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.216+0000 7fcbaa39b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 0x5576a8500320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.216+0000 7fcbaa39b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 0x5576a8500320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32794/0 (socket says 172.21.15.133:32794) 2023-07-14T18:15:52.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.216+0000 7fcbaa39b700 1 -- 172.21.15.133:0/3659755139 learned_addr learned my addr 172.21.15.133:0/3659755139 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:52.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.216+0000 7fcba9b9a700 1 --2- 172.21.15.133:0/3659755139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576a8500860 0x5576a8500e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.216+0000 7fcbaab9c700 1 --2- 172.21.15.133:0/3659755139 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5576a8501460 0x5576a85018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.216+0000 7fcbaa39b700 1 -- 172.21.15.133:0/3659755139 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5576a8501460 msgr2=0x5576a85018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:15:52.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.216+0000 7fcbaa39b700 1 --2- 172.21.15.133:0/3659755139 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5576a8501460 0x5576a85018d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.216+0000 7fcbaa39b700 1 -- 172.21.15.133:0/3659755139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576a8500860 msgr2=0x5576a8500e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:15:52.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.216+0000 7fcbaa39b700 1 --2- 172.21.15.133:0/3659755139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576a8500860 0x5576a8500e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.216+0000 7fcbaa39b700 1 -- 172.21.15.133:0/3659755139 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576a827ed90 con 0x5576a84fff10 2023-07-14T18:15:52.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.216+0000 7fcba9b9a700 1 --2- 172.21.15.133:0/3659755139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576a8500860 0x5576a8500e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:15:52.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.216+0000 7fcbaa39b700 1 --2- 172.21.15.133:0/3659755139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 0x5576a8500320 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcba00a2fd0 tx=0x7fcba009aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8287372b6c5344dc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:52.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.217+0000 7fcb98bfe700 1 -- 172.21.15.133:0/3659755139 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba00a3500 con 0x5576a84fff10 2023-07-14T18:15:52.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.217+0000 7fcb98bfe700 1 -- 172.21.15.133:0/3659755139 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcba00cb970 con 0x5576a84fff10 2023-07-14T18:15:52.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.217+0000 7fcb98bfe700 1 -- 172.21.15.133:0/3659755139 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba009a890 con 0x5576a84fff10 2023-07-14T18:15:52.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.217+0000 7fcbbffb3b80 1 -- 172.21.15.133:0/3659755139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 msgr2=0x5576a8500320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:52.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.217+0000 7fcbbffb3b80 1 --2- 172.21.15.133:0/3659755139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 0x5576a8500320 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcba00a2fd0 tx=0x7fcba009aaa0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.217+0000 7fcbbffb3b80 1 -- 172.21.15.133:0/3659755139 shutdown_connections 2023-07-14T18:15:52.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.217+0000 7fcbbffb3b80 1 --2- 172.21.15.133:0/3659755139 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5576a8501460 0x5576a85018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.217+0000 7fcbbffb3b80 1 --2- 172.21.15.133:0/3659755139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 0x5576a8500320 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.217+0000 7fcbbffb3b80 1 --2- 172.21.15.133:0/3659755139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576a8500860 0x5576a8500e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.272 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.217+0000 7fcbbffb3b80 1 -- 172.21.15.133:0/3659755139 >> 172.21.15.133:0/3659755139 conn(0x5576a84f5630 msgr2=0x5576a84fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:52.272 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.217+0000 7fcbbffb3b80 1 -- 172.21.15.133:0/3659755139 shutdown_connections 2023-07-14T18:15:52.272 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.217+0000 7fcbbffb3b80 1 -- 172.21.15.133:0/3659755139 wait complete. 2023-07-14T18:15:52.272 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.218+0000 7fcbbffb3b80 1 -- start start 2023-07-14T18:15:52.272 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.218+0000 7fcbbffb3b80 1 Processor -- start 2023-07-14T18:15:52.272 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.218+0000 7fcbbffb3b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 0x5576a8509510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.272 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.219+0000 7fcbbffb3b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5576a8500860 0x5576a8509a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.272 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.219+0000 7fcbbffb3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576a8501460 0x5576a8510a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.219+0000 7fcbbffb3b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a83f0a70 con 0x5576a84fff10 2023-07-14T18:15:52.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.219+0000 7fcbbffb3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a83f0770 con 0x5576a8501460 2023-07-14T18:15:52.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.219+0000 7fcbbffb3b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a82d48d0 con 0x5576a8500860 2023-07-14T18:15:52.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.219+0000 7fcbbffb3b80 10 client.0 _ll_register_callbacks cb 0x5576a83ee710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:15:52.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.219+0000 7fcbaa39b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 0x5576a8509510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.219+0000 7fcbaa39b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 0x5576a8509510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32796/0 (socket says 172.21.15.133:32796) 2023-07-14T18:15:52.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.219+0000 7fcbaa39b700 1 -- 172.21.15.133:0/1324594800 learned_addr learned my addr 172.21.15.133:0/1324594800 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:52.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.219+0000 7fcba9b9a700 1 --2- 172.21.15.133:0/1324594800 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5576a8500860 0x5576a8509a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.219+0000 7fcbaab9c700 1 --2- 172.21.15.133:0/1324594800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576a8501460 0x5576a8510a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.219+0000 7fcbaa39b700 1 -- 172.21.15.133:0/1324594800 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5576a8500860 msgr2=0x5576a8509a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:15:52.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.219+0000 7fcbaa39b700 1 --2- 172.21.15.133:0/1324594800 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5576a8500860 0x5576a8509a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.219+0000 7fcbaa39b700 1 -- 172.21.15.133:0/1324594800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576a8501460 msgr2=0x5576a8510a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:15:52.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.219+0000 7fcbaa39b700 1 --2- 172.21.15.133:0/1324594800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576a8501460 0x5576a8510a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.219+0000 7fcbaa39b700 1 -- 172.21.15.133:0/1324594800 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba419ffb0 con 0x5576a84fff10 2023-07-14T18:15:52.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.219+0000 7fcba9b9a700 1 --2- 172.21.15.133:0/1324594800 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5576a8500860 0x5576a8509a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:15:52.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.219+0000 7fcbaa39b700 1 --2- 172.21.15.133:0/1324594800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 0x5576a8509510 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fcba0005f50 tx=0x7fcba00975b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4c46d2364c9c37 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:52.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.219+0000 7fcb8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T18:15:52.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.219+0000 7fcb8dffb700 1 -- 172.21.15.133:0/1324594800 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba00a3280 con 0x5576a84fff10 2023-07-14T18:15:52.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.220+0000 7fcbbffb3b80 1 -- 172.21.15.133:0/1324594800 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5576a83aa4a0 con 0x5576a84fff10 2023-07-14T18:15:52.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.220+0000 7fcb8dffb700 1 -- 172.21.15.133:0/1324594800 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fcba000fb10 con 0x5576a84fff10 2023-07-14T18:15:52.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.220+0000 7fcb8dffb700 1 -- 172.21.15.133:0/1324594800 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcba000fcd0 con 0x5576a84fff10 2023-07-14T18:15:52.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.220+0000 7fcbbffb3b80 10 client.15162 fetch_fsmap learned FSMap version 25 2023-07-14T18:15:52.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.220+0000 7fcbbffb3b80 1 -- 172.21.15.133:0/1324594800 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5576a8267c10 con 0x5576a84fff10 2023-07-14T18:15:52.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.220+0000 7fcb8dffb700 1 -- 172.21.15.133:0/1324594800 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5576a83aa4a0 con 0x5576a84fff10 2023-07-14T18:15:52.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.220+0000 7fcb8dffb700 1 -- 172.21.15.133:0/1324594800 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcba00b3a90 con 0x5576a84fff10 2023-07-14T18:15:52.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.220+0000 7fcbbffb3b80 10 client.15162 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T18:15:52.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.220+0000 7fcbbffb3b80 10 client.15162 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-14T18:15:52.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.220+0000 7fcbbffb3b80 20 client.15162 populate_metadata read hostname 'smithi133' 2023-07-14T18:15:52.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.220+0000 7fcbbffb3b80 1 --2- 172.21.15.133:0/1324594800 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5576a85148d0 0x5576a85139c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.220+0000 7fcbbffb3b80 4 client.15162 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:15:52.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.220+0000 7fcbbffb3b80 1 -- 172.21.15.133:0/1324594800 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5576a84ced50 con 0x5576a85148d0 2023-07-14T18:15:52.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.221+0000 7fcba9b9a700 1 --2- 172.21.15.133:0/1324594800 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5576a85148d0 0x5576a85139c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.221+0000 7fcba9b9a700 1 --2- 172.21.15.133:0/1324594800 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5576a85148d0 0x5576a85139c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=788f8f0c4fb48306 server_cookie=124af8eb270c8958 in_seq=0 out_seq=0 2023-07-14T18:15:52.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.221+0000 7fcb8dffb700 10 client.15162 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:15:52.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.222+0000 7fcb8dffb700 1 -- 172.21.15.133:0/1324594800 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5576a84ced50 con 0x5576a85148d0 2023-07-14T18:15:52.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.222+0000 7fcb8dffb700 10 client.15162 handle_command_reply: tid=0 2023-07-14T18:15:52.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.222+0000 7fcbbffb3b80 1 client.15162 shutdown 2023-07-14T18:15:52.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.222+0000 7fcbbffb3b80 1 -- 172.21.15.133:0/1324594800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 msgr2=0x5576a8509510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:52.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.222+0000 7fcbbffb3b80 1 --2- 172.21.15.133:0/1324594800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 0x5576a8509510 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fcba0005f50 tx=0x7fcba00975b0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.222+0000 7fcbbffb3b80 1 -- 172.21.15.133:0/1324594800 shutdown_connections 2023-07-14T18:15:52.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.222+0000 7fcbbffb3b80 1 --2- 172.21.15.133:0/1324594800 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5576a85148d0 0x5576a85139c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.222+0000 7fcbbffb3b80 1 --2- 172.21.15.133:0/1324594800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576a8501460 0x5576a8510a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.222+0000 7fcbbffb3b80 1 --2- 172.21.15.133:0/1324594800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 0x5576a8509510 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.222+0000 7fcbbffb3b80 1 --2- 172.21.15.133:0/1324594800 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5576a8500860 0x5576a8509a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.222+0000 7fcbbffb3b80 1 -- 172.21.15.133:0/1324594800 >> 172.21.15.133:0/1324594800 conn(0x5576a84f5630 msgr2=0x5576a84f9600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:52.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.222+0000 7fcb8dffb700 0 client.15162 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:15:52.279 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.222+0000 7fcbbffb3b80 1 -- 172.21.15.133:0/1324594800 shutdown_connections 2023-07-14T18:15:52.279 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.222+0000 7fcbbffb3b80 1 -- 172.21.15.133:0/1324594800 wait complete. 2023-07-14T18:15:52.279 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.223+0000 7fcbbffb3b80 20 client.15162 trim_cache size 0 max 16384 2023-07-14T18:15:52.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.240+0000 7fcbbffb3b80 1 Processor -- start 2023-07-14T18:15:52.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.240+0000 7fcbbffb3b80 1 -- start start 2023-07-14T18:15:52.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.241+0000 7fcbbffb3b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 0x5576a85139c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.241+0000 7fcbbffb3b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5576a8500860 0x5576a8510a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.241+0000 7fcbbffb3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576a8501460 0x5576a84fa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.241+0000 7fcbbffb3b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a84ced50 con 0x5576a84fff10 2023-07-14T18:15:52.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.241+0000 7fcbbffb3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a83aa4a0 con 0x5576a8501460 2023-07-14T18:15:52.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.241+0000 7fcbbffb3b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a83fa470 con 0x5576a8500860 2023-07-14T18:15:52.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.241+0000 7fcbaa39b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 0x5576a85139c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.241+0000 7fcbaa39b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 0x5576a85139c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32812/0 (socket says 172.21.15.133:32812) 2023-07-14T18:15:52.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.241+0000 7fcbaa39b700 1 -- 172.21.15.133:0/2358438853 learned_addr learned my addr 172.21.15.133:0/2358438853 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:52.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.241+0000 7fcba9b9a700 1 --2- 172.21.15.133:0/2358438853 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5576a8500860 0x5576a8510a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.241+0000 7fcbaab9c700 1 --2- 172.21.15.133:0/2358438853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576a8501460 0x5576a84fa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.241+0000 7fcbaa39b700 1 -- 172.21.15.133:0/2358438853 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5576a8500860 msgr2=0x5576a8510a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:15:52.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.241+0000 7fcbaa39b700 1 --2- 172.21.15.133:0/2358438853 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5576a8500860 0x5576a8510a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.241+0000 7fcbaa39b700 1 -- 172.21.15.133:0/2358438853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576a8501460 msgr2=0x5576a84fa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:15:52.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.241+0000 7fcbaa39b700 1 --2- 172.21.15.133:0/2358438853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576a8501460 0x5576a84fa0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.241+0000 7fcbaa39b700 1 -- 172.21.15.133:0/2358438853 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576a82a5200 con 0x5576a84fff10 2023-07-14T18:15:52.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.241+0000 7fcbaa39b700 1 --2- 172.21.15.133:0/2358438853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 0x5576a85139c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcba009b0d0 tx=0x7fcba0021c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8326a1642dc324fa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:52.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.242+0000 7fcb993ff700 1 -- 172.21.15.133:0/2358438853 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba00b8ef0 con 0x5576a84fff10 2023-07-14T18:15:52.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.242+0000 7fcb993ff700 1 -- 172.21.15.133:0/2358438853 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcba00994e0 con 0x5576a84fff10 2023-07-14T18:15:52.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.242+0000 7fcb993ff700 1 -- 172.21.15.133:0/2358438853 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba00b9150 con 0x5576a84fff10 2023-07-14T18:15:52.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.242+0000 7fcbbffb3b80 1 -- 172.21.15.133:0/2358438853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 msgr2=0x5576a85139c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:52.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.242+0000 7fcbbffb3b80 1 --2- 172.21.15.133:0/2358438853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 0x5576a85139c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcba009b0d0 tx=0x7fcba0021c20 comp rx=0 tx=0).stop 2023-07-14T18:15:52.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.242+0000 7fcbbffb3b80 1 -- 172.21.15.133:0/2358438853 shutdown_connections 2023-07-14T18:15:52.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.242+0000 7fcbbffb3b80 1 --2- 172.21.15.133:0/2358438853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576a8501460 0x5576a84fa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.242+0000 7fcbbffb3b80 1 --2- 172.21.15.133:0/2358438853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 0x5576a85139c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.242+0000 7fcbbffb3b80 1 --2- 172.21.15.133:0/2358438853 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5576a8500860 0x5576a8510a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.242+0000 7fcbbffb3b80 1 -- 172.21.15.133:0/2358438853 >> 172.21.15.133:0/2358438853 conn(0x5576a84f5630 msgr2=0x5576a84f6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:52.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.242+0000 7fcbbffb3b80 1 -- 172.21.15.133:0/2358438853 shutdown_connections 2023-07-14T18:15:52.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.242+0000 7fcbbffb3b80 1 -- 172.21.15.133:0/2358438853 wait complete. 2023-07-14T18:15:52.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.243+0000 7fcbbffb3b80 1 -- start start 2023-07-14T18:15:52.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.243+0000 7fcbbffb3b80 1 Processor -- start 2023-07-14T18:15:52.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.243+0000 7fcbbffb3b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 0x5576a850b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.243+0000 7fcbbffb3b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5576a8500860 0x5576a8510a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.243+0000 7fcbbffb3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576a8501460 0x5576a850ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.243+0000 7fcbbffb3b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a850c160 con 0x5576a84fff10 2023-07-14T18:15:52.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.243+0000 7fcbbffb3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a8567f70 con 0x5576a8501460 2023-07-14T18:15:52.294 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.243+0000 7fcbbffb3b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a8568130 con 0x5576a8500860 2023-07-14T18:15:52.294 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.243+0000 7fcbbffb3b80 10 client.0 _ll_register_callbacks cb 0x5576a83ee710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:15:52.294 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.244+0000 7fcbaa39b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 0x5576a850b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.294 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.244+0000 7fcbaa39b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 0x5576a850b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32818/0 (socket says 172.21.15.133:32818) 2023-07-14T18:15:52.294 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.244+0000 7fcbaa39b700 1 -- 172.21.15.133:0/662851863 learned_addr learned my addr 172.21.15.133:0/662851863 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:52.294 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.244+0000 7fcbaab9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576a8501460 0x5576a850ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.294 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.244+0000 7fcba9b9a700 1 --2- 172.21.15.133:0/662851863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5576a8500860 0x5576a8510a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.244+0000 7fcbaa39b700 1 -- 172.21.15.133:0/662851863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5576a8500860 msgr2=0x5576a8510a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:15:52.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.244+0000 7fcbaa39b700 1 --2- 172.21.15.133:0/662851863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5576a8500860 0x5576a8510a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.244+0000 7fcbaa39b700 1 -- 172.21.15.133:0/662851863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576a8501460 msgr2=0x5576a850ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:15:52.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.244+0000 7fcbaa39b700 1 --2- 172.21.15.133:0/662851863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576a8501460 0x5576a850ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.244+0000 7fcbaa39b700 1 -- 172.21.15.133:0/662851863 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5576a8267c10 con 0x5576a84fff10 2023-07-14T18:15:52.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.244+0000 7fcbaab9c700 1 --2- 172.21.15.133:0/662851863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576a8501460 0x5576a850ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:52.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.244+0000 7fcbaa39b700 1 --2- 172.21.15.133:0/662851863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 0x5576a850b4f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcba009af80 tx=0x7fcba0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c4b520958e5efc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:52.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.244+0000 7fcb8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T18:15:52.296 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.244+0000 7fcb8dffb700 1 -- 172.21.15.133:0/662851863 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba0097480 con 0x5576a84fff10 2023-07-14T18:15:52.296 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.244+0000 7fcbbffb3b80 1 -- 172.21.15.133:0/662851863 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcba00cb970 con 0x5576a84fff10 2023-07-14T18:15:52.296 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.244+0000 7fcb8dffb700 1 -- 172.21.15.133:0/662851863 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fcba000fd20 con 0x5576a84fff10 2023-07-14T18:15:52.296 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.245+0000 7fcb8dffb700 1 -- 172.21.15.133:0/662851863 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcba00b9150 con 0x5576a84fff10 2023-07-14T18:15:52.296 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.245+0000 7fcbbffb3b80 10 client.15168 fetch_fsmap learned FSMap version 25 2023-07-14T18:15:52.296 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.245+0000 7fcbbffb3b80 1 -- 172.21.15.133:0/662851863 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5576a827d260 con 0x5576a84fff10 2023-07-14T18:15:52.296 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.245+0000 7fcb8dffb700 1 -- 172.21.15.133:0/662851863 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcba00cb970 con 0x5576a84fff10 2023-07-14T18:15:52.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.245+0000 7fcb8dffb700 1 -- 172.21.15.133:0/662851863 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcba009a230 con 0x5576a84fff10 2023-07-14T18:15:52.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.245+0000 7fcbbffb3b80 10 client.15168 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T18:15:52.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.245+0000 7fcbbffb3b80 10 client.15168 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-14T18:15:52.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.245+0000 7fcbbffb3b80 20 client.15168 populate_metadata read hostname 'smithi133' 2023-07-14T18:15:52.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.245+0000 7fcbbffb3b80 1 --2- 172.21.15.133:0/662851863 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5576a856c8f0 0x5576a8569380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:52.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.245+0000 7fcbbffb3b80 4 client.15168 mds_command: new command op to 24557 tid=0[{"prefix": "status"}] 2023-07-14T18:15:52.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.245+0000 7fcbbffb3b80 1 -- 172.21.15.133:0/662851863 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5576a8449e60 con 0x5576a856c8f0 2023-07-14T18:15:52.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.245+0000 7fcba9b9a700 1 --2- 172.21.15.133:0/662851863 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5576a856c8f0 0x5576a8569380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:52.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.246+0000 7fcba9b9a700 1 --2- 172.21.15.133:0/662851863 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5576a856c8f0 0x5576a8569380 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ebaa63e2506cd36 server_cookie=2ef634328fe1a191 in_seq=0 out_seq=0 2023-07-14T18:15:52.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.246+0000 7fcb8dffb700 10 client.15168 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:15:52.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.246+0000 7fcb8dffb700 1 -- 172.21.15.133:0/662851863 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x5576a8449e60 con 0x5576a856c8f0 2023-07-14T18:15:52.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.246+0000 7fcb8dffb700 10 client.15168 handle_command_reply: tid=0 2023-07-14T18:15:52.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.246+0000 7fcbbffb3b80 1 client.15168 shutdown 2023-07-14T18:15:52.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.246+0000 7fcbbffb3b80 1 -- 172.21.15.133:0/662851863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 msgr2=0x5576a850b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:52.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.246+0000 7fcbbffb3b80 1 --2- 172.21.15.133:0/662851863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 0x5576a850b4f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcba009af80 tx=0x7fcba0009ca0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.299 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.246+0000 7fcbbffb3b80 1 -- 172.21.15.133:0/662851863 shutdown_connections 2023-07-14T18:15:52.299 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.246+0000 7fcbbffb3b80 1 --2- 172.21.15.133:0/662851863 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5576a856c8f0 0x5576a8569380 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.299 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.246+0000 7fcbbffb3b80 1 --2- 172.21.15.133:0/662851863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576a8501460 0x5576a850ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.299 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.246+0000 7fcbbffb3b80 1 --2- 172.21.15.133:0/662851863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a84fff10 0x5576a850b4f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.299 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.246+0000 7fcbbffb3b80 1 --2- 172.21.15.133:0/662851863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5576a8500860 0x5576a8510a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.299 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.246+0000 7fcb8dffb700 0 client.15168 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:15:52.299 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.246+0000 7fcbbffb3b80 1 -- 172.21.15.133:0/662851863 >> 172.21.15.133:0/662851863 conn(0x5576a84f5630 msgr2=0x5576a85139c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:52.299 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.246+0000 7fcbbffb3b80 1 -- 172.21.15.133:0/662851863 shutdown_connections 2023-07-14T18:15:52.300 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.246+0000 7fcbbffb3b80 1 -- 172.21.15.133:0/662851863 wait complete. 2023-07-14T18:15:52.300 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.247+0000 7fcbbffb3b80 20 client.15168 trim_cache size 0 max 16384 2023-07-14T18:15:52.300 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:15:52.300 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "06d0f5fa-2271-11ee-9b33-001a4aab830c", 2023-07-14T18:15:52.301 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2023-07-14T18:15:52.301 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24557, 2023-07-14T18:15:52.301 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2023-07-14T18:15:52.301 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2023-07-14T18:15:52.301 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2023-07-14T18:15:52.301 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 94.319744477, 2023-07-14T18:15:52.301 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 25, 2023-07-14T18:15:52.301 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 92, 2023-07-14T18:15:52.302 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 90, 2023-07-14T18:15:52.302 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 101.333527422 2023-07-14T18:15:52.302 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:15:52.302 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.247+0000 7fcbac5ff700 1 -- 172.21.15.133:0/928992333 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fcb900a11e0 msgr2=0x7fcb900a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:52.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.247+0000 7fcbac5ff700 1 --2- 172.21.15.133:0/928992333 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fcb900a11e0 0x7fcb900a3690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcb94009c60 tx=0x7fcb94009260 comp rx=0 tx=0).stop 2023-07-14T18:15:52.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.247+0000 7fcbac5ff700 1 -- 172.21.15.133:0/928992333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcba40ffc20 msgr2=0x7fcba4197bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:52.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.247+0000 7fcbac5ff700 1 --2- 172.21.15.133:0/928992333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcba40ffc20 0x7fcba4197bf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcba0005740 tx=0x7fcba0005770 comp rx=0 tx=0).stop 2023-07-14T18:15:52.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.247+0000 7fcbac5ff700 1 -- 172.21.15.133:0/928992333 shutdown_connections 2023-07-14T18:15:52.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.247+0000 7fcbac5ff700 1 --2- 172.21.15.133:0/928992333 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fcb900a11e0 0x7fcb900a3690 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.247+0000 7fcbac5ff700 1 --2- 172.21.15.133:0/928992333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcba41017a0 0x7fcba419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.247+0000 7fcbac5ff700 1 --2- 172.21.15.133:0/928992333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcba40ffc20 0x7fcba4197bf0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.247+0000 7fcbac5ff700 1 --2- 172.21.15.133:0/928992333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcba4100e10 0x7fcba4198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:52.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.247+0000 7fcbac5ff700 1 -- 172.21.15.133:0/928992333 >> 172.21.15.133:0/928992333 conn(0x7fcba40fb260 msgr2=0x7fcba4106750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:52.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.247+0000 7fcbac5ff700 1 -- 172.21.15.133:0/928992333 shutdown_connections 2023-07-14T18:15:52.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:52.247+0000 7fcbac5ff700 1 -- 172.21.15.133:0/928992333 wait complete. 2023-07-14T18:15:52.310 INFO:tasks.cephadm:Setup complete, yielding 2023-07-14T18:15:52.311 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-14T18:15:52.324 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi133.front.sepia.ceph.com 2023-07-14T18:15:52.324 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- bash -c 'ceph orch status' 2023-07-14T18:15:52.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:51 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1653812202' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:15:52.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:51 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3070276312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:15:52.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:51 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1653812202' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:15:52.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:51 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3070276312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:15:53.039 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:53.314 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:52 smithi133 ceph-mon[126384]: pgmap v324: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 15 B/s, 0 objects/s recovering 2023-07-14T18:15:53.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:52 smithi161 ceph-mon[128993]: pgmap v324: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 15 B/s, 0 objects/s recovering 2023-07-14T18:15:53.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:52 smithi171 ceph-mon[129241]: pgmap v324: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 15 B/s, 0 objects/s recovering 2023-07-14T18:15:54.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.519+0000 7fec6a8a7700 1 -- 172.21.15.133:0/2759148436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec640ffcc0 msgr2=0x7fec641000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:54.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.519+0000 7fec6a8a7700 1 --2- 172.21.15.133:0/2759148436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec640ffcc0 0x7fec641000d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fec54009990 tx=0x7fec54009ca0 comp rx=0 tx=0).stop 2023-07-14T18:15:54.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.519+0000 7fec6a8a7700 1 -- 172.21.15.133:0/2759148436 shutdown_connections 2023-07-14T18:15:54.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.519+0000 7fec6a8a7700 1 --2- 172.21.15.133:0/2759148436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec64101850 0x7fec64108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:54.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.519+0000 7fec6a8a7700 1 --2- 172.21.15.133:0/2759148436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec640ffcc0 0x7fec641000d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:54.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.519+0000 7fec6a8a7700 1 --2- 172.21.15.133:0/2759148436 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec64100ec0 0x7fec64101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:54.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.519+0000 7fec6a8a7700 1 -- 172.21.15.133:0/2759148436 >> 172.21.15.133:0/2759148436 conn(0x7fec640fb2d0 msgr2=0x7fec640fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:54.566 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.519+0000 7fec6a8a7700 1 -- 172.21.15.133:0/2759148436 shutdown_connections 2023-07-14T18:15:54.566 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.519+0000 7fec6a8a7700 1 -- 172.21.15.133:0/2759148436 wait complete. 2023-07-14T18:15:54.567 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.521+0000 7fec6a8a7700 1 Processor -- start 2023-07-14T18:15:54.567 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.521+0000 7fec6a8a7700 1 -- start start 2023-07-14T18:15:54.567 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.521+0000 7fec6a8a7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec640ffcc0 0x7fec64195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:54.567 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.521+0000 7fec6a8a7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec64100ec0 0x7fec64195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:54.567 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.521+0000 7fec6a8a7700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec64101850 0x7fec6419cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:54.567 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.521+0000 7fec6a8a7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6419d5a0 con 0x7fec64100ec0 2023-07-14T18:15:54.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.521+0000 7fec6a8a7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6419d6e0 con 0x7fec640ffcc0 2023-07-14T18:15:54.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.521+0000 7fec6a8a7700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6419d870 con 0x7fec64101850 2023-07-14T18:15:54.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.522+0000 7fec63fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec640ffcc0 0x7fec64195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:54.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.522+0000 7fec637fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec64100ec0 0x7fec64195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:54.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.522+0000 7fec63fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec640ffcc0 0x7fec64195a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:48864/0 (socket says 172.21.15.133:48864) 2023-07-14T18:15:54.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.522+0000 7fec63fff700 1 -- 172.21.15.133:0/2983343353 learned_addr learned my addr 172.21.15.133:0/2983343353 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:54.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.522+0000 7fec68e44700 1 --2- 172.21.15.133:0/2983343353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec64101850 0x7fec6419cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:54.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.522+0000 7fec63fff700 1 -- 172.21.15.133:0/2983343353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec64101850 msgr2=0x7fec6419cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:54.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.522+0000 7fec63fff700 1 --2- 172.21.15.133:0/2983343353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec64101850 0x7fec6419cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:54.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.522+0000 7fec63fff700 1 -- 172.21.15.133:0/2983343353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec64100ec0 msgr2=0x7fec64195f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:54.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.522+0000 7fec63fff700 1 --2- 172.21.15.133:0/2983343353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec64100ec0 0x7fec64195f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:54.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.522+0000 7fec63fff700 1 -- 172.21.15.133:0/2983343353 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec540095f0 con 0x7fec640ffcc0 2023-07-14T18:15:54.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.522+0000 7fec637fe700 1 --2- 172.21.15.133:0/2983343353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec64100ec0 0x7fec64195f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:15:54.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.523+0000 7fec63fff700 1 --2- 172.21.15.133:0/2983343353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec640ffcc0 0x7fec64195a10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fec5400b680 tx=0x7fec54005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:54.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.523+0000 7fec617fa700 1 -- 172.21.15.133:0/2983343353 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5401d070 con 0x7fec640ffcc0 2023-07-14T18:15:54.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.523+0000 7fec6a8a7700 1 -- 172.21.15.133:0/2983343353 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec6419daf0 con 0x7fec640ffcc0 2023-07-14T18:15:54.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.523+0000 7fec617fa700 1 -- 172.21.15.133:0/2983343353 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec54021410 con 0x7fec640ffcc0 2023-07-14T18:15:54.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.523+0000 7fec617fa700 1 -- 172.21.15.133:0/2983343353 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec54017650 con 0x7fec640ffcc0 2023-07-14T18:15:54.572 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.523+0000 7fec6a8a7700 1 -- 172.21.15.133:0/2983343353 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec6419df60 con 0x7fec640ffcc0 2023-07-14T18:15:54.572 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.524+0000 7fec6a8a7700 1 -- 172.21.15.133:0/2983343353 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec64107300 con 0x7fec640ffcc0 2023-07-14T18:15:54.572 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.526+0000 7fec617fa700 1 -- 172.21.15.133:0/2983343353 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fec54021580 con 0x7fec640ffcc0 2023-07-14T18:15:54.573 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.527+0000 7fec617fa700 1 --2- 172.21.15.133:0/2983343353 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fec4c0a1260 0x7fec4c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:54.573 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.528+0000 7fec617fa700 1 -- 172.21.15.133:0/2983343353 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fec540ce810 con 0x7fec640ffcc0 2023-07-14T18:15:54.573 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.528+0000 7fec637fe700 1 --2- 172.21.15.133:0/2983343353 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fec4c0a1260 0x7fec4c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:54.573 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.528+0000 7fec637fe700 1 --2- 172.21.15.133:0/2983343353 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fec4c0a1260 0x7fec4c0a3710 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fec58011490 tx=0x7fec58009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:54.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.529+0000 7fec617fa700 1 -- 172.21.15.133:0/2983343353 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fec54097b00 con 0x7fec640ffcc0 2023-07-14T18:15:54.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.740+0000 7fec6a8a7700 1 -- 172.21.15.133:0/2983343353 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fec64060a80 con 0x7fec4c0a1260 2023-07-14T18:15:54.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.741+0000 7fec617fa700 1 -- 172.21.15.133:0/2983343353 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fec64060a80 con 0x7fec4c0a1260 2023-07-14T18:15:54.786 INFO:teuthology.orchestra.run.smithi133.stdout:Backend: cephadm 2023-07-14T18:15:54.786 INFO:teuthology.orchestra.run.smithi133.stdout:Available: Yes 2023-07-14T18:15:54.786 INFO:teuthology.orchestra.run.smithi133.stdout:Paused: No 2023-07-14T18:15:54.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.743+0000 7fec6a8a7700 1 -- 172.21.15.133:0/2983343353 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fec4c0a1260 msgr2=0x7fec4c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:54.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.743+0000 7fec6a8a7700 1 --2- 172.21.15.133:0/2983343353 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fec4c0a1260 0x7fec4c0a3710 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fec58011490 tx=0x7fec58009260 comp rx=0 tx=0).stop 2023-07-14T18:15:54.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.743+0000 7fec6a8a7700 1 -- 172.21.15.133:0/2983343353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec640ffcc0 msgr2=0x7fec64195a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:54.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.743+0000 7fec6a8a7700 1 --2- 172.21.15.133:0/2983343353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec640ffcc0 0x7fec64195a10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fec5400b680 tx=0x7fec54005740 comp rx=0 tx=0).stop 2023-07-14T18:15:54.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.743+0000 7fec6a8a7700 1 -- 172.21.15.133:0/2983343353 shutdown_connections 2023-07-14T18:15:54.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.743+0000 7fec6a8a7700 1 --2- 172.21.15.133:0/2983343353 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fec4c0a1260 0x7fec4c0a3710 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:54.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.743+0000 7fec6a8a7700 1 --2- 172.21.15.133:0/2983343353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec640ffcc0 0x7fec64195a10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:54.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.743+0000 7fec6a8a7700 1 --2- 172.21.15.133:0/2983343353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec64101850 0x7fec6419cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:54.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.743+0000 7fec6a8a7700 1 --2- 172.21.15.133:0/2983343353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec64100ec0 0x7fec64195f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:54.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.743+0000 7fec6a8a7700 1 -- 172.21.15.133:0/2983343353 >> 172.21.15.133:0/2983343353 conn(0x7fec640fb2d0 msgr2=0x7fec64104440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:54.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.744+0000 7fec6a8a7700 1 -- 172.21.15.133:0/2983343353 shutdown_connections 2023-07-14T18:15:54.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:54.744+0000 7fec6a8a7700 1 -- 172.21.15.133:0/2983343353 wait complete. 2023-07-14T18:15:55.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:54 smithi133 ceph-mon[126384]: pgmap v325: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 15 B/s, 0 objects/s recovering 2023-07-14T18:15:55.232 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- bash -c 'ceph orch ps' 2023-07-14T18:15:55.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:54 smithi161 ceph-mon[128993]: pgmap v325: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 15 B/s, 0 objects/s recovering 2023-07-14T18:15:55.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:54 smithi171 ceph-mon[129241]: pgmap v325: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 15 B/s, 0 objects/s recovering 2023-07-14T18:15:55.952 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:56.282 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:55 smithi133 ceph-mon[126384]: from='client.25157 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:15:56.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:55 smithi161 ceph-mon[128993]: from='client.25157 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:15:56.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:55 smithi171 ceph-mon[129241]: from='client.25157 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:15:57.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:56 smithi133 ceph-mon[126384]: pgmap v326: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 15 B/s, 0 objects/s recovering 2023-07-14T18:15:57.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:56 smithi161 ceph-mon[128993]: pgmap v326: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 15 B/s, 0 objects/s recovering 2023-07-14T18:15:57.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:56 smithi171 ceph-mon[129241]: pgmap v326: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 15 B/s, 0 objects/s recovering 2023-07-14T18:15:57.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.352+0000 7fe99542d700 1 -- 172.21.15.133:0/1740675330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe990101850 msgr2=0x7fe990108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:57.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.352+0000 7fe99542d700 1 --2- 172.21.15.133:0/1740675330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe990101850 0x7fe990108110 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe98800b0a0 tx=0x7fe98800b3b0 comp rx=0 tx=0).stop 2023-07-14T18:15:57.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.352+0000 7fe99542d700 1 -- 172.21.15.133:0/1740675330 shutdown_connections 2023-07-14T18:15:57.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.352+0000 7fe99542d700 1 --2- 172.21.15.133:0/1740675330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9900ffcc0 0x7fe9901000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:57.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.352+0000 7fe99542d700 1 --2- 172.21.15.133:0/1740675330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe990101850 0x7fe990108110 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:57.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.352+0000 7fe99542d700 1 --2- 172.21.15.133:0/1740675330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe990100ec0 0x7fe990101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:57.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.352+0000 7fe99542d700 1 -- 172.21.15.133:0/1740675330 >> 172.21.15.133:0/1740675330 conn(0x7fe9900fb2d0 msgr2=0x7fe9900fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:57.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.352+0000 7fe99542d700 1 -- 172.21.15.133:0/1740675330 shutdown_connections 2023-07-14T18:15:57.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.352+0000 7fe99542d700 1 -- 172.21.15.133:0/1740675330 wait complete. 2023-07-14T18:15:57.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.353+0000 7fe99542d700 1 Processor -- start 2023-07-14T18:15:57.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.353+0000 7fe99542d700 1 -- start start 2023-07-14T18:15:57.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.354+0000 7fe99542d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9900ffcc0 0x7fe990197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:57.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.354+0000 7fe99542d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe990100ec0 0x7fe9901981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:57.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.354+0000 7fe99542d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe990101850 0x7fe99019f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:57.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.354+0000 7fe99542d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99019f810 con 0x7fe9900ffcc0 2023-07-14T18:15:57.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.354+0000 7fe99542d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99019f950 con 0x7fe990101850 2023-07-14T18:15:57.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.354+0000 7fe99542d700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99019fb10 con 0x7fe990100ec0 2023-07-14T18:15:57.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.354+0000 7fe98f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe990101850 0x7fe99019f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:57.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.354+0000 7fe98effd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9900ffcc0 0x7fe990197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:57.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.354+0000 7fe98e7fc700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe990100ec0 0x7fe9901981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:57.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.354+0000 7fe98f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe990101850 0x7fe99019f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:35792/0 (socket says 172.21.15.133:35792) 2023-07-14T18:15:57.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.354+0000 7fe98f7fe700 1 -- 172.21.15.133:0/60051925 learned_addr learned my addr 172.21.15.133:0/60051925 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:15:57.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.354+0000 7fe98effd700 1 -- 172.21.15.133:0/60051925 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe990100ec0 msgr2=0x7fe9901981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:57.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.354+0000 7fe98effd700 1 --2- 172.21.15.133:0/60051925 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe990100ec0 0x7fe9901981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:57.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.354+0000 7fe98effd700 1 -- 172.21.15.133:0/60051925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe990101850 msgr2=0x7fe99019f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:57.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.354+0000 7fe98effd700 1 --2- 172.21.15.133:0/60051925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe990101850 0x7fe99019f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:57.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.354+0000 7fe98effd700 1 -- 172.21.15.133:0/60051925 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe988009d00 con 0x7fe9900ffcc0 2023-07-14T18:15:57.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.355+0000 7fe98effd700 1 --2- 172.21.15.133:0/60051925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9900ffcc0 0x7fe990197c90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe98000e9c0 tx=0x7fe98000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:57.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.355+0000 7fe977fff700 1 -- 172.21.15.133:0/60051925 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980004220 con 0x7fe9900ffcc0 2023-07-14T18:15:57.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.355+0000 7fe977fff700 1 -- 172.21.15.133:0/60051925 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe980004380 con 0x7fe9900ffcc0 2023-07-14T18:15:57.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.355+0000 7fe977fff700 1 -- 172.21.15.133:0/60051925 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980003be0 con 0x7fe9900ffcc0 2023-07-14T18:15:57.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.355+0000 7fe99542d700 1 -- 172.21.15.133:0/60051925 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe99019fdf0 con 0x7fe9900ffcc0 2023-07-14T18:15:57.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.355+0000 7fe99542d700 1 -- 172.21.15.133:0/60051925 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9901a0210 con 0x7fe9900ffcc0 2023-07-14T18:15:57.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.356+0000 7fe99542d700 1 -- 172.21.15.133:0/60051925 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe990192350 con 0x7fe9900ffcc0 2023-07-14T18:15:57.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.358+0000 7fe977fff700 1 -- 172.21.15.133:0/60051925 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fe980003d40 con 0x7fe9900ffcc0 2023-07-14T18:15:57.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.360+0000 7fe977fff700 1 --2- 172.21.15.133:0/60051925 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe9780a1260 0x7fe9780a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:15:57.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.360+0000 7fe977fff700 1 -- 172.21.15.133:0/60051925 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe980014070 con 0x7fe9900ffcc0 2023-07-14T18:15:57.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.360+0000 7fe98e7fc700 1 --2- 172.21.15.133:0/60051925 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe9780a1260 0x7fe9780a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:15:57.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.361+0000 7fe98e7fc700 1 --2- 172.21.15.133:0/60051925 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe9780a1260 0x7fe9780a3710 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe9901991a0 tx=0x7fe984009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:15:57.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.364+0000 7fe977fff700 1 -- 172.21.15.133:0/60051925 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fe980096280 con 0x7fe9900ffcc0 2023-07-14T18:15:57.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.570+0000 7fe99542d700 1 -- 172.21.15.133:0/60051925 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe99010b490 con 0x7fe9780a1260 2023-07-14T18:15:57.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.583+0000 7fe977fff700 1 -- 172.21.15.133:0/60051925 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7fe984011a40 con 0x7fe9780a1260 2023-07-14T18:15:57.628 INFO:teuthology.orchestra.run.smithi133.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-07-14T18:15:57.628 INFO:teuthology.orchestra.run.smithi133.stdout:mds.a smithi133 running (109s) - 108s 13.7M - 18.0.0-4940-g81c9387d 219b180907a8 a0aa1db88f11 2023-07-14T18:15:57.628 INFO:teuthology.orchestra.run.smithi133.stdout:mds.b smithi161 running (106s) - 106s 30.2M - 18.0.0-4940-g81c9387d 219b180907a8 243dee7a58e5 2023-07-14T18:15:57.629 INFO:teuthology.orchestra.run.smithi133.stdout:mds.c smithi171 running (119s) - 119s 18.4M - 18.0.0-4940-g81c9387d 219b180907a8 054f51021a87 2023-07-14T18:15:57.629 INFO:teuthology.orchestra.run.smithi133.stdout:mds.d smithi133 running (2m) - 2m 13.8M - 18.0.0-4940-g81c9387d 219b180907a8 5e4aae41fd12 2023-07-14T18:15:57.629 INFO:teuthology.orchestra.run.smithi133.stdout:mds.e smithi161 running (2m) - 2m 16.8M - 18.0.0-4940-g81c9387d 219b180907a8 d2d3f6e9dcbf 2023-07-14T18:15:57.629 INFO:teuthology.orchestra.run.smithi133.stdout:mds.f smithi171 running (2m) - 2m 20.5M - 18.0.0-4940-g81c9387d 219b180907a8 68f88c720a0e 2023-07-14T18:15:57.629 INFO:teuthology.orchestra.run.smithi133.stdout:mds.g smithi133 running (112s) - 111s 13.7M - 18.0.0-4940-g81c9387d 219b180907a8 c3f959794e1e 2023-07-14T18:15:57.629 INFO:teuthology.orchestra.run.smithi133.stdout:mds.h smithi161 running (117s) - 117s 17.2M - 18.0.0-4940-g81c9387d 219b180907a8 ff4ec1a7d739 2023-07-14T18:15:57.629 INFO:teuthology.orchestra.run.smithi133.stdout:mds.i smithi171 running (115s) - 115s 18.9M - 18.0.0-4940-g81c9387d 219b180907a8 3bb09798cd6d 2023-07-14T18:15:57.629 INFO:teuthology.orchestra.run.smithi133.stdout:mds.j smithi133 running (2m) - 2m 13.7M - 18.0.0-4940-g81c9387d 219b180907a8 04633dea74e4 2023-07-14T18:15:57.630 INFO:teuthology.orchestra.run.smithi133.stdout:mds.k smithi161 running (2m) - 2m 17.5M - 18.0.0-4940-g81c9387d 219b180907a8 c6cf83c92bd5 2023-07-14T18:15:57.630 INFO:teuthology.orchestra.run.smithi133.stdout:mds.l smithi171 running (2m) - 2m 15.0M - 18.0.0-4940-g81c9387d 219b180907a8 c15c523df221 2023-07-14T18:15:57.630 INFO:teuthology.orchestra.run.smithi133.stdout:mgr.x smithi133 *:9283,8765 running (9m) - 8m 522M - 18.0.0-4940-g81c9387d 219b180907a8 4dbf6a4c7596 2023-07-14T18:15:57.630 INFO:teuthology.orchestra.run.smithi133.stdout:mgr.y smithi161 *:8443,8765 running (7m) - 7m 449M - 18.0.0-4940-g81c9387d 219b180907a8 fa7f8318e072 2023-07-14T18:15:57.630 INFO:teuthology.orchestra.run.smithi133.stdout:mgr.z smithi171 *:8443,8765 running (7m) - 7m 448M - 18.0.0-4940-g81c9387d 219b180907a8 b11540419118 2023-07-14T18:15:57.630 INFO:teuthology.orchestra.run.smithi133.stdout:mon.a smithi133 running (9m) - 9m 94.0M 2048M 18.0.0-4940-g81c9387d 219b180907a8 e06732b37106 2023-07-14T18:15:57.630 INFO:teuthology.orchestra.run.smithi133.stdout:mon.b smithi161 running (7m) - 7m 62.4M 2048M 18.0.0-4940-g81c9387d 219b180907a8 553187cae4be 2023-07-14T18:15:57.630 INFO:teuthology.orchestra.run.smithi133.stdout:mon.c smithi171 running (7m) - 7m 62.8M 2048M 18.0.0-4940-g81c9387d 219b180907a8 df8474c85524 2023-07-14T18:15:57.631 INFO:teuthology.orchestra.run.smithi133.stdout:osd.0 smithi133 running (6m) - 6m 71.1M 4316M 18.0.0-4940-g81c9387d 219b180907a8 67bc1a856de4 2023-07-14T18:15:57.631 INFO:teuthology.orchestra.run.smithi133.stdout:osd.1 smithi161 running (6m) - 6m 78.5M 4292M 18.0.0-4940-g81c9387d 219b180907a8 d3726a65c0e7 2023-07-14T18:15:57.631 INFO:teuthology.orchestra.run.smithi133.stdout:osd.2 smithi171 running (6m) - 6m 75.3M 4292M 18.0.0-4940-g81c9387d 219b180907a8 b62de549f7a3 2023-07-14T18:15:57.631 INFO:teuthology.orchestra.run.smithi133.stdout:osd.3 smithi133 running (5m) - 5m 65.0M 4316M 18.0.0-4940-g81c9387d 219b180907a8 04e8b9e6212b 2023-07-14T18:15:57.631 INFO:teuthology.orchestra.run.smithi133.stdout:osd.4 smithi161 running (5m) - 5m 64.2M 4292M 18.0.0-4940-g81c9387d 219b180907a8 2f134f4a0f2e 2023-07-14T18:15:57.631 INFO:teuthology.orchestra.run.smithi133.stdout:osd.5 smithi171 running (4m) - 4m 68.5M 4292M 18.0.0-4940-g81c9387d 219b180907a8 6187a0e7ef9a 2023-07-14T18:15:57.631 INFO:teuthology.orchestra.run.smithi133.stdout:osd.6 smithi133 running (4m) - 4m 62.3M 4316M 18.0.0-4940-g81c9387d 219b180907a8 7d641f125453 2023-07-14T18:15:57.632 INFO:teuthology.orchestra.run.smithi133.stdout:osd.7 smithi161 running (4m) - 4m 67.8M 4292M 18.0.0-4940-g81c9387d 219b180907a8 a5c7a2d74ee0 2023-07-14T18:15:57.632 INFO:teuthology.orchestra.run.smithi133.stdout:osd.8 smithi171 running (3m) - 3m 65.4M 4292M 18.0.0-4940-g81c9387d 219b180907a8 5852d470240b 2023-07-14T18:15:57.632 INFO:teuthology.orchestra.run.smithi133.stdout:osd.9 smithi133 running (3m) - 3m 60.3M 4316M 18.0.0-4940-g81c9387d 219b180907a8 b4acc969ba8d 2023-07-14T18:15:57.632 INFO:teuthology.orchestra.run.smithi133.stdout:osd.10 smithi161 running (2m) - 2m 61.8M 4292M 18.0.0-4940-g81c9387d 219b180907a8 262682737a28 2023-07-14T18:15:57.632 INFO:teuthology.orchestra.run.smithi133.stdout:osd.11 smithi171 running (2m) - 2m 55.3M 4292M 18.0.0-4940-g81c9387d 219b180907a8 06074cb09d5f 2023-07-14T18:15:57.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.585+0000 7fe99542d700 1 -- 172.21.15.133:0/60051925 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe9780a1260 msgr2=0x7fe9780a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:57.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.585+0000 7fe99542d700 1 --2- 172.21.15.133:0/60051925 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe9780a1260 0x7fe9780a3710 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe9901991a0 tx=0x7fe984009260 comp rx=0 tx=0).stop 2023-07-14T18:15:57.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.585+0000 7fe99542d700 1 -- 172.21.15.133:0/60051925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9900ffcc0 msgr2=0x7fe990197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:15:57.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.585+0000 7fe99542d700 1 --2- 172.21.15.133:0/60051925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9900ffcc0 0x7fe990197c90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe98000e9c0 tx=0x7fe98000ecd0 comp rx=0 tx=0).stop 2023-07-14T18:15:57.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.585+0000 7fe99542d700 1 -- 172.21.15.133:0/60051925 shutdown_connections 2023-07-14T18:15:57.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.585+0000 7fe99542d700 1 --2- 172.21.15.133:0/60051925 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe9780a1260 0x7fe9780a3710 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:57.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.585+0000 7fe99542d700 1 --2- 172.21.15.133:0/60051925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe990101850 0x7fe99019f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:57.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.585+0000 7fe99542d700 1 --2- 172.21.15.133:0/60051925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9900ffcc0 0x7fe990197c90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:57.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.585+0000 7fe99542d700 1 --2- 172.21.15.133:0/60051925 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe990100ec0 0x7fe9901981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:15:57.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.585+0000 7fe99542d700 1 -- 172.21.15.133:0/60051925 >> 172.21.15.133:0/60051925 conn(0x7fe9900fb2d0 msgr2=0x7fe9901067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:15:57.635 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.585+0000 7fe99542d700 1 -- 172.21.15.133:0/60051925 shutdown_connections 2023-07-14T18:15:57.635 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:15:57.585+0000 7fe99542d700 1 -- 172.21.15.133:0/60051925 wait complete. 2023-07-14T18:15:58.146 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- bash -c 'ceph orch ls' 2023-07-14T18:15:58.867 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:15:59.331 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:58 smithi133 ceph-mon[126384]: pgmap v327: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 4 op/s; 14 B/s, 0 objects/s recovering 2023-07-14T18:15:59.331 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:15:58 smithi133 ceph-mon[126384]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:15:59.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:58 smithi161 ceph-mon[128993]: pgmap v327: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 4 op/s; 14 B/s, 0 objects/s recovering 2023-07-14T18:15:59.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:15:58 smithi161 ceph-mon[128993]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:15:59.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:58 smithi171 ceph-mon[129241]: pgmap v327: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 4 op/s; 14 B/s, 0 objects/s recovering 2023-07-14T18:15:59.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:15:58 smithi171 ceph-mon[129241]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:16:00.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.325+0000 7f1ea2ea0700 1 -- 172.21.15.133:0/1817430420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e9c1030f0 msgr2=0x7f1e9c103560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:00.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.325+0000 7f1ea2ea0700 1 --2- 172.21.15.133:0/1817430420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e9c1030f0 0x7f1e9c103560 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009990 tx=0x7f1e88009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:00.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.325+0000 7f1ea2ea0700 1 -- 172.21.15.133:0/1817430420 shutdown_connections 2023-07-14T18:16:00.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.325+0000 7f1ea2ea0700 1 --2- 172.21.15.133:0/1817430420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e9c103aa0 0x7f1e9c10a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:00.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.325+0000 7f1ea2ea0700 1 --2- 172.21.15.133:0/1817430420 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e9c101f10 0x7f1e9c102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:00.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.325+0000 7f1ea2ea0700 1 --2- 172.21.15.133:0/1817430420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e9c1030f0 0x7f1e9c103560 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:00.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.325+0000 7f1ea2ea0700 1 -- 172.21.15.133:0/1817430420 >> 172.21.15.133:0/1817430420 conn(0x7f1e9c0fd570 msgr2=0x7f1e9c0ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:00.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.325+0000 7f1ea2ea0700 1 -- 172.21.15.133:0/1817430420 shutdown_connections 2023-07-14T18:16:00.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.325+0000 7f1ea2ea0700 1 -- 172.21.15.133:0/1817430420 wait complete. 2023-07-14T18:16:00.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.326+0000 7f1ea2ea0700 1 Processor -- start 2023-07-14T18:16:00.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.326+0000 7f1ea2ea0700 1 -- start start 2023-07-14T18:16:00.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.327+0000 7f1ea2ea0700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e9c101f10 0x7f1e9c197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:00.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.327+0000 7f1ea2ea0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e9c1030f0 0x7f1e9c198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:00.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.327+0000 7f1ea2ea0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e9c103aa0 0x7f1e9c19f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:00.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.327+0000 7f1ea2ea0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c19f750 con 0x7f1e9c103aa0 2023-07-14T18:16:00.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.327+0000 7f1ea2ea0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c19f890 con 0x7f1e9c1030f0 2023-07-14T18:16:00.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.327+0000 7f1ea2ea0700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c19fa20 con 0x7f1e9c101f10 2023-07-14T18:16:00.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.327+0000 7f1ea143d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e9c103aa0 0x7f1e9c19f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:00.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.327+0000 7f1ea0c3c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e9c101f10 0x7f1e9c197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:00.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.327+0000 7f1ea143d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e9c103aa0 0x7f1e9c19f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41446/0 (socket says 172.21.15.133:41446) 2023-07-14T18:16:00.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.327+0000 7f1e9bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e9c1030f0 0x7f1e9c198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:00.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.327+0000 7f1ea143d700 1 -- 172.21.15.133:0/3205505999 learned_addr learned my addr 172.21.15.133:0/3205505999 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:00.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.328+0000 7f1ea143d700 1 -- 172.21.15.133:0/3205505999 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e9c101f10 msgr2=0x7f1e9c197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:00.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.328+0000 7f1ea143d700 1 --2- 172.21.15.133:0/3205505999 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e9c101f10 0x7f1e9c197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:00.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.328+0000 7f1ea143d700 1 -- 172.21.15.133:0/3205505999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e9c1030f0 msgr2=0x7f1e9c198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:00.377 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.328+0000 7f1ea143d700 1 --2- 172.21.15.133:0/3205505999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e9c1030f0 0x7f1e9c198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:00.377 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.328+0000 7f1ea143d700 1 -- 172.21.15.133:0/3205505999 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e880095f0 con 0x7f1e9c103aa0 2023-07-14T18:16:00.377 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.328+0000 7f1ea143d700 1 --2- 172.21.15.133:0/3205505999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e9c103aa0 0x7f1e9c19f180 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1e9400ef60 tx=0x7f1e9400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:00.377 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.328+0000 7f1e99ffb700 1 -- 172.21.15.133:0/3205505999 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e94018070 con 0x7f1e9c103aa0 2023-07-14T18:16:00.377 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.328+0000 7f1ea2ea0700 1 -- 172.21.15.133:0/3205505999 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e9c19fca0 con 0x7f1e9c103aa0 2023-07-14T18:16:00.377 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.328+0000 7f1e99ffb700 1 -- 172.21.15.133:0/3205505999 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e9400f040 con 0x7f1e9c103aa0 2023-07-14T18:16:00.377 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.328+0000 7f1e99ffb700 1 -- 172.21.15.133:0/3205505999 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9400ccc0 con 0x7f1e9c103aa0 2023-07-14T18:16:00.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.328+0000 7f1ea2ea0700 1 -- 172.21.15.133:0/3205505999 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e9c1a01a0 con 0x7f1e9c103aa0 2023-07-14T18:16:00.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.329+0000 7f1ea2ea0700 1 -- 172.21.15.133:0/3205505999 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e9c109570 con 0x7f1e9c103aa0 2023-07-14T18:16:00.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.331+0000 7f1e99ffb700 1 -- 172.21.15.133:0/3205505999 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f1e94003a00 con 0x7f1e9c103aa0 2023-07-14T18:16:00.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.332+0000 7f1e99ffb700 1 --2- 172.21.15.133:0/3205505999 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f1e8c0a11e0 0x7f1e8c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:00.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.333+0000 7f1e99ffb700 1 -- 172.21.15.133:0/3205505999 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1e940cc8f0 con 0x7f1e9c103aa0 2023-07-14T18:16:00.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.333+0000 7f1ea0c3c700 1 --2- 172.21.15.133:0/3205505999 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f1e8c0a11e0 0x7f1e8c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:00.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.333+0000 7f1ea0c3c700 1 --2- 172.21.15.133:0/3205505999 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f1e8c0a11e0 0x7f1e8c0a3690 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c101c40 tx=0x7f1e90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:00.381 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.336+0000 7f1e99ffb700 1 -- 172.21.15.133:0/3205505999 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f1e94095be0 con 0x7f1e9c103aa0 2023-07-14T18:16:00.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.538+0000 7f1ea2ea0700 1 -- 172.21.15.133:0/3205505999 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e9c060a80 con 0x7f1e8c0a11e0 2023-07-14T18:16:00.588 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.543+0000 7f1e99ffb700 1 -- 172.21.15.133:0/3205505999 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f1e9c060a80 con 0x7f1e8c0a11e0 2023-07-14T18:16:00.588 INFO:teuthology.orchestra.run.smithi133.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-07-14T18:16:00.588 INFO:teuthology.orchestra.run.smithi133.stdout:mds.all 12/12 - 2m smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2023-07-14T18:16:00.588 INFO:teuthology.orchestra.run.smithi133.stdout:mgr 3/3 - 7m smithi133=x;smithi161=y;smithi171=z;count:3 2023-07-14T18:16:00.589 INFO:teuthology.orchestra.run.smithi133.stdout:mon 3/3 - 7m smithi133:172.21.15.133=a;smithi161:172.21.15.161=b;smithi171:172.21.15.171=c;count:3 2023-07-14T18:16:00.589 INFO:teuthology.orchestra.run.smithi133.stdout:osd 12 - - 2023-07-14T18:16:00.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.546+0000 7f1ea2ea0700 1 -- 172.21.15.133:0/3205505999 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f1e8c0a11e0 msgr2=0x7f1e8c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:00.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.546+0000 7f1ea2ea0700 1 --2- 172.21.15.133:0/3205505999 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f1e8c0a11e0 0x7f1e8c0a3690 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c101c40 tx=0x7f1e90009260 comp rx=0 tx=0).stop 2023-07-14T18:16:00.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.546+0000 7f1ea2ea0700 1 -- 172.21.15.133:0/3205505999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e9c103aa0 msgr2=0x7f1e9c19f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:00.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.546+0000 7f1ea2ea0700 1 --2- 172.21.15.133:0/3205505999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e9c103aa0 0x7f1e9c19f180 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1e9400ef60 tx=0x7f1e9400ef90 comp rx=0 tx=0).stop 2023-07-14T18:16:00.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.546+0000 7f1ea2ea0700 1 -- 172.21.15.133:0/3205505999 shutdown_connections 2023-07-14T18:16:00.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.546+0000 7f1ea2ea0700 1 --2- 172.21.15.133:0/3205505999 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f1e8c0a11e0 0x7f1e8c0a3690 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:00.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.546+0000 7f1ea2ea0700 1 --2- 172.21.15.133:0/3205505999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e9c103aa0 0x7f1e9c19f180 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:00.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.546+0000 7f1ea2ea0700 1 --2- 172.21.15.133:0/3205505999 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e9c101f10 0x7f1e9c197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:00.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.546+0000 7f1ea2ea0700 1 --2- 172.21.15.133:0/3205505999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e9c1030f0 0x7f1e9c198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:00.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.546+0000 7f1ea2ea0700 1 -- 172.21.15.133:0/3205505999 >> 172.21.15.133:0/3205505999 conn(0x7f1e9c0fd570 msgr2=0x7f1e9c1066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:00.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.546+0000 7f1ea2ea0700 1 -- 172.21.15.133:0/3205505999 shutdown_connections 2023-07-14T18:16:00.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:00.546+0000 7f1ea2ea0700 1 -- 172.21.15.133:0/3205505999 wait complete. 2023-07-14T18:16:01.066 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- bash -c 'ceph orch host ls' 2023-07-14T18:16:01.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:00 smithi161 ceph-mon[128993]: pgmap v328: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 12 B/s, 0 objects/s recovering 2023-07-14T18:16:01.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:00 smithi171 ceph-mon[129241]: pgmap v328: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 12 B/s, 0 objects/s recovering 2023-07-14T18:16:01.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:00 smithi133 ceph-mon[126384]: pgmap v328: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 12 B/s, 0 objects/s recovering 2023-07-14T18:16:01.787 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:16:02.292 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:01 smithi133 ceph-mon[126384]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:16:02.350 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:01 smithi161 ceph-mon[128993]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:16:02.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:01 smithi171 ceph-mon[129241]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:16:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.144+0000 7ff6fde1e700 1 -- 172.21.15.133:0/678247579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6f8100ec0 msgr2=0x7ff6f8101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.144+0000 7ff6fde1e700 1 --2- 172.21.15.133:0/678247579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6f8100ec0 0x7ff6f8101310 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec009990 tx=0x7ff6ec009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.144+0000 7ff6fde1e700 1 -- 172.21.15.133:0/678247579 shutdown_connections 2023-07-14T18:16:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.144+0000 7ff6fde1e700 1 --2- 172.21.15.133:0/678247579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6f80ffcc0 0x7ff6f81000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:03.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.144+0000 7ff6fde1e700 1 --2- 172.21.15.133:0/678247579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6f8101850 0x7ff6f8108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:03.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.144+0000 7ff6fde1e700 1 --2- 172.21.15.133:0/678247579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6f8100ec0 0x7ff6f8101310 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:03.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.144+0000 7ff6fde1e700 1 -- 172.21.15.133:0/678247579 >> 172.21.15.133:0/678247579 conn(0x7ff6f80fb2d0 msgr2=0x7ff6f80fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:03.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.144+0000 7ff6fde1e700 1 -- 172.21.15.133:0/678247579 shutdown_connections 2023-07-14T18:16:03.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.144+0000 7ff6fde1e700 1 -- 172.21.15.133:0/678247579 wait complete. 2023-07-14T18:16:03.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.145+0000 7ff6fde1e700 1 Processor -- start 2023-07-14T18:16:03.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.145+0000 7ff6fde1e700 1 -- start start 2023-07-14T18:16:03.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.145+0000 7ff6fde1e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6f80ffcc0 0x7ff6f8195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:03.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.146+0000 7ff6fde1e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6f8100ec0 0x7ff6f8195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:03.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.146+0000 7ff6fde1e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6f8101850 0x7ff6f819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:03.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.146+0000 7ff6fde1e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f819d590 con 0x7ff6f80ffcc0 2023-07-14T18:16:03.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.146+0000 7ff6fde1e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f819d6d0 con 0x7ff6f8100ec0 2023-07-14T18:16:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.146+0000 7ff6fde1e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f819d860 con 0x7ff6f8101850 2023-07-14T18:16:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.146+0000 7ff6f77fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6f80ffcc0 0x7ff6f8195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.146+0000 7ff6f77fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6f80ffcc0 0x7ff6f8195a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41462/0 (socket says 172.21.15.133:41462) 2023-07-14T18:16:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.146+0000 7ff6f77fe700 1 -- 172.21.15.133:0/2462385737 learned_addr learned my addr 172.21.15.133:0/2462385737 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.146+0000 7ff6f6ffd700 1 --2- 172.21.15.133:0/2462385737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6f8100ec0 0x7ff6f8195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.146+0000 7ff6f7fff700 1 --2- 172.21.15.133:0/2462385737 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6f8101850 0x7ff6f819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.146+0000 7ff6f77fe700 1 -- 172.21.15.133:0/2462385737 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6f8101850 msgr2=0x7ff6f819cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.146+0000 7ff6f77fe700 1 --2- 172.21.15.133:0/2462385737 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6f8101850 0x7ff6f819cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:03.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.146+0000 7ff6f77fe700 1 -- 172.21.15.133:0/2462385737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6f8100ec0 msgr2=0x7ff6f8195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:03.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.146+0000 7ff6f77fe700 1 --2- 172.21.15.133:0/2462385737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6f8100ec0 0x7ff6f8195f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:03.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.146+0000 7ff6f77fe700 1 -- 172.21.15.133:0/2462385737 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ec0095f0 con 0x7ff6f80ffcc0 2023-07-14T18:16:03.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.146+0000 7ff6f77fe700 1 --2- 172.21.15.133:0/2462385737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6f80ffcc0 0x7ff6f8195a00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff6e800d9b0 tx=0x7ff6e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:03.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.147+0000 7ff6f4ff9700 1 -- 172.21.15.133:0/2462385737 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e800b950 con 0x7ff6f80ffcc0 2023-07-14T18:16:03.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.147+0000 7ff6fde1e700 1 -- 172.21.15.133:0/2462385737 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f819db40 con 0x7ff6f80ffcc0 2023-07-14T18:16:03.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.147+0000 7ff6f4ff9700 1 -- 172.21.15.133:0/2462385737 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6e8007620 con 0x7ff6f80ffcc0 2023-07-14T18:16:03.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.147+0000 7ff6f4ff9700 1 -- 172.21.15.133:0/2462385737 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e800f3c0 con 0x7ff6f80ffcc0 2023-07-14T18:16:03.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.147+0000 7ff6fde1e700 1 -- 172.21.15.133:0/2462385737 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f819e090 con 0x7ff6f80ffcc0 2023-07-14T18:16:03.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.148+0000 7ff6fde1e700 1 -- 172.21.15.133:0/2462385737 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f81072c0 con 0x7ff6f80ffcc0 2023-07-14T18:16:03.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.150+0000 7ff6f4ff9700 1 -- 172.21.15.133:0/2462385737 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7ff6e8010460 con 0x7ff6f80ffcc0 2023-07-14T18:16:03.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.151+0000 7ff6f4ff9700 1 --2- 172.21.15.133:0/2462385737 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff6e00a1260 0x7ff6e00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:03.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.151+0000 7ff6f4ff9700 1 -- 172.21.15.133:0/2462385737 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff6e80ccc30 con 0x7ff6f80ffcc0 2023-07-14T18:16:03.197 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.152+0000 7ff6f6ffd700 1 --2- 172.21.15.133:0/2462385737 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff6e00a1260 0x7ff6e00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:03.197 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.152+0000 7ff6f6ffd700 1 --2- 172.21.15.133:0/2462385737 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff6e00a1260 0x7ff6e00a3710 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec00b2e0 tx=0x7ff6ec005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:03.200 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.155+0000 7ff6f4ff9700 1 -- 172.21.15.133:0/2462385737 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7ff6e8095e90 con 0x7ff6f80ffcc0 2023-07-14T18:16:03.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:02 smithi161 ceph-mon[128993]: pgmap v329: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 10 B/s, 0 objects/s recovering 2023-07-14T18:16:03.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:02 smithi171 ceph-mon[129241]: pgmap v329: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 10 B/s, 0 objects/s recovering 2023-07-14T18:16:03.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.355+0000 7ff6fde1e700 1 -- 172.21.15.133:0/2462385737 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6f8060a80 con 0x7ff6e00a1260 2023-07-14T18:16:03.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.357+0000 7ff6f4ff9700 1 -- 172.21.15.133:0/2462385737 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7ff6f8060a80 con 0x7ff6e00a1260 2023-07-14T18:16:03.402 INFO:teuthology.orchestra.run.smithi133.stdout:HOST ADDR LABELS STATUS 2023-07-14T18:16:03.402 INFO:teuthology.orchestra.run.smithi133.stdout:smithi133 172.21.15.133 2023-07-14T18:16:03.402 INFO:teuthology.orchestra.run.smithi133.stdout:smithi161 172.21.15.161 2023-07-14T18:16:03.403 INFO:teuthology.orchestra.run.smithi133.stdout:smithi171 172.21.15.171 2023-07-14T18:16:03.403 INFO:teuthology.orchestra.run.smithi133.stdout:3 hosts in cluster 2023-07-14T18:16:03.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.359+0000 7ff6fde1e700 1 -- 172.21.15.133:0/2462385737 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff6e00a1260 msgr2=0x7ff6e00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:03.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.359+0000 7ff6fde1e700 1 --2- 172.21.15.133:0/2462385737 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff6e00a1260 0x7ff6e00a3710 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec00b2e0 tx=0x7ff6ec005fb0 comp rx=0 tx=0).stop 2023-07-14T18:16:03.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.360+0000 7ff6fde1e700 1 -- 172.21.15.133:0/2462385737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6f80ffcc0 msgr2=0x7ff6f8195a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:03.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.360+0000 7ff6fde1e700 1 --2- 172.21.15.133:0/2462385737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6f80ffcc0 0x7ff6f8195a00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff6e800d9b0 tx=0x7ff6e800dcc0 comp rx=0 tx=0).stop 2023-07-14T18:16:03.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.360+0000 7ff6fde1e700 1 -- 172.21.15.133:0/2462385737 shutdown_connections 2023-07-14T18:16:03.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.360+0000 7ff6fde1e700 1 --2- 172.21.15.133:0/2462385737 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff6e00a1260 0x7ff6e00a3710 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:03.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.360+0000 7ff6fde1e700 1 --2- 172.21.15.133:0/2462385737 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6f8101850 0x7ff6f819cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:03.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.360+0000 7ff6fde1e700 1 --2- 172.21.15.133:0/2462385737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6f80ffcc0 0x7ff6f8195a00 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:03.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.360+0000 7ff6fde1e700 1 --2- 172.21.15.133:0/2462385737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6f8100ec0 0x7ff6f8195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:03.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.360+0000 7ff6fde1e700 1 -- 172.21.15.133:0/2462385737 >> 172.21.15.133:0/2462385737 conn(0x7ff6f80fb2d0 msgr2=0x7ff6f8104400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:03.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.360+0000 7ff6fde1e700 1 -- 172.21.15.133:0/2462385737 shutdown_connections 2023-07-14T18:16:03.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:03.360+0000 7ff6fde1e700 1 -- 172.21.15.133:0/2462385737 wait complete. 2023-07-14T18:16:03.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:02 smithi133 ceph-mon[126384]: pgmap v329: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 10 B/s, 0 objects/s recovering 2023-07-14T18:16:03.875 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- bash -c 'ceph orch device ls' 2023-07-14T18:16:04.598 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:16:05.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:04 smithi133 ceph-mon[126384]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:16:05.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:04 smithi133 ceph-mon[126384]: pgmap v330: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-14T18:16:05.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:04 smithi161 ceph-mon[128993]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:16:05.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:04 smithi161 ceph-mon[128993]: pgmap v330: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-14T18:16:05.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:04 smithi171 ceph-mon[129241]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:16:05.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:04 smithi171 ceph-mon[129241]: pgmap v330: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-14T18:16:06.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.098+0000 7fe25d9af700 1 -- 172.21.15.133:0/2046853677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe258101850 msgr2=0x7fe258108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:06.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.098+0000 7fe25d9af700 1 --2- 172.21.15.133:0/2046853677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe258101850 0x7fe258108110 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe25000b0a0 tx=0x7fe25000b3b0 comp rx=0 tx=0).stop 2023-07-14T18:16:06.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.098+0000 7fe25d9af700 1 -- 172.21.15.133:0/2046853677 shutdown_connections 2023-07-14T18:16:06.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.098+0000 7fe25d9af700 1 --2- 172.21.15.133:0/2046853677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2580ffcc0 0x7fe2581000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:06.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.098+0000 7fe25d9af700 1 --2- 172.21.15.133:0/2046853677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe258101850 0x7fe258108110 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:06.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.098+0000 7fe25d9af700 1 --2- 172.21.15.133:0/2046853677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe258100ec0 0x7fe258101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:06.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.098+0000 7fe25d9af700 1 -- 172.21.15.133:0/2046853677 >> 172.21.15.133:0/2046853677 conn(0x7fe2580fb2d0 msgr2=0x7fe2580fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:06.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.099+0000 7fe25d9af700 1 -- 172.21.15.133:0/2046853677 shutdown_connections 2023-07-14T18:16:06.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.099+0000 7fe25d9af700 1 -- 172.21.15.133:0/2046853677 wait complete. 2023-07-14T18:16:06.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.100+0000 7fe25d9af700 1 Processor -- start 2023-07-14T18:16:06.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.100+0000 7fe25d9af700 1 -- start start 2023-07-14T18:16:06.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.100+0000 7fe25d9af700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2580ffcc0 0x7fe258197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:06.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.100+0000 7fe25d9af700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe258100ec0 0x7fe258198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:06.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.100+0000 7fe25d9af700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe258101850 0x7fe25819f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:06.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.100+0000 7fe25d9af700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25819f850 con 0x7fe2580ffcc0 2023-07-14T18:16:06.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.100+0000 7fe25d9af700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25819f990 con 0x7fe258100ec0 2023-07-14T18:16:06.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.100+0000 7fe25d9af700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25819fb50 con 0x7fe258101850 2023-07-14T18:16:06.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.101+0000 7fe256ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2580ffcc0 0x7fe258197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:06.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.101+0000 7fe256ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2580ffcc0 0x7fe258197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41478/0 (socket says 172.21.15.133:41478) 2023-07-14T18:16:06.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.101+0000 7fe256ffd700 1 -- 172.21.15.133:0/3402608020 learned_addr learned my addr 172.21.15.133:0/3402608020 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:06.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.101+0000 7fe2567fc700 1 --2- 172.21.15.133:0/3402608020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe258100ec0 0x7fe258198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:06.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.101+0000 7fe2577fe700 1 --2- 172.21.15.133:0/3402608020 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe258101850 0x7fe25819f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:06.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.101+0000 7fe256ffd700 1 -- 172.21.15.133:0/3402608020 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe258101850 msgr2=0x7fe25819f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:06.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.101+0000 7fe256ffd700 1 --2- 172.21.15.133:0/3402608020 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe258101850 0x7fe25819f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:06.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.101+0000 7fe256ffd700 1 -- 172.21.15.133:0/3402608020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe258100ec0 msgr2=0x7fe258198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:06.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.101+0000 7fe256ffd700 1 --2- 172.21.15.133:0/3402608020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe258100ec0 0x7fe258198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:06.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.101+0000 7fe256ffd700 1 -- 172.21.15.133:0/3402608020 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe250009d00 con 0x7fe2580ffcc0 2023-07-14T18:16:06.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.101+0000 7fe256ffd700 1 --2- 172.21.15.133:0/3402608020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2580ffcc0 0x7fe258197d00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe24800e970 tx=0x7fe24800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:06.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.101+0000 7fe25c9ad700 1 -- 172.21.15.133:0/3402608020 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe248009790 con 0x7fe2580ffcc0 2023-07-14T18:16:06.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.101+0000 7fe25c9ad700 1 -- 172.21.15.133:0/3402608020 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe24800c7e0 con 0x7fe2580ffcc0 2023-07-14T18:16:06.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.101+0000 7fe25c9ad700 1 -- 172.21.15.133:0/3402608020 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe248003c70 con 0x7fe2580ffcc0 2023-07-14T18:16:06.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.102+0000 7fe25d9af700 1 -- 172.21.15.133:0/3402608020 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe25819fe30 con 0x7fe2580ffcc0 2023-07-14T18:16:06.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.102+0000 7fe25d9af700 1 -- 172.21.15.133:0/3402608020 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2581a01f0 con 0x7fe2580ffcc0 2023-07-14T18:16:06.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.103+0000 7fe25d9af700 1 -- 172.21.15.133:0/3402608020 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2581923a0 con 0x7fe2580ffcc0 2023-07-14T18:16:06.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.105+0000 7fe25c9ad700 1 -- 172.21.15.133:0/3402608020 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fe24800cc50 con 0x7fe2580ffcc0 2023-07-14T18:16:06.151 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.107+0000 7fe25c9ad700 1 --2- 172.21.15.133:0/3402608020 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe2400a1260 0x7fe2400a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:06.152 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.107+0000 7fe25c9ad700 1 -- 172.21.15.133:0/3402608020 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe248014070 con 0x7fe2580ffcc0 2023-07-14T18:16:06.152 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.107+0000 7fe2567fc700 1 --2- 172.21.15.133:0/3402608020 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe2400a1260 0x7fe2400a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:06.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.108+0000 7fe2567fc700 1 --2- 172.21.15.133:0/3402608020 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe2400a1260 0x7fe2400a3710 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe24c007be0 tx=0x7fe24c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:06.156 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.111+0000 7fe25c9ad700 1 -- 172.21.15.133:0/3402608020 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fe248096170 con 0x7fe2580ffcc0 2023-07-14T18:16:06.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.313+0000 7fe25d9af700 1 -- 172.21.15.133:0/3402608020 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2580614a0 con 0x7fe2400a1260 2023-07-14T18:16:06.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.315+0000 7fe25c9ad700 1 -- 172.21.15.133:0/3402608020 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+492 (secure 0 0 0) 0x7fe2580614a0 con 0x7fe2400a1260 2023-07-14T18:16:06.360 INFO:teuthology.orchestra.run.smithi133.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-07-14T18:16:06.360 INFO:teuthology.orchestra.run.smithi133.stdout:smithi133 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620401AQ400BGN 372G No 3m ago LVM detected, locked 2023-07-14T18:16:06.360 INFO:teuthology.orchestra.run.smithi133.stdout:smithi161 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233001X400BGN 372G No 2m ago LVM detected, locked 2023-07-14T18:16:06.360 INFO:teuthology.orchestra.run.smithi133.stdout:smithi171 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233001S400BGN 372G No 2m ago LVM detected, locked 2023-07-14T18:16:06.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.317+0000 7fe25d9af700 1 -- 172.21.15.133:0/3402608020 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe2400a1260 msgr2=0x7fe2400a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:06.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.317+0000 7fe25d9af700 1 --2- 172.21.15.133:0/3402608020 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe2400a1260 0x7fe2400a3710 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe24c007be0 tx=0x7fe24c0073d0 comp rx=0 tx=0).stop 2023-07-14T18:16:06.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.317+0000 7fe25d9af700 1 -- 172.21.15.133:0/3402608020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2580ffcc0 msgr2=0x7fe258197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:06.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.317+0000 7fe25d9af700 1 --2- 172.21.15.133:0/3402608020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2580ffcc0 0x7fe258197d00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe24800e970 tx=0x7fe24800ec80 comp rx=0 tx=0).stop 2023-07-14T18:16:06.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.317+0000 7fe25d9af700 1 -- 172.21.15.133:0/3402608020 shutdown_connections 2023-07-14T18:16:06.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.317+0000 7fe25d9af700 1 --2- 172.21.15.133:0/3402608020 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe2400a1260 0x7fe2400a3710 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:06.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.317+0000 7fe25d9af700 1 --2- 172.21.15.133:0/3402608020 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe258101850 0x7fe25819f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:06.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.317+0000 7fe25d9af700 1 --2- 172.21.15.133:0/3402608020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2580ffcc0 0x7fe258197d00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:06.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.317+0000 7fe25d9af700 1 --2- 172.21.15.133:0/3402608020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe258100ec0 0x7fe258198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:06.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.317+0000 7fe25d9af700 1 -- 172.21.15.133:0/3402608020 >> 172.21.15.133:0/3402608020 conn(0x7fe2580fb2d0 msgr2=0x7fe2581067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:06.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.317+0000 7fe25d9af700 1 -- 172.21.15.133:0/3402608020 shutdown_connections 2023-07-14T18:16:06.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:06.317+0000 7fe25d9af700 1 -- 172.21.15.133:0/3402608020 wait complete. 2023-07-14T18:16:06.884 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-14T18:16:06.896 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi133.front.sepia.ceph.com 2023-07-14T18:16:06.897 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- bash -c 'ceph fs dump' 2023-07-14T18:16:07.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:07 smithi133 ceph-mon[126384]: pgmap v331: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T18:16:07.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:07 smithi161 ceph-mon[128993]: pgmap v331: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T18:16:07.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:07 smithi171 ceph-mon[129241]: pgmap v331: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T18:16:07.618 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:16:08.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:08 smithi133 ceph-mon[126384]: from='client.15198 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:16:08.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:08 smithi161 ceph-mon[128993]: from='client.15198 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:16:08.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:08 smithi171 ceph-mon[129241]: from='client.15198 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T18:16:09.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.012+0000 7f58b9e07700 1 -- 172.21.15.133:0/1319886024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58b40ffcc0 msgr2=0x7f58b41000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:09.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.012+0000 7f58b9e07700 1 --2- 172.21.15.133:0/1319886024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58b40ffcc0 0x7f58b41000d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f58a40098f0 tx=0x7f58a4009c00 comp rx=0 tx=0).stop 2023-07-14T18:16:09.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.012+0000 7f58b9e07700 1 -- 172.21.15.133:0/1319886024 shutdown_connections 2023-07-14T18:16:09.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.012+0000 7f58b9e07700 1 --2- 172.21.15.133:0/1319886024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58b4101850 0x7f58b4108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:09.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.012+0000 7f58b9e07700 1 --2- 172.21.15.133:0/1319886024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58b40ffcc0 0x7f58b41000d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:09.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.012+0000 7f58b9e07700 1 --2- 172.21.15.133:0/1319886024 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58b4100ec0 0x7f58b4101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:09.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.012+0000 7f58b9e07700 1 -- 172.21.15.133:0/1319886024 >> 172.21.15.133:0/1319886024 conn(0x7f58b40fb2d0 msgr2=0x7f58b40fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:09.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.013+0000 7f58b9e07700 1 -- 172.21.15.133:0/1319886024 shutdown_connections 2023-07-14T18:16:09.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.013+0000 7f58b9e07700 1 -- 172.21.15.133:0/1319886024 wait complete. 2023-07-14T18:16:09.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.014+0000 7f58b9e07700 1 Processor -- start 2023-07-14T18:16:09.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.014+0000 7f58b9e07700 1 -- start start 2023-07-14T18:16:09.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.014+0000 7f58b9e07700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58b40ffcc0 0x7f58b4197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:09.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.014+0000 7f58b9e07700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58b4100ec0 0x7f58b41981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:09.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.015+0000 7f58b2ffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58b4100ec0 0x7f58b41981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:09.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.015+0000 7f58b2ffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58b4100ec0 0x7f58b41981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:46426/0 (socket says 172.21.15.133:46426) 2023-07-14T18:16:09.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.015+0000 7f58b9e07700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58b4101850 0x7f58b419f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:09.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.015+0000 7f58b9e07700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b419f820 con 0x7f58b4101850 2023-07-14T18:16:09.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.015+0000 7f58b9e07700 1 -- 172.21.15.133:0/243917247 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b419f960 con 0x7f58b40ffcc0 2023-07-14T18:16:09.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.015+0000 7f58b2ffd700 1 -- 172.21.15.133:0/243917247 learned_addr learned my addr 172.21.15.133:0/243917247 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:09.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.015+0000 7f58b9e07700 1 -- 172.21.15.133:0/243917247 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b419fb20 con 0x7f58b4100ec0 2023-07-14T18:16:09.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.015+0000 7f58b37fe700 1 --2- 172.21.15.133:0/243917247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58b40ffcc0 0x7f58b4197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:09.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.015+0000 7f58b3fff700 1 --2- 172.21.15.133:0/243917247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58b4101850 0x7f58b419f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:09.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.015+0000 7f58b37fe700 1 -- 172.21.15.133:0/243917247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58b4100ec0 msgr2=0x7f58b41981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:09.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.015+0000 7f58b37fe700 1 --2- 172.21.15.133:0/243917247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58b4100ec0 0x7f58b41981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:09.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.015+0000 7f58b37fe700 1 -- 172.21.15.133:0/243917247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58b4101850 msgr2=0x7f58b419f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:09.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.015+0000 7f58b37fe700 1 --2- 172.21.15.133:0/243917247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58b4101850 0x7f58b419f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:09.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.015+0000 7f58b37fe700 1 -- 172.21.15.133:0/243917247 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a80095f0 con 0x7f58b40ffcc0 2023-07-14T18:16:09.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.015+0000 7f58b37fe700 1 --2- 172.21.15.133:0/243917247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58b40ffcc0 0x7f58b4197ca0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f58a4000c00 tx=0x7f58a400b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:09.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.016+0000 7f58b0ff9700 1 -- 172.21.15.133:0/243917247 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a401c070 con 0x7f58b40ffcc0 2023-07-14T18:16:09.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.016+0000 7f58b9e07700 1 -- 172.21.15.133:0/243917247 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a4009520 con 0x7f58b40ffcc0 2023-07-14T18:16:09.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.016+0000 7f58b0ff9700 1 -- 172.21.15.133:0/243917247 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58a400f7d0 con 0x7f58b40ffcc0 2023-07-14T18:16:09.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.016+0000 7f58b0ff9700 1 -- 172.21.15.133:0/243917247 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a40039b0 con 0x7f58b40ffcc0 2023-07-14T18:16:09.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.016+0000 7f58b9e07700 1 -- 172.21.15.133:0/243917247 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b41a0030 con 0x7f58b40ffcc0 2023-07-14T18:16:09.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.017+0000 7f58b9e07700 1 -- 172.21.15.133:0/243917247 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b4192390 con 0x7f58b40ffcc0 2023-07-14T18:16:09.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.019+0000 7f58b0ff9700 1 -- 172.21.15.133:0/243917247 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f58a4020030 con 0x7f58b40ffcc0 2023-07-14T18:16:09.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.020+0000 7f58b0ff9700 1 --2- 172.21.15.133:0/243917247 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f589c0a11e0 0x7f589c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:09.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.020+0000 7f58b0ff9700 1 -- 172.21.15.133:0/243917247 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f58a40ce230 con 0x7f58b40ffcc0 2023-07-14T18:16:09.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.020+0000 7f58b2ffd700 1 --2- 172.21.15.133:0/243917247 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f589c0a11e0 0x7f589c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:09.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.021+0000 7f58b2ffd700 1 --2- 172.21.15.133:0/243917247 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f589c0a11e0 0x7f589c0a3690 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f58b41991b0 tx=0x7f58a8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:09.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.022+0000 7f58b0ff9700 1 -- 172.21.15.133:0/243917247 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f58a4097520 con 0x7f58b40ffcc0 2023-07-14T18:16:09.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:09 smithi133 ceph-mon[126384]: pgmap v332: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T18:16:09.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.260+0000 7f58b9e07700 1 -- 172.21.15.133:0/243917247 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f58b402c8e0 con 0x7f58b40ffcc0 2023-07-14T18:16:09.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.260+0000 7f58b0ff9700 1 -- 172.21.15.133:0/243917247 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) v1 ==== 76+0+2913 (secure 0 0 0) 0x7f58a409ab40 con 0x7f58b40ffcc0 2023-07-14T18:16:09.306 INFO:teuthology.orchestra.run.smithi133.stdout:e25 2023-07-14T18:16:09.306 INFO:teuthology.orchestra.run.smithi133.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-07-14T18:16:09.306 INFO:teuthology.orchestra.run.smithi133.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-14T18:16:09.306 INFO:teuthology.orchestra.run.smithi133.stdout:legacy client fscid: 1 2023-07-14T18:16:09.306 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:09.306 INFO:teuthology.orchestra.run.smithi133.stdout:Filesystem 'cephfs' (1) 2023-07-14T18:16:09.307 INFO:teuthology.orchestra.run.smithi133.stdout:fs_name cephfs 2023-07-14T18:16:09.307 INFO:teuthology.orchestra.run.smithi133.stdout:epoch 25 2023-07-14T18:16:09.307 INFO:teuthology.orchestra.run.smithi133.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-07-14T18:16:09.307 INFO:teuthology.orchestra.run.smithi133.stdout:created 2023-07-14T18:14:17.440873+0000 2023-07-14T18:16:09.307 INFO:teuthology.orchestra.run.smithi133.stdout:modified 2023-07-14T18:14:50.886364+0000 2023-07-14T18:16:09.308 INFO:teuthology.orchestra.run.smithi133.stdout:tableserver 0 2023-07-14T18:16:09.308 INFO:teuthology.orchestra.run.smithi133.stdout:root 0 2023-07-14T18:16:09.308 INFO:teuthology.orchestra.run.smithi133.stdout:session_timeout 300 2023-07-14T18:16:09.308 INFO:teuthology.orchestra.run.smithi133.stdout:session_autoclose 300 2023-07-14T18:16:09.308 INFO:teuthology.orchestra.run.smithi133.stdout:max_file_size 1099511627776 2023-07-14T18:16:09.308 INFO:teuthology.orchestra.run.smithi133.stdout:max_xattr_size 65536 2023-07-14T18:16:09.309 INFO:teuthology.orchestra.run.smithi133.stdout:required_client_features {} 2023-07-14T18:16:09.309 INFO:teuthology.orchestra.run.smithi133.stdout:last_failure 0 2023-07-14T18:16:09.309 INFO:teuthology.orchestra.run.smithi133.stdout:last_failure_osd_epoch 0 2023-07-14T18:16:09.309 INFO:teuthology.orchestra.run.smithi133.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-14T18:16:09.309 INFO:teuthology.orchestra.run.smithi133.stdout:max_mds 3 2023-07-14T18:16:09.309 INFO:teuthology.orchestra.run.smithi133.stdout:in 0,1,2 2023-07-14T18:16:09.309 INFO:teuthology.orchestra.run.smithi133.stdout:up {0=24557,1=24541,2=24517} 2023-07-14T18:16:09.310 INFO:teuthology.orchestra.run.smithi133.stdout:failed 2023-07-14T18:16:09.310 INFO:teuthology.orchestra.run.smithi133.stdout:damaged 2023-07-14T18:16:09.310 INFO:teuthology.orchestra.run.smithi133.stdout:stopped 2023-07-14T18:16:09.310 INFO:teuthology.orchestra.run.smithi133.stdout:data_pools [3,4] 2023-07-14T18:16:09.310 INFO:teuthology.orchestra.run.smithi133.stdout:metadata_pool 2 2023-07-14T18:16:09.310 INFO:teuthology.orchestra.run.smithi133.stdout:inline_data disabled 2023-07-14T18:16:09.310 INFO:teuthology.orchestra.run.smithi133.stdout:balancer 2023-07-14T18:16:09.311 INFO:teuthology.orchestra.run.smithi133.stdout:bal_rank_mask -1 2023-07-14T18:16:09.311 INFO:teuthology.orchestra.run.smithi133.stdout:standby_count_wanted 0 2023-07-14T18:16:09.311 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b{0:24557} state up:active seq 3 addr [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T18:16:09.311 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c{0:24529} state up:standby-replay seq 1 addr [v2:172.21.15.171:6836/106205332,v1:172.21.15.171:6837/106205332] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T18:16:09.311 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.i{1:24541} state up:active seq 13 addr [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T18:16:09.311 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.h{1:24521} state up:standby-replay seq 1 addr [v2:172.21.15.161:6836/3458064798,v1:172.21.15.161:6837/3458064798] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T18:16:09.312 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.f{2:24517} state up:active seq 15 addr [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T18:16:09.312 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.e{2:24494} state up:standby-replay seq 1 addr [v2:172.21.15.161:6834/3725502564,v1:172.21.15.161:6835/3725502564] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T18:16:09.312 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:09.312 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:09.312 INFO:teuthology.orchestra.run.smithi133.stdout:Standby daemons: 2023-07-14T18:16:09.313 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:09.313 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d{-1:14487} state up:standby seq 1 addr [v2:172.21.15.133:6836/3168151271,v1:172.21.15.133:6837/3168151271] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T18:16:09.313 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.g{-1:14544} state up:standby seq 1 addr [v2:172.21.15.133:6838/3093144261,v1:172.21.15.133:6839/3093144261] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T18:16:09.313 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a{-1:14556} state up:standby seq 1 addr [v2:172.21.15.133:6840/3282902068,v1:172.21.15.133:6841/3282902068] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T18:16:09.313 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.k{-1:24458} state up:standby seq 1 addr [v2:172.21.15.161:6832/4023127259,v1:172.21.15.161:6833/4023127259] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T18:16:09.314 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.l{-1:24463} state up:standby seq 1 addr [v2:172.21.15.171:6832/2722373740,v1:172.21.15.171:6833/2722373740] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T18:16:09.314 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.j{-1:24487} state up:standby seq 1 addr [v2:172.21.15.133:6834/2399005074,v1:172.21.15.133:6835/2399005074] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T18:16:09.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.263+0000 7f58b9e07700 1 -- 172.21.15.133:0/243917247 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f589c0a11e0 msgr2=0x7f589c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:09.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.263+0000 7f58b9e07700 1 --2- 172.21.15.133:0/243917247 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f589c0a11e0 0x7f589c0a3690 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f58b41991b0 tx=0x7f58a8009310 comp rx=0 tx=0).stop 2023-07-14T18:16:09.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.263+0000 7f58b9e07700 1 -- 172.21.15.133:0/243917247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58b40ffcc0 msgr2=0x7f58b4197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:09.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.263+0000 7f58b9e07700 1 --2- 172.21.15.133:0/243917247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58b40ffcc0 0x7f58b4197ca0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f58a4000c00 tx=0x7f58a400b790 comp rx=0 tx=0).stop 2023-07-14T18:16:09.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.263+0000 7f58b9e07700 1 -- 172.21.15.133:0/243917247 shutdown_connections 2023-07-14T18:16:09.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.263+0000 7f58b9e07700 1 --2- 172.21.15.133:0/243917247 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f589c0a11e0 0x7f589c0a3690 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:09.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.263+0000 7f58b9e07700 1 --2- 172.21.15.133:0/243917247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58b40ffcc0 0x7f58b4197ca0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:09.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.263+0000 7f58b9e07700 1 --2- 172.21.15.133:0/243917247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58b4101850 0x7f58b419f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:09.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.263+0000 7f58b9e07700 1 --2- 172.21.15.133:0/243917247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58b4100ec0 0x7f58b41981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:09.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.263+0000 7f58b9e07700 1 -- 172.21.15.133:0/243917247 >> 172.21.15.133:0/243917247 conn(0x7f58b40fb2d0 msgr2=0x7f58b41068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:09.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.263+0000 7f58b9e07700 1 -- 172.21.15.133:0/243917247 shutdown_connections 2023-07-14T18:16:09.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:09.263+0000 7f58b9e07700 1 -- 172.21.15.133:0/243917247 wait complete. 2023-07-14T18:16:09.316 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 25 2023-07-14T18:16:09.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:09 smithi161 ceph-mon[128993]: pgmap v332: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T18:16:09.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:09 smithi171 ceph-mon[129241]: pgmap v332: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T18:16:09.808 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- bash -c 'ceph osd dump' 2023-07-14T18:16:10.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:10 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/243917247' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-14T18:16:10.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:10 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/243917247' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-14T18:16:10.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:10 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/243917247' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-14T18:16:10.550 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:16:11.140 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:11 smithi133 ceph-mon[126384]: pgmap v333: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-14T18:16:11.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:11 smithi161 ceph-mon[128993]: pgmap v333: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-14T18:16:11.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:11 smithi171 ceph-mon[129241]: pgmap v333: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-14T18:16:12.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.060+0000 7fb265cd3700 1 -- 172.21.15.133:0/395874720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb260100ec0 msgr2=0x7fb260101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:12.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.060+0000 7fb265cd3700 1 --2- 172.21.15.133:0/395874720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb260100ec0 0x7fb260101310 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb254009990 tx=0x7fb254009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:12.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.060+0000 7fb265cd3700 1 -- 172.21.15.133:0/395874720 shutdown_connections 2023-07-14T18:16:12.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.060+0000 7fb265cd3700 1 --2- 172.21.15.133:0/395874720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb260101850 0x7fb260108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:12.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.061+0000 7fb265cd3700 1 --2- 172.21.15.133:0/395874720 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2600ffcc0 0x7fb2601000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:12.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.061+0000 7fb265cd3700 1 --2- 172.21.15.133:0/395874720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb260100ec0 0x7fb260101310 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:12.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.061+0000 7fb265cd3700 1 -- 172.21.15.133:0/395874720 >> 172.21.15.133:0/395874720 conn(0x7fb2600fb2d0 msgr2=0x7fb2600fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:12.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.061+0000 7fb265cd3700 1 -- 172.21.15.133:0/395874720 shutdown_connections 2023-07-14T18:16:12.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.061+0000 7fb265cd3700 1 -- 172.21.15.133:0/395874720 wait complete. 2023-07-14T18:16:12.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.062+0000 7fb265cd3700 1 Processor -- start 2023-07-14T18:16:12.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.062+0000 7fb265cd3700 1 -- start start 2023-07-14T18:16:12.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.062+0000 7fb265cd3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2600ffcc0 0x7fb260197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:12.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.062+0000 7fb265cd3700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb260100ec0 0x7fb260198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:12.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.062+0000 7fb265cd3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb260101850 0x7fb26019f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:12.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.062+0000 7fb265cd3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26019f750 con 0x7fb2600ffcc0 2023-07-14T18:16:12.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.062+0000 7fb265cd3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26019f890 con 0x7fb260101850 2023-07-14T18:16:12.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.062+0000 7fb265cd3700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26019fa20 con 0x7fb260100ec0 2023-07-14T18:16:12.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.062+0000 7fb25f7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2600ffcc0 0x7fb260197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:12.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.062+0000 7fb25f7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2600ffcc0 0x7fb260197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58620/0 (socket says 172.21.15.133:58620) 2023-07-14T18:16:12.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.062+0000 7fb25f7fe700 1 -- 172.21.15.133:0/1868700633 learned_addr learned my addr 172.21.15.133:0/1868700633 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:12.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.062+0000 7fb25ffff700 1 --2- 172.21.15.133:0/1868700633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb260101850 0x7fb26019f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:12.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.062+0000 7fb25effd700 1 --2- 172.21.15.133:0/1868700633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb260100ec0 0x7fb260198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:12.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.063+0000 7fb25f7fe700 1 -- 172.21.15.133:0/1868700633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb260100ec0 msgr2=0x7fb260198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:12.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.063+0000 7fb25f7fe700 1 --2- 172.21.15.133:0/1868700633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb260100ec0 0x7fb260198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:12.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.063+0000 7fb25f7fe700 1 -- 172.21.15.133:0/1868700633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb260101850 msgr2=0x7fb26019f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:12.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.063+0000 7fb25f7fe700 1 --2- 172.21.15.133:0/1868700633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb260101850 0x7fb26019f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:12.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.063+0000 7fb25f7fe700 1 -- 172.21.15.133:0/1868700633 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2540095f0 con 0x7fb2600ffcc0 2023-07-14T18:16:12.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.063+0000 7fb25f7fe700 1 --2- 172.21.15.133:0/1868700633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2600ffcc0 0x7fb260197bc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb25000e9a0 tx=0x7fb25000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:12.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.063+0000 7fb25cff9700 1 -- 172.21.15.133:0/1868700633 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb250009790 con 0x7fb2600ffcc0 2023-07-14T18:16:12.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.063+0000 7fb25cff9700 1 -- 172.21.15.133:0/1868700633 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb25000c810 con 0x7fb2600ffcc0 2023-07-14T18:16:12.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.063+0000 7fb25cff9700 1 -- 172.21.15.133:0/1868700633 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb250003c70 con 0x7fb2600ffcc0 2023-07-14T18:16:12.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.063+0000 7fb265cd3700 1 -- 172.21.15.133:0/1868700633 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb26019fd00 con 0x7fb2600ffcc0 2023-07-14T18:16:12.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.064+0000 7fb265cd3700 1 -- 172.21.15.133:0/1868700633 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2601a01b0 con 0x7fb2600ffcc0 2023-07-14T18:16:12.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.065+0000 7fb265cd3700 1 -- 172.21.15.133:0/1868700633 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb260192340 con 0x7fb2600ffcc0 2023-07-14T18:16:12.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.066+0000 7fb25cff9700 1 -- 172.21.15.133:0/1868700633 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fb25000cc80 con 0x7fb2600ffcc0 2023-07-14T18:16:12.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.068+0000 7fb25cff9700 1 --2- 172.21.15.133:0/1868700633 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb2480a1260 0x7fb2480a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:12.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.068+0000 7fb25cff9700 1 -- 172.21.15.133:0/1868700633 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb250014070 con 0x7fb2600ffcc0 2023-07-14T18:16:12.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.068+0000 7fb25effd700 1 --2- 172.21.15.133:0/1868700633 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb2480a1260 0x7fb2480a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:12.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.069+0000 7fb25effd700 1 --2- 172.21.15.133:0/1868700633 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb2480a1260 0x7fb2480a3710 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb25400b2e0 tx=0x7fb254005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:12.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.072+0000 7fb25cff9700 1 -- 172.21.15.133:0/1868700633 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fb2500961b0 con 0x7fb2600ffcc0 2023-07-14T18:16:12.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.267+0000 7fb265cd3700 1 -- 172.21.15.133:0/1868700633 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fb26002c8e0 con 0x7fb2600ffcc0 2023-07-14T18:16:12.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.268+0000 7fb25cff9700 1 -- 172.21.15.133:0/1868700633 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v92) v1 ==== 56+0+5645 (secure 0 0 0) 0x7fb2500997d0 con 0x7fb2600ffcc0 2023-07-14T18:16:12.313 INFO:teuthology.orchestra.run.smithi133.stdout:epoch 92 2023-07-14T18:16:12.313 INFO:teuthology.orchestra.run.smithi133.stdout:fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c 2023-07-14T18:16:12.313 INFO:teuthology.orchestra.run.smithi133.stdout:created 2023-07-14T18:06:49.561582+0000 2023-07-14T18:16:12.313 INFO:teuthology.orchestra.run.smithi133.stdout:modified 2023-07-14T18:15:49.951050+0000 2023-07-14T18:16:12.314 INFO:teuthology.orchestra.run.smithi133.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-07-14T18:16:12.314 INFO:teuthology.orchestra.run.smithi133.stdout:crush_version 27 2023-07-14T18:16:12.314 INFO:teuthology.orchestra.run.smithi133.stdout:full_ratio 0.95 2023-07-14T18:16:12.314 INFO:teuthology.orchestra.run.smithi133.stdout:backfillfull_ratio 0.9 2023-07-14T18:16:12.314 INFO:teuthology.orchestra.run.smithi133.stdout:nearfull_ratio 0.85 2023-07-14T18:16:12.314 INFO:teuthology.orchestra.run.smithi133.stdout:require_min_compat_client luminous 2023-07-14T18:16:12.314 INFO:teuthology.orchestra.run.smithi133.stdout:min_compat_client luminous 2023-07-14T18:16:12.315 INFO:teuthology.orchestra.run.smithi133.stdout:require_osd_release reef 2023-07-14T18:16:12.315 INFO:teuthology.orchestra.run.smithi133.stdout:stretch_mode_enabled false 2023-07-14T18:16:12.315 INFO:teuthology.orchestra.run.smithi133.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-07-14T18:16:12.315 INFO:teuthology.orchestra.run.smithi133.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2023-07-14T18:16:12.315 INFO:teuthology.orchestra.run.smithi133.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-07-14T18:16:12.315 INFO:teuthology.orchestra.run.smithi133.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-07-14T18:16:12.315 INFO:teuthology.orchestra.run.smithi133.stdout:max_osd 12 2023-07-14T18:16:12.315 INFO:teuthology.orchestra.run.smithi133.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6802/3076510801,v1:172.21.15.133:6803/3076510801] [v2:172.21.15.133:6804/3076510801,v1:172.21.15.133:6805/3076510801] exists,up 85a29de7-d712-422a-b647-d9e90091aaf1 2023-07-14T18:16:12.316 INFO:teuthology.orchestra.run.smithi133.stdout:osd.1 up in weight 1 up_from 13 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6800/1841901852,v1:172.21.15.161:6801/1841901852] [v2:172.21.15.161:6802/1841901852,v1:172.21.15.161:6803/1841901852] exists,up 80ad4e78-7917-495e-b119-99f7c7c36243 2023-07-14T18:16:12.316 INFO:teuthology.orchestra.run.smithi133.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6800/346103099,v1:172.21.15.171:6801/346103099] [v2:172.21.15.171:6802/346103099,v1:172.21.15.171:6803/346103099] exists,up fc4bfd37-1fdd-47c6-a48e-6d910daf9e78 2023-07-14T18:16:12.316 INFO:teuthology.orchestra.run.smithi133.stdout:osd.3 up in weight 1 up_from 26 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6810/2152994143,v1:172.21.15.133:6811/2152994143] [v2:172.21.15.133:6812/2152994143,v1:172.21.15.133:6813/2152994143] exists,up 4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb 2023-07-14T18:16:12.316 INFO:teuthology.orchestra.run.smithi133.stdout:osd.4 up in weight 1 up_from 33 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6808/47213291,v1:172.21.15.161:6809/47213291] [v2:172.21.15.161:6810/47213291,v1:172.21.15.161:6811/47213291] exists,up 4601a1c2-254b-4c4d-9b15-8fc952b3d2f8 2023-07-14T18:16:12.316 INFO:teuthology.orchestra.run.smithi133.stdout:osd.5 up in weight 1 up_from 38 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6808/777556751,v1:172.21.15.171:6809/777556751] [v2:172.21.15.171:6810/777556751,v1:172.21.15.171:6811/777556751] exists,up 6cca180a-9140-4e49-835b-86842dae0574 2023-07-14T18:16:12.316 INFO:teuthology.orchestra.run.smithi133.stdout:osd.6 up in weight 1 up_from 46 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6818/2884152358,v1:172.21.15.133:6819/2884152358] [v2:172.21.15.133:6820/2884152358,v1:172.21.15.133:6821/2884152358] exists,up ea0299ff-7740-4494-a3a9-1503b3fc0ecd 2023-07-14T18:16:12.316 INFO:teuthology.orchestra.run.smithi133.stdout:osd.7 up in weight 1 up_from 51 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6816/4183705749,v1:172.21.15.161:6817/4183705749] [v2:172.21.15.161:6818/4183705749,v1:172.21.15.161:6819/4183705749] exists,up b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd 2023-07-14T18:16:12.317 INFO:teuthology.orchestra.run.smithi133.stdout:osd.8 up in weight 1 up_from 59 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6816/2771494483,v1:172.21.15.171:6817/2771494483] [v2:172.21.15.171:6818/2771494483,v1:172.21.15.171:6819/2771494483] exists,up 87dffc07-bf48-48c8-b418-5c4b509686d7 2023-07-14T18:16:12.317 INFO:teuthology.orchestra.run.smithi133.stdout:osd.9 up in weight 1 up_from 67 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6826/2066164149,v1:172.21.15.133:6827/2066164149] [v2:172.21.15.133:6828/2066164149,v1:172.21.15.133:6829/2066164149] exists,up e0832870-79ce-4bce-8045-58b7a7f6dc69 2023-07-14T18:16:12.317 INFO:teuthology.orchestra.run.smithi133.stdout:osd.10 up in weight 1 up_from 72 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6824/2162333626,v1:172.21.15.161:6825/2162333626] [v2:172.21.15.161:6826/2162333626,v1:172.21.15.161:6827/2162333626] exists,up 081f0484-d134-4b2e-9c48-ef8e6a646422 2023-07-14T18:16:12.317 INFO:teuthology.orchestra.run.smithi133.stdout:osd.11 up in weight 1 up_from 79 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6824/3403894313,v1:172.21.15.171:6825/3403894313] [v2:172.21.15.171:6826/3403894313,v1:172.21.15.171:6827/3403894313] exists,up 4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa 2023-07-14T18:16:12.317 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 2.0 [7,11] 2023-07-14T18:16:12.317 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 2.8 [2,11] 2023-07-14T18:16:12.317 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 2.1b [2,4] 2023-07-14T18:16:12.317 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 2.1f [8,1] 2023-07-14T18:16:12.318 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 3.1 [4,9] 2023-07-14T18:16:12.318 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 3.11 [5,9] 2023-07-14T18:16:12.318 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 3.24 [4,9] 2023-07-14T18:16:12.318 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 4.5 [9,10] 2023-07-14T18:16:12.318 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 4.6 [5,8] 2023-07-14T18:16:12.318 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 4.16 [9,8] 2023-07-14T18:16:12.318 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 4.1b [9,8] 2023-07-14T18:16:12.318 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:0/1832569241 expires 2023-07-15T18:07:47.346704+0000 2023-07-14T18:16:12.319 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:6801/2986586981 expires 2023-07-15T18:07:47.346704+0000 2023-07-14T18:16:12.319 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:6800/2986586981 expires 2023-07-15T18:07:47.346704+0000 2023-07-14T18:16:12.319 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:0/4219815304 expires 2023-07-15T18:07:47.346704+0000 2023-07-14T18:16:12.319 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:6800/58641721 expires 2023-07-15T18:07:19.109711+0000 2023-07-14T18:16:12.319 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:6801/58641721 expires 2023-07-15T18:07:19.109711+0000 2023-07-14T18:16:12.320 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:0/432253615 expires 2023-07-15T18:07:19.109711+0000 2023-07-14T18:16:12.320 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:0/1840700480 expires 2023-07-15T18:07:19.109711+0000 2023-07-14T18:16:12.320 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:0/1807775439 expires 2023-07-15T18:07:19.109711+0000 2023-07-14T18:16:12.320 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:0/1089030734 expires 2023-07-15T18:07:47.346704+0000 2023-07-14T18:16:12.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.270+0000 7fb265cd3700 1 -- 172.21.15.133:0/1868700633 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb2480a1260 msgr2=0x7fb2480a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:12.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.270+0000 7fb265cd3700 1 --2- 172.21.15.133:0/1868700633 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb2480a1260 0x7fb2480a3710 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb25400b2e0 tx=0x7fb254005fb0 comp rx=0 tx=0).stop 2023-07-14T18:16:12.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.270+0000 7fb265cd3700 1 -- 172.21.15.133:0/1868700633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2600ffcc0 msgr2=0x7fb260197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:12.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.270+0000 7fb265cd3700 1 --2- 172.21.15.133:0/1868700633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2600ffcc0 0x7fb260197bc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb25000e9a0 tx=0x7fb25000ecb0 comp rx=0 tx=0).stop 2023-07-14T18:16:12.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.270+0000 7fb265cd3700 1 -- 172.21.15.133:0/1868700633 shutdown_connections 2023-07-14T18:16:12.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.270+0000 7fb265cd3700 1 --2- 172.21.15.133:0/1868700633 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb2480a1260 0x7fb2480a3710 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:12.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.270+0000 7fb265cd3700 1 --2- 172.21.15.133:0/1868700633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb260101850 0x7fb26019f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:12.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.270+0000 7fb265cd3700 1 --2- 172.21.15.133:0/1868700633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2600ffcc0 0x7fb260197bc0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:12.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.270+0000 7fb265cd3700 1 --2- 172.21.15.133:0/1868700633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb260100ec0 0x7fb260198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:12.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.270+0000 7fb265cd3700 1 -- 172.21.15.133:0/1868700633 >> 172.21.15.133:0/1868700633 conn(0x7fb2600fb2d0 msgr2=0x7fb260106800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:12.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.270+0000 7fb265cd3700 1 -- 172.21.15.133:0/1868700633 shutdown_connections 2023-07-14T18:16:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:12.270+0000 7fb265cd3700 1 -- 172.21.15.133:0/1868700633 wait complete. 2023-07-14T18:16:12.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:12 smithi133 ceph-mon[126384]: pgmap v334: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T18:16:12.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:12 smithi161 ceph-mon[128993]: pgmap v334: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T18:16:12.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:12 smithi171 ceph-mon[129241]: pgmap v334: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-14T18:16:12.846 INFO:teuthology.run_tasks:Running task fs.ready... 2023-07-14T18:16:12.860 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T18:16:13.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.019+0000 7f525bfff700 1 -- 172.21.15.133:0/1046891219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5254105920 msgr2=0x7f5254107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:13.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.019+0000 7f525bfff700 1 --2- 172.21.15.133:0/1046891219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5254105920 0x7f5254107d20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f524c009f90 tx=0x7f524c00b330 comp rx=0 tx=0).stop 2023-07-14T18:16:13.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.019+0000 7f525bfff700 1 -- 172.21.15.133:0/1046891219 shutdown_connections 2023-07-14T18:16:13.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.019+0000 7f525bfff700 1 --2- 172.21.15.133:0/1046891219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5254105920 0x7f5254107d20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:13.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.019+0000 7f525bfff700 1 --2- 172.21.15.133:0/1046891219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52541006e0 0x7f5254102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:13.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.019+0000 7f525bfff700 1 --2- 172.21.15.133:0/1046891219 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5254103000 0x7f52541053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:13.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.019+0000 7f525bfff700 1 -- 172.21.15.133:0/1046891219 >> 172.21.15.133:0/1046891219 conn(0x7f52540fa100 msgr2=0x7f52540fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:13.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.019+0000 7f525bfff700 1 -- 172.21.15.133:0/1046891219 shutdown_connections 2023-07-14T18:16:13.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.019+0000 7f525bfff700 1 -- 172.21.15.133:0/1046891219 wait complete. 2023-07-14T18:16:13.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.020+0000 7f525bfff700 1 Processor -- start 2023-07-14T18:16:13.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.020+0000 7f525bfff700 1 -- start start 2023-07-14T18:16:13.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.020+0000 7f525bfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52541006e0 0x7f5254100050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:13.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.020+0000 7f525bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5254103000 0x7f52540fe690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:13.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.020+0000 7f525bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5254105920 0x7f52540febd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:13.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.020+0000 7f525bfff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52540ff370 con 0x7f5254103000 2023-07-14T18:16:13.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.020+0000 7f525bfff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52540ff4e0 con 0x7f5254105920 2023-07-14T18:16:13.068 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.020+0000 7f525bfff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525419fa50 con 0x7f52541006e0 2023-07-14T18:16:13.068 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.020+0000 7f525959a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5254103000 0x7f52540fe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:13.068 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.021+0000 7f525959a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5254103000 0x7f52540fe690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58644/0 (socket says 172.21.15.133:58644) 2023-07-14T18:16:13.068 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.021+0000 7f525a59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5254105920 0x7f52540febd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:13.068 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.021+0000 7f525959a700 1 -- 172.21.15.133:0/2174921842 learned_addr learned my addr 172.21.15.133:0/2174921842 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:13.068 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.021+0000 7f5259d9b700 1 --2- 172.21.15.133:0/2174921842 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52541006e0 0x7f5254100050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:13.068 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.021+0000 7f525a59c700 1 -- 172.21.15.133:0/2174921842 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52541006e0 msgr2=0x7f5254100050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:13.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.021+0000 7f525a59c700 1 --2- 172.21.15.133:0/2174921842 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52541006e0 0x7f5254100050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:13.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.021+0000 7f525a59c700 1 -- 172.21.15.133:0/2174921842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5254103000 msgr2=0x7f52540fe690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:13.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.021+0000 7f525a59c700 1 --2- 172.21.15.133:0/2174921842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5254103000 0x7f52540fe690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:13.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.021+0000 7f525a59c700 1 -- 172.21.15.133:0/2174921842 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52500095f0 con 0x7f5254105920 2023-07-14T18:16:13.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.021+0000 7f525959a700 1 --2- 172.21.15.133:0/2174921842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5254103000 0x7f52540fe690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:16:13.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.021+0000 7f525a59c700 1 --2- 172.21.15.133:0/2174921842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5254105920 0x7f52540febd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f524c00b6c0 tx=0x7f524c004c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:13.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.021+0000 7f524affd700 1 -- 172.21.15.133:0/2174921842 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c00e040 con 0x7f5254105920 2023-07-14T18:16:13.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.022+0000 7f524affd700 1 -- 172.21.15.133:0/2174921842 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f524c00bbf0 con 0x7f5254105920 2023-07-14T18:16:13.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.022+0000 7f525bfff700 1 -- 172.21.15.133:0/2174921842 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f524c009c40 con 0x7f5254105920 2023-07-14T18:16:13.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.022+0000 7f524affd700 1 -- 172.21.15.133:0/2174921842 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c005040 con 0x7f5254105920 2023-07-14T18:16:13.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.022+0000 7f525bfff700 1 -- 172.21.15.133:0/2174921842 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f525419ffc0 con 0x7f5254105920 2023-07-14T18:16:13.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.023+0000 7f525bfff700 1 -- 172.21.15.133:0/2174921842 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52540fbc70 con 0x7f5254105920 2023-07-14T18:16:13.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.025+0000 7f524affd700 1 -- 172.21.15.133:0/2174921842 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f524c01d470 con 0x7f5254105920 2023-07-14T18:16:13.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.026+0000 7f524affd700 1 --2- 172.21.15.133:0/2174921842 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f52400a1220 0x7f52400a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:13.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.026+0000 7f524affd700 1 -- 172.21.15.133:0/2174921842 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f524c0ce600 con 0x7f5254105920 2023-07-14T18:16:13.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.026+0000 7f5259d9b700 1 --2- 172.21.15.133:0/2174921842 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f52400a1220 0x7f52400a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:13.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.027+0000 7f5259d9b700 1 --2- 172.21.15.133:0/2174921842 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f52400a1220 0x7f52400a36d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5250005fd0 tx=0x7f5250009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:13.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.028+0000 7f524affd700 1 -- 172.21.15.133:0/2174921842 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f524c0978f0 con 0x7f5254105920 2023-07-14T18:16:13.272 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.228+0000 7f525bfff700 1 -- 172.21.15.133:0/2174921842 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f525402c8e0 con 0x7f5254105920 2023-07-14T18:16:13.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.230+0000 7f524affd700 1 -- 172.21.15.133:0/2174921842 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24520 (secure 0 0 0) 0x7f524c09af10 con 0x7f5254105920 2023-07-14T18:16:13.275 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:13.275 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":92,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","created":"2023-07-14T18:06:49.561582+0000","modified":"2023-07-14T18:15:49.951050+0000","last_up_change":"2023-07-14T18:13:29.836099+0000","last_in_change":"2023-07-14T18:13:11.606710+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T18:10:03.945039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T18:14:15.451340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T18:14:16.892215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T18:14:20.493175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"85a29de7-d712-422a-b647-d9e90091aaf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6803","nonce":3076510801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6805","nonce":3076510801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6809","nonce":3076510801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6807","nonce":3076510801}]},"public_addr":"172.21.15.133:6803/3076510801","cluster_addr":"172.21.15.133:6805/3076510801","heartbeat_back_addr":"172.21.15.133:6809/3076510801","heartbeat_front_addr":"172.21.15.133:6807/3076510801","state":["exists","up"]},{"osd":1,"uuid":"80ad4e78-7917-495e-b119-99f7c7c36243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6801","nonce":1841901852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6803","nonce":1841901852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6807","nonce":1841901852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6805","nonce":1841901852}]},"public_addr":"172.21.15.161:6801/1841901852","cluster_addr":"172.21.15.161:6803/1841901852","heartbeat_back_addr":"172.21.15.161:6807/1841901852","heartbeat_front_addr":"172.21.15.161:6805/1841901852","state":["exists","up"]},{"osd":2,"uuid":"fc4bfd37-1fdd-47c6-a48e-6d910daf9e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6801","nonce":346103099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6803","nonce":346103099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6807","nonce":346103099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6805","nonce":346103099}]},"public_addr":"172.21.15.171:6801/346103099","cluster_addr":"172.21.15.171:6803/346103099","heartbeat_back_addr":"172.21.15.171:6807/346103099","heartbeat_front_addr":"172.21.15.171:6805/346103099","state":["exists","up"]},{"osd":3,"uuid":"4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6811","nonce":2152994143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6813","nonce":2152994143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6817","nonce":2152994143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6815","nonce":2152994143}]},"public_addr":"172.21.15.133:6811/2152994143","cluster_addr":"172.21.15.133:6813/2152994143","heartbeat_back_addr":"172.21.15.133:6817/2152994143","heartbeat_front_addr":"172.21.15.133:6815/2152994143","state":["exists","up"]},{"osd":4,"uuid":"4601a1c2-254b-4c4d-9b15-8fc952b3d2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6809","nonce":47213291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6811","nonce":47213291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6815","nonce":47213291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6813","nonce":47213291}]},"public_addr":"172.21.15.161:6809/47213291","cluster_addr":"172.21.15.161:6811/47213291","heartbeat_back_addr":"172.21.15.161:6815/47213291","heartbeat_front_addr":"172.21.15.161:6813/47213291","state":["exists","up"]},{"osd":5,"uuid":"6cca180a-9140-4e49-835b-86842dae0574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6809","nonce":777556751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6811","nonce":777556751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6815","nonce":777556751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6813","nonce":777556751}]},"public_addr":"172.21.15.171:6809/777556751","cluster_addr":"172.21.15.171:6811/777556751","heartbeat_back_addr":"172.21.15.171:6815/777556751","heartbeat_front_addr":"172.21.15.171:6813/777556751","state":["exists","up"]},{"osd":6,"uuid":"ea0299ff-7740-4494-a3a9-1503b3fc0ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6819","nonce":2884152358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6821","nonce":2884152358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6825","nonce":2884152358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6823","nonce":2884152358}]},"public_addr":"172.21.15.133:6819/2884152358","cluster_addr":"172.21.15.133:6821/2884152358","heartbeat_back_addr":"172.21.15.133:6825/2884152358","heartbeat_front_addr":"172.21.15.133:6823/2884152358","state":["exists","up"]},{"osd":7,"uuid":"b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6817","nonce":4183705749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6819","nonce":4183705749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6823","nonce":4183705749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6821","nonce":4183705749}]},"public_addr":"172.21.15.161:6817/4183705749","cluster_addr":"172.21.15.161:6819/4183705749","heartbeat_back_addr":"172.21.15.161:6823/4183705749","heartbeat_front_addr":"172.21.15.161:6821/4183705749","state":["exists","up"]},{"osd":8,"uuid":"87dffc07-bf48-48c8-b418-5c4b509686d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6817","nonce":2771494483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6819","nonce":2771494483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6823","nonce":2771494483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6821","nonce":2771494483}]},"public_addr":"172.21.15.171:6817/2771494483","cluster_addr":"172.21.15.171:6819/2771494483","heartbeat_back_addr":"172.21.15.171:6823/2771494483","heartbeat_front_addr":"172.21.15.171:6821/2771494483","state":["exists","up"]},{"osd":9,"uuid":"e0832870-79ce-4bce-8045-58b7a7f6dc69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6827","nonce":2066164149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6829","nonce":2066164149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6833","nonce":2066164149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6831","nonce":2066164149}]},"public_addr":"172.21.15.133:6827/2066164149","cluster_addr":"172.21.15.133:6829/2066164149","heartbeat_back_addr":"172.21.15.133:6833/2066164149","heartbeat_front_addr":"172.21.15.133:6831/2066164149","state":["exists","up"]},{"osd":10,"uuid":"081f0484-d134-4b2e-9c48-ef8e6a646422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6825","nonce":2162333626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6827","nonce":2162333626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6831","nonce":2162333626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6829","nonce":2162333626}]},"public_addr":"172.21.15.161:6825/2162333626","cluster_addr":"172.21.15.161:6827/2162333626","heartbeat_back_addr":"172.21.15.161:6831/2162333626","heartbeat_front_addr":"172.21.15.161:6829/2162333626","state":["exists","up"]},{"osd":11,"uuid":"4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6825","nonce":3403894313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6827","nonce":3403894313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6831","nonce":3403894313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6829","nonce":3403894313}]},"public_addr":"172.21.15.171:6825/3403894313","cluster_addr":"172.21.15.171:6827/3403894313","heartbeat_back_addr":"172.21.15.171:6831/3403894313","heartbeat_front_addr":"172.21.15.171:6829/3403894313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:17.816229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:39.638543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:00.543317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:26.988921+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:48.644694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:10.049860+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:35.986508+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:57.753846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:19.018245+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:44.460450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:06.279960+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:27.955006+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/1832569241":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6801/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:0/4219815304":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:6801/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/432253615":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1840700480":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1807775439":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1089030734":"2023-07-15T18:07:47.346704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T18:16:13.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.232+0000 7f525bfff700 1 -- 172.21.15.133:0/2174921842 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f52400a1220 msgr2=0x7f52400a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:13.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.232+0000 7f525bfff700 1 --2- 172.21.15.133:0/2174921842 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f52400a1220 0x7f52400a36d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5250005fd0 tx=0x7f5250009310 comp rx=0 tx=0).stop 2023-07-14T18:16:13.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.232+0000 7f525bfff700 1 -- 172.21.15.133:0/2174921842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5254105920 msgr2=0x7f52540febd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:13.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.232+0000 7f525bfff700 1 --2- 172.21.15.133:0/2174921842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5254105920 0x7f52540febd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f524c00b6c0 tx=0x7f524c004c60 comp rx=0 tx=0).stop 2023-07-14T18:16:13.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.232+0000 7f525bfff700 1 -- 172.21.15.133:0/2174921842 shutdown_connections 2023-07-14T18:16:13.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.232+0000 7f525bfff700 1 --2- 172.21.15.133:0/2174921842 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f52400a1220 0x7f52400a36d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:13.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.232+0000 7f525bfff700 1 --2- 172.21.15.133:0/2174921842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5254105920 0x7f52540febd0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:13.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.232+0000 7f525bfff700 1 --2- 172.21.15.133:0/2174921842 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52541006e0 0x7f5254100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:13.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.232+0000 7f525bfff700 1 --2- 172.21.15.133:0/2174921842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5254103000 0x7f52540fe690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:13.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.232+0000 7f525bfff700 1 -- 172.21.15.133:0/2174921842 >> 172.21.15.133:0/2174921842 conn(0x7f52540fa100 msgr2=0x7f5254101340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:13.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.232+0000 7f525bfff700 1 -- 172.21.15.133:0/2174921842 shutdown_connections 2023-07-14T18:16:13.279 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.232+0000 7f525bfff700 1 -- 172.21.15.133:0/2174921842 wait complete. 2023-07-14T18:16:13.296 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T18:10:03.945039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T18:14:15.451340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T18:14:16.892215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T18:14:20.493175+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T18:16:13.297 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T18:16:13.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:13 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1868700633' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-14T18:16:13.498 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.453+0000 7efce11ff700 1 -- 172.21.15.133:0/686330789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcdc100e10 msgr2=0x7efcdc101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:13.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.453+0000 7efce11ff700 1 --2- 172.21.15.133:0/686330789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcdc100e10 0x7efcdc101260 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009990 tx=0x7efcd0009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:13.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.454+0000 7efce11ff700 1 -- 172.21.15.133:0/686330789 shutdown_connections 2023-07-14T18:16:13.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.454+0000 7efce11ff700 1 --2- 172.21.15.133:0/686330789 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcdc1017a0 0x7efcdc108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:13.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.454+0000 7efce11ff700 1 --2- 172.21.15.133:0/686330789 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efcdc0ffc20 0x7efcdc100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:13.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.454+0000 7efce11ff700 1 --2- 172.21.15.133:0/686330789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcdc100e10 0x7efcdc101260 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:13.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.454+0000 7efce11ff700 1 -- 172.21.15.133:0/686330789 >> 172.21.15.133:0/686330789 conn(0x7efcdc0fb260 msgr2=0x7efcdc0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:13.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.454+0000 7efce11ff700 1 -- 172.21.15.133:0/686330789 shutdown_connections 2023-07-14T18:16:13.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.454+0000 7efce11ff700 1 -- 172.21.15.133:0/686330789 wait complete. 2023-07-14T18:16:13.501 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.455+0000 7efce11ff700 1 Processor -- start 2023-07-14T18:16:13.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.455+0000 7efce11ff700 1 -- start start 2023-07-14T18:16:13.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.455+0000 7efce11ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcdc0ffc20 0x7efcdc0743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:13.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.455+0000 7efce11ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efcdc100e10 0x7efcdc072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:13.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.455+0000 7efce11ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcdc1017a0 0x7efcdc072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:13.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.455+0000 7efce11ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc073710 con 0x7efcdc0ffc20 2023-07-14T18:16:13.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.455+0000 7efce11ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc073880 con 0x7efcdc1017a0 2023-07-14T18:16:13.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.455+0000 7efce11ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc19f9e0 con 0x7efcdc100e10 2023-07-14T18:16:13.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.455+0000 7efcdad9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcdc0ffc20 0x7efcdc0743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:13.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.455+0000 7efcdad9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcdc0ffc20 0x7efcdc0743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58666/0 (socket says 172.21.15.133:58666) 2023-07-14T18:16:13.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.455+0000 7efcdad9d700 1 -- 172.21.15.133:0/904271941 learned_addr learned my addr 172.21.15.133:0/904271941 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:13.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.455+0000 7efcdb59e700 1 --2- 172.21.15.133:0/904271941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcdc1017a0 0x7efcdc072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:13.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.455+0000 7efcda59c700 1 --2- 172.21.15.133:0/904271941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efcdc100e10 0x7efcdc072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:13.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.456+0000 7efcdad9d700 1 -- 172.21.15.133:0/904271941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efcdc100e10 msgr2=0x7efcdc072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:13.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.456+0000 7efcdad9d700 1 --2- 172.21.15.133:0/904271941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efcdc100e10 0x7efcdc072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:13.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.456+0000 7efcdad9d700 1 -- 172.21.15.133:0/904271941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcdc1017a0 msgr2=0x7efcdc072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:13.504 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.456+0000 7efcdad9d700 1 --2- 172.21.15.133:0/904271941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcdc1017a0 0x7efcdc072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:13.504 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.456+0000 7efcdad9d700 1 -- 172.21.15.133:0/904271941 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd4009c40 con 0x7efcdc0ffc20 2023-07-14T18:16:13.504 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.456+0000 7efcdad9d700 1 --2- 172.21.15.133:0/904271941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcdc0ffc20 0x7efcdc0743f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efccc00e970 tx=0x7efccc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:13.504 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.456+0000 7efcc3fff700 1 -- 172.21.15.133:0/904271941 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc009790 con 0x7efcdc0ffc20 2023-07-14T18:16:13.504 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.456+0000 7efce11ff700 1 -- 172.21.15.133:0/904271941 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd00095f0 con 0x7efcdc0ffc20 2023-07-14T18:16:13.504 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.456+0000 7efcc3fff700 1 -- 172.21.15.133:0/904271941 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efccc00eeb0 con 0x7efcdc0ffc20 2023-07-14T18:16:13.504 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.456+0000 7efcc3fff700 1 -- 172.21.15.133:0/904271941 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc003d20 con 0x7efcdc0ffc20 2023-07-14T18:16:13.504 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.456+0000 7efce11ff700 1 -- 172.21.15.133:0/904271941 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcdc1a0090 con 0x7efcdc0ffc20 2023-07-14T18:16:13.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.457+0000 7efce11ff700 1 -- 172.21.15.133:0/904271941 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcdc1071c0 con 0x7efcdc0ffc20 2023-07-14T18:16:13.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.459+0000 7efcc3fff700 1 -- 172.21.15.133:0/904271941 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7efccc003e80 con 0x7efcdc0ffc20 2023-07-14T18:16:13.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.460+0000 7efcc3fff700 1 --2- 172.21.15.133:0/904271941 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efcc40a1260 0x7efcc40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:13.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.461+0000 7efcc3fff700 1 -- 172.21.15.133:0/904271941 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7efccc014070 con 0x7efcdc0ffc20 2023-07-14T18:16:13.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.461+0000 7efcda59c700 1 --2- 172.21.15.133:0/904271941 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efcc40a1260 0x7efcc40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:13.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.461+0000 7efcda59c700 1 --2- 172.21.15.133:0/904271941 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efcc40a1260 0x7efcc40a3710 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efcd000b2e0 tx=0x7efcd0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:13.509 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.464+0000 7efcc3fff700 1 -- 172.21.15.133:0/904271941 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7efccc096290 con 0x7efcdc0ffc20 2023-07-14T18:16:13.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:13 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1868700633' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-14T18:16:13.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:13 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1868700633' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-14T18:16:13.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.663+0000 7efce11ff700 1 -- 172.21.15.133:0/904271941 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efcdc02cd70 con 0x7efcdc0ffc20 2023-07-14T18:16:13.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.664+0000 7efcc3fff700 1 -- 172.21.15.133:0/904271941 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7efccc0998b0 con 0x7efcdc0ffc20 2023-07-14T18:16:13.709 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-07-14T18:16:13.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.666+0000 7efce11ff700 1 -- 172.21.15.133:0/904271941 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efcc40a1260 msgr2=0x7efcc40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:13.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.666+0000 7efce11ff700 1 --2- 172.21.15.133:0/904271941 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efcc40a1260 0x7efcc40a3710 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efcd000b2e0 tx=0x7efcd0005fb0 comp rx=0 tx=0).stop 2023-07-14T18:16:13.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.666+0000 7efce11ff700 1 -- 172.21.15.133:0/904271941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcdc0ffc20 msgr2=0x7efcdc0743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:13.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.666+0000 7efce11ff700 1 --2- 172.21.15.133:0/904271941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcdc0ffc20 0x7efcdc0743f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efccc00e970 tx=0x7efccc00ed30 comp rx=0 tx=0).stop 2023-07-14T18:16:13.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.666+0000 7efce11ff700 1 -- 172.21.15.133:0/904271941 shutdown_connections 2023-07-14T18:16:13.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.666+0000 7efce11ff700 1 --2- 172.21.15.133:0/904271941 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efcc40a1260 0x7efcc40a3710 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:13.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.666+0000 7efce11ff700 1 --2- 172.21.15.133:0/904271941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcdc1017a0 0x7efcdc072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:13.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.666+0000 7efce11ff700 1 --2- 172.21.15.133:0/904271941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcdc0ffc20 0x7efcdc0743f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:13.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.666+0000 7efce11ff700 1 --2- 172.21.15.133:0/904271941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efcdc100e10 0x7efcdc072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:13.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.666+0000 7efce11ff700 1 -- 172.21.15.133:0/904271941 >> 172.21.15.133:0/904271941 conn(0x7efcdc0fb260 msgr2=0x7efcdc104300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:13.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.666+0000 7efce11ff700 1 -- 172.21.15.133:0/904271941 shutdown_connections 2023-07-14T18:16:13.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.667+0000 7efce11ff700 1 -- 172.21.15.133:0/904271941 wait complete. 2023-07-14T18:16:13.730 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T18:16:13.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.884+0000 7fe514fff700 1 -- 172.21.15.133:0/3482456514 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5100ffc20 msgr2=0x7fe510100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:13.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.884+0000 7fe514fff700 1 --2- 172.21.15.133:0/3482456514 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5100ffc20 0x7fe510100030 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe5000098f0 tx=0x7fe500009c00 comp rx=0 tx=0).stop 2023-07-14T18:16:13.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.884+0000 7fe514fff700 1 -- 172.21.15.133:0/3482456514 shutdown_connections 2023-07-14T18:16:13.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.884+0000 7fe514fff700 1 --2- 172.21.15.133:0/3482456514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5101017a0 0x7fe510108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:13.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.884+0000 7fe514fff700 1 --2- 172.21.15.133:0/3482456514 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5100ffc20 0x7fe510100030 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:13.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.884+0000 7fe514fff700 1 --2- 172.21.15.133:0/3482456514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe510100e10 0x7fe510101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:13.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.884+0000 7fe514fff700 1 -- 172.21.15.133:0/3482456514 >> 172.21.15.133:0/3482456514 conn(0x7fe5100fb260 msgr2=0x7fe5100fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:13.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.884+0000 7fe514fff700 1 -- 172.21.15.133:0/3482456514 shutdown_connections 2023-07-14T18:16:13.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.884+0000 7fe514fff700 1 -- 172.21.15.133:0/3482456514 wait complete. 2023-07-14T18:16:13.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.885+0000 7fe514fff700 1 Processor -- start 2023-07-14T18:16:13.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.885+0000 7fe514fff700 1 -- start start 2023-07-14T18:16:13.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.886+0000 7fe514fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5100ffc20 0x7fe510197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:13.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.886+0000 7fe514fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe510100e10 0x7fe5101980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:13.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.886+0000 7fe514fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5101017a0 0x7fe51019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:13.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.886+0000 7fe514fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51019f700 con 0x7fe510100e10 2023-07-14T18:16:13.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.886+0000 7fe514fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51019f840 con 0x7fe5100ffc20 2023-07-14T18:16:13.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.886+0000 7fe514fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51019f9d0 con 0x7fe5101017a0 2023-07-14T18:16:13.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.886+0000 7fe50dd9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe510100e10 0x7fe5101980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:13.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.886+0000 7fe50dd9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe510100e10 0x7fe5101980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58674/0 (socket says 172.21.15.133:58674) 2023-07-14T18:16:13.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.886+0000 7fe50dd9b700 1 -- 172.21.15.133:0/2103468184 learned_addr learned my addr 172.21.15.133:0/2103468184 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:13.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.886+0000 7fe50e59c700 1 --2- 172.21.15.133:0/2103468184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5100ffc20 0x7fe510197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:13.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.886+0000 7fe50ed9d700 1 --2- 172.21.15.133:0/2103468184 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5101017a0 0x7fe51019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:13.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.886+0000 7fe50dd9b700 1 -- 172.21.15.133:0/2103468184 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5101017a0 msgr2=0x7fe51019f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:13.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.887+0000 7fe50dd9b700 1 --2- 172.21.15.133:0/2103468184 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5101017a0 0x7fe51019f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:13.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.887+0000 7fe50dd9b700 1 -- 172.21.15.133:0/2103468184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5100ffc20 msgr2=0x7fe510197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:13.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.887+0000 7fe50dd9b700 1 --2- 172.21.15.133:0/2103468184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5100ffc20 0x7fe510197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:13.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.887+0000 7fe50dd9b700 1 -- 172.21.15.133:0/2103468184 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5040095f0 con 0x7fe510100e10 2023-07-14T18:16:13.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.887+0000 7fe50dd9b700 1 --2- 172.21.15.133:0/2103468184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe510100e10 0x7fe5101980c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe50400ec10 tx=0x7fe50400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:13.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.887+0000 7fe4ff7fe700 1 -- 172.21.15.133:0/2103468184 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50400cc90 con 0x7fe510100e10 2023-07-14T18:16:13.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.887+0000 7fe514fff700 1 -- 172.21.15.133:0/2103468184 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe500009520 con 0x7fe510100e10 2023-07-14T18:16:13.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.887+0000 7fe4ff7fe700 1 -- 172.21.15.133:0/2103468184 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5040044e0 con 0x7fe510100e10 2023-07-14T18:16:13.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.887+0000 7fe4ff7fe700 1 -- 172.21.15.133:0/2103468184 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe504010430 con 0x7fe510100e10 2023-07-14T18:16:13.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.887+0000 7fe514fff700 1 -- 172.21.15.133:0/2103468184 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5101a0080 con 0x7fe510100e10 2023-07-14T18:16:13.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.888+0000 7fe514fff700 1 -- 172.21.15.133:0/2103468184 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5101922a0 con 0x7fe510100e10 2023-07-14T18:16:13.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.890+0000 7fe4ff7fe700 1 -- 172.21.15.133:0/2103468184 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fe504003f90 con 0x7fe510100e10 2023-07-14T18:16:13.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.891+0000 7fe4ff7fe700 1 --2- 172.21.15.133:0/2103468184 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe4f80a11e0 0x7fe4f80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:13.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.892+0000 7fe4ff7fe700 1 -- 172.21.15.133:0/2103468184 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe504014070 con 0x7fe510100e10 2023-07-14T18:16:13.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.892+0000 7fe50e59c700 1 --2- 172.21.15.133:0/2103468184 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe4f80a11e0 0x7fe4f80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:13.938 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.892+0000 7fe50e59c700 1 --2- 172.21.15.133:0/2103468184 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe4f80a11e0 0x7fe4f80a3690 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe500009fd0 tx=0x7fe500019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:13.938 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:13.893+0000 7fe4ff7fe700 1 -- 172.21.15.133:0/2103468184 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fe504096770 con 0x7fe510100e10 2023-07-14T18:16:14.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.097+0000 7fe514fff700 1 -- 172.21.15.133:0/2103468184 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe51002ca70 con 0x7fe510100e10 2023-07-14T18:16:14.143 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.098+0000 7fe4ff7fe700 1 -- 172.21.15.133:0/2103468184 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe504099d90 con 0x7fe510100e10 2023-07-14T18:16:14.143 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-07-14T18:16:14.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.100+0000 7fe514fff700 1 -- 172.21.15.133:0/2103468184 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe4f80a11e0 msgr2=0x7fe4f80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:14.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.100+0000 7fe514fff700 1 --2- 172.21.15.133:0/2103468184 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe4f80a11e0 0x7fe4f80a3690 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe500009fd0 tx=0x7fe500019040 comp rx=0 tx=0).stop 2023-07-14T18:16:14.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.100+0000 7fe514fff700 1 -- 172.21.15.133:0/2103468184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe510100e10 msgr2=0x7fe5101980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:14.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.100+0000 7fe514fff700 1 --2- 172.21.15.133:0/2103468184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe510100e10 0x7fe5101980c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe50400ec10 tx=0x7fe50400ef20 comp rx=0 tx=0).stop 2023-07-14T18:16:14.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.101+0000 7fe514fff700 1 -- 172.21.15.133:0/2103468184 shutdown_connections 2023-07-14T18:16:14.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.101+0000 7fe514fff700 1 --2- 172.21.15.133:0/2103468184 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe4f80a11e0 0x7fe4f80a3690 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:14.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.101+0000 7fe514fff700 1 --2- 172.21.15.133:0/2103468184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5100ffc20 0x7fe510197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:14.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.101+0000 7fe514fff700 1 --2- 172.21.15.133:0/2103468184 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5101017a0 0x7fe51019f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:14.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.101+0000 7fe514fff700 1 --2- 172.21.15.133:0/2103468184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe510100e10 0x7fe5101980c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:14.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.101+0000 7fe514fff700 1 -- 172.21.15.133:0/2103468184 >> 172.21.15.133:0/2103468184 conn(0x7fe5100fb260 msgr2=0x7fe510106770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:14.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.101+0000 7fe514fff700 1 -- 172.21.15.133:0/2103468184 shutdown_connections 2023-07-14T18:16:14.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.101+0000 7fe514fff700 1 -- 172.21.15.133:0/2103468184 wait complete. 2023-07-14T18:16:14.164 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T18:16:14.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.319+0000 7fd8ff7ff700 1 -- 172.21.15.133:0/3759812166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f80ffc10 msgr2=0x7fd8f8100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:14.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.319+0000 7fd8ff7ff700 1 --2- 172.21.15.133:0/3759812166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f80ffc10 0x7fd8f8100020 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4009990 tx=0x7fd8f4009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:14.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.319+0000 7fd8ff7ff700 1 -- 172.21.15.133:0/3759812166 shutdown_connections 2023-07-14T18:16:14.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.319+0000 7fd8ff7ff700 1 --2- 172.21.15.133:0/3759812166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8f8101790 0x7fd8f8108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:14.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.319+0000 7fd8ff7ff700 1 --2- 172.21.15.133:0/3759812166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f80ffc10 0x7fd8f8100020 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:14.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.319+0000 7fd8ff7ff700 1 --2- 172.21.15.133:0/3759812166 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd8f8100e00 0x7fd8f8101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:14.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.319+0000 7fd8ff7ff700 1 -- 172.21.15.133:0/3759812166 >> 172.21.15.133:0/3759812166 conn(0x7fd8f80fb210 msgr2=0x7fd8f80fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:14.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.319+0000 7fd8ff7ff700 1 -- 172.21.15.133:0/3759812166 shutdown_connections 2023-07-14T18:16:14.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.319+0000 7fd8ff7ff700 1 -- 172.21.15.133:0/3759812166 wait complete. 2023-07-14T18:16:14.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.320+0000 7fd8ff7ff700 1 Processor -- start 2023-07-14T18:16:14.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.321+0000 7fd8ff7ff700 1 -- start start 2023-07-14T18:16:14.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.321+0000 7fd8ff7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f80ffc10 0x7fd8f8197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:14.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.321+0000 7fd8ff7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8f8100e00 0x7fd8f81980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:14.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.321+0000 7fd8ff7ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd8f8101790 0x7fd8f819f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:14.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.321+0000 7fd8ff7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f819f700 con 0x7fd8f80ffc10 2023-07-14T18:16:14.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.321+0000 7fd8ff7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f819f840 con 0x7fd8f8100e00 2023-07-14T18:16:14.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.321+0000 7fd8ff7ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f819f9d0 con 0x7fd8f8101790 2023-07-14T18:16:14.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.321+0000 7fd8fd59b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f80ffc10 0x7fd8f8197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:14.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.321+0000 7fd8fd59b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f80ffc10 0x7fd8f8197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58680/0 (socket says 172.21.15.133:58680) 2023-07-14T18:16:14.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.321+0000 7fd8fd59b700 1 -- 172.21.15.133:0/3327748867 learned_addr learned my addr 172.21.15.133:0/3327748867 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:14.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.321+0000 7fd8fcd9a700 1 --2- 172.21.15.133:0/3327748867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8f8100e00 0x7fd8f81980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:14.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.321+0000 7fd8fdd9c700 1 --2- 172.21.15.133:0/3327748867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd8f8101790 0x7fd8f819f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:14.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.322+0000 7fd8fd59b700 1 -- 172.21.15.133:0/3327748867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd8f8101790 msgr2=0x7fd8f819f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:14.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.322+0000 7fd8fd59b700 1 --2- 172.21.15.133:0/3327748867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd8f8101790 0x7fd8f819f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:14.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.322+0000 7fd8fd59b700 1 -- 172.21.15.133:0/3327748867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8f8100e00 msgr2=0x7fd8f81980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:14.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.322+0000 7fd8fd59b700 1 --2- 172.21.15.133:0/3327748867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8f8100e00 0x7fd8f81980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:14.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.322+0000 7fd8fd59b700 1 -- 172.21.15.133:0/3327748867 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f40095f0 con 0x7fd8f80ffc10 2023-07-14T18:16:14.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.322+0000 7fd8fd59b700 1 --2- 172.21.15.133:0/3327748867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f80ffc10 0x7fd8f8197b80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4000c00 tx=0x7fd8f40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:14.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.322+0000 7fd8ee7fc700 1 -- 172.21.15.133:0/3327748867 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f401d070 con 0x7fd8f80ffc10 2023-07-14T18:16:14.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.322+0000 7fd8ee7fc700 1 -- 172.21.15.133:0/3327748867 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8f4021410 con 0x7fd8f80ffc10 2023-07-14T18:16:14.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.322+0000 7fd8ff7ff700 1 -- 172.21.15.133:0/3327748867 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f819fc50 con 0x7fd8f80ffc10 2023-07-14T18:16:14.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.322+0000 7fd8ee7fc700 1 -- 172.21.15.133:0/3327748867 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4017650 con 0x7fd8f80ffc10 2023-07-14T18:16:14.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.322+0000 7fd8ff7ff700 1 -- 172.21.15.133:0/3327748867 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f81a0100 con 0x7fd8f80ffc10 2023-07-14T18:16:14.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.323+0000 7fd8ff7ff700 1 -- 172.21.15.133:0/3327748867 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f81922a0 con 0x7fd8f80ffc10 2023-07-14T18:16:14.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.325+0000 7fd8ee7fc700 1 -- 172.21.15.133:0/3327748867 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fd8f4021580 con 0x7fd8f80ffc10 2023-07-14T18:16:14.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.326+0000 7fd8ee7fc700 1 --2- 172.21.15.133:0/3327748867 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd8e40a11e0 0x7fd8e40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:14.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.327+0000 7fd8ee7fc700 1 -- 172.21.15.133:0/3327748867 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd8f40ce850 con 0x7fd8f80ffc10 2023-07-14T18:16:14.372 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.327+0000 7fd8fcd9a700 1 --2- 172.21.15.133:0/3327748867 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd8e40a11e0 0x7fd8e40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:14.372 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.327+0000 7fd8fcd9a700 1 --2- 172.21.15.133:0/3327748867 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd8e40a11e0 0x7fd8e40a3690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8009c60 tx=0x7fd8e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:14.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.328+0000 7fd8ee7fc700 1 -- 172.21.15.133:0/3327748867 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fd8f4097b40 con 0x7fd8f80ffc10 2023-07-14T18:16:14.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:14 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2174921842' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:16:14.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:14 smithi133 ceph-mon[126384]: pgmap v335: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-14T18:16:14.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:14 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/904271941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:16:14.572 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.527+0000 7fd8ff7ff700 1 -- 172.21.15.133:0/3327748867 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd8f802cb50 con 0x7fd8f80ffc10 2023-07-14T18:16:14.572 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.527+0000 7fd8ee7fc700 1 -- 172.21.15.133:0/3327748867 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd8f409b160 con 0x7fd8f80ffc10 2023-07-14T18:16:14.573 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2023-07-14T18:16:14.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.529+0000 7fd8ff7ff700 1 -- 172.21.15.133:0/3327748867 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd8e40a11e0 msgr2=0x7fd8e40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:14.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.529+0000 7fd8ff7ff700 1 --2- 172.21.15.133:0/3327748867 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd8e40a11e0 0x7fd8e40a3690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8009c60 tx=0x7fd8e8009260 comp rx=0 tx=0).stop 2023-07-14T18:16:14.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.530+0000 7fd8ff7ff700 1 -- 172.21.15.133:0/3327748867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f80ffc10 msgr2=0x7fd8f8197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:14.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.530+0000 7fd8ff7ff700 1 --2- 172.21.15.133:0/3327748867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f80ffc10 0x7fd8f8197b80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4000c00 tx=0x7fd8f40057c0 comp rx=0 tx=0).stop 2023-07-14T18:16:14.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.530+0000 7fd8ff7ff700 1 -- 172.21.15.133:0/3327748867 shutdown_connections 2023-07-14T18:16:14.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.530+0000 7fd8ff7ff700 1 --2- 172.21.15.133:0/3327748867 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd8e40a11e0 0x7fd8e40a3690 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:14.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.530+0000 7fd8ff7ff700 1 --2- 172.21.15.133:0/3327748867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd8f8101790 0x7fd8f819f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:14.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.530+0000 7fd8ff7ff700 1 --2- 172.21.15.133:0/3327748867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f80ffc10 0x7fd8f8197b80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:14.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.530+0000 7fd8ff7ff700 1 --2- 172.21.15.133:0/3327748867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8f8100e00 0x7fd8f81980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:14.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.530+0000 7fd8ff7ff700 1 -- 172.21.15.133:0/3327748867 >> 172.21.15.133:0/3327748867 conn(0x7fd8f80fb210 msgr2=0x7fd8f8106730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:14.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.530+0000 7fd8ff7ff700 1 -- 172.21.15.133:0/3327748867 shutdown_connections 2023-07-14T18:16:14.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.530+0000 7fd8ff7ff700 1 -- 172.21.15.133:0/3327748867 wait complete. 2023-07-14T18:16:14.594 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T18:16:14.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:14 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2174921842' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:16:14.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:14 smithi161 ceph-mon[128993]: pgmap v335: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-14T18:16:14.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:14 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/904271941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:16:14.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:14 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2174921842' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:16:14.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:14 smithi171 ceph-mon[129241]: pgmap v335: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-14T18:16:14.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:14 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/904271941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:16:14.793 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.747+0000 7f586b7ff700 1 -- 172.21.15.133:0/779599790 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58640ffc20 msgr2=0x7f5864100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:14.793 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.747+0000 7f586b7ff700 1 --2- 172.21.15.133:0/779599790 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58640ffc20 0x7f5864100030 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5860009990 tx=0x7f5860009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:14.793 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.748+0000 7f586b7ff700 1 -- 172.21.15.133:0/779599790 shutdown_connections 2023-07-14T18:16:14.793 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.748+0000 7f586b7ff700 1 --2- 172.21.15.133:0/779599790 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58641017a0 0x7f5864108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:14.793 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.748+0000 7f586b7ff700 1 --2- 172.21.15.133:0/779599790 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58640ffc20 0x7f5864100030 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:14.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.748+0000 7f586b7ff700 1 --2- 172.21.15.133:0/779599790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5864100e10 0x7f5864101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:14.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.748+0000 7f586b7ff700 1 -- 172.21.15.133:0/779599790 >> 172.21.15.133:0/779599790 conn(0x7f58640fb260 msgr2=0x7f58640fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:14.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.748+0000 7f586b7ff700 1 -- 172.21.15.133:0/779599790 shutdown_connections 2023-07-14T18:16:14.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.748+0000 7f586b7ff700 1 -- 172.21.15.133:0/779599790 wait complete. 2023-07-14T18:16:14.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.749+0000 7f586b7ff700 1 Processor -- start 2023-07-14T18:16:14.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.749+0000 7f586b7ff700 1 -- start start 2023-07-14T18:16:14.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.749+0000 7f586b7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58640ffc20 0x7f58640743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:14.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.749+0000 7f586b7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5864100e10 0x7f5864072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:14.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.749+0000 7f586b7ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58641017a0 0x7f5864072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:14.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.749+0000 7f586b7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5864073710 con 0x7f5864100e10 2023-07-14T18:16:14.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.749+0000 7f586b7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5864073880 con 0x7f58640ffc20 2023-07-14T18:16:14.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.749+0000 7f586b7ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586419f9d0 con 0x7f58641017a0 2023-07-14T18:16:14.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.749+0000 7f586959b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58640ffc20 0x7f58640743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:14.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.750+0000 7f5868d9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5864100e10 0x7f5864072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:14.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.750+0000 7f586959b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58640ffc20 0x7f58640743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:37178/0 (socket says 172.21.15.133:37178) 2023-07-14T18:16:14.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.750+0000 7f586959b700 1 -- 172.21.15.133:0/513923497 learned_addr learned my addr 172.21.15.133:0/513923497 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:14.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.750+0000 7f5869d9c700 1 --2- 172.21.15.133:0/513923497 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58641017a0 0x7f5864072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:14.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.750+0000 7f586959b700 1 -- 172.21.15.133:0/513923497 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58641017a0 msgr2=0x7f5864072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:14.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.750+0000 7f586959b700 1 --2- 172.21.15.133:0/513923497 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58641017a0 0x7f5864072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:14.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.750+0000 7f586959b700 1 -- 172.21.15.133:0/513923497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5864100e10 msgr2=0x7f5864072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:14.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.750+0000 7f586959b700 1 --2- 172.21.15.133:0/513923497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5864100e10 0x7f5864072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:14.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.750+0000 7f586959b700 1 -- 172.21.15.133:0/513923497 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58600095f0 con 0x7f58640ffc20 2023-07-14T18:16:14.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.750+0000 7f5869d9c700 1 --2- 172.21.15.133:0/513923497 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58641017a0 0x7f5864072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:16:14.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.750+0000 7f5868d9a700 1 --2- 172.21.15.133:0/513923497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5864100e10 0x7f5864072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:16:14.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.750+0000 7f586959b700 1 --2- 172.21.15.133:0/513923497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58640ffc20 0x7f58640743f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f586000b2e0 tx=0x7f5860004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:14.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.750+0000 7f585a7fc700 1 -- 172.21.15.133:0/513923497 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586001d070 con 0x7f58640ffc20 2023-07-14T18:16:14.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.750+0000 7f586b7ff700 1 -- 172.21.15.133:0/513923497 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f586419fc50 con 0x7f58640ffc20 2023-07-14T18:16:14.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.750+0000 7f585a7fc700 1 -- 172.21.15.133:0/513923497 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5860005030 con 0x7f58640ffc20 2023-07-14T18:16:14.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.751+0000 7f585a7fc700 1 -- 172.21.15.133:0/513923497 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5860004cc0 con 0x7f58640ffc20 2023-07-14T18:16:14.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.751+0000 7f586b7ff700 1 -- 172.21.15.133:0/513923497 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58641a00d0 con 0x7f58640ffc20 2023-07-14T18:16:14.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.752+0000 7f586b7ff700 1 -- 172.21.15.133:0/513923497 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f586410b3e0 con 0x7f58640ffc20 2023-07-14T18:16:14.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.754+0000 7f585a7fc700 1 -- 172.21.15.133:0/513923497 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f58600051a0 con 0x7f58640ffc20 2023-07-14T18:16:14.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.755+0000 7f585a7fc700 1 --2- 172.21.15.133:0/513923497 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f58500a11e0 0x7f58500a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:14.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.755+0000 7f585a7fc700 1 -- 172.21.15.133:0/513923497 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f58600ce4c0 con 0x7f58640ffc20 2023-07-14T18:16:14.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.755+0000 7f5868d9a700 1 --2- 172.21.15.133:0/513923497 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f58500a11e0 0x7f58500a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:14.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.756+0000 7f5868d9a700 1 --2- 172.21.15.133:0/513923497 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f58500a11e0 0x7f58500a3690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5854011490 tx=0x7f5854009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:14.802 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.757+0000 7f585a7fc700 1 -- 172.21.15.133:0/513923497 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f58600977b0 con 0x7f58640ffc20 2023-07-14T18:16:15.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.957+0000 7f586b7ff700 1 -- 172.21.15.133:0/513923497 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5864106b40 con 0x7f58640ffc20 2023-07-14T18:16:15.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.958+0000 7f585a7fc700 1 -- 172.21.15.133:0/513923497 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5860005c00 con 0x7f58640ffc20 2023-07-14T18:16:15.003 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-07-14T18:16:15.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.960+0000 7f586b7ff700 1 -- 172.21.15.133:0/513923497 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f58500a11e0 msgr2=0x7f58500a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:15.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.960+0000 7f586b7ff700 1 --2- 172.21.15.133:0/513923497 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f58500a11e0 0x7f58500a3690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5854011490 tx=0x7f5854009260 comp rx=0 tx=0).stop 2023-07-14T18:16:15.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.960+0000 7f586b7ff700 1 -- 172.21.15.133:0/513923497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58640ffc20 msgr2=0x7f58640743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:15.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.960+0000 7f586b7ff700 1 --2- 172.21.15.133:0/513923497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58640ffc20 0x7f58640743f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f586000b2e0 tx=0x7f5860004730 comp rx=0 tx=0).stop 2023-07-14T18:16:15.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.960+0000 7f586b7ff700 1 -- 172.21.15.133:0/513923497 shutdown_connections 2023-07-14T18:16:15.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.960+0000 7f586b7ff700 1 --2- 172.21.15.133:0/513923497 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f58500a11e0 0x7f58500a3690 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:15.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.960+0000 7f586b7ff700 1 --2- 172.21.15.133:0/513923497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58640ffc20 0x7f58640743f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:15.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.960+0000 7f586b7ff700 1 --2- 172.21.15.133:0/513923497 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58641017a0 0x7f5864072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:15.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.960+0000 7f586b7ff700 1 --2- 172.21.15.133:0/513923497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5864100e10 0x7f5864072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:15.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.960+0000 7f586b7ff700 1 -- 172.21.15.133:0/513923497 >> 172.21.15.133:0/513923497 conn(0x7f58640fb260 msgr2=0x7f58640fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:15.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.960+0000 7f586b7ff700 1 -- 172.21.15.133:0/513923497 shutdown_connections 2023-07-14T18:16:15.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:14.960+0000 7f586b7ff700 1 -- 172.21.15.133:0/513923497 wait complete. 2023-07-14T18:16:15.025 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T18:16:15.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.177+0000 7f87a43ff700 1 -- 172.21.15.133:0/3378382420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f879c100e20 msgr2=0x7f879c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:15.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.177+0000 7f87a43ff700 1 --2- 172.21.15.133:0/3378382420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f879c100e20 0x7f879c101270 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f878c009990 tx=0x7f878c009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:15.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.177+0000 7f87a43ff700 1 -- 172.21.15.133:0/3378382420 shutdown_connections 2023-07-14T18:16:15.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.177+0000 7f87a43ff700 1 --2- 172.21.15.133:0/3378382420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f879c1017b0 0x7f879c108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:15.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.177+0000 7f87a43ff700 1 --2- 172.21.15.133:0/3378382420 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f879c0ffc30 0x7f879c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:15.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.177+0000 7f87a43ff700 1 --2- 172.21.15.133:0/3378382420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f879c100e20 0x7f879c101270 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:15.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.177+0000 7f87a43ff700 1 -- 172.21.15.133:0/3378382420 >> 172.21.15.133:0/3378382420 conn(0x7f879c0fb210 msgr2=0x7f879c0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:15.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.177+0000 7f87a43ff700 1 -- 172.21.15.133:0/3378382420 shutdown_connections 2023-07-14T18:16:15.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.177+0000 7f87a43ff700 1 -- 172.21.15.133:0/3378382420 wait complete. 2023-07-14T18:16:15.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.178+0000 7f87a43ff700 1 Processor -- start 2023-07-14T18:16:15.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.178+0000 7f87a43ff700 1 -- start start 2023-07-14T18:16:15.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.179+0000 7f87a43ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f879c0ffc30 0x7f879c0743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:15.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.179+0000 7f87a43ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f879c100e20 0x7f879c072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:15.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.179+0000 7f87a43ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f879c1017b0 0x7f879c072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:15.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.179+0000 7f87a43ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c073710 con 0x7f879c100e20 2023-07-14T18:16:15.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.179+0000 7f87a43ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c073880 con 0x7f879c1017b0 2023-07-14T18:16:15.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.179+0000 7f87a43ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c19f9c0 con 0x7f879c0ffc30 2023-07-14T18:16:15.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.179+0000 7f87a199a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f879c100e20 0x7f879c072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:15.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.179+0000 7f87a219b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f879c0ffc30 0x7f879c0743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:15.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.179+0000 7f87a199a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f879c100e20 0x7f879c072a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58728/0 (socket says 172.21.15.133:58728) 2023-07-14T18:16:15.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.179+0000 7f87a199a700 1 -- 172.21.15.133:0/1396889846 learned_addr learned my addr 172.21.15.133:0/1396889846 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:15.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.179+0000 7f87a299c700 1 --2- 172.21.15.133:0/1396889846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f879c1017b0 0x7f879c072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:15.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.179+0000 7f87a199a700 1 -- 172.21.15.133:0/1396889846 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f879c0ffc30 msgr2=0x7f879c0743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:15.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.179+0000 7f87a199a700 1 --2- 172.21.15.133:0/1396889846 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f879c0ffc30 0x7f879c0743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:15.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.180+0000 7f87a199a700 1 -- 172.21.15.133:0/1396889846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f879c1017b0 msgr2=0x7f879c072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:15.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.180+0000 7f87a199a700 1 --2- 172.21.15.133:0/1396889846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f879c1017b0 0x7f879c072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:15.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.180+0000 7f87a199a700 1 -- 172.21.15.133:0/1396889846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878c0095f0 con 0x7f879c100e20 2023-07-14T18:16:15.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.180+0000 7f87a199a700 1 --2- 172.21.15.133:0/1396889846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f879c100e20 0x7f879c072a30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f878c00b620 tx=0x7f878c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:15.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.180+0000 7f87937fe700 1 -- 172.21.15.133:0/1396889846 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c01d070 con 0x7f879c100e20 2023-07-14T18:16:15.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.180+0000 7f87937fe700 1 -- 172.21.15.133:0/1396889846 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f878c022470 con 0x7f879c100e20 2023-07-14T18:16:15.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.180+0000 7f87937fe700 1 -- 172.21.15.133:0/1396889846 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c017650 con 0x7f879c100e20 2023-07-14T18:16:15.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.180+0000 7f87a43ff700 1 -- 172.21.15.133:0/1396889846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f879c19fc40 con 0x7f879c100e20 2023-07-14T18:16:15.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.180+0000 7f87a43ff700 1 -- 172.21.15.133:0/1396889846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879c1a0060 con 0x7f879c100e20 2023-07-14T18:16:15.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.181+0000 7f87a43ff700 1 -- 172.21.15.133:0/1396889846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f879c10b4c0 con 0x7f879c100e20 2023-07-14T18:16:15.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.183+0000 7f87937fe700 1 -- 172.21.15.133:0/1396889846 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f878c0225e0 con 0x7f879c100e20 2023-07-14T18:16:15.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.186+0000 7f87937fe700 1 --2- 172.21.15.133:0/1396889846 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f87880a12a0 0x7f87880a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:15.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.186+0000 7f87937fe700 1 -- 172.21.15.133:0/1396889846 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f878c0ce330 con 0x7f879c100e20 2023-07-14T18:16:15.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.186+0000 7f87a219b700 1 --2- 172.21.15.133:0/1396889846 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f87880a12a0 0x7f87880a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:15.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.186+0000 7f87a219b700 1 --2- 172.21.15.133:0/1396889846 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f87880a12a0 0x7f87880a3750 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f879c100c80 tx=0x7f8798009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:15.234 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.190+0000 7f87937fe700 1 -- 172.21.15.133:0/1396889846 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f878c097620 con 0x7f879c100e20 2023-07-14T18:16:15.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:15 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2103468184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:16:15.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:15 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3327748867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:16:15.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:15 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/513923497' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:16:15.476 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.431+0000 7f87a43ff700 1 -- 172.21.15.133:0/1396889846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f879c106ac0 con 0x7f879c100e20 2023-07-14T18:16:15.518 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.432+0000 7f87937fe700 1 -- 172.21.15.133:0/1396889846 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10851 (secure 0 0 0) 0x7f878c09ac40 con 0x7f879c100e20 2023-07-14T18:16:15.518 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:15.519 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3168151271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3168151271},{"type":"v1","addr":"172.21.15.133:6837","nonce":3168151271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14544,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/3093144261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3093144261},{"type":"v1","addr":"172.21.15.133:6839","nonce":3093144261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14556,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/3282902068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":3282902068},{"type":"v1","addr":"172.21.15.133:6841","nonce":3282902068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/4023127259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4023127259},{"type":"v1","addr":"172.21.15.161:6833","nonce":4023127259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2722373740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2722373740},{"type":"v1","addr":"172.21.15.171:6833","nonce":2722373740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2399005074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2399005074},{"type":"v1","addr":"172.21.15.133:6835","nonce":2399005074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T18:14:17.440873+0000","modified":"2023-07-14T18:14:50.886364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24557,"mds_1":24541,"mds_2":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3725502564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3725502564},{"type":"v1","addr":"172.21.15.161:6835","nonce":3725502564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6835/1877362116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1877362116},{"type":"v1","addr":"172.21.15.171:6835","nonce":1877362116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/3458064798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3458064798},{"type":"v1","addr":"172.21.15.161:6837","nonce":3458064798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/106205332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":106205332},{"type":"v1","addr":"172.21.15.171:6837","nonce":106205332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.171:6839/816201198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":816201198},{"type":"v1","addr":"172.21.15.171:6839","nonce":816201198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/2503865871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2503865871},{"type":"v1","addr":"172.21.15.161:6839","nonce":2503865871}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T18:16:15.520 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.435+0000 7f87a43ff700 1 -- 172.21.15.133:0/1396889846 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f87880a12a0 msgr2=0x7f87880a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:15.520 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.435+0000 7f87a43ff700 1 --2- 172.21.15.133:0/1396889846 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f87880a12a0 0x7f87880a3750 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f879c100c80 tx=0x7f8798009260 comp rx=0 tx=0).stop 2023-07-14T18:16:15.520 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.435+0000 7f87a43ff700 1 -- 172.21.15.133:0/1396889846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f879c100e20 msgr2=0x7f879c072a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:15.521 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.435+0000 7f87a43ff700 1 --2- 172.21.15.133:0/1396889846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f879c100e20 0x7f879c072a30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f878c00b620 tx=0x7f878c005740 comp rx=0 tx=0).stop 2023-07-14T18:16:15.521 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.435+0000 7f87a43ff700 1 -- 172.21.15.133:0/1396889846 shutdown_connections 2023-07-14T18:16:15.521 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.435+0000 7f87a43ff700 1 --2- 172.21.15.133:0/1396889846 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f87880a12a0 0x7f87880a3750 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:15.521 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.435+0000 7f87a43ff700 1 --2- 172.21.15.133:0/1396889846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f879c1017b0 0x7f879c072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:15.521 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.435+0000 7f87a43ff700 1 --2- 172.21.15.133:0/1396889846 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f879c0ffc30 0x7f879c0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:15.521 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.435+0000 7f87a43ff700 1 --2- 172.21.15.133:0/1396889846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f879c100e20 0x7f879c072a30 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:15.521 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.435+0000 7f87a43ff700 1 -- 172.21.15.133:0/1396889846 >> 172.21.15.133:0/1396889846 conn(0x7f879c0fb210 msgr2=0x7f879c0fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:15.521 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.435+0000 7f87a43ff700 1 -- 172.21.15.133:0/1396889846 shutdown_connections 2023-07-14T18:16:15.522 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.435+0000 7f87a43ff700 1 -- 172.21.15.133:0/1396889846 wait complete. 2023-07-14T18:16:15.522 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 25 2023-07-14T18:16:15.522 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T18:16:15.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:15 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2103468184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:16:15.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:15 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3327748867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:16:15.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:15 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/513923497' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:16:15.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:15 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2103468184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:16:15.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:15 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3327748867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:16:15.617 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:15 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/513923497' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:16:15.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.674+0000 7f5fbfdff700 1 -- 172.21.15.133:0/752960353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fb80ffc30 msgr2=0x7f5fb8100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:15.721 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.674+0000 7f5fbfdff700 1 --2- 172.21.15.133:0/752960353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fb80ffc30 0x7f5fb8100040 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40098f0 tx=0x7f5fb4009c00 comp rx=0 tx=0).stop 2023-07-14T18:16:15.721 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.674+0000 7f5fbfdff700 1 -- 172.21.15.133:0/752960353 shutdown_connections 2023-07-14T18:16:15.721 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.674+0000 7f5fbfdff700 1 --2- 172.21.15.133:0/752960353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fb80ffc30 0x7f5fb8100040 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:15.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.674+0000 7f5fbfdff700 1 --2- 172.21.15.133:0/752960353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fb81017b0 0x7f5fb8108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:15.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.674+0000 7f5fbfdff700 1 --2- 172.21.15.133:0/752960353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fb8100e20 0x7f5fb8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:15.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.674+0000 7f5fbfdff700 1 -- 172.21.15.133:0/752960353 >> 172.21.15.133:0/752960353 conn(0x7f5fb80fb210 msgr2=0x7f5fb80fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:15.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.675+0000 7f5fbfdff700 1 -- 172.21.15.133:0/752960353 shutdown_connections 2023-07-14T18:16:15.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.675+0000 7f5fbfdff700 1 -- 172.21.15.133:0/752960353 wait complete. 2023-07-14T18:16:15.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.676+0000 7f5fbfdff700 1 Processor -- start 2023-07-14T18:16:15.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.676+0000 7f5fbfdff700 1 -- start start 2023-07-14T18:16:15.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.677+0000 7f5fbfdff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fb80ffc30 0x7f5fb80743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:15.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.677+0000 7f5fbfdff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fb8100e20 0x7f5fb8072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:15.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.677+0000 7f5fbfdff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fb81017b0 0x7f5fb8072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:15.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.677+0000 7f5fbfdff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb8073710 con 0x7f5fb80ffc30 2023-07-14T18:16:15.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.677+0000 7f5fbfdff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb8073880 con 0x7f5fb81017b0 2023-07-14T18:16:15.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.677+0000 7f5fbfdff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb819f9b0 con 0x7f5fb8100e20 2023-07-14T18:16:15.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.677+0000 7f5fbdb9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fb80ffc30 0x7f5fb80743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:15.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.677+0000 7f5fbdb9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fb80ffc30 0x7f5fb80743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58752/0 (socket says 172.21.15.133:58752) 2023-07-14T18:16:15.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.677+0000 7f5fbdb9b700 1 -- 172.21.15.133:0/3328978572 learned_addr learned my addr 172.21.15.133:0/3328978572 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:15.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.677+0000 7f5fbd39a700 1 --2- 172.21.15.133:0/3328978572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fb8100e20 0x7f5fb8072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:15.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.677+0000 7f5fbe39c700 1 --2- 172.21.15.133:0/3328978572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fb81017b0 0x7f5fb8072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:15.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.677+0000 7f5fbdb9b700 1 -- 172.21.15.133:0/3328978572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fb8100e20 msgr2=0x7f5fb8072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:15.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.677+0000 7f5fbdb9b700 1 --2- 172.21.15.133:0/3328978572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fb8100e20 0x7f5fb8072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:15.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.677+0000 7f5fbdb9b700 1 -- 172.21.15.133:0/3328978572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fb81017b0 msgr2=0x7f5fb8072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:15.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.677+0000 7f5fbdb9b700 1 --2- 172.21.15.133:0/3328978572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fb81017b0 0x7f5fb8072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:15.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.677+0000 7f5fbdb9b700 1 -- 172.21.15.133:0/3328978572 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb0009d00 con 0x7f5fb80ffc30 2023-07-14T18:16:15.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.677+0000 7f5fbdb9b700 1 --2- 172.21.15.133:0/3328978572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fb80ffc30 0x7f5fb80743f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4000c00 tx=0x7f5fb400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:15.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.678+0000 7f5faeffd700 1 -- 172.21.15.133:0/3328978572 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb401c070 con 0x7f5fb80ffc30 2023-07-14T18:16:15.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.678+0000 7f5fbfdff700 1 -- 172.21.15.133:0/3328978572 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb4009520 con 0x7f5fb80ffc30 2023-07-14T18:16:15.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.678+0000 7f5faeffd700 1 -- 172.21.15.133:0/3328978572 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fb400f7b0 con 0x7f5fb80ffc30 2023-07-14T18:16:15.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.678+0000 7f5faeffd700 1 -- 172.21.15.133:0/3328978572 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4003a00 con 0x7f5fb80ffc30 2023-07-14T18:16:15.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.678+0000 7f5fbfdff700 1 -- 172.21.15.133:0/3328978572 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb819fed0 con 0x7f5fb80ffc30 2023-07-14T18:16:15.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.679+0000 7f5fbfdff700 1 -- 172.21.15.133:0/3328978572 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb81071f0 con 0x7f5fb80ffc30 2023-07-14T18:16:15.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.681+0000 7f5faeffd700 1 -- 172.21.15.133:0/3328978572 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f5fb4020030 con 0x7f5fb80ffc30 2023-07-14T18:16:15.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.682+0000 7f5faeffd700 1 --2- 172.21.15.133:0/3328978572 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5fa40a1220 0x7f5fa40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:15.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.682+0000 7f5faeffd700 1 -- 172.21.15.133:0/3328978572 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5fb40cde70 con 0x7f5fb80ffc30 2023-07-14T18:16:15.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.682+0000 7f5fbd39a700 1 --2- 172.21.15.133:0/3328978572 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5fa40a1220 0x7f5fa40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:15.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.683+0000 7f5fbd39a700 1 --2- 172.21.15.133:0/3328978572 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5fa40a1220 0x7f5fa40a36d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009bf0 tx=0x7f5fa8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:15.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.686+0000 7f5faeffd700 1 -- 172.21.15.133:0/3328978572 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f5fb4097160 con 0x7f5fb80ffc30 2023-07-14T18:16:15.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.886+0000 7f5fbfdff700 1 -- 172.21.15.133:0/3328978572 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5fb802c8e0 con 0x7f5fb80ffc30 2023-07-14T18:16:15.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.888+0000 7f5faeffd700 1 -- 172.21.15.133:0/3328978572 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24520 (secure 0 0 0) 0x7f5fb409a780 con 0x7f5fb80ffc30 2023-07-14T18:16:15.933 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:15.933 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":92,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","created":"2023-07-14T18:06:49.561582+0000","modified":"2023-07-14T18:15:49.951050+0000","last_up_change":"2023-07-14T18:13:29.836099+0000","last_in_change":"2023-07-14T18:13:11.606710+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T18:10:03.945039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T18:14:15.451340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T18:14:16.892215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T18:14:20.493175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"85a29de7-d712-422a-b647-d9e90091aaf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6803","nonce":3076510801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6805","nonce":3076510801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6809","nonce":3076510801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6807","nonce":3076510801}]},"public_addr":"172.21.15.133:6803/3076510801","cluster_addr":"172.21.15.133:6805/3076510801","heartbeat_back_addr":"172.21.15.133:6809/3076510801","heartbeat_front_addr":"172.21.15.133:6807/3076510801","state":["exists","up"]},{"osd":1,"uuid":"80ad4e78-7917-495e-b119-99f7c7c36243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6801","nonce":1841901852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6803","nonce":1841901852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6807","nonce":1841901852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6805","nonce":1841901852}]},"public_addr":"172.21.15.161:6801/1841901852","cluster_addr":"172.21.15.161:6803/1841901852","heartbeat_back_addr":"172.21.15.161:6807/1841901852","heartbeat_front_addr":"172.21.15.161:6805/1841901852","state":["exists","up"]},{"osd":2,"uuid":"fc4bfd37-1fdd-47c6-a48e-6d910daf9e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6801","nonce":346103099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6803","nonce":346103099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6807","nonce":346103099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6805","nonce":346103099}]},"public_addr":"172.21.15.171:6801/346103099","cluster_addr":"172.21.15.171:6803/346103099","heartbeat_back_addr":"172.21.15.171:6807/346103099","heartbeat_front_addr":"172.21.15.171:6805/346103099","state":["exists","up"]},{"osd":3,"uuid":"4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6811","nonce":2152994143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6813","nonce":2152994143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6817","nonce":2152994143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6815","nonce":2152994143}]},"public_addr":"172.21.15.133:6811/2152994143","cluster_addr":"172.21.15.133:6813/2152994143","heartbeat_back_addr":"172.21.15.133:6817/2152994143","heartbeat_front_addr":"172.21.15.133:6815/2152994143","state":["exists","up"]},{"osd":4,"uuid":"4601a1c2-254b-4c4d-9b15-8fc952b3d2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6809","nonce":47213291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6811","nonce":47213291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6815","nonce":47213291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6813","nonce":47213291}]},"public_addr":"172.21.15.161:6809/47213291","cluster_addr":"172.21.15.161:6811/47213291","heartbeat_back_addr":"172.21.15.161:6815/47213291","heartbeat_front_addr":"172.21.15.161:6813/47213291","state":["exists","up"]},{"osd":5,"uuid":"6cca180a-9140-4e49-835b-86842dae0574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6809","nonce":777556751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6811","nonce":777556751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6815","nonce":777556751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6813","nonce":777556751}]},"public_addr":"172.21.15.171:6809/777556751","cluster_addr":"172.21.15.171:6811/777556751","heartbeat_back_addr":"172.21.15.171:6815/777556751","heartbeat_front_addr":"172.21.15.171:6813/777556751","state":["exists","up"]},{"osd":6,"uuid":"ea0299ff-7740-4494-a3a9-1503b3fc0ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6819","nonce":2884152358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6821","nonce":2884152358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6825","nonce":2884152358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6823","nonce":2884152358}]},"public_addr":"172.21.15.133:6819/2884152358","cluster_addr":"172.21.15.133:6821/2884152358","heartbeat_back_addr":"172.21.15.133:6825/2884152358","heartbeat_front_addr":"172.21.15.133:6823/2884152358","state":["exists","up"]},{"osd":7,"uuid":"b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6817","nonce":4183705749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6819","nonce":4183705749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6823","nonce":4183705749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6821","nonce":4183705749}]},"public_addr":"172.21.15.161:6817/4183705749","cluster_addr":"172.21.15.161:6819/4183705749","heartbeat_back_addr":"172.21.15.161:6823/4183705749","heartbeat_front_addr":"172.21.15.161:6821/4183705749","state":["exists","up"]},{"osd":8,"uuid":"87dffc07-bf48-48c8-b418-5c4b509686d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6817","nonce":2771494483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6819","nonce":2771494483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6823","nonce":2771494483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6821","nonce":2771494483}]},"public_addr":"172.21.15.171:6817/2771494483","cluster_addr":"172.21.15.171:6819/2771494483","heartbeat_back_addr":"172.21.15.171:6823/2771494483","heartbeat_front_addr":"172.21.15.171:6821/2771494483","state":["exists","up"]},{"osd":9,"uuid":"e0832870-79ce-4bce-8045-58b7a7f6dc69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6827","nonce":2066164149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6829","nonce":2066164149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6833","nonce":2066164149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6831","nonce":2066164149}]},"public_addr":"172.21.15.133:6827/2066164149","cluster_addr":"172.21.15.133:6829/2066164149","heartbeat_back_addr":"172.21.15.133:6833/2066164149","heartbeat_front_addr":"172.21.15.133:6831/2066164149","state":["exists","up"]},{"osd":10,"uuid":"081f0484-d134-4b2e-9c48-ef8e6a646422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6825","nonce":2162333626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6827","nonce":2162333626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6831","nonce":2162333626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6829","nonce":2162333626}]},"public_addr":"172.21.15.161:6825/2162333626","cluster_addr":"172.21.15.161:6827/2162333626","heartbeat_back_addr":"172.21.15.161:6831/2162333626","heartbeat_front_addr":"172.21.15.161:6829/2162333626","state":["exists","up"]},{"osd":11,"uuid":"4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6825","nonce":3403894313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6827","nonce":3403894313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6831","nonce":3403894313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6829","nonce":3403894313}]},"public_addr":"172.21.15.171:6825/3403894313","cluster_addr":"172.21.15.171:6827/3403894313","heartbeat_back_addr":"172.21.15.171:6831/3403894313","heartbeat_front_addr":"172.21.15.171:6829/3403894313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:17.816229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:39.638543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:00.543317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:26.988921+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:48.644694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:10.049860+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:35.986508+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:57.753846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:19.018245+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:44.460450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:06.279960+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:27.955006+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/1832569241":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6801/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:0/4219815304":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:6801/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/432253615":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1840700480":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1807775439":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1089030734":"2023-07-15T18:07:47.346704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T18:16:15.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.890+0000 7f5fbfdff700 1 -- 172.21.15.133:0/3328978572 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5fa40a1220 msgr2=0x7f5fa40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:15.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.890+0000 7f5fbfdff700 1 --2- 172.21.15.133:0/3328978572 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5fa40a1220 0x7f5fa40a36d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009bf0 tx=0x7f5fa8009260 comp rx=0 tx=0).stop 2023-07-14T18:16:15.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.890+0000 7f5fbfdff700 1 -- 172.21.15.133:0/3328978572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fb80ffc30 msgr2=0x7f5fb80743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:15.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.890+0000 7f5fbfdff700 1 --2- 172.21.15.133:0/3328978572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fb80ffc30 0x7f5fb80743f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4000c00 tx=0x7f5fb400b730 comp rx=0 tx=0).stop 2023-07-14T18:16:15.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.890+0000 7f5fbfdff700 1 -- 172.21.15.133:0/3328978572 shutdown_connections 2023-07-14T18:16:15.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.890+0000 7f5fbfdff700 1 --2- 172.21.15.133:0/3328978572 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5fa40a1220 0x7f5fa40a36d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:15.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.890+0000 7f5fbfdff700 1 --2- 172.21.15.133:0/3328978572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fb81017b0 0x7f5fb8072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:15.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.890+0000 7f5fbfdff700 1 --2- 172.21.15.133:0/3328978572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fb80ffc30 0x7f5fb80743f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:15.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.890+0000 7f5fbfdff700 1 --2- 172.21.15.133:0/3328978572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fb8100e20 0x7f5fb8072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:15.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.890+0000 7f5fbfdff700 1 -- 172.21.15.133:0/3328978572 >> 172.21.15.133:0/3328978572 conn(0x7f5fb80fb210 msgr2=0x7f5fb8104330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:15.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.890+0000 7f5fbfdff700 1 -- 172.21.15.133:0/3328978572 shutdown_connections 2023-07-14T18:16:15.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:15.890+0000 7f5fbfdff700 1 -- 172.21.15.133:0/3328978572 wait complete. 2023-07-14T18:16:15.955 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T18:10:03.945039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T18:14:15.451340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T18:14:16.892215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T18:14:20.493175+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T18:16:15.956 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T18:16:16.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.113+0000 7f02b93ff700 1 -- 172.21.15.133:0/970089480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02b4074620 msgr2=0x7f02b4072a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:16.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.113+0000 7f02b93ff700 1 --2- 172.21.15.133:0/970089480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02b4074620 0x7f02b4072a70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f02a4009990 tx=0x7f02a4009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:16.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.113+0000 7f02b93ff700 1 -- 172.21.15.133:0/970089480 shutdown_connections 2023-07-14T18:16:16.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.113+0000 7f02b93ff700 1 --2- 172.21.15.133:0/970089480 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02b4107ab0 0x7f02b4109f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:16.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.113+0000 7f02b93ff700 1 --2- 172.21.15.133:0/970089480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02b4074620 0x7f02b4072a70 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:16.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.113+0000 7f02b93ff700 1 --2- 172.21.15.133:0/970089480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02b4073040 0x7f02b40734b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:16.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.113+0000 7f02b93ff700 1 -- 172.21.15.133:0/970089480 >> 172.21.15.133:0/970089480 conn(0x7f02b40fb260 msgr2=0x7f02b40fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:16.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.113+0000 7f02b93ff700 1 -- 172.21.15.133:0/970089480 shutdown_connections 2023-07-14T18:16:16.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.113+0000 7f02b93ff700 1 -- 172.21.15.133:0/970089480 wait complete. 2023-07-14T18:16:16.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.114+0000 7f02b93ff700 1 Processor -- start 2023-07-14T18:16:16.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.114+0000 7f02b93ff700 1 -- start start 2023-07-14T18:16:16.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.114+0000 7f02b93ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02b4073040 0x7f02b4197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:16.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.114+0000 7f02b93ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02b4074620 0x7f02b4198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:16.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.114+0000 7f02b93ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02b4107ab0 0x7f02b419f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:16.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.114+0000 7f02b93ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b419f880 con 0x7f02b4107ab0 2023-07-14T18:16:16.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.114+0000 7f02b93ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b419f9f0 con 0x7f02b4074620 2023-07-14T18:16:16.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.114+0000 7f02b93ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b419fbb0 con 0x7f02b4073040 2023-07-14T18:16:16.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.115+0000 7f02b37fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02b4107ab0 0x7f02b419f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:16.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.115+0000 7f02b2ffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02b4073040 0x7f02b4197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:16.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.115+0000 7f02b27fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02b4074620 0x7f02b4198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:16.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.115+0000 7f02b37fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02b4107ab0 0x7f02b419f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58758/0 (socket says 172.21.15.133:58758) 2023-07-14T18:16:16.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.115+0000 7f02b37fe700 1 -- 172.21.15.133:0/1153969715 learned_addr learned my addr 172.21.15.133:0/1153969715 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:16.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.115+0000 7f02b27fc700 1 -- 172.21.15.133:0/1153969715 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02b4073040 msgr2=0x7f02b4197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:16.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.115+0000 7f02b27fc700 1 --2- 172.21.15.133:0/1153969715 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02b4073040 0x7f02b4197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:16.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.115+0000 7f02b27fc700 1 -- 172.21.15.133:0/1153969715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02b4107ab0 msgr2=0x7f02b419f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:16.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.115+0000 7f02b27fc700 1 --2- 172.21.15.133:0/1153969715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02b4107ab0 0x7f02b419f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:16.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.115+0000 7f02b27fc700 1 -- 172.21.15.133:0/1153969715 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a8009520 con 0x7f02b4074620 2023-07-14T18:16:16.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.115+0000 7f02b2ffd700 1 --2- 172.21.15.133:0/1153969715 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02b4073040 0x7f02b4197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:16:16.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.115+0000 7f02b27fc700 1 --2- 172.21.15.133:0/1153969715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02b4074620 0x7f02b4198280 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f02a800e9f0 tx=0x7f02a800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:16.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.115+0000 7f029bfff700 1 -- 172.21.15.133:0/1153969715 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a800cb70 con 0x7f02b4074620 2023-07-14T18:16:16.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.116+0000 7f029bfff700 1 -- 172.21.15.133:0/1153969715 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02a800ee90 con 0x7f02b4074620 2023-07-14T18:16:16.165 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.116+0000 7f02b93ff700 1 -- 172.21.15.133:0/1153969715 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a40095f0 con 0x7f02b4074620 2023-07-14T18:16:16.165 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.116+0000 7f029bfff700 1 -- 172.21.15.133:0/1153969715 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a8003cf0 con 0x7f02b4074620 2023-07-14T18:16:16.165 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.116+0000 7f02b93ff700 1 -- 172.21.15.133:0/1153969715 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b41a01c0 con 0x7f02b4074620 2023-07-14T18:16:16.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.117+0000 7f02b93ff700 1 -- 172.21.15.133:0/1153969715 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b410d620 con 0x7f02b4074620 2023-07-14T18:16:16.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.118+0000 7f029bfff700 1 -- 172.21.15.133:0/1153969715 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f02a800ccd0 con 0x7f02b4074620 2023-07-14T18:16:16.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.120+0000 7f029bfff700 1 --2- 172.21.15.133:0/1153969715 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f029c0a11e0 0x7f029c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:16.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.120+0000 7f029bfff700 1 -- 172.21.15.133:0/1153969715 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f02a8014070 con 0x7f02b4074620 2023-07-14T18:16:16.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.120+0000 7f02b2ffd700 1 --2- 172.21.15.133:0/1153969715 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f029c0a11e0 0x7f029c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:16.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.120+0000 7f02b2ffd700 1 --2- 172.21.15.133:0/1153969715 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f029c0a11e0 0x7f029c0a3690 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f02a400b2e0 tx=0x7f02a4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:16.169 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.124+0000 7f029bfff700 1 -- 172.21.15.133:0/1153969715 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f02a8096340 con 0x7f02b4074620 2023-07-14T18:16:16.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.319+0000 7f02b93ff700 1 -- 172.21.15.133:0/1153969715 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f02b41a0ad0 con 0x7f02b4074620 2023-07-14T18:16:16.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.320+0000 7f029bfff700 1 -- 172.21.15.133:0/1153969715 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f02a8099960 con 0x7f02b4074620 2023-07-14T18:16:16.365 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-07-14T18:16:16.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.322+0000 7f02b93ff700 1 -- 172.21.15.133:0/1153969715 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f029c0a11e0 msgr2=0x7f029c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:16.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.322+0000 7f02b93ff700 1 --2- 172.21.15.133:0/1153969715 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f029c0a11e0 0x7f029c0a3690 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f02a400b2e0 tx=0x7f02a4005fb0 comp rx=0 tx=0).stop 2023-07-14T18:16:16.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.322+0000 7f02b93ff700 1 -- 172.21.15.133:0/1153969715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02b4074620 msgr2=0x7f02b4198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:16.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.322+0000 7f02b93ff700 1 --2- 172.21.15.133:0/1153969715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02b4074620 0x7f02b4198280 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f02a800e9f0 tx=0x7f02a800ed00 comp rx=0 tx=0).stop 2023-07-14T18:16:16.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.322+0000 7f02b93ff700 1 -- 172.21.15.133:0/1153969715 shutdown_connections 2023-07-14T18:16:16.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.322+0000 7f02b93ff700 1 --2- 172.21.15.133:0/1153969715 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f029c0a11e0 0x7f029c0a3690 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:16.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.322+0000 7f02b93ff700 1 --2- 172.21.15.133:0/1153969715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02b4107ab0 0x7f02b419f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:16.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.322+0000 7f02b93ff700 1 --2- 172.21.15.133:0/1153969715 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02b4073040 0x7f02b4197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:16.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.322+0000 7f02b93ff700 1 --2- 172.21.15.133:0/1153969715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02b4074620 0x7f02b4198280 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:16.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.322+0000 7f02b93ff700 1 -- 172.21.15.133:0/1153969715 >> 172.21.15.133:0/1153969715 conn(0x7f02b40fb260 msgr2=0x7f02b4109d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:16.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.322+0000 7f02b93ff700 1 -- 172.21.15.133:0/1153969715 shutdown_connections 2023-07-14T18:16:16.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.322+0000 7f02b93ff700 1 -- 172.21.15.133:0/1153969715 wait complete. 2023-07-14T18:16:16.384 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:16 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1396889846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:16:16.384 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:16 smithi133 ceph-mon[126384]: pgmap v336: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-07-14T18:16:16.385 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:16 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3328978572' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:16:16.386 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T18:16:16.587 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.542+0000 7f6f443ff700 1 -- 172.21.15.133:0/382790975 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f3c100e00 msgr2=0x7f6f3c101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:16.588 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.542+0000 7f6f443ff700 1 --2- 172.21.15.133:0/382790975 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f3c100e00 0x7f6f3c101250 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009990 tx=0x7f6f2c009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:16.588 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.542+0000 7f6f443ff700 1 -- 172.21.15.133:0/382790975 shutdown_connections 2023-07-14T18:16:16.588 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.542+0000 7f6f443ff700 1 --2- 172.21.15.133:0/382790975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f3c101790 0x7f6f3c108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:16.588 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.542+0000 7f6f443ff700 1 --2- 172.21.15.133:0/382790975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f3c0ffc10 0x7f6f3c100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:16.588 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.542+0000 7f6f443ff700 1 --2- 172.21.15.133:0/382790975 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f3c100e00 0x7f6f3c101250 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:16.588 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.542+0000 7f6f443ff700 1 -- 172.21.15.133:0/382790975 >> 172.21.15.133:0/382790975 conn(0x7f6f3c0fb210 msgr2=0x7f6f3c0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:16.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.542+0000 7f6f443ff700 1 -- 172.21.15.133:0/382790975 shutdown_connections 2023-07-14T18:16:16.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.542+0000 7f6f443ff700 1 -- 172.21.15.133:0/382790975 wait complete. 2023-07-14T18:16:16.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.543+0000 7f6f443ff700 1 Processor -- start 2023-07-14T18:16:16.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.543+0000 7f6f443ff700 1 -- start start 2023-07-14T18:16:16.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.544+0000 7f6f443ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f3c0ffc10 0x7f6f3c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:16.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.544+0000 7f6f443ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f3c100e00 0x7f6f3c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:16.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.544+0000 7f6f443ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f3c101790 0x7f6f3c19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:16.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.544+0000 7f6f443ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c19f6e0 con 0x7f6f3c101790 2023-07-14T18:16:16.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.544+0000 7f6f443ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c19f820 con 0x7f6f3c0ffc10 2023-07-14T18:16:16.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.544+0000 7f6f443ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c19f9b0 con 0x7f6f3c100e00 2023-07-14T18:16:16.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.544+0000 7f6f4299c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f3c101790 0x7f6f3c19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:16.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.544+0000 7f6f4299c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f3c101790 0x7f6f3c19f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58786/0 (socket says 172.21.15.133:58786) 2023-07-14T18:16:16.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.544+0000 7f6f4299c700 1 -- 172.21.15.133:0/2230801271 learned_addr learned my addr 172.21.15.133:0/2230801271 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:16.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.544+0000 7f6f4219b700 1 --2- 172.21.15.133:0/2230801271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f3c0ffc10 0x7f6f3c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:16.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.544+0000 7f6f4199a700 1 --2- 172.21.15.133:0/2230801271 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f3c100e00 0x7f6f3c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:16.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.544+0000 7f6f4299c700 1 -- 172.21.15.133:0/2230801271 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f3c100e00 msgr2=0x7f6f3c1980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:16.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.544+0000 7f6f4299c700 1 --2- 172.21.15.133:0/2230801271 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f3c100e00 0x7f6f3c1980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:16.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.544+0000 7f6f4299c700 1 -- 172.21.15.133:0/2230801271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f3c0ffc10 msgr2=0x7f6f3c197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:16.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.544+0000 7f6f4299c700 1 --2- 172.21.15.133:0/2230801271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f3c0ffc10 0x7f6f3c197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:16.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.544+0000 7f6f4299c700 1 -- 172.21.15.133:0/2230801271 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f38009520 con 0x7f6f3c101790 2023-07-14T18:16:16.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.544+0000 7f6f4299c700 1 --2- 172.21.15.133:0/2230801271 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f3c101790 0x7f6f3c19f110 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6f3400ef30 tx=0x7f6f3400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:16.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.545+0000 7f6f337fe700 1 -- 172.21.15.133:0/2230801271 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f34018070 con 0x7f6f3c101790 2023-07-14T18:16:16.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.545+0000 7f6f443ff700 1 -- 172.21.15.133:0/2230801271 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f2c0095f0 con 0x7f6f3c101790 2023-07-14T18:16:16.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.545+0000 7f6f337fe700 1 -- 172.21.15.133:0/2230801271 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f3400f040 con 0x7f6f3c101790 2023-07-14T18:16:16.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.545+0000 7f6f337fe700 1 -- 172.21.15.133:0/2230801271 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3400cd00 con 0x7f6f3c101790 2023-07-14T18:16:16.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.545+0000 7f6f443ff700 1 -- 172.21.15.133:0/2230801271 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f3c1a0000 con 0x7f6f3c101790 2023-07-14T18:16:16.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.546+0000 7f6f443ff700 1 -- 172.21.15.133:0/2230801271 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f3c1922c0 con 0x7f6f3c101790 2023-07-14T18:16:16.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.548+0000 7f6f337fe700 1 -- 172.21.15.133:0/2230801271 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f6f34003a00 con 0x7f6f3c101790 2023-07-14T18:16:16.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.549+0000 7f6f337fe700 1 --2- 172.21.15.133:0/2230801271 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6f280a11e0 0x7f6f280a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:16.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.549+0000 7f6f337fe700 1 -- 172.21.15.133:0/2230801271 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6f340cc910 con 0x7f6f3c101790 2023-07-14T18:16:16.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.549+0000 7f6f4219b700 1 --2- 172.21.15.133:0/2230801271 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6f280a11e0 0x7f6f280a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:16.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.550+0000 7f6f4219b700 1 --2- 172.21.15.133:0/2230801271 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6f280a11e0 0x7f6f280a3690 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c100c60 tx=0x7f6f38009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:16.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.553+0000 7f6f337fe700 1 -- 172.21.15.133:0/2230801271 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f6f34095c00 con 0x7f6f3c101790 2023-07-14T18:16:16.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:16 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1396889846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:16:16.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:16 smithi161 ceph-mon[128993]: pgmap v336: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-07-14T18:16:16.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:16 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3328978572' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:16:16.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:16 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1396889846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:16:16.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:16 smithi171 ceph-mon[129241]: pgmap v336: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-07-14T18:16:16.617 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:16 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3328978572' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:16:16.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.752+0000 7f6f443ff700 1 -- 172.21.15.133:0/2230801271 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6f3c02cd50 con 0x7f6f3c101790 2023-07-14T18:16:16.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.752+0000 7f6f337fe700 1 -- 172.21.15.133:0/2230801271 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6f340b7e60 con 0x7f6f3c101790 2023-07-14T18:16:16.798 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-07-14T18:16:16.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.755+0000 7f6f443ff700 1 -- 172.21.15.133:0/2230801271 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6f280a11e0 msgr2=0x7f6f280a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:16.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.755+0000 7f6f443ff700 1 --2- 172.21.15.133:0/2230801271 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6f280a11e0 0x7f6f280a3690 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c100c60 tx=0x7f6f38009260 comp rx=0 tx=0).stop 2023-07-14T18:16:16.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.755+0000 7f6f443ff700 1 -- 172.21.15.133:0/2230801271 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f3c101790 msgr2=0x7f6f3c19f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:16.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.755+0000 7f6f443ff700 1 --2- 172.21.15.133:0/2230801271 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f3c101790 0x7f6f3c19f110 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6f3400ef30 tx=0x7f6f3400c3a0 comp rx=0 tx=0).stop 2023-07-14T18:16:16.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.755+0000 7f6f443ff700 1 -- 172.21.15.133:0/2230801271 shutdown_connections 2023-07-14T18:16:16.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.755+0000 7f6f443ff700 1 --2- 172.21.15.133:0/2230801271 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6f280a11e0 0x7f6f280a3690 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:16.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.755+0000 7f6f443ff700 1 --2- 172.21.15.133:0/2230801271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f3c0ffc10 0x7f6f3c197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:16.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.755+0000 7f6f443ff700 1 --2- 172.21.15.133:0/2230801271 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f3c101790 0x7f6f3c19f110 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:16.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.755+0000 7f6f443ff700 1 --2- 172.21.15.133:0/2230801271 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f3c100e00 0x7f6f3c1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:16.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.755+0000 7f6f443ff700 1 -- 172.21.15.133:0/2230801271 >> 172.21.15.133:0/2230801271 conn(0x7f6f3c0fb210 msgr2=0x7f6f3c106700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:16.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.755+0000 7f6f443ff700 1 -- 172.21.15.133:0/2230801271 shutdown_connections 2023-07-14T18:16:16.802 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.755+0000 7f6f443ff700 1 -- 172.21.15.133:0/2230801271 wait complete. 2023-07-14T18:16:16.819 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T18:16:17.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.975+0000 7f70b57ff700 1 -- 172.21.15.133:0/2488447260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70b00ffc20 msgr2=0x7f70b0100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:17.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.975+0000 7f70b57ff700 1 --2- 172.21.15.133:0/2488447260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70b00ffc20 0x7f70b0100030 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f70a0009990 tx=0x7f70a0009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:17.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.975+0000 7f70b57ff700 1 -- 172.21.15.133:0/2488447260 shutdown_connections 2023-07-14T18:16:17.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.975+0000 7f70b57ff700 1 --2- 172.21.15.133:0/2488447260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70b01017a0 0x7f70b0108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:17.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.975+0000 7f70b57ff700 1 --2- 172.21.15.133:0/2488447260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70b00ffc20 0x7f70b0100030 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:17.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.975+0000 7f70b57ff700 1 --2- 172.21.15.133:0/2488447260 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70b0100e10 0x7f70b0101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:17.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.975+0000 7f70b57ff700 1 -- 172.21.15.133:0/2488447260 >> 172.21.15.133:0/2488447260 conn(0x7f70b00fb260 msgr2=0x7f70b00fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:17.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.975+0000 7f70b57ff700 1 -- 172.21.15.133:0/2488447260 shutdown_connections 2023-07-14T18:16:17.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.975+0000 7f70b57ff700 1 -- 172.21.15.133:0/2488447260 wait complete. 2023-07-14T18:16:17.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.976+0000 7f70b57ff700 1 Processor -- start 2023-07-14T18:16:17.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.976+0000 7f70b57ff700 1 -- start start 2023-07-14T18:16:17.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.977+0000 7f70b57ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70b00ffc20 0x7f70b00743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:17.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.977+0000 7f70b57ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70b0100e10 0x7f70b0072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:17.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.977+0000 7f70b57ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70b01017a0 0x7f70b0072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:17.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.977+0000 7f70b57ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b0073710 con 0x7f70b0100e10 2023-07-14T18:16:17.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.977+0000 7f70b57ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b0073880 con 0x7f70b00ffc20 2023-07-14T18:16:17.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.977+0000 7f70b57ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b019f940 con 0x7f70b01017a0 2023-07-14T18:16:17.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.977+0000 7f70aeffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70b00ffc20 0x7f70b00743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:17.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.977+0000 7f70aeffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70b00ffc20 0x7f70b00743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:49790/0 (socket says 172.21.15.133:49790) 2023-07-14T18:16:17.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.977+0000 7f70aeffd700 1 -- 172.21.15.133:0/2365655965 learned_addr learned my addr 172.21.15.133:0/2365655965 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:17.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.977+0000 7f70ae7fc700 1 --2- 172.21.15.133:0/2365655965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70b0100e10 0x7f70b0072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:17.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.977+0000 7f70af7fe700 1 --2- 172.21.15.133:0/2365655965 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70b01017a0 0x7f70b0072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:17.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.978+0000 7f70ae7fc700 1 -- 172.21.15.133:0/2365655965 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70b01017a0 msgr2=0x7f70b0072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:17.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.978+0000 7f70ae7fc700 1 --2- 172.21.15.133:0/2365655965 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70b01017a0 0x7f70b0072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:17.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.978+0000 7f70ae7fc700 1 -- 172.21.15.133:0/2365655965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70b00ffc20 msgr2=0x7f70b00743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:17.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.978+0000 7f70ae7fc700 1 --2- 172.21.15.133:0/2365655965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70b00ffc20 0x7f70b00743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:17.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.978+0000 7f70ae7fc700 1 -- 172.21.15.133:0/2365655965 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a00095f0 con 0x7f70b0100e10 2023-07-14T18:16:17.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.978+0000 7f70aeffd700 1 --2- 172.21.15.133:0/2365655965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70b00ffc20 0x7f70b00743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:16:17.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.978+0000 7f70ae7fc700 1 --2- 172.21.15.133:0/2365655965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70b0100e10 0x7f70b0072a30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f70a400e9f0 tx=0x7f70a400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:17.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.978+0000 7f7097fff700 1 -- 172.21.15.133:0/2365655965 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a400c9c0 con 0x7f70b0100e10 2023-07-14T18:16:17.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.978+0000 7f70b57ff700 1 -- 172.21.15.133:0/2365655965 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b019fc20 con 0x7f70b0100e10 2023-07-14T18:16:17.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.978+0000 7f7097fff700 1 -- 172.21.15.133:0/2365655965 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70a4010430 con 0x7f70b0100e10 2023-07-14T18:16:17.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.978+0000 7f7097fff700 1 -- 172.21.15.133:0/2365655965 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a4018680 con 0x7f70b0100e10 2023-07-14T18:16:17.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.978+0000 7f70b57ff700 1 -- 172.21.15.133:0/2365655965 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b01a0180 con 0x7f70b0100e10 2023-07-14T18:16:17.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.979+0000 7f70b57ff700 1 -- 172.21.15.133:0/2365655965 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b0106b50 con 0x7f70b0100e10 2023-07-14T18:16:17.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.981+0000 7f7097fff700 1 -- 172.21.15.133:0/2365655965 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f70a4010860 con 0x7f70b0100e10 2023-07-14T18:16:17.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.982+0000 7f7097fff700 1 --2- 172.21.15.133:0/2365655965 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f70980a11e0 0x7f70980a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:17.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.982+0000 7f7097fff700 1 -- 172.21.15.133:0/2365655965 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f70a4014070 con 0x7f70b0100e10 2023-07-14T18:16:17.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.983+0000 7f70aeffd700 1 --2- 172.21.15.133:0/2365655965 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f70980a11e0 0x7f70980a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:17.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.983+0000 7f70aeffd700 1 --2- 172.21.15.133:0/2365655965 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f70980a11e0 0x7f70980a3690 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f70a0006010 tx=0x7f70a0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:17.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:16.986+0000 7f7097fff700 1 -- 172.21.15.133:0/2365655965 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f70a4051780 con 0x7f70b0100e10 2023-07-14T18:16:17.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.187+0000 7f70b57ff700 1 -- 172.21.15.133:0/2365655965 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f70b002caf0 con 0x7f70b0100e10 2023-07-14T18:16:17.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.188+0000 7f7097fff700 1 -- 172.21.15.133:0/2365655965 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f70a4096190 con 0x7f70b0100e10 2023-07-14T18:16:17.233 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2023-07-14T18:16:17.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.190+0000 7f70b57ff700 1 -- 172.21.15.133:0/2365655965 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f70980a11e0 msgr2=0x7f70980a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:17.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.190+0000 7f70b57ff700 1 --2- 172.21.15.133:0/2365655965 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f70980a11e0 0x7f70980a3690 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f70a0006010 tx=0x7f70a0005960 comp rx=0 tx=0).stop 2023-07-14T18:16:17.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.190+0000 7f70b57ff700 1 -- 172.21.15.133:0/2365655965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70b0100e10 msgr2=0x7f70b0072a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:17.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.190+0000 7f70b57ff700 1 --2- 172.21.15.133:0/2365655965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70b0100e10 0x7f70b0072a30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f70a400e9f0 tx=0x7f70a400ed00 comp rx=0 tx=0).stop 2023-07-14T18:16:17.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.191+0000 7f70b57ff700 1 -- 172.21.15.133:0/2365655965 shutdown_connections 2023-07-14T18:16:17.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.191+0000 7f70b57ff700 1 --2- 172.21.15.133:0/2365655965 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f70980a11e0 0x7f70980a3690 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:17.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.191+0000 7f70b57ff700 1 --2- 172.21.15.133:0/2365655965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70b00ffc20 0x7f70b00743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:17.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.191+0000 7f70b57ff700 1 --2- 172.21.15.133:0/2365655965 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70b01017a0 0x7f70b0072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:17.238 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.191+0000 7f70b57ff700 1 --2- 172.21.15.133:0/2365655965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70b0100e10 0x7f70b0072a30 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:17.238 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.191+0000 7f70b57ff700 1 -- 172.21.15.133:0/2365655965 >> 172.21.15.133:0/2365655965 conn(0x7f70b00fb260 msgr2=0x7f70b00fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:17.238 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.191+0000 7f70b57ff700 1 -- 172.21.15.133:0/2365655965 shutdown_connections 2023-07-14T18:16:17.238 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.191+0000 7f70b57ff700 1 -- 172.21.15.133:0/2365655965 wait complete. 2023-07-14T18:16:17.255 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T18:16:17.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:17 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1153969715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:16:17.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:17 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2230801271' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:16:17.457 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.411+0000 7f39dbfff700 1 -- 172.21.15.133:0/680337136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39d4100e10 msgr2=0x7f39d4101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:17.457 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.411+0000 7f39dbfff700 1 --2- 172.21.15.133:0/680337136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39d4100e10 0x7f39d4101260 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f39c4009990 tx=0x7f39c4009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:17.457 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.412+0000 7f39dbfff700 1 -- 172.21.15.133:0/680337136 shutdown_connections 2023-07-14T18:16:17.457 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.412+0000 7f39dbfff700 1 --2- 172.21.15.133:0/680337136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39d40ffc20 0x7f39d4100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:17.458 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.412+0000 7f39dbfff700 1 --2- 172.21.15.133:0/680337136 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39d41017a0 0x7f39d4108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:17.458 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.412+0000 7f39dbfff700 1 --2- 172.21.15.133:0/680337136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39d4100e10 0x7f39d4101260 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:17.458 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.412+0000 7f39dbfff700 1 -- 172.21.15.133:0/680337136 >> 172.21.15.133:0/680337136 conn(0x7f39d40fb260 msgr2=0x7f39d40fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:17.458 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.412+0000 7f39dbfff700 1 -- 172.21.15.133:0/680337136 shutdown_connections 2023-07-14T18:16:17.458 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.412+0000 7f39dbfff700 1 -- 172.21.15.133:0/680337136 wait complete. 2023-07-14T18:16:17.459 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.413+0000 7f39dbfff700 1 Processor -- start 2023-07-14T18:16:17.460 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.413+0000 7f39dbfff700 1 -- start start 2023-07-14T18:16:17.460 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.413+0000 7f39dbfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39d40ffc20 0x7f39d40743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:17.460 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.413+0000 7f39dbfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39d4100e10 0x7f39d4072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:17.460 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.413+0000 7f39dbfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39d41017a0 0x7f39d4072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:17.460 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.413+0000 7f39dbfff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d4073710 con 0x7f39d40ffc20 2023-07-14T18:16:17.460 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.413+0000 7f39dbfff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d4073880 con 0x7f39d41017a0 2023-07-14T18:16:17.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.413+0000 7f39dbfff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d419f9d0 con 0x7f39d4100e10 2023-07-14T18:16:17.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.413+0000 7f39d9d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39d40ffc20 0x7f39d40743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:17.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.414+0000 7f39d9d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39d40ffc20 0x7f39d40743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44222/0 (socket says 172.21.15.133:44222) 2023-07-14T18:16:17.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.414+0000 7f39d9d9b700 1 -- 172.21.15.133:0/579931458 learned_addr learned my addr 172.21.15.133:0/579931458 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:17.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.414+0000 7f39d959a700 1 --2- 172.21.15.133:0/579931458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39d4100e10 0x7f39d4072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:17.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.414+0000 7f39da59c700 1 --2- 172.21.15.133:0/579931458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39d41017a0 0x7f39d4072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:17.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.414+0000 7f39d959a700 1 -- 172.21.15.133:0/579931458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39d41017a0 msgr2=0x7f39d4072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:17.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.414+0000 7f39d959a700 1 --2- 172.21.15.133:0/579931458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39d41017a0 0x7f39d4072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:17.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.414+0000 7f39d959a700 1 -- 172.21.15.133:0/579931458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39d40ffc20 msgr2=0x7f39d40743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:17.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.414+0000 7f39d959a700 1 --2- 172.21.15.133:0/579931458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39d40ffc20 0x7f39d40743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:17.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.414+0000 7f39d959a700 1 -- 172.21.15.133:0/579931458 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c40095f0 con 0x7f39d4100e10 2023-07-14T18:16:17.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.414+0000 7f39da59c700 1 --2- 172.21.15.133:0/579931458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39d41017a0 0x7f39d4072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:16:17.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.414+0000 7f39d9d9b700 1 --2- 172.21.15.133:0/579931458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39d40ffc20 0x7f39d40743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:16:17.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.414+0000 7f39d959a700 1 --2- 172.21.15.133:0/579931458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39d4100e10 0x7f39d4072a30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f39c40098e0 tx=0x7f39c4005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:17.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.415+0000 7f39caffd700 1 -- 172.21.15.133:0/579931458 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c401d070 con 0x7f39d4100e10 2023-07-14T18:16:17.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.415+0000 7f39dbfff700 1 -- 172.21.15.133:0/579931458 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d419fc50 con 0x7f39d4100e10 2023-07-14T18:16:17.463 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.415+0000 7f39caffd700 1 -- 172.21.15.133:0/579931458 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39c4021410 con 0x7f39d4100e10 2023-07-14T18:16:17.463 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.415+0000 7f39caffd700 1 -- 172.21.15.133:0/579931458 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c4017650 con 0x7f39d4100e10 2023-07-14T18:16:17.463 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.415+0000 7f39dbfff700 1 -- 172.21.15.133:0/579931458 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d41a0080 con 0x7f39d4100e10 2023-07-14T18:16:17.463 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.416+0000 7f39dbfff700 1 -- 172.21.15.133:0/579931458 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d410b440 con 0x7f39d4100e10 2023-07-14T18:16:17.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.418+0000 7f39caffd700 1 -- 172.21.15.133:0/579931458 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f39c4021580 con 0x7f39d4100e10 2023-07-14T18:16:17.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.419+0000 7f39caffd700 1 --2- 172.21.15.133:0/579931458 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f39c00a11e0 0x7f39c00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:17.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.419+0000 7f39caffd700 1 -- 172.21.15.133:0/579931458 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f39c40ce270 con 0x7f39d4100e10 2023-07-14T18:16:17.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.419+0000 7f39d9d9b700 1 --2- 172.21.15.133:0/579931458 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f39c00a11e0 0x7f39c00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:17.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.420+0000 7f39d9d9b700 1 --2- 172.21.15.133:0/579931458 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f39c00a11e0 0x7f39c00a3690 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f39d0005950 tx=0x7f39d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:17.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.421+0000 7f39caffd700 1 -- 172.21.15.133:0/579931458 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f39c4097560 con 0x7f39d4100e10 2023-07-14T18:16:17.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:17 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1153969715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:16:17.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:17 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2230801271' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:16:17.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:17 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1153969715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:16:17.617 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:17 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2230801271' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:16:17.696 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.651+0000 7f39dbfff700 1 -- 172.21.15.133:0/579931458 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f39d4106b20 con 0x7f39d4100e10 2023-07-14T18:16:17.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.652+0000 7f39caffd700 1 -- 172.21.15.133:0/579931458 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f39c409ab80 con 0x7f39d4100e10 2023-07-14T18:16:17.697 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-07-14T18:16:17.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.654+0000 7f39dbfff700 1 -- 172.21.15.133:0/579931458 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f39c00a11e0 msgr2=0x7f39c00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:17.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.654+0000 7f39dbfff700 1 --2- 172.21.15.133:0/579931458 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f39c00a11e0 0x7f39c00a3690 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f39d0005950 tx=0x7f39d0009260 comp rx=0 tx=0).stop 2023-07-14T18:16:17.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.654+0000 7f39dbfff700 1 -- 172.21.15.133:0/579931458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39d4100e10 msgr2=0x7f39d4072a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:17.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.654+0000 7f39dbfff700 1 --2- 172.21.15.133:0/579931458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39d4100e10 0x7f39d4072a30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f39c40098e0 tx=0x7f39c4005740 comp rx=0 tx=0).stop 2023-07-14T18:16:17.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.655+0000 7f39dbfff700 1 -- 172.21.15.133:0/579931458 shutdown_connections 2023-07-14T18:16:17.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.655+0000 7f39dbfff700 1 --2- 172.21.15.133:0/579931458 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f39c00a11e0 0x7f39c00a3690 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:17.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.655+0000 7f39dbfff700 1 --2- 172.21.15.133:0/579931458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39d41017a0 0x7f39d4072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:17.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.655+0000 7f39dbfff700 1 --2- 172.21.15.133:0/579931458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39d40ffc20 0x7f39d40743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:17.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.655+0000 7f39dbfff700 1 --2- 172.21.15.133:0/579931458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39d4100e10 0x7f39d4072a30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:17.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.655+0000 7f39dbfff700 1 -- 172.21.15.133:0/579931458 >> 172.21.15.133:0/579931458 conn(0x7f39d40fb260 msgr2=0x7f39d40fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:17.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.655+0000 7f39dbfff700 1 -- 172.21.15.133:0/579931458 shutdown_connections 2023-07-14T18:16:17.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.655+0000 7f39dbfff700 1 -- 172.21.15.133:0/579931458 wait complete. 2023-07-14T18:16:17.719 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T18:16:17.915 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.870+0000 7f26dcbff700 1 -- 172.21.15.133:0/1070753684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26d8074680 msgr2=0x7f26d8072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:17.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.870+0000 7f26dcbff700 1 --2- 172.21.15.133:0/1070753684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26d8074680 0x7f26d8072ad0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009990 tx=0x7f26c8009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:17.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.870+0000 7f26dcbff700 1 -- 172.21.15.133:0/1070753684 shutdown_connections 2023-07-14T18:16:17.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.870+0000 7f26dcbff700 1 --2- 172.21.15.133:0/1070753684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26d8107af0 0x7f26d8109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:17.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.870+0000 7f26dcbff700 1 --2- 172.21.15.133:0/1070753684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26d8074680 0x7f26d8072ad0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:17.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.870+0000 7f26dcbff700 1 --2- 172.21.15.133:0/1070753684 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26d80730a0 0x7f26d8073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:17.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.870+0000 7f26dcbff700 1 -- 172.21.15.133:0/1070753684 >> 172.21.15.133:0/1070753684 conn(0x7f26d80fb210 msgr2=0x7f26d80fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:17.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.870+0000 7f26dcbff700 1 -- 172.21.15.133:0/1070753684 shutdown_connections 2023-07-14T18:16:17.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.870+0000 7f26dcbff700 1 -- 172.21.15.133:0/1070753684 wait complete. 2023-07-14T18:16:17.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.871+0000 7f26dcbff700 1 Processor -- start 2023-07-14T18:16:17.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.872+0000 7f26dcbff700 1 -- start start 2023-07-14T18:16:17.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.872+0000 7f26dcbff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26d80730a0 0x7f26d8197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:17.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.872+0000 7f26dcbff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26d8074680 0x7f26d81982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:17.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.872+0000 7f26dcbff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26d8107af0 0x7f26d819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:17.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.872+0000 7f26dcbff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d819f8e0 con 0x7f26d8107af0 2023-07-14T18:16:17.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.872+0000 7f26dcbff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d819fa20 con 0x7f26d8074680 2023-07-14T18:16:17.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.872+0000 7f26dcbff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d819fbb0 con 0x7f26d80730a0 2023-07-14T18:16:17.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.872+0000 7f26d6d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26d8107af0 0x7f26d819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:17.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.872+0000 7f26d6d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26d8107af0 0x7f26d819f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44252/0 (socket says 172.21.15.133:44252) 2023-07-14T18:16:17.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.872+0000 7f26d6d9d700 1 -- 172.21.15.133:0/2268298862 learned_addr learned my addr 172.21.15.133:0/2268298862 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:17.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.872+0000 7f26d659c700 1 --2- 172.21.15.133:0/2268298862 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26d80730a0 0x7f26d8197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:17.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.872+0000 7f26d5d9b700 1 --2- 172.21.15.133:0/2268298862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26d8074680 0x7f26d81982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:17.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.873+0000 7f26d6d9d700 1 -- 172.21.15.133:0/2268298862 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26d80730a0 msgr2=0x7f26d8197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:17.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.873+0000 7f26d6d9d700 1 --2- 172.21.15.133:0/2268298862 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26d80730a0 0x7f26d8197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:17.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.873+0000 7f26d6d9d700 1 -- 172.21.15.133:0/2268298862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26d8074680 msgr2=0x7f26d81982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:17.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.873+0000 7f26d6d9d700 1 --2- 172.21.15.133:0/2268298862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26d8074680 0x7f26d81982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:17.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.873+0000 7f26d6d9d700 1 -- 172.21.15.133:0/2268298862 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c80095f0 con 0x7f26d8107af0 2023-07-14T18:16:17.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.873+0000 7f26d6d9d700 1 --2- 172.21.15.133:0/2268298862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26d8107af0 0x7f26d819f310 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f26d000ef30 tx=0x7f26d000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:17.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.873+0000 7f26c77fe700 1 -- 172.21.15.133:0/2268298862 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d0018070 con 0x7f26d8107af0 2023-07-14T18:16:17.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.873+0000 7f26c77fe700 1 -- 172.21.15.133:0/2268298862 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26d000f040 con 0x7f26d8107af0 2023-07-14T18:16:17.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.873+0000 7f26dcbff700 1 -- 172.21.15.133:0/2268298862 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d819fe30 con 0x7f26d8107af0 2023-07-14T18:16:17.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.873+0000 7f26c77fe700 1 -- 172.21.15.133:0/2268298862 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d000cd00 con 0x7f26d8107af0 2023-07-14T18:16:17.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.873+0000 7f26dcbff700 1 -- 172.21.15.133:0/2268298862 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d81a0260 con 0x7f26d8107af0 2023-07-14T18:16:17.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.874+0000 7f26dcbff700 1 -- 172.21.15.133:0/2268298862 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d810d660 con 0x7f26d8107af0 2023-07-14T18:16:17.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.876+0000 7f26c77fe700 1 -- 172.21.15.133:0/2268298862 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f26d0003a00 con 0x7f26d8107af0 2023-07-14T18:16:17.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.877+0000 7f26c77fe700 1 --2- 172.21.15.133:0/2268298862 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f26c00a1220 0x7f26c00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:17.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.877+0000 7f26c77fe700 1 -- 172.21.15.133:0/2268298862 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f26d00cc970 con 0x7f26d8107af0 2023-07-14T18:16:17.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.878+0000 7f26d659c700 1 --2- 172.21.15.133:0/2268298862 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f26c00a1220 0x7f26c00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:17.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.878+0000 7f26d659c700 1 --2- 172.21.15.133:0/2268298862 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f26c00a1220 0x7f26c00a36d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f26c800b2e0 tx=0x7f26c8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:17.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:17.881+0000 7f26c77fe700 1 -- 172.21.15.133:0/2268298862 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f26d0095c60 con 0x7f26d8107af0 2023-07-14T18:16:18.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.113+0000 7f26c77fe700 1 -- 172.21.15.133:0/2268298862 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f26d0003cb0 con 0x7f26d8107af0 2023-07-14T18:16:18.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.134+0000 7f26dcbff700 1 -- 172.21.15.133:0/2268298862 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26d81a0870 con 0x7f26d8107af0 2023-07-14T18:16:18.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.135+0000 7f26c77fe700 1 -- 172.21.15.133:0/2268298862 <== mon.0 v2:172.21.15.133:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10851 (secure 0 0 0) 0x7f26d0081e60 con 0x7f26d8107af0 2023-07-14T18:16:18.180 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:18.181 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3168151271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3168151271},{"type":"v1","addr":"172.21.15.133:6837","nonce":3168151271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14544,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/3093144261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3093144261},{"type":"v1","addr":"172.21.15.133:6839","nonce":3093144261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14556,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/3282902068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":3282902068},{"type":"v1","addr":"172.21.15.133:6841","nonce":3282902068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/4023127259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4023127259},{"type":"v1","addr":"172.21.15.161:6833","nonce":4023127259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2722373740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2722373740},{"type":"v1","addr":"172.21.15.171:6833","nonce":2722373740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2399005074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2399005074},{"type":"v1","addr":"172.21.15.133:6835","nonce":2399005074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T18:14:17.440873+0000","modified":"2023-07-14T18:14:50.886364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24557,"mds_1":24541,"mds_2":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3725502564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3725502564},{"type":"v1","addr":"172.21.15.161:6835","nonce":3725502564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6835/1877362116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1877362116},{"type":"v1","addr":"172.21.15.171:6835","nonce":1877362116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/3458064798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3458064798},{"type":"v1","addr":"172.21.15.161:6837","nonce":3458064798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/106205332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":106205332},{"type":"v1","addr":"172.21.15.171:6837","nonce":106205332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.171:6839/816201198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":816201198},{"type":"v1","addr":"172.21.15.171:6839","nonce":816201198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/2503865871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2503865871},{"type":"v1","addr":"172.21.15.161:6839","nonce":2503865871}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T18:16:18.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.138+0000 7f26dcbff700 1 -- 172.21.15.133:0/2268298862 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f26c00a1220 msgr2=0x7f26c00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:18.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.138+0000 7f26dcbff700 1 --2- 172.21.15.133:0/2268298862 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f26c00a1220 0x7f26c00a36d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f26c800b2e0 tx=0x7f26c8005fb0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.138+0000 7f26dcbff700 1 -- 172.21.15.133:0/2268298862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26d8107af0 msgr2=0x7f26d819f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:18.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.138+0000 7f26dcbff700 1 --2- 172.21.15.133:0/2268298862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26d8107af0 0x7f26d819f310 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f26d000ef30 tx=0x7f26d000c3a0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.138+0000 7f26dcbff700 1 -- 172.21.15.133:0/2268298862 shutdown_connections 2023-07-14T18:16:18.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.138+0000 7f26dcbff700 1 --2- 172.21.15.133:0/2268298862 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f26c00a1220 0x7f26c00a36d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.138+0000 7f26dcbff700 1 --2- 172.21.15.133:0/2268298862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26d8107af0 0x7f26d819f310 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.138+0000 7f26dcbff700 1 --2- 172.21.15.133:0/2268298862 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26d80730a0 0x7f26d8197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.138+0000 7f26dcbff700 1 --2- 172.21.15.133:0/2268298862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26d8074680 0x7f26d81982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.138+0000 7f26dcbff700 1 -- 172.21.15.133:0/2268298862 >> 172.21.15.133:0/2268298862 conn(0x7f26d80fb210 msgr2=0x7f26d8109dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:18.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.138+0000 7f26dcbff700 1 -- 172.21.15.133:0/2268298862 shutdown_connections 2023-07-14T18:16:18.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.138+0000 7f26dcbff700 1 -- 172.21.15.133:0/2268298862 wait complete. 2023-07-14T18:16:18.185 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 25 2023-07-14T18:16:18.202 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T18:16:18.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.354+0000 7f26667ff700 1 -- 172.21.15.133:0/4054852297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26600ffc30 msgr2=0x7f2660100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:18.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.354+0000 7f26667ff700 1 --2- 172.21.15.133:0/4054852297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26600ffc30 0x7f2660100040 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2650009990 tx=0x7f2650009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.354+0000 7f26667ff700 1 -- 172.21.15.133:0/4054852297 shutdown_connections 2023-07-14T18:16:18.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.354+0000 7f26667ff700 1 --2- 172.21.15.133:0/4054852297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26601017b0 0x7f2660108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.354+0000 7f26667ff700 1 --2- 172.21.15.133:0/4054852297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26600ffc30 0x7f2660100040 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.354+0000 7f26667ff700 1 --2- 172.21.15.133:0/4054852297 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2660100e20 0x7f2660101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.354+0000 7f26667ff700 1 -- 172.21.15.133:0/4054852297 >> 172.21.15.133:0/4054852297 conn(0x7f26600fb210 msgr2=0x7f26600fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:18.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.355+0000 7f26667ff700 1 -- 172.21.15.133:0/4054852297 shutdown_connections 2023-07-14T18:16:18.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.355+0000 7f26667ff700 1 -- 172.21.15.133:0/4054852297 wait complete. 2023-07-14T18:16:18.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.355+0000 7f26667ff700 1 Processor -- start 2023-07-14T18:16:18.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.356+0000 7f26667ff700 1 -- start start 2023-07-14T18:16:18.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.356+0000 7f26667ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26600ffc30 0x7f2660197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:18.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.356+0000 7f26667ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2660100e20 0x7f26601980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:18.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.356+0000 7f26667ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26601017b0 0x7f266019f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:18.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.356+0000 7f26667ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266019f6e0 con 0x7f26601017b0 2023-07-14T18:16:18.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.356+0000 7f26667ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266019f820 con 0x7f26600ffc30 2023-07-14T18:16:18.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.356+0000 7f26667ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266019f9b0 con 0x7f2660100e20 2023-07-14T18:16:18.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.356+0000 7f2664d9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26601017b0 0x7f266019f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:18.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.356+0000 7f265f7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2660100e20 0x7f26601980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:18.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.356+0000 7f265f7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2660100e20 0x7f26601980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:55974/0 (socket says 172.21.15.133:55974) 2023-07-14T18:16:18.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.356+0000 7f2664d9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26601017b0 0x7f266019f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44284/0 (socket says 172.21.15.133:44284) 2023-07-14T18:16:18.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.356+0000 7f265f7fe700 1 -- 172.21.15.133:0/2230630586 learned_addr learned my addr 172.21.15.133:0/2230630586 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:18.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.356+0000 7f265ffff700 1 --2- 172.21.15.133:0/2230630586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26600ffc30 0x7f2660197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:18.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.357+0000 7f265f7fe700 1 -- 172.21.15.133:0/2230630586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26600ffc30 msgr2=0x7f2660197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:18.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.357+0000 7f265f7fe700 1 --2- 172.21.15.133:0/2230630586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26600ffc30 0x7f2660197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.357+0000 7f265f7fe700 1 -- 172.21.15.133:0/2230630586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26601017b0 msgr2=0x7f266019f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:18.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.357+0000 7f265f7fe700 1 --2- 172.21.15.133:0/2230630586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26601017b0 0x7f266019f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.357+0000 7f265f7fe700 1 -- 172.21.15.133:0/2230630586 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26500095f0 con 0x7f2660100e20 2023-07-14T18:16:18.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.357+0000 7f265ffff700 1 --2- 172.21.15.133:0/2230630586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26600ffc30 0x7f2660197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:16:18.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.357+0000 7f2664d9c700 1 --2- 172.21.15.133:0/2230630586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26601017b0 0x7f266019f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:16:18.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.357+0000 7f265f7fe700 1 --2- 172.21.15.133:0/2230630586 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2660100e20 0x7f26601980a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f265400e9a0 tx=0x7f265400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:18.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.357+0000 7f265d7fa700 1 -- 172.21.15.133:0/2230630586 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2654009790 con 0x7f2660100e20 2023-07-14T18:16:18.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.357+0000 7f26667ff700 1 -- 172.21.15.133:0/2230630586 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f266019fc90 con 0x7f2660100e20 2023-07-14T18:16:18.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.357+0000 7f265d7fa700 1 -- 172.21.15.133:0/2230630586 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f265400cbd0 con 0x7f2660100e20 2023-07-14T18:16:18.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.357+0000 7f265d7fa700 1 -- 172.21.15.133:0/2230630586 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2654018680 con 0x7f2660100e20 2023-07-14T18:16:18.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.357+0000 7f26667ff700 1 -- 172.21.15.133:0/2230630586 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26601a01a0 con 0x7f2660100e20 2023-07-14T18:16:18.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.358+0000 7f26667ff700 1 -- 172.21.15.133:0/2230630586 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f266010b450 con 0x7f2660100e20 2023-07-14T18:16:18.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.360+0000 7f265d7fa700 1 -- 172.21.15.133:0/2230630586 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f265400cd40 con 0x7f2660100e20 2023-07-14T18:16:18.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.361+0000 7f265d7fa700 1 --2- 172.21.15.133:0/2230630586 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f26480a11a0 0x7f26480a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:18.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.361+0000 7f265d7fa700 1 -- 172.21.15.133:0/2230630586 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f2654014070 con 0x7f2660100e20 2023-07-14T18:16:18.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.362+0000 7f265ffff700 1 --2- 172.21.15.133:0/2230630586 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f26480a11a0 0x7f26480a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:18.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:18 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2365655965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:16:18.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:18 smithi133 ceph-mon[126384]: pgmap v337: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2023-07-14T18:16:18.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:18 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/579931458' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:16:18.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.362+0000 7f265ffff700 1 --2- 172.21.15.133:0/2230630586 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f26480a11a0 0x7f26480a3650 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2650000c00 tx=0x7f2650005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:18.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.365+0000 7f265d7fa700 1 -- 172.21.15.133:0/2230630586 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f2654051780 con 0x7f2660100e20 2023-07-14T18:16:18.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:18 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2365655965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:16:18.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:18 smithi161 ceph-mon[128993]: pgmap v337: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2023-07-14T18:16:18.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:18 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/579931458' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:16:18.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.566+0000 7f26667ff700 1 -- 172.21.15.133:0/2230630586 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f266010b640 con 0x7f2660100e20 2023-07-14T18:16:18.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.568+0000 7f265d7fa700 1 -- 172.21.15.133:0/2230630586 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24520 (secure 0 0 0) 0x7f2654096300 con 0x7f2660100e20 2023-07-14T18:16:18.614 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:18.614 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":92,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","created":"2023-07-14T18:06:49.561582+0000","modified":"2023-07-14T18:15:49.951050+0000","last_up_change":"2023-07-14T18:13:29.836099+0000","last_in_change":"2023-07-14T18:13:11.606710+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T18:10:03.945039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T18:14:15.451340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T18:14:16.892215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T18:14:20.493175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"85a29de7-d712-422a-b647-d9e90091aaf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6803","nonce":3076510801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6805","nonce":3076510801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6809","nonce":3076510801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6807","nonce":3076510801}]},"public_addr":"172.21.15.133:6803/3076510801","cluster_addr":"172.21.15.133:6805/3076510801","heartbeat_back_addr":"172.21.15.133:6809/3076510801","heartbeat_front_addr":"172.21.15.133:6807/3076510801","state":["exists","up"]},{"osd":1,"uuid":"80ad4e78-7917-495e-b119-99f7c7c36243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6801","nonce":1841901852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6803","nonce":1841901852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6807","nonce":1841901852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6805","nonce":1841901852}]},"public_addr":"172.21.15.161:6801/1841901852","cluster_addr":"172.21.15.161:6803/1841901852","heartbeat_back_addr":"172.21.15.161:6807/1841901852","heartbeat_front_addr":"172.21.15.161:6805/1841901852","state":["exists","up"]},{"osd":2,"uuid":"fc4bfd37-1fdd-47c6-a48e-6d910daf9e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6801","nonce":346103099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6803","nonce":346103099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6807","nonce":346103099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6805","nonce":346103099}]},"public_addr":"172.21.15.171:6801/346103099","cluster_addr":"172.21.15.171:6803/346103099","heartbeat_back_addr":"172.21.15.171:6807/346103099","heartbeat_front_addr":"172.21.15.171:6805/346103099","state":["exists","up"]},{"osd":3,"uuid":"4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6811","nonce":2152994143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6813","nonce":2152994143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6817","nonce":2152994143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6815","nonce":2152994143}]},"public_addr":"172.21.15.133:6811/2152994143","cluster_addr":"172.21.15.133:6813/2152994143","heartbeat_back_addr":"172.21.15.133:6817/2152994143","heartbeat_front_addr":"172.21.15.133:6815/2152994143","state":["exists","up"]},{"osd":4,"uuid":"4601a1c2-254b-4c4d-9b15-8fc952b3d2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6809","nonce":47213291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6811","nonce":47213291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6815","nonce":47213291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6813","nonce":47213291}]},"public_addr":"172.21.15.161:6809/47213291","cluster_addr":"172.21.15.161:6811/47213291","heartbeat_back_addr":"172.21.15.161:6815/47213291","heartbeat_front_addr":"172.21.15.161:6813/47213291","state":["exists","up"]},{"osd":5,"uuid":"6cca180a-9140-4e49-835b-86842dae0574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6809","nonce":777556751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6811","nonce":777556751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6815","nonce":777556751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6813","nonce":777556751}]},"public_addr":"172.21.15.171:6809/777556751","cluster_addr":"172.21.15.171:6811/777556751","heartbeat_back_addr":"172.21.15.171:6815/777556751","heartbeat_front_addr":"172.21.15.171:6813/777556751","state":["exists","up"]},{"osd":6,"uuid":"ea0299ff-7740-4494-a3a9-1503b3fc0ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6819","nonce":2884152358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6821","nonce":2884152358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6825","nonce":2884152358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6823","nonce":2884152358}]},"public_addr":"172.21.15.133:6819/2884152358","cluster_addr":"172.21.15.133:6821/2884152358","heartbeat_back_addr":"172.21.15.133:6825/2884152358","heartbeat_front_addr":"172.21.15.133:6823/2884152358","state":["exists","up"]},{"osd":7,"uuid":"b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6817","nonce":4183705749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6819","nonce":4183705749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6823","nonce":4183705749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6821","nonce":4183705749}]},"public_addr":"172.21.15.161:6817/4183705749","cluster_addr":"172.21.15.161:6819/4183705749","heartbeat_back_addr":"172.21.15.161:6823/4183705749","heartbeat_front_addr":"172.21.15.161:6821/4183705749","state":["exists","up"]},{"osd":8,"uuid":"87dffc07-bf48-48c8-b418-5c4b509686d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6817","nonce":2771494483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6819","nonce":2771494483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6823","nonce":2771494483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6821","nonce":2771494483}]},"public_addr":"172.21.15.171:6817/2771494483","cluster_addr":"172.21.15.171:6819/2771494483","heartbeat_back_addr":"172.21.15.171:6823/2771494483","heartbeat_front_addr":"172.21.15.171:6821/2771494483","state":["exists","up"]},{"osd":9,"uuid":"e0832870-79ce-4bce-8045-58b7a7f6dc69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6827","nonce":2066164149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6829","nonce":2066164149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6833","nonce":2066164149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6831","nonce":2066164149}]},"public_addr":"172.21.15.133:6827/2066164149","cluster_addr":"172.21.15.133:6829/2066164149","heartbeat_back_addr":"172.21.15.133:6833/2066164149","heartbeat_front_addr":"172.21.15.133:6831/2066164149","state":["exists","up"]},{"osd":10,"uuid":"081f0484-d134-4b2e-9c48-ef8e6a646422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6825","nonce":2162333626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6827","nonce":2162333626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6831","nonce":2162333626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6829","nonce":2162333626}]},"public_addr":"172.21.15.161:6825/2162333626","cluster_addr":"172.21.15.161:6827/2162333626","heartbeat_back_addr":"172.21.15.161:6831/2162333626","heartbeat_front_addr":"172.21.15.161:6829/2162333626","state":["exists","up"]},{"osd":11,"uuid":"4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6825","nonce":3403894313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6827","nonce":3403894313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6831","nonce":3403894313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6829","nonce":3403894313}]},"public_addr":"172.21.15.171:6825/3403894313","cluster_addr":"172.21.15.171:6827/3403894313","heartbeat_back_addr":"172.21.15.171:6831/3403894313","heartbeat_front_addr":"172.21.15.171:6829/3403894313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:17.816229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:39.638543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:00.543317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:26.988921+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:48.644694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:10.049860+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:35.986508+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:57.753846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:19.018245+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:44.460450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:06.279960+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:27.955006+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/1832569241":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6801/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:0/4219815304":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:6801/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/432253615":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1840700480":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1807775439":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1089030734":"2023-07-15T18:07:47.346704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T18:16:18.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.571+0000 7f26667ff700 1 -- 172.21.15.133:0/2230630586 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f26480a11a0 msgr2=0x7f26480a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:18.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.571+0000 7f26667ff700 1 --2- 172.21.15.133:0/2230630586 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f26480a11a0 0x7f26480a3650 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2650000c00 tx=0x7f2650005fb0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.571+0000 7f26667ff700 1 -- 172.21.15.133:0/2230630586 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2660100e20 msgr2=0x7f26601980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:18.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.571+0000 7f26667ff700 1 --2- 172.21.15.133:0/2230630586 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2660100e20 0x7f26601980a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f265400e9a0 tx=0x7f265400ecb0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.571+0000 7f26667ff700 1 -- 172.21.15.133:0/2230630586 shutdown_connections 2023-07-14T18:16:18.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.571+0000 7f26667ff700 1 --2- 172.21.15.133:0/2230630586 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f26480a11a0 0x7f26480a3650 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.571+0000 7f26667ff700 1 --2- 172.21.15.133:0/2230630586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26600ffc30 0x7f2660197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.571+0000 7f26667ff700 1 --2- 172.21.15.133:0/2230630586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26601017b0 0x7f266019f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.571+0000 7f26667ff700 1 --2- 172.21.15.133:0/2230630586 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2660100e20 0x7f26601980a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.571+0000 7f26667ff700 1 -- 172.21.15.133:0/2230630586 >> 172.21.15.133:0/2230630586 conn(0x7f26600fb210 msgr2=0x7f26601066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:18.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.571+0000 7f26667ff700 1 -- 172.21.15.133:0/2230630586 shutdown_connections 2023-07-14T18:16:18.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.571+0000 7f26667ff700 1 -- 172.21.15.133:0/2230630586 wait complete. 2023-07-14T18:16:18.618 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:18 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2365655965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:16:18.618 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:18 smithi171 ceph-mon[129241]: pgmap v337: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2023-07-14T18:16:18.618 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:18 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/579931458' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:16:18.635 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-07-14T18:14:17.440873+0000', 'modified': '2023-07-14T18:14:50.886364+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24557, 'mds_1': 24541, 'mds_2': 24517}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24494': {'gid': 24494, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6835/3725502564', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 3725502564}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 3725502564}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24517': {'gid': 24517, 'name': 'f', 'rank': 2, 'incarnation': 23, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.171:6835/1877362116', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6834', 'nonce': 1877362116}, {'type': 'v1', 'addr': '172.21.15.171:6835', 'nonce': 1877362116}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24521': {'gid': 24521, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6837/3458064798', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 3458064798}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 3458064798}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24529': {'gid': 24529, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6837/106205332', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6836', 'nonce': 106205332}, {'type': 'v1', 'addr': '172.21.15.171:6837', 'nonce': 106205332}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24541': {'gid': 24541, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.171:6839/816201198', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6838', 'nonce': 816201198}, {'type': 'v1', 'addr': '172.21.15.171:6839', 'nonce': 816201198}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24557': {'gid': 24557, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6839/2503865871', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 2503865871}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 2503865871}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-14T18:16:18.635 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-07-14T18:16:18.635 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-07-14T18:16:18.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.787+0000 7fa1909ff700 1 -- 172.21.15.133:0/1918472169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa18c105950 msgr2=0x7fa18c107d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:18.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.787+0000 7fa1909ff700 1 --2- 172.21.15.133:0/1918472169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa18c105950 0x7fa18c107d30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa18400b0a0 tx=0x7fa18400b3b0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.787+0000 7fa1909ff700 1 -- 172.21.15.133:0/1918472169 shutdown_connections 2023-07-14T18:16:18.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.787+0000 7fa1909ff700 1 --2- 172.21.15.133:0/1918472169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa18c105950 0x7fa18c107d30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.787+0000 7fa1909ff700 1 --2- 172.21.15.133:0/1918472169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa18c100710 0x7fa18c102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.787+0000 7fa1909ff700 1 --2- 172.21.15.133:0/1918472169 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa18c103030 0x7fa18c105410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.787+0000 7fa1909ff700 1 -- 172.21.15.133:0/1918472169 >> 172.21.15.133:0/1918472169 conn(0x7fa18c0fa130 msgr2=0x7fa18c0fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:18.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.787+0000 7fa1909ff700 1 -- 172.21.15.133:0/1918472169 shutdown_connections 2023-07-14T18:16:18.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.787+0000 7fa1909ff700 1 -- 172.21.15.133:0/1918472169 wait complete. 2023-07-14T18:16:18.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.788+0000 7fa1909ff700 1 Processor -- start 2023-07-14T18:16:18.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.789+0000 7fa1909ff700 1 -- start start 2023-07-14T18:16:18.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.789+0000 7fa1909ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa18c100710 0x7fa18c195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:18.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.789+0000 7fa1909ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa18c103030 0x7fa18c195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:18.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.789+0000 7fa1909ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa18c105950 0x7fa18c19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:18.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.789+0000 7fa1909ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c19d5c0 con 0x7fa18c103030 2023-07-14T18:16:18.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.789+0000 7fa1909ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c19d700 con 0x7fa18c100710 2023-07-14T18:16:18.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.789+0000 7fa1909ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c19d890 con 0x7fa18c105950 2023-07-14T18:16:18.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.789+0000 7fa183fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa18c103030 0x7fa18c195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:18.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.789+0000 7fa183fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa18c103030 0x7fa18c195f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44310/0 (socket says 172.21.15.133:44310) 2023-07-14T18:16:18.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.789+0000 7fa183fff700 1 -- 172.21.15.133:0/2001369977 learned_addr learned my addr 172.21.15.133:0/2001369977 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:18.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.789+0000 7fa18a59c700 1 --2- 172.21.15.133:0/2001369977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa18c100710 0x7fa18c195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:18.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.789+0000 7fa18ad9d700 1 --2- 172.21.15.133:0/2001369977 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa18c105950 0x7fa18c19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:18.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.790+0000 7fa183fff700 1 -- 172.21.15.133:0/2001369977 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa18c105950 msgr2=0x7fa18c19cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:18.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.790+0000 7fa183fff700 1 --2- 172.21.15.133:0/2001369977 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa18c105950 0x7fa18c19cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.790+0000 7fa183fff700 1 -- 172.21.15.133:0/2001369977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa18c100710 msgr2=0x7fa18c195a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:18.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.790+0000 7fa183fff700 1 --2- 172.21.15.133:0/2001369977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa18c100710 0x7fa18c195a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.790+0000 7fa183fff700 1 -- 172.21.15.133:0/2001369977 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa184009d00 con 0x7fa18c103030 2023-07-14T18:16:18.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.790+0000 7fa183fff700 1 --2- 172.21.15.133:0/2001369977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa18c103030 0x7fa18c195f80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa17400e9a0 tx=0x7fa17400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:18.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.790+0000 7fa1837fe700 1 -- 172.21.15.133:0/2001369977 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174009790 con 0x7fa18c103030 2023-07-14T18:16:18.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.790+0000 7fa1837fe700 1 -- 172.21.15.133:0/2001369977 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa174010430 con 0x7fa18c103030 2023-07-14T18:16:18.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.790+0000 7fa1909ff700 1 -- 172.21.15.133:0/2001369977 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa18c19db70 con 0x7fa18c103030 2023-07-14T18:16:18.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.790+0000 7fa1837fe700 1 -- 172.21.15.133:0/2001369977 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174018680 con 0x7fa18c103030 2023-07-14T18:16:18.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.790+0000 7fa1909ff700 1 -- 172.21.15.133:0/2001369977 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa18c19e0d0 con 0x7fa18c103030 2023-07-14T18:16:18.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.793+0000 7fa1837fe700 1 -- 172.21.15.133:0/2001369977 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fa1740105a0 con 0x7fa18c103030 2023-07-14T18:16:18.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.795+0000 7fa1a4239b80 1 Processor -- start 2023-07-14T18:16:18.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.795+0000 7fa1837fe700 1 --2- 172.21.15.133:0/2001369977 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa1780a11d0 0x7fa1780a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:18.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.795+0000 7fa1837fe700 1 -- 172.21.15.133:0/2001369977 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa174014070 con 0x7fa18c103030 2023-07-14T18:16:18.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.795+0000 7fa1a4239b80 1 -- start start 2023-07-14T18:16:18.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.795+0000 7fa18a59c700 1 --2- 172.21.15.133:0/2001369977 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa1780a11d0 0x7fa1780a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:18.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.795+0000 7fa1a4239b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610e0637f10 0x5610e0638320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:18.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.795+0000 7fa1a4239b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610e0638860 0x5610e0638e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:18.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.795+0000 7fa1a4239b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0639460 0x5610e06398d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:18.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.795+0000 7fa1a4239b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e053cc60 con 0x5610e0639460 2023-07-14T18:16:18.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.795+0000 7fa1a4239b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e058f9f0 con 0x5610e0638860 2023-07-14T18:16:18.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.795+0000 7fa1a4239b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e0588e80 con 0x5610e0637f10 2023-07-14T18:16:18.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.795+0000 7fa18ad9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0639460 0x5610e06398d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:18.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.796+0000 7fa18ad9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0639460 0x5610e06398d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44324/0 (socket says 172.21.15.133:44324) 2023-07-14T18:16:18.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.796+0000 7fa18ad9d700 1 -- 172.21.15.133:0/3670613928 learned_addr learned my addr 172.21.15.133:0/3670613928 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:18.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.796+0000 7fa18a59c700 1 --2- 172.21.15.133:0/3670613928 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610e0637f10 0x5610e0638320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:18.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.796+0000 7fa183fff700 1 --2- 172.21.15.133:0/3670613928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610e0638860 0x5610e0638e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:18.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.796+0000 7fa18ad9d700 1 -- 172.21.15.133:0/3670613928 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610e0637f10 msgr2=0x5610e0638320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:18.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.796+0000 7fa18ad9d700 1 --2- 172.21.15.133:0/3670613928 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610e0637f10 0x5610e0638320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.796+0000 7fa18ad9d700 1 -- 172.21.15.133:0/3670613928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610e0638860 msgr2=0x5610e0638e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:18.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.796+0000 7fa18ad9d700 1 --2- 172.21.15.133:0/3670613928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610e0638860 0x5610e0638e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.796+0000 7fa18ad9d700 1 -- 172.21.15.133:0/3670613928 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610e03b6d90 con 0x5610e0639460 2023-07-14T18:16:18.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.796+0000 7fa18a59c700 1 --2- 172.21.15.133:0/2001369977 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa1780a11d0 0x7fa1780a3680 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa17c009c70 tx=0x7fa17c011720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:18.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.796+0000 7fa18ad9d700 1 --2- 172.21.15.133:0/3670613928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0639460 0x5610e06398d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa184009c40 tx=0x7fa184009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f9a05f4f3a42bb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:18.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.796+0000 7fa180ff9700 1 -- 172.21.15.133:0/3670613928 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18400e040 con 0x5610e0639460 2023-07-14T18:16:18.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.796+0000 7fa180ff9700 1 -- 172.21.15.133:0/3670613928 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa18400bab0 con 0x5610e0639460 2023-07-14T18:16:18.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.796+0000 7fa180ff9700 1 -- 172.21.15.133:0/3670613928 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18401c6a0 con 0x5610e0639460 2023-07-14T18:16:18.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.797+0000 7fa1a4239b80 1 -- 172.21.15.133:0/3670613928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0639460 msgr2=0x5610e06398d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:18.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.797+0000 7fa1a4239b80 1 --2- 172.21.15.133:0/3670613928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0639460 0x5610e06398d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa184009c40 tx=0x7fa184009c70 comp rx=0 tx=0).stop 2023-07-14T18:16:18.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.797+0000 7fa1a4239b80 1 -- 172.21.15.133:0/3670613928 shutdown_connections 2023-07-14T18:16:18.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.797+0000 7fa1a4239b80 1 --2- 172.21.15.133:0/3670613928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0639460 0x5610e06398d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.797+0000 7fa1a4239b80 1 --2- 172.21.15.133:0/3670613928 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610e0637f10 0x5610e0638320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.797+0000 7fa1a4239b80 1 --2- 172.21.15.133:0/3670613928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610e0638860 0x5610e0638e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.797+0000 7fa1a4239b80 1 -- 172.21.15.133:0/3670613928 >> 172.21.15.133:0/3670613928 conn(0x5610e062d630 msgr2=0x5610e0632980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:18.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.797+0000 7fa1a4239b80 1 -- 172.21.15.133:0/3670613928 shutdown_connections 2023-07-14T18:16:18.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.797+0000 7fa1a4239b80 1 -- 172.21.15.133:0/3670613928 wait complete. 2023-07-14T18:16:18.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.798+0000 7fa1a4239b80 1 -- start start 2023-07-14T18:16:18.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.798+0000 7fa1a4239b80 1 Processor -- start 2023-07-14T18:16:18.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.798+0000 7fa1a4239b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0637f10 0x5610e0645cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:18.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.798+0000 7fa1a4239b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610e0638860 0x5610e06461f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:18.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.798+0000 7fa1a4239b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610e0639460 0x5610e064d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:18.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.798+0000 7fa1a4239b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e0528a70 con 0x5610e0637f10 2023-07-14T18:16:18.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.798+0000 7fa1a4239b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e0528770 con 0x5610e0639460 2023-07-14T18:16:18.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.798+0000 7fa1a4239b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e040c8d0 con 0x5610e0638860 2023-07-14T18:16:18.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.798+0000 7fa1a4239b80 10 client.0 _ll_register_callbacks cb 0x5610e0526710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:16:18.852 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.799+0000 7fa18a59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0637f10 0x5610e0645cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:18.852 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.799+0000 7fa18ad9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610e0639460 0x5610e064d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:18.852 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.799+0000 7fa18ad9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610e0639460 0x5610e064d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:49900/0 (socket says 172.21.15.133:49900) 2023-07-14T18:16:18.852 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.799+0000 7fa18ad9d700 1 -- 172.21.15.133:0/639564351 learned_addr learned my addr 172.21.15.133:0/639564351 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:18.852 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.799+0000 7fa18a59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0637f10 0x5610e0645cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44330/0 (socket says 172.21.15.133:44330) 2023-07-14T18:16:18.852 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.799+0000 7fa183fff700 1 --2- 172.21.15.133:0/639564351 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610e0638860 0x5610e06461f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:18.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.799+0000 7fa18ad9d700 1 -- 172.21.15.133:0/639564351 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610e0638860 msgr2=0x5610e06461f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:18.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.799+0000 7fa18ad9d700 1 --2- 172.21.15.133:0/639564351 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610e0638860 0x5610e06461f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.799+0000 7fa18ad9d700 1 -- 172.21.15.133:0/639564351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0637f10 msgr2=0x5610e0645cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:18.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.799+0000 7fa18ad9d700 1 --2- 172.21.15.133:0/639564351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0637f10 0x5610e0645cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.799+0000 7fa18ad9d700 1 -- 172.21.15.133:0/639564351 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa18c108920 con 0x5610e0639460 2023-07-14T18:16:18.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.799+0000 7fa18a59c700 1 --2- 172.21.15.133:0/639564351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0637f10 0x5610e0645cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:16:18.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.799+0000 7fa18ad9d700 1 --2- 172.21.15.133:0/639564351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610e0639460 0x5610e064d230 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa184006230 tx=0x7fa184006260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5009546fe93c28 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:18.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.799+0000 7fa171ffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:16:18.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.800+0000 7fa171ffb700 1 -- 172.21.15.133:0/639564351 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18400e040 con 0x5610e0639460 2023-07-14T18:16:18.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.800+0000 7fa1a4239b80 1 -- 172.21.15.133:0/639564351 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5610e04e24a0 con 0x5610e0639460 2023-07-14T18:16:18.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.800+0000 7fa171ffb700 1 -- 172.21.15.133:0/639564351 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa184005070 con 0x5610e0639460 2023-07-14T18:16:18.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.800+0000 7fa171ffb700 1 -- 172.21.15.133:0/639564351 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa18402d670 con 0x5610e0639460 2023-07-14T18:16:18.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.800+0000 7fa1a4239b80 10 client.25283 fetch_fsmap learned FSMap version 25 2023-07-14T18:16:18.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.800+0000 7fa1a4239b80 1 -- 172.21.15.133:0/639564351 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5610e039fc10 con 0x5610e0639460 2023-07-14T18:16:18.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.800+0000 7fa171ffb700 1 -- 172.21.15.133:0/639564351 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5610e04e24a0 con 0x5610e0639460 2023-07-14T18:16:18.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.800+0000 7fa171ffb700 1 -- 172.21.15.133:0/639564351 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa184019030 con 0x5610e0639460 2023-07-14T18:16:18.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.800+0000 7fa1a4239b80 10 client.25283 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T18:16:18.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.800+0000 7fa1a4239b80 10 client.25283 resolve_mds: resolved name 'f' to daemon mds.f 2023-07-14T18:16:18.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.800+0000 7fa1a4239b80 20 client.25283 populate_metadata read hostname 'smithi133' 2023-07-14T18:16:18.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.801+0000 7fa1a4239b80 1 --2- 172.21.15.133:0/639564351 >> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] conn(0x5610e0650aa0 0x5610e0650160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:18.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.801+0000 7fa1a4239b80 4 client.25283 mds_command: new command op to 24517 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:16:18.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.801+0000 7fa1a4239b80 1 -- 172.21.15.133:0/639564351 --> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5610e0606d50 con 0x5610e0650aa0 2023-07-14T18:16:18.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.801+0000 7fa18a59c700 1 --2- 172.21.15.133:0/639564351 >> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] conn(0x5610e0650aa0 0x5610e0650160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:18.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.801+0000 7fa18a59c700 1 --2- 172.21.15.133:0/639564351 >> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] conn(0x5610e0650aa0 0x5610e0650160 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=463e25fa1b6736ea server_cookie=5774e9529b3209a7 in_seq=0 out_seq=0 2023-07-14T18:16:18.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.801+0000 7fa171ffb700 10 client.25283 ms_handle_connect on v2:172.21.15.171:6834/1877362116 2023-07-14T18:16:18.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.802+0000 7fa171ffb700 1 -- 172.21.15.133:0/639564351 <== mds.2 v2:172.21.15.171:6834/1877362116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5610e0606d50 con 0x5610e0650aa0 2023-07-14T18:16:18.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.802+0000 7fa171ffb700 10 client.25283 handle_command_reply: tid=0 2023-07-14T18:16:18.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.802+0000 7fa1a4239b80 1 client.25283 shutdown 2023-07-14T18:16:18.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.803+0000 7fa1a4239b80 1 -- 172.21.15.133:0/639564351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610e0639460 msgr2=0x5610e064d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:18.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.803+0000 7fa1a4239b80 1 --2- 172.21.15.133:0/639564351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610e0639460 0x5610e064d230 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa184006230 tx=0x7fa184006260 comp rx=0 tx=0).stop 2023-07-14T18:16:18.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.803+0000 7fa1a4239b80 1 -- 172.21.15.133:0/639564351 shutdown_connections 2023-07-14T18:16:18.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.803+0000 7fa1a4239b80 1 --2- 172.21.15.133:0/639564351 >> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] conn(0x5610e0650aa0 0x5610e0650160 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.803+0000 7fa1a4239b80 1 --2- 172.21.15.133:0/639564351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610e0639460 0x5610e064d230 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.803+0000 7fa1a4239b80 1 --2- 172.21.15.133:0/639564351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0637f10 0x5610e0645cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.803+0000 7fa171ffb700 0 client.25283 ms_handle_reset on v2:172.21.15.171:6834/1877362116 2023-07-14T18:16:18.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.803+0000 7fa1a4239b80 1 --2- 172.21.15.133:0/639564351 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610e0638860 0x5610e06461f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.803+0000 7fa1a4239b80 1 -- 172.21.15.133:0/639564351 >> 172.21.15.133:0/639564351 conn(0x5610e062d630 msgr2=0x5610e0631600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:18.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.803+0000 7fa1a4239b80 1 -- 172.21.15.133:0/639564351 shutdown_connections 2023-07-14T18:16:18.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.803+0000 7fa1a4239b80 1 -- 172.21.15.133:0/639564351 wait complete. 2023-07-14T18:16:18.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.803+0000 7fa1a4239b80 20 client.25283 trim_cache size 0 max 16384 2023-07-14T18:16:18.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.821+0000 7fa1a4239b80 1 Processor -- start 2023-07-14T18:16:18.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.821+0000 7fa1a4239b80 1 -- start start 2023-07-14T18:16:18.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.821+0000 7fa1a4239b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0637f10 0x5610e064da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:18.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.821+0000 7fa1a4239b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610e0638860 0x5610e064d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:18.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.821+0000 7fa1a4239b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610e0639460 0x5610e063f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:18.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.821+0000 7fa1a4239b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e04e24a0 con 0x5610e0637f10 2023-07-14T18:16:18.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.821+0000 7fa1a4239b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e0646ea0 con 0x5610e0638860 2023-07-14T18:16:18.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.821+0000 7fa1a4239b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e0557790 con 0x5610e0639460 2023-07-14T18:16:18.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.821+0000 7fa18a59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0637f10 0x5610e064da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:18.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.821+0000 7fa18a59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0637f10 0x5610e064da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44336/0 (socket says 172.21.15.133:44336) 2023-07-14T18:16:18.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.821+0000 7fa18a59c700 1 -- 172.21.15.133:0/456501492 learned_addr learned my addr 172.21.15.133:0/456501492 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:18.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.821+0000 7fa183fff700 1 --2- 172.21.15.133:0/456501492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610e0638860 0x5610e064d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:18.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.821+0000 7fa18ad9d700 1 --2- 172.21.15.133:0/456501492 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610e0639460 0x5610e063f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:18.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.822+0000 7fa18a59c700 1 -- 172.21.15.133:0/456501492 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610e0639460 msgr2=0x5610e063f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:18.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.822+0000 7fa18a59c700 1 --2- 172.21.15.133:0/456501492 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610e0639460 0x5610e063f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.822+0000 7fa18a59c700 1 -- 172.21.15.133:0/456501492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610e0638860 msgr2=0x5610e064d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:18.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.822+0000 7fa18a59c700 1 --2- 172.21.15.133:0/456501492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610e0638860 0x5610e064d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.822+0000 7fa18a59c700 1 -- 172.21.15.133:0/456501492 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610e03dd200 con 0x5610e0637f10 2023-07-14T18:16:18.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.822+0000 7fa18a59c700 1 --2- 172.21.15.133:0/456501492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0637f10 0x5610e064da60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x5610e0528b60 tx=0x7fa17c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31afe3215e4dbf4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:18.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.822+0000 7fa1817fa700 1 -- 172.21.15.133:0/456501492 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c023040 con 0x5610e0637f10 2023-07-14T18:16:18.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.822+0000 7fa1817fa700 1 -- 172.21.15.133:0/456501492 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa17c02b8c0 con 0x5610e0637f10 2023-07-14T18:16:18.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.822+0000 7fa1817fa700 1 -- 172.21.15.133:0/456501492 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c004a90 con 0x5610e0637f10 2023-07-14T18:16:18.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.823+0000 7fa1a4239b80 1 -- 172.21.15.133:0/456501492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0637f10 msgr2=0x5610e064da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:18.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.823+0000 7fa1a4239b80 1 --2- 172.21.15.133:0/456501492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0637f10 0x5610e064da60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x5610e0528b60 tx=0x7fa17c01c000 comp rx=0 tx=0).stop 2023-07-14T18:16:18.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.823+0000 7fa1a4239b80 1 -- 172.21.15.133:0/456501492 shutdown_connections 2023-07-14T18:16:18.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.823+0000 7fa1a4239b80 1 --2- 172.21.15.133:0/456501492 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610e0639460 0x5610e063f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.823+0000 7fa1a4239b80 1 --2- 172.21.15.133:0/456501492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0637f10 0x5610e064da60 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.823+0000 7fa1a4239b80 1 --2- 172.21.15.133:0/456501492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610e0638860 0x5610e064d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.823+0000 7fa1a4239b80 1 -- 172.21.15.133:0/456501492 >> 172.21.15.133:0/456501492 conn(0x5610e062d630 msgr2=0x5610e06320a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:18.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.823+0000 7fa1a4239b80 1 -- 172.21.15.133:0/456501492 shutdown_connections 2023-07-14T18:16:18.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.823+0000 7fa1a4239b80 1 -- 172.21.15.133:0/456501492 wait complete. 2023-07-14T18:16:18.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.823+0000 7fa1a4239b80 1 -- start start 2023-07-14T18:16:18.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.823+0000 7fa1a4239b80 1 Processor -- start 2023-07-14T18:16:18.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.824+0000 7fa1a4239b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610e0637f10 0x5610e0648810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:18.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.824+0000 7fa1a4239b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0638860 0x5610e0648d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:18.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.824+0000 7fa1a4239b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610e0639460 0x5610e064bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:18.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.824+0000 7fa1a4239b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e064c2b0 con 0x5610e0638860 2023-07-14T18:16:18.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.824+0000 7fa1a4239b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e064c420 con 0x5610e0637f10 2023-07-14T18:16:18.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.824+0000 7fa1a4239b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e064c5e0 con 0x5610e0639460 2023-07-14T18:16:18.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.824+0000 7fa1a4239b80 10 client.0 _ll_register_callbacks cb 0x7fa174005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:16:18.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.824+0000 7fa183fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0638860 0x5610e0648d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:18.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.824+0000 7fa18ad9d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610e0639460 0x5610e064bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:18.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.824+0000 7fa183fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0638860 0x5610e0648d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44338/0 (socket says 172.21.15.133:44338) 2023-07-14T18:16:18.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.824+0000 7fa183fff700 1 -- 172.21.15.133:0/2934963490 learned_addr learned my addr 172.21.15.133:0/2934963490 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:18.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.824+0000 7fa18a59c700 1 --2- 172.21.15.133:0/2934963490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610e0637f10 0x5610e0648810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:18.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.824+0000 7fa183fff700 1 -- 172.21.15.133:0/2934963490 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610e0639460 msgr2=0x5610e064bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:18.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.824+0000 7fa183fff700 1 --2- 172.21.15.133:0/2934963490 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610e0639460 0x5610e064bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.824+0000 7fa183fff700 1 -- 172.21.15.133:0/2934963490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610e0637f10 msgr2=0x5610e0648810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:18.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.824+0000 7fa183fff700 1 --2- 172.21.15.133:0/2934963490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610e0637f10 0x5610e0648810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.824+0000 7fa183fff700 1 -- 172.21.15.133:0/2934963490 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa17c01d570 con 0x5610e0638860 2023-07-14T18:16:18.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.824+0000 7fa18a59c700 1 --2- 172.21.15.133:0/2934963490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610e0637f10 0x5610e0648810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:16:18.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.825+0000 7fa183fff700 1 --2- 172.21.15.133:0/2934963490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0638860 0x5610e0648d50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa174005830 tx=0x7fa17401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6499f02fda9dbc0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:18.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.825+0000 7fa171ffb700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T18:16:18.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.825+0000 7fa171ffb700 1 -- 172.21.15.133:0/2934963490 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1740a22c0 con 0x5610e0638860 2023-07-14T18:16:18.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.825+0000 7fa1a4239b80 1 -- 172.21.15.133:0/2934963490 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa18400bab0 con 0x5610e0638860 2023-07-14T18:16:18.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.825+0000 7fa171ffb700 1 -- 172.21.15.133:0/2934963490 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa1740a1e30 con 0x5610e0638860 2023-07-14T18:16:18.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.825+0000 7fa171ffb700 1 -- 172.21.15.133:0/2934963490 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1740a1ff0 con 0x5610e0638860 2023-07-14T18:16:18.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.825+0000 7fa1a4239b80 10 client.15306 fetch_fsmap learned FSMap version 25 2023-07-14T18:16:18.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.825+0000 7fa1a4239b80 1 -- 172.21.15.133:0/2934963490 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5610e03b5260 con 0x5610e0638860 2023-07-14T18:16:18.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.825+0000 7fa171ffb700 1 -- 172.21.15.133:0/2934963490 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa17407c080 con 0x5610e0638860 2023-07-14T18:16:18.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.825+0000 7fa171ffb700 1 -- 172.21.15.133:0/2934963490 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa1740adc00 con 0x5610e0638860 2023-07-14T18:16:18.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.825+0000 7fa1a4239b80 10 client.15306 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T18:16:18.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.825+0000 7fa1a4239b80 10 client.15306 resolve_mds: resolved name 'f' to daemon mds.f 2023-07-14T18:16:18.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.825+0000 7fa1a4239b80 20 client.15306 populate_metadata read hostname 'smithi133' 2023-07-14T18:16:18.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.825+0000 7fa1a4239b80 1 --2- 172.21.15.133:0/2934963490 >> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] conn(0x5610e06aa390 0x5610e06a6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:18.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.826+0000 7fa1a4239b80 4 client.15306 mds_command: new command op to 24517 tid=0[{"prefix": "status"}] 2023-07-14T18:16:18.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.826+0000 7fa1a4239b80 1 -- 172.21.15.133:0/2934963490 --> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5610e0581e60 con 0x5610e06aa390 2023-07-14T18:16:18.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.826+0000 7fa18a59c700 1 --2- 172.21.15.133:0/2934963490 >> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] conn(0x5610e06aa390 0x5610e06a6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:18.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.826+0000 7fa171ffb700 10 client.15306 ms_handle_connect on v2:172.21.15.171:6834/1877362116 2023-07-14T18:16:18.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.826+0000 7fa18a59c700 1 --2- 172.21.15.133:0/2934963490 >> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] conn(0x5610e06aa390 0x5610e06a6ce0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=760c5e92f3fa50d3 server_cookie=ec5b3cd80285e304 in_seq=0 out_seq=0 2023-07-14T18:16:18.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.827+0000 7fa171ffb700 1 -- 172.21.15.133:0/2934963490 <== mds.2 v2:172.21.15.171:6834/1877362116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5610e0581e60 con 0x5610e06aa390 2023-07-14T18:16:18.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.827+0000 7fa171ffb700 10 client.15306 handle_command_reply: tid=0 2023-07-14T18:16:18.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.827+0000 7fa1a4239b80 1 client.15306 shutdown 2023-07-14T18:16:18.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.827+0000 7fa1a4239b80 1 -- 172.21.15.133:0/2934963490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0638860 msgr2=0x5610e0648d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:18.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.827+0000 7fa1a4239b80 1 --2- 172.21.15.133:0/2934963490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0638860 0x5610e0648d50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa174005830 tx=0x7fa17401f000 comp rx=0 tx=0).stop 2023-07-14T18:16:18.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.827+0000 7fa1a4239b80 1 -- 172.21.15.133:0/2934963490 shutdown_connections 2023-07-14T18:16:18.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.827+0000 7fa1a4239b80 1 --2- 172.21.15.133:0/2934963490 >> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] conn(0x5610e06aa390 0x5610e06a6ce0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.827+0000 7fa1a4239b80 1 --2- 172.21.15.133:0/2934963490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610e0637f10 0x5610e0648810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.827+0000 7fa1a4239b80 1 --2- 172.21.15.133:0/2934963490 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5610e0639460 0x5610e064bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.827+0000 7fa171ffb700 0 client.15306 ms_handle_reset on v2:172.21.15.171:6834/1877362116 2023-07-14T18:16:18.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.827+0000 7fa1a4239b80 1 --2- 172.21.15.133:0/2934963490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610e0638860 0x5610e0648d50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.827+0000 7fa1a4239b80 1 -- 172.21.15.133:0/2934963490 >> 172.21.15.133:0/2934963490 conn(0x5610e062d630 msgr2=0x5610e064da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:18.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.827+0000 7fa1a4239b80 1 -- 172.21.15.133:0/2934963490 shutdown_connections 2023-07-14T18:16:18.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.827+0000 7fa1a4239b80 1 -- 172.21.15.133:0/2934963490 wait complete. 2023-07-14T18:16:18.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.827+0000 7fa1a4239b80 20 client.15306 trim_cache size 0 max 16384 2023-07-14T18:16:18.883 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:16:18.883 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "06d0f5fa-2271-11ee-9b33-001a4aab830c", 2023-07-14T18:16:18.883 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 2, 2023-07-14T18:16:18.884 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24517, 2023-07-14T18:16:18.884 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2023-07-14T18:16:18.884 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2023-07-14T18:16:18.884 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2023-07-14T18:16:18.884 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 88.944457283999995, 2023-07-14T18:16:18.884 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 25, 2023-07-14T18:16:18.884 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 92, 2023-07-14T18:16:18.885 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 90, 2023-07-14T18:16:18.885 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 142.76608263400001 2023-07-14T18:16:18.885 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:16:18.886 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.828+0000 7fa1909ff700 1 -- 172.21.15.133:0/2001369977 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa1780a11d0 msgr2=0x7fa1780a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:18.886 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.828+0000 7fa1909ff700 1 --2- 172.21.15.133:0/2001369977 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa1780a11d0 0x7fa1780a3680 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa17c009c70 tx=0x7fa17c011720 comp rx=0 tx=0).stop 2023-07-14T18:16:18.886 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.828+0000 7fa1909ff700 1 -- 172.21.15.133:0/2001369977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa18c103030 msgr2=0x7fa18c195f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:18.886 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.828+0000 7fa1909ff700 1 --2- 172.21.15.133:0/2001369977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa18c103030 0x7fa18c195f80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa17400e9a0 tx=0x7fa17400ecb0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.886 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.828+0000 7fa1909ff700 1 -- 172.21.15.133:0/2001369977 shutdown_connections 2023-07-14T18:16:18.886 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.828+0000 7fa1909ff700 1 --2- 172.21.15.133:0/2001369977 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa1780a11d0 0x7fa1780a3680 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.886 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.828+0000 7fa1909ff700 1 --2- 172.21.15.133:0/2001369977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa18c100710 0x7fa18c195a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.828+0000 7fa1909ff700 1 --2- 172.21.15.133:0/2001369977 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa18c105950 0x7fa18c19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.828+0000 7fa1909ff700 1 --2- 172.21.15.133:0/2001369977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa18c103030 0x7fa18c195f80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:18.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.828+0000 7fa1909ff700 1 -- 172.21.15.133:0/2001369977 >> 172.21.15.133:0/2001369977 conn(0x7fa18c0fa130 msgr2=0x7fa18c101370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:18.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.828+0000 7fa1909ff700 1 -- 172.21.15.133:0/2001369977 shutdown_connections 2023-07-14T18:16:18.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:18.828+0000 7fa1909ff700 1 -- 172.21.15.133:0/2001369977 wait complete. 2023-07-14T18:16:18.891 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-07-14T18:16:19.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.044+0000 7fe03e5ff700 1 -- 172.21.15.133:0/3189122623 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe0381017a0 msgr2=0x7fe038108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:19.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.044+0000 7fe03e5ff700 1 --2- 172.21.15.133:0/3189122623 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe0381017a0 0x7fe038108060 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe030009f90 tx=0x7fe03000b330 comp rx=0 tx=0).stop 2023-07-14T18:16:19.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.044+0000 7fe03e5ff700 1 -- 172.21.15.133:0/3189122623 shutdown_connections 2023-07-14T18:16:19.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.044+0000 7fe03e5ff700 1 --2- 172.21.15.133:0/3189122623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0380ffc20 0x7fe038100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.044+0000 7fe03e5ff700 1 --2- 172.21.15.133:0/3189122623 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe0381017a0 0x7fe038108060 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.044+0000 7fe03e5ff700 1 --2- 172.21.15.133:0/3189122623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe038100e10 0x7fe038101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.044+0000 7fe03e5ff700 1 -- 172.21.15.133:0/3189122623 >> 172.21.15.133:0/3189122623 conn(0x7fe0380fb260 msgr2=0x7fe0380fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:19.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.044+0000 7fe03e5ff700 1 -- 172.21.15.133:0/3189122623 shutdown_connections 2023-07-14T18:16:19.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.044+0000 7fe03e5ff700 1 -- 172.21.15.133:0/3189122623 wait complete. 2023-07-14T18:16:19.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.045+0000 7fe03e5ff700 1 Processor -- start 2023-07-14T18:16:19.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.045+0000 7fe03e5ff700 1 -- start start 2023-07-14T18:16:19.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.045+0000 7fe03e5ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0380ffc20 0x7fe0380743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.046+0000 7fe03e5ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe038100e10 0x7fe038072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.046+0000 7fe03e5ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe0381017a0 0x7fe038072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.046+0000 7fe03e5ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe038073710 con 0x7fe0380ffc20 2023-07-14T18:16:19.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.046+0000 7fe03e5ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe038073880 con 0x7fe038100e10 2023-07-14T18:16:19.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.046+0000 7fe03e5ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03819f8f0 con 0x7fe0381017a0 2023-07-14T18:16:19.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.046+0000 7fe037fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0380ffc20 0x7fe0380743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.046+0000 7fe037fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0380ffc20 0x7fe0380743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44360/0 (socket says 172.21.15.133:44360) 2023-07-14T18:16:19.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.046+0000 7fe037fff700 1 -- 172.21.15.133:0/56998318 learned_addr learned my addr 172.21.15.133:0/56998318 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:19.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.046+0000 7fe0377fe700 1 --2- 172.21.15.133:0/56998318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe038100e10 0x7fe038072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.046+0000 7fe03cb9c700 1 --2- 172.21.15.133:0/56998318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe0381017a0 0x7fe038072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.046+0000 7fe037fff700 1 -- 172.21.15.133:0/56998318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe0381017a0 msgr2=0x7fe038072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:19.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.046+0000 7fe037fff700 1 --2- 172.21.15.133:0/56998318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe0381017a0 0x7fe038072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.046+0000 7fe037fff700 1 -- 172.21.15.133:0/56998318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe038100e10 msgr2=0x7fe038072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:19.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.046+0000 7fe037fff700 1 --2- 172.21.15.133:0/56998318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe038100e10 0x7fe038072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.046+0000 7fe037fff700 1 -- 172.21.15.133:0/56998318 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe030009c40 con 0x7fe0380ffc20 2023-07-14T18:16:19.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.046+0000 7fe037fff700 1 --2- 172.21.15.133:0/56998318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0380ffc20 0x7fe0380743f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe02800e970 tx=0x7fe02800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:19.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.047+0000 7fe0357fa700 1 -- 172.21.15.133:0/56998318 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe028009790 con 0x7fe0380ffc20 2023-07-14T18:16:19.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.047+0000 7fe03e5ff700 1 -- 172.21.15.133:0/56998318 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe03819fbd0 con 0x7fe0380ffc20 2023-07-14T18:16:19.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.047+0000 7fe0357fa700 1 -- 172.21.15.133:0/56998318 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe02800c7c0 con 0x7fe0380ffc20 2023-07-14T18:16:19.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.047+0000 7fe0357fa700 1 -- 172.21.15.133:0/56998318 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe028003c20 con 0x7fe0380ffc20 2023-07-14T18:16:19.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.047+0000 7fe03e5ff700 1 -- 172.21.15.133:0/56998318 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0381a0100 con 0x7fe0380ffc20 2023-07-14T18:16:19.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.050+0000 7fe0357fa700 1 -- 172.21.15.133:0/56998318 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe02800c930 con 0x7fe0380ffc20 2023-07-14T18:16:19.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.051+0000 7fe051f40b80 1 Processor -- start 2023-07-14T18:16:19.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.051+0000 7fe0357fa700 1 --2- 172.21.15.133:0/56998318 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe0200a11d0 0x7fe0200a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.051+0000 7fe051f40b80 1 -- start start 2023-07-14T18:16:19.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.051+0000 7fe0357fa700 1 -- 172.21.15.133:0/56998318 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe028014070 con 0x7fe0380ffc20 2023-07-14T18:16:19.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.051+0000 7fe03cb9c700 1 --2- 172.21.15.133:0/56998318 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe0200a11d0 0x7fe0200a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.051+0000 7fe051f40b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d74fc4f10 0x563d74fc5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.052+0000 7fe051f40b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d74fc5860 0x563d74fc5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.052+0000 7fe051f40b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d74fc6460 0x563d74fc68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.052+0000 7fe051f40b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d74ec9c60 con 0x563d74fc5860 2023-07-14T18:16:19.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.052+0000 7fe051f40b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d74f1c9f0 con 0x563d74fc6460 2023-07-14T18:16:19.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.052+0000 7fe051f40b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d74f15e80 con 0x563d74fc4f10 2023-07-14T18:16:19.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.052+0000 7fe037fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d74fc4f10 0x563d74fc5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.052+0000 7fe0377fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d74fc5860 0x563d74fc5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.052+0000 7fe037fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d74fc4f10 0x563d74fc5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:56048/0 (socket says 172.21.15.133:56048) 2023-07-14T18:16:19.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.052+0000 7fe037fff700 1 -- 172.21.15.133:0/3181581948 learned_addr learned my addr 172.21.15.133:0/3181581948 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:19.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.052+0000 7fe03cb9c700 1 --2- 172.21.15.133:0/3181581948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d74fc6460 0x563d74fc68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.052+0000 7fe037fff700 1 -- 172.21.15.133:0/3181581948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d74fc6460 msgr2=0x563d74fc68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:19.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.052+0000 7fe037fff700 1 --2- 172.21.15.133:0/3181581948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d74fc6460 0x563d74fc68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.052+0000 7fe037fff700 1 -- 172.21.15.133:0/3181581948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d74fc5860 msgr2=0x563d74fc5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:19.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.052+0000 7fe037fff700 1 --2- 172.21.15.133:0/3181581948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d74fc5860 0x563d74fc5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.052+0000 7fe037fff700 1 -- 172.21.15.133:0/3181581948 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d74d43d90 con 0x563d74fc4f10 2023-07-14T18:16:19.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.052+0000 7fe0377fe700 1 --2- 172.21.15.133:0/3181581948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d74fc5860 0x563d74fc5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:16:19.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.052+0000 7fe03cb9c700 1 --2- 172.21.15.133:0/56998318 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe0200a11d0 0x7fe0200a3680 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe0300079e0 tx=0x7fe030007580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:19.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.053+0000 7fe037fff700 1 --2- 172.21.15.133:0/3181581948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d74fc4f10 0x563d74fc5320 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe0280a1d80 tx=0x7fe028099e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ba50bd812c6ab19 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:19.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.053+0000 7fe01e7fc700 1 -- 172.21.15.133:0/3181581948 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0280a22b0 con 0x563d74fc4f10 2023-07-14T18:16:19.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.053+0000 7fe01e7fc700 1 -- 172.21.15.133:0/3181581948 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe028098200 con 0x563d74fc4f10 2023-07-14T18:16:19.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.053+0000 7fe01e7fc700 1 -- 172.21.15.133:0/3181581948 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe028099350 con 0x563d74fc4f10 2023-07-14T18:16:19.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.053+0000 7fe051f40b80 1 -- 172.21.15.133:0/3181581948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d74fc4f10 msgr2=0x563d74fc5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:19.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.053+0000 7fe051f40b80 1 --2- 172.21.15.133:0/3181581948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d74fc4f10 0x563d74fc5320 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe0280a1d80 tx=0x7fe028099e60 comp rx=0 tx=0).stop 2023-07-14T18:16:19.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.054+0000 7fe051f40b80 1 -- 172.21.15.133:0/3181581948 shutdown_connections 2023-07-14T18:16:19.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.054+0000 7fe051f40b80 1 --2- 172.21.15.133:0/3181581948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d74fc6460 0x563d74fc68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.054+0000 7fe051f40b80 1 --2- 172.21.15.133:0/3181581948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d74fc4f10 0x563d74fc5320 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.054+0000 7fe051f40b80 1 --2- 172.21.15.133:0/3181581948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d74fc5860 0x563d74fc5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.054+0000 7fe051f40b80 1 -- 172.21.15.133:0/3181581948 >> 172.21.15.133:0/3181581948 conn(0x563d74fba630 msgr2=0x563d74fbf980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:19.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.054+0000 7fe051f40b80 1 -- 172.21.15.133:0/3181581948 shutdown_connections 2023-07-14T18:16:19.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.054+0000 7fe051f40b80 1 -- 172.21.15.133:0/3181581948 wait complete. 2023-07-14T18:16:19.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.054+0000 7fe051f40b80 1 -- start start 2023-07-14T18:16:19.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.054+0000 7fe051f40b80 1 Processor -- start 2023-07-14T18:16:19.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.055+0000 7fe051f40b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d74fc4f10 0x563d74fce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.055+0000 7fe051f40b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d74fc5860 0x563d74fcee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.055+0000 7fe051f40b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d74fc6460 0x563d74fd5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.055+0000 7fe051f40b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d74eb5a70 con 0x563d74fc5860 2023-07-14T18:16:19.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.055+0000 7fe051f40b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d74eb5770 con 0x563d74fc6460 2023-07-14T18:16:19.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.055+0000 7fe051f40b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d74d998d0 con 0x563d74fc4f10 2023-07-14T18:16:19.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.055+0000 7fe051f40b80 10 client.0 _ll_register_callbacks cb 0x563d74eb3710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:16:19.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.055+0000 7fe0377fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d74fc5860 0x563d74fcee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.055+0000 7fe037fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d74fc4f10 0x563d74fce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.055+0000 7fe037fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d74fc4f10 0x563d74fce8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:56056/0 (socket says 172.21.15.133:56056) 2023-07-14T18:16:19.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.055+0000 7fe037fff700 1 -- 172.21.15.133:0/708598610 learned_addr learned my addr 172.21.15.133:0/708598610 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:19.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.055+0000 7fe03cb9c700 1 --2- 172.21.15.133:0/708598610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d74fc6460 0x563d74fd5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.055+0000 7fe037fff700 1 -- 172.21.15.133:0/708598610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d74fc6460 msgr2=0x563d74fd5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:19.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.055+0000 7fe037fff700 1 --2- 172.21.15.133:0/708598610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d74fc6460 0x563d74fd5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.055+0000 7fe037fff700 1 -- 172.21.15.133:0/708598610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d74fc5860 msgr2=0x563d74fcee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:19.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.055+0000 7fe037fff700 1 --2- 172.21.15.133:0/708598610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d74fc5860 0x563d74fcee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.055+0000 7fe037fff700 1 -- 172.21.15.133:0/708598610 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe02c009920 con 0x563d74fc4f10 2023-07-14T18:16:19.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.055+0000 7fe0377fe700 1 --2- 172.21.15.133:0/708598610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d74fc5860 0x563d74fcee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:16:19.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.056+0000 7fe037fff700 1 --2- 172.21.15.133:0/708598610 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d74fc4f10 0x563d74fce8d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe028096360 tx=0x7fe02800cca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9ba31fed76b70f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:19.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.056+0000 7fe0177fe700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:16:19.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.056+0000 7fe0177fe700 1 -- 172.21.15.133:0/708598610 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0280ca270 con 0x563d74fc4f10 2023-07-14T18:16:19.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.056+0000 7fe051f40b80 1 -- 172.21.15.133:0/708598610 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d74e6f4a0 con 0x563d74fc4f10 2023-07-14T18:16:19.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.056+0000 7fe0177fe700 1 -- 172.21.15.133:0/708598610 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe028099350 con 0x563d74fc4f10 2023-07-14T18:16:19.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.056+0000 7fe0177fe700 1 -- 172.21.15.133:0/708598610 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe028099510 con 0x563d74fc4f10 2023-07-14T18:16:19.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.056+0000 7fe051f40b80 10 client.25336 fetch_fsmap learned FSMap version 25 2023-07-14T18:16:19.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.056+0000 7fe051f40b80 1 -- 172.21.15.133:0/708598610 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fe0381a0100 con 0x563d74fc4f10 2023-07-14T18:16:19.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.056+0000 7fe0177fe700 1 -- 172.21.15.133:0/708598610 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563d74e6f4a0 con 0x563d74fc4f10 2023-07-14T18:16:19.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.056+0000 7fe0177fe700 1 -- 172.21.15.133:0/708598610 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe028083120 con 0x563d74fc4f10 2023-07-14T18:16:19.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.057+0000 7fe051f40b80 10 client.25336 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T18:16:19.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.057+0000 7fe051f40b80 10 client.25336 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-14T18:16:19.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.057+0000 7fe051f40b80 20 client.25336 populate_metadata read hostname 'smithi133' 2023-07-14T18:16:19.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.057+0000 7fe051f40b80 1 --2- 172.21.15.133:0/708598610 >> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] conn(0x563d74fd9d90 0x563d74fd8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.057+0000 7fe051f40b80 4 client.25336 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:16:19.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.057+0000 7fe051f40b80 1 -- 172.21.15.133:0/708598610 --> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d74f93d50 con 0x563d74fd9d90 2023-07-14T18:16:19.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.057+0000 7fe0377fe700 1 --2- 172.21.15.133:0/708598610 >> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] conn(0x563d74fd9d90 0x563d74fd8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.057+0000 7fe0377fe700 1 --2- 172.21.15.133:0/708598610 >> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] conn(0x563d74fd9d90 0x563d74fd8d80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b59271f8ecaa246a server_cookie=49f5f0e63b5ea70 in_seq=0 out_seq=0 2023-07-14T18:16:19.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.057+0000 7fe0177fe700 10 client.25336 ms_handle_connect on v2:172.21.15.171:6838/816201198 2023-07-14T18:16:19.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.058+0000 7fe0177fe700 1 -- 172.21.15.133:0/708598610 <== mds.1 v2:172.21.15.171:6838/816201198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x563d74f93d50 con 0x563d74fd9d90 2023-07-14T18:16:19.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.058+0000 7fe0177fe700 10 client.25336 handle_command_reply: tid=0 2023-07-14T18:16:19.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.058+0000 7fe051f40b80 1 client.25336 shutdown 2023-07-14T18:16:19.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.058+0000 7fe051f40b80 1 -- 172.21.15.133:0/708598610 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d74fc4f10 msgr2=0x563d74fce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:19.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.058+0000 7fe051f40b80 1 --2- 172.21.15.133:0/708598610 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d74fc4f10 0x563d74fce8d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe028096360 tx=0x7fe02800cca0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.058+0000 7fe051f40b80 1 -- 172.21.15.133:0/708598610 shutdown_connections 2023-07-14T18:16:19.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.058+0000 7fe051f40b80 1 --2- 172.21.15.133:0/708598610 >> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] conn(0x563d74fd9d90 0x563d74fd8d80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.058+0000 7fe051f40b80 1 --2- 172.21.15.133:0/708598610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d74fc6460 0x563d74fd5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.058+0000 7fe0177fe700 0 client.25336 ms_handle_reset on v2:172.21.15.171:6838/816201198 2023-07-14T18:16:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.058+0000 7fe051f40b80 1 --2- 172.21.15.133:0/708598610 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d74fc4f10 0x563d74fce8d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.058+0000 7fe051f40b80 1 --2- 172.21.15.133:0/708598610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d74fc5860 0x563d74fcee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.058+0000 7fe051f40b80 1 -- 172.21.15.133:0/708598610 >> 172.21.15.133:0/708598610 conn(0x563d74fba630 msgr2=0x563d74fbe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.059+0000 7fe051f40b80 1 -- 172.21.15.133:0/708598610 shutdown_connections 2023-07-14T18:16:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.059+0000 7fe051f40b80 1 -- 172.21.15.133:0/708598610 wait complete. 2023-07-14T18:16:19.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.059+0000 7fe051f40b80 20 client.25336 trim_cache size 0 max 16384 2023-07-14T18:16:19.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.076+0000 7fe051f40b80 1 Processor -- start 2023-07-14T18:16:19.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.076+0000 7fe051f40b80 1 -- start start 2023-07-14T18:16:19.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.077+0000 7fe051f40b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d74fc4f10 0x563d74fd8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.077+0000 7fe051f40b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d74fc5860 0x563d74fd5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.077+0000 7fe051f40b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d74fc6460 0x563d74fcc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.077+0000 7fe051f40b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d74ebf470 con 0x563d74fc4f10 2023-07-14T18:16:19.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.077+0000 7fe051f40b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d74ee4790 con 0x563d74fc5860 2023-07-14T18:16:19.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.077+0000 7fe051f40b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d74ed4480 con 0x563d74fc6460 2023-07-14T18:16:19.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.077+0000 7fe037fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d74fc4f10 0x563d74fd8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.077+0000 7fe037fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d74fc4f10 0x563d74fd8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44366/0 (socket says 172.21.15.133:44366) 2023-07-14T18:16:19.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.077+0000 7fe037fff700 1 -- 172.21.15.133:0/1084032674 learned_addr learned my addr 172.21.15.133:0/1084032674 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:19.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.077+0000 7fe03cb9c700 1 --2- 172.21.15.133:0/1084032674 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d74fc6460 0x563d74fcc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.077+0000 7fe0377fe700 1 --2- 172.21.15.133:0/1084032674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d74fc5860 0x563d74fd5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.077+0000 7fe037fff700 1 -- 172.21.15.133:0/1084032674 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d74fc6460 msgr2=0x563d74fcc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:19.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.077+0000 7fe037fff700 1 --2- 172.21.15.133:0/1084032674 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d74fc6460 0x563d74fcc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.077+0000 7fe037fff700 1 -- 172.21.15.133:0/1084032674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d74fc5860 msgr2=0x563d74fd5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:19.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.077+0000 7fe037fff700 1 --2- 172.21.15.133:0/1084032674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d74fc5860 0x563d74fd5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.078+0000 7fe037fff700 1 -- 172.21.15.133:0/1084032674 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d74d2cc10 con 0x563d74fc4f10 2023-07-14T18:16:19.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.078+0000 7fe03cb9c700 1 --2- 172.21.15.133:0/1084032674 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d74fc6460 0x563d74fcc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:16:19.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.078+0000 7fe037fff700 1 --2- 172.21.15.133:0/1084032674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d74fc4f10 0x563d74fd8d80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe02800c5b0 tx=0x7fe02809a0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f753ba425687ae8e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:19.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.078+0000 7fe01effd700 1 -- 172.21.15.133:0/1084032674 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe028004870 con 0x563d74fc4f10 2023-07-14T18:16:19.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.078+0000 7fe01effd700 1 -- 172.21.15.133:0/1084032674 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0280a22b0 con 0x563d74fc4f10 2023-07-14T18:16:19.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.078+0000 7fe01effd700 1 -- 172.21.15.133:0/1084032674 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0280995b0 con 0x563d74fc4f10 2023-07-14T18:16:19.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.078+0000 7fe051f40b80 1 -- 172.21.15.133:0/1084032674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d74fc4f10 msgr2=0x563d74fd8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:19.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.078+0000 7fe051f40b80 1 --2- 172.21.15.133:0/1084032674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d74fc4f10 0x563d74fd8d80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe02800c5b0 tx=0x7fe02809a0c0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.079+0000 7fe051f40b80 1 -- 172.21.15.133:0/1084032674 shutdown_connections 2023-07-14T18:16:19.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.079+0000 7fe051f40b80 1 --2- 172.21.15.133:0/1084032674 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d74fc6460 0x563d74fcc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.079+0000 7fe051f40b80 1 --2- 172.21.15.133:0/1084032674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d74fc4f10 0x563d74fd8d80 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.079+0000 7fe051f40b80 1 --2- 172.21.15.133:0/1084032674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d74fc5860 0x563d74fd5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.079+0000 7fe051f40b80 1 -- 172.21.15.133:0/1084032674 >> 172.21.15.133:0/1084032674 conn(0x563d74fba630 msgr2=0x563d74fbbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:19.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.079+0000 7fe051f40b80 1 -- 172.21.15.133:0/1084032674 shutdown_connections 2023-07-14T18:16:19.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.079+0000 7fe051f40b80 1 -- 172.21.15.133:0/1084032674 wait complete. 2023-07-14T18:16:19.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.079+0000 7fe051f40b80 1 -- start start 2023-07-14T18:16:19.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.079+0000 7fe051f40b80 1 Processor -- start 2023-07-14T18:16:19.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.080+0000 7fe051f40b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d74fc4f10 0x563d74fbf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.080+0000 7fe051f40b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d74fc5860 0x563d74fd5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.080+0000 7fe051f40b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d74fc6460 0x563d74fbf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.080+0000 7fe051f40b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d7502d400 con 0x563d74fc4f10 2023-07-14T18:16:19.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.080+0000 7fe051f40b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d7502d540 con 0x563d74fc5860 2023-07-14T18:16:19.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.080+0000 7fe051f40b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d7502d6b0 con 0x563d74fc6460 2023-07-14T18:16:19.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.080+0000 7fe051f40b80 10 client.0 _ll_register_callbacks cb 0x7fe02c009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:16:19.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.080+0000 7fe037fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d74fc4f10 0x563d74fbf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.080+0000 7fe0377fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d74fc5860 0x563d74fd5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.080+0000 7fe0377fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d74fc5860 0x563d74fd5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:49978/0 (socket says 172.21.15.133:49978) 2023-07-14T18:16:19.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.080+0000 7fe0377fe700 1 -- 172.21.15.133:0/2085548727 learned_addr learned my addr 172.21.15.133:0/2085548727 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.080+0000 7fe03cb9c700 1 --2- 172.21.15.133:0/2085548727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d74fc6460 0x563d74fbf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.080+0000 7fe0377fe700 1 -- 172.21.15.133:0/2085548727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d74fc6460 msgr2=0x563d74fbf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.080+0000 7fe0377fe700 1 --2- 172.21.15.133:0/2085548727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d74fc6460 0x563d74fbf920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.080+0000 7fe0377fe700 1 -- 172.21.15.133:0/2085548727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d74fc4f10 msgr2=0x563d74fbf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.080+0000 7fe0377fe700 1 --2- 172.21.15.133:0/2085548727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d74fc4f10 0x563d74fbf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.080+0000 7fe0377fe700 1 -- 172.21.15.133:0/2085548727 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe030026050 con 0x563d74fc5860 2023-07-14T18:16:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.081+0000 7fe0377fe700 1 --2- 172.21.15.133:0/2085548727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d74fc5860 0x563d74fd5e10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x563d74fc5c70 tx=0x7fe02c021d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1020376525db893 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.081+0000 7fe0177fe700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:16:19.139 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.081+0000 7fe0177fe700 1 -- 172.21.15.133:0/2085548727 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02c017040 con 0x563d74fc5860 2023-07-14T18:16:19.139 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.081+0000 7fe051f40b80 1 -- 172.21.15.133:0/2085548727 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe028098200 con 0x563d74fc5860 2023-07-14T18:16:19.139 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.081+0000 7fe0177fe700 1 -- 172.21.15.133:0/2085548727 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe02c012d30 con 0x563d74fc5860 2023-07-14T18:16:19.139 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.081+0000 7fe0177fe700 1 -- 172.21.15.133:0/2085548727 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe02c0122e0 con 0x563d74fc5860 2023-07-14T18:16:19.139 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.081+0000 7fe051f40b80 10 client.25307 fetch_fsmap learned FSMap version 25 2023-07-14T18:16:19.139 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.081+0000 7fe051f40b80 1 -- 172.21.15.133:0/2085548727 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fe0381a0100 con 0x563d74fc5860 2023-07-14T18:16:19.139 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.081+0000 7fe0177fe700 1 -- 172.21.15.133:0/2085548727 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe028098200 con 0x563d74fc5860 2023-07-14T18:16:19.139 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.082+0000 7fe0177fe700 1 -- 172.21.15.133:0/2085548727 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe02c010a10 con 0x563d74fc5860 2023-07-14T18:16:19.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.082+0000 7fe051f40b80 10 client.25307 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T18:16:19.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.082+0000 7fe051f40b80 10 client.25307 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-14T18:16:19.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.082+0000 7fe051f40b80 20 client.25307 populate_metadata read hostname 'smithi133' 2023-07-14T18:16:19.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.082+0000 7fe051f40b80 1 --2- 172.21.15.133:0/2085548727 >> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] conn(0x563d75032490 0x563d7502eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.082+0000 7fe051f40b80 4 client.25307 mds_command: new command op to 24541 tid=0[{"prefix": "status"}] 2023-07-14T18:16:19.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.082+0000 7fe051f40b80 1 -- 172.21.15.133:0/2085548727 --> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563d74f0ee60 con 0x563d75032490 2023-07-14T18:16:19.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.082+0000 7fe037fff700 1 --2- 172.21.15.133:0/2085548727 >> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] conn(0x563d75032490 0x563d7502eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.083+0000 7fe0177fe700 10 client.25307 ms_handle_connect on v2:172.21.15.171:6838/816201198 2023-07-14T18:16:19.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.083+0000 7fe037fff700 1 --2- 172.21.15.133:0/2085548727 >> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] conn(0x563d75032490 0x563d7502eea0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2864ea6dd12278e2 server_cookie=65d0faaa5eaf23d2 in_seq=0 out_seq=0 2023-07-14T18:16:19.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.083+0000 7fe0177fe700 1 -- 172.21.15.133:0/2085548727 <== mds.1 v2:172.21.15.171:6838/816201198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x563d74f0ee60 con 0x563d75032490 2023-07-14T18:16:19.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.083+0000 7fe0177fe700 10 client.25307 handle_command_reply: tid=0 2023-07-14T18:16:19.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.083+0000 7fe051f40b80 1 client.25307 shutdown 2023-07-14T18:16:19.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.083+0000 7fe051f40b80 1 -- 172.21.15.133:0/2085548727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d74fc5860 msgr2=0x563d74fd5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:19.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.083+0000 7fe051f40b80 1 --2- 172.21.15.133:0/2085548727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d74fc5860 0x563d74fd5e10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x563d74fc5c70 tx=0x7fe02c021d80 comp rx=0 tx=0).stop 2023-07-14T18:16:19.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.083+0000 7fe051f40b80 1 -- 172.21.15.133:0/2085548727 shutdown_connections 2023-07-14T18:16:19.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.083+0000 7fe051f40b80 1 --2- 172.21.15.133:0/2085548727 >> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] conn(0x563d75032490 0x563d7502eea0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.083+0000 7fe051f40b80 1 --2- 172.21.15.133:0/2085548727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d74fc6460 0x563d74fbf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.083+0000 7fe0177fe700 0 client.25307 ms_handle_reset on v2:172.21.15.171:6838/816201198 2023-07-14T18:16:19.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.083+0000 7fe051f40b80 1 --2- 172.21.15.133:0/2085548727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d74fc4f10 0x563d74fbf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.083+0000 7fe051f40b80 1 --2- 172.21.15.133:0/2085548727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d74fc5860 0x563d74fd5e10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.083+0000 7fe051f40b80 1 -- 172.21.15.133:0/2085548727 >> 172.21.15.133:0/2085548727 conn(0x563d74fba630 msgr2=0x563d74fd8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:19.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.083+0000 7fe051f40b80 1 -- 172.21.15.133:0/2085548727 shutdown_connections 2023-07-14T18:16:19.143 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.083+0000 7fe051f40b80 1 -- 172.21.15.133:0/2085548727 wait complete. 2023-07-14T18:16:19.143 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.083+0000 7fe051f40b80 20 client.25307 trim_cache size 0 max 16384 2023-07-14T18:16:19.143 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:16:19.143 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "06d0f5fa-2271-11ee-9b33-001a4aab830c", 2023-07-14T18:16:19.143 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 1, 2023-07-14T18:16:19.143 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24541, 2023-07-14T18:16:19.144 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2023-07-14T18:16:19.144 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2023-07-14T18:16:19.144 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2023-07-14T18:16:19.144 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 91.135614118999996, 2023-07-14T18:16:19.144 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 25, 2023-07-14T18:16:19.144 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 92, 2023-07-14T18:16:19.144 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 90, 2023-07-14T18:16:19.145 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 136.51402783699999 2023-07-14T18:16:19.145 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:16:19.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.084+0000 7fe03e5ff700 1 -- 172.21.15.133:0/56998318 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe0200a11d0 msgr2=0x7fe0200a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:19.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.084+0000 7fe03e5ff700 1 --2- 172.21.15.133:0/56998318 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe0200a11d0 0x7fe0200a3680 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe0300079e0 tx=0x7fe030007580 comp rx=0 tx=0).stop 2023-07-14T18:16:19.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.084+0000 7fe03e5ff700 1 -- 172.21.15.133:0/56998318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0380ffc20 msgr2=0x7fe0380743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:19.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.084+0000 7fe03e5ff700 1 --2- 172.21.15.133:0/56998318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0380ffc20 0x7fe0380743f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe02800e970 tx=0x7fe02800ec80 comp rx=0 tx=0).stop 2023-07-14T18:16:19.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.084+0000 7fe03e5ff700 1 -- 172.21.15.133:0/56998318 shutdown_connections 2023-07-14T18:16:19.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.084+0000 7fe03e5ff700 1 --2- 172.21.15.133:0/56998318 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe0200a11d0 0x7fe0200a3680 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.084+0000 7fe03e5ff700 1 --2- 172.21.15.133:0/56998318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe0381017a0 0x7fe038072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.084+0000 7fe03e5ff700 1 --2- 172.21.15.133:0/56998318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0380ffc20 0x7fe0380743f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.084+0000 7fe03e5ff700 1 --2- 172.21.15.133:0/56998318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe038100e10 0x7fe038072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.084+0000 7fe03e5ff700 1 -- 172.21.15.133:0/56998318 >> 172.21.15.133:0/56998318 conn(0x7fe0380fb260 msgr2=0x7fe0381042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.084+0000 7fe03e5ff700 1 -- 172.21.15.133:0/56998318 shutdown_connections 2023-07-14T18:16:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.084+0000 7fe03e5ff700 1 -- 172.21.15.133:0/56998318 wait complete. 2023-07-14T18:16:19.148 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-14T18:16:19.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.300+0000 7fac719ff700 1 -- 172.21.15.133:0/2314157441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac6c0ffc20 msgr2=0x7fac6c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:19.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.300+0000 7fac719ff700 1 --2- 172.21.15.133:0/2314157441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac6c0ffc20 0x7fac6c100030 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fac5c009990 tx=0x7fac5c009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.300+0000 7fac719ff700 1 -- 172.21.15.133:0/2314157441 shutdown_connections 2023-07-14T18:16:19.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.300+0000 7fac719ff700 1 --2- 172.21.15.133:0/2314157441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac6c1017a0 0x7fac6c108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.300+0000 7fac719ff700 1 --2- 172.21.15.133:0/2314157441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac6c0ffc20 0x7fac6c100030 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.300+0000 7fac719ff700 1 --2- 172.21.15.133:0/2314157441 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac6c100e10 0x7fac6c101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.300+0000 7fac719ff700 1 -- 172.21.15.133:0/2314157441 >> 172.21.15.133:0/2314157441 conn(0x7fac6c0fb260 msgr2=0x7fac6c0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:19.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.300+0000 7fac719ff700 1 -- 172.21.15.133:0/2314157441 shutdown_connections 2023-07-14T18:16:19.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.300+0000 7fac719ff700 1 -- 172.21.15.133:0/2314157441 wait complete. 2023-07-14T18:16:19.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.301+0000 7fac719ff700 1 Processor -- start 2023-07-14T18:16:19.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.301+0000 7fac719ff700 1 -- start start 2023-07-14T18:16:19.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.302+0000 7fac719ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac6c0ffc20 0x7fac6c0743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.302+0000 7fac719ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac6c100e10 0x7fac6c072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.302+0000 7fac719ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac6c1017a0 0x7fac6c072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.302+0000 7fac719ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c073710 con 0x7fac6c1017a0 2023-07-14T18:16:19.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.302+0000 7fac719ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c073880 con 0x7fac6c100e10 2023-07-14T18:16:19.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.302+0000 7fac719ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c19fa10 con 0x7fac6c0ffc20 2023-07-14T18:16:19.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.302+0000 7fac6b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac6c1017a0 0x7fac6c072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.302+0000 7fac6affd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac6c0ffc20 0x7fac6c0743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.302+0000 7fac6affd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac6c0ffc20 0x7fac6c0743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:56096/0 (socket says 172.21.15.133:56096) 2023-07-14T18:16:19.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.302+0000 7fac6b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac6c1017a0 0x7fac6c072f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44392/0 (socket says 172.21.15.133:44392) 2023-07-14T18:16:19.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.302+0000 7fac6affd700 1 -- 172.21.15.133:0/3837287912 learned_addr learned my addr 172.21.15.133:0/3837287912 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:19.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.302+0000 7fac6a7fc700 1 --2- 172.21.15.133:0/3837287912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac6c100e10 0x7fac6c072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.303+0000 7fac6affd700 1 -- 172.21.15.133:0/3837287912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac6c100e10 msgr2=0x7fac6c072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:19.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.303+0000 7fac6affd700 1 --2- 172.21.15.133:0/3837287912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac6c100e10 0x7fac6c072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.303+0000 7fac6affd700 1 -- 172.21.15.133:0/3837287912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac6c1017a0 msgr2=0x7fac6c072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:19.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.303+0000 7fac6affd700 1 --2- 172.21.15.133:0/3837287912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac6c1017a0 0x7fac6c072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.303+0000 7fac6affd700 1 -- 172.21.15.133:0/3837287912 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac5c0095f0 con 0x7fac6c0ffc20 2023-07-14T18:16:19.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.303+0000 7fac6b7fe700 1 --2- 172.21.15.133:0/3837287912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac6c1017a0 0x7fac6c072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:16:19.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.303+0000 7fac6affd700 1 --2- 172.21.15.133:0/3837287912 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac6c0ffc20 0x7fac6c0743f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fac5c00b620 tx=0x7fac5c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:19.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.303+0000 7fac709fd700 1 -- 172.21.15.133:0/3837287912 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5c01d070 con 0x7fac6c0ffc20 2023-07-14T18:16:19.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.303+0000 7fac719ff700 1 -- 172.21.15.133:0/3837287912 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac6c19fc90 con 0x7fac6c0ffc20 2023-07-14T18:16:19.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.303+0000 7fac709fd700 1 -- 172.21.15.133:0/3837287912 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac5c021410 con 0x7fac6c0ffc20 2023-07-14T18:16:19.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.303+0000 7fac709fd700 1 -- 172.21.15.133:0/3837287912 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5c017650 con 0x7fac6c0ffc20 2023-07-14T18:16:19.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.303+0000 7fac719ff700 1 -- 172.21.15.133:0/3837287912 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac6c1a0110 con 0x7fac6c0ffc20 2023-07-14T18:16:19.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.307+0000 7fac709fd700 1 -- 172.21.15.133:0/3837287912 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fac5c021580 con 0x7fac6c0ffc20 2023-07-14T18:16:19.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.307+0000 7fac85331b80 1 Processor -- start 2023-07-14T18:16:19.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.308+0000 7fac85331b80 1 -- start start 2023-07-14T18:16:19.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.308+0000 7fac85331b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4842f10 0x55afd4843320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.308+0000 7fac85331b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afd4843860 0x55afd4843e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.308+0000 7fac85331b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afd4844460 0x55afd48448d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.354 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.308+0000 7fac85331b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd4747c60 con 0x55afd4842f10 2023-07-14T18:16:19.354 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.308+0000 7fac85331b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd479a9f0 con 0x55afd4844460 2023-07-14T18:16:19.354 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.308+0000 7fac85331b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd4793e80 con 0x55afd4843860 2023-07-14T18:16:19.355 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.308+0000 7fac6b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4842f10 0x55afd4843320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.355 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.308+0000 7fac6b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4842f10 0x55afd4843320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44402/0 (socket says 172.21.15.133:44402) 2023-07-14T18:16:19.356 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.308+0000 7fac6b7fe700 1 -- 172.21.15.133:0/2149606782 learned_addr learned my addr 172.21.15.133:0/2149606782 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:19.356 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.308+0000 7fac6affd700 1 --2- 172.21.15.133:0/2149606782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afd4843860 0x55afd4843e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.356 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.308+0000 7fac6a7fc700 1 --2- 172.21.15.133:0/2149606782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afd4844460 0x55afd48448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.356 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.309+0000 7fac6b7fe700 1 -- 172.21.15.133:0/2149606782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afd4843860 msgr2=0x55afd4843e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:19.356 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.309+0000 7fac6b7fe700 1 --2- 172.21.15.133:0/2149606782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afd4843860 0x55afd4843e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.356 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.309+0000 7fac6b7fe700 1 -- 172.21.15.133:0/2149606782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afd4844460 msgr2=0x55afd48448d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:19.356 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.309+0000 7fac6b7fe700 1 --2- 172.21.15.133:0/2149606782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afd4844460 0x55afd48448d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.309+0000 7fac6b7fe700 1 -- 172.21.15.133:0/2149606782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afd45c1d90 con 0x55afd4842f10 2023-07-14T18:16:19.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.309+0000 7fac709fd700 1 --2- 172.21.15.133:0/3837287912 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fac540a1150 0x7fac540a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.309+0000 7fac6a7fc700 1 --2- 172.21.15.133:0/2149606782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afd4844460 0x55afd48448d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:16:19.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.309+0000 7fac709fd700 1 -- 172.21.15.133:0/3837287912 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fac5c0ce140 con 0x7fac6c0ffc20 2023-07-14T18:16:19.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.309+0000 7fac6b7fe700 1 --2- 172.21.15.133:0/3837287912 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fac540a1150 0x7fac540a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.309+0000 7fac6b7fe700 1 --2- 172.21.15.133:0/2149606782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4842f10 0x55afd4843320 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fac640076f0 tx=0x7fac64007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7970a805408d26e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:19.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.309+0000 7fac51ffb700 1 -- 172.21.15.133:0/2149606782 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac64012930 con 0x55afd4842f10 2023-07-14T18:16:19.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.309+0000 7fac51ffb700 1 -- 172.21.15.133:0/2149606782 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac6400f040 con 0x55afd4842f10 2023-07-14T18:16:19.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.310+0000 7fac51ffb700 1 -- 172.21.15.133:0/2149606782 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac64020610 con 0x55afd4842f10 2023-07-14T18:16:19.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.310+0000 7fac85331b80 1 -- 172.21.15.133:0/2149606782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4842f10 msgr2=0x55afd4843320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:19.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.310+0000 7fac6b7fe700 1 --2- 172.21.15.133:0/3837287912 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fac540a1150 0x7fac540a3600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fac6401f000 tx=0x7fac64020d10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:19.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.310+0000 7fac85331b80 1 --2- 172.21.15.133:0/2149606782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4842f10 0x55afd4843320 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fac640076f0 tx=0x7fac64007a00 comp rx=0 tx=0).stop 2023-07-14T18:16:19.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.310+0000 7fac85331b80 1 -- 172.21.15.133:0/2149606782 shutdown_connections 2023-07-14T18:16:19.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.310+0000 7fac85331b80 1 --2- 172.21.15.133:0/2149606782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afd4844460 0x55afd48448d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.310+0000 7fac85331b80 1 --2- 172.21.15.133:0/2149606782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4842f10 0x55afd4843320 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.310+0000 7fac85331b80 1 --2- 172.21.15.133:0/2149606782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afd4843860 0x55afd4843e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.310+0000 7fac85331b80 1 -- 172.21.15.133:0/2149606782 >> 172.21.15.133:0/2149606782 conn(0x55afd4838630 msgr2=0x55afd483d980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:19.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.310+0000 7fac85331b80 1 -- 172.21.15.133:0/2149606782 shutdown_connections 2023-07-14T18:16:19.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.310+0000 7fac85331b80 1 -- 172.21.15.133:0/2149606782 wait complete. 2023-07-14T18:16:19.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.311+0000 7fac85331b80 1 -- start start 2023-07-14T18:16:19.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.311+0000 7fac85331b80 1 Processor -- start 2023-07-14T18:16:19.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.312+0000 7fac85331b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4842f10 0x55afd484c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.312+0000 7fac85331b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afd4843860 0x55afd484cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.312+0000 7fac85331b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afd4844460 0x55afd4853e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.312+0000 7fac85331b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd4733a70 con 0x55afd4842f10 2023-07-14T18:16:19.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.312+0000 7fac85331b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd4733770 con 0x55afd4843860 2023-07-14T18:16:19.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.312+0000 7fac85331b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd46178d0 con 0x55afd4844460 2023-07-14T18:16:19.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.312+0000 7fac85331b80 10 client.0 _ll_register_callbacks cb 0x55afd4731710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:16:19.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.312+0000 7fac6affd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4842f10 0x55afd484c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.312+0000 7fac6affd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4842f10 0x55afd484c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44404/0 (socket says 172.21.15.133:44404) 2023-07-14T18:16:19.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.312+0000 7fac6affd700 1 -- 172.21.15.133:0/3804608131 learned_addr learned my addr 172.21.15.133:0/3804608131 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:19.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.312+0000 7fac6a7fc700 1 --2- 172.21.15.133:0/3804608131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afd4843860 0x55afd484cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.312+0000 7fac6b7fe700 1 --2- 172.21.15.133:0/3804608131 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afd4844460 0x55afd4853e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.313+0000 7fac6affd700 1 -- 172.21.15.133:0/3804608131 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afd4844460 msgr2=0x55afd4853e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:19.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.313+0000 7fac6affd700 1 --2- 172.21.15.133:0/3804608131 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afd4844460 0x55afd4853e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.313+0000 7fac6affd700 1 -- 172.21.15.133:0/3804608131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afd4843860 msgr2=0x55afd484cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:19.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.313+0000 7fac6affd700 1 --2- 172.21.15.133:0/3804608131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afd4843860 0x55afd484cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.313+0000 7fac6affd700 1 -- 172.21.15.133:0/3804608131 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac64009e20 con 0x55afd4842f10 2023-07-14T18:16:19.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.313+0000 7fac6b7fe700 1 --2- 172.21.15.133:0/3804608131 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afd4844460 0x55afd4853e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:16:19.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.313+0000 7fac6a7fc700 1 --2- 172.21.15.133:0/3804608131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afd4843860 0x55afd484cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:16:19.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.313+0000 7fac6affd700 1 --2- 172.21.15.133:0/3804608131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4842f10 0x55afd484c8a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fac5c0cb910 tx=0x7fac5c021b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422a50d7c79aef42 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:19.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.313+0000 7fac4affd700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T18:16:19.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.313+0000 7fac4affd700 1 -- 172.21.15.133:0/3804608131 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5c09adf0 con 0x55afd4842f10 2023-07-14T18:16:19.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.313+0000 7fac85331b80 1 -- 172.21.15.133:0/3804608131 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55afd46ed4a0 con 0x55afd4842f10 2023-07-14T18:16:19.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.313+0000 7fac4affd700 1 -- 172.21.15.133:0/3804608131 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fac5c09b2b0 con 0x55afd4842f10 2023-07-14T18:16:19.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.313+0000 7fac4affd700 1 -- 172.21.15.133:0/3804608131 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac5c09ac10 con 0x55afd4842f10 2023-07-14T18:16:19.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.313+0000 7fac85331b80 10 client.15336 fetch_fsmap learned FSMap version 25 2023-07-14T18:16:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.313+0000 7fac85331b80 1 -- 172.21.15.133:0/3804608131 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55afd45aac10 con 0x55afd4842f10 2023-07-14T18:16:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.313+0000 7fac4affd700 1 -- 172.21.15.133:0/3804608131 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55afd46ed4a0 con 0x55afd4842f10 2023-07-14T18:16:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.314+0000 7fac4affd700 1 -- 172.21.15.133:0/3804608131 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fac5c09b510 con 0x55afd4842f10 2023-07-14T18:16:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.314+0000 7fac85331b80 10 client.15336 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T18:16:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.314+0000 7fac85331b80 10 client.15336 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-14T18:16:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.314+0000 7fac85331b80 20 client.15336 populate_metadata read hostname 'smithi133' 2023-07-14T18:16:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.314+0000 7fac85331b80 1 --2- 172.21.15.133:0/3804608131 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55afd4857700 0x55afd4856d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.314+0000 7fac85331b80 4 client.15336 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:16:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.314+0000 7fac85331b80 1 -- 172.21.15.133:0/3804608131 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55afd4811d50 con 0x55afd4857700 2023-07-14T18:16:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.314+0000 7fac6a7fc700 1 --2- 172.21.15.133:0/3804608131 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55afd4857700 0x55afd4856d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.315+0000 7fac4affd700 10 client.15336 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:16:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.315+0000 7fac6a7fc700 1 --2- 172.21.15.133:0/3804608131 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55afd4857700 0x55afd4856d50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37edb0f14c4176fa server_cookie=fe4bde095490e23a in_seq=0 out_seq=0 2023-07-14T18:16:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.315+0000 7fac4affd700 1 -- 172.21.15.133:0/3804608131 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55afd4811d50 con 0x55afd4857700 2023-07-14T18:16:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.315+0000 7fac4affd700 10 client.15336 handle_command_reply: tid=0 2023-07-14T18:16:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.316+0000 7fac85331b80 1 client.15336 shutdown 2023-07-14T18:16:19.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.316+0000 7fac85331b80 1 -- 172.21.15.133:0/3804608131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4842f10 msgr2=0x55afd484c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:19.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.316+0000 7fac85331b80 1 --2- 172.21.15.133:0/3804608131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4842f10 0x55afd484c8a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fac5c0cb910 tx=0x7fac5c021b40 comp rx=0 tx=0).stop 2023-07-14T18:16:19.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.316+0000 7fac85331b80 1 -- 172.21.15.133:0/3804608131 shutdown_connections 2023-07-14T18:16:19.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.316+0000 7fac85331b80 1 --2- 172.21.15.133:0/3804608131 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55afd4857700 0x55afd4856d50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.316+0000 7fac85331b80 1 --2- 172.21.15.133:0/3804608131 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afd4844460 0x55afd4853e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.316+0000 7fac4affd700 0 client.15336 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:16:19.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.316+0000 7fac85331b80 1 --2- 172.21.15.133:0/3804608131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4842f10 0x55afd484c8a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.316+0000 7fac85331b80 1 --2- 172.21.15.133:0/3804608131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afd4843860 0x55afd484cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.316+0000 7fac85331b80 1 -- 172.21.15.133:0/3804608131 >> 172.21.15.133:0/3804608131 conn(0x55afd4838630 msgr2=0x55afd483c600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:19.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.316+0000 7fac85331b80 1 -- 172.21.15.133:0/3804608131 shutdown_connections 2023-07-14T18:16:19.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.316+0000 7fac85331b80 1 -- 172.21.15.133:0/3804608131 wait complete. 2023-07-14T18:16:19.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.316+0000 7fac85331b80 20 client.15336 trim_cache size 0 max 16384 2023-07-14T18:16:19.381 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.334+0000 7fac85331b80 1 Processor -- start 2023-07-14T18:16:19.381 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.334+0000 7fac85331b80 1 -- start start 2023-07-14T18:16:19.381 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.334+0000 7fac85331b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afd4842f10 0x55afd484b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.381 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.335+0000 7fac85331b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afd4843860 0x55afd4853de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.381 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.335+0000 7fac85331b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4844460 0x55afd484a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.381 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.335+0000 7fac85331b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd4811d50 con 0x55afd4844460 2023-07-14T18:16:19.381 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.335+0000 7fac85331b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd46ed4a0 con 0x55afd4843860 2023-07-14T18:16:19.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.335+0000 7fac85331b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd484da90 con 0x55afd4842f10 2023-07-14T18:16:19.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.335+0000 7fac6b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4844460 0x55afd484a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.335+0000 7fac6b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4844460 0x55afd484a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44410/0 (socket says 172.21.15.133:44410) 2023-07-14T18:16:19.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.335+0000 7fac6b7fe700 1 -- 172.21.15.133:0/1347925869 learned_addr learned my addr 172.21.15.133:0/1347925869 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:19.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.335+0000 7fac6a7fc700 1 --2- 172.21.15.133:0/1347925869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afd4843860 0x55afd4853de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.335+0000 7fac6affd700 1 --2- 172.21.15.133:0/1347925869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afd4842f10 0x55afd484b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.335+0000 7fac6b7fe700 1 -- 172.21.15.133:0/1347925869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afd4842f10 msgr2=0x55afd484b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:19.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.335+0000 7fac6b7fe700 1 --2- 172.21.15.133:0/1347925869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afd4842f10 0x55afd484b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.335+0000 7fac6b7fe700 1 -- 172.21.15.133:0/1347925869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afd4843860 msgr2=0x55afd4853de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:19.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.335+0000 7fac6b7fe700 1 --2- 172.21.15.133:0/1347925869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afd4843860 0x55afd4853de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.335+0000 7fac6b7fe700 1 -- 172.21.15.133:0/1347925869 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afd45e8200 con 0x55afd4844460 2023-07-14T18:16:19.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.335+0000 7fac6b7fe700 1 --2- 172.21.15.133:0/1347925869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4844460 0x55afd484a7f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fac64014680 tx=0x7fac64023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c547fdd75db7bfff server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:19.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.336+0000 7fac527fc700 1 -- 172.21.15.133:0/1347925869 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac64015cc0 con 0x55afd4844460 2023-07-14T18:16:19.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.336+0000 7fac527fc700 1 -- 172.21.15.133:0/1347925869 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac64025070 con 0x55afd4844460 2023-07-14T18:16:19.384 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.336+0000 7fac527fc700 1 -- 172.21.15.133:0/1347925869 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac64015680 con 0x55afd4844460 2023-07-14T18:16:19.387 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.336+0000 7fac85331b80 1 -- 172.21.15.133:0/1347925869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4844460 msgr2=0x55afd484a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:19.387 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.336+0000 7fac85331b80 1 --2- 172.21.15.133:0/1347925869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4844460 0x55afd484a7f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fac64014680 tx=0x7fac64023000 comp rx=0 tx=0).stop 2023-07-14T18:16:19.387 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.336+0000 7fac85331b80 1 -- 172.21.15.133:0/1347925869 shutdown_connections 2023-07-14T18:16:19.388 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.336+0000 7fac85331b80 1 --2- 172.21.15.133:0/1347925869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4844460 0x55afd484a7f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.388 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.336+0000 7fac85331b80 1 --2- 172.21.15.133:0/1347925869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afd4842f10 0x55afd484b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.388 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.336+0000 7fac85331b80 1 --2- 172.21.15.133:0/1347925869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afd4843860 0x55afd4853de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.388 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.336+0000 7fac85331b80 1 -- 172.21.15.133:0/1347925869 >> 172.21.15.133:0/1347925869 conn(0x55afd4838630 msgr2=0x55afd4856d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:19.388 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.336+0000 7fac85331b80 1 -- 172.21.15.133:0/1347925869 shutdown_connections 2023-07-14T18:16:19.388 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.336+0000 7fac85331b80 1 -- 172.21.15.133:0/1347925869 wait complete. 2023-07-14T18:16:19.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.337+0000 7fac85331b80 1 -- start start 2023-07-14T18:16:19.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.337+0000 7fac85331b80 1 Processor -- start 2023-07-14T18:16:19.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.337+0000 7fac85331b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4842f10 0x55afd484f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.337+0000 7fac85331b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afd4843860 0x55afd484f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.337+0000 7fac85331b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afd4844460 0x55afd4852720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.337+0000 7fac85331b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd4852c60 con 0x55afd4842f10 2023-07-14T18:16:19.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.338+0000 7fac85331b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd4852dd0 con 0x55afd4843860 2023-07-14T18:16:19.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.338+0000 7fac85331b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd4852f90 con 0x55afd4844460 2023-07-14T18:16:19.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.338+0000 7fac85331b80 10 client.0 _ll_register_callbacks cb 0x7fac5c005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:16:19.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.338+0000 7fac6affd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4842f10 0x55afd484f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.338+0000 7fac6affd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4842f10 0x55afd484f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44414/0 (socket says 172.21.15.133:44414) 2023-07-14T18:16:19.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.338+0000 7fac6affd700 1 -- 172.21.15.133:0/1686117905 learned_addr learned my addr 172.21.15.133:0/1686117905 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:19.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.338+0000 7fac6b7fe700 1 --2- 172.21.15.133:0/1686117905 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afd4844460 0x55afd4852720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.338+0000 7fac6a7fc700 1 --2- 172.21.15.133:0/1686117905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afd4843860 0x55afd484f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.338+0000 7fac6affd700 1 -- 172.21.15.133:0/1686117905 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afd4844460 msgr2=0x55afd4852720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:19.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.338+0000 7fac6affd700 1 --2- 172.21.15.133:0/1686117905 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afd4844460 0x55afd4852720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.338+0000 7fac6affd700 1 -- 172.21.15.133:0/1686117905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afd4843860 msgr2=0x55afd484f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:19.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.338+0000 7fac6affd700 1 --2- 172.21.15.133:0/1686117905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afd4843860 0x55afd484f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.338+0000 7fac6affd700 1 -- 172.21.15.133:0/1686117905 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac6401a040 con 0x55afd4842f10 2023-07-14T18:16:19.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.338+0000 7fac6a7fc700 1 --2- 172.21.15.133:0/1686117905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afd4843860 0x55afd484f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:16:19.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.338+0000 7fac6affd700 1 --2- 172.21.15.133:0/1686117905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4842f10 0x55afd484f1f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fac5c097d20 tx=0x7fac5c0cb960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2293b09d96be2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:19.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.338+0000 7fac4affd700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T18:16:19.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.339+0000 7fac4affd700 1 -- 172.21.15.133:0/1686117905 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5c0a33e0 con 0x55afd4842f10 2023-07-14T18:16:19.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.339+0000 7fac4affd700 1 -- 172.21.15.133:0/1686117905 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fac5c09b2b0 con 0x55afd4842f10 2023-07-14T18:16:19.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.339+0000 7fac85331b80 1 -- 172.21.15.133:0/1686117905 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac6400f040 con 0x55afd4842f10 2023-07-14T18:16:19.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.339+0000 7fac4affd700 1 -- 172.21.15.133:0/1686117905 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac5c09adf0 con 0x55afd4842f10 2023-07-14T18:16:19.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.339+0000 7fac85331b80 10 client.15342 fetch_fsmap learned FSMap version 25 2023-07-14T18:16:19.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.339+0000 7fac85331b80 1 -- 172.21.15.133:0/1686117905 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55afd45c0260 con 0x55afd4842f10 2023-07-14T18:16:19.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.339+0000 7fac4affd700 1 -- 172.21.15.133:0/1686117905 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac6400f040 con 0x55afd4842f10 2023-07-14T18:16:19.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.339+0000 7fac4affd700 1 -- 172.21.15.133:0/1686117905 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fac5c0970a0 con 0x55afd4842f10 2023-07-14T18:16:19.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.339+0000 7fac85331b80 10 client.15342 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T18:16:19.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.339+0000 7fac85331b80 10 client.15342 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-14T18:16:19.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.339+0000 7fac85331b80 20 client.15342 populate_metadata read hostname 'smithi133' 2023-07-14T18:16:19.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.339+0000 7fac85331b80 1 --2- 172.21.15.133:0/1686117905 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55afd48b0620 0x55afd48ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:19.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.339+0000 7fac85331b80 4 client.15342 mds_command: new command op to 24557 tid=0[{"prefix": "status"}] 2023-07-14T18:16:19.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.339+0000 7fac85331b80 1 -- 172.21.15.133:0/1686117905 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55afd478ce60 con 0x55afd48b0620 2023-07-14T18:16:19.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.340+0000 7fac6a7fc700 1 --2- 172.21.15.133:0/1686117905 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55afd48b0620 0x55afd48ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:19.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.340+0000 7fac4affd700 10 client.15342 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:16:19.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.340+0000 7fac6a7fc700 1 --2- 172.21.15.133:0/1686117905 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55afd48b0620 0x55afd48ad350 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fef25d92b7f51d24 server_cookie=24601d1f32aff092 in_seq=0 out_seq=0 2023-07-14T18:16:19.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.340+0000 7fac4affd700 1 -- 172.21.15.133:0/1686117905 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x55afd478ce60 con 0x55afd48b0620 2023-07-14T18:16:19.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.340+0000 7fac4affd700 10 client.15342 handle_command_reply: tid=0 2023-07-14T18:16:19.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.340+0000 7fac85331b80 1 client.15342 shutdown 2023-07-14T18:16:19.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.340+0000 7fac85331b80 1 -- 172.21.15.133:0/1686117905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4842f10 msgr2=0x55afd484f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:19.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.340+0000 7fac85331b80 1 --2- 172.21.15.133:0/1686117905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4842f10 0x55afd484f1f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fac5c097d20 tx=0x7fac5c0cb960 comp rx=0 tx=0).stop 2023-07-14T18:16:19.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.340+0000 7fac85331b80 1 -- 172.21.15.133:0/1686117905 shutdown_connections 2023-07-14T18:16:19.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.340+0000 7fac85331b80 1 --2- 172.21.15.133:0/1686117905 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55afd48b0620 0x55afd48ad350 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.340+0000 7fac85331b80 1 --2- 172.21.15.133:0/1686117905 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55afd4844460 0x55afd4852720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.340+0000 7fac4affd700 0 client.15342 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:16:19.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.340+0000 7fac85331b80 1 --2- 172.21.15.133:0/1686117905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afd4842f10 0x55afd484f1f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.340+0000 7fac85331b80 1 --2- 172.21.15.133:0/1686117905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55afd4843860 0x55afd484f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.340+0000 7fac85331b80 1 -- 172.21.15.133:0/1686117905 >> 172.21.15.133:0/1686117905 conn(0x55afd4838630 msgr2=0x55afd484a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:19.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.341+0000 7fac85331b80 1 -- 172.21.15.133:0/1686117905 shutdown_connections 2023-07-14T18:16:19.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.341+0000 7fac85331b80 1 -- 172.21.15.133:0/1686117905 wait complete. 2023-07-14T18:16:19.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.341+0000 7fac85331b80 20 client.15342 trim_cache size 0 max 16384 2023-07-14T18:16:19.396 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:16:19.396 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "06d0f5fa-2271-11ee-9b33-001a4aab830c", 2023-07-14T18:16:19.396 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2023-07-14T18:16:19.396 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24557, 2023-07-14T18:16:19.397 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2023-07-14T18:16:19.397 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2023-07-14T18:16:19.397 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2023-07-14T18:16:19.397 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 121.413843566, 2023-07-14T18:16:19.397 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 25, 2023-07-14T18:16:19.397 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 92, 2023-07-14T18:16:19.397 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 90, 2023-07-14T18:16:19.398 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 128.427624759 2023-07-14T18:16:19.398 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:16:19.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.341+0000 7fac719ff700 1 -- 172.21.15.133:0/3837287912 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fac540a1150 msgr2=0x7fac540a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:19.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.341+0000 7fac719ff700 1 --2- 172.21.15.133:0/3837287912 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fac540a1150 0x7fac540a3600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fac6401f000 tx=0x7fac64020d10 comp rx=0 tx=0).stop 2023-07-14T18:16:19.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.341+0000 7fac719ff700 1 -- 172.21.15.133:0/3837287912 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac6c0ffc20 msgr2=0x7fac6c0743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:19.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.341+0000 7fac719ff700 1 --2- 172.21.15.133:0/3837287912 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac6c0ffc20 0x7fac6c0743f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fac5c00b620 tx=0x7fac5c005740 comp rx=0 tx=0).stop 2023-07-14T18:16:19.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.341+0000 7fac719ff700 1 -- 172.21.15.133:0/3837287912 shutdown_connections 2023-07-14T18:16:19.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.341+0000 7fac719ff700 1 --2- 172.21.15.133:0/3837287912 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fac540a1150 0x7fac540a3600 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.341+0000 7fac719ff700 1 --2- 172.21.15.133:0/3837287912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac6c1017a0 0x7fac6c072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.341+0000 7fac719ff700 1 --2- 172.21.15.133:0/3837287912 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac6c0ffc20 0x7fac6c0743f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.341+0000 7fac719ff700 1 --2- 172.21.15.133:0/3837287912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac6c100e10 0x7fac6c072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:19.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.341+0000 7fac719ff700 1 -- 172.21.15.133:0/3837287912 >> 172.21.15.133:0/3837287912 conn(0x7fac6c0fb260 msgr2=0x7fac6c0fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:19.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.341+0000 7fac719ff700 1 -- 172.21.15.133:0/3837287912 shutdown_connections 2023-07-14T18:16:19.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:19.342+0000 7fac719ff700 1 -- 172.21.15.133:0/3837287912 wait complete. 2023-07-14T18:16:19.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:19 smithi133 ceph-mon[126384]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-07-14T18:16:19.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:19 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2268298862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:16:19.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:19 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2230630586' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:16:19.404 INFO:teuthology.run_tasks:Running task kclient... 2023-07-14T18:16:19.419 INFO:tasks.kclient:Mounting kernel clients... 2023-07-14T18:16:19.419 INFO:tasks.kclient:config is {'client.0': {}} 2023-07-14T18:16:19.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:16:19.420 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2023-07-14T18:16:19.463 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-07-14T18:16:19.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:16:19.465 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-07-14T18:16:19.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:16:19.574 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link delete ceph-brx 2023-07-14T18:16:19.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:19 smithi161 ceph-mon[128993]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-07-14T18:16:19.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:19 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2268298862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:16:19.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:19 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2230630586' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:16:19.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:19 smithi171 ceph-mon[129241]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-07-14T18:16:19.617 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:19 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2268298862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:16:19.617 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:19 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2230630586' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:16:19.696 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v2', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2023-07-14T18:16:19.696 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-14T18:16:19.696 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2023-07-14T18:16:19.697 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2023-07-14T18:16:19.697 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph fs ls 2023-07-14T18:16:20.422 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:16:21.285 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:20 smithi133 ceph-mon[126384]: pgmap v338: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2023-07-14T18:16:21.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:20 smithi161 ceph-mon[128993]: pgmap v338: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2023-07-14T18:16:21.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:20 smithi171 ceph-mon[129241]: pgmap v338: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2023-07-14T18:16:21.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.878+0000 7f040d2d8700 1 -- 172.21.15.133:0/2222120027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0408100f20 msgr2=0x7f04081012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:21.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.878+0000 7f040d2d8700 1 --2- 172.21.15.133:0/2222120027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0408100f20 0x7f04081012f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009990 tx=0x7f03f8009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:21.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.878+0000 7f040d2d8700 1 -- 172.21.15.133:0/2222120027 shutdown_connections 2023-07-14T18:16:21.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.878+0000 7f040d2d8700 1 --2- 172.21.15.133:0/2222120027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f040810a300 0x7f040810c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:21.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.878+0000 7f040d2d8700 1 --2- 172.21.15.133:0/2222120027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0408100f20 0x7f04081012f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:21.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.878+0000 7f040d2d8700 1 --2- 172.21.15.133:0/2222120027 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0408101830 0x7f0408109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:21.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.878+0000 7f040d2d8700 1 -- 172.21.15.133:0/2222120027 >> 172.21.15.133:0/2222120027 conn(0x7f04080fa720 msgr2=0x7f04080fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:21.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.879+0000 7f040d2d8700 1 -- 172.21.15.133:0/2222120027 shutdown_connections 2023-07-14T18:16:21.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.879+0000 7f040d2d8700 1 -- 172.21.15.133:0/2222120027 wait complete. 2023-07-14T18:16:21.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.880+0000 7f040d2d8700 1 Processor -- start 2023-07-14T18:16:21.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.880+0000 7f040d2d8700 1 -- start start 2023-07-14T18:16:21.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.880+0000 7f040d2d8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0408100f20 0x7f0408195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:21.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.880+0000 7f040d2d8700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0408101830 0x7f04081962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:21.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.880+0000 7f040d2d8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f040810a300 0x7f040819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:21.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.880+0000 7f040d2d8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040819acf0 con 0x7f040810a300 2023-07-14T18:16:21.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.880+0000 7f040d2d8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040819ae60 con 0x7f0408100f20 2023-07-14T18:16:21.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.880+0000 7f040d2d8700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040819b020 con 0x7f0408101830 2023-07-14T18:16:21.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.880+0000 7f0406ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0408100f20 0x7f0408195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:21.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.880+0000 7f0406ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0408100f20 0x7f0408195da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:50050/0 (socket says 172.21.15.133:50050) 2023-07-14T18:16:21.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.880+0000 7f0406ffd700 1 -- 172.21.15.133:0/1338507454 learned_addr learned my addr 172.21.15.133:0/1338507454 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:21.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.880+0000 7f04077fe700 1 --2- 172.21.15.133:0/1338507454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f040810a300 0x7f040819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:21.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.881+0000 7f04067fc700 1 --2- 172.21.15.133:0/1338507454 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0408101830 0x7f04081962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:21.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.881+0000 7f0406ffd700 1 -- 172.21.15.133:0/1338507454 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0408101830 msgr2=0x7f04081962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:21.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.881+0000 7f0406ffd700 1 --2- 172.21.15.133:0/1338507454 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0408101830 0x7f04081962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:21.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.881+0000 7f0406ffd700 1 -- 172.21.15.133:0/1338507454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f040810a300 msgr2=0x7f040819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:21.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.881+0000 7f0406ffd700 1 --2- 172.21.15.133:0/1338507454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f040810a300 0x7f040819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:21.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.881+0000 7f0406ffd700 1 -- 172.21.15.133:0/1338507454 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f80095f0 con 0x7f0408100f20 2023-07-14T18:16:21.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.881+0000 7f0406ffd700 1 --2- 172.21.15.133:0/1338507454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0408100f20 0x7f0408195da0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f03f8005910 tx=0x7f03f8005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:21.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.881+0000 7f03effff700 1 -- 172.21.15.133:0/1338507454 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f801d070 con 0x7f0408100f20 2023-07-14T18:16:21.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.881+0000 7f040d2d8700 1 -- 172.21.15.133:0/1338507454 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f040819b2a0 con 0x7f0408100f20 2023-07-14T18:16:21.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.881+0000 7f03effff700 1 -- 172.21.15.133:0/1338507454 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03f8022470 con 0x7f0408100f20 2023-07-14T18:16:21.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.881+0000 7f03effff700 1 -- 172.21.15.133:0/1338507454 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f8017650 con 0x7f0408100f20 2023-07-14T18:16:21.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.882+0000 7f040d2d8700 1 -- 172.21.15.133:0/1338507454 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04080fec80 con 0x7f0408100f20 2023-07-14T18:16:21.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.882+0000 7f040d2d8700 1 -- 172.21.15.133:0/1338507454 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04080682d0 con 0x7f0408100f20 2023-07-14T18:16:21.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.885+0000 7f03effff700 1 -- 172.21.15.133:0/1338507454 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f03f800fad0 con 0x7f0408100f20 2023-07-14T18:16:21.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.886+0000 7f03effff700 1 --2- 172.21.15.133:0/1338507454 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f03f00a1150 0x7f03f00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:21.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.886+0000 7f03effff700 1 -- 172.21.15.133:0/1338507454 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f03f80ce850 con 0x7f0408100f20 2023-07-14T18:16:21.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.886+0000 7f04067fc700 1 --2- 172.21.15.133:0/1338507454 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f03f00a1150 0x7f03f00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:21.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.887+0000 7f04067fc700 1 --2- 172.21.15.133:0/1338507454 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f03f00a1150 0x7f03f00a3600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0408197350 tx=0x7f03fc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:21.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:21.888+0000 7f03effff700 1 -- 172.21.15.133:0/1338507454 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f03f8097ca0 con 0x7f0408100f20 2023-07-14T18:16:22.176 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:22.131+0000 7f040d2d8700 1 -- 172.21.15.133:0/1338507454 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f04080feea0 con 0x7f0408100f20 2023-07-14T18:16:22.177 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:22.132+0000 7f03effff700 1 -- 172.21.15.133:0/1338507454 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+88 (secure 0 0 0) 0x7f03f809b2c0 con 0x7f0408100f20 2023-07-14T18:16:22.177 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-07-14T18:16:22.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:22.134+0000 7f040d2d8700 1 -- 172.21.15.133:0/1338507454 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f03f00a1150 msgr2=0x7f03f00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:22.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:22.134+0000 7f040d2d8700 1 --2- 172.21.15.133:0/1338507454 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f03f00a1150 0x7f03f00a3600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0408197350 tx=0x7f03fc00b410 comp rx=0 tx=0).stop 2023-07-14T18:16:22.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:22.134+0000 7f040d2d8700 1 -- 172.21.15.133:0/1338507454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0408100f20 msgr2=0x7f0408195da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:22.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:22.134+0000 7f040d2d8700 1 --2- 172.21.15.133:0/1338507454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0408100f20 0x7f0408195da0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f03f8005910 tx=0x7f03f8005740 comp rx=0 tx=0).stop 2023-07-14T18:16:22.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:22.135+0000 7f040d2d8700 1 -- 172.21.15.133:0/1338507454 shutdown_connections 2023-07-14T18:16:22.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:22.135+0000 7f040d2d8700 1 --2- 172.21.15.133:0/1338507454 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f03f00a1150 0x7f03f00a3600 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:22.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:22.135+0000 7f040d2d8700 1 --2- 172.21.15.133:0/1338507454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0408100f20 0x7f0408195da0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:22.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:22.135+0000 7f040d2d8700 1 --2- 172.21.15.133:0/1338507454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f040810a300 0x7f040819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:22.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:22.135+0000 7f040d2d8700 1 --2- 172.21.15.133:0/1338507454 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0408101830 0x7f04081962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:22.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:22.135+0000 7f040d2d8700 1 -- 172.21.15.133:0/1338507454 >> 172.21.15.133:0/1338507454 conn(0x7f04080fa720 msgr2=0x7f040810c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:22.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:22.135+0000 7f040d2d8700 1 -- 172.21.15.133:0/1338507454 shutdown_connections 2023-07-14T18:16:22.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:22.135+0000 7f040d2d8700 1 -- 172.21.15.133:0/1338507454 wait complete. 2023-07-14T18:16:22.594 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-14T18:16:22.595 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-14T18:16:22.596 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2023-07-14T18:16:22.596 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-14T18:16:22.597 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2023-07-14T18:16:22.597 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-14T18:16:22.597 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-14T18:16:22.598 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-14T18:16:22.598 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2023-07-14T18:16:22.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:16:22.599 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2023-07-14T18:16:22.644 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-14T18:16:22.645 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-14T18:16:22.645 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2023-07-14T18:16:22.645 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-07-14T18:16:22.646 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2023-07-14T18:16:22.646 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-07-14T18:16:22.646 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-14T18:16:22.646 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2023-07-14T18:16:22.646 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2023-07-14T18:16:22.647 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-14T18:16:22.647 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2023-07-14T18:16:22.647 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2023-07-14T18:16:22.647 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-14T18:16:22.647 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2023-07-14T18:16:22.648 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-14T18:16:22.648 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2023-07-14T18:16:22.648 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-14T18:16:22.648 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 41411sec preferred_lft 41411sec 2023-07-14T18:16:22.649 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::1cfd:5bbf:6db9:2d9/64 scope link noprefixroute 2023-07-14T18:16:22.649 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-07-14T18:16:22.649 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-14T18:16:22.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:16:22.650 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-07-14T18:16:22.651 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-07-14T18:16:22.651 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add name ceph-brx type bridge 2023-07-14T18:16:22.652 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr flush dev ceph-brx 2023-07-14T18:16:22.652 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set ceph-brx up 2023-07-14T18:16:22.653 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-14T18:16:22.653 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-07-14T18:16:22.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:16:22.891 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-14T18:16:22.960 INFO:teuthology.orchestra.run.smithi133.stdout:1 2023-07-14T18:16:22.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:16:22.964 DEBUG:teuthology.orchestra.run.smithi133:> route 2023-07-14T18:16:23.011 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2023-07-14T18:16:23.011 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-14T18:16:23.011 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-14T18:16:23.011 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-14T18:16:23.011 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-14T18:16:23.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:16:23.013 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-07-14T18:16:23.013 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-07-14T18:16:23.013 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-14T18:16:23.014 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-14T18:16:23.014 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-14T18:16:23.014 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-07-14T18:16:23.116 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:22 smithi133 ceph-mon[126384]: pgmap v339: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 341 B/s wr, 5 op/s 2023-07-14T18:16:23.116 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:22 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1338507454' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-14T18:16:23.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:16:23.187 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2023-07-14T18:16:23.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:16:23.231 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list-id 2023-07-14T18:16:23.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:16:23.314 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-07-14T18:16:23.314 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-07-14T18:16:23.314 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns add ceph-ns--tmp-cephfs 2023-07-14T18:16:23.314 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns set ceph-ns--tmp-cephfs 0 2023-07-14T18:16:23.315 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-07-14T18:16:23.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:22 smithi161 ceph-mon[128993]: pgmap v339: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 341 B/s wr, 5 op/s 2023-07-14T18:16:23.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:22 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1338507454' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-14T18:16:23.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:22 smithi171 ceph-mon[129241]: pgmap v339: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 341 B/s wr, 5 op/s 2023-07-14T18:16:23.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:22 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1338507454' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-14T18:16:23.495 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2023-07-14T18:16:23.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:16:23.496 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-07-14T18:16:23.496 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-07-14T18:16:23.497 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2023-07-14T18:16:23.497 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-14T18:16:23.497 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2023-07-14T18:16:23.497 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2023-07-14T18:16:23.498 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2023-07-14T18:16:23.498 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-07-14T18:16:23.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:16:23.745 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-07-14T18:16:23.746 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-07-14T18:16:23.746 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2023-07-14T18:16:23.746 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set dev brx.0 master ceph-brx 2023-07-14T18:16:23.746 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-07-14T18:16:23.919 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T18:16:23.920 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /tmp/cephfs 2023-07-14T18:16:23.963 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/tmp/cephfs' 2023-07-14T18:16:23.964 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T18:16:23.964 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /tmp/cephfs 2023-07-14T18:16:24.046 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-07-14T18:16:24.047 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-07-14T18:16:24.047 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/_nogroup/sv_1/63cee60f-94c5-4324-af48-1f2bdf1c218e 2023-07-14T18:16:24.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:16:24.048 DEBUG:teuthology.orchestra.run.smithi133:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/_nogroup/sv_1/63cee60f-94c5-4324-af48-1f2bdf1c218e /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,context=system_u:object_r:postgresql_db_t:s0,nofallback 2023-07-14T18:16:24.192 INFO:teuthology.orchestra.run.smithi133.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2023-07-14T18:16:24.192 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2023-07-14T18:16:24.666 INFO:teuthology.orchestra.run.smithi133.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2023-07-14T18:16:24.666 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2023-07-14T18:16:24.667 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: resolved to: "172.21.15.133:6789,172.21.15.161:6789,172.21.15.171:6789" 2023-07-14T18:16:24.667 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: trying mount with new device syntax: 0@06d0f5fa-2271-11ee-9b33-001a4aab830c.cephfs=/volumes/_nogroup/sv_1/63cee60f-94c5-4324-af48-1f2bdf1c218e 2023-07-14T18:16:24.667 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mon_addr=172.21.15.133:6789/172.21.15.161:6789/172.21.15.171:6789" will pass to kernel 2023-07-14T18:16:24.673 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-14T18:16:24.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:16:24.674 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /tmp/cephfs 2023-07-14T18:16:24.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:16:24.745 DEBUG:teuthology.orchestra.run.smithi133:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-14T18:16:24.831 INFO:teuthology.orchestra.run.smithi133.stdout:client25334 2023-07-14T18:16:24.832 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T18:16:24.832 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T18:16:24.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:16:24.876 DEBUG:teuthology.orchestra.run.smithi133:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-14T18:16:25.003 INFO:teuthology.orchestra.run.smithi133.stdout:client25334 2023-07-14T18:16:25.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:16:25.004 DEBUG:teuthology.orchestra.run.smithi133:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-14T18:16:25.090 INFO:teuthology.orchestra.run.smithi133.stdout:client25334 2023-07-14T18:16:25.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:16:25.091 DEBUG:teuthology.orchestra.run.smithi133:> (cd /tmp/cephfs && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c.client25334/status') 2023-07-14T18:16:25.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:24 smithi133 ceph-mon[126384]: pgmap v340: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-14T18:16:25.198 INFO:teuthology.orchestra.run.smithi133.stdout:instance: client.25334 (3)172.21.15.133:0/3881128325 2023-07-14T18:16:25.198 INFO:teuthology.orchestra.run.smithi133.stdout:blocklisted: false 2023-07-14T18:16:25.199 INFO:teuthology.orchestra.run.smithi133.stderr:0+1 records in 2023-07-14T18:16:25.199 INFO:teuthology.orchestra.run.smithi133.stderr:0+1 records out 2023-07-14T18:16:25.199 INFO:teuthology.orchestra.run.smithi133.stderr:72 bytes copied, 4.8074e-05 s, 1.5 MB/s 2023-07-14T18:16:25.201 INFO:teuthology.run_tasks:Running task exec... 2023-07-14T18:16:25.211 INFO:teuthology.task.exec:Executing custom commands... 2023-07-14T18:16:25.211 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi133.front.sepia.ceph.com 2023-07-14T18:16:25.212 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2023-07-14T18:16:25.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:24 smithi161 ceph-mon[128993]: pgmap v340: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-14T18:16:25.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:24 smithi171 ceph-mon[129241]: pgmap v340: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-14T18:16:25.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.358+0000 7f46b95ff700 1 -- 172.21.15.133:0/696073800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46b40ffc40 msgr2=0x7f46b4100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:25.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.358+0000 7f46b95ff700 1 --2- 172.21.15.133:0/696073800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46b40ffc40 0x7f46b4100050 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f46a40098f0 tx=0x7f46a4009c00 comp rx=0 tx=0).stop 2023-07-14T18:16:25.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.358+0000 7f46b95ff700 1 -- 172.21.15.133:0/696073800 shutdown_connections 2023-07-14T18:16:25.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.358+0000 7f46b95ff700 1 --2- 172.21.15.133:0/696073800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46b40ffc40 0x7f46b4100050 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:25.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.358+0000 7f46b95ff700 1 --2- 172.21.15.133:0/696073800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46b41017c0 0x7f46b4108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:25.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.358+0000 7f46b95ff700 1 --2- 172.21.15.133:0/696073800 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46b4100e30 0x7f46b4101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:25.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.358+0000 7f46b95ff700 1 -- 172.21.15.133:0/696073800 >> 172.21.15.133:0/696073800 conn(0x7f46b40fb260 msgr2=0x7f46b40fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:25.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.358+0000 7f46b95ff700 1 -- 172.21.15.133:0/696073800 shutdown_connections 2023-07-14T18:16:25.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.358+0000 7f46b95ff700 1 -- 172.21.15.133:0/696073800 wait complete. 2023-07-14T18:16:25.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.359+0000 7f46b95ff700 1 Processor -- start 2023-07-14T18:16:25.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.359+0000 7f46b95ff700 1 -- start start 2023-07-14T18:16:25.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.360+0000 7f46b95ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46b40ffc40 0x7f46b4197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:25.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.360+0000 7f46b95ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46b4100e30 0x7f46b4198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:25.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.360+0000 7f46b95ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46b41017c0 0x7f46b4199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:25.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.360+0000 7f46b95ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b4199740 con 0x7f46b41017c0 2023-07-14T18:16:25.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.360+0000 7f46b95ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b41998b0 con 0x7f46b4100e30 2023-07-14T18:16:25.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.360+0000 7f46b95ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b4199a70 con 0x7f46b40ffc40 2023-07-14T18:16:25.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.360+0000 7f46b2ffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46b40ffc40 0x7f46b4197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:25.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.360+0000 7f46b37fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46b41017c0 0x7f46b4199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:25.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.360+0000 7f46b2ffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46b40ffc40 0x7f46b4197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:56192/0 (socket says 172.21.15.133:56192) 2023-07-14T18:16:25.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.360+0000 7f46b2ffd700 1 -- 172.21.15.133:0/2357189582 learned_addr learned my addr 172.21.15.133:0/2357189582 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:25.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.360+0000 7f46b27fc700 1 --2- 172.21.15.133:0/2357189582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46b4100e30 0x7f46b4198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:25.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.360+0000 7f46b2ffd700 1 -- 172.21.15.133:0/2357189582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46b4100e30 msgr2=0x7f46b4198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:25.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.360+0000 7f46b2ffd700 1 --2- 172.21.15.133:0/2357189582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46b4100e30 0x7f46b4198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:25.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.360+0000 7f46b2ffd700 1 -- 172.21.15.133:0/2357189582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46b41017c0 msgr2=0x7f46b4199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:25.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.360+0000 7f46b2ffd700 1 --2- 172.21.15.133:0/2357189582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46b41017c0 0x7f46b4199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:25.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.360+0000 7f46b2ffd700 1 -- 172.21.15.133:0/2357189582 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ac009d00 con 0x7f46b40ffc40 2023-07-14T18:16:25.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.361+0000 7f46b2ffd700 1 --2- 172.21.15.133:0/2357189582 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46b40ffc40 0x7f46b4197d50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f46a4005e90 tx=0x7f46a400b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:25.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.361+0000 7f469bfff700 1 -- 172.21.15.133:0/2357189582 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a401c070 con 0x7f46b40ffc40 2023-07-14T18:16:25.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.361+0000 7f46b95ff700 1 -- 172.21.15.133:0/2357189582 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a4009520 con 0x7f46b40ffc40 2023-07-14T18:16:25.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.361+0000 7f469bfff700 1 -- 172.21.15.133:0/2357189582 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46a400f7f0 con 0x7f46b40ffc40 2023-07-14T18:16:25.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.361+0000 7f469bfff700 1 -- 172.21.15.133:0/2357189582 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a4003990 con 0x7f46b40ffc40 2023-07-14T18:16:25.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.361+0000 7f46b95ff700 1 -- 172.21.15.133:0/2357189582 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b41a0280 con 0x7f46b40ffc40 2023-07-14T18:16:25.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.362+0000 7f46b95ff700 1 -- 172.21.15.133:0/2357189582 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b4192320 con 0x7f46b40ffc40 2023-07-14T18:16:25.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.364+0000 7f469bfff700 1 -- 172.21.15.133:0/2357189582 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f46a4020030 con 0x7f46b40ffc40 2023-07-14T18:16:25.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.365+0000 7f469bfff700 1 --2- 172.21.15.133:0/2357189582 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f469c0a1150 0x7f469c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:25.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.365+0000 7f469bfff700 1 -- 172.21.15.133:0/2357189582 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f46a40cdfe0 con 0x7f46b40ffc40 2023-07-14T18:16:25.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.366+0000 7f46b27fc700 1 --2- 172.21.15.133:0/2357189582 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f469c0a1150 0x7f469c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:25.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.366+0000 7f46b27fc700 1 --2- 172.21.15.133:0/2357189582 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f469c0a1150 0x7f469c0a3600 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f46a8009c60 tx=0x7f46a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:25.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.367+0000 7f469bfff700 1 -- 172.21.15.133:0/2357189582 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f46a4097430 con 0x7f46b40ffc40 2023-07-14T18:16:25.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.611+0000 7f46b95ff700 1 -- 172.21.15.133:0/2357189582 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) v1 -- 0x7f46b404e420 con 0x7f46b40ffc40 2023-07-14T18:16:25.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.938+0000 7f469bfff700 1 -- 172.21.15.133:0/2357189582 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v26) v1 ==== 117+0+0 (secure 0 0 0) 0x7f46a409aa50 con 0x7f46b40ffc40 2023-07-14T18:16:25.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.941+0000 7f46b95ff700 1 -- 172.21.15.133:0/2357189582 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f469c0a1150 msgr2=0x7f469c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:25.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.941+0000 7f46b95ff700 1 --2- 172.21.15.133:0/2357189582 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f469c0a1150 0x7f469c0a3600 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f46a8009c60 tx=0x7f46a8009260 comp rx=0 tx=0).stop 2023-07-14T18:16:25.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.941+0000 7f46b95ff700 1 -- 172.21.15.133:0/2357189582 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46b40ffc40 msgr2=0x7f46b4197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:25.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.941+0000 7f46b95ff700 1 --2- 172.21.15.133:0/2357189582 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46b40ffc40 0x7f46b4197d50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f46a4005e90 tx=0x7f46a400b730 comp rx=0 tx=0).stop 2023-07-14T18:16:25.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.941+0000 7f46b95ff700 1 -- 172.21.15.133:0/2357189582 shutdown_connections 2023-07-14T18:16:25.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.941+0000 7f46b95ff700 1 --2- 172.21.15.133:0/2357189582 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f469c0a1150 0x7f469c0a3600 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:25.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.941+0000 7f46b95ff700 1 --2- 172.21.15.133:0/2357189582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46b41017c0 0x7f46b4199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:25.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.941+0000 7f46b95ff700 1 --2- 172.21.15.133:0/2357189582 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46b40ffc40 0x7f46b4197d50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:25.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.941+0000 7f46b95ff700 1 --2- 172.21.15.133:0/2357189582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46b4100e30 0x7f46b4198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:25.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.941+0000 7f46b95ff700 1 -- 172.21.15.133:0/2357189582 >> 172.21.15.133:0/2357189582 conn(0x7f46b40fb260 msgr2=0x7f46b4106790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:25.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.941+0000 7f46b95ff700 1 -- 172.21.15.133:0/2357189582 shutdown_connections 2023-07-14T18:16:25.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:25.941+0000 7f46b95ff700 1 -- 172.21.15.133:0/2357189582 wait complete. 2023-07-14T18:16:26.005 INFO:teuthology.run_tasks:Running task check-counter... 2023-07-14T18:16:26.023 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-07-14T18:16:26.033 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T18:16:26.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.186+0000 7f23b65ff700 1 -- 172.21.15.133:0/3406695028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23b00ffc30 msgr2=0x7f23b0100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.186+0000 7f23b65ff700 1 --2- 172.21.15.133:0/3406695028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23b00ffc30 0x7f23b0100040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f23a0009990 tx=0x7f23a0009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.186+0000 7f23b65ff700 1 -- 172.21.15.133:0/3406695028 shutdown_connections 2023-07-14T18:16:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.186+0000 7f23b65ff700 1 --2- 172.21.15.133:0/3406695028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23b01017b0 0x7f23b0108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.186+0000 7f23b65ff700 1 --2- 172.21.15.133:0/3406695028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23b00ffc30 0x7f23b0100040 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.186+0000 7f23b65ff700 1 --2- 172.21.15.133:0/3406695028 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f23b0100e20 0x7f23b0101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.186+0000 7f23b65ff700 1 -- 172.21.15.133:0/3406695028 >> 172.21.15.133:0/3406695028 conn(0x7f23b00fb210 msgr2=0x7f23b00fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:26.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.186+0000 7f23b65ff700 1 -- 172.21.15.133:0/3406695028 shutdown_connections 2023-07-14T18:16:26.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.186+0000 7f23b65ff700 1 -- 172.21.15.133:0/3406695028 wait complete. 2023-07-14T18:16:26.234 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.187+0000 7f23b65ff700 1 Processor -- start 2023-07-14T18:16:26.234 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.187+0000 7f23b65ff700 1 -- start start 2023-07-14T18:16:26.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.187+0000 7f23b65ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f23b00ffc30 0x7f23b0195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:26.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.188+0000 7f23b65ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23b0100e20 0x7f23b0195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:26.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.188+0000 7f23b65ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23b01017b0 0x7f23b019cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:26.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.188+0000 7f23b65ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b019d4d0 con 0x7f23b0100e20 2023-07-14T18:16:26.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.188+0000 7f23b65ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b019d610 con 0x7f23b01017b0 2023-07-14T18:16:26.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.188+0000 7f23b65ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b019d7a0 con 0x7f23b00ffc30 2023-07-14T18:16:26.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.188+0000 7f23af7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23b0100e20 0x7f23b0195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:26.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.188+0000 7f23af7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23b0100e20 0x7f23b0195e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44474/0 (socket says 172.21.15.133:44474) 2023-07-14T18:16:26.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.188+0000 7f23af7fe700 1 -- 172.21.15.133:0/816313558 learned_addr learned my addr 172.21.15.133:0/816313558 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:26.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.188+0000 7f23affff700 1 --2- 172.21.15.133:0/816313558 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f23b00ffc30 0x7f23b0195950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:26.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.188+0000 7f23b4b9c700 1 --2- 172.21.15.133:0/816313558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23b01017b0 0x7f23b019cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:26.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.188+0000 7f23af7fe700 1 -- 172.21.15.133:0/816313558 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f23b00ffc30 msgr2=0x7f23b0195950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:26.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.188+0000 7f23af7fe700 1 --2- 172.21.15.133:0/816313558 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f23b00ffc30 0x7f23b0195950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:26.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.188+0000 7f23af7fe700 1 -- 172.21.15.133:0/816313558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23b01017b0 msgr2=0x7f23b019cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:26.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.188+0000 7f23af7fe700 1 --2- 172.21.15.133:0/816313558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23b01017b0 0x7f23b019cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:26.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.188+0000 7f23af7fe700 1 -- 172.21.15.133:0/816313558 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a00095f0 con 0x7f23b0100e20 2023-07-14T18:16:26.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.188+0000 7f23af7fe700 1 --2- 172.21.15.133:0/816313558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23b0100e20 0x7f23b0195e90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f23a400e9a0 tx=0x7f23a400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:26.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.189+0000 7f23ad7fa700 1 -- 172.21.15.133:0/816313558 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a4009790 con 0x7f23b0100e20 2023-07-14T18:16:26.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.189+0000 7f23ad7fa700 1 -- 172.21.15.133:0/816313558 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23a400cbd0 con 0x7f23b0100e20 2023-07-14T18:16:26.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.189+0000 7f23ad7fa700 1 -- 172.21.15.133:0/816313558 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a4018680 con 0x7f23b0100e20 2023-07-14T18:16:26.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.189+0000 7f23b65ff700 1 -- 172.21.15.133:0/816313558 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b019da80 con 0x7f23b0100e20 2023-07-14T18:16:26.238 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.189+0000 7f23b65ff700 1 -- 172.21.15.133:0/816313558 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b019df40 con 0x7f23b0100e20 2023-07-14T18:16:26.238 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.190+0000 7f23b65ff700 1 -- 172.21.15.133:0/816313558 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b0107180 con 0x7f23b0100e20 2023-07-14T18:16:26.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.192+0000 7f23ad7fa700 1 -- 172.21.15.133:0/816313558 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f23a400cd40 con 0x7f23b0100e20 2023-07-14T18:16:26.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.193+0000 7f23ad7fa700 1 --2- 172.21.15.133:0/816313558 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f23980a1220 0x7f23980a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:26.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.193+0000 7f23ad7fa700 1 -- 172.21.15.133:0/816313558 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f23a4014070 con 0x7f23b0100e20 2023-07-14T18:16:26.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.193+0000 7f23affff700 1 --2- 172.21.15.133:0/816313558 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f23980a1220 0x7f23980a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:26.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.194+0000 7f23affff700 1 --2- 172.21.15.133:0/816313558 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f23980a1220 0x7f23980a36d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f23a000b2e0 tx=0x7f23a0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:26.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.195+0000 7f23ad7fa700 1 -- 172.21.15.133:0/816313558 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f23a4096500 con 0x7f23b0100e20 2023-07-14T18:16:26.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:25 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2357189582' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2023-07-14T18:16:26.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:25 smithi161 ceph-mon[128993]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2023-07-14T18:16:26.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:25 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2357189582' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2023-07-14T18:16:26.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:25 smithi171 ceph-mon[129241]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2023-07-14T18:16:26.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:25 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2357189582' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2023-07-14T18:16:26.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:25 smithi133 ceph-mon[126384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2023-07-14T18:16:26.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.397+0000 7f23b65ff700 1 -- 172.21.15.133:0/816313558 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23b002cb30 con 0x7f23b0100e20 2023-07-14T18:16:26.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.399+0000 7f23ad7fa700 1 -- 172.21.15.133:0/816313558 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24520 (secure 0 0 0) 0x7f23a4026090 con 0x7f23b0100e20 2023-07-14T18:16:26.444 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:26.444 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":92,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","created":"2023-07-14T18:06:49.561582+0000","modified":"2023-07-14T18:15:49.951050+0000","last_up_change":"2023-07-14T18:13:29.836099+0000","last_in_change":"2023-07-14T18:13:11.606710+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T18:10:03.945039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T18:14:15.451340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T18:14:16.892215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T18:14:20.493175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"85a29de7-d712-422a-b647-d9e90091aaf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6803","nonce":3076510801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6805","nonce":3076510801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6809","nonce":3076510801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6807","nonce":3076510801}]},"public_addr":"172.21.15.133:6803/3076510801","cluster_addr":"172.21.15.133:6805/3076510801","heartbeat_back_addr":"172.21.15.133:6809/3076510801","heartbeat_front_addr":"172.21.15.133:6807/3076510801","state":["exists","up"]},{"osd":1,"uuid":"80ad4e78-7917-495e-b119-99f7c7c36243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6801","nonce":1841901852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6803","nonce":1841901852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6807","nonce":1841901852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6805","nonce":1841901852}]},"public_addr":"172.21.15.161:6801/1841901852","cluster_addr":"172.21.15.161:6803/1841901852","heartbeat_back_addr":"172.21.15.161:6807/1841901852","heartbeat_front_addr":"172.21.15.161:6805/1841901852","state":["exists","up"]},{"osd":2,"uuid":"fc4bfd37-1fdd-47c6-a48e-6d910daf9e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6801","nonce":346103099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6803","nonce":346103099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6807","nonce":346103099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6805","nonce":346103099}]},"public_addr":"172.21.15.171:6801/346103099","cluster_addr":"172.21.15.171:6803/346103099","heartbeat_back_addr":"172.21.15.171:6807/346103099","heartbeat_front_addr":"172.21.15.171:6805/346103099","state":["exists","up"]},{"osd":3,"uuid":"4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6811","nonce":2152994143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6813","nonce":2152994143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6817","nonce":2152994143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6815","nonce":2152994143}]},"public_addr":"172.21.15.133:6811/2152994143","cluster_addr":"172.21.15.133:6813/2152994143","heartbeat_back_addr":"172.21.15.133:6817/2152994143","heartbeat_front_addr":"172.21.15.133:6815/2152994143","state":["exists","up"]},{"osd":4,"uuid":"4601a1c2-254b-4c4d-9b15-8fc952b3d2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6809","nonce":47213291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6811","nonce":47213291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6815","nonce":47213291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6813","nonce":47213291}]},"public_addr":"172.21.15.161:6809/47213291","cluster_addr":"172.21.15.161:6811/47213291","heartbeat_back_addr":"172.21.15.161:6815/47213291","heartbeat_front_addr":"172.21.15.161:6813/47213291","state":["exists","up"]},{"osd":5,"uuid":"6cca180a-9140-4e49-835b-86842dae0574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6809","nonce":777556751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6811","nonce":777556751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6815","nonce":777556751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6813","nonce":777556751}]},"public_addr":"172.21.15.171:6809/777556751","cluster_addr":"172.21.15.171:6811/777556751","heartbeat_back_addr":"172.21.15.171:6815/777556751","heartbeat_front_addr":"172.21.15.171:6813/777556751","state":["exists","up"]},{"osd":6,"uuid":"ea0299ff-7740-4494-a3a9-1503b3fc0ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6819","nonce":2884152358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6821","nonce":2884152358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6825","nonce":2884152358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6823","nonce":2884152358}]},"public_addr":"172.21.15.133:6819/2884152358","cluster_addr":"172.21.15.133:6821/2884152358","heartbeat_back_addr":"172.21.15.133:6825/2884152358","heartbeat_front_addr":"172.21.15.133:6823/2884152358","state":["exists","up"]},{"osd":7,"uuid":"b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6817","nonce":4183705749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6819","nonce":4183705749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6823","nonce":4183705749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6821","nonce":4183705749}]},"public_addr":"172.21.15.161:6817/4183705749","cluster_addr":"172.21.15.161:6819/4183705749","heartbeat_back_addr":"172.21.15.161:6823/4183705749","heartbeat_front_addr":"172.21.15.161:6821/4183705749","state":["exists","up"]},{"osd":8,"uuid":"87dffc07-bf48-48c8-b418-5c4b509686d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6817","nonce":2771494483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6819","nonce":2771494483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6823","nonce":2771494483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6821","nonce":2771494483}]},"public_addr":"172.21.15.171:6817/2771494483","cluster_addr":"172.21.15.171:6819/2771494483","heartbeat_back_addr":"172.21.15.171:6823/2771494483","heartbeat_front_addr":"172.21.15.171:6821/2771494483","state":["exists","up"]},{"osd":9,"uuid":"e0832870-79ce-4bce-8045-58b7a7f6dc69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6827","nonce":2066164149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6829","nonce":2066164149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6833","nonce":2066164149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6831","nonce":2066164149}]},"public_addr":"172.21.15.133:6827/2066164149","cluster_addr":"172.21.15.133:6829/2066164149","heartbeat_back_addr":"172.21.15.133:6833/2066164149","heartbeat_front_addr":"172.21.15.133:6831/2066164149","state":["exists","up"]},{"osd":10,"uuid":"081f0484-d134-4b2e-9c48-ef8e6a646422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6825","nonce":2162333626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6827","nonce":2162333626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6831","nonce":2162333626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6829","nonce":2162333626}]},"public_addr":"172.21.15.161:6825/2162333626","cluster_addr":"172.21.15.161:6827/2162333626","heartbeat_back_addr":"172.21.15.161:6831/2162333626","heartbeat_front_addr":"172.21.15.161:6829/2162333626","state":["exists","up"]},{"osd":11,"uuid":"4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6825","nonce":3403894313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6827","nonce":3403894313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6831","nonce":3403894313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6829","nonce":3403894313}]},"public_addr":"172.21.15.171:6825/3403894313","cluster_addr":"172.21.15.171:6827/3403894313","heartbeat_back_addr":"172.21.15.171:6831/3403894313","heartbeat_front_addr":"172.21.15.171:6829/3403894313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:17.816229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:39.638543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:00.543317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:26.988921+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:48.644694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:10.049860+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:35.986508+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:57.753846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:19.018245+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:44.460450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:06.279960+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:27.955006+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/1832569241":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6801/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:0/4219815304":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:6801/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/432253615":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1840700480":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1807775439":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1089030734":"2023-07-15T18:07:47.346704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T18:16:26.446 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.401+0000 7f23b65ff700 1 -- 172.21.15.133:0/816313558 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f23980a1220 msgr2=0x7f23980a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:26.446 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.401+0000 7f23b65ff700 1 --2- 172.21.15.133:0/816313558 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f23980a1220 0x7f23980a36d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f23a000b2e0 tx=0x7f23a0005fb0 comp rx=0 tx=0).stop 2023-07-14T18:16:26.447 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.401+0000 7f23b65ff700 1 -- 172.21.15.133:0/816313558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23b0100e20 msgr2=0x7f23b0195e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:26.447 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.401+0000 7f23b65ff700 1 --2- 172.21.15.133:0/816313558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23b0100e20 0x7f23b0195e90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f23a400e9a0 tx=0x7f23a400ecb0 comp rx=0 tx=0).stop 2023-07-14T18:16:26.447 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.401+0000 7f23b65ff700 1 -- 172.21.15.133:0/816313558 shutdown_connections 2023-07-14T18:16:26.447 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.401+0000 7f23b65ff700 1 --2- 172.21.15.133:0/816313558 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f23980a1220 0x7f23980a36d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:26.447 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.401+0000 7f23b65ff700 1 --2- 172.21.15.133:0/816313558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23b01017b0 0x7f23b019cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:26.447 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.401+0000 7f23b65ff700 1 --2- 172.21.15.133:0/816313558 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f23b00ffc30 0x7f23b0195950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:26.447 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.401+0000 7f23b65ff700 1 --2- 172.21.15.133:0/816313558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23b0100e20 0x7f23b0195e90 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:26.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.401+0000 7f23b65ff700 1 -- 172.21.15.133:0/816313558 >> 172.21.15.133:0/816313558 conn(0x7f23b00fb210 msgr2=0x7f23b01042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:26.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.401+0000 7f23b65ff700 1 -- 172.21.15.133:0/816313558 shutdown_connections 2023-07-14T18:16:26.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.401+0000 7f23b65ff700 1 -- 172.21.15.133:0/816313558 wait complete. 2023-07-14T18:16:26.466 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T18:10:03.945039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T18:14:15.451340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T18:14:16.892215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T18:14:20.493175+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T18:16:26.467 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T18:16:26.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.624+0000 7f662fdff700 1 -- 172.21.15.133:0/2517479947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6628100e10 msgr2=0x7f6628101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:26.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.624+0000 7f662fdff700 1 --2- 172.21.15.133:0/2517479947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6628100e10 0x7f6628101260 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6618009990 tx=0x7f6618009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:26.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.624+0000 7f662fdff700 1 -- 172.21.15.133:0/2517479947 shutdown_connections 2023-07-14T18:16:26.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.624+0000 7f662fdff700 1 --2- 172.21.15.133:0/2517479947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66281017a0 0x7f6628108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:26.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.624+0000 7f662fdff700 1 --2- 172.21.15.133:0/2517479947 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66280ffc20 0x7f6628100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:26.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.624+0000 7f662fdff700 1 --2- 172.21.15.133:0/2517479947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6628100e10 0x7f6628101260 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:26.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.624+0000 7f662fdff700 1 -- 172.21.15.133:0/2517479947 >> 172.21.15.133:0/2517479947 conn(0x7f66280fb260 msgr2=0x7f66280fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:26.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.624+0000 7f662fdff700 1 -- 172.21.15.133:0/2517479947 shutdown_connections 2023-07-14T18:16:26.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.624+0000 7f662fdff700 1 -- 172.21.15.133:0/2517479947 wait complete. 2023-07-14T18:16:26.671 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.625+0000 7f662fdff700 1 Processor -- start 2023-07-14T18:16:26.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.626+0000 7f662fdff700 1 -- start start 2023-07-14T18:16:26.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.626+0000 7f662fdff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66280ffc20 0x7f6628195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:26.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.626+0000 7f662fdff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6628100e10 0x7f6628195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:26.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.626+0000 7f662fdff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66281017a0 0x7f662819cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:26.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.626+0000 7f662fdff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662819d4f0 con 0x7f66280ffc20 2023-07-14T18:16:26.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.626+0000 7f662fdff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662819d630 con 0x7f6628100e10 2023-07-14T18:16:26.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.626+0000 7f662fdff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662819d7c0 con 0x7f66281017a0 2023-07-14T18:16:26.673 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.626+0000 7f662db9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66280ffc20 0x7f6628195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:26.673 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.626+0000 7f662db9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66280ffc20 0x7f6628195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44484/0 (socket says 172.21.15.133:44484) 2023-07-14T18:16:26.673 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.626+0000 7f662db9b700 1 -- 172.21.15.133:0/297190997 learned_addr learned my addr 172.21.15.133:0/297190997 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:26.673 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.626+0000 7f662d39a700 1 --2- 172.21.15.133:0/297190997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6628100e10 0x7f6628195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:26.673 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.626+0000 7f662e39c700 1 --2- 172.21.15.133:0/297190997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66281017a0 0x7f662819cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:26.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.627+0000 7f662db9b700 1 -- 172.21.15.133:0/297190997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66281017a0 msgr2=0x7f662819cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:26.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.627+0000 7f662db9b700 1 --2- 172.21.15.133:0/297190997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66281017a0 0x7f662819cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:26.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.627+0000 7f662db9b700 1 -- 172.21.15.133:0/297190997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6628100e10 msgr2=0x7f6628195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:26.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.627+0000 7f662db9b700 1 --2- 172.21.15.133:0/297190997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6628100e10 0x7f6628195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:26.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.627+0000 7f662db9b700 1 -- 172.21.15.133:0/297190997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66180095f0 con 0x7f66280ffc20 2023-07-14T18:16:26.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.627+0000 7f662db9b700 1 --2- 172.21.15.133:0/297190997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66280ffc20 0x7f6628195970 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f662400e970 tx=0x7f662400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:26.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.627+0000 7f661effd700 1 -- 172.21.15.133:0/297190997 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6624009790 con 0x7f66280ffc20 2023-07-14T18:16:26.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.627+0000 7f662fdff700 1 -- 172.21.15.133:0/297190997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f662819daa0 con 0x7f66280ffc20 2023-07-14T18:16:26.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.627+0000 7f661effd700 1 -- 172.21.15.133:0/297190997 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f662400c7e0 con 0x7f66280ffc20 2023-07-14T18:16:26.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.627+0000 7f661effd700 1 -- 172.21.15.133:0/297190997 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6624003c70 con 0x7f66280ffc20 2023-07-14T18:16:26.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.627+0000 7f662fdff700 1 -- 172.21.15.133:0/297190997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f662819dfb0 con 0x7f66280ffc20 2023-07-14T18:16:26.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.628+0000 7f662fdff700 1 -- 172.21.15.133:0/297190997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f662810b440 con 0x7f66280ffc20 2023-07-14T18:16:26.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.630+0000 7f661effd700 1 -- 172.21.15.133:0/297190997 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f662400cc50 con 0x7f66280ffc20 2023-07-14T18:16:26.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.631+0000 7f661effd700 1 --2- 172.21.15.133:0/297190997 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f66140a1150 0x7f66140a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:26.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.632+0000 7f661effd700 1 -- 172.21.15.133:0/297190997 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6624014070 con 0x7f66280ffc20 2023-07-14T18:16:26.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.632+0000 7f662d39a700 1 --2- 172.21.15.133:0/297190997 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f66140a1150 0x7f66140a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:26.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.632+0000 7f662d39a700 1 --2- 172.21.15.133:0/297190997 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f66140a1150 0x7f66140a3600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f661800b2e0 tx=0x7f6618005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:26.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.635+0000 7f661effd700 1 -- 172.21.15.133:0/297190997 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f66240961f0 con 0x7f66280ffc20 2023-07-14T18:16:26.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.831+0000 7f662fdff700 1 -- 172.21.15.133:0/297190997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6628106b20 con 0x7f66280ffc20 2023-07-14T18:16:26.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.832+0000 7f661effd700 1 -- 172.21.15.133:0/297190997 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6624003dd0 con 0x7f66280ffc20 2023-07-14T18:16:26.877 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-07-14T18:16:26.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.834+0000 7f662fdff700 1 -- 172.21.15.133:0/297190997 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f66140a1150 msgr2=0x7f66140a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:26.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.834+0000 7f662fdff700 1 --2- 172.21.15.133:0/297190997 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f66140a1150 0x7f66140a3600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f661800b2e0 tx=0x7f6618005fb0 comp rx=0 tx=0).stop 2023-07-14T18:16:26.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.834+0000 7f662fdff700 1 -- 172.21.15.133:0/297190997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66280ffc20 msgr2=0x7f6628195970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:26.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.834+0000 7f662fdff700 1 --2- 172.21.15.133:0/297190997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66280ffc20 0x7f6628195970 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f662400e970 tx=0x7f662400ec80 comp rx=0 tx=0).stop 2023-07-14T18:16:26.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.834+0000 7f662fdff700 1 -- 172.21.15.133:0/297190997 shutdown_connections 2023-07-14T18:16:26.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.834+0000 7f662fdff700 1 --2- 172.21.15.133:0/297190997 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f66140a1150 0x7f66140a3600 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:26.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.834+0000 7f662fdff700 1 --2- 172.21.15.133:0/297190997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66281017a0 0x7f662819cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:26.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.834+0000 7f662fdff700 1 --2- 172.21.15.133:0/297190997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66280ffc20 0x7f6628195970 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:26.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.834+0000 7f662fdff700 1 --2- 172.21.15.133:0/297190997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6628100e10 0x7f6628195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:26.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.834+0000 7f662fdff700 1 -- 172.21.15.133:0/297190997 >> 172.21.15.133:0/297190997 conn(0x7f66280fb260 msgr2=0x7f66280fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:26.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.835+0000 7f662fdff700 1 -- 172.21.15.133:0/297190997 shutdown_connections 2023-07-14T18:16:26.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:26.835+0000 7f662fdff700 1 -- 172.21.15.133:0/297190997 wait complete. 2023-07-14T18:16:26.899 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T18:16:27.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.056+0000 7fc409bff700 1 -- 172.21.15.133:0/3939158966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc404100e10 msgr2=0x7fc404101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:27.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.056+0000 7fc409bff700 1 --2- 172.21.15.133:0/3939158966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc404100e10 0x7fc404101260 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8009990 tx=0x7fc3f8009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.056+0000 7fc409bff700 1 -- 172.21.15.133:0/3939158966 shutdown_connections 2023-07-14T18:16:27.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.056+0000 7fc409bff700 1 --2- 172.21.15.133:0/3939158966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4041017a0 0x7fc404108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.056+0000 7fc409bff700 1 --2- 172.21.15.133:0/3939158966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc4040ffc20 0x7fc404100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.056+0000 7fc409bff700 1 --2- 172.21.15.133:0/3939158966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc404100e10 0x7fc404101260 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.056+0000 7fc409bff700 1 -- 172.21.15.133:0/3939158966 >> 172.21.15.133:0/3939158966 conn(0x7fc4040fb260 msgr2=0x7fc4040fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:27.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.056+0000 7fc409bff700 1 -- 172.21.15.133:0/3939158966 shutdown_connections 2023-07-14T18:16:27.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.056+0000 7fc409bff700 1 -- 172.21.15.133:0/3939158966 wait complete. 2023-07-14T18:16:27.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.057+0000 7fc409bff700 1 Processor -- start 2023-07-14T18:16:27.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.058+0000 7fc409bff700 1 -- start start 2023-07-14T18:16:27.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.058+0000 7fc409bff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc4040ffc20 0x7fc4040743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:27.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.058+0000 7fc409bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc404100e10 0x7fc404072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:27.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.058+0000 7fc409bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4041017a0 0x7fc404072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:27.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.058+0000 7fc409bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc404073710 con 0x7fc404100e10 2023-07-14T18:16:27.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.058+0000 7fc409bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc404073880 con 0x7fc4041017a0 2023-07-14T18:16:27.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.058+0000 7fc409bff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40419f9d0 con 0x7fc4040ffc20 2023-07-14T18:16:27.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.058+0000 7fc402ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc404100e10 0x7fc404072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:27.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.058+0000 7fc4037fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc4040ffc20 0x7fc4040743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:27.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.058+0000 7fc4037fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc4040ffc20 0x7fc4040743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:44044/0 (socket says 172.21.15.133:44044) 2023-07-14T18:16:27.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.058+0000 7fc4037fe700 1 -- 172.21.15.133:0/2894387281 learned_addr learned my addr 172.21.15.133:0/2894387281 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:27.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.058+0000 7fc403fff700 1 --2- 172.21.15.133:0/2894387281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4041017a0 0x7fc404072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:27.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.059+0000 7fc4037fe700 1 -- 172.21.15.133:0/2894387281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4041017a0 msgr2=0x7fc404072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:27.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.059+0000 7fc4037fe700 1 --2- 172.21.15.133:0/2894387281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4041017a0 0x7fc404072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.059+0000 7fc4037fe700 1 -- 172.21.15.133:0/2894387281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc404100e10 msgr2=0x7fc404072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:27.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.059+0000 7fc4037fe700 1 --2- 172.21.15.133:0/2894387281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc404100e10 0x7fc404072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.059+0000 7fc4037fe700 1 -- 172.21.15.133:0/2894387281 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f80095f0 con 0x7fc4040ffc20 2023-07-14T18:16:27.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.059+0000 7fc402ffd700 1 --2- 172.21.15.133:0/2894387281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc404100e10 0x7fc404072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:16:27.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.059+0000 7fc4037fe700 1 --2- 172.21.15.133:0/2894387281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc4040ffc20 0x7fc4040743f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc3f400e970 tx=0x7fc3f400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:27.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.059+0000 7fc400ff9700 1 -- 172.21.15.133:0/2894387281 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4009790 con 0x7fc4040ffc20 2023-07-14T18:16:27.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.059+0000 7fc409bff700 1 -- 172.21.15.133:0/2894387281 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc40419fcb0 con 0x7fc4040ffc20 2023-07-14T18:16:27.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.059+0000 7fc400ff9700 1 -- 172.21.15.133:0/2894387281 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3f400c7e0 con 0x7fc4040ffc20 2023-07-14T18:16:27.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.059+0000 7fc400ff9700 1 -- 172.21.15.133:0/2894387281 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4003c70 con 0x7fc4040ffc20 2023-07-14T18:16:27.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.059+0000 7fc409bff700 1 -- 172.21.15.133:0/2894387281 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4041a01c0 con 0x7fc4040ffc20 2023-07-14T18:16:27.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.060+0000 7fc409bff700 1 -- 172.21.15.133:0/2894387281 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc40410b440 con 0x7fc4040ffc20 2023-07-14T18:16:27.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.062+0000 7fc400ff9700 1 -- 172.21.15.133:0/2894387281 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fc3f400cc50 con 0x7fc4040ffc20 2023-07-14T18:16:27.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.064+0000 7fc400ff9700 1 --2- 172.21.15.133:0/2894387281 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc3ec0a1150 0x7fc3ec0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:27.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.064+0000 7fc400ff9700 1 -- 172.21.15.133:0/2894387281 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc3f4014070 con 0x7fc4040ffc20 2023-07-14T18:16:27.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.064+0000 7fc402ffd700 1 --2- 172.21.15.133:0/2894387281 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc3ec0a1150 0x7fc3ec0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:27.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.064+0000 7fc402ffd700 1 --2- 172.21.15.133:0/2894387281 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc3ec0a1150 0x7fc3ec0a3600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc3f800b2e0 tx=0x7fc3f8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:27.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.066+0000 7fc400ff9700 1 -- 172.21.15.133:0/2894387281 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fc3f4096800 con 0x7fc4040ffc20 2023-07-14T18:16:27.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:26 smithi133 ceph-mon[126384]: pgmap v341: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2023-07-14T18:16:27.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:26 smithi133 ceph-mon[126384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2023-07-14T18:16:27.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:26 smithi133 ceph-mon[126384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:16:27.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:26 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/816313558' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:16:27.158 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:26 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/297190997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:16:27.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.264+0000 7fc409bff700 1 -- 172.21.15.133:0/2894387281 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc404106b20 con 0x7fc4040ffc20 2023-07-14T18:16:27.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.264+0000 7fc400ff9700 1 -- 172.21.15.133:0/2894387281 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc3f4003dd0 con 0x7fc4040ffc20 2023-07-14T18:16:27.310 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-07-14T18:16:27.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.267+0000 7fc409bff700 1 -- 172.21.15.133:0/2894387281 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc3ec0a1150 msgr2=0x7fc3ec0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:27.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.267+0000 7fc409bff700 1 --2- 172.21.15.133:0/2894387281 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc3ec0a1150 0x7fc3ec0a3600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc3f800b2e0 tx=0x7fc3f8005fb0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.267+0000 7fc409bff700 1 -- 172.21.15.133:0/2894387281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc4040ffc20 msgr2=0x7fc4040743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:27.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.267+0000 7fc409bff700 1 --2- 172.21.15.133:0/2894387281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc4040ffc20 0x7fc4040743f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc3f400e970 tx=0x7fc3f400ec80 comp rx=0 tx=0).stop 2023-07-14T18:16:27.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.267+0000 7fc409bff700 1 -- 172.21.15.133:0/2894387281 shutdown_connections 2023-07-14T18:16:27.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.267+0000 7fc409bff700 1 --2- 172.21.15.133:0/2894387281 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc3ec0a1150 0x7fc3ec0a3600 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.267+0000 7fc409bff700 1 --2- 172.21.15.133:0/2894387281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4041017a0 0x7fc404072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.267+0000 7fc409bff700 1 --2- 172.21.15.133:0/2894387281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc4040ffc20 0x7fc4040743f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.267+0000 7fc409bff700 1 --2- 172.21.15.133:0/2894387281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc404100e10 0x7fc404072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.267+0000 7fc409bff700 1 -- 172.21.15.133:0/2894387281 >> 172.21.15.133:0/2894387281 conn(0x7fc4040fb260 msgr2=0x7fc4040fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:27.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.267+0000 7fc409bff700 1 -- 172.21.15.133:0/2894387281 shutdown_connections 2023-07-14T18:16:27.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.267+0000 7fc409bff700 1 -- 172.21.15.133:0/2894387281 wait complete. 2023-07-14T18:16:27.331 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T18:16:27.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:26 smithi161 ceph-mon[128993]: pgmap v341: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2023-07-14T18:16:27.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:26 smithi161 ceph-mon[128993]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2023-07-14T18:16:27.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:26 smithi161 ceph-mon[128993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:16:27.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:26 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/816313558' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:16:27.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:26 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/297190997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:16:27.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:26 smithi171 ceph-mon[129241]: pgmap v341: 129 pgs: 129 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2023-07-14T18:16:27.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:26 smithi171 ceph-mon[129241]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2023-07-14T18:16:27.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:26 smithi171 ceph-mon[129241]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:16:27.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:26 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/816313558' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:16:27.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:26 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/297190997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:16:27.530 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.485+0000 7fbc4f9ff700 1 -- 172.21.15.133:0/3066604298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc480ffc10 msgr2=0x7fbc48100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:27.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.485+0000 7fbc4f9ff700 1 --2- 172.21.15.133:0/3066604298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc480ffc10 0x7fbc48100020 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009990 tx=0x7fbc44009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.485+0000 7fbc4f9ff700 1 -- 172.21.15.133:0/3066604298 shutdown_connections 2023-07-14T18:16:27.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.485+0000 7fbc4f9ff700 1 --2- 172.21.15.133:0/3066604298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc48101790 0x7fbc48108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.485+0000 7fbc4f9ff700 1 --2- 172.21.15.133:0/3066604298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc480ffc10 0x7fbc48100020 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.485+0000 7fbc4f9ff700 1 --2- 172.21.15.133:0/3066604298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc48100e00 0x7fbc48101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.485+0000 7fbc4f9ff700 1 -- 172.21.15.133:0/3066604298 >> 172.21.15.133:0/3066604298 conn(0x7fbc480fb210 msgr2=0x7fbc480fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:27.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.485+0000 7fbc4f9ff700 1 -- 172.21.15.133:0/3066604298 shutdown_connections 2023-07-14T18:16:27.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.485+0000 7fbc4f9ff700 1 -- 172.21.15.133:0/3066604298 wait complete. 2023-07-14T18:16:27.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.486+0000 7fbc4f9ff700 1 Processor -- start 2023-07-14T18:16:27.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.486+0000 7fbc4f9ff700 1 -- start start 2023-07-14T18:16:27.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.487+0000 7fbc4f9ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc480ffc10 0x7fbc48195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:27.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.487+0000 7fbc4f9ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc48100e00 0x7fbc48195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:27.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.487+0000 7fbc4f9ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc48101790 0x7fbc4819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:27.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.487+0000 7fbc4f9ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4819d4e0 con 0x7fbc48100e00 2023-07-14T18:16:27.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.487+0000 7fbc4f9ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4819d620 con 0x7fbc48101790 2023-07-14T18:16:27.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.487+0000 7fbc4f9ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4819d7b0 con 0x7fbc480ffc10 2023-07-14T18:16:27.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.487+0000 7fbc4cf9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc48100e00 0x7fbc48195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:27.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.487+0000 7fbc4cf9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc48100e00 0x7fbc48195ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43512/0 (socket says 172.21.15.133:43512) 2023-07-14T18:16:27.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.487+0000 7fbc4cf9a700 1 -- 172.21.15.133:0/694292524 learned_addr learned my addr 172.21.15.133:0/694292524 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:27.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.487+0000 7fbc4d79b700 1 --2- 172.21.15.133:0/694292524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc480ffc10 0x7fbc48195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:27.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.487+0000 7fbc4df9c700 1 --2- 172.21.15.133:0/694292524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc48101790 0x7fbc4819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:27.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.487+0000 7fbc4cf9a700 1 -- 172.21.15.133:0/694292524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc480ffc10 msgr2=0x7fbc48195960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:27.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.487+0000 7fbc4cf9a700 1 --2- 172.21.15.133:0/694292524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc480ffc10 0x7fbc48195960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.487+0000 7fbc4cf9a700 1 -- 172.21.15.133:0/694292524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc48101790 msgr2=0x7fbc4819cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:27.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.487+0000 7fbc4cf9a700 1 --2- 172.21.15.133:0/694292524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc48101790 0x7fbc4819cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.487+0000 7fbc4cf9a700 1 -- 172.21.15.133:0/694292524 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc440095f0 con 0x7fbc48100e00 2023-07-14T18:16:27.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.487+0000 7fbc4d79b700 1 --2- 172.21.15.133:0/694292524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc480ffc10 0x7fbc48195960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:16:27.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.487+0000 7fbc4df9c700 1 --2- 172.21.15.133:0/694292524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc48101790 0x7fbc4819cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:16:27.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.488+0000 7fbc4cf9a700 1 --2- 172.21.15.133:0/694292524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc48100e00 0x7fbc48195ea0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800e9a0 tx=0x7fbc3800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:27.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.488+0000 7fbc3e7fc700 1 -- 172.21.15.133:0/694292524 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc38009790 con 0x7fbc48100e00 2023-07-14T18:16:27.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.488+0000 7fbc4f9ff700 1 -- 172.21.15.133:0/694292524 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc4819da90 con 0x7fbc48100e00 2023-07-14T18:16:27.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.488+0000 7fbc3e7fc700 1 -- 172.21.15.133:0/694292524 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc3800cbd0 con 0x7fbc48100e00 2023-07-14T18:16:27.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.488+0000 7fbc3e7fc700 1 -- 172.21.15.133:0/694292524 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc38018680 con 0x7fbc48100e00 2023-07-14T18:16:27.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.488+0000 7fbc4f9ff700 1 -- 172.21.15.133:0/694292524 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc4819dfa0 con 0x7fbc48100e00 2023-07-14T18:16:27.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.489+0000 7fbc4f9ff700 1 -- 172.21.15.133:0/694292524 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc48106b90 con 0x7fbc48100e00 2023-07-14T18:16:27.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.491+0000 7fbc3e7fc700 1 -- 172.21.15.133:0/694292524 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fbc3800cd40 con 0x7fbc48100e00 2023-07-14T18:16:27.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.492+0000 7fbc3e7fc700 1 --2- 172.21.15.133:0/694292524 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fbc340a1150 0x7fbc340a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:27.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.492+0000 7fbc3e7fc700 1 -- 172.21.15.133:0/694292524 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbc38014070 con 0x7fbc48100e00 2023-07-14T18:16:27.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.492+0000 7fbc4d79b700 1 --2- 172.21.15.133:0/694292524 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fbc340a1150 0x7fbc340a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:27.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.493+0000 7fbc4d79b700 1 --2- 172.21.15.133:0/694292524 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fbc340a1150 0x7fbc340a3600 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400b2e0 tx=0x7fbc44005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:27.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.494+0000 7fbc3e7fc700 1 -- 172.21.15.133:0/694292524 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fbc38096590 con 0x7fbc48100e00 2023-07-14T18:16:27.735 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.690+0000 7fbc4f9ff700 1 -- 172.21.15.133:0/694292524 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbc4802ca70 con 0x7fbc48100e00 2023-07-14T18:16:27.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.690+0000 7fbc3e7fc700 1 -- 172.21.15.133:0/694292524 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbc38094e60 con 0x7fbc48100e00 2023-07-14T18:16:27.736 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2023-07-14T18:16:27.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.693+0000 7fbc4f9ff700 1 -- 172.21.15.133:0/694292524 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fbc340a1150 msgr2=0x7fbc340a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:27.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.693+0000 7fbc4f9ff700 1 --2- 172.21.15.133:0/694292524 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fbc340a1150 0x7fbc340a3600 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400b2e0 tx=0x7fbc44005fb0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.693+0000 7fbc4f9ff700 1 -- 172.21.15.133:0/694292524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc48100e00 msgr2=0x7fbc48195ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:27.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.693+0000 7fbc4f9ff700 1 --2- 172.21.15.133:0/694292524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc48100e00 0x7fbc48195ea0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800e9a0 tx=0x7fbc3800ecb0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.693+0000 7fbc4f9ff700 1 -- 172.21.15.133:0/694292524 shutdown_connections 2023-07-14T18:16:27.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.693+0000 7fbc4f9ff700 1 --2- 172.21.15.133:0/694292524 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fbc340a1150 0x7fbc340a3600 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.693+0000 7fbc4f9ff700 1 --2- 172.21.15.133:0/694292524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc48101790 0x7fbc4819cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.693+0000 7fbc4f9ff700 1 --2- 172.21.15.133:0/694292524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc480ffc10 0x7fbc48195960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.693+0000 7fbc4f9ff700 1 --2- 172.21.15.133:0/694292524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc48100e00 0x7fbc48195ea0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.693+0000 7fbc4f9ff700 1 -- 172.21.15.133:0/694292524 >> 172.21.15.133:0/694292524 conn(0x7fbc480fb210 msgr2=0x7fbc480fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:27.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.693+0000 7fbc4f9ff700 1 -- 172.21.15.133:0/694292524 shutdown_connections 2023-07-14T18:16:27.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.693+0000 7fbc4f9ff700 1 -- 172.21.15.133:0/694292524 wait complete. 2023-07-14T18:16:27.756 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T18:16:27.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.911+0000 7f58211ff700 1 -- 172.21.15.133:0/3157720548 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f581c100e10 msgr2=0x7f581c101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:27.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.911+0000 7f58211ff700 1 --2- 172.21.15.133:0/3157720548 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f581c100e10 0x7f581c101260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f58100098f0 tx=0x7f5810009c00 comp rx=0 tx=0).stop 2023-07-14T18:16:27.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.911+0000 7f58211ff700 1 -- 172.21.15.133:0/3157720548 shutdown_connections 2023-07-14T18:16:27.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.911+0000 7f58211ff700 1 --2- 172.21.15.133:0/3157720548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f581c0ffc20 0x7f581c100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.911+0000 7f58211ff700 1 --2- 172.21.15.133:0/3157720548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f581c1017a0 0x7f581c108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.911+0000 7f58211ff700 1 --2- 172.21.15.133:0/3157720548 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f581c100e10 0x7f581c101260 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.911+0000 7f58211ff700 1 -- 172.21.15.133:0/3157720548 >> 172.21.15.133:0/3157720548 conn(0x7f581c0fb260 msgr2=0x7f581c0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:27.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.911+0000 7f58211ff700 1 -- 172.21.15.133:0/3157720548 shutdown_connections 2023-07-14T18:16:27.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.911+0000 7f58211ff700 1 -- 172.21.15.133:0/3157720548 wait complete. 2023-07-14T18:16:27.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.912+0000 7f58211ff700 1 Processor -- start 2023-07-14T18:16:27.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.912+0000 7f58211ff700 1 -- start start 2023-07-14T18:16:27.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.912+0000 7f58211ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f581c0ffc20 0x7f581c197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:27.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.912+0000 7f58211ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f581c100e10 0x7f581c198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:27.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.912+0000 7f58211ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f581c1017a0 0x7f581c19f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:27.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.912+0000 7f58211ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c19f740 con 0x7f581c1017a0 2023-07-14T18:16:27.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.912+0000 7f58211ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c19f880 con 0x7f581c0ffc20 2023-07-14T18:16:27.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.912+0000 7f58211ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c19fa10 con 0x7f581c100e10 2023-07-14T18:16:27.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.912+0000 7f581ad9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f581c0ffc20 0x7f581c197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:27.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.913+0000 7f581a59c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f581c100e10 0x7f581c198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:27.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.913+0000 7f581ad9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f581c0ffc20 0x7f581c197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:33536/0 (socket says 172.21.15.133:33536) 2023-07-14T18:16:27.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.913+0000 7f581ad9d700 1 -- 172.21.15.133:0/3783347242 learned_addr learned my addr 172.21.15.133:0/3783347242 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:27.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.913+0000 7f581b59e700 1 --2- 172.21.15.133:0/3783347242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f581c1017a0 0x7f581c19f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:27.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.913+0000 7f581a59c700 1 -- 172.21.15.133:0/3783347242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f581c0ffc20 msgr2=0x7f581c197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:27.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.913+0000 7f581a59c700 1 --2- 172.21.15.133:0/3783347242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f581c0ffc20 0x7f581c197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.913+0000 7f581a59c700 1 -- 172.21.15.133:0/3783347242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f581c1017a0 msgr2=0x7f581c19f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:27.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.913+0000 7f581a59c700 1 --2- 172.21.15.133:0/3783347242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f581c1017a0 0x7f581c19f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:27.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.913+0000 7f581a59c700 1 -- 172.21.15.133:0/3783347242 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c0095f0 con 0x7f581c100e10 2023-07-14T18:16:27.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.913+0000 7f581a59c700 1 --2- 172.21.15.133:0/3783347242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f581c100e10 0x7f581c198100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5810009230 tx=0x7f581000b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:27.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.913+0000 7f5803fff700 1 -- 172.21.15.133:0/3783347242 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581001c070 con 0x7f581c100e10 2023-07-14T18:16:27.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.913+0000 7f58211ff700 1 -- 172.21.15.133:0/3783347242 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5814009c40 con 0x7f581c100e10 2023-07-14T18:16:27.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.914+0000 7f5803fff700 1 -- 172.21.15.133:0/3783347242 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f581000f7d0 con 0x7f581c100e10 2023-07-14T18:16:27.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.914+0000 7f58211ff700 1 -- 172.21.15.133:0/3783347242 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5810009520 con 0x7f581c100e10 2023-07-14T18:16:27.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.914+0000 7f5803fff700 1 -- 172.21.15.133:0/3783347242 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5810003970 con 0x7f581c100e10 2023-07-14T18:16:27.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.914+0000 7f58211ff700 1 -- 172.21.15.133:0/3783347242 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f581c067430 con 0x7f581c100e10 2023-07-14T18:16:27.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.917+0000 7f5803fff700 1 -- 172.21.15.133:0/3783347242 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f5810020030 con 0x7f581c100e10 2023-07-14T18:16:27.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.918+0000 7f5803fff700 1 --2- 172.21.15.133:0/3783347242 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f58040a11d0 0x7f58040a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:27.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.918+0000 7f5803fff700 1 -- 172.21.15.133:0/3783347242 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f58100cdd20 con 0x7f581c100e10 2023-07-14T18:16:27.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.918+0000 7f581ad9d700 1 --2- 172.21.15.133:0/3783347242 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f58040a11d0 0x7f58040a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:27.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.919+0000 7f581ad9d700 1 --2- 172.21.15.133:0/3783347242 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f58040a11d0 0x7f58040a3680 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f580c005950 tx=0x7f580c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:27.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:27.920+0000 7f5803fff700 1 -- 172.21.15.133:0/3783347242 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f5810097290 con 0x7f581c100e10 2023-07-14T18:16:28.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:27 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2894387281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:16:28.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:27 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/694292524' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:16:28.168 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.123+0000 7f58211ff700 1 -- 172.21.15.133:0/3783347242 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f581c0673c0 con 0x7f581c100e10 2023-07-14T18:16:28.169 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.124+0000 7f5803fff700 1 -- 172.21.15.133:0/3783347242 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f581009a8b0 con 0x7f581c100e10 2023-07-14T18:16:28.169 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-07-14T18:16:28.171 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.126+0000 7f58211ff700 1 -- 172.21.15.133:0/3783347242 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f58040a11d0 msgr2=0x7f58040a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:28.171 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.126+0000 7f58211ff700 1 --2- 172.21.15.133:0/3783347242 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f58040a11d0 0x7f58040a3680 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f580c005950 tx=0x7f580c009310 comp rx=0 tx=0).stop 2023-07-14T18:16:28.172 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.126+0000 7f58211ff700 1 -- 172.21.15.133:0/3783347242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f581c100e10 msgr2=0x7f581c198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:28.172 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.126+0000 7f58211ff700 1 --2- 172.21.15.133:0/3783347242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f581c100e10 0x7f581c198100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5810009230 tx=0x7f581000b790 comp rx=0 tx=0).stop 2023-07-14T18:16:28.172 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.126+0000 7f58211ff700 1 -- 172.21.15.133:0/3783347242 shutdown_connections 2023-07-14T18:16:28.172 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.126+0000 7f58211ff700 1 --2- 172.21.15.133:0/3783347242 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f58040a11d0 0x7f58040a3680 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:28.172 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.126+0000 7f58211ff700 1 --2- 172.21.15.133:0/3783347242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f581c0ffc20 0x7f581c197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:28.172 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.126+0000 7f58211ff700 1 --2- 172.21.15.133:0/3783347242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f581c1017a0 0x7f581c19f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:28.172 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.126+0000 7f58211ff700 1 --2- 172.21.15.133:0/3783347242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f581c100e10 0x7f581c198100 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:28.173 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.126+0000 7f58211ff700 1 -- 172.21.15.133:0/3783347242 >> 172.21.15.133:0/3783347242 conn(0x7f581c0fb260 msgr2=0x7f581c0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:28.173 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.126+0000 7f58211ff700 1 -- 172.21.15.133:0/3783347242 shutdown_connections 2023-07-14T18:16:28.173 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.126+0000 7f58211ff700 1 -- 172.21.15.133:0/3783347242 wait complete. 2023-07-14T18:16:28.190 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T18:16:28.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:27 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2894387281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:16:28.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:27 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/694292524' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:16:28.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:27 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2894387281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:16:28.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:27 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/694292524' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:16:28.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.343+0000 7f12e09ff700 1 -- 172.21.15.133:0/132933750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12dc0ffc30 msgr2=0x7f12dc100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:28.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.343+0000 7f12e09ff700 1 --2- 172.21.15.133:0/132933750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12dc0ffc30 0x7f12dc100040 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009990 tx=0x7f12cc009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:28.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.343+0000 7f12e09ff700 1 -- 172.21.15.133:0/132933750 shutdown_connections 2023-07-14T18:16:28.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.343+0000 7f12e09ff700 1 --2- 172.21.15.133:0/132933750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12dc1017b0 0x7f12dc108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:28.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.343+0000 7f12e09ff700 1 --2- 172.21.15.133:0/132933750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12dc0ffc30 0x7f12dc100040 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:28.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.343+0000 7f12e09ff700 1 --2- 172.21.15.133:0/132933750 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12dc100e20 0x7f12dc101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:28.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.343+0000 7f12e09ff700 1 -- 172.21.15.133:0/132933750 >> 172.21.15.133:0/132933750 conn(0x7f12dc0fb210 msgr2=0x7f12dc0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:28.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.343+0000 7f12e09ff700 1 -- 172.21.15.133:0/132933750 shutdown_connections 2023-07-14T18:16:28.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.344+0000 7f12e09ff700 1 -- 172.21.15.133:0/132933750 wait complete. 2023-07-14T18:16:28.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.344+0000 7f12e09ff700 1 Processor -- start 2023-07-14T18:16:28.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.345+0000 7f12e09ff700 1 -- start start 2023-07-14T18:16:28.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.345+0000 7f12e09ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12dc0ffc30 0x7f12dc197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:28.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.345+0000 7f12e09ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12dc100e20 0x7f12dc198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:28.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.345+0000 7f12e09ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12dc1017b0 0x7f12dc19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:28.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.345+0000 7f12e09ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc19f6d0 con 0x7f12dc0ffc30 2023-07-14T18:16:28.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.345+0000 7f12e09ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc19f810 con 0x7f12dc1017b0 2023-07-14T18:16:28.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.345+0000 7f12e09ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc19f9a0 con 0x7f12dc100e20 2023-07-14T18:16:28.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.345+0000 7f12da59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12dc0ffc30 0x7f12dc197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:28.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.345+0000 7f12da59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12dc0ffc30 0x7f12dc197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43558/0 (socket says 172.21.15.133:43558) 2023-07-14T18:16:28.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.345+0000 7f12da59c700 1 -- 172.21.15.133:0/1180550213 learned_addr learned my addr 172.21.15.133:0/1180550213 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:28.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.345+0000 7f12d9d9b700 1 --2- 172.21.15.133:0/1180550213 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12dc100e20 0x7f12dc198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:28.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.345+0000 7f12dad9d700 1 --2- 172.21.15.133:0/1180550213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12dc1017b0 0x7f12dc19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:28.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.346+0000 7f12da59c700 1 -- 172.21.15.133:0/1180550213 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12dc100e20 msgr2=0x7f12dc198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:28.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.346+0000 7f12da59c700 1 --2- 172.21.15.133:0/1180550213 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12dc100e20 0x7f12dc198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:28.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.346+0000 7f12da59c700 1 -- 172.21.15.133:0/1180550213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12dc1017b0 msgr2=0x7f12dc19f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:28.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.346+0000 7f12da59c700 1 --2- 172.21.15.133:0/1180550213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12dc1017b0 0x7f12dc19f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:28.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.346+0000 7f12da59c700 1 -- 172.21.15.133:0/1180550213 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d0009520 con 0x7f12dc0ffc30 2023-07-14T18:16:28.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.346+0000 7f12da59c700 1 --2- 172.21.15.133:0/1180550213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12dc0ffc30 0x7f12dc197b50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f12cc000c00 tx=0x7f12cc0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:28.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.346+0000 7f12cb7fe700 1 -- 172.21.15.133:0/1180550213 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc01d070 con 0x7f12dc0ffc30 2023-07-14T18:16:28.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.346+0000 7f12e09ff700 1 -- 172.21.15.133:0/1180550213 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12cc0095f0 con 0x7f12dc0ffc30 2023-07-14T18:16:28.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.346+0000 7f12cb7fe700 1 -- 172.21.15.133:0/1180550213 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12cc021410 con 0x7f12dc0ffc30 2023-07-14T18:16:28.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.346+0000 7f12cb7fe700 1 -- 172.21.15.133:0/1180550213 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc017650 con 0x7f12dc0ffc30 2023-07-14T18:16:28.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.346+0000 7f12e09ff700 1 -- 172.21.15.133:0/1180550213 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12dc19ff40 con 0x7f12dc0ffc30 2023-07-14T18:16:28.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.347+0000 7f12e09ff700 1 -- 172.21.15.133:0/1180550213 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12dc1922c0 con 0x7f12dc0ffc30 2023-07-14T18:16:28.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.349+0000 7f12cb7fe700 1 -- 172.21.15.133:0/1180550213 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f12cc021580 con 0x7f12dc0ffc30 2023-07-14T18:16:28.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.350+0000 7f12cb7fe700 1 --2- 172.21.15.133:0/1180550213 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f12c40a11a0 0x7f12c40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:28.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.351+0000 7f12cb7fe700 1 -- 172.21.15.133:0/1180550213 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f12cc0cdff0 con 0x7f12dc0ffc30 2023-07-14T18:16:28.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.351+0000 7f12d9d9b700 1 --2- 172.21.15.133:0/1180550213 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f12c40a11a0 0x7f12c40a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:28.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.351+0000 7f12d9d9b700 1 --2- 172.21.15.133:0/1180550213 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f12c40a11a0 0x7f12c40a3650 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009df0 tx=0x7f12d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:28.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.354+0000 7f12cb7fe700 1 -- 172.21.15.133:0/1180550213 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f12cc097460 con 0x7f12dc0ffc30 2023-07-14T18:16:28.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.556+0000 7f12e09ff700 1 -- 172.21.15.133:0/1180550213 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12dc02c8e0 con 0x7f12dc0ffc30 2023-07-14T18:16:28.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.558+0000 7f12cb7fe700 1 -- 172.21.15.133:0/1180550213 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24520 (secure 0 0 0) 0x7f12cc09aa80 con 0x7f12dc0ffc30 2023-07-14T18:16:28.604 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:28.604 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":92,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","created":"2023-07-14T18:06:49.561582+0000","modified":"2023-07-14T18:15:49.951050+0000","last_up_change":"2023-07-14T18:13:29.836099+0000","last_in_change":"2023-07-14T18:13:11.606710+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T18:10:03.945039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T18:14:15.451340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T18:14:16.892215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T18:14:20.493175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"85a29de7-d712-422a-b647-d9e90091aaf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6803","nonce":3076510801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6805","nonce":3076510801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6809","nonce":3076510801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6807","nonce":3076510801}]},"public_addr":"172.21.15.133:6803/3076510801","cluster_addr":"172.21.15.133:6805/3076510801","heartbeat_back_addr":"172.21.15.133:6809/3076510801","heartbeat_front_addr":"172.21.15.133:6807/3076510801","state":["exists","up"]},{"osd":1,"uuid":"80ad4e78-7917-495e-b119-99f7c7c36243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6801","nonce":1841901852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6803","nonce":1841901852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6807","nonce":1841901852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6805","nonce":1841901852}]},"public_addr":"172.21.15.161:6801/1841901852","cluster_addr":"172.21.15.161:6803/1841901852","heartbeat_back_addr":"172.21.15.161:6807/1841901852","heartbeat_front_addr":"172.21.15.161:6805/1841901852","state":["exists","up"]},{"osd":2,"uuid":"fc4bfd37-1fdd-47c6-a48e-6d910daf9e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6801","nonce":346103099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6803","nonce":346103099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6807","nonce":346103099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6805","nonce":346103099}]},"public_addr":"172.21.15.171:6801/346103099","cluster_addr":"172.21.15.171:6803/346103099","heartbeat_back_addr":"172.21.15.171:6807/346103099","heartbeat_front_addr":"172.21.15.171:6805/346103099","state":["exists","up"]},{"osd":3,"uuid":"4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6811","nonce":2152994143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6813","nonce":2152994143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6817","nonce":2152994143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6815","nonce":2152994143}]},"public_addr":"172.21.15.133:6811/2152994143","cluster_addr":"172.21.15.133:6813/2152994143","heartbeat_back_addr":"172.21.15.133:6817/2152994143","heartbeat_front_addr":"172.21.15.133:6815/2152994143","state":["exists","up"]},{"osd":4,"uuid":"4601a1c2-254b-4c4d-9b15-8fc952b3d2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6809","nonce":47213291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6811","nonce":47213291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6815","nonce":47213291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6813","nonce":47213291}]},"public_addr":"172.21.15.161:6809/47213291","cluster_addr":"172.21.15.161:6811/47213291","heartbeat_back_addr":"172.21.15.161:6815/47213291","heartbeat_front_addr":"172.21.15.161:6813/47213291","state":["exists","up"]},{"osd":5,"uuid":"6cca180a-9140-4e49-835b-86842dae0574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6809","nonce":777556751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6811","nonce":777556751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6815","nonce":777556751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6813","nonce":777556751}]},"public_addr":"172.21.15.171:6809/777556751","cluster_addr":"172.21.15.171:6811/777556751","heartbeat_back_addr":"172.21.15.171:6815/777556751","heartbeat_front_addr":"172.21.15.171:6813/777556751","state":["exists","up"]},{"osd":6,"uuid":"ea0299ff-7740-4494-a3a9-1503b3fc0ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6819","nonce":2884152358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6821","nonce":2884152358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6825","nonce":2884152358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6823","nonce":2884152358}]},"public_addr":"172.21.15.133:6819/2884152358","cluster_addr":"172.21.15.133:6821/2884152358","heartbeat_back_addr":"172.21.15.133:6825/2884152358","heartbeat_front_addr":"172.21.15.133:6823/2884152358","state":["exists","up"]},{"osd":7,"uuid":"b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6817","nonce":4183705749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6819","nonce":4183705749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6823","nonce":4183705749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6821","nonce":4183705749}]},"public_addr":"172.21.15.161:6817/4183705749","cluster_addr":"172.21.15.161:6819/4183705749","heartbeat_back_addr":"172.21.15.161:6823/4183705749","heartbeat_front_addr":"172.21.15.161:6821/4183705749","state":["exists","up"]},{"osd":8,"uuid":"87dffc07-bf48-48c8-b418-5c4b509686d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6817","nonce":2771494483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6819","nonce":2771494483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6823","nonce":2771494483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6821","nonce":2771494483}]},"public_addr":"172.21.15.171:6817/2771494483","cluster_addr":"172.21.15.171:6819/2771494483","heartbeat_back_addr":"172.21.15.171:6823/2771494483","heartbeat_front_addr":"172.21.15.171:6821/2771494483","state":["exists","up"]},{"osd":9,"uuid":"e0832870-79ce-4bce-8045-58b7a7f6dc69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6827","nonce":2066164149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6829","nonce":2066164149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6833","nonce":2066164149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6831","nonce":2066164149}]},"public_addr":"172.21.15.133:6827/2066164149","cluster_addr":"172.21.15.133:6829/2066164149","heartbeat_back_addr":"172.21.15.133:6833/2066164149","heartbeat_front_addr":"172.21.15.133:6831/2066164149","state":["exists","up"]},{"osd":10,"uuid":"081f0484-d134-4b2e-9c48-ef8e6a646422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6825","nonce":2162333626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6827","nonce":2162333626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6831","nonce":2162333626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6829","nonce":2162333626}]},"public_addr":"172.21.15.161:6825/2162333626","cluster_addr":"172.21.15.161:6827/2162333626","heartbeat_back_addr":"172.21.15.161:6831/2162333626","heartbeat_front_addr":"172.21.15.161:6829/2162333626","state":["exists","up"]},{"osd":11,"uuid":"4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6825","nonce":3403894313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6827","nonce":3403894313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6831","nonce":3403894313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6829","nonce":3403894313}]},"public_addr":"172.21.15.171:6825/3403894313","cluster_addr":"172.21.15.171:6827/3403894313","heartbeat_back_addr":"172.21.15.171:6831/3403894313","heartbeat_front_addr":"172.21.15.171:6829/3403894313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:17.816229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:39.638543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:00.543317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:26.988921+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:48.644694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:10.049860+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:35.986508+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:57.753846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:19.018245+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:44.460450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:06.279960+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:27.955006+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/1832569241":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6801/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:0/4219815304":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:6801/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/432253615":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1840700480":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1807775439":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1089030734":"2023-07-15T18:07:47.346704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T18:16:28.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.561+0000 7f12e09ff700 1 -- 172.21.15.133:0/1180550213 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f12c40a11a0 msgr2=0x7f12c40a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:28.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.561+0000 7f12e09ff700 1 --2- 172.21.15.133:0/1180550213 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f12c40a11a0 0x7f12c40a3650 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009df0 tx=0x7f12d0009260 comp rx=0 tx=0).stop 2023-07-14T18:16:28.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.561+0000 7f12e09ff700 1 -- 172.21.15.133:0/1180550213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12dc0ffc30 msgr2=0x7f12dc197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:28.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.561+0000 7f12e09ff700 1 --2- 172.21.15.133:0/1180550213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12dc0ffc30 0x7f12dc197b50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f12cc000c00 tx=0x7f12cc0057c0 comp rx=0 tx=0).stop 2023-07-14T18:16:28.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.561+0000 7f12e09ff700 1 -- 172.21.15.133:0/1180550213 shutdown_connections 2023-07-14T18:16:28.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.561+0000 7f12e09ff700 1 --2- 172.21.15.133:0/1180550213 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f12c40a11a0 0x7f12c40a3650 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:28.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.561+0000 7f12e09ff700 1 --2- 172.21.15.133:0/1180550213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12dc1017b0 0x7f12dc19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:28.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.561+0000 7f12e09ff700 1 --2- 172.21.15.133:0/1180550213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12dc0ffc30 0x7f12dc197b50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:28.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.561+0000 7f12e09ff700 1 --2- 172.21.15.133:0/1180550213 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12dc100e20 0x7f12dc198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:28.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.561+0000 7f12e09ff700 1 -- 172.21.15.133:0/1180550213 >> 172.21.15.133:0/1180550213 conn(0x7f12dc0fb210 msgr2=0x7f12dc106710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:28.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.561+0000 7f12e09ff700 1 -- 172.21.15.133:0/1180550213 shutdown_connections 2023-07-14T18:16:28.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.561+0000 7f12e09ff700 1 -- 172.21.15.133:0/1180550213 wait complete. 2023-07-14T18:16:28.625 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T18:10:03.945039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T18:14:15.451340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T18:14:16.892215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T18:14:20.493175+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T18:16:28.625 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T18:16:28.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.781+0000 7fdded7ff700 1 -- 172.21.15.133:0/744182259 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdde8100750 msgr2=0x7fdde8102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:28.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.781+0000 7fdded7ff700 1 --2- 172.21.15.133:0/744182259 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdde8100750 0x7fdde8102b30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fddd8009990 tx=0x7fddd8009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:28.827 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.781+0000 7fdded7ff700 1 -- 172.21.15.133:0/744182259 shutdown_connections 2023-07-14T18:16:28.827 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.781+0000 7fdded7ff700 1 --2- 172.21.15.133:0/744182259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdde8105990 0x7fdde8107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:28.827 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.781+0000 7fdded7ff700 1 --2- 172.21.15.133:0/744182259 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdde8100750 0x7fdde8102b30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:28.827 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.781+0000 7fdded7ff700 1 --2- 172.21.15.133:0/744182259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdde8103070 0x7fdde8105450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:28.827 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.781+0000 7fdded7ff700 1 -- 172.21.15.133:0/744182259 >> 172.21.15.133:0/744182259 conn(0x7fdde80fa130 msgr2=0x7fdde80fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:28.827 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.781+0000 7fdded7ff700 1 -- 172.21.15.133:0/744182259 shutdown_connections 2023-07-14T18:16:28.827 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.781+0000 7fdded7ff700 1 -- 172.21.15.133:0/744182259 wait complete. 2023-07-14T18:16:28.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.782+0000 7fdded7ff700 1 Processor -- start 2023-07-14T18:16:28.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.782+0000 7fdded7ff700 1 -- start start 2023-07-14T18:16:28.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.782+0000 7fdded7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdde8100750 0x7fdde8197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:28.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.782+0000 7fdded7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdde8103070 0x7fdde81980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:28.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.782+0000 7fdded7ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdde8105990 0x7fdde819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:28.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.782+0000 7fdded7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde819f720 con 0x7fdde8103070 2023-07-14T18:16:28.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.782+0000 7fdded7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde819f860 con 0x7fdde8100750 2023-07-14T18:16:28.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.782+0000 7fdded7ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde819f9f0 con 0x7fdde8105990 2023-07-14T18:16:28.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.783+0000 7fdde67fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdde8103070 0x7fdde81980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:28.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.783+0000 7fdde67fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdde8103070 0x7fdde81980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43562/0 (socket says 172.21.15.133:43562) 2023-07-14T18:16:28.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.783+0000 7fdde67fc700 1 -- 172.21.15.133:0/631973120 learned_addr learned my addr 172.21.15.133:0/631973120 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:28.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.783+0000 7fdde6ffd700 1 --2- 172.21.15.133:0/631973120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdde8100750 0x7fdde8197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:28.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.783+0000 7fdde77fe700 1 --2- 172.21.15.133:0/631973120 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdde8105990 0x7fdde819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:28.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.783+0000 7fdde67fc700 1 -- 172.21.15.133:0/631973120 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdde8105990 msgr2=0x7fdde819f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:28.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.783+0000 7fdde67fc700 1 --2- 172.21.15.133:0/631973120 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdde8105990 0x7fdde819f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:28.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.783+0000 7fdde67fc700 1 -- 172.21.15.133:0/631973120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdde8100750 msgr2=0x7fdde8197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:28.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.783+0000 7fdde67fc700 1 --2- 172.21.15.133:0/631973120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdde8100750 0x7fdde8197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:28.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.783+0000 7fdde67fc700 1 -- 172.21.15.133:0/631973120 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde0009c40 con 0x7fdde8103070 2023-07-14T18:16:28.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.783+0000 7fdde67fc700 1 --2- 172.21.15.133:0/631973120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdde8103070 0x7fdde81980e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00e9a0 tx=0x7fdddc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:28.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.783+0000 7fddcffff700 1 -- 172.21.15.133:0/631973120 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc009790 con 0x7fdde8103070 2023-07-14T18:16:28.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.783+0000 7fdded7ff700 1 -- 172.21.15.133:0/631973120 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd80095f0 con 0x7fdde8103070 2023-07-14T18:16:28.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.783+0000 7fddcffff700 1 -- 172.21.15.133:0/631973120 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdddc00ce80 con 0x7fdde8103070 2023-07-14T18:16:28.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.784+0000 7fddcffff700 1 -- 172.21.15.133:0/631973120 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc018680 con 0x7fdde8103070 2023-07-14T18:16:28.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.784+0000 7fdded7ff700 1 -- 172.21.15.133:0/631973120 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde81a0050 con 0x7fdde8103070 2023-07-14T18:16:28.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.785+0000 7fdded7ff700 1 -- 172.21.15.133:0/631973120 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdde810b410 con 0x7fdde8103070 2023-07-14T18:16:28.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.787+0000 7fddcffff700 1 -- 172.21.15.133:0/631973120 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fdddc010430 con 0x7fdde8103070 2023-07-14T18:16:28.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.788+0000 7fddcffff700 1 --2- 172.21.15.133:0/631973120 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fddd00a1150 0x7fddd00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:28.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.788+0000 7fddcffff700 1 -- 172.21.15.133:0/631973120 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdddc014070 con 0x7fdde8103070 2023-07-14T18:16:28.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.788+0000 7fdde6ffd700 1 --2- 172.21.15.133:0/631973120 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fddd00a1150 0x7fddd00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:28.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.789+0000 7fdde6ffd700 1 --2- 172.21.15.133:0/631973120 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fddd00a1150 0x7fddd00a3600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fddd800b2e0 tx=0x7fddd8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:28.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.791+0000 7fddcffff700 1 -- 172.21.15.133:0/631973120 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fdddc051780 con 0x7fdde8103070 2023-07-14T18:16:29.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.992+0000 7fdded7ff700 1 -- 172.21.15.133:0/631973120 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdde81a0600 con 0x7fdde8103070 2023-07-14T18:16:29.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.992+0000 7fddcffff700 1 -- 172.21.15.133:0/631973120 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdddc096220 con 0x7fdde8103070 2023-07-14T18:16:29.038 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-07-14T18:16:29.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.995+0000 7fdded7ff700 1 -- 172.21.15.133:0/631973120 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fddd00a1150 msgr2=0x7fddd00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:29.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.995+0000 7fdded7ff700 1 --2- 172.21.15.133:0/631973120 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fddd00a1150 0x7fddd00a3600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fddd800b2e0 tx=0x7fddd8005fb0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.995+0000 7fdded7ff700 1 -- 172.21.15.133:0/631973120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdde8103070 msgr2=0x7fdde81980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:29.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.995+0000 7fdded7ff700 1 --2- 172.21.15.133:0/631973120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdde8103070 0x7fdde81980e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00e9a0 tx=0x7fdddc00ecb0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.995+0000 7fdded7ff700 1 -- 172.21.15.133:0/631973120 shutdown_connections 2023-07-14T18:16:29.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.995+0000 7fdded7ff700 1 --2- 172.21.15.133:0/631973120 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fddd00a1150 0x7fddd00a3600 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.995+0000 7fdded7ff700 1 --2- 172.21.15.133:0/631973120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdde8100750 0x7fdde8197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.995+0000 7fdded7ff700 1 --2- 172.21.15.133:0/631973120 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdde8105990 0x7fdde819f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.995+0000 7fdded7ff700 1 --2- 172.21.15.133:0/631973120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdde8103070 0x7fdde81980e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.995+0000 7fdded7ff700 1 -- 172.21.15.133:0/631973120 >> 172.21.15.133:0/631973120 conn(0x7fdde80fa130 msgr2=0x7fdde8101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:29.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.995+0000 7fdded7ff700 1 -- 172.21.15.133:0/631973120 shutdown_connections 2023-07-14T18:16:29.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:28.995+0000 7fdded7ff700 1 -- 172.21.15.133:0/631973120 wait complete. 2023-07-14T18:16:29.059 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T18:16:29.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.214+0000 7ff5eddff700 1 -- 172.21.15.133:0/153772010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5e8102fb0 msgr2=0x7ff5e8105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:29.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.214+0000 7ff5eddff700 1 --2- 172.21.15.133:0/153772010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5e8102fb0 0x7ff5e8105390 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc009990 tx=0x7ff5dc009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.215+0000 7ff5eddff700 1 -- 172.21.15.133:0/153772010 shutdown_connections 2023-07-14T18:16:29.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.215+0000 7ff5eddff700 1 --2- 172.21.15.133:0/153772010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5e8100690 0x7ff5e8102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.215+0000 7ff5eddff700 1 --2- 172.21.15.133:0/153772010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5e81058d0 0x7ff5e8107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.215+0000 7ff5eddff700 1 --2- 172.21.15.133:0/153772010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5e8102fb0 0x7ff5e8105390 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.215+0000 7ff5eddff700 1 -- 172.21.15.133:0/153772010 >> 172.21.15.133:0/153772010 conn(0x7ff5e80fa130 msgr2=0x7ff5e80fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:29.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.215+0000 7ff5eddff700 1 -- 172.21.15.133:0/153772010 shutdown_connections 2023-07-14T18:16:29.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.215+0000 7ff5eddff700 1 -- 172.21.15.133:0/153772010 wait complete. 2023-07-14T18:16:29.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.216+0000 7ff5eddff700 1 Processor -- start 2023-07-14T18:16:29.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.216+0000 7ff5eddff700 1 -- start start 2023-07-14T18:16:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.216+0000 7ff5eddff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5e8100690 0x7ff5e8197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.216+0000 7ff5eddff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5e8102fb0 0x7ff5e81980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.216+0000 7ff5eddff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5e81058d0 0x7ff5e819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.216+0000 7ff5eddff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e819f720 con 0x7ff5e8100690 2023-07-14T18:16:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.216+0000 7ff5eddff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e819f860 con 0x7ff5e8102fb0 2023-07-14T18:16:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.216+0000 7ff5eddff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e819f9f0 con 0x7ff5e81058d0 2023-07-14T18:16:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.216+0000 7ff5e77fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5e8100690 0x7ff5e8197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.216+0000 7ff5e77fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5e8100690 0x7ff5e8197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43582/0 (socket says 172.21.15.133:43582) 2023-07-14T18:16:29.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.216+0000 7ff5e77fe700 1 -- 172.21.15.133:0/1394612888 learned_addr learned my addr 172.21.15.133:0/1394612888 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:29.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.216+0000 7ff5e6ffd700 1 --2- 172.21.15.133:0/1394612888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5e8102fb0 0x7ff5e81980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:29.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.217+0000 7ff5e7fff700 1 --2- 172.21.15.133:0/1394612888 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5e81058d0 0x7ff5e819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:29.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.217+0000 7ff5e77fe700 1 -- 172.21.15.133:0/1394612888 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5e81058d0 msgr2=0x7ff5e819f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:29.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.217+0000 7ff5e77fe700 1 --2- 172.21.15.133:0/1394612888 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5e81058d0 0x7ff5e819f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.217+0000 7ff5e77fe700 1 -- 172.21.15.133:0/1394612888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5e8102fb0 msgr2=0x7ff5e81980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:29.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.217+0000 7ff5e77fe700 1 --2- 172.21.15.133:0/1394612888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5e8102fb0 0x7ff5e81980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.217+0000 7ff5e77fe700 1 -- 172.21.15.133:0/1394612888 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5dc0095f0 con 0x7ff5e8100690 2023-07-14T18:16:29.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.217+0000 7ff5e6ffd700 1 --2- 172.21.15.133:0/1394612888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5e8102fb0 0x7ff5e81980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:16:29.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.217+0000 7ff5e77fe700 1 --2- 172.21.15.133:0/1394612888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5e8100690 0x7ff5e8197ba0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800e9a0 tx=0x7ff5d800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:29.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.217+0000 7ff5e4ff9700 1 -- 172.21.15.133:0/1394612888 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8009790 con 0x7ff5e8100690 2023-07-14T18:16:29.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.217+0000 7ff5eddff700 1 -- 172.21.15.133:0/1394612888 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e819fcd0 con 0x7ff5e8100690 2023-07-14T18:16:29.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.217+0000 7ff5e4ff9700 1 -- 172.21.15.133:0/1394612888 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5d8010430 con 0x7ff5e8100690 2023-07-14T18:16:29.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.217+0000 7ff5e4ff9700 1 -- 172.21.15.133:0/1394612888 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8018680 con 0x7ff5e8100690 2023-07-14T18:16:29.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.217+0000 7ff5eddff700 1 -- 172.21.15.133:0/1394612888 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e81a0180 con 0x7ff5e8100690 2023-07-14T18:16:29.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.218+0000 7ff5eddff700 1 -- 172.21.15.133:0/1394612888 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e8107490 con 0x7ff5e8100690 2023-07-14T18:16:29.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.220+0000 7ff5e4ff9700 1 -- 172.21.15.133:0/1394612888 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff5d80105a0 con 0x7ff5e8100690 2023-07-14T18:16:29.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.222+0000 7ff5e4ff9700 1 --2- 172.21.15.133:0/1394612888 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff5d00a1150 0x7ff5d00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:29.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.222+0000 7ff5e4ff9700 1 -- 172.21.15.133:0/1394612888 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff5d8014070 con 0x7ff5e8100690 2023-07-14T18:16:29.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.222+0000 7ff5e6ffd700 1 --2- 172.21.15.133:0/1394612888 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff5d00a1150 0x7ff5d00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:29.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.222+0000 7ff5e6ffd700 1 --2- 172.21.15.133:0/1394612888 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff5d00a1150 0x7ff5d00a3600 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00b2e0 tx=0x7ff5dc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:29.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.225+0000 7ff5e4ff9700 1 -- 172.21.15.133:0/1394612888 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7ff5d8051780 con 0x7ff5e8100690 2023-07-14T18:16:29.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:28 smithi161 ceph-mon[128993]: pgmap v342: 129 pgs: 129 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2023-07-14T18:16:29.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:28 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3783347242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:16:29.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:28 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1180550213' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:16:29.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:28 smithi171 ceph-mon[129241]: pgmap v342: 129 pgs: 129 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2023-07-14T18:16:29.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:28 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3783347242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:16:29.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:28 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1180550213' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:16:29.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:28 smithi133 ceph-mon[126384]: pgmap v342: 129 pgs: 129 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2023-07-14T18:16:29.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:28 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3783347242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:16:29.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:28 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1180550213' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:16:29.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.426+0000 7ff5eddff700 1 -- 172.21.15.133:0/1394612888 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff5e802c8e0 con 0x7ff5e8100690 2023-07-14T18:16:29.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.426+0000 7ff5e4ff9700 1 -- 172.21.15.133:0/1394612888 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff5e802c8e0 con 0x7ff5e8100690 2023-07-14T18:16:29.472 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-07-14T18:16:29.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.428+0000 7ff5eddff700 1 -- 172.21.15.133:0/1394612888 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff5d00a1150 msgr2=0x7ff5d00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:29.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.428+0000 7ff5eddff700 1 --2- 172.21.15.133:0/1394612888 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff5d00a1150 0x7ff5d00a3600 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00b2e0 tx=0x7ff5dc005fb0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.428+0000 7ff5eddff700 1 -- 172.21.15.133:0/1394612888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5e8100690 msgr2=0x7ff5e8197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:29.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.428+0000 7ff5eddff700 1 --2- 172.21.15.133:0/1394612888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5e8100690 0x7ff5e8197ba0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800e9a0 tx=0x7ff5d800ecb0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.429+0000 7ff5eddff700 1 -- 172.21.15.133:0/1394612888 shutdown_connections 2023-07-14T18:16:29.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.429+0000 7ff5eddff700 1 --2- 172.21.15.133:0/1394612888 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff5d00a1150 0x7ff5d00a3600 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.429+0000 7ff5eddff700 1 --2- 172.21.15.133:0/1394612888 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5e81058d0 0x7ff5e819f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.429+0000 7ff5eddff700 1 --2- 172.21.15.133:0/1394612888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5e8100690 0x7ff5e8197ba0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.429+0000 7ff5eddff700 1 --2- 172.21.15.133:0/1394612888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5e8102fb0 0x7ff5e81980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.429+0000 7ff5eddff700 1 -- 172.21.15.133:0/1394612888 >> 172.21.15.133:0/1394612888 conn(0x7ff5e80fa130 msgr2=0x7ff5e80fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:29.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.429+0000 7ff5eddff700 1 -- 172.21.15.133:0/1394612888 shutdown_connections 2023-07-14T18:16:29.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.429+0000 7ff5eddff700 1 -- 172.21.15.133:0/1394612888 wait complete. 2023-07-14T18:16:29.492 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T18:16:29.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.647+0000 7f0cc31ff700 1 -- 172.21.15.133:0/1554595119 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cbc100e10 msgr2=0x7f0cbc101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:29.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.647+0000 7f0cc31ff700 1 --2- 172.21.15.133:0/1554595119 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cbc100e10 0x7f0cbc101260 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8009990 tx=0x7f0ca8009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.647+0000 7f0cc31ff700 1 -- 172.21.15.133:0/1554595119 shutdown_connections 2023-07-14T18:16:29.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.647+0000 7f0cc31ff700 1 --2- 172.21.15.133:0/1554595119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cbc0ffc20 0x7f0cbc100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.647+0000 7f0cc31ff700 1 --2- 172.21.15.133:0/1554595119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cbc1017a0 0x7f0cbc108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.647+0000 7f0cc31ff700 1 --2- 172.21.15.133:0/1554595119 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cbc100e10 0x7f0cbc101260 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.694 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.647+0000 7f0cc31ff700 1 -- 172.21.15.133:0/1554595119 >> 172.21.15.133:0/1554595119 conn(0x7f0cbc0fb260 msgr2=0x7f0cbc0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:29.694 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.647+0000 7f0cc31ff700 1 -- 172.21.15.133:0/1554595119 shutdown_connections 2023-07-14T18:16:29.694 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.647+0000 7f0cc31ff700 1 -- 172.21.15.133:0/1554595119 wait complete. 2023-07-14T18:16:29.694 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.649+0000 7f0cc31ff700 1 Processor -- start 2023-07-14T18:16:29.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.649+0000 7f0cc31ff700 1 -- start start 2023-07-14T18:16:29.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.649+0000 7f0cc31ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cbc0ffc20 0x7f0cbc0743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:29.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.649+0000 7f0cc31ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cbc100e10 0x7f0cbc072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:29.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.649+0000 7f0cc31ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cbc1017a0 0x7f0cbc072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:29.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.649+0000 7f0cc31ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc073710 con 0x7f0cbc100e10 2023-07-14T18:16:29.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.649+0000 7f0cc31ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc073880 con 0x7f0cbc1017a0 2023-07-14T18:16:29.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.649+0000 7f0cc31ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc19f9a0 con 0x7f0cbc0ffc20 2023-07-14T18:16:29.696 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.650+0000 7f0cbbfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cbc100e10 0x7f0cbc072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:29.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.650+0000 7f0cbbfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cbc100e10 0x7f0cbc072a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43592/0 (socket says 172.21.15.133:43592) 2023-07-14T18:16:29.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.650+0000 7f0cbbfff700 1 -- 172.21.15.133:0/827492326 learned_addr learned my addr 172.21.15.133:0/827492326 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:29.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.650+0000 7f0cc0f9b700 1 --2- 172.21.15.133:0/827492326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cbc0ffc20 0x7f0cbc0743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:29.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.650+0000 7f0cc179c700 1 --2- 172.21.15.133:0/827492326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cbc1017a0 0x7f0cbc072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:29.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.650+0000 7f0cbbfff700 1 -- 172.21.15.133:0/827492326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cbc0ffc20 msgr2=0x7f0cbc0743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:29.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.650+0000 7f0cbbfff700 1 --2- 172.21.15.133:0/827492326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cbc0ffc20 0x7f0cbc0743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.650+0000 7f0cbbfff700 1 -- 172.21.15.133:0/827492326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cbc1017a0 msgr2=0x7f0cbc072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:29.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.650+0000 7f0cbbfff700 1 --2- 172.21.15.133:0/827492326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cbc1017a0 0x7f0cbc072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.650+0000 7f0cbbfff700 1 -- 172.21.15.133:0/827492326 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb0009520 con 0x7f0cbc100e10 2023-07-14T18:16:29.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.650+0000 7f0cbbfff700 1 --2- 172.21.15.133:0/827492326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cbc100e10 0x7f0cbc072a30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0ca800b680 tx=0x7f0ca8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:29.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.651+0000 7f0cb9ffb700 1 -- 172.21.15.133:0/827492326 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca801d070 con 0x7f0cbc100e10 2023-07-14T18:16:29.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.651+0000 7f0cc31ff700 1 -- 172.21.15.133:0/827492326 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca80095f0 con 0x7f0cbc100e10 2023-07-14T18:16:29.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.651+0000 7f0cb9ffb700 1 -- 172.21.15.133:0/827492326 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ca8022470 con 0x7f0cbc100e10 2023-07-14T18:16:29.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.651+0000 7f0cb9ffb700 1 -- 172.21.15.133:0/827492326 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8017650 con 0x7f0cbc100e10 2023-07-14T18:16:29.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.651+0000 7f0cc31ff700 1 -- 172.21.15.133:0/827492326 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cbc19ff10 con 0x7f0cbc100e10 2023-07-14T18:16:29.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.652+0000 7f0cc31ff700 1 -- 172.21.15.133:0/827492326 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cbc1071c0 con 0x7f0cbc100e10 2023-07-14T18:16:29.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.654+0000 7f0cb9ffb700 1 -- 172.21.15.133:0/827492326 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f0ca80225e0 con 0x7f0cbc100e10 2023-07-14T18:16:29.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.655+0000 7f0cb9ffb700 1 --2- 172.21.15.133:0/827492326 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0cac0a1150 0x7f0cac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:29.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.655+0000 7f0cb9ffb700 1 -- 172.21.15.133:0/827492326 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f0ca80ce050 con 0x7f0cbc100e10 2023-07-14T18:16:29.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.655+0000 7f0cc0f9b700 1 --2- 172.21.15.133:0/827492326 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0cac0a1150 0x7f0cac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:29.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.656+0000 7f0cc0f9b700 1 --2- 172.21.15.133:0/827492326 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0cac0a1150 0x7f0cac0a3600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc100c70 tx=0x7f0cb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:29.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.659+0000 7f0cb9ffb700 1 -- 172.21.15.133:0/827492326 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f0ca80974e0 con 0x7f0cbc100e10 2023-07-14T18:16:29.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.854+0000 7f0cc31ff700 1 -- 172.21.15.133:0/827492326 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0cbc061250 con 0x7f0cbc100e10 2023-07-14T18:16:29.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.855+0000 7f0cb9ffb700 1 -- 172.21.15.133:0/827492326 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0ca809ab00 con 0x7f0cbc100e10 2023-07-14T18:16:29.901 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2023-07-14T18:16:29.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.857+0000 7f0cc31ff700 1 -- 172.21.15.133:0/827492326 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0cac0a1150 msgr2=0x7f0cac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:29.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.857+0000 7f0cc31ff700 1 --2- 172.21.15.133:0/827492326 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0cac0a1150 0x7f0cac0a3600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc100c70 tx=0x7f0cb0009260 comp rx=0 tx=0).stop 2023-07-14T18:16:29.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.857+0000 7f0cc31ff700 1 -- 172.21.15.133:0/827492326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cbc100e10 msgr2=0x7f0cbc072a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:29.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.857+0000 7f0cc31ff700 1 --2- 172.21.15.133:0/827492326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cbc100e10 0x7f0cbc072a30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0ca800b680 tx=0x7f0ca8005740 comp rx=0 tx=0).stop 2023-07-14T18:16:29.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.857+0000 7f0cc31ff700 1 -- 172.21.15.133:0/827492326 shutdown_connections 2023-07-14T18:16:29.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.857+0000 7f0cc31ff700 1 --2- 172.21.15.133:0/827492326 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0cac0a1150 0x7f0cac0a3600 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.857+0000 7f0cc31ff700 1 --2- 172.21.15.133:0/827492326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cbc1017a0 0x7f0cbc072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.857+0000 7f0cc31ff700 1 --2- 172.21.15.133:0/827492326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cbc0ffc20 0x7f0cbc0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.857+0000 7f0cc31ff700 1 --2- 172.21.15.133:0/827492326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cbc100e10 0x7f0cbc072a30 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:29.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.857+0000 7f0cc31ff700 1 -- 172.21.15.133:0/827492326 >> 172.21.15.133:0/827492326 conn(0x7f0cbc0fb260 msgr2=0x7f0cbc104300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:29.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.857+0000 7f0cc31ff700 1 -- 172.21.15.133:0/827492326 shutdown_connections 2023-07-14T18:16:29.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:29.857+0000 7f0cc31ff700 1 -- 172.21.15.133:0/827492326 wait complete. 2023-07-14T18:16:29.922 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T18:16:30.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.080+0000 7f0f8c7ff700 1 -- 172.21.15.133:0/3131938875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f84101790 msgr2=0x7f0f84108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:30.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.080+0000 7f0f8c7ff700 1 --2- 172.21.15.133:0/3131938875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f84101790 0x7f0f84108050 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00b0a0 tx=0x7f0f7c00b3b0 comp rx=0 tx=0).stop 2023-07-14T18:16:30.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.080+0000 7f0f8c7ff700 1 -- 172.21.15.133:0/3131938875 shutdown_connections 2023-07-14T18:16:30.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.080+0000 7f0f8c7ff700 1 --2- 172.21.15.133:0/3131938875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f84101790 0x7f0f84108050 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:30.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.080+0000 7f0f8c7ff700 1 --2- 172.21.15.133:0/3131938875 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f840ffc10 0x7f0f84100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:30.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.080+0000 7f0f8c7ff700 1 --2- 172.21.15.133:0/3131938875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f84100e00 0x7f0f84101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:30.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.080+0000 7f0f8c7ff700 1 -- 172.21.15.133:0/3131938875 >> 172.21.15.133:0/3131938875 conn(0x7f0f840fb210 msgr2=0x7f0f840fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:30.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.080+0000 7f0f8c7ff700 1 -- 172.21.15.133:0/3131938875 shutdown_connections 2023-07-14T18:16:30.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.080+0000 7f0f8c7ff700 1 -- 172.21.15.133:0/3131938875 wait complete. 2023-07-14T18:16:30.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.081+0000 7f0f8c7ff700 1 Processor -- start 2023-07-14T18:16:30.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.081+0000 7f0f8c7ff700 1 -- start start 2023-07-14T18:16:30.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.082+0000 7f0f8c7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f840ffc10 0x7f0f84197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:30.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.082+0000 7f0f8c7ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f84100e00 0x7f0f841981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:30.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.082+0000 7f0f8c7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f84101790 0x7f0f8419f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:30.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.082+0000 7f0f8c7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8419f800 con 0x7f0f84101790 2023-07-14T18:16:30.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.082+0000 7f0f8c7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8419f970 con 0x7f0f840ffc10 2023-07-14T18:16:30.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.082+0000 7f0f8c7ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8419fb30 con 0x7f0f84100e00 2023-07-14T18:16:30.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.082+0000 7f0f8ad9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f84101790 0x7f0f8419f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:30.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.082+0000 7f0f8ad9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f84101790 0x7f0f8419f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43608/0 (socket says 172.21.15.133:43608) 2023-07-14T18:16:30.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.082+0000 7f0f8a59b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f840ffc10 0x7f0f84197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:30.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.082+0000 7f0f8ad9c700 1 -- 172.21.15.133:0/1015134702 learned_addr learned my addr 172.21.15.133:0/1015134702 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:30.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.082+0000 7f0f89d9a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f84100e00 0x7f0f841981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:30.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.082+0000 7f0f8ad9c700 1 -- 172.21.15.133:0/1015134702 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f84100e00 msgr2=0x7f0f841981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:30.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.082+0000 7f0f8ad9c700 1 --2- 172.21.15.133:0/1015134702 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f84100e00 0x7f0f841981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:30.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.082+0000 7f0f8ad9c700 1 -- 172.21.15.133:0/1015134702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f840ffc10 msgr2=0x7f0f84197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:30.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.082+0000 7f0f8ad9c700 1 --2- 172.21.15.133:0/1015134702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f840ffc10 0x7f0f84197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:30.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.082+0000 7f0f8ad9c700 1 -- 172.21.15.133:0/1015134702 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7c009d00 con 0x7f0f84101790 2023-07-14T18:16:30.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.083+0000 7f0f8ad9c700 1 --2- 172.21.15.133:0/1015134702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f84101790 0x7f0f8419f230 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00d010 tx=0x7f0f7c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:30.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.083+0000 7f0f7b7fe700 1 -- 172.21.15.133:0/1015134702 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c00e040 con 0x7f0f84101790 2023-07-14T18:16:30.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.083+0000 7f0f7b7fe700 1 -- 172.21.15.133:0/1015134702 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f7c003e10 con 0x7f0f84101790 2023-07-14T18:16:30.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.083+0000 7f0f8c7ff700 1 -- 172.21.15.133:0/1015134702 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f8419fdb0 con 0x7f0f84101790 2023-07-14T18:16:30.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.083+0000 7f0f7b7fe700 1 -- 172.21.15.133:0/1015134702 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c0052a0 con 0x7f0f84101790 2023-07-14T18:16:30.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.083+0000 7f0f8c7ff700 1 -- 172.21.15.133:0/1015134702 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f841a01d0 con 0x7f0f84101790 2023-07-14T18:16:30.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.084+0000 7f0f8c7ff700 1 -- 172.21.15.133:0/1015134702 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f8410b440 con 0x7f0f84101790 2023-07-14T18:16:30.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.086+0000 7f0f7b7fe700 1 -- 172.21.15.133:0/1015134702 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f0f7c01d480 con 0x7f0f84101790 2023-07-14T18:16:30.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.087+0000 7f0f7b7fe700 1 --2- 172.21.15.133:0/1015134702 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0f700a1150 0x7f0f700a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:30.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.087+0000 7f0f7b7fe700 1 -- 172.21.15.133:0/1015134702 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f0f7c0cddd0 con 0x7f0f84101790 2023-07-14T18:16:30.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.087+0000 7f0f8a59b700 1 --2- 172.21.15.133:0/1015134702 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0f700a1150 0x7f0f700a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:30.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.088+0000 7f0f8a59b700 1 --2- 172.21.15.133:0/1015134702 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0f700a1150 0x7f0f700a3600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0f84198b60 tx=0x7f0f80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:30.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.091+0000 7f0f7b7fe700 1 -- 172.21.15.133:0/1015134702 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f0f7c097220 con 0x7f0f84101790 2023-07-14T18:16:30.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.290+0000 7f0f8c7ff700 1 -- 172.21.15.133:0/1015134702 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0f841a0780 con 0x7f0f84101790 2023-07-14T18:16:30.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.290+0000 7f0f7b7fe700 1 -- 172.21.15.133:0/1015134702 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0f7c09a840 con 0x7f0f84101790 2023-07-14T18:16:30.336 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-07-14T18:16:30.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.292+0000 7f0f8c7ff700 1 -- 172.21.15.133:0/1015134702 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0f700a1150 msgr2=0x7f0f700a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:30.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.292+0000 7f0f8c7ff700 1 --2- 172.21.15.133:0/1015134702 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0f700a1150 0x7f0f700a3600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0f84198b60 tx=0x7f0f80009260 comp rx=0 tx=0).stop 2023-07-14T18:16:30.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.292+0000 7f0f8c7ff700 1 -- 172.21.15.133:0/1015134702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f84101790 msgr2=0x7f0f8419f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:30.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.292+0000 7f0f8c7ff700 1 --2- 172.21.15.133:0/1015134702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f84101790 0x7f0f8419f230 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00d010 tx=0x7f0f7c00bc80 comp rx=0 tx=0).stop 2023-07-14T18:16:30.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.292+0000 7f0f8c7ff700 1 -- 172.21.15.133:0/1015134702 shutdown_connections 2023-07-14T18:16:30.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.292+0000 7f0f8c7ff700 1 --2- 172.21.15.133:0/1015134702 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0f700a1150 0x7f0f700a3600 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:30.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.292+0000 7f0f8c7ff700 1 --2- 172.21.15.133:0/1015134702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f840ffc10 0x7f0f84197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:30.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.292+0000 7f0f8c7ff700 1 --2- 172.21.15.133:0/1015134702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f84101790 0x7f0f8419f230 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:30.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.292+0000 7f0f8c7ff700 1 --2- 172.21.15.133:0/1015134702 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f84100e00 0x7f0f841981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:30.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.292+0000 7f0f8c7ff700 1 -- 172.21.15.133:0/1015134702 >> 172.21.15.133:0/1015134702 conn(0x7f0f840fb210 msgr2=0x7f0f840fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:30.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.293+0000 7f0f8c7ff700 1 -- 172.21.15.133:0/1015134702 shutdown_connections 2023-07-14T18:16:30.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.293+0000 7f0f8c7ff700 1 -- 172.21.15.133:0/1015134702 wait complete. 2023-07-14T18:16:30.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:29 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/631973120' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:16:30.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:29 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1394612888' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:16:30.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:29 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/827492326' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:16:30.354 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:29 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/631973120' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:16:30.355 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:29 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1394612888' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:16:30.355 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:29 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/827492326' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:16:30.356 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-07-14T18:16:30.357 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T18:16:30.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:29 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/631973120' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:16:30.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:29 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1394612888' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:16:30.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:29 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/827492326' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:16:30.556 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.511+0000 7efe59dff700 1 -- 172.21.15.133:0/1661254116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe54100e20 msgr2=0x7efe54101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:30.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.511+0000 7efe59dff700 1 --2- 172.21.15.133:0/1661254116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe54100e20 0x7efe54101270 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7efe480098f0 tx=0x7efe48009c00 comp rx=0 tx=0).stop 2023-07-14T18:16:30.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.511+0000 7efe59dff700 1 -- 172.21.15.133:0/1661254116 shutdown_connections 2023-07-14T18:16:30.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.511+0000 7efe59dff700 1 --2- 172.21.15.133:0/1661254116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe541017b0 0x7efe54108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:30.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.511+0000 7efe59dff700 1 --2- 172.21.15.133:0/1661254116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe540ffc30 0x7efe54100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:30.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.511+0000 7efe59dff700 1 --2- 172.21.15.133:0/1661254116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe54100e20 0x7efe54101270 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:30.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.511+0000 7efe59dff700 1 -- 172.21.15.133:0/1661254116 >> 172.21.15.133:0/1661254116 conn(0x7efe540fb210 msgr2=0x7efe540fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:30.558 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.511+0000 7efe59dff700 1 -- 172.21.15.133:0/1661254116 shutdown_connections 2023-07-14T18:16:30.558 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.511+0000 7efe59dff700 1 -- 172.21.15.133:0/1661254116 wait complete. 2023-07-14T18:16:30.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.512+0000 7efe59dff700 1 Processor -- start 2023-07-14T18:16:30.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.512+0000 7efe59dff700 1 -- start start 2023-07-14T18:16:30.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.512+0000 7efe59dff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe540ffc30 0x7efe54197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:30.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.513+0000 7efe59dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe54100e20 0x7efe54198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:30.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.513+0000 7efe59dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe541017b0 0x7efe5419f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:30.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.513+0000 7efe59dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5419f750 con 0x7efe54100e20 2023-07-14T18:16:30.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.513+0000 7efe59dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5419f890 con 0x7efe541017b0 2023-07-14T18:16:30.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.513+0000 7efe59dff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5419fa20 con 0x7efe540ffc30 2023-07-14T18:16:30.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.513+0000 7efe52ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe54100e20 0x7efe54198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:30.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.513+0000 7efe52ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe54100e20 0x7efe54198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43626/0 (socket says 172.21.15.133:43626) 2023-07-14T18:16:30.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.513+0000 7efe52ffd700 1 -- 172.21.15.133:0/2173150128 learned_addr learned my addr 172.21.15.133:0/2173150128 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:30.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.513+0000 7efe537fe700 1 --2- 172.21.15.133:0/2173150128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe540ffc30 0x7efe54197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:30.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.513+0000 7efe53fff700 1 --2- 172.21.15.133:0/2173150128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe541017b0 0x7efe5419f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:30.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.513+0000 7efe52ffd700 1 -- 172.21.15.133:0/2173150128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe540ffc30 msgr2=0x7efe54197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:30.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.513+0000 7efe52ffd700 1 --2- 172.21.15.133:0/2173150128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe540ffc30 0x7efe54197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:30.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.513+0000 7efe52ffd700 1 -- 172.21.15.133:0/2173150128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe541017b0 msgr2=0x7efe5419f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:30.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.513+0000 7efe52ffd700 1 --2- 172.21.15.133:0/2173150128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe541017b0 0x7efe5419f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:30.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.513+0000 7efe52ffd700 1 -- 172.21.15.133:0/2173150128 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe48009520 con 0x7efe54100e20 2023-07-14T18:16:30.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.513+0000 7efe537fe700 1 --2- 172.21.15.133:0/2173150128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe540ffc30 0x7efe54197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:16:30.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.513+0000 7efe52ffd700 1 --2- 172.21.15.133:0/2173150128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe54100e20 0x7efe54198110 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7efe48005e90 tx=0x7efe4800b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:30.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.514+0000 7efe50ff9700 1 -- 172.21.15.133:0/2173150128 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4801c070 con 0x7efe54100e20 2023-07-14T18:16:30.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.514+0000 7efe59dff700 1 -- 172.21.15.133:0/2173150128 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe5419fca0 con 0x7efe54100e20 2023-07-14T18:16:30.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.514+0000 7efe50ff9700 1 -- 172.21.15.133:0/2173150128 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe4800f7f0 con 0x7efe54100e20 2023-07-14T18:16:30.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.514+0000 7efe50ff9700 1 -- 172.21.15.133:0/2173150128 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe48004a40 con 0x7efe54100e20 2023-07-14T18:16:30.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.514+0000 7efe59dff700 1 -- 172.21.15.133:0/2173150128 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe541a0120 con 0x7efe54100e20 2023-07-14T18:16:30.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.515+0000 7efe59dff700 1 -- 172.21.15.133:0/2173150128 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe5410b480 con 0x7efe54100e20 2023-07-14T18:16:30.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.517+0000 7efe50ff9700 1 -- 172.21.15.133:0/2173150128 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7efe48020070 con 0x7efe54100e20 2023-07-14T18:16:30.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.518+0000 7efe50ff9700 1 --2- 172.21.15.133:0/2173150128 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efe3c0a1220 0x7efe3c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.518+0000 7efe50ff9700 1 -- 172.21.15.133:0/2173150128 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7efe480cd900 con 0x7efe54100e20 2023-07-14T18:16:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.518+0000 7efe537fe700 1 --2- 172.21.15.133:0/2173150128 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efe3c0a1220 0x7efe3c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.519+0000 7efe537fe700 1 --2- 172.21.15.133:0/2173150128 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efe3c0a1220 0x7efe3c0a36d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efe54100c80 tx=0x7efe44009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:30.567 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.522+0000 7efe50ff9700 1 -- 172.21.15.133:0/2173150128 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7efe48096d50 con 0x7efe54100e20 2023-07-14T18:16:30.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.764+0000 7efe59dff700 1 -- 172.21.15.133:0/2173150128 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe541a06d0 con 0x7efe54100e20 2023-07-14T18:16:30.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.764+0000 7efe50ff9700 1 -- 172.21.15.133:0/2173150128 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10853 (secure 0 0 0) 0x7efe4809a370 con 0x7efe54100e20 2023-07-14T18:16:30.810 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:30.810 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3168151271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3168151271},{"type":"v1","addr":"172.21.15.133:6837","nonce":3168151271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14544,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/3093144261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3093144261},{"type":"v1","addr":"172.21.15.133:6839","nonce":3093144261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14556,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/3282902068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":3282902068},{"type":"v1","addr":"172.21.15.133:6841","nonce":3282902068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/4023127259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4023127259},{"type":"v1","addr":"172.21.15.161:6833","nonce":4023127259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2722373740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2722373740},{"type":"v1","addr":"172.21.15.171:6833","nonce":2722373740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2399005074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2399005074},{"type":"v1","addr":"172.21.15.133:6835","nonce":2399005074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":26,"flags":178,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"balance_automate":true},"ever_allowed_features":160,"explicitly_allowed_features":160,"created":"2023-07-14T18:14:17.440873+0000","modified":"2023-07-14T18:16:25.929058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24557,"mds_1":24541,"mds_2":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3725502564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3725502564},{"type":"v1","addr":"172.21.15.161:6835","nonce":3725502564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6835/1877362116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1877362116},{"type":"v1","addr":"172.21.15.171:6835","nonce":1877362116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/3458064798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3458064798},{"type":"v1","addr":"172.21.15.161:6837","nonce":3458064798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/106205332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":106205332},{"type":"v1","addr":"172.21.15.171:6837","nonce":106205332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.171:6839/816201198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":816201198},{"type":"v1","addr":"172.21.15.171:6839","nonce":816201198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/2503865871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2503865871},{"type":"v1","addr":"172.21.15.161:6839","nonce":2503865871}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T18:16:30.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.767+0000 7efe59dff700 1 -- 172.21.15.133:0/2173150128 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efe3c0a1220 msgr2=0x7efe3c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:30.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.767+0000 7efe59dff700 1 --2- 172.21.15.133:0/2173150128 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efe3c0a1220 0x7efe3c0a36d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efe54100c80 tx=0x7efe44009310 comp rx=0 tx=0).stop 2023-07-14T18:16:30.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.767+0000 7efe59dff700 1 -- 172.21.15.133:0/2173150128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe54100e20 msgr2=0x7efe54198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:30.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.767+0000 7efe59dff700 1 --2- 172.21.15.133:0/2173150128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe54100e20 0x7efe54198110 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7efe48005e90 tx=0x7efe4800b730 comp rx=0 tx=0).stop 2023-07-14T18:16:30.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.767+0000 7efe59dff700 1 -- 172.21.15.133:0/2173150128 shutdown_connections 2023-07-14T18:16:30.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.767+0000 7efe59dff700 1 --2- 172.21.15.133:0/2173150128 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efe3c0a1220 0x7efe3c0a36d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:30.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.767+0000 7efe59dff700 1 --2- 172.21.15.133:0/2173150128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe541017b0 0x7efe5419f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:30.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.767+0000 7efe59dff700 1 --2- 172.21.15.133:0/2173150128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe540ffc30 0x7efe54197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:30.814 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.767+0000 7efe59dff700 1 --2- 172.21.15.133:0/2173150128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe54100e20 0x7efe54198110 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:30.814 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.767+0000 7efe59dff700 1 -- 172.21.15.133:0/2173150128 >> 172.21.15.133:0/2173150128 conn(0x7efe540fb210 msgr2=0x7efe540fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:30.814 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.767+0000 7efe59dff700 1 -- 172.21.15.133:0/2173150128 shutdown_connections 2023-07-14T18:16:30.814 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:30.767+0000 7efe59dff700 1 -- 172.21.15.133:0/2173150128 wait complete. 2023-07-14T18:16:30.814 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 26 2023-07-14T18:16:30.833 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-07-14T18:16:30.896 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-07-14T18:16:30.897 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T18:16:31.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.051+0000 7fa63f3ff700 1 -- 172.21.15.133:0/279123111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6380ffc30 msgr2=0x7fa638100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:31.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.051+0000 7fa63f3ff700 1 --2- 172.21.15.133:0/279123111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6380ffc30 0x7fa638100040 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa634009990 tx=0x7fa634009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:31.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.051+0000 7fa63f3ff700 1 -- 172.21.15.133:0/279123111 shutdown_connections 2023-07-14T18:16:31.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.051+0000 7fa63f3ff700 1 --2- 172.21.15.133:0/279123111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6381017b0 0x7fa638108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:31.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.051+0000 7fa63f3ff700 1 --2- 172.21.15.133:0/279123111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6380ffc30 0x7fa638100040 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:31.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.051+0000 7fa63f3ff700 1 --2- 172.21.15.133:0/279123111 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa638100e20 0x7fa638101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:31.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.051+0000 7fa63f3ff700 1 -- 172.21.15.133:0/279123111 >> 172.21.15.133:0/279123111 conn(0x7fa6380fb210 msgr2=0x7fa6380fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:31.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.051+0000 7fa63f3ff700 1 -- 172.21.15.133:0/279123111 shutdown_connections 2023-07-14T18:16:31.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.051+0000 7fa63f3ff700 1 -- 172.21.15.133:0/279123111 wait complete. 2023-07-14T18:16:31.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.052+0000 7fa63f3ff700 1 Processor -- start 2023-07-14T18:16:31.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.053+0000 7fa63f3ff700 1 -- start start 2023-07-14T18:16:31.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.053+0000 7fa63f3ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6380ffc30 0x7fa638197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:31.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.053+0000 7fa63f3ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa638100e20 0x7fa638198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:31.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.053+0000 7fa63f3ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6381017b0 0x7fa63819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:31.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.053+0000 7fa63f3ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63819f6d0 con 0x7fa6381017b0 2023-07-14T18:16:31.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.053+0000 7fa63f3ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63819f810 con 0x7fa638100e20 2023-07-14T18:16:31.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.053+0000 7fa63f3ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63819f9a0 con 0x7fa6380ffc30 2023-07-14T18:16:31.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.053+0000 7fa63d19b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6380ffc30 0x7fa638197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:31.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.053+0000 7fa63c99a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa638100e20 0x7fa638198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:31.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.053+0000 7fa63d99c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6381017b0 0x7fa63819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:31.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.053+0000 7fa63d19b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6380ffc30 0x7fa638197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:44222/0 (socket says 172.21.15.133:44222) 2023-07-14T18:16:31.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.053+0000 7fa63d19b700 1 -- 172.21.15.133:0/2998672809 learned_addr learned my addr 172.21.15.133:0/2998672809 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:31.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.054+0000 7fa63d99c700 1 -- 172.21.15.133:0/2998672809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6380ffc30 msgr2=0x7fa638197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:31.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.054+0000 7fa63d99c700 1 --2- 172.21.15.133:0/2998672809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6380ffc30 0x7fa638197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:31.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.054+0000 7fa63d99c700 1 -- 172.21.15.133:0/2998672809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa638100e20 msgr2=0x7fa638198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:31.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.054+0000 7fa63d99c700 1 --2- 172.21.15.133:0/2998672809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa638100e20 0x7fa638198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:31.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.054+0000 7fa63d99c700 1 -- 172.21.15.133:0/2998672809 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6340095f0 con 0x7fa6381017b0 2023-07-14T18:16:31.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.054+0000 7fa63c99a700 1 --2- 172.21.15.133:0/2998672809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa638100e20 0x7fa638198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:16:31.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.054+0000 7fa63d99c700 1 --2- 172.21.15.133:0/2998672809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6381017b0 0x7fa63819f100 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa63000efc0 tx=0x7fa63000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:31.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.054+0000 7fa62e7fc700 1 -- 172.21.15.133:0/2998672809 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa630018070 con 0x7fa6381017b0 2023-07-14T18:16:31.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.054+0000 7fa63f3ff700 1 -- 172.21.15.133:0/2998672809 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa63819fc20 con 0x7fa6381017b0 2023-07-14T18:16:31.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.054+0000 7fa62e7fc700 1 -- 172.21.15.133:0/2998672809 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa63000f040 con 0x7fa6381017b0 2023-07-14T18:16:31.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.054+0000 7fa62e7fc700 1 -- 172.21.15.133:0/2998672809 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63000cd40 con 0x7fa6381017b0 2023-07-14T18:16:31.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.054+0000 7fa63f3ff700 1 -- 172.21.15.133:0/2998672809 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6381a00e0 con 0x7fa6381017b0 2023-07-14T18:16:31.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.055+0000 7fa63f3ff700 1 -- 172.21.15.133:0/2998672809 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6381922c0 con 0x7fa6381017b0 2023-07-14T18:16:31.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.057+0000 7fa62e7fc700 1 -- 172.21.15.133:0/2998672809 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fa630003a00 con 0x7fa6381017b0 2023-07-14T18:16:31.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.058+0000 7fa62e7fc700 1 --2- 172.21.15.133:0/2998672809 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa6240a11a0 0x7fa6240a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:31.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.059+0000 7fa62e7fc700 1 -- 172.21.15.133:0/2998672809 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa6300cc880 con 0x7fa6381017b0 2023-07-14T18:16:31.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.059+0000 7fa63d19b700 1 --2- 172.21.15.133:0/2998672809 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa6240a11a0 0x7fa6240a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:31.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.059+0000 7fa63d19b700 1 --2- 172.21.15.133:0/2998672809 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa6240a11a0 0x7fa6240a3650 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa634006010 tx=0x7fa634005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:31.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.062+0000 7fa62e7fc700 1 -- 172.21.15.133:0/2998672809 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fa630095cd0 con 0x7fa6381017b0 2023-07-14T18:16:31.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:30 smithi133 ceph-mon[126384]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2023-07-14T18:16:31.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:30 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1015134702' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:16:31.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:30 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2173150128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:16:31.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.259+0000 7fa63f3ff700 1 -- 172.21.15.133:0/2998672809 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa638060a80 con 0x7fa6240a11a0 2023-07-14T18:16:31.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.273+0000 7fa62e7fc700 1 -- 172.21.15.133:0/2998672809 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426640 (secure 0 0 0) 0x7fa638060a80 con 0x7fa6240a11a0 2023-07-14T18:16:31.322 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:31.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.278+0000 7fa63f3ff700 1 -- 172.21.15.133:0/2998672809 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa6240a11a0 msgr2=0x7fa6240a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:31.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.278+0000 7fa63f3ff700 1 --2- 172.21.15.133:0/2998672809 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa6240a11a0 0x7fa6240a3650 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa634006010 tx=0x7fa634005960 comp rx=0 tx=0).stop 2023-07-14T18:16:31.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.278+0000 7fa63f3ff700 1 -- 172.21.15.133:0/2998672809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6381017b0 msgr2=0x7fa63819f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:31.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.278+0000 7fa63f3ff700 1 --2- 172.21.15.133:0/2998672809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6381017b0 0x7fa63819f100 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa63000efc0 tx=0x7fa63000c3a0 comp rx=0 tx=0).stop 2023-07-14T18:16:31.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.278+0000 7fa63f3ff700 1 -- 172.21.15.133:0/2998672809 shutdown_connections 2023-07-14T18:16:31.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.278+0000 7fa63f3ff700 1 --2- 172.21.15.133:0/2998672809 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa6240a11a0 0x7fa6240a3650 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:31.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.278+0000 7fa63f3ff700 1 --2- 172.21.15.133:0/2998672809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6381017b0 0x7fa63819f100 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:31.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.278+0000 7fa63f3ff700 1 --2- 172.21.15.133:0/2998672809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6380ffc30 0x7fa638197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:31.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.278+0000 7fa63f3ff700 1 --2- 172.21.15.133:0/2998672809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa638100e20 0x7fa638198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:31.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.278+0000 7fa63f3ff700 1 -- 172.21.15.133:0/2998672809 >> 172.21.15.133:0/2998672809 conn(0x7fa6380fb210 msgr2=0x7fa638106710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:31.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.278+0000 7fa63f3ff700 1 -- 172.21.15.133:0/2998672809 shutdown_connections 2023-07-14T18:16:31.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.278+0000 7fa63f3ff700 1 -- 172.21.15.133:0/2998672809 wait complete. 2023-07-14T18:16:31.327 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2023-07-14T18:16:31.342 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2023-07-14T18:16:29.466224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660727,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":741,"num_read_kb":493,"num_write":352,"num_write_kb":3066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1181441,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336488,"kb_used_data":12296,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124605336,"statfs":{"total":1151940427776,"available":1151595864064,"internally_reserved":0,"allocated":12591104,"data_stored":8093022,"data_compressed":390240,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6401,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":55,"num_write":12,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003179"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.487205+0000","last_change":"2023-07-14T18:14:17.927179+0000","last_active":"2023-07-14T18:16:17.487205+0000","last_peered":"2023-07-14T18:16:17.487205+0000","last_clean":"2023-07-14T18:16:17.487205+0000","last_became_active":"2023-07-14T18:14:17.926880+0000","last_became_peered":"2023-07-14T18:14:17.926880+0000","last_unstale":"2023-07-14T18:16:17.487205+0000","last_undegraded":"2023-07-14T18:16:17.487205+0000","last_fullsized":"2023-07-14T18:16:17.487205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:41:35.598778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.483578+0000","last_change":"2023-07-14T18:14:17.933834+0000","last_active":"2023-07-14T18:16:17.483578+0000","last_peered":"2023-07-14T18:16:17.483578+0000","last_clean":"2023-07-14T18:16:17.483578+0000","last_became_active":"2023-07-14T18:14:17.933529+0000","last_became_peered":"2023-07-14T18:14:17.933529+0000","last_unstale":"2023-07-14T18:16:17.483578+0000","last_undegraded":"2023-07-14T18:16:17.483578+0000","last_fullsized":"2023-07-14T18:16:17.483578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:48:46.885347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.479455+0000","last_change":"2023-07-14T18:14:17.928400+0000","last_active":"2023-07-14T18:16:17.479455+0000","last_peered":"2023-07-14T18:16:17.479455+0000","last_clean":"2023-07-14T18:16:17.479455+0000","last_became_active":"2023-07-14T18:14:17.928077+0000","last_became_peered":"2023-07-14T18:14:17.928077+0000","last_unstale":"2023-07-14T18:16:17.479455+0000","last_undegraded":"2023-07-14T18:16:17.479455+0000","last_fullsized":"2023-07-14T18:16:17.479455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:52:23.012871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.472507+0000","last_change":"2023-07-14T18:14:17.932623+0000","last_active":"2023-07-14T18:16:17.472507+0000","last_peered":"2023-07-14T18:16:17.472507+0000","last_clean":"2023-07-14T18:16:17.472507+0000","last_became_active":"2023-07-14T18:14:17.932298+0000","last_became_peered":"2023-07-14T18:14:17.932298+0000","last_unstale":"2023-07-14T18:16:17.472507+0000","last_undegraded":"2023-07-14T18:16:17.472507+0000","last_fullsized":"2023-07-14T18:16:17.472507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:09:59.192311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.469953+0000","last_change":"2023-07-14T18:14:17.927109+0000","last_active":"2023-07-14T18:16:17.469953+0000","last_peered":"2023-07-14T18:16:17.469953+0000","last_clean":"2023-07-14T18:16:17.469953+0000","last_became_active":"2023-07-14T18:14:17.926800+0000","last_became_peered":"2023-07-14T18:14:17.926800+0000","last_unstale":"2023-07-14T18:16:17.469953+0000","last_undegraded":"2023-07-14T18:16:17.469953+0000","last_fullsized":"2023-07-14T18:16:17.469953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:59:27.287187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.481171+0000","last_change":"2023-07-14T18:14:17.935380+0000","last_active":"2023-07-14T18:16:17.481171+0000","last_peered":"2023-07-14T18:16:17.481171+0000","last_clean":"2023-07-14T18:16:17.481171+0000","last_became_active":"2023-07-14T18:14:17.935099+0000","last_became_peered":"2023-07-14T18:14:17.935099+0000","last_unstale":"2023-07-14T18:16:17.481171+0000","last_undegraded":"2023-07-14T18:16:17.481171+0000","last_fullsized":"2023-07-14T18:16:17.481171+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:22:36.754253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.482663+0000","last_change":"2023-07-14T18:14:17.930207+0000","last_active":"2023-07-14T18:16:17.482663+0000","last_peered":"2023-07-14T18:16:17.482663+0000","last_clean":"2023-07-14T18:16:17.482663+0000","last_became_active":"2023-07-14T18:14:17.929898+0000","last_became_peered":"2023-07-14T18:14:17.929898+0000","last_unstale":"2023-07-14T18:16:17.482663+0000","last_undegraded":"2023-07-14T18:16:17.482663+0000","last_fullsized":"2023-07-14T18:16:17.482663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:53:19.205250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.473699+0000","last_change":"2023-07-14T18:14:17.928645+0000","last_active":"2023-07-14T18:16:17.473699+0000","last_peered":"2023-07-14T18:16:17.473699+0000","last_clean":"2023-07-14T18:16:17.473699+0000","last_became_active":"2023-07-14T18:14:17.928288+0000","last_became_peered":"2023-07-14T18:14:17.928288+0000","last_unstale":"2023-07-14T18:16:17.473699+0000","last_undegraded":"2023-07-14T18:16:17.473699+0000","last_fullsized":"2023-07-14T18:16:17.473699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:37:39.330884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.477077+0000","last_change":"2023-07-14T18:14:17.925888+0000","last_active":"2023-07-14T18:16:17.477077+0000","last_peered":"2023-07-14T18:16:17.477077+0000","last_clean":"2023-07-14T18:16:17.477077+0000","last_became_active":"2023-07-14T18:14:17.925580+0000","last_became_peered":"2023-07-14T18:14:17.925580+0000","last_unstale":"2023-07-14T18:16:17.477077+0000","last_undegraded":"2023-07-14T18:16:17.477077+0000","last_fullsized":"2023-07-14T18:16:17.477077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:09:13.877679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.482686+0000","last_change":"2023-07-14T18:14:17.924440+0000","last_active":"2023-07-14T18:16:17.482686+0000","last_peered":"2023-07-14T18:16:17.482686+0000","last_clean":"2023-07-14T18:16:17.482686+0000","last_became_active":"2023-07-14T18:14:17.923839+0000","last_became_peered":"2023-07-14T18:14:17.923839+0000","last_unstale":"2023-07-14T18:16:17.482686+0000","last_undegraded":"2023-07-14T18:16:17.482686+0000","last_fullsized":"2023-07-14T18:16:17.482686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:29:23.092881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.486380+0000","last_change":"2023-07-14T18:14:17.931067+0000","last_active":"2023-07-14T18:16:17.486380+0000","last_peered":"2023-07-14T18:16:17.486380+0000","last_clean":"2023-07-14T18:16:17.486380+0000","last_became_active":"2023-07-14T18:14:17.930633+0000","last_became_peered":"2023-07-14T18:14:17.930633+0000","last_unstale":"2023-07-14T18:16:17.486380+0000","last_undegraded":"2023-07-14T18:16:17.486380+0000","last_fullsized":"2023-07-14T18:16:17.486380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:04:56.675009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.471892+0000","last_change":"2023-07-14T18:14:17.932770+0000","last_active":"2023-07-14T18:16:17.471892+0000","last_peered":"2023-07-14T18:16:17.471892+0000","last_clean":"2023-07-14T18:16:17.471892+0000","last_became_active":"2023-07-14T18:14:17.932438+0000","last_became_peered":"2023-07-14T18:14:17.932438+0000","last_unstale":"2023-07-14T18:16:17.471892+0000","last_undegraded":"2023-07-14T18:16:17.471892+0000","last_fullsized":"2023-07-14T18:16:17.471892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:18:53.813312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.482161+0000","last_change":"2023-07-14T18:14:17.938213+0000","last_active":"2023-07-14T18:16:17.482161+0000","last_peered":"2023-07-14T18:16:17.482161+0000","last_clean":"2023-07-14T18:16:17.482161+0000","last_became_active":"2023-07-14T18:14:17.937565+0000","last_became_peered":"2023-07-14T18:14:17.937565+0000","last_unstale":"2023-07-14T18:16:17.482161+0000","last_undegraded":"2023-07-14T18:16:17.482161+0000","last_fullsized":"2023-07-14T18:16:17.482161+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:51:47.121447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.483823+0000","last_change":"2023-07-14T18:14:17.934677+0000","last_active":"2023-07-14T18:16:17.483823+0000","last_peered":"2023-07-14T18:16:17.483823+0000","last_clean":"2023-07-14T18:16:17.483823+0000","last_became_active":"2023-07-14T18:14:17.933622+0000","last_became_peered":"2023-07-14T18:14:17.933622+0000","last_unstale":"2023-07-14T18:16:17.483823+0000","last_undegraded":"2023-07-14T18:16:17.483823+0000","last_fullsized":"2023-07-14T18:16:17.483823+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:26:28.437669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.484282+0000","last_change":"2023-07-14T18:14:17.935317+0000","last_active":"2023-07-14T18:16:17.484282+0000","last_peered":"2023-07-14T18:16:17.484282+0000","last_clean":"2023-07-14T18:16:17.484282+0000","last_became_active":"2023-07-14T18:14:17.935034+0000","last_became_peered":"2023-07-14T18:14:17.935034+0000","last_unstale":"2023-07-14T18:16:17.484282+0000","last_undegraded":"2023-07-14T18:16:17.484282+0000","last_fullsized":"2023-07-14T18:16:17.484282+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:42:03.287591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.488504+0000","last_change":"2023-07-14T18:14:17.928222+0000","last_active":"2023-07-14T18:16:17.488504+0000","last_peered":"2023-07-14T18:16:17.488504+0000","last_clean":"2023-07-14T18:16:17.488504+0000","last_became_active":"2023-07-14T18:14:17.927964+0000","last_became_peered":"2023-07-14T18:14:17.927964+0000","last_unstale":"2023-07-14T18:16:17.488504+0000","last_undegraded":"2023-07-14T18:16:17.488504+0000","last_fullsized":"2023-07-14T18:16:17.488504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:42:45.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.486729+0000","last_change":"2023-07-14T18:14:17.927276+0000","last_active":"2023-07-14T18:16:17.486729+0000","last_peered":"2023-07-14T18:16:17.486729+0000","last_clean":"2023-07-14T18:16:17.486729+0000","last_became_active":"2023-07-14T18:14:17.926952+0000","last_became_peered":"2023-07-14T18:14:17.926952+0000","last_unstale":"2023-07-14T18:16:17.486729+0000","last_undegraded":"2023-07-14T18:16:17.486729+0000","last_fullsized":"2023-07-14T18:16:17.486729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:00:09.825668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.476643+0000","last_change":"2023-07-14T18:14:17.937364+0000","last_active":"2023-07-14T18:16:17.476643+0000","last_peered":"2023-07-14T18:16:17.476643+0000","last_clean":"2023-07-14T18:16:17.476643+0000","last_became_active":"2023-07-14T18:14:17.937014+0000","last_became_peered":"2023-07-14T18:14:17.937014+0000","last_unstale":"2023-07-14T18:16:17.476643+0000","last_undegraded":"2023-07-14T18:16:17.476643+0000","last_fullsized":"2023-07-14T18:16:17.476643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:31:10.117800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.472419+0000","last_change":"2023-07-14T18:14:17.920358+0000","last_active":"2023-07-14T18:16:17.472419+0000","last_peered":"2023-07-14T18:16:17.472419+0000","last_clean":"2023-07-14T18:16:17.472419+0000","last_became_active":"2023-07-14T18:14:17.920042+0000","last_became_peered":"2023-07-14T18:14:17.920042+0000","last_unstale":"2023-07-14T18:16:17.472419+0000","last_undegraded":"2023-07-14T18:16:17.472419+0000","last_fullsized":"2023-07-14T18:16:17.472419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:07:30.258643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093399+0000","last_change":"2023-07-14T18:14:22.045376+0000","last_active":"2023-07-14T18:15:50.093399+0000","last_peered":"2023-07-14T18:15:50.093399+0000","last_clean":"2023-07-14T18:15:50.093399+0000","last_became_active":"2023-07-14T18:14:22.044984+0000","last_became_peered":"2023-07-14T18:14:22.044984+0000","last_unstale":"2023-07-14T18:15:50.093399+0000","last_undegraded":"2023-07-14T18:15:50.093399+0000","last_fullsized":"2023-07-14T18:15:50.093399+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:26:13.981694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964742+0000","last_change":"2023-07-14T18:14:48.954270+0000","last_active":"2023-07-14T18:15:49.964742+0000","last_peered":"2023-07-14T18:15:49.964742+0000","last_clean":"2023-07-14T18:15:49.964742+0000","last_became_active":"2023-07-14T18:14:48.947868+0000","last_became_peered":"2023-07-14T18:14:48.947868+0000","last_unstale":"2023-07-14T18:15:49.964742+0000","last_undegraded":"2023-07-14T18:15:49.964742+0000","last_fullsized":"2023-07-14T18:15:49.964742+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:56:18.186337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.472946+0000","last_change":"2023-07-14T18:14:17.925822+0000","last_active":"2023-07-14T18:16:17.472946+0000","last_peered":"2023-07-14T18:16:17.472946+0000","last_clean":"2023-07-14T18:16:17.472946+0000","last_became_active":"2023-07-14T18:14:17.925508+0000","last_became_peered":"2023-07-14T18:14:17.925508+0000","last_unstale":"2023-07-14T18:16:17.472946+0000","last_undegraded":"2023-07-14T18:16:17.472946+0000","last_fullsized":"2023-07-14T18:16:17.472946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:05:04.319300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958344+0000","last_change":"2023-07-14T18:14:22.043098+0000","last_active":"2023-07-14T18:15:49.958344+0000","last_peered":"2023-07-14T18:15:49.958344+0000","last_clean":"2023-07-14T18:15:49.958344+0000","last_became_active":"2023-07-14T18:14:22.042517+0000","last_became_peered":"2023-07-14T18:14:22.042517+0000","last_unstale":"2023-07-14T18:15:49.958344+0000","last_undegraded":"2023-07-14T18:15:49.958344+0000","last_fullsized":"2023-07-14T18:15:49.958344+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:46:20.942876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.095016+0000","last_change":"2023-07-14T18:14:16.927612+0000","last_active":"2023-07-14T18:15:50.095016+0000","last_peered":"2023-07-14T18:15:50.095016+0000","last_clean":"2023-07-14T18:15:50.095016+0000","last_became_active":"2023-07-14T18:14:16.927284+0000","last_became_peered":"2023-07-14T18:14:16.927284+0000","last_unstale":"2023-07-14T18:15:50.095016+0000","last_undegraded":"2023-07-14T18:15:50.095016+0000","last_fullsized":"2023-07-14T18:15:50.095016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:49:41.365291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.488089+0000","last_change":"2023-07-14T18:14:17.930624+0000","last_active":"2023-07-14T18:16:17.488089+0000","last_peered":"2023-07-14T18:16:17.488089+0000","last_clean":"2023-07-14T18:16:17.488089+0000","last_became_active":"2023-07-14T18:14:17.930273+0000","last_became_peered":"2023-07-14T18:14:17.930273+0000","last_unstale":"2023-07-14T18:16:17.488089+0000","last_undegraded":"2023-07-14T18:16:17.488089+0000","last_fullsized":"2023-07-14T18:16:17.488089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:08:45.137173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958582+0000","last_change":"2023-07-14T18:14:49.543208+0000","last_active":"2023-07-14T18:15:49.958582+0000","last_peered":"2023-07-14T18:15:49.958582+0000","last_clean":"2023-07-14T18:15:49.958582+0000","last_became_active":"2023-07-14T18:14:49.542685+0000","last_became_peered":"2023-07-14T18:14:49.542685+0000","last_unstale":"2023-07-14T18:15:49.958582+0000","last_undegraded":"2023-07-14T18:15:49.958582+0000","last_fullsized":"2023-07-14T18:15:49.958582+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:34:49.230001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.959423+0000","last_change":"2023-07-14T18:14:16.926185+0000","last_active":"2023-07-14T18:15:49.959423+0000","last_peered":"2023-07-14T18:15:49.959423+0000","last_clean":"2023-07-14T18:15:49.959423+0000","last_became_active":"2023-07-14T18:14:16.925874+0000","last_became_peered":"2023-07-14T18:14:16.925874+0000","last_unstale":"2023-07-14T18:15:49.959423+0000","last_undegraded":"2023-07-14T18:15:49.959423+0000","last_fullsized":"2023-07-14T18:15:49.959423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:54:20.071669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.483223+0000","last_change":"2023-07-14T18:14:17.936601+0000","last_active":"2023-07-14T18:16:17.483223+0000","last_peered":"2023-07-14T18:16:17.483223+0000","last_clean":"2023-07-14T18:16:17.483223+0000","last_became_active":"2023-07-14T18:14:17.936291+0000","last_became_peered":"2023-07-14T18:14:17.936291+0000","last_unstale":"2023-07-14T18:16:17.483223+0000","last_undegraded":"2023-07-14T18:16:17.483223+0000","last_fullsized":"2023-07-14T18:16:17.483223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:32:59.220779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958529+0000","last_change":"2023-07-14T18:14:22.042952+0000","last_active":"2023-07-14T18:15:49.958529+0000","last_peered":"2023-07-14T18:15:49.958529+0000","last_clean":"2023-07-14T18:15:49.958529+0000","last_became_active":"2023-07-14T18:14:22.042256+0000","last_became_peered":"2023-07-14T18:14:22.042256+0000","last_unstale":"2023-07-14T18:15:49.958529+0000","last_undegraded":"2023-07-14T18:15:49.958529+0000","last_fullsized":"2023-07-14T18:15:49.958529+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:00:33.594352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958938+0000","last_change":"2023-07-14T18:14:16.920902+0000","last_active":"2023-07-14T18:15:49.958938+0000","last_peered":"2023-07-14T18:15:49.958938+0000","last_clean":"2023-07-14T18:15:49.958938+0000","last_became_active":"2023-07-14T18:14:16.920554+0000","last_became_peered":"2023-07-14T18:14:16.920554+0000","last_unstale":"2023-07-14T18:15:49.958938+0000","last_undegraded":"2023-07-14T18:15:49.958938+0000","last_fullsized":"2023-07-14T18:15:49.958938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:42:47.908351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.477004+0000","last_change":"2023-07-14T18:14:17.925005+0000","last_active":"2023-07-14T18:16:17.477004+0000","last_peered":"2023-07-14T18:16:17.477004+0000","last_clean":"2023-07-14T18:16:17.477004+0000","last_became_active":"2023-07-14T18:14:17.924723+0000","last_became_peered":"2023-07-14T18:14:17.924723+0000","last_unstale":"2023-07-14T18:16:17.477004+0000","last_undegraded":"2023-07-14T18:16:17.477004+0000","last_fullsized":"2023-07-14T18:16:17.477004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:23.634518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093304+0000","last_change":"2023-07-14T18:14:22.038482+0000","last_active":"2023-07-14T18:15:50.093304+0000","last_peered":"2023-07-14T18:15:50.093304+0000","last_clean":"2023-07-14T18:15:50.093304+0000","last_became_active":"2023-07-14T18:14:22.038059+0000","last_became_peered":"2023-07-14T18:14:22.038059+0000","last_unstale":"2023-07-14T18:15:50.093304+0000","last_undegraded":"2023-07-14T18:15:50.093304+0000","last_fullsized":"2023-07-14T18:15:50.093304+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:14:37.242779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'8","reported_seq":146,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:25.035755+0000","last_change":"2023-07-14T18:14:48.949585+0000","last_active":"2023-07-14T18:16:25.035755+0000","last_peered":"2023-07-14T18:16:25.035755+0000","last_clean":"2023-07-14T18:16:25.035755+0000","last_became_active":"2023-07-14T18:14:48.949106+0000","last_became_peered":"2023-07-14T18:14:48.949106+0000","last_unstale":"2023-07-14T18:16:25.035755+0000","last_undegraded":"2023-07-14T18:16:25.035755+0000","last_fullsized":"2023-07-14T18:16:25.035755+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:10:07.660525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1421,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.471828+0000","last_change":"2023-07-14T18:14:17.934894+0000","last_active":"2023-07-14T18:16:17.471828+0000","last_peered":"2023-07-14T18:16:17.471828+0000","last_clean":"2023-07-14T18:16:17.471828+0000","last_became_active":"2023-07-14T18:14:17.934651+0000","last_became_peered":"2023-07-14T18:14:17.934651+0000","last_unstale":"2023-07-14T18:16:17.471828+0000","last_undegraded":"2023-07-14T18:16:17.471828+0000","last_fullsized":"2023-07-14T18:16:17.471828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:56:46.813395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093307+0000","last_change":"2023-07-14T18:14:22.043611+0000","last_active":"2023-07-14T18:15:50.093307+0000","last_peered":"2023-07-14T18:15:50.093307+0000","last_clean":"2023-07-14T18:15:50.093307+0000","last_became_active":"2023-07-14T18:14:22.043101+0000","last_became_peered":"2023-07-14T18:14:22.043101+0000","last_unstale":"2023-07-14T18:15:50.093307+0000","last_undegraded":"2023-07-14T18:15:50.093307+0000","last_fullsized":"2023-07-14T18:15:50.093307+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:14:51.438990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.983824+0000","last_change":"2023-07-14T18:14:16.925321+0000","last_active":"2023-07-14T18:15:49.983824+0000","last_peered":"2023-07-14T18:15:49.983824+0000","last_clean":"2023-07-14T18:15:49.983824+0000","last_became_active":"2023-07-14T18:14:16.925016+0000","last_became_peered":"2023-07-14T18:14:16.925016+0000","last_unstale":"2023-07-14T18:15:49.983824+0000","last_undegraded":"2023-07-14T18:15:49.983824+0000","last_fullsized":"2023-07-14T18:15:49.983824+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:41:08.173620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.464867+0000","last_change":"2023-07-14T18:14:17.928838+0000","last_active":"2023-07-14T18:16:17.464867+0000","last_peered":"2023-07-14T18:16:17.464867+0000","last_clean":"2023-07-14T18:16:17.464867+0000","last_became_active":"2023-07-14T18:14:17.928511+0000","last_became_peered":"2023-07-14T18:14:17.928511+0000","last_unstale":"2023-07-14T18:16:17.464867+0000","last_undegraded":"2023-07-14T18:16:17.464867+0000","last_fullsized":"2023-07-14T18:16:17.464867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:35:21.406449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964551+0000","last_change":"2023-07-14T18:14:22.043182+0000","last_active":"2023-07-14T18:15:49.964551+0000","last_peered":"2023-07-14T18:15:49.964551+0000","last_clean":"2023-07-14T18:15:49.964551+0000","last_became_active":"2023-07-14T18:14:22.042779+0000","last_became_peered":"2023-07-14T18:14:22.042779+0000","last_unstale":"2023-07-14T18:15:49.964551+0000","last_undegraded":"2023-07-14T18:15:49.964551+0000","last_fullsized":"2023-07-14T18:15:49.964551+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:22:42.125442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958189+0000","last_change":"2023-07-14T18:14:16.935321+0000","last_active":"2023-07-14T18:15:49.958189+0000","last_peered":"2023-07-14T18:15:49.958189+0000","last_clean":"2023-07-14T18:15:49.958189+0000","last_became_active":"2023-07-14T18:14:16.934494+0000","last_became_peered":"2023-07-14T18:14:16.934494+0000","last_unstale":"2023-07-14T18:15:49.958189+0000","last_undegraded":"2023-07-14T18:15:49.958189+0000","last_fullsized":"2023-07-14T18:15:49.958189+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:11:15.794692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.480564+0000","last_change":"2023-07-14T18:14:17.922139+0000","last_active":"2023-07-14T18:16:17.480564+0000","last_peered":"2023-07-14T18:16:17.480564+0000","last_clean":"2023-07-14T18:16:17.480564+0000","last_became_active":"2023-07-14T18:14:17.921829+0000","last_became_peered":"2023-07-14T18:14:17.921829+0000","last_unstale":"2023-07-14T18:16:17.480564+0000","last_undegraded":"2023-07-14T18:16:17.480564+0000","last_fullsized":"2023-07-14T18:16:17.480564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:19:51.159357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.094035+0000","last_change":"2023-07-14T18:14:22.033625+0000","last_active":"2023-07-14T18:15:50.094035+0000","last_peered":"2023-07-14T18:15:50.094035+0000","last_clean":"2023-07-14T18:15:50.094035+0000","last_became_active":"2023-07-14T18:14:22.033253+0000","last_became_peered":"2023-07-14T18:14:22.033253+0000","last_unstale":"2023-07-14T18:15:50.094035+0000","last_undegraded":"2023-07-14T18:15:50.094035+0000","last_fullsized":"2023-07-14T18:15:50.094035+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:36:04.591067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964644+0000","last_change":"2023-07-14T18:14:16.923864+0000","last_active":"2023-07-14T18:15:49.964644+0000","last_peered":"2023-07-14T18:15:49.964644+0000","last_clean":"2023-07-14T18:15:49.964644+0000","last_became_active":"2023-07-14T18:14:16.923546+0000","last_became_peered":"2023-07-14T18:14:16.923546+0000","last_unstale":"2023-07-14T18:15:49.964644+0000","last_undegraded":"2023-07-14T18:15:49.964644+0000","last_fullsized":"2023-07-14T18:15:49.964644+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:07:13.280104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.475115+0000","last_change":"2023-07-14T18:14:17.936473+0000","last_active":"2023-07-14T18:16:17.475115+0000","last_peered":"2023-07-14T18:16:17.475115+0000","last_clean":"2023-07-14T18:16:17.475115+0000","last_became_active":"2023-07-14T18:14:17.935387+0000","last_became_peered":"2023-07-14T18:14:17.935387+0000","last_unstale":"2023-07-14T18:16:17.475115+0000","last_undegraded":"2023-07-14T18:16:17.475115+0000","last_fullsized":"2023-07-14T18:16:17.475115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:05:12.007411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093186+0000","last_change":"2023-07-14T18:14:22.041544+0000","last_active":"2023-07-14T18:15:50.093186+0000","last_peered":"2023-07-14T18:15:50.093186+0000","last_clean":"2023-07-14T18:15:50.093186+0000","last_became_active":"2023-07-14T18:14:22.041183+0000","last_became_peered":"2023-07-14T18:14:22.041183+0000","last_unstale":"2023-07-14T18:15:50.093186+0000","last_undegraded":"2023-07-14T18:15:50.093186+0000","last_fullsized":"2023-07-14T18:15:50.093186+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:08:52.284739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'4","reported_seq":130,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:26.037087+0000","last_change":"2023-07-14T18:14:16.928860+0000","last_active":"2023-07-14T18:16:26.037087+0000","last_peered":"2023-07-14T18:16:26.037087+0000","last_clean":"2023-07-14T18:16:26.037087+0000","last_became_active":"2023-07-14T18:14:16.928567+0000","last_became_peered":"2023-07-14T18:14:16.928567+0000","last_unstale":"2023-07-14T18:16:26.037087+0000","last_undegraded":"2023-07-14T18:16:26.037087+0000","last_fullsized":"2023-07-14T18:16:26.037087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:26:39.917215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.484494+0000","last_change":"2023-07-14T18:14:17.938456+0000","last_active":"2023-07-14T18:16:17.484494+0000","last_peered":"2023-07-14T18:16:17.484494+0000","last_clean":"2023-07-14T18:16:17.484494+0000","last_became_active":"2023-07-14T18:14:17.938086+0000","last_became_peered":"2023-07-14T18:14:17.938086+0000","last_unstale":"2023-07-14T18:16:17.484494+0000","last_undegraded":"2023-07-14T18:16:17.484494+0000","last_fullsized":"2023-07-14T18:16:17.484494+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:19:25.486023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964227+0000","last_change":"2023-07-14T18:14:22.042785+0000","last_active":"2023-07-14T18:15:49.964227+0000","last_peered":"2023-07-14T18:15:49.964227+0000","last_clean":"2023-07-14T18:15:49.964227+0000","last_became_active":"2023-07-14T18:14:22.042346+0000","last_became_peered":"2023-07-14T18:14:22.042346+0000","last_unstale":"2023-07-14T18:15:49.964227+0000","last_undegraded":"2023-07-14T18:15:49.964227+0000","last_fullsized":"2023-07-14T18:15:49.964227+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:49:14.017758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.962653+0000","last_change":"2023-07-14T18:14:16.921168+0000","last_active":"2023-07-14T18:15:49.962653+0000","last_peered":"2023-07-14T18:15:49.962653+0000","last_clean":"2023-07-14T18:15:49.962653+0000","last_became_active":"2023-07-14T18:14:16.920808+0000","last_became_peered":"2023-07-14T18:14:16.920808+0000","last_unstale":"2023-07-14T18:15:49.962653+0000","last_undegraded":"2023-07-14T18:15:49.962653+0000","last_fullsized":"2023-07-14T18:15:49.962653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:13:22.466581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.487015+0000","last_change":"2023-07-14T18:14:17.935100+0000","last_active":"2023-07-14T18:16:17.487015+0000","last_peered":"2023-07-14T18:16:17.487015+0000","last_clean":"2023-07-14T18:16:17.487015+0000","last_became_active":"2023-07-14T18:14:17.934794+0000","last_became_peered":"2023-07-14T18:14:17.934794+0000","last_unstale":"2023-07-14T18:16:17.487015+0000","last_undegraded":"2023-07-14T18:16:17.487015+0000","last_fullsized":"2023-07-14T18:16:17.487015+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:06:24.196022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958851+0000","last_change":"2023-07-14T18:14:22.043495+0000","last_active":"2023-07-14T18:15:49.958851+0000","last_peered":"2023-07-14T18:15:49.958851+0000","last_clean":"2023-07-14T18:15:49.958851+0000","last_became_active":"2023-07-14T18:14:22.042686+0000","last_became_peered":"2023-07-14T18:14:22.042686+0000","last_unstale":"2023-07-14T18:15:49.958851+0000","last_undegraded":"2023-07-14T18:15:49.958851+0000","last_fullsized":"2023-07-14T18:15:49.958851+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:50:08.855428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.965517+0000","last_change":"2023-07-14T18:14:16.926006+0000","last_active":"2023-07-14T18:15:49.965517+0000","last_peered":"2023-07-14T18:15:49.965517+0000","last_clean":"2023-07-14T18:15:49.965517+0000","last_became_active":"2023-07-14T18:14:16.925719+0000","last_became_peered":"2023-07-14T18:14:16.925719+0000","last_unstale":"2023-07-14T18:15:49.965517+0000","last_undegraded":"2023-07-14T18:15:49.965517+0000","last_fullsized":"2023-07-14T18:15:49.965517+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:16:46.148528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.480595+0000","last_change":"2023-07-14T18:14:17.934155+0000","last_active":"2023-07-14T18:16:17.480595+0000","last_peered":"2023-07-14T18:16:17.480595+0000","last_clean":"2023-07-14T18:16:17.480595+0000","last_became_active":"2023-07-14T18:14:17.933756+0000","last_became_peered":"2023-07-14T18:14:17.933756+0000","last_unstale":"2023-07-14T18:16:17.480595+0000","last_undegraded":"2023-07-14T18:16:17.480595+0000","last_fullsized":"2023-07-14T18:16:17.480595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:10.480686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964266+0000","last_change":"2023-07-14T18:14:22.039198+0000","last_active":"2023-07-14T18:15:49.964266+0000","last_peered":"2023-07-14T18:15:49.964266+0000","last_clean":"2023-07-14T18:15:49.964266+0000","last_became_active":"2023-07-14T18:14:22.038842+0000","last_became_peered":"2023-07-14T18:14:22.038842+0000","last_unstale":"2023-07-14T18:15:49.964266+0000","last_undegraded":"2023-07-14T18:15:49.964266+0000","last_fullsized":"2023-07-14T18:15:49.964266+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:31:14.995258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'38","reported_seq":273,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:27.039579+0000","last_change":"2023-07-14T18:14:16.920918+0000","last_active":"2023-07-14T18:16:27.039579+0000","last_peered":"2023-07-14T18:16:27.039579+0000","last_clean":"2023-07-14T18:16:27.039579+0000","last_became_active":"2023-07-14T18:14:16.920572+0000","last_became_peered":"2023-07-14T18:14:16.920572+0000","last_unstale":"2023-07-14T18:16:27.039579+0000","last_undegraded":"2023-07-14T18:16:27.039579+0000","last_fullsized":"2023-07-14T18:16:27.039579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:02:06.484223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65752,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":207,"num_read_kb":168,"num_write":39,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.483336+0000","last_change":"2023-07-14T18:14:17.929413+0000","last_active":"2023-07-14T18:16:17.483336+0000","last_peered":"2023-07-14T18:16:17.483336+0000","last_clean":"2023-07-14T18:16:17.483336+0000","last_became_active":"2023-07-14T18:14:17.929064+0000","last_became_peered":"2023-07-14T18:14:17.929064+0000","last_unstale":"2023-07-14T18:16:17.483336+0000","last_undegraded":"2023-07-14T18:16:17.483336+0000","last_fullsized":"2023-07-14T18:16:17.483336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:56:39.264420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.965827+0000","last_change":"2023-07-14T18:14:22.041983+0000","last_active":"2023-07-14T18:15:49.965827+0000","last_peered":"2023-07-14T18:15:49.965827+0000","last_clean":"2023-07-14T18:15:49.965827+0000","last_became_active":"2023-07-14T18:14:22.041553+0000","last_became_peered":"2023-07-14T18:14:22.041553+0000","last_unstale":"2023-07-14T18:15:49.965827+0000","last_undegraded":"2023-07-14T18:15:49.965827+0000","last_fullsized":"2023-07-14T18:15:49.965827+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:02:11.442175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093843+0000","last_change":"2023-07-14T18:14:16.925444+0000","last_active":"2023-07-14T18:15:50.093843+0000","last_peered":"2023-07-14T18:15:50.093843+0000","last_clean":"2023-07-14T18:15:50.093843+0000","last_became_active":"2023-07-14T18:14:16.925138+0000","last_became_peered":"2023-07-14T18:14:16.925138+0000","last_unstale":"2023-07-14T18:15:50.093843+0000","last_undegraded":"2023-07-14T18:15:50.093843+0000","last_fullsized":"2023-07-14T18:15:50.093843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:19:37.684882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.487136+0000","last_change":"2023-07-14T18:14:17.922351+0000","last_active":"2023-07-14T18:16:17.487136+0000","last_peered":"2023-07-14T18:16:17.487136+0000","last_clean":"2023-07-14T18:16:17.487136+0000","last_became_active":"2023-07-14T18:14:17.921973+0000","last_became_peered":"2023-07-14T18:14:17.921973+0000","last_unstale":"2023-07-14T18:16:17.487136+0000","last_undegraded":"2023-07-14T18:16:17.487136+0000","last_fullsized":"2023-07-14T18:16:17.487136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:27:35.200279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964677+0000","last_change":"2023-07-14T18:14:22.048274+0000","last_active":"2023-07-14T18:15:49.964677+0000","last_peered":"2023-07-14T18:15:49.964677+0000","last_clean":"2023-07-14T18:15:49.964677+0000","last_became_active":"2023-07-14T18:14:22.047946+0000","last_became_peered":"2023-07-14T18:14:22.047946+0000","last_unstale":"2023-07-14T18:15:49.964677+0000","last_undegraded":"2023-07-14T18:15:49.964677+0000","last_fullsized":"2023-07-14T18:15:49.964677+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:43:43.819233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.965500+0000","last_change":"2023-07-14T18:14:16.933566+0000","last_active":"2023-07-14T18:15:49.965500+0000","last_peered":"2023-07-14T18:15:49.965500+0000","last_clean":"2023-07-14T18:15:49.965500+0000","last_became_active":"2023-07-14T18:14:16.933300+0000","last_became_peered":"2023-07-14T18:14:16.933300+0000","last_unstale":"2023-07-14T18:15:49.965500+0000","last_undegraded":"2023-07-14T18:15:49.965500+0000","last_fullsized":"2023-07-14T18:15:49.965500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:37:32.251937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.488953+0000","last_change":"2023-07-14T18:14:17.928521+0000","last_active":"2023-07-14T18:16:17.488953+0000","last_peered":"2023-07-14T18:16:17.488953+0000","last_clean":"2023-07-14T18:16:17.488953+0000","last_became_active":"2023-07-14T18:14:17.928025+0000","last_became_peered":"2023-07-14T18:14:17.928025+0000","last_unstale":"2023-07-14T18:16:17.488953+0000","last_undegraded":"2023-07-14T18:16:17.488953+0000","last_fullsized":"2023-07-14T18:16:17.488953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:58:12.486113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964540+0000","last_change":"2023-07-14T18:14:22.046820+0000","last_active":"2023-07-14T18:15:49.964540+0000","last_peered":"2023-07-14T18:15:49.964540+0000","last_clean":"2023-07-14T18:15:49.964540+0000","last_became_active":"2023-07-14T18:14:22.046440+0000","last_became_peered":"2023-07-14T18:14:22.046440+0000","last_unstale":"2023-07-14T18:15:49.964540+0000","last_undegraded":"2023-07-14T18:15:49.964540+0000","last_fullsized":"2023-07-14T18:15:49.964540+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:40:11.333149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":130,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:24.033320+0000","last_change":"2023-07-14T18:14:16.922075+0000","last_active":"2023-07-14T18:16:24.033320+0000","last_peered":"2023-07-14T18:16:24.033320+0000","last_clean":"2023-07-14T18:16:24.033320+0000","last_became_active":"2023-07-14T18:14:16.921570+0000","last_became_peered":"2023-07-14T18:14:16.921570+0000","last_unstale":"2023-07-14T18:16:24.033320+0000","last_undegraded":"2023-07-14T18:16:24.033320+0000","last_fullsized":"2023-07-14T18:16:24.033320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:57:51.347575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.487451+0000","last_change":"2023-07-14T18:14:17.936439+0000","last_active":"2023-07-14T18:16:17.487451+0000","last_peered":"2023-07-14T18:16:17.487451+0000","last_clean":"2023-07-14T18:16:17.487451+0000","last_became_active":"2023-07-14T18:14:17.936138+0000","last_became_peered":"2023-07-14T18:14:17.936138+0000","last_unstale":"2023-07-14T18:16:17.487451+0000","last_undegraded":"2023-07-14T18:16:17.487451+0000","last_fullsized":"2023-07-14T18:16:17.487451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:32:25.986869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958834+0000","last_change":"2023-07-14T18:14:49.543302+0000","last_active":"2023-07-14T18:15:49.958834+0000","last_peered":"2023-07-14T18:15:49.958834+0000","last_clean":"2023-07-14T18:15:49.958834+0000","last_became_active":"2023-07-14T18:14:49.542768+0000","last_became_peered":"2023-07-14T18:14:49.542768+0000","last_unstale":"2023-07-14T18:15:49.958834+0000","last_undegraded":"2023-07-14T18:15:49.958834+0000","last_fullsized":"2023-07-14T18:15:49.958834+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:05:11.701773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958699+0000","last_change":"2023-07-14T18:14:16.926417+0000","last_active":"2023-07-14T18:15:49.958699+0000","last_peered":"2023-07-14T18:15:49.958699+0000","last_clean":"2023-07-14T18:15:49.958699+0000","last_became_active":"2023-07-14T18:14:16.926101+0000","last_became_peered":"2023-07-14T18:14:16.926101+0000","last_unstale":"2023-07-14T18:15:49.958699+0000","last_undegraded":"2023-07-14T18:15:49.958699+0000","last_fullsized":"2023-07-14T18:15:49.958699+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:16:34.754140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.476341+0000","last_change":"2023-07-14T18:14:48.950649+0000","last_active":"2023-07-14T18:16:17.476341+0000","last_peered":"2023-07-14T18:16:17.476341+0000","last_clean":"2023-07-14T18:16:17.476341+0000","last_became_active":"2023-07-14T18:14:48.950303+0000","last_became_peered":"2023-07-14T18:14:48.950303+0000","last_unstale":"2023-07-14T18:16:17.476341+0000","last_undegraded":"2023-07-14T18:16:17.476341+0000","last_fullsized":"2023-07-14T18:16:17.476341+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:03:15.248073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.965148+0000","last_change":"2023-07-14T18:14:22.042171+0000","last_active":"2023-07-14T18:15:49.965148+0000","last_peered":"2023-07-14T18:15:49.965148+0000","last_clean":"2023-07-14T18:15:49.965148+0000","last_became_active":"2023-07-14T18:14:22.041692+0000","last_became_peered":"2023-07-14T18:14:22.041692+0000","last_unstale":"2023-07-14T18:15:49.965148+0000","last_undegraded":"2023-07-14T18:15:49.965148+0000","last_fullsized":"2023-07-14T18:15:49.965148+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:51:41.773405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.959284+0000","last_change":"2023-07-14T18:14:16.922883+0000","last_active":"2023-07-14T18:15:49.959284+0000","last_peered":"2023-07-14T18:15:49.959284+0000","last_clean":"2023-07-14T18:15:49.959284+0000","last_became_active":"2023-07-14T18:14:16.922501+0000","last_became_peered":"2023-07-14T18:14:16.922501+0000","last_unstale":"2023-07-14T18:15:49.959284+0000","last_undegraded":"2023-07-14T18:15:49.959284+0000","last_fullsized":"2023-07-14T18:15:49.959284+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:53:01.489131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.478108+0000","last_change":"2023-07-14T18:14:17.922899+0000","last_active":"2023-07-14T18:16:17.478108+0000","last_peered":"2023-07-14T18:16:17.478108+0000","last_clean":"2023-07-14T18:16:17.478108+0000","last_became_active":"2023-07-14T18:14:17.922608+0000","last_became_peered":"2023-07-14T18:14:17.922608+0000","last_unstale":"2023-07-14T18:16:17.478108+0000","last_undegraded":"2023-07-14T18:16:17.478108+0000","last_fullsized":"2023-07-14T18:16:17.478108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:07:33.479250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.959164+0000","last_change":"2023-07-14T18:14:22.327484+0000","last_active":"2023-07-14T18:15:49.959164+0000","last_peered":"2023-07-14T18:15:49.959164+0000","last_clean":"2023-07-14T18:15:49.959164+0000","last_became_active":"2023-07-14T18:14:22.327010+0000","last_became_peered":"2023-07-14T18:14:22.327010+0000","last_unstale":"2023-07-14T18:15:49.959164+0000","last_undegraded":"2023-07-14T18:15:49.959164+0000","last_fullsized":"2023-07-14T18:15:49.959164+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:28:55.842693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964720+0000","last_change":"2023-07-14T18:14:16.930606+0000","last_active":"2023-07-14T18:15:49.964720+0000","last_peered":"2023-07-14T18:15:49.964720+0000","last_clean":"2023-07-14T18:15:49.964720+0000","last_became_active":"2023-07-14T18:14:16.930175+0000","last_became_peered":"2023-07-14T18:14:16.930175+0000","last_unstale":"2023-07-14T18:15:49.964720+0000","last_undegraded":"2023-07-14T18:15:49.964720+0000","last_fullsized":"2023-07-14T18:15:49.964720+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:30:16.214141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.471509+0000","last_change":"2023-07-14T18:14:17.931962+0000","last_active":"2023-07-14T18:16:17.471509+0000","last_peered":"2023-07-14T18:16:17.471509+0000","last_clean":"2023-07-14T18:16:17.471509+0000","last_became_active":"2023-07-14T18:14:17.931621+0000","last_became_peered":"2023-07-14T18:14:17.931621+0000","last_unstale":"2023-07-14T18:16:17.471509+0000","last_undegraded":"2023-07-14T18:16:17.471509+0000","last_fullsized":"2023-07-14T18:16:17.471509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:53:21.866251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958462+0000","last_change":"2023-07-14T18:14:22.038977+0000","last_active":"2023-07-14T18:15:49.958462+0000","last_peered":"2023-07-14T18:15:49.958462+0000","last_clean":"2023-07-14T18:15:49.958462+0000","last_became_active":"2023-07-14T18:14:22.038515+0000","last_became_peered":"2023-07-14T18:14:22.038515+0000","last_unstale":"2023-07-14T18:15:49.958462+0000","last_undegraded":"2023-07-14T18:15:49.958462+0000","last_fullsized":"2023-07-14T18:15:49.958462+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:18:48.042709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.962465+0000","last_change":"2023-07-14T18:14:16.925457+0000","last_active":"2023-07-14T18:15:49.962465+0000","last_peered":"2023-07-14T18:15:49.962465+0000","last_clean":"2023-07-14T18:15:49.962465+0000","last_became_active":"2023-07-14T18:14:16.925150+0000","last_became_peered":"2023-07-14T18:14:16.925150+0000","last_unstale":"2023-07-14T18:15:49.962465+0000","last_undegraded":"2023-07-14T18:15:49.962465+0000","last_fullsized":"2023-07-14T18:15:49.962465+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:15:42.815134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.481369+0000","last_change":"2023-07-14T18:14:17.926492+0000","last_active":"2023-07-14T18:16:17.481369+0000","last_peered":"2023-07-14T18:16:17.481369+0000","last_clean":"2023-07-14T18:16:17.481369+0000","last_became_active":"2023-07-14T18:14:17.926218+0000","last_became_peered":"2023-07-14T18:14:17.926218+0000","last_unstale":"2023-07-14T18:16:17.481369+0000","last_undegraded":"2023-07-14T18:16:17.481369+0000","last_fullsized":"2023-07-14T18:16:17.481369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:03:08.979533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958737+0000","last_change":"2023-07-14T18:14:22.327242+0000","last_active":"2023-07-14T18:15:49.958737+0000","last_peered":"2023-07-14T18:15:49.958737+0000","last_clean":"2023-07-14T18:15:49.958737+0000","last_became_active":"2023-07-14T18:14:22.326787+0000","last_became_peered":"2023-07-14T18:14:22.326787+0000","last_unstale":"2023-07-14T18:15:49.958737+0000","last_undegraded":"2023-07-14T18:15:49.958737+0000","last_fullsized":"2023-07-14T18:15:49.958737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:20:16.313522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.469654+0000","last_change":"2023-07-14T18:14:17.925035+0000","last_active":"2023-07-14T18:16:17.469654+0000","last_peered":"2023-07-14T18:16:17.469654+0000","last_clean":"2023-07-14T18:16:17.469654+0000","last_became_active":"2023-07-14T18:14:17.924713+0000","last_became_peered":"2023-07-14T18:14:17.924713+0000","last_unstale":"2023-07-14T18:16:17.469654+0000","last_undegraded":"2023-07-14T18:16:17.469654+0000","last_fullsized":"2023-07-14T18:16:17.469654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:32:01.731363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":132,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:26.035225+0000","last_change":"2023-07-14T18:14:16.935561+0000","last_active":"2023-07-14T18:16:26.035225+0000","last_peered":"2023-07-14T18:16:26.035225+0000","last_clean":"2023-07-14T18:16:26.035225+0000","last_became_active":"2023-07-14T18:14:16.935022+0000","last_became_peered":"2023-07-14T18:14:16.935022+0000","last_unstale":"2023-07-14T18:16:26.035225+0000","last_undegraded":"2023-07-14T18:16:26.035225+0000","last_fullsized":"2023-07-14T18:16:26.035225+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:46:30.440549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.989513+0000","last_change":"2023-07-14T18:15:49.989513+0000","last_active":"2023-07-14T18:15:49.989513+0000","last_peered":"2023-07-14T18:15:49.989513+0000","last_clean":"2023-07-14T18:15:49.989513+0000","last_became_active":"2023-07-14T18:15:49.985094+0000","last_became_peered":"2023-07-14T18:15:49.985094+0000","last_unstale":"2023-07-14T18:15:49.989513+0000","last_undegraded":"2023-07-14T18:15:49.989513+0000","last_fullsized":"2023-07-14T18:15:49.989513+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:12:15.915672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.476140+0000","last_change":"2023-07-14T18:14:17.930211+0000","last_active":"2023-07-14T18:16:17.476140+0000","last_peered":"2023-07-14T18:16:17.476140+0000","last_clean":"2023-07-14T18:16:17.476140+0000","last_became_active":"2023-07-14T18:14:17.929851+0000","last_became_peered":"2023-07-14T18:14:17.929851+0000","last_unstale":"2023-07-14T18:16:17.476140+0000","last_undegraded":"2023-07-14T18:16:17.476140+0000","last_fullsized":"2023-07-14T18:16:17.476140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:03:36.180205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958812+0000","last_change":"2023-07-14T18:14:16.928995+0000","last_active":"2023-07-14T18:15:49.958812+0000","last_peered":"2023-07-14T18:15:49.958812+0000","last_clean":"2023-07-14T18:15:49.958812+0000","last_became_active":"2023-07-14T18:14:16.928612+0000","last_became_peered":"2023-07-14T18:14:16.928612+0000","last_unstale":"2023-07-14T18:15:49.958812+0000","last_undegraded":"2023-07-14T18:15:49.958812+0000","last_fullsized":"2023-07-14T18:15:49.958812+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:01:02.502166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":345,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.962275+0000","last_change":"2023-07-14T18:13:11.614507+0000","last_active":"2023-07-14T18:15:49.962275+0000","last_peered":"2023-07-14T18:15:49.962275+0000","last_clean":"2023-07-14T18:15:49.962275+0000","last_became_active":"2023-07-14T18:13:11.614063+0000","last_became_peered":"2023-07-14T18:13:11.614063+0000","last_unstale":"2023-07-14T18:15:49.962275+0000","last_undegraded":"2023-07-14T18:15:49.962275+0000","last_fullsized":"2023-07-14T18:15:49.962275+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:10:03.995193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:10:03.995193+0000","last_clean_scrub_stamp":"2023-07-14T18:10:03.995193+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:00:53.238854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.959283+0000","last_change":"2023-07-14T18:14:49.542604+0000","last_active":"2023-07-14T18:15:49.959283+0000","last_peered":"2023-07-14T18:15:49.959283+0000","last_clean":"2023-07-14T18:15:49.959283+0000","last_became_active":"2023-07-14T18:14:49.542111+0000","last_became_peered":"2023-07-14T18:14:49.542111+0000","last_unstale":"2023-07-14T18:15:49.959283+0000","last_undegraded":"2023-07-14T18:15:49.959283+0000","last_fullsized":"2023-07-14T18:15:49.959283+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:25:58.915436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.479234+0000","last_change":"2023-07-14T18:14:49.543511+0000","last_active":"2023-07-14T18:16:17.479234+0000","last_peered":"2023-07-14T18:16:17.479234+0000","last_clean":"2023-07-14T18:16:17.479234+0000","last_became_active":"2023-07-14T18:14:49.542407+0000","last_became_peered":"2023-07-14T18:14:49.542407+0000","last_unstale":"2023-07-14T18:16:17.479234+0000","last_undegraded":"2023-07-14T18:16:17.479234+0000","last_fullsized":"2023-07-14T18:16:17.479234+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:18:32.901622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964257+0000","last_change":"2023-07-14T18:14:48.950022+0000","last_active":"2023-07-14T18:15:49.964257+0000","last_peered":"2023-07-14T18:15:49.964257+0000","last_clean":"2023-07-14T18:15:49.964257+0000","last_became_active":"2023-07-14T18:14:48.949149+0000","last_became_peered":"2023-07-14T18:14:48.949149+0000","last_unstale":"2023-07-14T18:15:49.964257+0000","last_undegraded":"2023-07-14T18:15:49.964257+0000","last_fullsized":"2023-07-14T18:15:49.964257+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:10:45.323165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964875+0000","last_change":"2023-07-14T18:14:22.038428+0000","last_active":"2023-07-14T18:15:49.964875+0000","last_peered":"2023-07-14T18:15:49.964875+0000","last_clean":"2023-07-14T18:15:49.964875+0000","last_became_active":"2023-07-14T18:14:22.038038+0000","last_became_peered":"2023-07-14T18:14:22.038038+0000","last_unstale":"2023-07-14T18:15:49.964875+0000","last_undegraded":"2023-07-14T18:15:49.964875+0000","last_fullsized":"2023-07-14T18:15:49.964875+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:16:27.576741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.959318+0000","last_change":"2023-07-14T18:14:16.933110+0000","last_active":"2023-07-14T18:15:49.959318+0000","last_peered":"2023-07-14T18:15:49.959318+0000","last_clean":"2023-07-14T18:15:49.959318+0000","last_became_active":"2023-07-14T18:14:16.932844+0000","last_became_peered":"2023-07-14T18:14:16.932844+0000","last_unstale":"2023-07-14T18:15:49.959318+0000","last_undegraded":"2023-07-14T18:15:49.959318+0000","last_fullsized":"2023-07-14T18:15:49.959318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:08:17.512405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.477926+0000","last_change":"2023-07-14T18:14:17.920168+0000","last_active":"2023-07-14T18:16:17.477926+0000","last_peered":"2023-07-14T18:16:17.477926+0000","last_clean":"2023-07-14T18:16:17.477926+0000","last_became_active":"2023-07-14T18:14:17.919576+0000","last_became_peered":"2023-07-14T18:14:17.919576+0000","last_unstale":"2023-07-14T18:16:17.477926+0000","last_undegraded":"2023-07-14T18:16:17.477926+0000","last_fullsized":"2023-07-14T18:16:17.477926+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:55:49.314127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093284+0000","last_change":"2023-07-14T18:14:22.327206+0000","last_active":"2023-07-14T18:15:50.093284+0000","last_peered":"2023-07-14T18:15:50.093284+0000","last_clean":"2023-07-14T18:15:50.093284+0000","last_became_active":"2023-07-14T18:14:22.326743+0000","last_became_peered":"2023-07-14T18:14:22.326743+0000","last_unstale":"2023-07-14T18:15:50.093284+0000","last_undegraded":"2023-07-14T18:15:50.093284+0000","last_fullsized":"2023-07-14T18:15:50.093284+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:50:43.439856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'8","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:25.919981+0000","last_change":"2023-07-14T18:14:16.928994+0000","last_active":"2023-07-14T18:16:25.919981+0000","last_peered":"2023-07-14T18:16:25.919981+0000","last_clean":"2023-07-14T18:16:25.919981+0000","last_became_active":"2023-07-14T18:14:16.928468+0000","last_became_peered":"2023-07-14T18:14:16.928468+0000","last_unstale":"2023-07-14T18:16:25.919981+0000","last_undegraded":"2023-07-14T18:16:25.919981+0000","last_fullsized":"2023-07-14T18:16:25.919981+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:31:02.519665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.487174+0000","last_change":"2023-07-14T18:14:17.919147+0000","last_active":"2023-07-14T18:16:17.487174+0000","last_peered":"2023-07-14T18:16:17.487174+0000","last_clean":"2023-07-14T18:16:17.487174+0000","last_became_active":"2023-07-14T18:14:17.918811+0000","last_became_peered":"2023-07-14T18:14:17.918811+0000","last_unstale":"2023-07-14T18:16:17.487174+0000","last_undegraded":"2023-07-14T18:16:17.487174+0000","last_fullsized":"2023-07-14T18:16:17.487174+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:48:51.557600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093309+0000","last_change":"2023-07-14T18:14:22.032632+0000","last_active":"2023-07-14T18:15:50.093309+0000","last_peered":"2023-07-14T18:15:50.093309+0000","last_clean":"2023-07-14T18:15:50.093309+0000","last_became_active":"2023-07-14T18:14:22.032233+0000","last_became_peered":"2023-07-14T18:14:22.032233+0000","last_unstale":"2023-07-14T18:15:50.093309+0000","last_undegraded":"2023-07-14T18:15:50.093309+0000","last_fullsized":"2023-07-14T18:15:50.093309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:10:59.598294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.959256+0000","last_change":"2023-07-14T18:14:16.924993+0000","last_active":"2023-07-14T18:15:49.959256+0000","last_peered":"2023-07-14T18:15:49.959256+0000","last_clean":"2023-07-14T18:15:49.959256+0000","last_became_active":"2023-07-14T18:14:16.924702+0000","last_became_peered":"2023-07-14T18:14:16.924702+0000","last_unstale":"2023-07-14T18:15:49.959256+0000","last_undegraded":"2023-07-14T18:15:49.959256+0000","last_fullsized":"2023-07-14T18:15:49.959256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:46:18.671640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.483640+0000","last_change":"2023-07-14T18:14:17.943397+0000","last_active":"2023-07-14T18:16:17.483640+0000","last_peered":"2023-07-14T18:16:17.483640+0000","last_clean":"2023-07-14T18:16:17.483640+0000","last_became_active":"2023-07-14T18:14:17.942114+0000","last_became_peered":"2023-07-14T18:14:17.942114+0000","last_unstale":"2023-07-14T18:16:17.483640+0000","last_undegraded":"2023-07-14T18:16:17.483640+0000","last_fullsized":"2023-07-14T18:16:17.483640+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:59:53.996642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958856+0000","last_change":"2023-07-14T18:14:22.328273+0000","last_active":"2023-07-14T18:15:49.958856+0000","last_peered":"2023-07-14T18:15:49.958856+0000","last_clean":"2023-07-14T18:15:49.958856+0000","last_became_active":"2023-07-14T18:14:22.327843+0000","last_became_peered":"2023-07-14T18:14:22.327843+0000","last_unstale":"2023-07-14T18:15:49.958856+0000","last_undegraded":"2023-07-14T18:15:49.958856+0000","last_fullsized":"2023-07-14T18:15:49.958856+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:29:12.613515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.094321+0000","last_change":"2023-07-14T18:14:49.543525+0000","last_active":"2023-07-14T18:15:50.094321+0000","last_peered":"2023-07-14T18:15:50.094321+0000","last_clean":"2023-07-14T18:15:50.094321+0000","last_became_active":"2023-07-14T18:14:49.542962+0000","last_became_peered":"2023-07-14T18:14:49.542962+0000","last_unstale":"2023-07-14T18:15:50.094321+0000","last_undegraded":"2023-07-14T18:15:50.094321+0000","last_fullsized":"2023-07-14T18:15:50.094321+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:43:37.767679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.464347+0000","last_change":"2023-07-14T18:14:17.932181+0000","last_active":"2023-07-14T18:16:17.464347+0000","last_peered":"2023-07-14T18:16:17.464347+0000","last_clean":"2023-07-14T18:16:17.464347+0000","last_became_active":"2023-07-14T18:14:17.931811+0000","last_became_peered":"2023-07-14T18:14:17.931811+0000","last_unstale":"2023-07-14T18:16:17.464347+0000","last_undegraded":"2023-07-14T18:16:17.464347+0000","last_fullsized":"2023-07-14T18:16:17.464347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:59:38.610413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964687+0000","last_change":"2023-07-14T18:14:22.047281+0000","last_active":"2023-07-14T18:15:49.964687+0000","last_peered":"2023-07-14T18:15:49.964687+0000","last_clean":"2023-07-14T18:15:49.964687+0000","last_became_active":"2023-07-14T18:14:22.045531+0000","last_became_peered":"2023-07-14T18:14:22.045531+0000","last_unstale":"2023-07-14T18:15:49.964687+0000","last_undegraded":"2023-07-14T18:15:49.964687+0000","last_fullsized":"2023-07-14T18:15:49.964687+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:53:44.716384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093515+0000","last_change":"2023-07-14T18:14:16.926904+0000","last_active":"2023-07-14T18:15:50.093515+0000","last_peered":"2023-07-14T18:15:50.093515+0000","last_clean":"2023-07-14T18:15:50.093515+0000","last_became_active":"2023-07-14T18:14:16.926579+0000","last_became_peered":"2023-07-14T18:14:16.926579+0000","last_unstale":"2023-07-14T18:15:50.093515+0000","last_undegraded":"2023-07-14T18:15:50.093515+0000","last_fullsized":"2023-07-14T18:15:50.093515+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:55:21.011226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.478178+0000","last_change":"2023-07-14T18:14:17.926005+0000","last_active":"2023-07-14T18:16:17.478178+0000","last_peered":"2023-07-14T18:16:17.478178+0000","last_clean":"2023-07-14T18:16:17.478178+0000","last_became_active":"2023-07-14T18:14:17.925266+0000","last_became_peered":"2023-07-14T18:14:17.925266+0000","last_unstale":"2023-07-14T18:16:17.478178+0000","last_undegraded":"2023-07-14T18:16:17.478178+0000","last_fullsized":"2023-07-14T18:16:17.478178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:18:03.629066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958581+0000","last_change":"2023-07-14T18:14:22.043219+0000","last_active":"2023-07-14T18:15:49.958581+0000","last_peered":"2023-07-14T18:15:49.958581+0000","last_clean":"2023-07-14T18:15:49.958581+0000","last_became_active":"2023-07-14T18:14:22.042654+0000","last_became_peered":"2023-07-14T18:14:22.042654+0000","last_unstale":"2023-07-14T18:15:49.958581+0000","last_undegraded":"2023-07-14T18:15:49.958581+0000","last_fullsized":"2023-07-14T18:15:49.958581+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:20:43.065883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093962+0000","last_change":"2023-07-14T18:14:16.923130+0000","last_active":"2023-07-14T18:15:50.093962+0000","last_peered":"2023-07-14T18:15:50.093962+0000","last_clean":"2023-07-14T18:15:50.093962+0000","last_became_active":"2023-07-14T18:14:16.922835+0000","last_became_peered":"2023-07-14T18:14:16.922835+0000","last_unstale":"2023-07-14T18:15:50.093962+0000","last_undegraded":"2023-07-14T18:15:50.093962+0000","last_fullsized":"2023-07-14T18:15:50.093962+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:29:20.116131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.478830+0000","last_change":"2023-07-14T18:14:17.931024+0000","last_active":"2023-07-14T18:16:17.478830+0000","last_peered":"2023-07-14T18:16:17.478830+0000","last_clean":"2023-07-14T18:16:17.478830+0000","last_became_active":"2023-07-14T18:14:17.929529+0000","last_became_peered":"2023-07-14T18:14:17.929529+0000","last_unstale":"2023-07-14T18:16:17.478830+0000","last_undegraded":"2023-07-14T18:16:17.478830+0000","last_fullsized":"2023-07-14T18:16:17.478830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:06:29.694951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958757+0000","last_change":"2023-07-14T18:14:22.327313+0000","last_active":"2023-07-14T18:15:49.958757+0000","last_peered":"2023-07-14T18:15:49.958757+0000","last_clean":"2023-07-14T18:15:49.958757+0000","last_became_active":"2023-07-14T18:14:22.326922+0000","last_became_peered":"2023-07-14T18:14:22.326922+0000","last_unstale":"2023-07-14T18:15:49.958757+0000","last_undegraded":"2023-07-14T18:15:49.958757+0000","last_fullsized":"2023-07-14T18:15:49.958757+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:27:15.790634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958923+0000","last_change":"2023-07-14T18:14:16.926632+0000","last_active":"2023-07-14T18:15:49.958923+0000","last_peered":"2023-07-14T18:15:49.958923+0000","last_clean":"2023-07-14T18:15:49.958923+0000","last_became_active":"2023-07-14T18:14:16.926279+0000","last_became_peered":"2023-07-14T18:14:16.926279+0000","last_unstale":"2023-07-14T18:15:49.958923+0000","last_undegraded":"2023-07-14T18:15:49.958923+0000","last_fullsized":"2023-07-14T18:15:49.958923+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:55:17.792126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.487272+0000","last_change":"2023-07-14T18:14:17.924416+0000","last_active":"2023-07-14T18:16:17.487272+0000","last_peered":"2023-07-14T18:16:17.487272+0000","last_clean":"2023-07-14T18:16:17.487272+0000","last_became_active":"2023-07-14T18:14:17.924140+0000","last_became_peered":"2023-07-14T18:14:17.924140+0000","last_unstale":"2023-07-14T18:16:17.487272+0000","last_undegraded":"2023-07-14T18:16:17.487272+0000","last_fullsized":"2023-07-14T18:16:17.487272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:43:36.553540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.094029+0000","last_change":"2023-07-14T18:14:22.037127+0000","last_active":"2023-07-14T18:15:50.094029+0000","last_peered":"2023-07-14T18:15:50.094029+0000","last_clean":"2023-07-14T18:15:50.094029+0000","last_became_active":"2023-07-14T18:14:22.036647+0000","last_became_peered":"2023-07-14T18:14:22.036647+0000","last_unstale":"2023-07-14T18:15:50.094029+0000","last_undegraded":"2023-07-14T18:15:50.094029+0000","last_fullsized":"2023-07-14T18:15:50.094029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:51:25.981240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.094215+0000","last_change":"2023-07-14T18:14:16.921494+0000","last_active":"2023-07-14T18:15:50.094215+0000","last_peered":"2023-07-14T18:15:50.094215+0000","last_clean":"2023-07-14T18:15:50.094215+0000","last_became_active":"2023-07-14T18:14:16.919622+0000","last_became_peered":"2023-07-14T18:14:16.919622+0000","last_unstale":"2023-07-14T18:15:50.094215+0000","last_undegraded":"2023-07-14T18:15:50.094215+0000","last_fullsized":"2023-07-14T18:15:50.094215+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:09:12.990342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.486524+0000","last_change":"2023-07-14T18:14:17.922266+0000","last_active":"2023-07-14T18:16:17.486524+0000","last_peered":"2023-07-14T18:16:17.486524+0000","last_clean":"2023-07-14T18:16:17.486524+0000","last_became_active":"2023-07-14T18:14:17.921461+0000","last_became_peered":"2023-07-14T18:14:17.921461+0000","last_unstale":"2023-07-14T18:16:17.486524+0000","last_undegraded":"2023-07-14T18:16:17.486524+0000","last_fullsized":"2023-07-14T18:16:17.486524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:04:46.997019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093532+0000","last_change":"2023-07-14T18:14:22.044080+0000","last_active":"2023-07-14T18:15:50.093532+0000","last_peered":"2023-07-14T18:15:50.093532+0000","last_clean":"2023-07-14T18:15:50.093532+0000","last_became_active":"2023-07-14T18:14:22.043612+0000","last_became_peered":"2023-07-14T18:14:22.043612+0000","last_unstale":"2023-07-14T18:15:50.093532+0000","last_undegraded":"2023-07-14T18:15:50.093532+0000","last_fullsized":"2023-07-14T18:15:50.093532+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:33:05.435083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.965710+0000","last_change":"2023-07-14T18:14:16.931871+0000","last_active":"2023-07-14T18:15:49.965710+0000","last_peered":"2023-07-14T18:15:49.965710+0000","last_clean":"2023-07-14T18:15:49.965710+0000","last_became_active":"2023-07-14T18:14:16.931549+0000","last_became_peered":"2023-07-14T18:14:16.931549+0000","last_unstale":"2023-07-14T18:15:49.965710+0000","last_undegraded":"2023-07-14T18:15:49.965710+0000","last_fullsized":"2023-07-14T18:15:49.965710+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:41:43.337550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.483151+0000","last_change":"2023-07-14T18:14:17.929550+0000","last_active":"2023-07-14T18:16:17.483151+0000","last_peered":"2023-07-14T18:16:17.483151+0000","last_clean":"2023-07-14T18:16:17.483151+0000","last_became_active":"2023-07-14T18:14:17.929204+0000","last_became_peered":"2023-07-14T18:14:17.929204+0000","last_unstale":"2023-07-14T18:16:17.483151+0000","last_undegraded":"2023-07-14T18:16:17.483151+0000","last_fullsized":"2023-07-14T18:16:17.483151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:15:39.711388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964416+0000","last_change":"2023-07-14T18:14:22.034899+0000","last_active":"2023-07-14T18:15:49.964416+0000","last_peered":"2023-07-14T18:15:49.964416+0000","last_clean":"2023-07-14T18:15:49.964416+0000","last_became_active":"2023-07-14T18:14:22.034524+0000","last_became_peered":"2023-07-14T18:14:22.034524+0000","last_unstale":"2023-07-14T18:15:49.964416+0000","last_undegraded":"2023-07-14T18:15:49.964416+0000","last_fullsized":"2023-07-14T18:15:49.964416+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:33:24.357185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958833+0000","last_change":"2023-07-14T18:14:16.935285+0000","last_active":"2023-07-14T18:15:49.958833+0000","last_peered":"2023-07-14T18:15:49.958833+0000","last_clean":"2023-07-14T18:15:49.958833+0000","last_became_active":"2023-07-14T18:14:16.934888+0000","last_became_peered":"2023-07-14T18:14:16.934888+0000","last_unstale":"2023-07-14T18:15:49.958833+0000","last_undegraded":"2023-07-14T18:15:49.958833+0000","last_fullsized":"2023-07-14T18:15:49.958833+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:24:43.441076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.485394+0000","last_change":"2023-07-14T18:14:17.930982+0000","last_active":"2023-07-14T18:16:17.485394+0000","last_peered":"2023-07-14T18:16:17.485394+0000","last_clean":"2023-07-14T18:16:17.485394+0000","last_became_active":"2023-07-14T18:14:17.930661+0000","last_became_peered":"2023-07-14T18:14:17.930661+0000","last_unstale":"2023-07-14T18:16:17.485394+0000","last_undegraded":"2023-07-14T18:16:17.485394+0000","last_fullsized":"2023-07-14T18:16:17.485394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:02:02.738482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.472552+0000","last_change":"2023-07-14T18:14:17.926204+0000","last_active":"2023-07-14T18:16:17.472552+0000","last_peered":"2023-07-14T18:16:17.472552+0000","last_clean":"2023-07-14T18:16:17.472552+0000","last_became_active":"2023-07-14T18:14:17.925689+0000","last_became_peered":"2023-07-14T18:14:17.925689+0000","last_unstale":"2023-07-14T18:16:17.472552+0000","last_undegraded":"2023-07-14T18:16:17.472552+0000","last_fullsized":"2023-07-14T18:16:17.472552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:24:10.100519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.476582+0000","last_change":"2023-07-14T18:14:17.923146+0000","last_active":"2023-07-14T18:16:17.476582+0000","last_peered":"2023-07-14T18:16:17.476582+0000","last_clean":"2023-07-14T18:16:17.476582+0000","last_became_active":"2023-07-14T18:14:17.922734+0000","last_became_peered":"2023-07-14T18:14:17.922734+0000","last_unstale":"2023-07-14T18:16:17.476582+0000","last_undegraded":"2023-07-14T18:16:17.476582+0000","last_fullsized":"2023-07-14T18:16:17.476582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:36:48.282746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.483489+0000","last_change":"2023-07-14T18:14:17.927397+0000","last_active":"2023-07-14T18:16:17.483489+0000","last_peered":"2023-07-14T18:16:17.483489+0000","last_clean":"2023-07-14T18:16:17.483489+0000","last_became_active":"2023-07-14T18:14:17.927000+0000","last_became_peered":"2023-07-14T18:14:17.927000+0000","last_unstale":"2023-07-14T18:16:17.483489+0000","last_undegraded":"2023-07-14T18:16:17.483489+0000","last_fullsized":"2023-07-14T18:16:17.483489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:55:36.682437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.482209+0000","last_change":"2023-07-14T18:14:17.935266+0000","last_active":"2023-07-14T18:16:17.482209+0000","last_peered":"2023-07-14T18:16:17.482209+0000","last_clean":"2023-07-14T18:16:17.482209+0000","last_became_active":"2023-07-14T18:14:17.934944+0000","last_became_peered":"2023-07-14T18:14:17.934944+0000","last_unstale":"2023-07-14T18:16:17.482209+0000","last_undegraded":"2023-07-14T18:16:17.482209+0000","last_fullsized":"2023-07-14T18:16:17.482209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:18:53.622505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.483800+0000","last_change":"2023-07-14T18:14:49.542810+0000","last_active":"2023-07-14T18:16:17.483800+0000","last_peered":"2023-07-14T18:16:17.483800+0000","last_clean":"2023-07-14T18:16:17.483800+0000","last_became_active":"2023-07-14T18:14:49.542421+0000","last_became_peered":"2023-07-14T18:14:49.542421+0000","last_unstale":"2023-07-14T18:16:17.483800+0000","last_undegraded":"2023-07-14T18:16:17.483800+0000","last_fullsized":"2023-07-14T18:16:17.483800+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:27:08.747661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.466139+0000","last_change":"2023-07-14T18:14:17.930757+0000","last_active":"2023-07-14T18:16:17.466139+0000","last_peered":"2023-07-14T18:16:17.466139+0000","last_clean":"2023-07-14T18:16:17.466139+0000","last_became_active":"2023-07-14T18:14:17.930302+0000","last_became_peered":"2023-07-14T18:14:17.930302+0000","last_unstale":"2023-07-14T18:16:17.466139+0000","last_undegraded":"2023-07-14T18:16:17.466139+0000","last_fullsized":"2023-07-14T18:16:17.466139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:00:25.569289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.475262+0000","last_change":"2023-07-14T18:14:17.930337+0000","last_active":"2023-07-14T18:16:17.475262+0000","last_peered":"2023-07-14T18:16:17.475262+0000","last_clean":"2023-07-14T18:16:17.475262+0000","last_became_active":"2023-07-14T18:14:17.930037+0000","last_became_peered":"2023-07-14T18:14:17.930037+0000","last_unstale":"2023-07-14T18:16:17.475262+0000","last_undegraded":"2023-07-14T18:16:17.475262+0000","last_fullsized":"2023-07-14T18:16:17.475262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:50:04.589039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.478977+0000","last_change":"2023-07-14T18:14:17.912750+0000","last_active":"2023-07-14T18:16:17.478977+0000","last_peered":"2023-07-14T18:16:17.478977+0000","last_clean":"2023-07-14T18:16:17.478977+0000","last_became_active":"2023-07-14T18:14:17.912402+0000","last_became_peered":"2023-07-14T18:14:17.912402+0000","last_unstale":"2023-07-14T18:16:17.478977+0000","last_undegraded":"2023-07-14T18:16:17.478977+0000","last_fullsized":"2023-07-14T18:16:17.478977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:29:09.484261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.471891+0000","last_change":"2023-07-14T18:14:17.930004+0000","last_active":"2023-07-14T18:16:17.471891+0000","last_peered":"2023-07-14T18:16:17.471891+0000","last_clean":"2023-07-14T18:16:17.471891+0000","last_became_active":"2023-07-14T18:14:17.929719+0000","last_became_peered":"2023-07-14T18:14:17.929719+0000","last_unstale":"2023-07-14T18:16:17.471891+0000","last_undegraded":"2023-07-14T18:16:17.471891+0000","last_fullsized":"2023-07-14T18:16:17.471891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:51:52.223909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.475685+0000","last_change":"2023-07-14T18:14:17.919044+0000","last_active":"2023-07-14T18:16:17.475685+0000","last_peered":"2023-07-14T18:16:17.475685+0000","last_clean":"2023-07-14T18:16:17.475685+0000","last_became_active":"2023-07-14T18:14:17.918079+0000","last_became_peered":"2023-07-14T18:14:17.918079+0000","last_unstale":"2023-07-14T18:16:17.475685+0000","last_undegraded":"2023-07-14T18:16:17.475685+0000","last_fullsized":"2023-07-14T18:16:17.475685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:44:33.328875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.488913+0000","last_change":"2023-07-14T18:14:17.921560+0000","last_active":"2023-07-14T18:16:17.488913+0000","last_peered":"2023-07-14T18:16:17.488913+0000","last_clean":"2023-07-14T18:16:17.488913+0000","last_became_active":"2023-07-14T18:14:17.921243+0000","last_became_peered":"2023-07-14T18:14:17.921243+0000","last_unstale":"2023-07-14T18:16:17.488913+0000","last_undegraded":"2023-07-14T18:16:17.488913+0000","last_fullsized":"2023-07-14T18:16:17.488913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:09:32.651260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.487411+0000","last_change":"2023-07-14T18:14:17.927504+0000","last_active":"2023-07-14T18:16:17.487411+0000","last_peered":"2023-07-14T18:16:17.487411+0000","last_clean":"2023-07-14T18:16:17.487411+0000","last_became_active":"2023-07-14T18:14:17.927233+0000","last_became_peered":"2023-07-14T18:14:17.927233+0000","last_unstale":"2023-07-14T18:16:17.487411+0000","last_undegraded":"2023-07-14T18:16:17.487411+0000","last_fullsized":"2023-07-14T18:16:17.487411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:59:31.666727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.478342+0000","last_change":"2023-07-14T18:14:17.932317+0000","last_active":"2023-07-14T18:16:17.478342+0000","last_peered":"2023-07-14T18:16:17.478342+0000","last_clean":"2023-07-14T18:16:17.478342+0000","last_became_active":"2023-07-14T18:14:17.931973+0000","last_became_peered":"2023-07-14T18:14:17.931973+0000","last_unstale":"2023-07-14T18:16:17.478342+0000","last_undegraded":"2023-07-14T18:16:17.478342+0000","last_fullsized":"2023-07-14T18:16:17.478342+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:12:59.198777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281026,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70113,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":615,"num_read_kb":384,"num_write":169,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":210339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416422,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":511749,"data_compressed":23584,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.94199999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.36099999999999999}]},{"osd":1,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63900000000000001,"15min":0.63800000000000001},"min":{"1min":0.44500000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.41899999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77700000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.63600000000000001}]},{"osd":2,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72299999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72699999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54100000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":4,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68600000000000005,"15min":0.68000000000000005},"min":{"1min":0.48099999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89900000000000002,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.746},{"interface":"front","average":{"1min":0.628,"5min":0.66400000000000003,"15min":0.67000000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86799999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.52500000000000002}]},{"osd":5,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.61099999999999999,"15min":0.625},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68300000000000005,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.38700000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96399999999999997,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.78400000000000003}]},{"osd":6,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.749,"15min":0.75800000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.95999999999999996,"5min":1.351,"15min":1.351},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.79000000000000004,"15min":0.79400000000000004},"min":{"1min":0.48299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":3.2410000000000001,"5min":3.2410000000000001,"15min":3.2410000000000001},"last":0.56200000000000006}]},{"osd":7,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77800000000000002,"15min":0.77900000000000003},"min":{"1min":0.58099999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.026,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.74399999999999999,"15min":0.749},"min":{"1min":0.51600000000000001,"5min":0.505,"15min":0.505},"max":{"1min":0.93100000000000005,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.69999999999999996}]},{"osd":8,"last update":"Fri Jul 14 18:16:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69699999999999995,"15min":0.70599999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.78100000000000003,"5min":0.878,"15min":0.878},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.78800000000000003,"15min":0.79400000000000004},"min":{"1min":0.55400000000000005,"5min":0.504,"15min":0.504},"max":{"1min":0.95299999999999996,"5min":1,"15min":1},"last":0.82299999999999995}]},{"osd":9,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80200000000000005,"15min":0.80600000000000005},"min":{"1min":0.58099999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1279999999999999,"5min":1.635,"15min":1.635},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.84899999999999998,"15min":0.85099999999999998},"min":{"1min":0.59899999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.05,"5min":1.147,"15min":1.147},"last":0.84899999999999998}]},{"osd":10,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80500000000000005,"15min":0.80900000000000005},"min":{"1min":0.57599999999999996,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.97299999999999998,"5min":1.272,"15min":1.272},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.83999999999999997,"15min":0.84999999999999998},"min":{"1min":0.59699999999999998,"5min":0.501,"15min":0.501},"max":{"1min":0.97099999999999997,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.80000000000000004}]}]},{"osd":10,"up_from":72,"seq":309237645355,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1155072,"data_stored":1168486,"data_compressed":59328,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61499999999999999,"15min":0.623},"min":{"1min":0.44400000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.76100000000000001,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.68000000000000005,"15min":0.68799999999999994},"min":{"1min":0.48499999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.83499999999999996,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.53200000000000003}]},{"osd":1,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.441,"15min":0.44400000000000001},"min":{"1min":0.32600000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61599999999999999,"5min":0.626,"15min":0.626},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47799999999999998,"15min":0.48399999999999999},"min":{"1min":0.29999999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.66900000000000004,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.47699999999999998}]},{"osd":2,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.38600000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.78400000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.42299999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77500000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.71199999999999997}]},{"osd":3,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63900000000000001,"15min":0.63500000000000001},"min":{"1min":0.439,"5min":0.38,"15min":0.38},"max":{"1min":1.4179999999999999,"5min":3.4129999999999998,"15min":3.4129999999999998},"last":0.504},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.70399999999999996,"15min":0.68100000000000005},"min":{"1min":0.45400000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4870000000000001,"5min":5.9939999999999998,"15min":5.9939999999999998},"last":0.58299999999999996}]},{"osd":4,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55200000000000005,"15min":0.56399999999999995},"min":{"1min":0.40500000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68400000000000005,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.68799999999999994,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.62}]},{"osd":5,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.66700000000000004,"15min":0.65800000000000003},"min":{"1min":0.46700000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81000000000000005,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.746},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.47299999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.55400000000000005}]},{"osd":6,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70999999999999996,"15min":0.72399999999999998},"min":{"1min":0.48399999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.747,"15min":0.76500000000000001},"min":{"1min":0.504,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.69299999999999995}]},{"osd":7,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.42199999999999999}]},{"osd":8,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.73599999999999999,"15min":0.745},"min":{"1min":0.54400000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.84899999999999998,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.75,"15min":0.75800000000000001},"min":{"1min":0.57999999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0149999999999999,"5min":1.026,"15min":1.026},"last":0.79500000000000004}]},{"osd":9,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.81100000000000005,"15min":0.81599999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.3879999999999999,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.80800000000000005,"15min":0.80600000000000005},"min":{"1min":0.60499999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":0.82499999999999996}]},{"osd":11,"last update":"Fri Jul 14 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.752,"15min":0.751},"min":{"1min":0.61499999999999999,"5min":0.499,"15min":0.499},"max":{"1min":0.89300000000000002,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.76500000000000001,"15min":0.76200000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.96699999999999997,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.80800000000000005}]}]},{"osd":9,"up_from":67,"seq":287762808880,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":511985,"data_compressed":23584,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.51100000000000001,"15min":0.53600000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.58699999999999997,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54800000000000004,"15min":0.56799999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Jul 14 18:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64800000000000002,"15min":0.65800000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.90600000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.69899999999999995,"15min":0.72799999999999998},"min":{"1min":0.45000000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88200000000000001,"5min":1.085,"15min":1.085},"last":0.70099999999999996}]},{"osd":2,"last update":"Fri Jul 14 18:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.68700000000000006,"15min":0.69399999999999995},"min":{"1min":0.44800000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91800000000000004,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.73199999999999998,"15min":0.754},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93799999999999994,"5min":1.147,"15min":1.147},"last":0.57399999999999995}]},{"osd":3,"last update":"Fri Jul 14 18:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.63700000000000001,"15min":0.66900000000000004},"min":{"1min":0.40100000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.63500000000000001,"15min":0.66300000000000003},"min":{"1min":0.39400000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.45500000000000002}]},{"osd":4,"last update":"Fri Jul 14 18:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.745,"15min":0.78100000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.878,"5min":1.198,"15min":1.198},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.751,"15min":0.78100000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.99199999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.71599999999999997}]},{"osd":5,"last update":"Fri Jul 14 18:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.77700000000000002,"15min":0.80600000000000005},"min":{"1min":0.53600000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.97199999999999998,"5min":1.224,"15min":1.224},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.73299999999999998,"15min":0.752},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95399999999999996,"5min":1.113,"15min":1.113},"last":0.64400000000000002}]},{"osd":6,"last update":"Fri Jul 14 18:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.75900000000000001}]},{"osd":7,"last update":"Fri Jul 14 18:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.82299999999999995,"15min":0.871},"min":{"1min":0.55800000000000005,"5min":0.503,"15min":0.503},"max":{"1min":0.92300000000000004,"5min":1.323,"15min":1.323},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.80800000000000005,"15min":0.85499999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.88100000000000001,"5min":1.411,"15min":1.411},"last":0.77700000000000002}]},{"osd":8,"last update":"Fri Jul 14 18:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.86599999999999999,"15min":0.91000000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.98799999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.872,"15min":0.91100000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.01,"5min":1.371,"15min":1.371},"last":0.88900000000000001}]},{"osd":10,"last update":"Fri Jul 14 18:16:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.83299999999999996,"15min":0.86899999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0409999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.84199999999999997,"15min":0.86599999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0489999999999999,"5min":1.095,"15min":1.095},"last":0.79300000000000004}]},{"osd":11,"last update":"Fri Jul 14 18:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.80600000000000005,"15min":0.80200000000000005},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.79600000000000004,"15min":0.80000000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.97899999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.83899999999999997}]}]},{"osd":8,"up_from":59,"seq":253403070517,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":511323,"data_compressed":23584,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77000000000000002,"5min":1.238,"15min":1.238},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.622,"15min":0.57999999999999996},"min":{"1min":0.46700000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96199999999999997,"5min":2.5950000000000002,"15min":2.5950000000000002},"last":0.62}]},{"osd":1,"last update":"Fri Jul 14 18:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.60499999999999998,"15min":0.57299999999999995},"min":{"1min":0.47999999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.63,"15min":0.59599999999999997},"min":{"1min":0.56100000000000005,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89800000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.77900000000000003}]},{"osd":2,"last update":"Fri Jul 14 18:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.48299999999999998,"15min":0.47499999999999998},"min":{"1min":0.42499999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.72699999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.49199999999999999,"15min":0.47199999999999998},"min":{"1min":0.42899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.79900000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.75600000000000001}]},{"osd":3,"last update":"Fri Jul 14 18:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.74399999999999999,"15min":0.66200000000000003},"min":{"1min":0.53600000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":3.8090000000000002,"5min":5.9749999999999996,"15min":5.9749999999999996},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.71099999999999997,"15min":0.63100000000000001},"min":{"1min":0.44,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":5.0810000000000004,"5min":5.0810000000000004,"15min":5.0810000000000004},"last":0.67100000000000004}]},{"osd":4,"last update":"Fri Jul 14 18:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.65400000000000003,"15min":0.63200000000000001},"min":{"1min":0.439,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.68700000000000006,"15min":0.66400000000000003},"min":{"1min":0.61699999999999999,"5min":0.434,"15min":0.434},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.92400000000000004}]},{"osd":5,"last update":"Fri Jul 14 18:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.59899999999999998,"15min":0.57099999999999995},"min":{"1min":0.40000000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.86799999999999999,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.57999999999999996,"15min":0.55400000000000005},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.88200000000000001}]},{"osd":6,"last update":"Fri Jul 14 18:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.75800000000000001,"15min":0.73399999999999999},"min":{"1min":0.51000000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.90400000000000003,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.82199999999999995,"15min":0.74399999999999999},"min":{"1min":0.65600000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":3.6339999999999999,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":0.90800000000000003}]},{"osd":7,"last update":"Fri Jul 14 18:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.75900000000000001,"15min":0.73699999999999999},"min":{"1min":0.73999999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.73799999999999999,"15min":0.71099999999999997},"min":{"1min":0.59799999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":1.042}]},{"osd":9,"last update":"Fri Jul 14 18:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":0.88900000000000001,"15min":0.82099999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":5.9370000000000003,"5min":5.9370000000000003,"15min":5.9370000000000003},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.2529999999999999,"5min":0.88,"15min":0.82399999999999995},"min":{"1min":0.57299999999999995,"5min":0.502,"15min":0.502},"max":{"1min":5.9039999999999999,"5min":5.9039999999999999,"15min":5.9039999999999999},"last":1.016}]},{"osd":10,"last update":"Fri Jul 14 18:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.79100000000000004,"15min":0.77000000000000002},"min":{"1min":0.57199999999999995,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":1.03},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.79400000000000004,"15min":0.77900000000000003},"min":{"1min":0.58399999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":1.0649999999999999}]},{"osd":11,"last update":"Fri Jul 14 18:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.79600000000000004,"15min":0.79100000000000004},"min":{"1min":0.57199999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.81899999999999995,"15min":0.81200000000000006},"min":{"1min":0.60399999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.97699999999999998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.995}]}]},{"osd":7,"up_from":51,"seq":219043332151,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":1108174,"data_compressed":59328,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.755,"15min":0.77300000000000002},"min":{"1min":0.38200000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.024,"5min":5.843,"15min":5.843},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.51100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.81299999999999994,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.40899999999999997}]},{"osd":1,"last update":"Fri Jul 14 18:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.42999999999999999,"15min":0.42799999999999999},"min":{"1min":0.32500000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.62,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46500000000000002,"15min":0.46200000000000002},"min":{"1min":0.35599999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.62,"5min":0.752,"15min":0.752},"last":0.52600000000000002}]},{"osd":2,"last update":"Fri Jul 14 18:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.64100000000000001,"15min":0.60399999999999998},"min":{"1min":0.42999999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.92800000000000005,"5min":3.4889999999999999,"15min":3.4889999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.45200000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.84999999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.66300000000000003}]},{"osd":3,"last update":"Fri Jul 14 18:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.504,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.95099999999999996,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.629},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.434,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.78800000000000003,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.64600000000000002}]},{"osd":4,"last update":"Fri Jul 14 18:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.51700000000000002,"15min":0.52900000000000003},"min":{"1min":0.379,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72199999999999998,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.5,"15min":0.498},"min":{"1min":0.379,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.58699999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.38500000000000001}]},{"osd":5,"last update":"Fri Jul 14 18:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.63700000000000001,"15min":0.62},"min":{"1min":0.52300000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.873,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68899999999999995,"15min":0.68600000000000005},"min":{"1min":0.55900000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86199999999999999,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.57999999999999996}]},{"osd":6,"last update":"Fri Jul 14 18:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.92700000000000005,"15min":1.1140000000000001},"min":{"1min":0.47899999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.129,"5min":13.189,"15min":13.189},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.48499999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.92400000000000004,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.72799999999999998}]},{"osd":8,"last update":"Fri Jul 14 18:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.71599999999999997,"15min":0.69799999999999995},"min":{"1min":0.59899999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86899999999999999,"5min":1.04,"15min":1.04},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69799999999999995,"15min":0.69199999999999995},"min":{"1min":0.54700000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85799999999999998,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.59999999999999998}]},{"osd":9,"last update":"Fri Jul 14 18:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.89700000000000002,"15min":0.95899999999999996},"min":{"1min":0.56100000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.96999999999999997,"5min":5.8250000000000002,"15min":5.8250000000000002},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77400000000000002,"15min":0.77100000000000002},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.86799999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.746}]},{"osd":10,"last update":"Fri Jul 14 18:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71799999999999997,"15min":0.71699999999999997},"min":{"1min":0.48599999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1,"5min":1,"15min":1},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76900000000000002,"15min":0.77000000000000002},"min":{"1min":0.59499999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.98499999999999999,"5min":0.995,"15min":0.995},"last":0.504}]},{"osd":11,"last update":"Fri Jul 14 18:16:19 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.85499999999999998,"15min":0.85399999999999998},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.875,"15min":0.876},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.79300000000000004}]}]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":507161,"data_compressed":23584,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46800000000000003,"15min":0.45600000000000002},"min":{"1min":0.316,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.51600000000000001,"15min":0.48999999999999999},"min":{"1min":0.41499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.39900000000000002}]},{"osd":1,"last update":"Fri Jul 14 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.66600000000000004,"15min":0.64600000000000002},"min":{"1min":0.46700000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.69499999999999995,"15min":0.68000000000000005},"min":{"1min":0.48199999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.76000000000000001}]},{"osd":2,"last update":"Fri Jul 14 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.65500000000000003,"15min":0.63200000000000001},"min":{"1min":0.50900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93799999999999994,"5min":1.087,"15min":1.087},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69399999999999995,"15min":0.68100000000000005},"min":{"1min":0.45500000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.629}]},{"osd":3,"last update":"Fri Jul 14 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.63700000000000001,"15min":0.61699999999999999},"min":{"1min":0.41999999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.504},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.65100000000000002,"15min":0.65200000000000002},"min":{"1min":0.433,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.64900000000000002}]},{"osd":4,"last update":"Fri Jul 14 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.66300000000000003,"15min":0.64900000000000002},"min":{"1min":0.52700000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.72999999999999998,"15min":0.72499999999999998},"min":{"1min":0.57299999999999995,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.79600000000000004}]},{"osd":5,"last update":"Fri Jul 14 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.76200000000000001,"15min":0.76500000000000001},"min":{"1min":0.54700000000000004,"5min":0.435,"15min":0.435},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.75900000000000001,"15min":0.73099999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":4.1449999999999996,"5min":4.1449999999999996,"15min":4.1449999999999996},"last":0.85299999999999998}]},{"osd":7,"last update":"Fri Jul 14 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.78500000000000003,"15min":0.74299999999999999},"min":{"1min":0.58899999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":4.2750000000000004,"5min":4.2750000000000004,"15min":4.2750000000000004},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.747,"15min":0.72699999999999998},"min":{"1min":0.51300000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0860000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.93100000000000005}]},{"osd":8,"last update":"Fri Jul 14 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.79300000000000004,"15min":0.80400000000000005},"min":{"1min":0.54800000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.77300000000000002,"15min":0.75900000000000001},"min":{"1min":0.623,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0980000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.69199999999999995}]},{"osd":9,"last update":"Fri Jul 14 18:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.79300000000000004,"15min":0.81499999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.996,"5min":1.147,"15min":1.147},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83099999999999996,"15min":0.84499999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.133,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.748}]},{"osd":10,"last update":"Fri Jul 14 18:16:13 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86499999999999999,"15min":0.86899999999999999},"min":{"1min":0.67900000000000005,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.028,"5min":0.89000000000000001,"15min":0.86599999999999999},"min":{"1min":0.64200000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":4.1909999999999998,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.97099999999999997}]},{"osd":11,"last update":"Fri Jul 14 18:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.90600000000000003,"15min":0.91200000000000003},"min":{"1min":0.72499999999999998,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.1180000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.84899999999999998,"15min":0.84799999999999998},"min":{"1min":0.73799999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.77300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":568026,"data_compressed":23584,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56999999999999995,"15min":0.52000000000000002},"min":{"1min":0.34100000000000003,"5min":0.30599999999999999,"15min":0.28199999999999997},"max":{"1min":1.3129999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":1.151,"5min":0.70399999999999996,"15min":0.57399999999999995},"min":{"1min":0.44800000000000001,"5min":0.32100000000000001,"15min":0.30299999999999999},"max":{"1min":7.226,"5min":7.226,"15min":7.226},"last":0.47499999999999998}]},{"osd":2,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.626,"15min":0.55400000000000005},"min":{"1min":0.47899999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59999999999999998,"15min":0.53500000000000003},"min":{"1min":0.48999999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84199999999999997,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.75600000000000001}]},{"osd":3,"last update":"Fri Jul 14 18:15:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.59299999999999997,"15min":0.53300000000000003},"min":{"1min":0.42799999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.125,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.65600000000000003,"15min":0.59999999999999998},"min":{"1min":0.501,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.5009999999999999,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":0.53500000000000003}]},{"osd":4,"last update":"Fri Jul 14 18:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.51000000000000001,"15min":0.46400000000000002},"min":{"1min":0.34599999999999997,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.91100000000000003,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.503,"15min":0.45900000000000002},"min":{"1min":0.36699999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.50800000000000001}]},{"osd":5,"last update":"Fri Jul 14 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.65700000000000003,"15min":0.61399999999999999},"min":{"1min":0.51600000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.65700000000000003,"15min":0.61599999999999999},"min":{"1min":0.61599999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.61899999999999999}]},{"osd":6,"last update":"Fri Jul 14 18:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.68600000000000005,"15min":0.67400000000000004},"min":{"1min":0.443,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.92500000000000004,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68000000000000005,"15min":0.66900000000000004},"min":{"1min":0.51800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.91400000000000003,"5min":1.23,"15min":1.23},"last":0.64200000000000002}]},{"osd":7,"last update":"Fri Jul 14 18:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.59999999999999998,"15min":0.56299999999999994},"min":{"1min":0.438,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.98299999999999998,"5min":1.04,"15min":1.04},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.628,"15min":0.58999999999999997},"min":{"1min":0.45900000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.93799999999999994,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.79700000000000004}]},{"osd":8,"last update":"Fri Jul 14 18:15:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.66500000000000004,"15min":0.64300000000000002},"min":{"1min":0.51000000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.72399999999999998,"15min":0.68999999999999995},"min":{"1min":0.629,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.026,"5min":1.052,"15min":1.052},"last":0.81999999999999995}]},{"osd":9,"last update":"Fri Jul 14 18:15:56 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.81899999999999995,"15min":0.78000000000000003},"min":{"1min":0.63300000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":4.0750000000000002,"5min":4.0750000000000002,"15min":4.0750000000000002},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.054,"5min":0.874,"15min":0.83899999999999997},"min":{"1min":0.69299999999999995,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":4.0369999999999999,"5min":4.0369999999999999,"15min":4.0369999999999999},"last":0.879}]},{"osd":10,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.81100000000000005,"15min":0.80500000000000005},"min":{"1min":0.57699999999999996,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.052,"5min":1.167,"15min":1.167},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76800000000000002,"15min":0.75700000000000001},"min":{"1min":0.55900000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0389999999999999,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.81000000000000005}]},{"osd":11,"last update":"Fri Jul 14 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.92000000000000004,"15min":0.92800000000000005},"min":{"1min":0.74099999999999999,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.097,"5min":1.169,"15min":1.169},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.86599999999999999,"15min":0.86799999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.0820000000000001,"5min":1.117,"15min":1.117},"last":0.78600000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":508549,"data_compressed":23584,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 18:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60799999999999998,"15min":0.55500000000000005},"min":{"1min":0.39400000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89800000000000002,"5min":1.698,"15min":1.698},"last":0.624},{"interface":"front","average":{"1min":0.621,"5min":0.61399999999999999,"15min":0.58199999999999996},"min":{"1min":0.40899999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.89600000000000002,"5min":1.179,"15min":1.179},"last":0.61099999999999999}]},{"osd":2,"last update":"Fri Jul 14 18:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.65600000000000003,"15min":0.57499999999999996},"min":{"1min":0.53400000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86799999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68200000000000005,"15min":0.59599999999999997},"min":{"1min":0.499,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89800000000000002,"5min":1.258,"15min":1.258},"last":0.74399999999999999}]},{"osd":3,"last update":"Fri Jul 14 18:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.55900000000000005,"15min":0.56699999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77400000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.44},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.64000000000000001,"15min":0.64600000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.64900000000000002}]},{"osd":4,"last update":"Fri Jul 14 18:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.61399999999999999,"15min":0.56899999999999995},"min":{"1min":0.441,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.80600000000000005,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66900000000000004,"15min":0.627},"min":{"1min":0.44,"5min":0.379,"15min":0.379},"max":{"1min":0.83699999999999997,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.59199999999999997}]},{"osd":5,"last update":"Fri Jul 14 18:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.72999999999999998,"15min":0.68899999999999995},"min":{"1min":0.45800000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90100000000000002,"5min":3.3740000000000001,"15min":3.3740000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.71499999999999997,"15min":0.68700000000000006},"min":{"1min":0.48099999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.97899999999999998,"5min":1.228,"15min":1.228},"last":0.79900000000000004}]},{"osd":6,"last update":"Fri Jul 14 18:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.64000000000000001,"15min":0.61599999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93600000000000005,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.65700000000000003,"15min":0.63700000000000001},"min":{"1min":0.41799999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88600000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.48499999999999999}]},{"osd":7,"last update":"Fri Jul 14 18:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.70199999999999996,"15min":0.70999999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.82299999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.97899999999999998,"5min":1.006,"15min":1.006},"last":0.76900000000000002}]},{"osd":8,"last update":"Fri Jul 14 18:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.80900000000000005,"15min":0.79600000000000004},"min":{"1min":0.60599999999999998,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.028,"5min":1.839,"15min":1.839},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.79700000000000004,"15min":0.77000000000000002},"min":{"1min":0.57599999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0029999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.83799999999999997}]},{"osd":9,"last update":"Fri Jul 14 18:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.79600000000000004,"5min":1.0669999999999999,"15min":1.1850000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1259999999999999,"5min":7.8689999999999998,"15min":7.8689999999999998},"last":0.70499999999999996}]},{"osd":10,"last update":"Fri Jul 14 18:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84999999999999998,"15min":0.85899999999999999},"min":{"1min":0.62,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.875},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82199999999999995,"15min":0.82599999999999996},"min":{"1min":0.69199999999999995,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0569999999999999,"5min":1.081,"15min":1.081},"last":0.82699999999999996}]},{"osd":11,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.90300000000000002,"15min":0.90700000000000003},"min":{"1min":0.65300000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.093,"5min":1.869,"15min":1.869},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.92900000000000005,"15min":0.93500000000000005},"min":{"1min":0.74399999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.0629999999999999,"5min":1.758,"15min":1.758},"last":0.90600000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":573551,"data_compressed":23584,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.629,"15min":0.55700000000000005},"min":{"1min":0.40200000000000002,"5min":0.313,"15min":0.28899999999999998},"max":{"1min":6.5220000000000002,"5min":6.5220000000000002,"15min":6.5220000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.67400000000000004,"15min":0.56399999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":8.984,"5min":8.984,"15min":8.984},"last":0.61399999999999999}]},{"osd":1,"last update":"Fri Jul 14 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.627,"15min":0.53700000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.33400000000000002},"max":{"1min":0.94599999999999995,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.61599999999999999,"15min":0.53400000000000003},"min":{"1min":0.441,"5min":0.36899999999999999,"15min":0.32000000000000001},"max":{"1min":0.89600000000000002,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.66600000000000004}]},{"osd":3,"last update":"Fri Jul 14 18:15:40 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":0.71899999999999997,"15min":0.60999999999999999},"min":{"1min":0.47599999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":9.1110000000000007,"5min":9.1110000000000007,"15min":9.1110000000000007},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.64900000000000002,"15min":0.65700000000000003},"min":{"1min":0.48199999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88100000000000001,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":0.65000000000000002}]},{"osd":4,"last update":"Fri Jul 14 18:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.72899999999999998,"15min":0.74199999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84699999999999998,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.73899999999999999,"15min":0.751},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.89900000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68999999999999995}]},{"osd":5,"last update":"Fri Jul 14 18:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.47799999999999998,"15min":0.46000000000000002},"min":{"1min":0.33200000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.68100000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52300000000000002,"15min":0.49099999999999999},"min":{"1min":0.33100000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.752,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.49099999999999999}]},{"osd":6,"last update":"Fri Jul 14 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.71199999999999997,"15min":0.65600000000000003},"min":{"1min":0.52600000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":6.5389999999999997,"5min":6.5389999999999997,"15min":6.5389999999999997},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67100000000000004,"15min":0.63600000000000001},"min":{"1min":0.46899999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.94599999999999995,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.69799999999999995}]},{"osd":7,"last update":"Fri Jul 14 18:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.79200000000000004,"15min":0.79400000000000004},"min":{"1min":0.59999999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.97299999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.79300000000000004,"15min":0.80400000000000005},"min":{"1min":0.53500000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94599999999999995,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.76200000000000001}]},{"osd":8,"last update":"Fri Jul 14 18:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.65400000000000003,"15min":0.64600000000000002},"min":{"1min":0.50900000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.64800000000000002,"15min":0.63800000000000001},"min":{"1min":0.40500000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.94499999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63300000000000001}]},{"osd":9,"last update":"Fri Jul 14 18:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.78700000000000003,"15min":0.78600000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0149999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.82299999999999995,"15min":0.81399999999999995},"min":{"1min":0.65800000000000003,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.73699999999999999}]},{"osd":10,"last update":"Fri Jul 14 18:16:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.79000000000000004,"15min":0.76200000000000001},"min":{"1min":0.63800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0529999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.79200000000000004,"15min":0.77300000000000002},"min":{"1min":0.54100000000000004,"5min":0.438,"15min":0.438},"max":{"1min":1.0109999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.71199999999999997}]},{"osd":11,"last update":"Fri Jul 14 18:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.82999999999999996,"15min":0.83899999999999997},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.99399999999999999,"5min":1.04,"15min":1.04},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.85799999999999998,"15min":0.86399999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0229999999999999,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.751}]}]},{"osd":3,"up_from":26,"seq":111669149771,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":507355,"data_compressed":23584,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.44900000000000001,"15min":0.39700000000000002},"min":{"1min":0.34300000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.74099999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.47099999999999997,"15min":0.42099999999999999},"min":{"1min":0.311,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.96199999999999997,"5min":0.998,"15min":0.998},"last":0.42099999999999999}]},{"osd":1,"last update":"Fri Jul 14 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.61899999999999999,"15min":0.58299999999999996},"min":{"1min":0.48799999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.64100000000000001,"15min":0.59499999999999997},"min":{"1min":0.51400000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.874,"5min":1.29,"15min":1.29},"last":0.53800000000000003}]},{"osd":2,"last update":"Fri Jul 14 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.64900000000000002,"15min":0.59099999999999997},"min":{"1min":0.55200000000000005,"5min":0.44,"15min":0.44},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.64000000000000001,"15min":0.60699999999999998},"min":{"1min":0.46999999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.81299999999999994}]},{"osd":4,"last update":"Fri Jul 14 18:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.65400000000000003,"15min":0.59799999999999998},"min":{"1min":0.48699999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.123,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.68000000000000005,"15min":0.64000000000000001},"min":{"1min":0.50800000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94999999999999996,"5min":1.153,"15min":1.153},"last":0.68500000000000005}]},{"osd":5,"last update":"Fri Jul 14 18:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.67100000000000004,"15min":0.63800000000000001},"min":{"1min":0.46300000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.86599999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.753},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68200000000000005,"15min":0.627},"min":{"1min":0.53900000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.877,"5min":1.149,"15min":1.149},"last":0.79000000000000004}]},{"osd":6,"last update":"Fri Jul 14 18:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.61299999999999999,"15min":0.59699999999999998},"min":{"1min":0.41799999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85099999999999998,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.66000000000000003,"15min":0.624},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0649999999999999,"5min":1.504,"15min":1.504},"last":0.83399999999999996}]},{"osd":7,"last update":"Fri Jul 14 18:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72299999999999998,"15min":0.69299999999999995},"min":{"1min":0.51600000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.105,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.75,"15min":0.71199999999999997},"min":{"1min":0.60299999999999998,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.96199999999999997,"5min":1.407,"15min":1.407},"last":0.71999999999999997}]},{"osd":8,"last update":"Fri Jul 14 18:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.72799999999999998}]},{"osd":9,"last update":"Fri Jul 14 18:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.79200000000000004,"15min":0.81299999999999994},"min":{"1min":0.51700000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.2450000000000001,"5min":1.619,"15min":1.619},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.86299999999999999,"15min":0.90100000000000002},"min":{"1min":0.57799999999999996,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2689999999999999,"5min":1.573,"15min":1.573},"last":0.80200000000000005}]},{"osd":10,"last update":"Fri Jul 14 18:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.85899999999999999,"15min":0.88100000000000001},"min":{"1min":0.65700000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.94399999999999995,"5min":1.542,"15min":1.542},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.82399999999999995,"15min":0.84999999999999998},"min":{"1min":0.61899999999999999,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.089,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.77900000000000003}]},{"osd":11,"last update":"Fri Jul 14 18:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.84999999999999998,"15min":0.85699999999999998},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":0.93400000000000005,"5min":1.472,"15min":1.472},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.79600000000000004,"15min":0.80700000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.92100000000000004,"5min":1.206,"15min":1.206},"last":0.82599999999999996}]}]},{"osd":4,"up_from":33,"seq":141733920838,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":513184,"data_compressed":23584,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.48999999999999999,"15min":0.46000000000000002},"min":{"1min":0.39700000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.83799999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.50800000000000001,"15min":0.47799999999999998},"min":{"1min":0.46700000000000003,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.747,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.39600000000000002}]},{"osd":1,"last update":"Fri Jul 14 18:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.45000000000000001,"15min":0.38200000000000001},"min":{"1min":0.26400000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.55000000000000004,"5min":3.7890000000000001,"15min":3.7890000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.435,"15min":0.39300000000000002},"min":{"1min":0.23899999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.57799999999999996,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.56399999999999995}]},{"osd":2,"last update":"Fri Jul 14 18:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60399999999999998,"15min":0.55800000000000005},"min":{"1min":0.34699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.875,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.65000000000000002,"15min":0.56599999999999995},"min":{"1min":0.47099999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.80500000000000005,"5min":3.6989999999999998,"15min":3.6989999999999998},"last":0.82799999999999996}]},{"osd":3,"last update":"Fri Jul 14 18:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.63300000000000001,"15min":0.54600000000000004},"min":{"1min":0.34100000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":3.5190000000000001,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.623,"15min":0.54900000000000004},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":3.593,"5min":3.593,"15min":3.593},"last":0.59199999999999997}]},{"osd":5,"last update":"Fri Jul 14 18:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.67600000000000005,"15min":0.58799999999999997},"min":{"1min":0.41299999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83999999999999997,"5min":3.7490000000000001,"15min":3.7490000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65900000000000003,"15min":0.58599999999999997},"min":{"1min":0.434,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.78600000000000003}]},{"osd":6,"last update":"Fri Jul 14 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.747,"15min":0.68300000000000005},"min":{"1min":0.46700000000000003,"5min":0.441,"15min":0.441},"max":{"1min":4.117,"5min":4.117,"15min":4.117},"last":0.747},{"interface":"front","average":{"1min":1.054,"5min":0.76200000000000001,"15min":0.69199999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":4.1440000000000001,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":0.88500000000000001}]},{"osd":7,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.66900000000000004,"15min":0.59699999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.872,"5min":3.8999999999999999,"15min":3.8999999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67400000000000004,"15min":0.60999999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.95799999999999996,"5min":3.2250000000000001,"15min":3.2250000000000001},"last":0.90600000000000003}]},{"osd":8,"last update":"Fri Jul 14 18:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.68999999999999995,"15min":0.65400000000000003},"min":{"1min":0.51000000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.874,"5min":2.359,"15min":2.359},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.80400000000000005,"15min":0.76600000000000001},"min":{"1min":0.52000000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.0149999999999999,"5min":3.1739999999999999,"15min":3.1739999999999999},"last":0.85399999999999998}]},{"osd":9,"last update":"Fri Jul 14 18:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.84399999999999997,"15min":0.80800000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":4.3310000000000004,"5min":4.3310000000000004,"15min":4.3310000000000004},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.876,"5min":0.76700000000000002,"15min":0.71899999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":3.766,"5min":3.766,"15min":3.766},"last":0.81699999999999995}]},{"osd":10,"last update":"Fri Jul 14 18:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.80800000000000005,"15min":0.79000000000000004},"min":{"1min":0.39300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91300000000000003,"5min":3.8239999999999998,"15min":3.8239999999999998},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.74099999999999999,"15min":0.73599999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91100000000000003,"5min":2.52,"15min":2.52},"last":0.873}]},{"osd":11,"last update":"Fri Jul 14 18:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.96099999999999997,"15min":0.99099999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.002,"5min":3.548,"15min":3.548},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.93200000000000005,"15min":0.94799999999999995},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.0269999999999999,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":0.96199999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":1103479,"data_compressed":59328,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.36799999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68100000000000005,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.55400000000000005,"15min":0.57399999999999995},"min":{"1min":0.379,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.621,"5min":1.645,"15min":1.645},"last":0.52900000000000003}]},{"osd":1,"last update":"Fri Jul 14 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.61299999999999999,"15min":0.56599999999999995},"min":{"1min":0.40999999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.874,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60299999999999998,"15min":0.57699999999999996},"min":{"1min":0.378,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80700000000000005,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.621}]},{"osd":2,"last update":"Fri Jul 14 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.43099999999999999,"15min":0.42699999999999999},"min":{"1min":0.33200000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46000000000000002,"15min":0.48699999999999999},"min":{"1min":0.28699999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.71099999999999997,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.71099999999999997}]},{"osd":3,"last update":"Fri Jul 14 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58199999999999996,"15min":0.61099999999999999},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77300000000000002,"5min":1.591,"15min":1.591},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.46700000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.84699999999999998}]},{"osd":4,"last update":"Fri Jul 14 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.626,"15min":0.62},"min":{"1min":0.47499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82799999999999996,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.63600000000000001,"15min":0.624},"min":{"1min":0.35999999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91200000000000003,"5min":1.504,"15min":1.504},"last":0.68999999999999995}]},{"osd":6,"last update":"Fri Jul 14 18:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64400000000000002,"15min":0.65700000000000003},"min":{"1min":0.45100000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.82699999999999996,"5min":1.627,"15min":1.627},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65900000000000003,"15min":0.63900000000000001},"min":{"1min":0.45800000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.86499999999999999,"5min":1.036,"15min":1.036},"last":0.89600000000000002}]},{"osd":7,"last update":"Fri Jul 14 18:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68400000000000005,"15min":0.68500000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0089999999999999,"5min":1.554,"15min":1.554},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.68799999999999994,"15min":0.66100000000000003},"min":{"1min":0.54000000000000004,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92500000000000004,"5min":1.01,"15min":1.01},"last":0.93899999999999995}]},{"osd":8,"last update":"Fri Jul 14 18:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59399999999999997,"15min":0.57999999999999996},"min":{"1min":0.36599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80600000000000005,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59499999999999997,"15min":0.58299999999999996},"min":{"1min":0.41199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.86299999999999999}]},{"osd":9,"last update":"Fri Jul 14 18:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.57099999999999995,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.88400000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.02,"5min":0.81499999999999995,"15min":0.78500000000000003},"min":{"1min":0.58699999999999997,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":6.0460000000000003,"5min":6.0460000000000003,"15min":6.0460000000000003},"last":0.97399999999999998}]},{"osd":10,"last update":"Fri Jul 14 18:16:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.746,"15min":0.72999999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.77200000000000002,"15min":0.755},"min":{"1min":0.56899999999999995,"5min":0.505,"15min":0.505},"max":{"1min":0.96799999999999997,"5min":1.107,"15min":1.107},"last":1.008}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34102,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34102,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34102,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":66480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T18:16:31.355 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T18:16:31.356 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:30 smithi161 ceph-mon[128993]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2023-07-14T18:16:31.356 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:30 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1015134702' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:16:31.356 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:30 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2173150128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:16:31.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:30 smithi171 ceph-mon[129241]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2023-07-14T18:16:31.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:30 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1015134702' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:16:31.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:30 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2173150128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:16:31.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.533+0000 7f05d3fff700 1 -- 172.21.15.133:0/2407018854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05d40ffb20 msgr2=0x7f05d40fff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:31.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.533+0000 7f05d3fff700 1 --2- 172.21.15.133:0/2407018854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05d40ffb20 0x7f05d40fff30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f05c4009990 tx=0x7f05c4009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:31.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.533+0000 7f05d3fff700 1 -- 172.21.15.133:0/2407018854 shutdown_connections 2023-07-14T18:16:31.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.533+0000 7f05d3fff700 1 --2- 172.21.15.133:0/2407018854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05d41016a0 0x7f05d4107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:31.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.533+0000 7f05d3fff700 1 --2- 172.21.15.133:0/2407018854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05d40ffb20 0x7f05d40fff30 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:31.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.533+0000 7f05d3fff700 1 --2- 172.21.15.133:0/2407018854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05d4100d10 0x7f05d4101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:31.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.533+0000 7f05d3fff700 1 -- 172.21.15.133:0/2407018854 >> 172.21.15.133:0/2407018854 conn(0x7f05d40fb140 msgr2=0x7f05d40fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:31.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.533+0000 7f05d3fff700 1 -- 172.21.15.133:0/2407018854 shutdown_connections 2023-07-14T18:16:31.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.533+0000 7f05d3fff700 1 -- 172.21.15.133:0/2407018854 wait complete. 2023-07-14T18:16:31.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.534+0000 7f05d3fff700 1 Processor -- start 2023-07-14T18:16:31.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.534+0000 7f05d3fff700 1 -- start start 2023-07-14T18:16:31.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.535+0000 7f05d3fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05d40ffb20 0x7f05d40743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:31.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.535+0000 7f05d3fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05d4100d10 0x7f05d4072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:31.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.535+0000 7f05d3fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05d41016a0 0x7f05d4072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:31.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.535+0000 7f05d3fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d4073710 con 0x7f05d40ffb20 2023-07-14T18:16:31.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.535+0000 7f05d3fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d4073880 con 0x7f05d4100d10 2023-07-14T18:16:31.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.535+0000 7f05d3fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d419f710 con 0x7f05d41016a0 2023-07-14T18:16:31.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.535+0000 7f05d2ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05d40ffb20 0x7f05d40743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:31.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.535+0000 7f05d2ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05d40ffb20 0x7f05d40743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43672/0 (socket says 172.21.15.133:43672) 2023-07-14T18:16:31.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.535+0000 7f05d2ffd700 1 -- 172.21.15.133:0/3253899062 learned_addr learned my addr 172.21.15.133:0/3253899062 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:31.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.535+0000 7f05d37fe700 1 --2- 172.21.15.133:0/3253899062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05d41016a0 0x7f05d4072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:31.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.535+0000 7f05cbfff700 1 --2- 172.21.15.133:0/3253899062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05d4100d10 0x7f05d4072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:31.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.535+0000 7f05d2ffd700 1 -- 172.21.15.133:0/3253899062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05d41016a0 msgr2=0x7f05d4072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:31.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.535+0000 7f05d2ffd700 1 --2- 172.21.15.133:0/3253899062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05d41016a0 0x7f05d4072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:31.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.535+0000 7f05d2ffd700 1 -- 172.21.15.133:0/3253899062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05d4100d10 msgr2=0x7f05d4072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:31.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.535+0000 7f05d2ffd700 1 --2- 172.21.15.133:0/3253899062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05d4100d10 0x7f05d4072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:31.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.535+0000 7f05d2ffd700 1 -- 172.21.15.133:0/3253899062 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c40095f0 con 0x7f05d40ffb20 2023-07-14T18:16:31.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.535+0000 7f05cbfff700 1 --2- 172.21.15.133:0/3253899062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05d4100d10 0x7f05d4072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:16:31.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.536+0000 7f05d2ffd700 1 --2- 172.21.15.133:0/3253899062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05d40ffb20 0x7f05d40743f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f05c4005740 tx=0x7f05c4005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:31.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.536+0000 7f05d0ff9700 1 -- 172.21.15.133:0/3253899062 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c401d070 con 0x7f05d40ffb20 2023-07-14T18:16:31.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.536+0000 7f05d3fff700 1 -- 172.21.15.133:0/3253899062 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05d419f990 con 0x7f05d40ffb20 2023-07-14T18:16:31.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.536+0000 7f05d0ff9700 1 -- 172.21.15.133:0/3253899062 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05c4021410 con 0x7f05d40ffb20 2023-07-14T18:16:31.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.536+0000 7f05d0ff9700 1 -- 172.21.15.133:0/3253899062 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c4017650 con 0x7f05d40ffb20 2023-07-14T18:16:31.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.536+0000 7f05d3fff700 1 -- 172.21.15.133:0/3253899062 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d419fe10 con 0x7f05d40ffb20 2023-07-14T18:16:31.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.537+0000 7f05d3fff700 1 -- 172.21.15.133:0/3253899062 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05d4106fc0 con 0x7f05d40ffb20 2023-07-14T18:16:31.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.539+0000 7f05d0ff9700 1 -- 172.21.15.133:0/3253899062 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f05c4021580 con 0x7f05d40ffb20 2023-07-14T18:16:31.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.540+0000 7f05d0ff9700 1 --2- 172.21.15.133:0/3253899062 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f05c00a1240 0x7f05c00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:31.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.540+0000 7f05d0ff9700 1 -- 172.21.15.133:0/3253899062 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f05c40ce620 con 0x7f05d40ffb20 2023-07-14T18:16:31.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.540+0000 7f05cbfff700 1 --2- 172.21.15.133:0/3253899062 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f05c00a1240 0x7f05c00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:31.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.541+0000 7f05cbfff700 1 --2- 172.21.15.133:0/3253899062 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f05c00a1240 0x7f05c00a36f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f05bc009cb0 tx=0x7f05bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:31.587 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.542+0000 7f05d0ff9700 1 -- 172.21.15.133:0/3253899062 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f05c4097b00 con 0x7f05d40ffb20 2023-07-14T18:16:31.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.738+0000 7f05d3fff700 1 -- 172.21.15.133:0/3253899062 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f05d4060a80 con 0x7f05c00a1240 2023-07-14T18:16:31.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.750+0000 7f05d0ff9700 1 -- 172.21.15.133:0/3253899062 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426624 (secure 0 0 0) 0x7f05d4060a80 con 0x7f05c00a1240 2023-07-14T18:16:31.800 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:31.803 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.755+0000 7f05d3fff700 1 -- 172.21.15.133:0/3253899062 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f05c00a1240 msgr2=0x7f05c00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:31.803 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.755+0000 7f05d3fff700 1 --2- 172.21.15.133:0/3253899062 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f05c00a1240 0x7f05c00a36f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f05bc009cb0 tx=0x7f05bc009260 comp rx=0 tx=0).stop 2023-07-14T18:16:31.803 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.755+0000 7f05d3fff700 1 -- 172.21.15.133:0/3253899062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05d40ffb20 msgr2=0x7f05d40743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:31.803 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.755+0000 7f05d3fff700 1 --2- 172.21.15.133:0/3253899062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05d40ffb20 0x7f05d40743f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f05c4005740 tx=0x7f05c4005770 comp rx=0 tx=0).stop 2023-07-14T18:16:31.803 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.756+0000 7f05d3fff700 1 -- 172.21.15.133:0/3253899062 shutdown_connections 2023-07-14T18:16:31.803 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.756+0000 7f05d3fff700 1 --2- 172.21.15.133:0/3253899062 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f05c00a1240 0x7f05c00a36f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:31.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.756+0000 7f05d3fff700 1 --2- 172.21.15.133:0/3253899062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05d41016a0 0x7f05d4072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:31.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.756+0000 7f05d3fff700 1 --2- 172.21.15.133:0/3253899062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05d40ffb20 0x7f05d40743f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:31.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.756+0000 7f05d3fff700 1 --2- 172.21.15.133:0/3253899062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05d4100d10 0x7f05d4072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:31.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.756+0000 7f05d3fff700 1 -- 172.21.15.133:0/3253899062 >> 172.21.15.133:0/3253899062 conn(0x7f05d40fb140 msgr2=0x7f05d4104100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:31.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.756+0000 7f05d3fff700 1 -- 172.21.15.133:0/3253899062 shutdown_connections 2023-07-14T18:16:31.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:31.756+0000 7f05d3fff700 1 -- 172.21.15.133:0/3253899062 wait complete. 2023-07-14T18:16:31.804 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2023-07-14T18:16:31.820 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2023-07-14T18:16:31.466798+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660727,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":746,"num_read_kb":493,"num_write":352,"num_write_kb":3066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1181441,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336492,"kb_used_data":12300,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124605332,"statfs":{"total":1151940427776,"available":1151595859968,"internally_reserved":0,"allocated":12595200,"data_stored":8098401,"data_compressed":390240,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6152,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":40,"num_write":8,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003021"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.487205+0000","last_change":"2023-07-14T18:14:17.927179+0000","last_active":"2023-07-14T18:16:17.487205+0000","last_peered":"2023-07-14T18:16:17.487205+0000","last_clean":"2023-07-14T18:16:17.487205+0000","last_became_active":"2023-07-14T18:14:17.926880+0000","last_became_peered":"2023-07-14T18:14:17.926880+0000","last_unstale":"2023-07-14T18:16:17.487205+0000","last_undegraded":"2023-07-14T18:16:17.487205+0000","last_fullsized":"2023-07-14T18:16:17.487205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:41:35.598778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.483578+0000","last_change":"2023-07-14T18:14:17.933834+0000","last_active":"2023-07-14T18:16:17.483578+0000","last_peered":"2023-07-14T18:16:17.483578+0000","last_clean":"2023-07-14T18:16:17.483578+0000","last_became_active":"2023-07-14T18:14:17.933529+0000","last_became_peered":"2023-07-14T18:14:17.933529+0000","last_unstale":"2023-07-14T18:16:17.483578+0000","last_undegraded":"2023-07-14T18:16:17.483578+0000","last_fullsized":"2023-07-14T18:16:17.483578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:48:46.885347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.479455+0000","last_change":"2023-07-14T18:14:17.928400+0000","last_active":"2023-07-14T18:16:17.479455+0000","last_peered":"2023-07-14T18:16:17.479455+0000","last_clean":"2023-07-14T18:16:17.479455+0000","last_became_active":"2023-07-14T18:14:17.928077+0000","last_became_peered":"2023-07-14T18:14:17.928077+0000","last_unstale":"2023-07-14T18:16:17.479455+0000","last_undegraded":"2023-07-14T18:16:17.479455+0000","last_fullsized":"2023-07-14T18:16:17.479455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:52:23.012871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.472507+0000","last_change":"2023-07-14T18:14:17.932623+0000","last_active":"2023-07-14T18:16:17.472507+0000","last_peered":"2023-07-14T18:16:17.472507+0000","last_clean":"2023-07-14T18:16:17.472507+0000","last_became_active":"2023-07-14T18:14:17.932298+0000","last_became_peered":"2023-07-14T18:14:17.932298+0000","last_unstale":"2023-07-14T18:16:17.472507+0000","last_undegraded":"2023-07-14T18:16:17.472507+0000","last_fullsized":"2023-07-14T18:16:17.472507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:09:59.192311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.469953+0000","last_change":"2023-07-14T18:14:17.927109+0000","last_active":"2023-07-14T18:16:17.469953+0000","last_peered":"2023-07-14T18:16:17.469953+0000","last_clean":"2023-07-14T18:16:17.469953+0000","last_became_active":"2023-07-14T18:14:17.926800+0000","last_became_peered":"2023-07-14T18:14:17.926800+0000","last_unstale":"2023-07-14T18:16:17.469953+0000","last_undegraded":"2023-07-14T18:16:17.469953+0000","last_fullsized":"2023-07-14T18:16:17.469953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:59:27.287187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.481171+0000","last_change":"2023-07-14T18:14:17.935380+0000","last_active":"2023-07-14T18:16:17.481171+0000","last_peered":"2023-07-14T18:16:17.481171+0000","last_clean":"2023-07-14T18:16:17.481171+0000","last_became_active":"2023-07-14T18:14:17.935099+0000","last_became_peered":"2023-07-14T18:14:17.935099+0000","last_unstale":"2023-07-14T18:16:17.481171+0000","last_undegraded":"2023-07-14T18:16:17.481171+0000","last_fullsized":"2023-07-14T18:16:17.481171+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:22:36.754253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.482663+0000","last_change":"2023-07-14T18:14:17.930207+0000","last_active":"2023-07-14T18:16:17.482663+0000","last_peered":"2023-07-14T18:16:17.482663+0000","last_clean":"2023-07-14T18:16:17.482663+0000","last_became_active":"2023-07-14T18:14:17.929898+0000","last_became_peered":"2023-07-14T18:14:17.929898+0000","last_unstale":"2023-07-14T18:16:17.482663+0000","last_undegraded":"2023-07-14T18:16:17.482663+0000","last_fullsized":"2023-07-14T18:16:17.482663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:53:19.205250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.473699+0000","last_change":"2023-07-14T18:14:17.928645+0000","last_active":"2023-07-14T18:16:17.473699+0000","last_peered":"2023-07-14T18:16:17.473699+0000","last_clean":"2023-07-14T18:16:17.473699+0000","last_became_active":"2023-07-14T18:14:17.928288+0000","last_became_peered":"2023-07-14T18:14:17.928288+0000","last_unstale":"2023-07-14T18:16:17.473699+0000","last_undegraded":"2023-07-14T18:16:17.473699+0000","last_fullsized":"2023-07-14T18:16:17.473699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:37:39.330884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.477077+0000","last_change":"2023-07-14T18:14:17.925888+0000","last_active":"2023-07-14T18:16:17.477077+0000","last_peered":"2023-07-14T18:16:17.477077+0000","last_clean":"2023-07-14T18:16:17.477077+0000","last_became_active":"2023-07-14T18:14:17.925580+0000","last_became_peered":"2023-07-14T18:14:17.925580+0000","last_unstale":"2023-07-14T18:16:17.477077+0000","last_undegraded":"2023-07-14T18:16:17.477077+0000","last_fullsized":"2023-07-14T18:16:17.477077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:09:13.877679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.482686+0000","last_change":"2023-07-14T18:14:17.924440+0000","last_active":"2023-07-14T18:16:17.482686+0000","last_peered":"2023-07-14T18:16:17.482686+0000","last_clean":"2023-07-14T18:16:17.482686+0000","last_became_active":"2023-07-14T18:14:17.923839+0000","last_became_peered":"2023-07-14T18:14:17.923839+0000","last_unstale":"2023-07-14T18:16:17.482686+0000","last_undegraded":"2023-07-14T18:16:17.482686+0000","last_fullsized":"2023-07-14T18:16:17.482686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:29:23.092881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.486380+0000","last_change":"2023-07-14T18:14:17.931067+0000","last_active":"2023-07-14T18:16:17.486380+0000","last_peered":"2023-07-14T18:16:17.486380+0000","last_clean":"2023-07-14T18:16:17.486380+0000","last_became_active":"2023-07-14T18:14:17.930633+0000","last_became_peered":"2023-07-14T18:14:17.930633+0000","last_unstale":"2023-07-14T18:16:17.486380+0000","last_undegraded":"2023-07-14T18:16:17.486380+0000","last_fullsized":"2023-07-14T18:16:17.486380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:04:56.675009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.471892+0000","last_change":"2023-07-14T18:14:17.932770+0000","last_active":"2023-07-14T18:16:17.471892+0000","last_peered":"2023-07-14T18:16:17.471892+0000","last_clean":"2023-07-14T18:16:17.471892+0000","last_became_active":"2023-07-14T18:14:17.932438+0000","last_became_peered":"2023-07-14T18:14:17.932438+0000","last_unstale":"2023-07-14T18:16:17.471892+0000","last_undegraded":"2023-07-14T18:16:17.471892+0000","last_fullsized":"2023-07-14T18:16:17.471892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:18:53.813312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.482161+0000","last_change":"2023-07-14T18:14:17.938213+0000","last_active":"2023-07-14T18:16:17.482161+0000","last_peered":"2023-07-14T18:16:17.482161+0000","last_clean":"2023-07-14T18:16:17.482161+0000","last_became_active":"2023-07-14T18:14:17.937565+0000","last_became_peered":"2023-07-14T18:14:17.937565+0000","last_unstale":"2023-07-14T18:16:17.482161+0000","last_undegraded":"2023-07-14T18:16:17.482161+0000","last_fullsized":"2023-07-14T18:16:17.482161+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:51:47.121447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.483823+0000","last_change":"2023-07-14T18:14:17.934677+0000","last_active":"2023-07-14T18:16:17.483823+0000","last_peered":"2023-07-14T18:16:17.483823+0000","last_clean":"2023-07-14T18:16:17.483823+0000","last_became_active":"2023-07-14T18:14:17.933622+0000","last_became_peered":"2023-07-14T18:14:17.933622+0000","last_unstale":"2023-07-14T18:16:17.483823+0000","last_undegraded":"2023-07-14T18:16:17.483823+0000","last_fullsized":"2023-07-14T18:16:17.483823+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:26:28.437669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.484282+0000","last_change":"2023-07-14T18:14:17.935317+0000","last_active":"2023-07-14T18:16:17.484282+0000","last_peered":"2023-07-14T18:16:17.484282+0000","last_clean":"2023-07-14T18:16:17.484282+0000","last_became_active":"2023-07-14T18:14:17.935034+0000","last_became_peered":"2023-07-14T18:14:17.935034+0000","last_unstale":"2023-07-14T18:16:17.484282+0000","last_undegraded":"2023-07-14T18:16:17.484282+0000","last_fullsized":"2023-07-14T18:16:17.484282+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:42:03.287591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.488504+0000","last_change":"2023-07-14T18:14:17.928222+0000","last_active":"2023-07-14T18:16:17.488504+0000","last_peered":"2023-07-14T18:16:17.488504+0000","last_clean":"2023-07-14T18:16:17.488504+0000","last_became_active":"2023-07-14T18:14:17.927964+0000","last_became_peered":"2023-07-14T18:14:17.927964+0000","last_unstale":"2023-07-14T18:16:17.488504+0000","last_undegraded":"2023-07-14T18:16:17.488504+0000","last_fullsized":"2023-07-14T18:16:17.488504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:42:45.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.486729+0000","last_change":"2023-07-14T18:14:17.927276+0000","last_active":"2023-07-14T18:16:17.486729+0000","last_peered":"2023-07-14T18:16:17.486729+0000","last_clean":"2023-07-14T18:16:17.486729+0000","last_became_active":"2023-07-14T18:14:17.926952+0000","last_became_peered":"2023-07-14T18:14:17.926952+0000","last_unstale":"2023-07-14T18:16:17.486729+0000","last_undegraded":"2023-07-14T18:16:17.486729+0000","last_fullsized":"2023-07-14T18:16:17.486729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:00:09.825668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.476643+0000","last_change":"2023-07-14T18:14:17.937364+0000","last_active":"2023-07-14T18:16:17.476643+0000","last_peered":"2023-07-14T18:16:17.476643+0000","last_clean":"2023-07-14T18:16:17.476643+0000","last_became_active":"2023-07-14T18:14:17.937014+0000","last_became_peered":"2023-07-14T18:14:17.937014+0000","last_unstale":"2023-07-14T18:16:17.476643+0000","last_undegraded":"2023-07-14T18:16:17.476643+0000","last_fullsized":"2023-07-14T18:16:17.476643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:31:10.117800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.472419+0000","last_change":"2023-07-14T18:14:17.920358+0000","last_active":"2023-07-14T18:16:17.472419+0000","last_peered":"2023-07-14T18:16:17.472419+0000","last_clean":"2023-07-14T18:16:17.472419+0000","last_became_active":"2023-07-14T18:14:17.920042+0000","last_became_peered":"2023-07-14T18:14:17.920042+0000","last_unstale":"2023-07-14T18:16:17.472419+0000","last_undegraded":"2023-07-14T18:16:17.472419+0000","last_fullsized":"2023-07-14T18:16:17.472419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:07:30.258643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093399+0000","last_change":"2023-07-14T18:14:22.045376+0000","last_active":"2023-07-14T18:15:50.093399+0000","last_peered":"2023-07-14T18:15:50.093399+0000","last_clean":"2023-07-14T18:15:50.093399+0000","last_became_active":"2023-07-14T18:14:22.044984+0000","last_became_peered":"2023-07-14T18:14:22.044984+0000","last_unstale":"2023-07-14T18:15:50.093399+0000","last_undegraded":"2023-07-14T18:15:50.093399+0000","last_fullsized":"2023-07-14T18:15:50.093399+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:26:13.981694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964742+0000","last_change":"2023-07-14T18:14:48.954270+0000","last_active":"2023-07-14T18:15:49.964742+0000","last_peered":"2023-07-14T18:15:49.964742+0000","last_clean":"2023-07-14T18:15:49.964742+0000","last_became_active":"2023-07-14T18:14:48.947868+0000","last_became_peered":"2023-07-14T18:14:48.947868+0000","last_unstale":"2023-07-14T18:15:49.964742+0000","last_undegraded":"2023-07-14T18:15:49.964742+0000","last_fullsized":"2023-07-14T18:15:49.964742+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:56:18.186337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.472946+0000","last_change":"2023-07-14T18:14:17.925822+0000","last_active":"2023-07-14T18:16:17.472946+0000","last_peered":"2023-07-14T18:16:17.472946+0000","last_clean":"2023-07-14T18:16:17.472946+0000","last_became_active":"2023-07-14T18:14:17.925508+0000","last_became_peered":"2023-07-14T18:14:17.925508+0000","last_unstale":"2023-07-14T18:16:17.472946+0000","last_undegraded":"2023-07-14T18:16:17.472946+0000","last_fullsized":"2023-07-14T18:16:17.472946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:05:04.319300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958344+0000","last_change":"2023-07-14T18:14:22.043098+0000","last_active":"2023-07-14T18:15:49.958344+0000","last_peered":"2023-07-14T18:15:49.958344+0000","last_clean":"2023-07-14T18:15:49.958344+0000","last_became_active":"2023-07-14T18:14:22.042517+0000","last_became_peered":"2023-07-14T18:14:22.042517+0000","last_unstale":"2023-07-14T18:15:49.958344+0000","last_undegraded":"2023-07-14T18:15:49.958344+0000","last_fullsized":"2023-07-14T18:15:49.958344+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:46:20.942876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.095016+0000","last_change":"2023-07-14T18:14:16.927612+0000","last_active":"2023-07-14T18:15:50.095016+0000","last_peered":"2023-07-14T18:15:50.095016+0000","last_clean":"2023-07-14T18:15:50.095016+0000","last_became_active":"2023-07-14T18:14:16.927284+0000","last_became_peered":"2023-07-14T18:14:16.927284+0000","last_unstale":"2023-07-14T18:15:50.095016+0000","last_undegraded":"2023-07-14T18:15:50.095016+0000","last_fullsized":"2023-07-14T18:15:50.095016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:49:41.365291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.488089+0000","last_change":"2023-07-14T18:14:17.930624+0000","last_active":"2023-07-14T18:16:17.488089+0000","last_peered":"2023-07-14T18:16:17.488089+0000","last_clean":"2023-07-14T18:16:17.488089+0000","last_became_active":"2023-07-14T18:14:17.930273+0000","last_became_peered":"2023-07-14T18:14:17.930273+0000","last_unstale":"2023-07-14T18:16:17.488089+0000","last_undegraded":"2023-07-14T18:16:17.488089+0000","last_fullsized":"2023-07-14T18:16:17.488089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:08:45.137173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958582+0000","last_change":"2023-07-14T18:14:49.543208+0000","last_active":"2023-07-14T18:15:49.958582+0000","last_peered":"2023-07-14T18:15:49.958582+0000","last_clean":"2023-07-14T18:15:49.958582+0000","last_became_active":"2023-07-14T18:14:49.542685+0000","last_became_peered":"2023-07-14T18:14:49.542685+0000","last_unstale":"2023-07-14T18:15:49.958582+0000","last_undegraded":"2023-07-14T18:15:49.958582+0000","last_fullsized":"2023-07-14T18:15:49.958582+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:34:49.230001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.959423+0000","last_change":"2023-07-14T18:14:16.926185+0000","last_active":"2023-07-14T18:15:49.959423+0000","last_peered":"2023-07-14T18:15:49.959423+0000","last_clean":"2023-07-14T18:15:49.959423+0000","last_became_active":"2023-07-14T18:14:16.925874+0000","last_became_peered":"2023-07-14T18:14:16.925874+0000","last_unstale":"2023-07-14T18:15:49.959423+0000","last_undegraded":"2023-07-14T18:15:49.959423+0000","last_fullsized":"2023-07-14T18:15:49.959423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:54:20.071669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.483223+0000","last_change":"2023-07-14T18:14:17.936601+0000","last_active":"2023-07-14T18:16:17.483223+0000","last_peered":"2023-07-14T18:16:17.483223+0000","last_clean":"2023-07-14T18:16:17.483223+0000","last_became_active":"2023-07-14T18:14:17.936291+0000","last_became_peered":"2023-07-14T18:14:17.936291+0000","last_unstale":"2023-07-14T18:16:17.483223+0000","last_undegraded":"2023-07-14T18:16:17.483223+0000","last_fullsized":"2023-07-14T18:16:17.483223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:32:59.220779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958529+0000","last_change":"2023-07-14T18:14:22.042952+0000","last_active":"2023-07-14T18:15:49.958529+0000","last_peered":"2023-07-14T18:15:49.958529+0000","last_clean":"2023-07-14T18:15:49.958529+0000","last_became_active":"2023-07-14T18:14:22.042256+0000","last_became_peered":"2023-07-14T18:14:22.042256+0000","last_unstale":"2023-07-14T18:15:49.958529+0000","last_undegraded":"2023-07-14T18:15:49.958529+0000","last_fullsized":"2023-07-14T18:15:49.958529+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:00:33.594352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958938+0000","last_change":"2023-07-14T18:14:16.920902+0000","last_active":"2023-07-14T18:15:49.958938+0000","last_peered":"2023-07-14T18:15:49.958938+0000","last_clean":"2023-07-14T18:15:49.958938+0000","last_became_active":"2023-07-14T18:14:16.920554+0000","last_became_peered":"2023-07-14T18:14:16.920554+0000","last_unstale":"2023-07-14T18:15:49.958938+0000","last_undegraded":"2023-07-14T18:15:49.958938+0000","last_fullsized":"2023-07-14T18:15:49.958938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:42:47.908351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.477004+0000","last_change":"2023-07-14T18:14:17.925005+0000","last_active":"2023-07-14T18:16:17.477004+0000","last_peered":"2023-07-14T18:16:17.477004+0000","last_clean":"2023-07-14T18:16:17.477004+0000","last_became_active":"2023-07-14T18:14:17.924723+0000","last_became_peered":"2023-07-14T18:14:17.924723+0000","last_unstale":"2023-07-14T18:16:17.477004+0000","last_undegraded":"2023-07-14T18:16:17.477004+0000","last_fullsized":"2023-07-14T18:16:17.477004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:23.634518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093304+0000","last_change":"2023-07-14T18:14:22.038482+0000","last_active":"2023-07-14T18:15:50.093304+0000","last_peered":"2023-07-14T18:15:50.093304+0000","last_clean":"2023-07-14T18:15:50.093304+0000","last_became_active":"2023-07-14T18:14:22.038059+0000","last_became_peered":"2023-07-14T18:14:22.038059+0000","last_unstale":"2023-07-14T18:15:50.093304+0000","last_undegraded":"2023-07-14T18:15:50.093304+0000","last_fullsized":"2023-07-14T18:15:50.093304+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:14:37.242779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'8","reported_seq":146,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:25.035755+0000","last_change":"2023-07-14T18:14:48.949585+0000","last_active":"2023-07-14T18:16:25.035755+0000","last_peered":"2023-07-14T18:16:25.035755+0000","last_clean":"2023-07-14T18:16:25.035755+0000","last_became_active":"2023-07-14T18:14:48.949106+0000","last_became_peered":"2023-07-14T18:14:48.949106+0000","last_unstale":"2023-07-14T18:16:25.035755+0000","last_undegraded":"2023-07-14T18:16:25.035755+0000","last_fullsized":"2023-07-14T18:16:25.035755+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:10:07.660525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1421,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.471828+0000","last_change":"2023-07-14T18:14:17.934894+0000","last_active":"2023-07-14T18:16:17.471828+0000","last_peered":"2023-07-14T18:16:17.471828+0000","last_clean":"2023-07-14T18:16:17.471828+0000","last_became_active":"2023-07-14T18:14:17.934651+0000","last_became_peered":"2023-07-14T18:14:17.934651+0000","last_unstale":"2023-07-14T18:16:17.471828+0000","last_undegraded":"2023-07-14T18:16:17.471828+0000","last_fullsized":"2023-07-14T18:16:17.471828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:56:46.813395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093307+0000","last_change":"2023-07-14T18:14:22.043611+0000","last_active":"2023-07-14T18:15:50.093307+0000","last_peered":"2023-07-14T18:15:50.093307+0000","last_clean":"2023-07-14T18:15:50.093307+0000","last_became_active":"2023-07-14T18:14:22.043101+0000","last_became_peered":"2023-07-14T18:14:22.043101+0000","last_unstale":"2023-07-14T18:15:50.093307+0000","last_undegraded":"2023-07-14T18:15:50.093307+0000","last_fullsized":"2023-07-14T18:15:50.093307+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:14:51.438990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.983824+0000","last_change":"2023-07-14T18:14:16.925321+0000","last_active":"2023-07-14T18:15:49.983824+0000","last_peered":"2023-07-14T18:15:49.983824+0000","last_clean":"2023-07-14T18:15:49.983824+0000","last_became_active":"2023-07-14T18:14:16.925016+0000","last_became_peered":"2023-07-14T18:14:16.925016+0000","last_unstale":"2023-07-14T18:15:49.983824+0000","last_undegraded":"2023-07-14T18:15:49.983824+0000","last_fullsized":"2023-07-14T18:15:49.983824+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:41:08.173620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.464867+0000","last_change":"2023-07-14T18:14:17.928838+0000","last_active":"2023-07-14T18:16:17.464867+0000","last_peered":"2023-07-14T18:16:17.464867+0000","last_clean":"2023-07-14T18:16:17.464867+0000","last_became_active":"2023-07-14T18:14:17.928511+0000","last_became_peered":"2023-07-14T18:14:17.928511+0000","last_unstale":"2023-07-14T18:16:17.464867+0000","last_undegraded":"2023-07-14T18:16:17.464867+0000","last_fullsized":"2023-07-14T18:16:17.464867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:35:21.406449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964630+0000","last_change":"2023-07-14T18:14:22.043182+0000","last_active":"2023-07-14T18:15:49.964630+0000","last_peered":"2023-07-14T18:15:49.964630+0000","last_clean":"2023-07-14T18:15:49.964630+0000","last_became_active":"2023-07-14T18:14:22.042779+0000","last_became_peered":"2023-07-14T18:14:22.042779+0000","last_unstale":"2023-07-14T18:15:49.964630+0000","last_undegraded":"2023-07-14T18:15:49.964630+0000","last_fullsized":"2023-07-14T18:15:49.964630+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:22:42.125442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958189+0000","last_change":"2023-07-14T18:14:16.935321+0000","last_active":"2023-07-14T18:15:49.958189+0000","last_peered":"2023-07-14T18:15:49.958189+0000","last_clean":"2023-07-14T18:15:49.958189+0000","last_became_active":"2023-07-14T18:14:16.934494+0000","last_became_peered":"2023-07-14T18:14:16.934494+0000","last_unstale":"2023-07-14T18:15:49.958189+0000","last_undegraded":"2023-07-14T18:15:49.958189+0000","last_fullsized":"2023-07-14T18:15:49.958189+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:11:15.794692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.480564+0000","last_change":"2023-07-14T18:14:17.922139+0000","last_active":"2023-07-14T18:16:17.480564+0000","last_peered":"2023-07-14T18:16:17.480564+0000","last_clean":"2023-07-14T18:16:17.480564+0000","last_became_active":"2023-07-14T18:14:17.921829+0000","last_became_peered":"2023-07-14T18:14:17.921829+0000","last_unstale":"2023-07-14T18:16:17.480564+0000","last_undegraded":"2023-07-14T18:16:17.480564+0000","last_fullsized":"2023-07-14T18:16:17.480564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:19:51.159357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.094035+0000","last_change":"2023-07-14T18:14:22.033625+0000","last_active":"2023-07-14T18:15:50.094035+0000","last_peered":"2023-07-14T18:15:50.094035+0000","last_clean":"2023-07-14T18:15:50.094035+0000","last_became_active":"2023-07-14T18:14:22.033253+0000","last_became_peered":"2023-07-14T18:14:22.033253+0000","last_unstale":"2023-07-14T18:15:50.094035+0000","last_undegraded":"2023-07-14T18:15:50.094035+0000","last_fullsized":"2023-07-14T18:15:50.094035+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:36:04.591067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964644+0000","last_change":"2023-07-14T18:14:16.923864+0000","last_active":"2023-07-14T18:15:49.964644+0000","last_peered":"2023-07-14T18:15:49.964644+0000","last_clean":"2023-07-14T18:15:49.964644+0000","last_became_active":"2023-07-14T18:14:16.923546+0000","last_became_peered":"2023-07-14T18:14:16.923546+0000","last_unstale":"2023-07-14T18:15:49.964644+0000","last_undegraded":"2023-07-14T18:15:49.964644+0000","last_fullsized":"2023-07-14T18:15:49.964644+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:07:13.280104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.475115+0000","last_change":"2023-07-14T18:14:17.936473+0000","last_active":"2023-07-14T18:16:17.475115+0000","last_peered":"2023-07-14T18:16:17.475115+0000","last_clean":"2023-07-14T18:16:17.475115+0000","last_became_active":"2023-07-14T18:14:17.935387+0000","last_became_peered":"2023-07-14T18:14:17.935387+0000","last_unstale":"2023-07-14T18:16:17.475115+0000","last_undegraded":"2023-07-14T18:16:17.475115+0000","last_fullsized":"2023-07-14T18:16:17.475115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:05:12.007411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093186+0000","last_change":"2023-07-14T18:14:22.041544+0000","last_active":"2023-07-14T18:15:50.093186+0000","last_peered":"2023-07-14T18:15:50.093186+0000","last_clean":"2023-07-14T18:15:50.093186+0000","last_became_active":"2023-07-14T18:14:22.041183+0000","last_became_peered":"2023-07-14T18:14:22.041183+0000","last_unstale":"2023-07-14T18:15:50.093186+0000","last_undegraded":"2023-07-14T18:15:50.093186+0000","last_fullsized":"2023-07-14T18:15:50.093186+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:08:52.284739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'4","reported_seq":130,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:26.037087+0000","last_change":"2023-07-14T18:14:16.928860+0000","last_active":"2023-07-14T18:16:26.037087+0000","last_peered":"2023-07-14T18:16:26.037087+0000","last_clean":"2023-07-14T18:16:26.037087+0000","last_became_active":"2023-07-14T18:14:16.928567+0000","last_became_peered":"2023-07-14T18:14:16.928567+0000","last_unstale":"2023-07-14T18:16:26.037087+0000","last_undegraded":"2023-07-14T18:16:26.037087+0000","last_fullsized":"2023-07-14T18:16:26.037087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:26:39.917215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.484494+0000","last_change":"2023-07-14T18:14:17.938456+0000","last_active":"2023-07-14T18:16:17.484494+0000","last_peered":"2023-07-14T18:16:17.484494+0000","last_clean":"2023-07-14T18:16:17.484494+0000","last_became_active":"2023-07-14T18:14:17.938086+0000","last_became_peered":"2023-07-14T18:14:17.938086+0000","last_unstale":"2023-07-14T18:16:17.484494+0000","last_undegraded":"2023-07-14T18:16:17.484494+0000","last_fullsized":"2023-07-14T18:16:17.484494+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:19:25.486023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964227+0000","last_change":"2023-07-14T18:14:22.042785+0000","last_active":"2023-07-14T18:15:49.964227+0000","last_peered":"2023-07-14T18:15:49.964227+0000","last_clean":"2023-07-14T18:15:49.964227+0000","last_became_active":"2023-07-14T18:14:22.042346+0000","last_became_peered":"2023-07-14T18:14:22.042346+0000","last_unstale":"2023-07-14T18:15:49.964227+0000","last_undegraded":"2023-07-14T18:15:49.964227+0000","last_fullsized":"2023-07-14T18:15:49.964227+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:49:14.017758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.962653+0000","last_change":"2023-07-14T18:14:16.921168+0000","last_active":"2023-07-14T18:15:49.962653+0000","last_peered":"2023-07-14T18:15:49.962653+0000","last_clean":"2023-07-14T18:15:49.962653+0000","last_became_active":"2023-07-14T18:14:16.920808+0000","last_became_peered":"2023-07-14T18:14:16.920808+0000","last_unstale":"2023-07-14T18:15:49.962653+0000","last_undegraded":"2023-07-14T18:15:49.962653+0000","last_fullsized":"2023-07-14T18:15:49.962653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:13:22.466581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.487015+0000","last_change":"2023-07-14T18:14:17.935100+0000","last_active":"2023-07-14T18:16:17.487015+0000","last_peered":"2023-07-14T18:16:17.487015+0000","last_clean":"2023-07-14T18:16:17.487015+0000","last_became_active":"2023-07-14T18:14:17.934794+0000","last_became_peered":"2023-07-14T18:14:17.934794+0000","last_unstale":"2023-07-14T18:16:17.487015+0000","last_undegraded":"2023-07-14T18:16:17.487015+0000","last_fullsized":"2023-07-14T18:16:17.487015+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:06:24.196022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958851+0000","last_change":"2023-07-14T18:14:22.043495+0000","last_active":"2023-07-14T18:15:49.958851+0000","last_peered":"2023-07-14T18:15:49.958851+0000","last_clean":"2023-07-14T18:15:49.958851+0000","last_became_active":"2023-07-14T18:14:22.042686+0000","last_became_peered":"2023-07-14T18:14:22.042686+0000","last_unstale":"2023-07-14T18:15:49.958851+0000","last_undegraded":"2023-07-14T18:15:49.958851+0000","last_fullsized":"2023-07-14T18:15:49.958851+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:50:08.855428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.965517+0000","last_change":"2023-07-14T18:14:16.926006+0000","last_active":"2023-07-14T18:15:49.965517+0000","last_peered":"2023-07-14T18:15:49.965517+0000","last_clean":"2023-07-14T18:15:49.965517+0000","last_became_active":"2023-07-14T18:14:16.925719+0000","last_became_peered":"2023-07-14T18:14:16.925719+0000","last_unstale":"2023-07-14T18:15:49.965517+0000","last_undegraded":"2023-07-14T18:15:49.965517+0000","last_fullsized":"2023-07-14T18:15:49.965517+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:16:46.148528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.480595+0000","last_change":"2023-07-14T18:14:17.934155+0000","last_active":"2023-07-14T18:16:17.480595+0000","last_peered":"2023-07-14T18:16:17.480595+0000","last_clean":"2023-07-14T18:16:17.480595+0000","last_became_active":"2023-07-14T18:14:17.933756+0000","last_became_peered":"2023-07-14T18:14:17.933756+0000","last_unstale":"2023-07-14T18:16:17.480595+0000","last_undegraded":"2023-07-14T18:16:17.480595+0000","last_fullsized":"2023-07-14T18:16:17.480595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:10.480686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964266+0000","last_change":"2023-07-14T18:14:22.039198+0000","last_active":"2023-07-14T18:15:49.964266+0000","last_peered":"2023-07-14T18:15:49.964266+0000","last_clean":"2023-07-14T18:15:49.964266+0000","last_became_active":"2023-07-14T18:14:22.038842+0000","last_became_peered":"2023-07-14T18:14:22.038842+0000","last_unstale":"2023-07-14T18:15:49.964266+0000","last_undegraded":"2023-07-14T18:15:49.964266+0000","last_fullsized":"2023-07-14T18:15:49.964266+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:31:14.995258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'38","reported_seq":273,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:27.039579+0000","last_change":"2023-07-14T18:14:16.920918+0000","last_active":"2023-07-14T18:16:27.039579+0000","last_peered":"2023-07-14T18:16:27.039579+0000","last_clean":"2023-07-14T18:16:27.039579+0000","last_became_active":"2023-07-14T18:14:16.920572+0000","last_became_peered":"2023-07-14T18:14:16.920572+0000","last_unstale":"2023-07-14T18:16:27.039579+0000","last_undegraded":"2023-07-14T18:16:27.039579+0000","last_fullsized":"2023-07-14T18:16:27.039579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:02:06.484223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65752,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":207,"num_read_kb":168,"num_write":39,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.483336+0000","last_change":"2023-07-14T18:14:17.929413+0000","last_active":"2023-07-14T18:16:17.483336+0000","last_peered":"2023-07-14T18:16:17.483336+0000","last_clean":"2023-07-14T18:16:17.483336+0000","last_became_active":"2023-07-14T18:14:17.929064+0000","last_became_peered":"2023-07-14T18:14:17.929064+0000","last_unstale":"2023-07-14T18:16:17.483336+0000","last_undegraded":"2023-07-14T18:16:17.483336+0000","last_fullsized":"2023-07-14T18:16:17.483336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:56:39.264420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.965827+0000","last_change":"2023-07-14T18:14:22.041983+0000","last_active":"2023-07-14T18:15:49.965827+0000","last_peered":"2023-07-14T18:15:49.965827+0000","last_clean":"2023-07-14T18:15:49.965827+0000","last_became_active":"2023-07-14T18:14:22.041553+0000","last_became_peered":"2023-07-14T18:14:22.041553+0000","last_unstale":"2023-07-14T18:15:49.965827+0000","last_undegraded":"2023-07-14T18:15:49.965827+0000","last_fullsized":"2023-07-14T18:15:49.965827+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:02:11.442175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093843+0000","last_change":"2023-07-14T18:14:16.925444+0000","last_active":"2023-07-14T18:15:50.093843+0000","last_peered":"2023-07-14T18:15:50.093843+0000","last_clean":"2023-07-14T18:15:50.093843+0000","last_became_active":"2023-07-14T18:14:16.925138+0000","last_became_peered":"2023-07-14T18:14:16.925138+0000","last_unstale":"2023-07-14T18:15:50.093843+0000","last_undegraded":"2023-07-14T18:15:50.093843+0000","last_fullsized":"2023-07-14T18:15:50.093843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:19:37.684882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.487136+0000","last_change":"2023-07-14T18:14:17.922351+0000","last_active":"2023-07-14T18:16:17.487136+0000","last_peered":"2023-07-14T18:16:17.487136+0000","last_clean":"2023-07-14T18:16:17.487136+0000","last_became_active":"2023-07-14T18:14:17.921973+0000","last_became_peered":"2023-07-14T18:14:17.921973+0000","last_unstale":"2023-07-14T18:16:17.487136+0000","last_undegraded":"2023-07-14T18:16:17.487136+0000","last_fullsized":"2023-07-14T18:16:17.487136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:27:35.200279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964677+0000","last_change":"2023-07-14T18:14:22.048274+0000","last_active":"2023-07-14T18:15:49.964677+0000","last_peered":"2023-07-14T18:15:49.964677+0000","last_clean":"2023-07-14T18:15:49.964677+0000","last_became_active":"2023-07-14T18:14:22.047946+0000","last_became_peered":"2023-07-14T18:14:22.047946+0000","last_unstale":"2023-07-14T18:15:49.964677+0000","last_undegraded":"2023-07-14T18:15:49.964677+0000","last_fullsized":"2023-07-14T18:15:49.964677+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:43:43.819233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.965500+0000","last_change":"2023-07-14T18:14:16.933566+0000","last_active":"2023-07-14T18:15:49.965500+0000","last_peered":"2023-07-14T18:15:49.965500+0000","last_clean":"2023-07-14T18:15:49.965500+0000","last_became_active":"2023-07-14T18:14:16.933300+0000","last_became_peered":"2023-07-14T18:14:16.933300+0000","last_unstale":"2023-07-14T18:15:49.965500+0000","last_undegraded":"2023-07-14T18:15:49.965500+0000","last_fullsized":"2023-07-14T18:15:49.965500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:37:32.251937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.488953+0000","last_change":"2023-07-14T18:14:17.928521+0000","last_active":"2023-07-14T18:16:17.488953+0000","last_peered":"2023-07-14T18:16:17.488953+0000","last_clean":"2023-07-14T18:16:17.488953+0000","last_became_active":"2023-07-14T18:14:17.928025+0000","last_became_peered":"2023-07-14T18:14:17.928025+0000","last_unstale":"2023-07-14T18:16:17.488953+0000","last_undegraded":"2023-07-14T18:16:17.488953+0000","last_fullsized":"2023-07-14T18:16:17.488953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:58:12.486113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964540+0000","last_change":"2023-07-14T18:14:22.046820+0000","last_active":"2023-07-14T18:15:49.964540+0000","last_peered":"2023-07-14T18:15:49.964540+0000","last_clean":"2023-07-14T18:15:49.964540+0000","last_became_active":"2023-07-14T18:14:22.046440+0000","last_became_peered":"2023-07-14T18:14:22.046440+0000","last_unstale":"2023-07-14T18:15:49.964540+0000","last_undegraded":"2023-07-14T18:15:49.964540+0000","last_fullsized":"2023-07-14T18:15:49.964540+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:40:11.333149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":135,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:29.040866+0000","last_change":"2023-07-14T18:14:16.922075+0000","last_active":"2023-07-14T18:16:29.040866+0000","last_peered":"2023-07-14T18:16:29.040866+0000","last_clean":"2023-07-14T18:16:29.040866+0000","last_became_active":"2023-07-14T18:14:16.921570+0000","last_became_peered":"2023-07-14T18:14:16.921570+0000","last_unstale":"2023-07-14T18:16:29.040866+0000","last_undegraded":"2023-07-14T18:16:29.040866+0000","last_fullsized":"2023-07-14T18:16:29.040866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:57:51.347575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.487451+0000","last_change":"2023-07-14T18:14:17.936439+0000","last_active":"2023-07-14T18:16:17.487451+0000","last_peered":"2023-07-14T18:16:17.487451+0000","last_clean":"2023-07-14T18:16:17.487451+0000","last_became_active":"2023-07-14T18:14:17.936138+0000","last_became_peered":"2023-07-14T18:14:17.936138+0000","last_unstale":"2023-07-14T18:16:17.487451+0000","last_undegraded":"2023-07-14T18:16:17.487451+0000","last_fullsized":"2023-07-14T18:16:17.487451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:32:25.986869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958834+0000","last_change":"2023-07-14T18:14:49.543302+0000","last_active":"2023-07-14T18:15:49.958834+0000","last_peered":"2023-07-14T18:15:49.958834+0000","last_clean":"2023-07-14T18:15:49.958834+0000","last_became_active":"2023-07-14T18:14:49.542768+0000","last_became_peered":"2023-07-14T18:14:49.542768+0000","last_unstale":"2023-07-14T18:15:49.958834+0000","last_undegraded":"2023-07-14T18:15:49.958834+0000","last_fullsized":"2023-07-14T18:15:49.958834+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:05:11.701773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958699+0000","last_change":"2023-07-14T18:14:16.926417+0000","last_active":"2023-07-14T18:15:49.958699+0000","last_peered":"2023-07-14T18:15:49.958699+0000","last_clean":"2023-07-14T18:15:49.958699+0000","last_became_active":"2023-07-14T18:14:16.926101+0000","last_became_peered":"2023-07-14T18:14:16.926101+0000","last_unstale":"2023-07-14T18:15:49.958699+0000","last_undegraded":"2023-07-14T18:15:49.958699+0000","last_fullsized":"2023-07-14T18:15:49.958699+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:16:34.754140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.476341+0000","last_change":"2023-07-14T18:14:48.950649+0000","last_active":"2023-07-14T18:16:17.476341+0000","last_peered":"2023-07-14T18:16:17.476341+0000","last_clean":"2023-07-14T18:16:17.476341+0000","last_became_active":"2023-07-14T18:14:48.950303+0000","last_became_peered":"2023-07-14T18:14:48.950303+0000","last_unstale":"2023-07-14T18:16:17.476341+0000","last_undegraded":"2023-07-14T18:16:17.476341+0000","last_fullsized":"2023-07-14T18:16:17.476341+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:03:15.248073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.965148+0000","last_change":"2023-07-14T18:14:22.042171+0000","last_active":"2023-07-14T18:15:49.965148+0000","last_peered":"2023-07-14T18:15:49.965148+0000","last_clean":"2023-07-14T18:15:49.965148+0000","last_became_active":"2023-07-14T18:14:22.041692+0000","last_became_peered":"2023-07-14T18:14:22.041692+0000","last_unstale":"2023-07-14T18:15:49.965148+0000","last_undegraded":"2023-07-14T18:15:49.965148+0000","last_fullsized":"2023-07-14T18:15:49.965148+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:51:41.773405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.959284+0000","last_change":"2023-07-14T18:14:16.922883+0000","last_active":"2023-07-14T18:15:49.959284+0000","last_peered":"2023-07-14T18:15:49.959284+0000","last_clean":"2023-07-14T18:15:49.959284+0000","last_became_active":"2023-07-14T18:14:16.922501+0000","last_became_peered":"2023-07-14T18:14:16.922501+0000","last_unstale":"2023-07-14T18:15:49.959284+0000","last_undegraded":"2023-07-14T18:15:49.959284+0000","last_fullsized":"2023-07-14T18:15:49.959284+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:53:01.489131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.478108+0000","last_change":"2023-07-14T18:14:17.922899+0000","last_active":"2023-07-14T18:16:17.478108+0000","last_peered":"2023-07-14T18:16:17.478108+0000","last_clean":"2023-07-14T18:16:17.478108+0000","last_became_active":"2023-07-14T18:14:17.922608+0000","last_became_peered":"2023-07-14T18:14:17.922608+0000","last_unstale":"2023-07-14T18:16:17.478108+0000","last_undegraded":"2023-07-14T18:16:17.478108+0000","last_fullsized":"2023-07-14T18:16:17.478108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:07:33.479250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.959164+0000","last_change":"2023-07-14T18:14:22.327484+0000","last_active":"2023-07-14T18:15:49.959164+0000","last_peered":"2023-07-14T18:15:49.959164+0000","last_clean":"2023-07-14T18:15:49.959164+0000","last_became_active":"2023-07-14T18:14:22.327010+0000","last_became_peered":"2023-07-14T18:14:22.327010+0000","last_unstale":"2023-07-14T18:15:49.959164+0000","last_undegraded":"2023-07-14T18:15:49.959164+0000","last_fullsized":"2023-07-14T18:15:49.959164+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:28:55.842693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964720+0000","last_change":"2023-07-14T18:14:16.930606+0000","last_active":"2023-07-14T18:15:49.964720+0000","last_peered":"2023-07-14T18:15:49.964720+0000","last_clean":"2023-07-14T18:15:49.964720+0000","last_became_active":"2023-07-14T18:14:16.930175+0000","last_became_peered":"2023-07-14T18:14:16.930175+0000","last_unstale":"2023-07-14T18:15:49.964720+0000","last_undegraded":"2023-07-14T18:15:49.964720+0000","last_fullsized":"2023-07-14T18:15:49.964720+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:30:16.214141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.471509+0000","last_change":"2023-07-14T18:14:17.931962+0000","last_active":"2023-07-14T18:16:17.471509+0000","last_peered":"2023-07-14T18:16:17.471509+0000","last_clean":"2023-07-14T18:16:17.471509+0000","last_became_active":"2023-07-14T18:14:17.931621+0000","last_became_peered":"2023-07-14T18:14:17.931621+0000","last_unstale":"2023-07-14T18:16:17.471509+0000","last_undegraded":"2023-07-14T18:16:17.471509+0000","last_fullsized":"2023-07-14T18:16:17.471509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:53:21.866251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958462+0000","last_change":"2023-07-14T18:14:22.038977+0000","last_active":"2023-07-14T18:15:49.958462+0000","last_peered":"2023-07-14T18:15:49.958462+0000","last_clean":"2023-07-14T18:15:49.958462+0000","last_became_active":"2023-07-14T18:14:22.038515+0000","last_became_peered":"2023-07-14T18:14:22.038515+0000","last_unstale":"2023-07-14T18:15:49.958462+0000","last_undegraded":"2023-07-14T18:15:49.958462+0000","last_fullsized":"2023-07-14T18:15:49.958462+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:18:48.042709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.962465+0000","last_change":"2023-07-14T18:14:16.925457+0000","last_active":"2023-07-14T18:15:49.962465+0000","last_peered":"2023-07-14T18:15:49.962465+0000","last_clean":"2023-07-14T18:15:49.962465+0000","last_became_active":"2023-07-14T18:14:16.925150+0000","last_became_peered":"2023-07-14T18:14:16.925150+0000","last_unstale":"2023-07-14T18:15:49.962465+0000","last_undegraded":"2023-07-14T18:15:49.962465+0000","last_fullsized":"2023-07-14T18:15:49.962465+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:15:42.815134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.481369+0000","last_change":"2023-07-14T18:14:17.926492+0000","last_active":"2023-07-14T18:16:17.481369+0000","last_peered":"2023-07-14T18:16:17.481369+0000","last_clean":"2023-07-14T18:16:17.481369+0000","last_became_active":"2023-07-14T18:14:17.926218+0000","last_became_peered":"2023-07-14T18:14:17.926218+0000","last_unstale":"2023-07-14T18:16:17.481369+0000","last_undegraded":"2023-07-14T18:16:17.481369+0000","last_fullsized":"2023-07-14T18:16:17.481369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:03:08.979533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958737+0000","last_change":"2023-07-14T18:14:22.327242+0000","last_active":"2023-07-14T18:15:49.958737+0000","last_peered":"2023-07-14T18:15:49.958737+0000","last_clean":"2023-07-14T18:15:49.958737+0000","last_became_active":"2023-07-14T18:14:22.326787+0000","last_became_peered":"2023-07-14T18:14:22.326787+0000","last_unstale":"2023-07-14T18:15:49.958737+0000","last_undegraded":"2023-07-14T18:15:49.958737+0000","last_fullsized":"2023-07-14T18:15:49.958737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:20:16.313522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.469654+0000","last_change":"2023-07-14T18:14:17.925035+0000","last_active":"2023-07-14T18:16:17.469654+0000","last_peered":"2023-07-14T18:16:17.469654+0000","last_clean":"2023-07-14T18:16:17.469654+0000","last_became_active":"2023-07-14T18:14:17.924713+0000","last_became_peered":"2023-07-14T18:14:17.924713+0000","last_unstale":"2023-07-14T18:16:17.469654+0000","last_undegraded":"2023-07-14T18:16:17.469654+0000","last_fullsized":"2023-07-14T18:16:17.469654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:32:01.731363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":132,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:26.035225+0000","last_change":"2023-07-14T18:14:16.935561+0000","last_active":"2023-07-14T18:16:26.035225+0000","last_peered":"2023-07-14T18:16:26.035225+0000","last_clean":"2023-07-14T18:16:26.035225+0000","last_became_active":"2023-07-14T18:14:16.935022+0000","last_became_peered":"2023-07-14T18:14:16.935022+0000","last_unstale":"2023-07-14T18:16:26.035225+0000","last_undegraded":"2023-07-14T18:16:26.035225+0000","last_fullsized":"2023-07-14T18:16:26.035225+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:46:30.440549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.989880+0000","last_change":"2023-07-14T18:15:49.989513+0000","last_active":"2023-07-14T18:15:49.989880+0000","last_peered":"2023-07-14T18:15:49.989880+0000","last_clean":"2023-07-14T18:15:49.989880+0000","last_became_active":"2023-07-14T18:15:49.985094+0000","last_became_peered":"2023-07-14T18:15:49.985094+0000","last_unstale":"2023-07-14T18:15:49.989880+0000","last_undegraded":"2023-07-14T18:15:49.989880+0000","last_fullsized":"2023-07-14T18:15:49.989880+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:12:15.915672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.476140+0000","last_change":"2023-07-14T18:14:17.930211+0000","last_active":"2023-07-14T18:16:17.476140+0000","last_peered":"2023-07-14T18:16:17.476140+0000","last_clean":"2023-07-14T18:16:17.476140+0000","last_became_active":"2023-07-14T18:14:17.929851+0000","last_became_peered":"2023-07-14T18:14:17.929851+0000","last_unstale":"2023-07-14T18:16:17.476140+0000","last_undegraded":"2023-07-14T18:16:17.476140+0000","last_fullsized":"2023-07-14T18:16:17.476140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:03:36.180205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958812+0000","last_change":"2023-07-14T18:14:16.928995+0000","last_active":"2023-07-14T18:15:49.958812+0000","last_peered":"2023-07-14T18:15:49.958812+0000","last_clean":"2023-07-14T18:15:49.958812+0000","last_became_active":"2023-07-14T18:14:16.928612+0000","last_became_peered":"2023-07-14T18:14:16.928612+0000","last_unstale":"2023-07-14T18:15:49.958812+0000","last_undegraded":"2023-07-14T18:15:49.958812+0000","last_fullsized":"2023-07-14T18:15:49.958812+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:01:02.502166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":345,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.962275+0000","last_change":"2023-07-14T18:13:11.614507+0000","last_active":"2023-07-14T18:15:49.962275+0000","last_peered":"2023-07-14T18:15:49.962275+0000","last_clean":"2023-07-14T18:15:49.962275+0000","last_became_active":"2023-07-14T18:13:11.614063+0000","last_became_peered":"2023-07-14T18:13:11.614063+0000","last_unstale":"2023-07-14T18:15:49.962275+0000","last_undegraded":"2023-07-14T18:15:49.962275+0000","last_fullsized":"2023-07-14T18:15:49.962275+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:10:03.995193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:10:03.995193+0000","last_clean_scrub_stamp":"2023-07-14T18:10:03.995193+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:00:53.238854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.959283+0000","last_change":"2023-07-14T18:14:49.542604+0000","last_active":"2023-07-14T18:15:49.959283+0000","last_peered":"2023-07-14T18:15:49.959283+0000","last_clean":"2023-07-14T18:15:49.959283+0000","last_became_active":"2023-07-14T18:14:49.542111+0000","last_became_peered":"2023-07-14T18:14:49.542111+0000","last_unstale":"2023-07-14T18:15:49.959283+0000","last_undegraded":"2023-07-14T18:15:49.959283+0000","last_fullsized":"2023-07-14T18:15:49.959283+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:25:58.915436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.479234+0000","last_change":"2023-07-14T18:14:49.543511+0000","last_active":"2023-07-14T18:16:17.479234+0000","last_peered":"2023-07-14T18:16:17.479234+0000","last_clean":"2023-07-14T18:16:17.479234+0000","last_became_active":"2023-07-14T18:14:49.542407+0000","last_became_peered":"2023-07-14T18:14:49.542407+0000","last_unstale":"2023-07-14T18:16:17.479234+0000","last_undegraded":"2023-07-14T18:16:17.479234+0000","last_fullsized":"2023-07-14T18:16:17.479234+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:18:32.901622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964257+0000","last_change":"2023-07-14T18:14:48.950022+0000","last_active":"2023-07-14T18:15:49.964257+0000","last_peered":"2023-07-14T18:15:49.964257+0000","last_clean":"2023-07-14T18:15:49.964257+0000","last_became_active":"2023-07-14T18:14:48.949149+0000","last_became_peered":"2023-07-14T18:14:48.949149+0000","last_unstale":"2023-07-14T18:15:49.964257+0000","last_undegraded":"2023-07-14T18:15:49.964257+0000","last_fullsized":"2023-07-14T18:15:49.964257+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:10:45.323165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964919+0000","last_change":"2023-07-14T18:14:22.038428+0000","last_active":"2023-07-14T18:15:49.964919+0000","last_peered":"2023-07-14T18:15:49.964919+0000","last_clean":"2023-07-14T18:15:49.964919+0000","last_became_active":"2023-07-14T18:14:22.038038+0000","last_became_peered":"2023-07-14T18:14:22.038038+0000","last_unstale":"2023-07-14T18:15:49.964919+0000","last_undegraded":"2023-07-14T18:15:49.964919+0000","last_fullsized":"2023-07-14T18:15:49.964919+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:16:27.576741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.959318+0000","last_change":"2023-07-14T18:14:16.933110+0000","last_active":"2023-07-14T18:15:49.959318+0000","last_peered":"2023-07-14T18:15:49.959318+0000","last_clean":"2023-07-14T18:15:49.959318+0000","last_became_active":"2023-07-14T18:14:16.932844+0000","last_became_peered":"2023-07-14T18:14:16.932844+0000","last_unstale":"2023-07-14T18:15:49.959318+0000","last_undegraded":"2023-07-14T18:15:49.959318+0000","last_fullsized":"2023-07-14T18:15:49.959318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:08:17.512405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.477926+0000","last_change":"2023-07-14T18:14:17.920168+0000","last_active":"2023-07-14T18:16:17.477926+0000","last_peered":"2023-07-14T18:16:17.477926+0000","last_clean":"2023-07-14T18:16:17.477926+0000","last_became_active":"2023-07-14T18:14:17.919576+0000","last_became_peered":"2023-07-14T18:14:17.919576+0000","last_unstale":"2023-07-14T18:16:17.477926+0000","last_undegraded":"2023-07-14T18:16:17.477926+0000","last_fullsized":"2023-07-14T18:16:17.477926+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:55:49.314127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093284+0000","last_change":"2023-07-14T18:14:22.327206+0000","last_active":"2023-07-14T18:15:50.093284+0000","last_peered":"2023-07-14T18:15:50.093284+0000","last_clean":"2023-07-14T18:15:50.093284+0000","last_became_active":"2023-07-14T18:14:22.326743+0000","last_became_peered":"2023-07-14T18:14:22.326743+0000","last_unstale":"2023-07-14T18:15:50.093284+0000","last_undegraded":"2023-07-14T18:15:50.093284+0000","last_fullsized":"2023-07-14T18:15:50.093284+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:50:43.439856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'8","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:25.919981+0000","last_change":"2023-07-14T18:14:16.928994+0000","last_active":"2023-07-14T18:16:25.919981+0000","last_peered":"2023-07-14T18:16:25.919981+0000","last_clean":"2023-07-14T18:16:25.919981+0000","last_became_active":"2023-07-14T18:14:16.928468+0000","last_became_peered":"2023-07-14T18:14:16.928468+0000","last_unstale":"2023-07-14T18:16:25.919981+0000","last_undegraded":"2023-07-14T18:16:25.919981+0000","last_fullsized":"2023-07-14T18:16:25.919981+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:31:02.519665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.487174+0000","last_change":"2023-07-14T18:14:17.919147+0000","last_active":"2023-07-14T18:16:17.487174+0000","last_peered":"2023-07-14T18:16:17.487174+0000","last_clean":"2023-07-14T18:16:17.487174+0000","last_became_active":"2023-07-14T18:14:17.918811+0000","last_became_peered":"2023-07-14T18:14:17.918811+0000","last_unstale":"2023-07-14T18:16:17.487174+0000","last_undegraded":"2023-07-14T18:16:17.487174+0000","last_fullsized":"2023-07-14T18:16:17.487174+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:48:51.557600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093309+0000","last_change":"2023-07-14T18:14:22.032632+0000","last_active":"2023-07-14T18:15:50.093309+0000","last_peered":"2023-07-14T18:15:50.093309+0000","last_clean":"2023-07-14T18:15:50.093309+0000","last_became_active":"2023-07-14T18:14:22.032233+0000","last_became_peered":"2023-07-14T18:14:22.032233+0000","last_unstale":"2023-07-14T18:15:50.093309+0000","last_undegraded":"2023-07-14T18:15:50.093309+0000","last_fullsized":"2023-07-14T18:15:50.093309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:10:59.598294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.959256+0000","last_change":"2023-07-14T18:14:16.924993+0000","last_active":"2023-07-14T18:15:49.959256+0000","last_peered":"2023-07-14T18:15:49.959256+0000","last_clean":"2023-07-14T18:15:49.959256+0000","last_became_active":"2023-07-14T18:14:16.924702+0000","last_became_peered":"2023-07-14T18:14:16.924702+0000","last_unstale":"2023-07-14T18:15:49.959256+0000","last_undegraded":"2023-07-14T18:15:49.959256+0000","last_fullsized":"2023-07-14T18:15:49.959256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:46:18.671640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.483683+0000","last_change":"2023-07-14T18:14:17.943397+0000","last_active":"2023-07-14T18:16:17.483683+0000","last_peered":"2023-07-14T18:16:17.483683+0000","last_clean":"2023-07-14T18:16:17.483683+0000","last_became_active":"2023-07-14T18:14:17.942114+0000","last_became_peered":"2023-07-14T18:14:17.942114+0000","last_unstale":"2023-07-14T18:16:17.483683+0000","last_undegraded":"2023-07-14T18:16:17.483683+0000","last_fullsized":"2023-07-14T18:16:17.483683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:59:53.996642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958856+0000","last_change":"2023-07-14T18:14:22.328273+0000","last_active":"2023-07-14T18:15:49.958856+0000","last_peered":"2023-07-14T18:15:49.958856+0000","last_clean":"2023-07-14T18:15:49.958856+0000","last_became_active":"2023-07-14T18:14:22.327843+0000","last_became_peered":"2023-07-14T18:14:22.327843+0000","last_unstale":"2023-07-14T18:15:49.958856+0000","last_undegraded":"2023-07-14T18:15:49.958856+0000","last_fullsized":"2023-07-14T18:15:49.958856+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:29:12.613515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.094321+0000","last_change":"2023-07-14T18:14:49.543525+0000","last_active":"2023-07-14T18:15:50.094321+0000","last_peered":"2023-07-14T18:15:50.094321+0000","last_clean":"2023-07-14T18:15:50.094321+0000","last_became_active":"2023-07-14T18:14:49.542962+0000","last_became_peered":"2023-07-14T18:14:49.542962+0000","last_unstale":"2023-07-14T18:15:50.094321+0000","last_undegraded":"2023-07-14T18:15:50.094321+0000","last_fullsized":"2023-07-14T18:15:50.094321+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:43:37.767679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.464347+0000","last_change":"2023-07-14T18:14:17.932181+0000","last_active":"2023-07-14T18:16:17.464347+0000","last_peered":"2023-07-14T18:16:17.464347+0000","last_clean":"2023-07-14T18:16:17.464347+0000","last_became_active":"2023-07-14T18:14:17.931811+0000","last_became_peered":"2023-07-14T18:14:17.931811+0000","last_unstale":"2023-07-14T18:16:17.464347+0000","last_undegraded":"2023-07-14T18:16:17.464347+0000","last_fullsized":"2023-07-14T18:16:17.464347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:59:38.610413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964687+0000","last_change":"2023-07-14T18:14:22.047281+0000","last_active":"2023-07-14T18:15:49.964687+0000","last_peered":"2023-07-14T18:15:49.964687+0000","last_clean":"2023-07-14T18:15:49.964687+0000","last_became_active":"2023-07-14T18:14:22.045531+0000","last_became_peered":"2023-07-14T18:14:22.045531+0000","last_unstale":"2023-07-14T18:15:49.964687+0000","last_undegraded":"2023-07-14T18:15:49.964687+0000","last_fullsized":"2023-07-14T18:15:49.964687+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:53:44.716384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093515+0000","last_change":"2023-07-14T18:14:16.926904+0000","last_active":"2023-07-14T18:15:50.093515+0000","last_peered":"2023-07-14T18:15:50.093515+0000","last_clean":"2023-07-14T18:15:50.093515+0000","last_became_active":"2023-07-14T18:14:16.926579+0000","last_became_peered":"2023-07-14T18:14:16.926579+0000","last_unstale":"2023-07-14T18:15:50.093515+0000","last_undegraded":"2023-07-14T18:15:50.093515+0000","last_fullsized":"2023-07-14T18:15:50.093515+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:55:21.011226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.478290+0000","last_change":"2023-07-14T18:14:17.926005+0000","last_active":"2023-07-14T18:16:17.478290+0000","last_peered":"2023-07-14T18:16:17.478290+0000","last_clean":"2023-07-14T18:16:17.478290+0000","last_became_active":"2023-07-14T18:14:17.925266+0000","last_became_peered":"2023-07-14T18:14:17.925266+0000","last_unstale":"2023-07-14T18:16:17.478290+0000","last_undegraded":"2023-07-14T18:16:17.478290+0000","last_fullsized":"2023-07-14T18:16:17.478290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:18:03.629066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958581+0000","last_change":"2023-07-14T18:14:22.043219+0000","last_active":"2023-07-14T18:15:49.958581+0000","last_peered":"2023-07-14T18:15:49.958581+0000","last_clean":"2023-07-14T18:15:49.958581+0000","last_became_active":"2023-07-14T18:14:22.042654+0000","last_became_peered":"2023-07-14T18:14:22.042654+0000","last_unstale":"2023-07-14T18:15:49.958581+0000","last_undegraded":"2023-07-14T18:15:49.958581+0000","last_fullsized":"2023-07-14T18:15:49.958581+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:20:43.065883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093962+0000","last_change":"2023-07-14T18:14:16.923130+0000","last_active":"2023-07-14T18:15:50.093962+0000","last_peered":"2023-07-14T18:15:50.093962+0000","last_clean":"2023-07-14T18:15:50.093962+0000","last_became_active":"2023-07-14T18:14:16.922835+0000","last_became_peered":"2023-07-14T18:14:16.922835+0000","last_unstale":"2023-07-14T18:15:50.093962+0000","last_undegraded":"2023-07-14T18:15:50.093962+0000","last_fullsized":"2023-07-14T18:15:50.093962+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:29:20.116131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.478830+0000","last_change":"2023-07-14T18:14:17.931024+0000","last_active":"2023-07-14T18:16:17.478830+0000","last_peered":"2023-07-14T18:16:17.478830+0000","last_clean":"2023-07-14T18:16:17.478830+0000","last_became_active":"2023-07-14T18:14:17.929529+0000","last_became_peered":"2023-07-14T18:14:17.929529+0000","last_unstale":"2023-07-14T18:16:17.478830+0000","last_undegraded":"2023-07-14T18:16:17.478830+0000","last_fullsized":"2023-07-14T18:16:17.478830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:06:29.694951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958757+0000","last_change":"2023-07-14T18:14:22.327313+0000","last_active":"2023-07-14T18:15:49.958757+0000","last_peered":"2023-07-14T18:15:49.958757+0000","last_clean":"2023-07-14T18:15:49.958757+0000","last_became_active":"2023-07-14T18:14:22.326922+0000","last_became_peered":"2023-07-14T18:14:22.326922+0000","last_unstale":"2023-07-14T18:15:49.958757+0000","last_undegraded":"2023-07-14T18:15:49.958757+0000","last_fullsized":"2023-07-14T18:15:49.958757+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:27:15.790634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958923+0000","last_change":"2023-07-14T18:14:16.926632+0000","last_active":"2023-07-14T18:15:49.958923+0000","last_peered":"2023-07-14T18:15:49.958923+0000","last_clean":"2023-07-14T18:15:49.958923+0000","last_became_active":"2023-07-14T18:14:16.926279+0000","last_became_peered":"2023-07-14T18:14:16.926279+0000","last_unstale":"2023-07-14T18:15:49.958923+0000","last_undegraded":"2023-07-14T18:15:49.958923+0000","last_fullsized":"2023-07-14T18:15:49.958923+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:55:17.792126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.487420+0000","last_change":"2023-07-14T18:14:17.924416+0000","last_active":"2023-07-14T18:16:17.487420+0000","last_peered":"2023-07-14T18:16:17.487420+0000","last_clean":"2023-07-14T18:16:17.487420+0000","last_became_active":"2023-07-14T18:14:17.924140+0000","last_became_peered":"2023-07-14T18:14:17.924140+0000","last_unstale":"2023-07-14T18:16:17.487420+0000","last_undegraded":"2023-07-14T18:16:17.487420+0000","last_fullsized":"2023-07-14T18:16:17.487420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:43:36.553540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.094104+0000","last_change":"2023-07-14T18:14:22.037127+0000","last_active":"2023-07-14T18:15:50.094104+0000","last_peered":"2023-07-14T18:15:50.094104+0000","last_clean":"2023-07-14T18:15:50.094104+0000","last_became_active":"2023-07-14T18:14:22.036647+0000","last_became_peered":"2023-07-14T18:14:22.036647+0000","last_unstale":"2023-07-14T18:15:50.094104+0000","last_undegraded":"2023-07-14T18:15:50.094104+0000","last_fullsized":"2023-07-14T18:15:50.094104+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:51:25.981240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.094215+0000","last_change":"2023-07-14T18:14:16.921494+0000","last_active":"2023-07-14T18:15:50.094215+0000","last_peered":"2023-07-14T18:15:50.094215+0000","last_clean":"2023-07-14T18:15:50.094215+0000","last_became_active":"2023-07-14T18:14:16.919622+0000","last_became_peered":"2023-07-14T18:14:16.919622+0000","last_unstale":"2023-07-14T18:15:50.094215+0000","last_undegraded":"2023-07-14T18:15:50.094215+0000","last_fullsized":"2023-07-14T18:15:50.094215+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:09:12.990342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.486524+0000","last_change":"2023-07-14T18:14:17.922266+0000","last_active":"2023-07-14T18:16:17.486524+0000","last_peered":"2023-07-14T18:16:17.486524+0000","last_clean":"2023-07-14T18:16:17.486524+0000","last_became_active":"2023-07-14T18:14:17.921461+0000","last_became_peered":"2023-07-14T18:14:17.921461+0000","last_unstale":"2023-07-14T18:16:17.486524+0000","last_undegraded":"2023-07-14T18:16:17.486524+0000","last_fullsized":"2023-07-14T18:16:17.486524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:04:46.997019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093532+0000","last_change":"2023-07-14T18:14:22.044080+0000","last_active":"2023-07-14T18:15:50.093532+0000","last_peered":"2023-07-14T18:15:50.093532+0000","last_clean":"2023-07-14T18:15:50.093532+0000","last_became_active":"2023-07-14T18:14:22.043612+0000","last_became_peered":"2023-07-14T18:14:22.043612+0000","last_unstale":"2023-07-14T18:15:50.093532+0000","last_undegraded":"2023-07-14T18:15:50.093532+0000","last_fullsized":"2023-07-14T18:15:50.093532+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:33:05.435083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.965710+0000","last_change":"2023-07-14T18:14:16.931871+0000","last_active":"2023-07-14T18:15:49.965710+0000","last_peered":"2023-07-14T18:15:49.965710+0000","last_clean":"2023-07-14T18:15:49.965710+0000","last_became_active":"2023-07-14T18:14:16.931549+0000","last_became_peered":"2023-07-14T18:14:16.931549+0000","last_unstale":"2023-07-14T18:15:49.965710+0000","last_undegraded":"2023-07-14T18:15:49.965710+0000","last_fullsized":"2023-07-14T18:15:49.965710+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:41:43.337550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.483151+0000","last_change":"2023-07-14T18:14:17.929550+0000","last_active":"2023-07-14T18:16:17.483151+0000","last_peered":"2023-07-14T18:16:17.483151+0000","last_clean":"2023-07-14T18:16:17.483151+0000","last_became_active":"2023-07-14T18:14:17.929204+0000","last_became_peered":"2023-07-14T18:14:17.929204+0000","last_unstale":"2023-07-14T18:16:17.483151+0000","last_undegraded":"2023-07-14T18:16:17.483151+0000","last_fullsized":"2023-07-14T18:16:17.483151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:15:39.711388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964416+0000","last_change":"2023-07-14T18:14:22.034899+0000","last_active":"2023-07-14T18:15:49.964416+0000","last_peered":"2023-07-14T18:15:49.964416+0000","last_clean":"2023-07-14T18:15:49.964416+0000","last_became_active":"2023-07-14T18:14:22.034524+0000","last_became_peered":"2023-07-14T18:14:22.034524+0000","last_unstale":"2023-07-14T18:15:49.964416+0000","last_undegraded":"2023-07-14T18:15:49.964416+0000","last_fullsized":"2023-07-14T18:15:49.964416+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:33:24.357185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958833+0000","last_change":"2023-07-14T18:14:16.935285+0000","last_active":"2023-07-14T18:15:49.958833+0000","last_peered":"2023-07-14T18:15:49.958833+0000","last_clean":"2023-07-14T18:15:49.958833+0000","last_became_active":"2023-07-14T18:14:16.934888+0000","last_became_peered":"2023-07-14T18:14:16.934888+0000","last_unstale":"2023-07-14T18:15:49.958833+0000","last_undegraded":"2023-07-14T18:15:49.958833+0000","last_fullsized":"2023-07-14T18:15:49.958833+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:24:43.441076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.485394+0000","last_change":"2023-07-14T18:14:17.930982+0000","last_active":"2023-07-14T18:16:17.485394+0000","last_peered":"2023-07-14T18:16:17.485394+0000","last_clean":"2023-07-14T18:16:17.485394+0000","last_became_active":"2023-07-14T18:14:17.930661+0000","last_became_peered":"2023-07-14T18:14:17.930661+0000","last_unstale":"2023-07-14T18:16:17.485394+0000","last_undegraded":"2023-07-14T18:16:17.485394+0000","last_fullsized":"2023-07-14T18:16:17.485394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:02:02.738482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.472552+0000","last_change":"2023-07-14T18:14:17.926204+0000","last_active":"2023-07-14T18:16:17.472552+0000","last_peered":"2023-07-14T18:16:17.472552+0000","last_clean":"2023-07-14T18:16:17.472552+0000","last_became_active":"2023-07-14T18:14:17.925689+0000","last_became_peered":"2023-07-14T18:14:17.925689+0000","last_unstale":"2023-07-14T18:16:17.472552+0000","last_undegraded":"2023-07-14T18:16:17.472552+0000","last_fullsized":"2023-07-14T18:16:17.472552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:24:10.100519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.476582+0000","last_change":"2023-07-14T18:14:17.923146+0000","last_active":"2023-07-14T18:16:17.476582+0000","last_peered":"2023-07-14T18:16:17.476582+0000","last_clean":"2023-07-14T18:16:17.476582+0000","last_became_active":"2023-07-14T18:14:17.922734+0000","last_became_peered":"2023-07-14T18:14:17.922734+0000","last_unstale":"2023-07-14T18:16:17.476582+0000","last_undegraded":"2023-07-14T18:16:17.476582+0000","last_fullsized":"2023-07-14T18:16:17.476582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:36:48.282746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.483489+0000","last_change":"2023-07-14T18:14:17.927397+0000","last_active":"2023-07-14T18:16:17.483489+0000","last_peered":"2023-07-14T18:16:17.483489+0000","last_clean":"2023-07-14T18:16:17.483489+0000","last_became_active":"2023-07-14T18:14:17.927000+0000","last_became_peered":"2023-07-14T18:14:17.927000+0000","last_unstale":"2023-07-14T18:16:17.483489+0000","last_undegraded":"2023-07-14T18:16:17.483489+0000","last_fullsized":"2023-07-14T18:16:17.483489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:55:36.682437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.482209+0000","last_change":"2023-07-14T18:14:17.935266+0000","last_active":"2023-07-14T18:16:17.482209+0000","last_peered":"2023-07-14T18:16:17.482209+0000","last_clean":"2023-07-14T18:16:17.482209+0000","last_became_active":"2023-07-14T18:14:17.934944+0000","last_became_peered":"2023-07-14T18:14:17.934944+0000","last_unstale":"2023-07-14T18:16:17.482209+0000","last_undegraded":"2023-07-14T18:16:17.482209+0000","last_fullsized":"2023-07-14T18:16:17.482209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:18:53.622505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.483800+0000","last_change":"2023-07-14T18:14:49.542810+0000","last_active":"2023-07-14T18:16:17.483800+0000","last_peered":"2023-07-14T18:16:17.483800+0000","last_clean":"2023-07-14T18:16:17.483800+0000","last_became_active":"2023-07-14T18:14:49.542421+0000","last_became_peered":"2023-07-14T18:14:49.542421+0000","last_unstale":"2023-07-14T18:16:17.483800+0000","last_undegraded":"2023-07-14T18:16:17.483800+0000","last_fullsized":"2023-07-14T18:16:17.483800+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:27:08.747661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.466271+0000","last_change":"2023-07-14T18:14:17.930757+0000","last_active":"2023-07-14T18:16:17.466271+0000","last_peered":"2023-07-14T18:16:17.466271+0000","last_clean":"2023-07-14T18:16:17.466271+0000","last_became_active":"2023-07-14T18:14:17.930302+0000","last_became_peered":"2023-07-14T18:14:17.930302+0000","last_unstale":"2023-07-14T18:16:17.466271+0000","last_undegraded":"2023-07-14T18:16:17.466271+0000","last_fullsized":"2023-07-14T18:16:17.466271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:00:25.569289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.475262+0000","last_change":"2023-07-14T18:14:17.930337+0000","last_active":"2023-07-14T18:16:17.475262+0000","last_peered":"2023-07-14T18:16:17.475262+0000","last_clean":"2023-07-14T18:16:17.475262+0000","last_became_active":"2023-07-14T18:14:17.930037+0000","last_became_peered":"2023-07-14T18:14:17.930037+0000","last_unstale":"2023-07-14T18:16:17.475262+0000","last_undegraded":"2023-07-14T18:16:17.475262+0000","last_fullsized":"2023-07-14T18:16:17.475262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:50:04.589039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.478977+0000","last_change":"2023-07-14T18:14:17.912750+0000","last_active":"2023-07-14T18:16:17.478977+0000","last_peered":"2023-07-14T18:16:17.478977+0000","last_clean":"2023-07-14T18:16:17.478977+0000","last_became_active":"2023-07-14T18:14:17.912402+0000","last_became_peered":"2023-07-14T18:14:17.912402+0000","last_unstale":"2023-07-14T18:16:17.478977+0000","last_undegraded":"2023-07-14T18:16:17.478977+0000","last_fullsized":"2023-07-14T18:16:17.478977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:29:09.484261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.471891+0000","last_change":"2023-07-14T18:14:17.930004+0000","last_active":"2023-07-14T18:16:17.471891+0000","last_peered":"2023-07-14T18:16:17.471891+0000","last_clean":"2023-07-14T18:16:17.471891+0000","last_became_active":"2023-07-14T18:14:17.929719+0000","last_became_peered":"2023-07-14T18:14:17.929719+0000","last_unstale":"2023-07-14T18:16:17.471891+0000","last_undegraded":"2023-07-14T18:16:17.471891+0000","last_fullsized":"2023-07-14T18:16:17.471891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:51:52.223909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.475685+0000","last_change":"2023-07-14T18:14:17.919044+0000","last_active":"2023-07-14T18:16:17.475685+0000","last_peered":"2023-07-14T18:16:17.475685+0000","last_clean":"2023-07-14T18:16:17.475685+0000","last_became_active":"2023-07-14T18:14:17.918079+0000","last_became_peered":"2023-07-14T18:14:17.918079+0000","last_unstale":"2023-07-14T18:16:17.475685+0000","last_undegraded":"2023-07-14T18:16:17.475685+0000","last_fullsized":"2023-07-14T18:16:17.475685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:44:33.328875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.488913+0000","last_change":"2023-07-14T18:14:17.921560+0000","last_active":"2023-07-14T18:16:17.488913+0000","last_peered":"2023-07-14T18:16:17.488913+0000","last_clean":"2023-07-14T18:16:17.488913+0000","last_became_active":"2023-07-14T18:14:17.921243+0000","last_became_peered":"2023-07-14T18:14:17.921243+0000","last_unstale":"2023-07-14T18:16:17.488913+0000","last_undegraded":"2023-07-14T18:16:17.488913+0000","last_fullsized":"2023-07-14T18:16:17.488913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:09:32.651260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.487411+0000","last_change":"2023-07-14T18:14:17.927504+0000","last_active":"2023-07-14T18:16:17.487411+0000","last_peered":"2023-07-14T18:16:17.487411+0000","last_clean":"2023-07-14T18:16:17.487411+0000","last_became_active":"2023-07-14T18:14:17.927233+0000","last_became_peered":"2023-07-14T18:14:17.927233+0000","last_unstale":"2023-07-14T18:16:17.487411+0000","last_undegraded":"2023-07-14T18:16:17.487411+0000","last_fullsized":"2023-07-14T18:16:17.487411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:59:31.666727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.478342+0000","last_change":"2023-07-14T18:14:17.932317+0000","last_active":"2023-07-14T18:16:17.478342+0000","last_peered":"2023-07-14T18:16:17.478342+0000","last_clean":"2023-07-14T18:16:17.478342+0000","last_became_active":"2023-07-14T18:14:17.931973+0000","last_became_peered":"2023-07-14T18:14:17.931973+0000","last_unstale":"2023-07-14T18:16:17.478342+0000","last_undegraded":"2023-07-14T18:16:17.478342+0000","last_fullsized":"2023-07-14T18:16:17.478342+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:12:59.198777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281026,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70113,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":620,"num_read_kb":384,"num_write":169,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":210339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416423,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":511749,"data_compressed":23584,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.94199999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.53000000000000003}]},{"osd":1,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63900000000000001,"15min":0.63800000000000001},"min":{"1min":0.44500000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.41899999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77700000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.66600000000000004}]},{"osd":2,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72299999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.502,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72699999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.67600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":4,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68600000000000005,"15min":0.68000000000000005},"min":{"1min":0.48099999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89900000000000002,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.66400000000000003,"15min":0.67000000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86799999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.626}]},{"osd":5,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.61099999999999999,"15min":0.625},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68300000000000005,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.38700000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96399999999999997,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.81299999999999994}]},{"osd":6,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.749,"15min":0.75800000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.95999999999999996,"5min":1.351,"15min":1.351},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.79000000000000004,"15min":0.79400000000000004},"min":{"1min":0.48299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":3.2410000000000001,"5min":3.2410000000000001,"15min":3.2410000000000001},"last":0.77700000000000002}]},{"osd":7,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77800000000000002,"15min":0.77900000000000003},"min":{"1min":0.58099999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.026,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.74399999999999999,"15min":0.749},"min":{"1min":0.51600000000000001,"5min":0.505,"15min":0.505},"max":{"1min":0.93100000000000005,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.79800000000000004}]},{"osd":8,"last update":"Fri Jul 14 18:16:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69699999999999995,"15min":0.70599999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.78100000000000003,"5min":0.878,"15min":0.878},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.78800000000000003,"15min":0.79400000000000004},"min":{"1min":0.55400000000000005,"5min":0.504,"15min":0.504},"max":{"1min":0.95299999999999996,"5min":1,"15min":1},"last":0.89700000000000002}]},{"osd":9,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80200000000000005,"15min":0.80600000000000005},"min":{"1min":0.58099999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1279999999999999,"5min":1.635,"15min":1.635},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.84899999999999998,"15min":0.85099999999999998},"min":{"1min":0.59899999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.05,"5min":1.147,"15min":1.147},"last":0.95499999999999996}]},{"osd":10,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80500000000000005,"15min":0.80900000000000005},"min":{"1min":0.57599999999999996,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.97299999999999998,"5min":1.272,"15min":1.272},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.83999999999999997,"15min":0.84999999999999998},"min":{"1min":0.59699999999999998,"5min":0.501,"15min":0.501},"max":{"1min":0.97099999999999997,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.878}]}]},{"osd":10,"up_from":72,"seq":309237645355,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1155072,"data_stored":1168486,"data_compressed":59328,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61499999999999999,"15min":0.623},"min":{"1min":0.44400000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.76100000000000001,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.68000000000000005,"15min":0.68799999999999994},"min":{"1min":0.48499999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.83499999999999996,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.53200000000000003}]},{"osd":1,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.441,"15min":0.44400000000000001},"min":{"1min":0.32600000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61599999999999999,"5min":0.626,"15min":0.626},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47799999999999998,"15min":0.48399999999999999},"min":{"1min":0.29999999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.66900000000000004,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.47699999999999998}]},{"osd":2,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.38600000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.78400000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.42299999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77500000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.71199999999999997}]},{"osd":3,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63900000000000001,"15min":0.63500000000000001},"min":{"1min":0.439,"5min":0.38,"15min":0.38},"max":{"1min":1.4179999999999999,"5min":3.4129999999999998,"15min":3.4129999999999998},"last":0.504},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.70399999999999996,"15min":0.68100000000000005},"min":{"1min":0.45400000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4870000000000001,"5min":5.9939999999999998,"15min":5.9939999999999998},"last":0.58299999999999996}]},{"osd":4,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55200000000000005,"15min":0.56399999999999995},"min":{"1min":0.40500000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68400000000000005,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.68799999999999994,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.62}]},{"osd":5,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.66700000000000004,"15min":0.65800000000000003},"min":{"1min":0.46700000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81000000000000005,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.746},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.47299999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.55400000000000005}]},{"osd":6,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70999999999999996,"15min":0.72399999999999998},"min":{"1min":0.48399999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.747,"15min":0.76500000000000001},"min":{"1min":0.504,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.69299999999999995}]},{"osd":7,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.42199999999999999}]},{"osd":8,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.73599999999999999,"15min":0.745},"min":{"1min":0.54400000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.84899999999999998,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.75,"15min":0.75800000000000001},"min":{"1min":0.57999999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0149999999999999,"5min":1.026,"15min":1.026},"last":0.79500000000000004}]},{"osd":9,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.81100000000000005,"15min":0.81599999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.3879999999999999,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.80800000000000005,"15min":0.80600000000000005},"min":{"1min":0.60499999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":0.82499999999999996}]},{"osd":11,"last update":"Fri Jul 14 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.752,"15min":0.751},"min":{"1min":0.61499999999999999,"5min":0.499,"15min":0.499},"max":{"1min":0.89300000000000002,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.76500000000000001,"15min":0.76200000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.96699999999999997,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.80800000000000005}]}]},{"osd":9,"up_from":67,"seq":287762808880,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":511985,"data_compressed":23584,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.51100000000000001,"15min":0.53600000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.58699999999999997,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54800000000000004,"15min":0.56799999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Jul 14 18:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64800000000000002,"15min":0.65800000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.90600000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.69899999999999995,"15min":0.72799999999999998},"min":{"1min":0.45000000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88200000000000001,"5min":1.085,"15min":1.085},"last":0.70099999999999996}]},{"osd":2,"last update":"Fri Jul 14 18:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.68700000000000006,"15min":0.69399999999999995},"min":{"1min":0.44800000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91800000000000004,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.73199999999999998,"15min":0.754},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93799999999999994,"5min":1.147,"15min":1.147},"last":0.57399999999999995}]},{"osd":3,"last update":"Fri Jul 14 18:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.63700000000000001,"15min":0.66900000000000004},"min":{"1min":0.40100000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.63500000000000001,"15min":0.66300000000000003},"min":{"1min":0.39400000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.45500000000000002}]},{"osd":4,"last update":"Fri Jul 14 18:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.745,"15min":0.78100000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.878,"5min":1.198,"15min":1.198},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.751,"15min":0.78100000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.99199999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.71599999999999997}]},{"osd":5,"last update":"Fri Jul 14 18:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.77700000000000002,"15min":0.80600000000000005},"min":{"1min":0.53600000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.97199999999999998,"5min":1.224,"15min":1.224},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.73299999999999998,"15min":0.752},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95399999999999996,"5min":1.113,"15min":1.113},"last":0.64400000000000002}]},{"osd":6,"last update":"Fri Jul 14 18:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.75900000000000001}]},{"osd":7,"last update":"Fri Jul 14 18:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.82299999999999995,"15min":0.871},"min":{"1min":0.55800000000000005,"5min":0.503,"15min":0.503},"max":{"1min":0.92300000000000004,"5min":1.323,"15min":1.323},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.80800000000000005,"15min":0.85499999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.88100000000000001,"5min":1.411,"15min":1.411},"last":0.77700000000000002}]},{"osd":8,"last update":"Fri Jul 14 18:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.86599999999999999,"15min":0.91000000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.98799999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.872,"15min":0.91100000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.01,"5min":1.371,"15min":1.371},"last":0.88900000000000001}]},{"osd":10,"last update":"Fri Jul 14 18:16:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.83299999999999996,"15min":0.86899999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0409999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.84199999999999997,"15min":0.86599999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0489999999999999,"5min":1.095,"15min":1.095},"last":0.79300000000000004}]},{"osd":11,"last update":"Fri Jul 14 18:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.80600000000000005,"15min":0.80200000000000005},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.79600000000000004,"15min":0.80000000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.97899999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.83899999999999997}]}]},{"osd":8,"up_from":59,"seq":253403070517,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":511323,"data_compressed":23584,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77000000000000002,"5min":1.238,"15min":1.238},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.622,"15min":0.57999999999999996},"min":{"1min":0.46700000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96199999999999997,"5min":2.5950000000000002,"15min":2.5950000000000002},"last":0.62}]},{"osd":1,"last update":"Fri Jul 14 18:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.60499999999999998,"15min":0.57299999999999995},"min":{"1min":0.47999999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.63,"15min":0.59599999999999997},"min":{"1min":0.56100000000000005,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89800000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.77900000000000003}]},{"osd":2,"last update":"Fri Jul 14 18:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.48299999999999998,"15min":0.47499999999999998},"min":{"1min":0.42499999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.72699999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.49199999999999999,"15min":0.47199999999999998},"min":{"1min":0.42899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.79900000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.75600000000000001}]},{"osd":3,"last update":"Fri Jul 14 18:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.74399999999999999,"15min":0.66200000000000003},"min":{"1min":0.53600000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":3.8090000000000002,"5min":5.9749999999999996,"15min":5.9749999999999996},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.71099999999999997,"15min":0.63100000000000001},"min":{"1min":0.44,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":5.0810000000000004,"5min":5.0810000000000004,"15min":5.0810000000000004},"last":0.67100000000000004}]},{"osd":4,"last update":"Fri Jul 14 18:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.65400000000000003,"15min":0.63200000000000001},"min":{"1min":0.439,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.68700000000000006,"15min":0.66400000000000003},"min":{"1min":0.61699999999999999,"5min":0.434,"15min":0.434},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.92400000000000004}]},{"osd":5,"last update":"Fri Jul 14 18:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.59899999999999998,"15min":0.57099999999999995},"min":{"1min":0.40000000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.86799999999999999,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.57999999999999996,"15min":0.55400000000000005},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.88200000000000001}]},{"osd":6,"last update":"Fri Jul 14 18:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.75800000000000001,"15min":0.73399999999999999},"min":{"1min":0.51000000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.90400000000000003,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.82199999999999995,"15min":0.74399999999999999},"min":{"1min":0.65600000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":3.6339999999999999,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":0.90800000000000003}]},{"osd":7,"last update":"Fri Jul 14 18:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.75900000000000001,"15min":0.73699999999999999},"min":{"1min":0.73999999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.73799999999999999,"15min":0.71099999999999997},"min":{"1min":0.59799999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":1.042}]},{"osd":9,"last update":"Fri Jul 14 18:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":0.88900000000000001,"15min":0.82099999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":5.9370000000000003,"5min":5.9370000000000003,"15min":5.9370000000000003},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.2529999999999999,"5min":0.88,"15min":0.82399999999999995},"min":{"1min":0.57299999999999995,"5min":0.502,"15min":0.502},"max":{"1min":5.9039999999999999,"5min":5.9039999999999999,"15min":5.9039999999999999},"last":1.016}]},{"osd":10,"last update":"Fri Jul 14 18:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.79100000000000004,"15min":0.77000000000000002},"min":{"1min":0.57199999999999995,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":1.03},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.79400000000000004,"15min":0.77900000000000003},"min":{"1min":0.58399999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":1.0649999999999999}]},{"osd":11,"last update":"Fri Jul 14 18:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.79600000000000004,"15min":0.79100000000000004},"min":{"1min":0.57199999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.81899999999999995,"15min":0.81200000000000006},"min":{"1min":0.60399999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.97699999999999998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.995}]}]},{"osd":7,"up_from":51,"seq":219043332152,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":1108174,"data_compressed":59328,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.755,"15min":0.77300000000000002},"min":{"1min":0.38200000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.024,"5min":5.843,"15min":5.843},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.51100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.81299999999999994,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.40899999999999997}]},{"osd":1,"last update":"Fri Jul 14 18:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.42999999999999999,"15min":0.42799999999999999},"min":{"1min":0.32500000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.62,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46500000000000002,"15min":0.46200000000000002},"min":{"1min":0.35599999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.62,"5min":0.752,"15min":0.752},"last":0.51500000000000001}]},{"osd":2,"last update":"Fri Jul 14 18:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.64100000000000001,"15min":0.60399999999999998},"min":{"1min":0.42999999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.92800000000000005,"5min":3.4889999999999999,"15min":3.4889999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.45200000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.84999999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.746}]},{"osd":3,"last update":"Fri Jul 14 18:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.504,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.95099999999999996,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.434,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.78800000000000003,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.57199999999999995}]},{"osd":4,"last update":"Fri Jul 14 18:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.51700000000000002,"15min":0.52900000000000003},"min":{"1min":0.379,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72199999999999998,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.5,"15min":0.498},"min":{"1min":0.379,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.58699999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.54300000000000004}]},{"osd":5,"last update":"Fri Jul 14 18:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.63700000000000001,"15min":0.62},"min":{"1min":0.52300000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.873,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68899999999999995,"15min":0.68600000000000005},"min":{"1min":0.55900000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86199999999999999,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.78500000000000003}]},{"osd":6,"last update":"Fri Jul 14 18:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.92700000000000005,"15min":1.1140000000000001},"min":{"1min":0.47899999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.129,"5min":13.189,"15min":13.189},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.48499999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.92400000000000004,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.68400000000000005}]},{"osd":8,"last update":"Fri Jul 14 18:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.71599999999999997,"15min":0.69799999999999995},"min":{"1min":0.59899999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86899999999999999,"5min":1.04,"15min":1.04},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69799999999999995,"15min":0.69199999999999995},"min":{"1min":0.54700000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85799999999999998,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.72699999999999998}]},{"osd":9,"last update":"Fri Jul 14 18:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.89700000000000002,"15min":0.95899999999999996},"min":{"1min":0.56100000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.96999999999999997,"5min":5.8250000000000002,"15min":5.8250000000000002},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77400000000000002,"15min":0.77100000000000002},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.86799999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.80900000000000005}]},{"osd":10,"last update":"Fri Jul 14 18:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71799999999999997,"15min":0.71699999999999997},"min":{"1min":0.48599999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1,"5min":1,"15min":1},"last":0.629},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76900000000000002,"15min":0.77000000000000002},"min":{"1min":0.59499999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.98499999999999999,"5min":0.995,"15min":0.995},"last":0.85799999999999998}]},{"osd":11,"last update":"Fri Jul 14 18:16:19 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.85499999999999998,"15min":0.85399999999999998},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.875,"15min":0.876},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.88100000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":507161,"data_compressed":23584,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46800000000000003,"15min":0.45600000000000002},"min":{"1min":0.316,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.51600000000000001,"15min":0.48999999999999999},"min":{"1min":0.41499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.39900000000000002}]},{"osd":1,"last update":"Fri Jul 14 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.66600000000000004,"15min":0.64600000000000002},"min":{"1min":0.46700000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.69499999999999995,"15min":0.68000000000000005},"min":{"1min":0.48199999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.76000000000000001}]},{"osd":2,"last update":"Fri Jul 14 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.65500000000000003,"15min":0.63200000000000001},"min":{"1min":0.50900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93799999999999994,"5min":1.087,"15min":1.087},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69399999999999995,"15min":0.68100000000000005},"min":{"1min":0.45500000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.629}]},{"osd":3,"last update":"Fri Jul 14 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.63700000000000001,"15min":0.61699999999999999},"min":{"1min":0.41999999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.504},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.65100000000000002,"15min":0.65200000000000002},"min":{"1min":0.433,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.64900000000000002}]},{"osd":4,"last update":"Fri Jul 14 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.66300000000000003,"15min":0.64900000000000002},"min":{"1min":0.52700000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.72999999999999998,"15min":0.72499999999999998},"min":{"1min":0.57299999999999995,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.79600000000000004}]},{"osd":5,"last update":"Fri Jul 14 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.76200000000000001,"15min":0.76500000000000001},"min":{"1min":0.54700000000000004,"5min":0.435,"15min":0.435},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.75900000000000001,"15min":0.73099999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":4.1449999999999996,"5min":4.1449999999999996,"15min":4.1449999999999996},"last":0.85299999999999998}]},{"osd":7,"last update":"Fri Jul 14 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.78500000000000003,"15min":0.74299999999999999},"min":{"1min":0.58899999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":4.2750000000000004,"5min":4.2750000000000004,"15min":4.2750000000000004},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.747,"15min":0.72699999999999998},"min":{"1min":0.51300000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0860000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.93100000000000005}]},{"osd":8,"last update":"Fri Jul 14 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.79300000000000004,"15min":0.80400000000000005},"min":{"1min":0.54800000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.77300000000000002,"15min":0.75900000000000001},"min":{"1min":0.623,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0980000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.69199999999999995}]},{"osd":9,"last update":"Fri Jul 14 18:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.79300000000000004,"15min":0.81499999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.996,"5min":1.147,"15min":1.147},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83099999999999996,"15min":0.84499999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.133,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.748}]},{"osd":10,"last update":"Fri Jul 14 18:16:13 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86499999999999999,"15min":0.86899999999999999},"min":{"1min":0.67900000000000005,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.028,"5min":0.89000000000000001,"15min":0.86599999999999999},"min":{"1min":0.64200000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":4.1909999999999998,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.97099999999999997}]},{"osd":11,"last update":"Fri Jul 14 18:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.90600000000000003,"15min":0.91200000000000003},"min":{"1min":0.72499999999999998,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.1180000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.84899999999999998,"15min":0.84799999999999998},"min":{"1min":0.73799999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.77300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":573405,"data_compressed":23584,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56999999999999995,"15min":0.52000000000000002},"min":{"1min":0.34100000000000003,"5min":0.30599999999999999,"15min":0.28199999999999997},"max":{"1min":1.3129999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":1.151,"5min":0.70399999999999996,"15min":0.57399999999999995},"min":{"1min":0.44800000000000001,"5min":0.32100000000000001,"15min":0.30299999999999999},"max":{"1min":7.226,"5min":7.226,"15min":7.226},"last":0.36199999999999999}]},{"osd":2,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.626,"15min":0.55400000000000005},"min":{"1min":0.47899999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59999999999999998,"15min":0.53500000000000003},"min":{"1min":0.48999999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84199999999999997,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.752}]},{"osd":3,"last update":"Fri Jul 14 18:15:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.59299999999999997,"15min":0.53300000000000003},"min":{"1min":0.42799999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.125,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.65600000000000003,"15min":0.59999999999999998},"min":{"1min":0.501,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.5009999999999999,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":0.52900000000000003}]},{"osd":4,"last update":"Fri Jul 14 18:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.51000000000000001,"15min":0.46400000000000002},"min":{"1min":0.34599999999999997,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.91100000000000003,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.503,"15min":0.45900000000000002},"min":{"1min":0.36699999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.42699999999999999}]},{"osd":5,"last update":"Fri Jul 14 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.65700000000000003,"15min":0.61399999999999999},"min":{"1min":0.51600000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.65700000000000003,"15min":0.61599999999999999},"min":{"1min":0.61599999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.66300000000000003}]},{"osd":6,"last update":"Fri Jul 14 18:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.68600000000000005,"15min":0.67400000000000004},"min":{"1min":0.443,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.92500000000000004,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68000000000000005,"15min":0.66900000000000004},"min":{"1min":0.51800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.91400000000000003,"5min":1.23,"15min":1.23},"last":0.64700000000000002}]},{"osd":7,"last update":"Fri Jul 14 18:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.59999999999999998,"15min":0.56299999999999994},"min":{"1min":0.438,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.98299999999999998,"5min":1.04,"15min":1.04},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.628,"15min":0.58999999999999997},"min":{"1min":0.45900000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.93799999999999994,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.58199999999999996}]},{"osd":8,"last update":"Fri Jul 14 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.69099999999999995,"15min":0.65100000000000002},"min":{"1min":0.53900000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.008,"5min":1.071,"15min":1.071},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.75,"15min":0.69799999999999995},"min":{"1min":0.68100000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.03,"5min":1.052,"15min":1.052},"last":0.73399999999999999}]},{"osd":9,"last update":"Fri Jul 14 18:15:56 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.81899999999999995,"15min":0.78000000000000003},"min":{"1min":0.63300000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":4.0750000000000002,"5min":4.0750000000000002,"15min":4.0750000000000002},"last":0.69799999999999995},{"interface":"front","average":{"1min":1.054,"5min":0.874,"15min":0.83899999999999997},"min":{"1min":0.69299999999999995,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":4.0369999999999999,"5min":4.0369999999999999,"15min":4.0369999999999999},"last":0.68100000000000005}]},{"osd":10,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.81100000000000005,"15min":0.80500000000000005},"min":{"1min":0.57699999999999996,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.052,"5min":1.167,"15min":1.167},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76800000000000002,"15min":0.75700000000000001},"min":{"1min":0.55900000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0389999999999999,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.71899999999999997}]},{"osd":11,"last update":"Fri Jul 14 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.92000000000000004,"15min":0.92800000000000005},"min":{"1min":0.74099999999999999,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.097,"5min":1.169,"15min":1.169},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.86599999999999999,"15min":0.86799999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.0820000000000001,"5min":1.117,"15min":1.117},"last":0.78500000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":508549,"data_compressed":23584,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 18:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60799999999999998,"15min":0.55500000000000005},"min":{"1min":0.39400000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89800000000000002,"5min":1.698,"15min":1.698},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.61399999999999999,"15min":0.58199999999999996},"min":{"1min":0.40899999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.89600000000000002,"5min":1.179,"15min":1.179},"last":0.63800000000000001}]},{"osd":2,"last update":"Fri Jul 14 18:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.65600000000000003,"15min":0.57499999999999996},"min":{"1min":0.53400000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86799999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68200000000000005,"15min":0.59599999999999997},"min":{"1min":0.499,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89800000000000002,"5min":1.258,"15min":1.258},"last":0.89900000000000002}]},{"osd":3,"last update":"Fri Jul 14 18:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.55900000000000005,"15min":0.56699999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77400000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.64000000000000001,"15min":0.64600000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.85199999999999998}]},{"osd":4,"last update":"Fri Jul 14 18:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.61399999999999999,"15min":0.56899999999999995},"min":{"1min":0.441,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.80600000000000005,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66900000000000004,"15min":0.627},"min":{"1min":0.44,"5min":0.379,"15min":0.379},"max":{"1min":0.83699999999999997,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.58299999999999996}]},{"osd":5,"last update":"Fri Jul 14 18:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.72999999999999998,"15min":0.68899999999999995},"min":{"1min":0.45800000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90100000000000002,"5min":3.3740000000000001,"15min":3.3740000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.71499999999999997,"15min":0.68700000000000006},"min":{"1min":0.48099999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.97899999999999998,"5min":1.228,"15min":1.228},"last":0.97799999999999998}]},{"osd":6,"last update":"Fri Jul 14 18:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.64000000000000001,"15min":0.61599999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93600000000000005,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.65700000000000003,"15min":0.63700000000000001},"min":{"1min":0.41799999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88600000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.751}]},{"osd":7,"last update":"Fri Jul 14 18:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.70199999999999996,"15min":0.70999999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.82299999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.97899999999999998,"5min":1.006,"15min":1.006},"last":0.70999999999999996}]},{"osd":8,"last update":"Fri Jul 14 18:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.80900000000000005,"15min":0.79600000000000004},"min":{"1min":0.60599999999999998,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.028,"5min":1.839,"15min":1.839},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.79700000000000004,"15min":0.77000000000000002},"min":{"1min":0.57599999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0029999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.94499999999999995}]},{"osd":9,"last update":"Fri Jul 14 18:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.79600000000000004,"5min":1.0669999999999999,"15min":1.1850000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1259999999999999,"5min":7.8689999999999998,"15min":7.8689999999999998},"last":0.91000000000000003}]},{"osd":10,"last update":"Fri Jul 14 18:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84999999999999998,"15min":0.85899999999999999},"min":{"1min":0.62,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82199999999999995,"15min":0.82599999999999996},"min":{"1min":0.69199999999999995,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0569999999999999,"5min":1.081,"15min":1.081},"last":0.872}]},{"osd":11,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.90300000000000002,"15min":0.90700000000000003},"min":{"1min":0.65300000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.093,"5min":1.869,"15min":1.869},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.92900000000000005,"15min":0.93500000000000005},"min":{"1min":0.74399999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.0629999999999999,"5min":1.758,"15min":1.758},"last":1.044}]}]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":573551,"data_compressed":23584,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.629,"15min":0.55700000000000005},"min":{"1min":0.40200000000000002,"5min":0.313,"15min":0.28899999999999998},"max":{"1min":6.5220000000000002,"5min":6.5220000000000002,"15min":6.5220000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.67400000000000004,"15min":0.56399999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":8.984,"5min":8.984,"15min":8.984},"last":0.61399999999999999}]},{"osd":1,"last update":"Fri Jul 14 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.627,"15min":0.53700000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.33400000000000002},"max":{"1min":0.94599999999999995,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.61599999999999999,"15min":0.53400000000000003},"min":{"1min":0.441,"5min":0.36899999999999999,"15min":0.32000000000000001},"max":{"1min":0.89600000000000002,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.66600000000000004}]},{"osd":3,"last update":"Fri Jul 14 18:15:40 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":0.71899999999999997,"15min":0.60999999999999999},"min":{"1min":0.47599999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":9.1110000000000007,"5min":9.1110000000000007,"15min":9.1110000000000007},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.64900000000000002,"15min":0.65700000000000003},"min":{"1min":0.48199999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88100000000000001,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":0.65000000000000002}]},{"osd":4,"last update":"Fri Jul 14 18:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.72899999999999998,"15min":0.74199999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84699999999999998,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.73899999999999999,"15min":0.751},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.89900000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68999999999999995}]},{"osd":5,"last update":"Fri Jul 14 18:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.47799999999999998,"15min":0.46000000000000002},"min":{"1min":0.33200000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.68100000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52300000000000002,"15min":0.49099999999999999},"min":{"1min":0.33100000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.752,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.49099999999999999}]},{"osd":6,"last update":"Fri Jul 14 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.71199999999999997,"15min":0.65600000000000003},"min":{"1min":0.52600000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":6.5389999999999997,"5min":6.5389999999999997,"15min":6.5389999999999997},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67100000000000004,"15min":0.63600000000000001},"min":{"1min":0.46899999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.94599999999999995,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.69799999999999995}]},{"osd":7,"last update":"Fri Jul 14 18:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.79200000000000004,"15min":0.79400000000000004},"min":{"1min":0.59999999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.97299999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.79300000000000004,"15min":0.80400000000000005},"min":{"1min":0.53500000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94599999999999995,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.76200000000000001}]},{"osd":8,"last update":"Fri Jul 14 18:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.65400000000000003,"15min":0.64600000000000002},"min":{"1min":0.50900000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.64800000000000002,"15min":0.63800000000000001},"min":{"1min":0.40500000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.94499999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63300000000000001}]},{"osd":9,"last update":"Fri Jul 14 18:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.78700000000000003,"15min":0.78600000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0149999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.82299999999999995,"15min":0.81399999999999995},"min":{"1min":0.65800000000000003,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.73699999999999999}]},{"osd":10,"last update":"Fri Jul 14 18:16:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.79000000000000004,"15min":0.76200000000000001},"min":{"1min":0.63800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0529999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.79200000000000004,"15min":0.77300000000000002},"min":{"1min":0.54100000000000004,"5min":0.438,"15min":0.438},"max":{"1min":1.0109999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.71199999999999997}]},{"osd":11,"last update":"Fri Jul 14 18:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.82999999999999996,"15min":0.83899999999999997},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.99399999999999999,"5min":1.04,"15min":1.04},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.85799999999999998,"15min":0.86399999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0229999999999999,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.751}]}]},{"osd":3,"up_from":26,"seq":111669149771,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":507355,"data_compressed":23584,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.44900000000000001,"15min":0.39700000000000002},"min":{"1min":0.34300000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.74099999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.47099999999999997,"15min":0.42099999999999999},"min":{"1min":0.311,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.96199999999999997,"5min":0.998,"15min":0.998},"last":0.42099999999999999}]},{"osd":1,"last update":"Fri Jul 14 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.61899999999999999,"15min":0.58299999999999996},"min":{"1min":0.48799999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.64100000000000001,"15min":0.59499999999999997},"min":{"1min":0.51400000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.874,"5min":1.29,"15min":1.29},"last":0.53800000000000003}]},{"osd":2,"last update":"Fri Jul 14 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.64900000000000002,"15min":0.59099999999999997},"min":{"1min":0.55200000000000005,"5min":0.44,"15min":0.44},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.64000000000000001,"15min":0.60699999999999998},"min":{"1min":0.46999999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.81299999999999994}]},{"osd":4,"last update":"Fri Jul 14 18:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.65400000000000003,"15min":0.59799999999999998},"min":{"1min":0.48699999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.123,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.68000000000000005,"15min":0.64000000000000001},"min":{"1min":0.50800000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94999999999999996,"5min":1.153,"15min":1.153},"last":0.68500000000000005}]},{"osd":5,"last update":"Fri Jul 14 18:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.67100000000000004,"15min":0.63800000000000001},"min":{"1min":0.46300000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.86599999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.753},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68200000000000005,"15min":0.627},"min":{"1min":0.53900000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.877,"5min":1.149,"15min":1.149},"last":0.79000000000000004}]},{"osd":6,"last update":"Fri Jul 14 18:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.61299999999999999,"15min":0.59699999999999998},"min":{"1min":0.41799999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85099999999999998,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.66000000000000003,"15min":0.624},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0649999999999999,"5min":1.504,"15min":1.504},"last":0.83399999999999996}]},{"osd":7,"last update":"Fri Jul 14 18:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72299999999999998,"15min":0.69299999999999995},"min":{"1min":0.51600000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.105,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.75,"15min":0.71199999999999997},"min":{"1min":0.60299999999999998,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.96199999999999997,"5min":1.407,"15min":1.407},"last":0.71999999999999997}]},{"osd":8,"last update":"Fri Jul 14 18:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.72799999999999998}]},{"osd":9,"last update":"Fri Jul 14 18:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.79200000000000004,"15min":0.81299999999999994},"min":{"1min":0.51700000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.2450000000000001,"5min":1.619,"15min":1.619},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.86299999999999999,"15min":0.90100000000000002},"min":{"1min":0.57799999999999996,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2689999999999999,"5min":1.573,"15min":1.573},"last":0.80200000000000005}]},{"osd":10,"last update":"Fri Jul 14 18:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.85899999999999999,"15min":0.88100000000000001},"min":{"1min":0.65700000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.94399999999999995,"5min":1.542,"15min":1.542},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.82399999999999995,"15min":0.84999999999999998},"min":{"1min":0.61899999999999999,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.089,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.77900000000000003}]},{"osd":11,"last update":"Fri Jul 14 18:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.84999999999999998,"15min":0.85699999999999998},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":0.93400000000000005,"5min":1.472,"15min":1.472},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.79600000000000004,"15min":0.80700000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.92100000000000004,"5min":1.206,"15min":1.206},"last":0.82599999999999996}]}]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":513184,"data_compressed":23584,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.48999999999999999,"15min":0.46000000000000002},"min":{"1min":0.39700000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.83799999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.50800000000000001,"15min":0.47799999999999998},"min":{"1min":0.46700000000000003,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.747,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.39600000000000002}]},{"osd":1,"last update":"Fri Jul 14 18:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.45000000000000001,"15min":0.38200000000000001},"min":{"1min":0.26400000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.55000000000000004,"5min":3.7890000000000001,"15min":3.7890000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.435,"15min":0.39300000000000002},"min":{"1min":0.23899999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.57799999999999996,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.439}]},{"osd":2,"last update":"Fri Jul 14 18:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60399999999999998,"15min":0.55800000000000005},"min":{"1min":0.34699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.875,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.65000000000000002,"15min":0.56599999999999995},"min":{"1min":0.47099999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.80500000000000005,"5min":3.6989999999999998,"15min":3.6989999999999998},"last":0.57899999999999996}]},{"osd":3,"last update":"Fri Jul 14 18:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.63300000000000001,"15min":0.54600000000000004},"min":{"1min":0.34100000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":3.5190000000000001,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":0.497},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.623,"15min":0.54900000000000004},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":3.593,"5min":3.593,"15min":3.593},"last":0.46600000000000003}]},{"osd":5,"last update":"Fri Jul 14 18:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.67600000000000005,"15min":0.58799999999999997},"min":{"1min":0.41299999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83999999999999997,"5min":3.7490000000000001,"15min":3.7490000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65900000000000003,"15min":0.58599999999999997},"min":{"1min":0.434,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.68700000000000006}]},{"osd":6,"last update":"Fri Jul 14 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.747,"15min":0.68300000000000005},"min":{"1min":0.46700000000000003,"5min":0.441,"15min":0.441},"max":{"1min":4.117,"5min":4.117,"15min":4.117},"last":0.59699999999999998},{"interface":"front","average":{"1min":1.054,"5min":0.76200000000000001,"15min":0.69199999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":4.1440000000000001,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":0.65600000000000003}]},{"osd":7,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.66900000000000004,"15min":0.59699999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.872,"5min":3.8999999999999999,"15min":3.8999999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67400000000000004,"15min":0.60999999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.95799999999999996,"5min":3.2250000000000001,"15min":3.2250000000000001},"last":0.627}]},{"osd":8,"last update":"Fri Jul 14 18:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.68999999999999995,"15min":0.65400000000000003},"min":{"1min":0.51000000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.874,"5min":2.359,"15min":2.359},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.80400000000000005,"15min":0.76600000000000001},"min":{"1min":0.52000000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.0149999999999999,"5min":3.1739999999999999,"15min":3.1739999999999999},"last":0.76700000000000002}]},{"osd":9,"last update":"Fri Jul 14 18:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.84399999999999997,"15min":0.80800000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":4.3310000000000004,"5min":4.3310000000000004,"15min":4.3310000000000004},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.876,"5min":0.76700000000000002,"15min":0.71899999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":3.766,"5min":3.766,"15min":3.766},"last":0.61299999999999999}]},{"osd":10,"last update":"Fri Jul 14 18:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.80800000000000005,"15min":0.79000000000000004},"min":{"1min":0.39300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91300000000000003,"5min":3.8239999999999998,"15min":3.8239999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.74099999999999999,"15min":0.73599999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91100000000000003,"5min":2.52,"15min":2.52},"last":0.64200000000000002}]},{"osd":11,"last update":"Fri Jul 14 18:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.96099999999999997,"15min":0.99099999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.002,"5min":3.548,"15min":3.548},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.93200000000000005,"15min":0.94799999999999995},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.0269999999999999,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":0.81899999999999995}]}]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":1103479,"data_compressed":59328,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.36799999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68100000000000005,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.55400000000000005,"15min":0.57399999999999995},"min":{"1min":0.379,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.621,"5min":1.645,"15min":1.645},"last":0.52900000000000003}]},{"osd":1,"last update":"Fri Jul 14 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.61299999999999999,"15min":0.56599999999999995},"min":{"1min":0.40999999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.874,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60299999999999998,"15min":0.57699999999999996},"min":{"1min":0.378,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80700000000000005,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.621}]},{"osd":2,"last update":"Fri Jul 14 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.43099999999999999,"15min":0.42699999999999999},"min":{"1min":0.33200000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46000000000000002,"15min":0.48699999999999999},"min":{"1min":0.28699999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.71099999999999997,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.71099999999999997}]},{"osd":3,"last update":"Fri Jul 14 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58199999999999996,"15min":0.61099999999999999},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77300000000000002,"5min":1.591,"15min":1.591},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.46700000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.84699999999999998}]},{"osd":4,"last update":"Fri Jul 14 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.626,"15min":0.62},"min":{"1min":0.47499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82799999999999996,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.63600000000000001,"15min":0.624},"min":{"1min":0.35999999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91200000000000003,"5min":1.504,"15min":1.504},"last":0.68999999999999995}]},{"osd":6,"last update":"Fri Jul 14 18:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64400000000000002,"15min":0.65700000000000003},"min":{"1min":0.45100000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.82699999999999996,"5min":1.627,"15min":1.627},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65900000000000003,"15min":0.63900000000000001},"min":{"1min":0.45800000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.86499999999999999,"5min":1.036,"15min":1.036},"last":0.89600000000000002}]},{"osd":7,"last update":"Fri Jul 14 18:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68400000000000005,"15min":0.68500000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0089999999999999,"5min":1.554,"15min":1.554},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.68799999999999994,"15min":0.66100000000000003},"min":{"1min":0.54000000000000004,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92500000000000004,"5min":1.01,"15min":1.01},"last":0.93899999999999995}]},{"osd":8,"last update":"Fri Jul 14 18:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59399999999999997,"15min":0.57999999999999996},"min":{"1min":0.36599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80600000000000005,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59499999999999997,"15min":0.58299999999999996},"min":{"1min":0.41199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.86299999999999999}]},{"osd":9,"last update":"Fri Jul 14 18:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.57099999999999995,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.88400000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.02,"5min":0.81499999999999995,"15min":0.78500000000000003},"min":{"1min":0.58699999999999997,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":6.0460000000000003,"5min":6.0460000000000003,"15min":6.0460000000000003},"last":0.97399999999999998}]},{"osd":10,"last update":"Fri Jul 14 18:16:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.746,"15min":0.72999999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.77200000000000002,"15min":0.755},"min":{"1min":0.56899999999999995,"5min":0.505,"15min":0.505},"max":{"1min":0.96799999999999997,"5min":1.107,"15min":1.107},"last":1.008}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34102,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34102,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34102,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":66480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T18:16:31.830 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-07-14T18:16:31.830 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T18:16:32.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.015+0000 7f84c41ff700 1 -- 172.21.15.133:0/1991055697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84bc1017b0 msgr2=0x7f84bc108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:32.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.015+0000 7f84c41ff700 1 --2- 172.21.15.133:0/1991055697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84bc1017b0 0x7f84bc108070 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f84b400b0a0 tx=0x7f84b400b3b0 comp rx=0 tx=0).stop 2023-07-14T18:16:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.015+0000 7f84c41ff700 1 -- 172.21.15.133:0/1991055697 shutdown_connections 2023-07-14T18:16:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.015+0000 7f84c41ff700 1 --2- 172.21.15.133:0/1991055697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84bc0ffc30 0x7f84bc100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.015+0000 7f84c41ff700 1 --2- 172.21.15.133:0/1991055697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84bc1017b0 0x7f84bc108070 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.015+0000 7f84c41ff700 1 --2- 172.21.15.133:0/1991055697 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84bc100e20 0x7f84bc101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.015+0000 7f84c41ff700 1 -- 172.21.15.133:0/1991055697 >> 172.21.15.133:0/1991055697 conn(0x7f84bc0fb210 msgr2=0x7f84bc0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.015+0000 7f84c41ff700 1 -- 172.21.15.133:0/1991055697 shutdown_connections 2023-07-14T18:16:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.015+0000 7f84c41ff700 1 -- 172.21.15.133:0/1991055697 wait complete. 2023-07-14T18:16:32.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.016+0000 7f84c41ff700 1 Processor -- start 2023-07-14T18:16:32.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.016+0000 7f84c41ff700 1 -- start start 2023-07-14T18:16:32.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.016+0000 7f84c41ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84bc0ffc30 0x7f84bc195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:32.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.016+0000 7f84c41ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84bc100e20 0x7f84bc195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:32.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.016+0000 7f84c41ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84bc1017b0 0x7f84bc19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:32.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.016+0000 7f84c41ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc19d580 con 0x7f84bc1017b0 2023-07-14T18:16:32.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.016+0000 7f84c41ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc19d6c0 con 0x7f84bc0ffc30 2023-07-14T18:16:32.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.016+0000 7f84c41ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc19d850 con 0x7f84bc100e20 2023-07-14T18:16:32.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.017+0000 7f84c1f9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84bc0ffc30 0x7f84bc195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:32.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.017+0000 7f84c179a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84bc100e20 0x7f84bc195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:32.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.017+0000 7f84c1f9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84bc0ffc30 0x7f84bc195a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:33712/0 (socket says 172.21.15.133:33712) 2023-07-14T18:16:32.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.017+0000 7f84c1f9b700 1 -- 172.21.15.133:0/1946284885 learned_addr learned my addr 172.21.15.133:0/1946284885 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:32.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.017+0000 7f84c279c700 1 --2- 172.21.15.133:0/1946284885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84bc1017b0 0x7f84bc19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:32.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.017+0000 7f84c179a700 1 -- 172.21.15.133:0/1946284885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84bc0ffc30 msgr2=0x7f84bc195a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:32.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.017+0000 7f84c179a700 1 --2- 172.21.15.133:0/1946284885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84bc0ffc30 0x7f84bc195a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:32.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.017+0000 7f84c179a700 1 -- 172.21.15.133:0/1946284885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84bc1017b0 msgr2=0x7f84bc19cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:32.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.017+0000 7f84c179a700 1 --2- 172.21.15.133:0/1946284885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84bc1017b0 0x7f84bc19cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:32.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.017+0000 7f84c179a700 1 -- 172.21.15.133:0/1946284885 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b4009d00 con 0x7f84bc100e20 2023-07-14T18:16:32.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.017+0000 7f84c1f9b700 1 --2- 172.21.15.133:0/1946284885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84bc0ffc30 0x7f84bc195a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:16:32.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.017+0000 7f84c279c700 1 --2- 172.21.15.133:0/1946284885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84bc1017b0 0x7f84bc19cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:16:32.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.017+0000 7f84c179a700 1 --2- 172.21.15.133:0/1946284885 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84bc100e20 0x7f84bc195f40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f84ac00e9f0 tx=0x7f84ac00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:32.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.017+0000 7f84b2ffd700 1 -- 172.21.15.133:0/1946284885 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac00c9c0 con 0x7f84bc100e20 2023-07-14T18:16:32.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.018+0000 7f84b2ffd700 1 -- 172.21.15.133:0/1946284885 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84ac010430 con 0x7f84bc100e20 2023-07-14T18:16:32.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.018+0000 7f84b2ffd700 1 -- 172.21.15.133:0/1946284885 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac018680 con 0x7f84bc100e20 2023-07-14T18:16:32.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.018+0000 7f84c41ff700 1 -- 172.21.15.133:0/1946284885 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84bc19db30 con 0x7f84bc100e20 2023-07-14T18:16:32.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.018+0000 7f84c41ff700 1 -- 172.21.15.133:0/1946284885 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84bc19dff0 con 0x7f84bc100e20 2023-07-14T18:16:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.019+0000 7f84c41ff700 1 -- 172.21.15.133:0/1946284885 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84bc1070d0 con 0x7f84bc100e20 2023-07-14T18:16:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.021+0000 7f84b2ffd700 1 -- 172.21.15.133:0/1946284885 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f84ac010860 con 0x7f84bc100e20 2023-07-14T18:16:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.022+0000 7f84b2ffd700 1 --2- 172.21.15.133:0/1946284885 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f84a80a11a0 0x7f84a80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.023+0000 7f84b2ffd700 1 -- 172.21.15.133:0/1946284885 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f84ac014070 con 0x7f84bc100e20 2023-07-14T18:16:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.023+0000 7f84c1f9b700 1 --2- 172.21.15.133:0/1946284885 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f84a80a11a0 0x7f84a80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:32.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.023+0000 7f84c1f9b700 1 --2- 172.21.15.133:0/1946284885 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f84a80a11a0 0x7f84a80a3650 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f84b8005950 tx=0x7f84b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:32.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.024+0000 7f84b2ffd700 1 -- 172.21.15.133:0/1946284885 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f84ac096130 con 0x7f84bc100e20 2023-07-14T18:16:32.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.219+0000 7f84c41ff700 1 -- 172.21.15.133:0/1946284885 --> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f84bc060a80 con 0x7f84a80a11a0 2023-07-14T18:16:32.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.231+0000 7f84b2ffd700 1 -- 172.21.15.133:0/1946284885 <== mgr.14152 v2:172.21.15.133:6800/196197529 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426624 (secure 0 0 0) 0x7f84bc060a80 con 0x7f84a80a11a0 2023-07-14T18:16:32.281 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:32.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.236+0000 7f84c41ff700 1 -- 172.21.15.133:0/1946284885 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f84a80a11a0 msgr2=0x7f84a80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:32.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.236+0000 7f84c41ff700 1 --2- 172.21.15.133:0/1946284885 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f84a80a11a0 0x7f84a80a3650 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f84b8005950 tx=0x7f84b8009260 comp rx=0 tx=0).stop 2023-07-14T18:16:32.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.236+0000 7f84c41ff700 1 -- 172.21.15.133:0/1946284885 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84bc100e20 msgr2=0x7f84bc195f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:32.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.236+0000 7f84c41ff700 1 --2- 172.21.15.133:0/1946284885 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84bc100e20 0x7f84bc195f40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f84ac00e9f0 tx=0x7f84ac00ed00 comp rx=0 tx=0).stop 2023-07-14T18:16:32.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.236+0000 7f84c41ff700 1 -- 172.21.15.133:0/1946284885 shutdown_connections 2023-07-14T18:16:32.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.236+0000 7f84c41ff700 1 --2- 172.21.15.133:0/1946284885 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f84a80a11a0 0x7f84a80a3650 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:32.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.236+0000 7f84c41ff700 1 --2- 172.21.15.133:0/1946284885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84bc0ffc30 0x7f84bc195a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:32.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.236+0000 7f84c41ff700 1 --2- 172.21.15.133:0/1946284885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84bc1017b0 0x7f84bc19cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:32.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.236+0000 7f84c41ff700 1 --2- 172.21.15.133:0/1946284885 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84bc100e20 0x7f84bc195f40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:32.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.237+0000 7f84c41ff700 1 -- 172.21.15.133:0/1946284885 >> 172.21.15.133:0/1946284885 conn(0x7f84bc0fb210 msgr2=0x7f84bc104210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:32.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.237+0000 7f84c41ff700 1 -- 172.21.15.133:0/1946284885 shutdown_connections 2023-07-14T18:16:32.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.237+0000 7f84c41ff700 1 -- 172.21.15.133:0/1946284885 wait complete. 2023-07-14T18:16:32.285 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2023-07-14T18:16:32.301 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2023-07-14T18:16:31.466798+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660727,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":746,"num_read_kb":493,"num_write":352,"num_write_kb":3066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1181441,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336492,"kb_used_data":12300,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124605332,"statfs":{"total":1151940427776,"available":1151595859968,"internally_reserved":0,"allocated":12595200,"data_stored":8098401,"data_compressed":390240,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6152,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":40,"num_write":8,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003021"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.487205+0000","last_change":"2023-07-14T18:14:17.927179+0000","last_active":"2023-07-14T18:16:17.487205+0000","last_peered":"2023-07-14T18:16:17.487205+0000","last_clean":"2023-07-14T18:16:17.487205+0000","last_became_active":"2023-07-14T18:14:17.926880+0000","last_became_peered":"2023-07-14T18:14:17.926880+0000","last_unstale":"2023-07-14T18:16:17.487205+0000","last_undegraded":"2023-07-14T18:16:17.487205+0000","last_fullsized":"2023-07-14T18:16:17.487205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:41:35.598778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.483578+0000","last_change":"2023-07-14T18:14:17.933834+0000","last_active":"2023-07-14T18:16:17.483578+0000","last_peered":"2023-07-14T18:16:17.483578+0000","last_clean":"2023-07-14T18:16:17.483578+0000","last_became_active":"2023-07-14T18:14:17.933529+0000","last_became_peered":"2023-07-14T18:14:17.933529+0000","last_unstale":"2023-07-14T18:16:17.483578+0000","last_undegraded":"2023-07-14T18:16:17.483578+0000","last_fullsized":"2023-07-14T18:16:17.483578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:48:46.885347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.479455+0000","last_change":"2023-07-14T18:14:17.928400+0000","last_active":"2023-07-14T18:16:17.479455+0000","last_peered":"2023-07-14T18:16:17.479455+0000","last_clean":"2023-07-14T18:16:17.479455+0000","last_became_active":"2023-07-14T18:14:17.928077+0000","last_became_peered":"2023-07-14T18:14:17.928077+0000","last_unstale":"2023-07-14T18:16:17.479455+0000","last_undegraded":"2023-07-14T18:16:17.479455+0000","last_fullsized":"2023-07-14T18:16:17.479455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:52:23.012871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.472507+0000","last_change":"2023-07-14T18:14:17.932623+0000","last_active":"2023-07-14T18:16:17.472507+0000","last_peered":"2023-07-14T18:16:17.472507+0000","last_clean":"2023-07-14T18:16:17.472507+0000","last_became_active":"2023-07-14T18:14:17.932298+0000","last_became_peered":"2023-07-14T18:14:17.932298+0000","last_unstale":"2023-07-14T18:16:17.472507+0000","last_undegraded":"2023-07-14T18:16:17.472507+0000","last_fullsized":"2023-07-14T18:16:17.472507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:09:59.192311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.469953+0000","last_change":"2023-07-14T18:14:17.927109+0000","last_active":"2023-07-14T18:16:17.469953+0000","last_peered":"2023-07-14T18:16:17.469953+0000","last_clean":"2023-07-14T18:16:17.469953+0000","last_became_active":"2023-07-14T18:14:17.926800+0000","last_became_peered":"2023-07-14T18:14:17.926800+0000","last_unstale":"2023-07-14T18:16:17.469953+0000","last_undegraded":"2023-07-14T18:16:17.469953+0000","last_fullsized":"2023-07-14T18:16:17.469953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:59:27.287187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.481171+0000","last_change":"2023-07-14T18:14:17.935380+0000","last_active":"2023-07-14T18:16:17.481171+0000","last_peered":"2023-07-14T18:16:17.481171+0000","last_clean":"2023-07-14T18:16:17.481171+0000","last_became_active":"2023-07-14T18:14:17.935099+0000","last_became_peered":"2023-07-14T18:14:17.935099+0000","last_unstale":"2023-07-14T18:16:17.481171+0000","last_undegraded":"2023-07-14T18:16:17.481171+0000","last_fullsized":"2023-07-14T18:16:17.481171+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:22:36.754253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.482663+0000","last_change":"2023-07-14T18:14:17.930207+0000","last_active":"2023-07-14T18:16:17.482663+0000","last_peered":"2023-07-14T18:16:17.482663+0000","last_clean":"2023-07-14T18:16:17.482663+0000","last_became_active":"2023-07-14T18:14:17.929898+0000","last_became_peered":"2023-07-14T18:14:17.929898+0000","last_unstale":"2023-07-14T18:16:17.482663+0000","last_undegraded":"2023-07-14T18:16:17.482663+0000","last_fullsized":"2023-07-14T18:16:17.482663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:53:19.205250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.473699+0000","last_change":"2023-07-14T18:14:17.928645+0000","last_active":"2023-07-14T18:16:17.473699+0000","last_peered":"2023-07-14T18:16:17.473699+0000","last_clean":"2023-07-14T18:16:17.473699+0000","last_became_active":"2023-07-14T18:14:17.928288+0000","last_became_peered":"2023-07-14T18:14:17.928288+0000","last_unstale":"2023-07-14T18:16:17.473699+0000","last_undegraded":"2023-07-14T18:16:17.473699+0000","last_fullsized":"2023-07-14T18:16:17.473699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:37:39.330884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.477077+0000","last_change":"2023-07-14T18:14:17.925888+0000","last_active":"2023-07-14T18:16:17.477077+0000","last_peered":"2023-07-14T18:16:17.477077+0000","last_clean":"2023-07-14T18:16:17.477077+0000","last_became_active":"2023-07-14T18:14:17.925580+0000","last_became_peered":"2023-07-14T18:14:17.925580+0000","last_unstale":"2023-07-14T18:16:17.477077+0000","last_undegraded":"2023-07-14T18:16:17.477077+0000","last_fullsized":"2023-07-14T18:16:17.477077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:09:13.877679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.482686+0000","last_change":"2023-07-14T18:14:17.924440+0000","last_active":"2023-07-14T18:16:17.482686+0000","last_peered":"2023-07-14T18:16:17.482686+0000","last_clean":"2023-07-14T18:16:17.482686+0000","last_became_active":"2023-07-14T18:14:17.923839+0000","last_became_peered":"2023-07-14T18:14:17.923839+0000","last_unstale":"2023-07-14T18:16:17.482686+0000","last_undegraded":"2023-07-14T18:16:17.482686+0000","last_fullsized":"2023-07-14T18:16:17.482686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:29:23.092881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.486380+0000","last_change":"2023-07-14T18:14:17.931067+0000","last_active":"2023-07-14T18:16:17.486380+0000","last_peered":"2023-07-14T18:16:17.486380+0000","last_clean":"2023-07-14T18:16:17.486380+0000","last_became_active":"2023-07-14T18:14:17.930633+0000","last_became_peered":"2023-07-14T18:14:17.930633+0000","last_unstale":"2023-07-14T18:16:17.486380+0000","last_undegraded":"2023-07-14T18:16:17.486380+0000","last_fullsized":"2023-07-14T18:16:17.486380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:04:56.675009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.471892+0000","last_change":"2023-07-14T18:14:17.932770+0000","last_active":"2023-07-14T18:16:17.471892+0000","last_peered":"2023-07-14T18:16:17.471892+0000","last_clean":"2023-07-14T18:16:17.471892+0000","last_became_active":"2023-07-14T18:14:17.932438+0000","last_became_peered":"2023-07-14T18:14:17.932438+0000","last_unstale":"2023-07-14T18:16:17.471892+0000","last_undegraded":"2023-07-14T18:16:17.471892+0000","last_fullsized":"2023-07-14T18:16:17.471892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:18:53.813312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.482161+0000","last_change":"2023-07-14T18:14:17.938213+0000","last_active":"2023-07-14T18:16:17.482161+0000","last_peered":"2023-07-14T18:16:17.482161+0000","last_clean":"2023-07-14T18:16:17.482161+0000","last_became_active":"2023-07-14T18:14:17.937565+0000","last_became_peered":"2023-07-14T18:14:17.937565+0000","last_unstale":"2023-07-14T18:16:17.482161+0000","last_undegraded":"2023-07-14T18:16:17.482161+0000","last_fullsized":"2023-07-14T18:16:17.482161+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:51:47.121447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.483823+0000","last_change":"2023-07-14T18:14:17.934677+0000","last_active":"2023-07-14T18:16:17.483823+0000","last_peered":"2023-07-14T18:16:17.483823+0000","last_clean":"2023-07-14T18:16:17.483823+0000","last_became_active":"2023-07-14T18:14:17.933622+0000","last_became_peered":"2023-07-14T18:14:17.933622+0000","last_unstale":"2023-07-14T18:16:17.483823+0000","last_undegraded":"2023-07-14T18:16:17.483823+0000","last_fullsized":"2023-07-14T18:16:17.483823+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:26:28.437669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.484282+0000","last_change":"2023-07-14T18:14:17.935317+0000","last_active":"2023-07-14T18:16:17.484282+0000","last_peered":"2023-07-14T18:16:17.484282+0000","last_clean":"2023-07-14T18:16:17.484282+0000","last_became_active":"2023-07-14T18:14:17.935034+0000","last_became_peered":"2023-07-14T18:14:17.935034+0000","last_unstale":"2023-07-14T18:16:17.484282+0000","last_undegraded":"2023-07-14T18:16:17.484282+0000","last_fullsized":"2023-07-14T18:16:17.484282+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:42:03.287591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.488504+0000","last_change":"2023-07-14T18:14:17.928222+0000","last_active":"2023-07-14T18:16:17.488504+0000","last_peered":"2023-07-14T18:16:17.488504+0000","last_clean":"2023-07-14T18:16:17.488504+0000","last_became_active":"2023-07-14T18:14:17.927964+0000","last_became_peered":"2023-07-14T18:14:17.927964+0000","last_unstale":"2023-07-14T18:16:17.488504+0000","last_undegraded":"2023-07-14T18:16:17.488504+0000","last_fullsized":"2023-07-14T18:16:17.488504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:42:45.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.486729+0000","last_change":"2023-07-14T18:14:17.927276+0000","last_active":"2023-07-14T18:16:17.486729+0000","last_peered":"2023-07-14T18:16:17.486729+0000","last_clean":"2023-07-14T18:16:17.486729+0000","last_became_active":"2023-07-14T18:14:17.926952+0000","last_became_peered":"2023-07-14T18:14:17.926952+0000","last_unstale":"2023-07-14T18:16:17.486729+0000","last_undegraded":"2023-07-14T18:16:17.486729+0000","last_fullsized":"2023-07-14T18:16:17.486729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:00:09.825668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.476643+0000","last_change":"2023-07-14T18:14:17.937364+0000","last_active":"2023-07-14T18:16:17.476643+0000","last_peered":"2023-07-14T18:16:17.476643+0000","last_clean":"2023-07-14T18:16:17.476643+0000","last_became_active":"2023-07-14T18:14:17.937014+0000","last_became_peered":"2023-07-14T18:14:17.937014+0000","last_unstale":"2023-07-14T18:16:17.476643+0000","last_undegraded":"2023-07-14T18:16:17.476643+0000","last_fullsized":"2023-07-14T18:16:17.476643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:31:10.117800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.472419+0000","last_change":"2023-07-14T18:14:17.920358+0000","last_active":"2023-07-14T18:16:17.472419+0000","last_peered":"2023-07-14T18:16:17.472419+0000","last_clean":"2023-07-14T18:16:17.472419+0000","last_became_active":"2023-07-14T18:14:17.920042+0000","last_became_peered":"2023-07-14T18:14:17.920042+0000","last_unstale":"2023-07-14T18:16:17.472419+0000","last_undegraded":"2023-07-14T18:16:17.472419+0000","last_fullsized":"2023-07-14T18:16:17.472419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:07:30.258643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093399+0000","last_change":"2023-07-14T18:14:22.045376+0000","last_active":"2023-07-14T18:15:50.093399+0000","last_peered":"2023-07-14T18:15:50.093399+0000","last_clean":"2023-07-14T18:15:50.093399+0000","last_became_active":"2023-07-14T18:14:22.044984+0000","last_became_peered":"2023-07-14T18:14:22.044984+0000","last_unstale":"2023-07-14T18:15:50.093399+0000","last_undegraded":"2023-07-14T18:15:50.093399+0000","last_fullsized":"2023-07-14T18:15:50.093399+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:26:13.981694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964742+0000","last_change":"2023-07-14T18:14:48.954270+0000","last_active":"2023-07-14T18:15:49.964742+0000","last_peered":"2023-07-14T18:15:49.964742+0000","last_clean":"2023-07-14T18:15:49.964742+0000","last_became_active":"2023-07-14T18:14:48.947868+0000","last_became_peered":"2023-07-14T18:14:48.947868+0000","last_unstale":"2023-07-14T18:15:49.964742+0000","last_undegraded":"2023-07-14T18:15:49.964742+0000","last_fullsized":"2023-07-14T18:15:49.964742+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:56:18.186337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.472946+0000","last_change":"2023-07-14T18:14:17.925822+0000","last_active":"2023-07-14T18:16:17.472946+0000","last_peered":"2023-07-14T18:16:17.472946+0000","last_clean":"2023-07-14T18:16:17.472946+0000","last_became_active":"2023-07-14T18:14:17.925508+0000","last_became_peered":"2023-07-14T18:14:17.925508+0000","last_unstale":"2023-07-14T18:16:17.472946+0000","last_undegraded":"2023-07-14T18:16:17.472946+0000","last_fullsized":"2023-07-14T18:16:17.472946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:05:04.319300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958344+0000","last_change":"2023-07-14T18:14:22.043098+0000","last_active":"2023-07-14T18:15:49.958344+0000","last_peered":"2023-07-14T18:15:49.958344+0000","last_clean":"2023-07-14T18:15:49.958344+0000","last_became_active":"2023-07-14T18:14:22.042517+0000","last_became_peered":"2023-07-14T18:14:22.042517+0000","last_unstale":"2023-07-14T18:15:49.958344+0000","last_undegraded":"2023-07-14T18:15:49.958344+0000","last_fullsized":"2023-07-14T18:15:49.958344+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:46:20.942876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.095016+0000","last_change":"2023-07-14T18:14:16.927612+0000","last_active":"2023-07-14T18:15:50.095016+0000","last_peered":"2023-07-14T18:15:50.095016+0000","last_clean":"2023-07-14T18:15:50.095016+0000","last_became_active":"2023-07-14T18:14:16.927284+0000","last_became_peered":"2023-07-14T18:14:16.927284+0000","last_unstale":"2023-07-14T18:15:50.095016+0000","last_undegraded":"2023-07-14T18:15:50.095016+0000","last_fullsized":"2023-07-14T18:15:50.095016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:49:41.365291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.488089+0000","last_change":"2023-07-14T18:14:17.930624+0000","last_active":"2023-07-14T18:16:17.488089+0000","last_peered":"2023-07-14T18:16:17.488089+0000","last_clean":"2023-07-14T18:16:17.488089+0000","last_became_active":"2023-07-14T18:14:17.930273+0000","last_became_peered":"2023-07-14T18:14:17.930273+0000","last_unstale":"2023-07-14T18:16:17.488089+0000","last_undegraded":"2023-07-14T18:16:17.488089+0000","last_fullsized":"2023-07-14T18:16:17.488089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:08:45.137173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958582+0000","last_change":"2023-07-14T18:14:49.543208+0000","last_active":"2023-07-14T18:15:49.958582+0000","last_peered":"2023-07-14T18:15:49.958582+0000","last_clean":"2023-07-14T18:15:49.958582+0000","last_became_active":"2023-07-14T18:14:49.542685+0000","last_became_peered":"2023-07-14T18:14:49.542685+0000","last_unstale":"2023-07-14T18:15:49.958582+0000","last_undegraded":"2023-07-14T18:15:49.958582+0000","last_fullsized":"2023-07-14T18:15:49.958582+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:34:49.230001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.959423+0000","last_change":"2023-07-14T18:14:16.926185+0000","last_active":"2023-07-14T18:15:49.959423+0000","last_peered":"2023-07-14T18:15:49.959423+0000","last_clean":"2023-07-14T18:15:49.959423+0000","last_became_active":"2023-07-14T18:14:16.925874+0000","last_became_peered":"2023-07-14T18:14:16.925874+0000","last_unstale":"2023-07-14T18:15:49.959423+0000","last_undegraded":"2023-07-14T18:15:49.959423+0000","last_fullsized":"2023-07-14T18:15:49.959423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:54:20.071669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.483223+0000","last_change":"2023-07-14T18:14:17.936601+0000","last_active":"2023-07-14T18:16:17.483223+0000","last_peered":"2023-07-14T18:16:17.483223+0000","last_clean":"2023-07-14T18:16:17.483223+0000","last_became_active":"2023-07-14T18:14:17.936291+0000","last_became_peered":"2023-07-14T18:14:17.936291+0000","last_unstale":"2023-07-14T18:16:17.483223+0000","last_undegraded":"2023-07-14T18:16:17.483223+0000","last_fullsized":"2023-07-14T18:16:17.483223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:32:59.220779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958529+0000","last_change":"2023-07-14T18:14:22.042952+0000","last_active":"2023-07-14T18:15:49.958529+0000","last_peered":"2023-07-14T18:15:49.958529+0000","last_clean":"2023-07-14T18:15:49.958529+0000","last_became_active":"2023-07-14T18:14:22.042256+0000","last_became_peered":"2023-07-14T18:14:22.042256+0000","last_unstale":"2023-07-14T18:15:49.958529+0000","last_undegraded":"2023-07-14T18:15:49.958529+0000","last_fullsized":"2023-07-14T18:15:49.958529+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:00:33.594352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958938+0000","last_change":"2023-07-14T18:14:16.920902+0000","last_active":"2023-07-14T18:15:49.958938+0000","last_peered":"2023-07-14T18:15:49.958938+0000","last_clean":"2023-07-14T18:15:49.958938+0000","last_became_active":"2023-07-14T18:14:16.920554+0000","last_became_peered":"2023-07-14T18:14:16.920554+0000","last_unstale":"2023-07-14T18:15:49.958938+0000","last_undegraded":"2023-07-14T18:15:49.958938+0000","last_fullsized":"2023-07-14T18:15:49.958938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:42:47.908351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.477004+0000","last_change":"2023-07-14T18:14:17.925005+0000","last_active":"2023-07-14T18:16:17.477004+0000","last_peered":"2023-07-14T18:16:17.477004+0000","last_clean":"2023-07-14T18:16:17.477004+0000","last_became_active":"2023-07-14T18:14:17.924723+0000","last_became_peered":"2023-07-14T18:14:17.924723+0000","last_unstale":"2023-07-14T18:16:17.477004+0000","last_undegraded":"2023-07-14T18:16:17.477004+0000","last_fullsized":"2023-07-14T18:16:17.477004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:23.634518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093304+0000","last_change":"2023-07-14T18:14:22.038482+0000","last_active":"2023-07-14T18:15:50.093304+0000","last_peered":"2023-07-14T18:15:50.093304+0000","last_clean":"2023-07-14T18:15:50.093304+0000","last_became_active":"2023-07-14T18:14:22.038059+0000","last_became_peered":"2023-07-14T18:14:22.038059+0000","last_unstale":"2023-07-14T18:15:50.093304+0000","last_undegraded":"2023-07-14T18:15:50.093304+0000","last_fullsized":"2023-07-14T18:15:50.093304+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:14:37.242779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'8","reported_seq":146,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:25.035755+0000","last_change":"2023-07-14T18:14:48.949585+0000","last_active":"2023-07-14T18:16:25.035755+0000","last_peered":"2023-07-14T18:16:25.035755+0000","last_clean":"2023-07-14T18:16:25.035755+0000","last_became_active":"2023-07-14T18:14:48.949106+0000","last_became_peered":"2023-07-14T18:14:48.949106+0000","last_unstale":"2023-07-14T18:16:25.035755+0000","last_undegraded":"2023-07-14T18:16:25.035755+0000","last_fullsized":"2023-07-14T18:16:25.035755+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:10:07.660525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1421,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.471828+0000","last_change":"2023-07-14T18:14:17.934894+0000","last_active":"2023-07-14T18:16:17.471828+0000","last_peered":"2023-07-14T18:16:17.471828+0000","last_clean":"2023-07-14T18:16:17.471828+0000","last_became_active":"2023-07-14T18:14:17.934651+0000","last_became_peered":"2023-07-14T18:14:17.934651+0000","last_unstale":"2023-07-14T18:16:17.471828+0000","last_undegraded":"2023-07-14T18:16:17.471828+0000","last_fullsized":"2023-07-14T18:16:17.471828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:56:46.813395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093307+0000","last_change":"2023-07-14T18:14:22.043611+0000","last_active":"2023-07-14T18:15:50.093307+0000","last_peered":"2023-07-14T18:15:50.093307+0000","last_clean":"2023-07-14T18:15:50.093307+0000","last_became_active":"2023-07-14T18:14:22.043101+0000","last_became_peered":"2023-07-14T18:14:22.043101+0000","last_unstale":"2023-07-14T18:15:50.093307+0000","last_undegraded":"2023-07-14T18:15:50.093307+0000","last_fullsized":"2023-07-14T18:15:50.093307+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:14:51.438990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.983824+0000","last_change":"2023-07-14T18:14:16.925321+0000","last_active":"2023-07-14T18:15:49.983824+0000","last_peered":"2023-07-14T18:15:49.983824+0000","last_clean":"2023-07-14T18:15:49.983824+0000","last_became_active":"2023-07-14T18:14:16.925016+0000","last_became_peered":"2023-07-14T18:14:16.925016+0000","last_unstale":"2023-07-14T18:15:49.983824+0000","last_undegraded":"2023-07-14T18:15:49.983824+0000","last_fullsized":"2023-07-14T18:15:49.983824+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:41:08.173620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.464867+0000","last_change":"2023-07-14T18:14:17.928838+0000","last_active":"2023-07-14T18:16:17.464867+0000","last_peered":"2023-07-14T18:16:17.464867+0000","last_clean":"2023-07-14T18:16:17.464867+0000","last_became_active":"2023-07-14T18:14:17.928511+0000","last_became_peered":"2023-07-14T18:14:17.928511+0000","last_unstale":"2023-07-14T18:16:17.464867+0000","last_undegraded":"2023-07-14T18:16:17.464867+0000","last_fullsized":"2023-07-14T18:16:17.464867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:35:21.406449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964630+0000","last_change":"2023-07-14T18:14:22.043182+0000","last_active":"2023-07-14T18:15:49.964630+0000","last_peered":"2023-07-14T18:15:49.964630+0000","last_clean":"2023-07-14T18:15:49.964630+0000","last_became_active":"2023-07-14T18:14:22.042779+0000","last_became_peered":"2023-07-14T18:14:22.042779+0000","last_unstale":"2023-07-14T18:15:49.964630+0000","last_undegraded":"2023-07-14T18:15:49.964630+0000","last_fullsized":"2023-07-14T18:15:49.964630+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:22:42.125442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958189+0000","last_change":"2023-07-14T18:14:16.935321+0000","last_active":"2023-07-14T18:15:49.958189+0000","last_peered":"2023-07-14T18:15:49.958189+0000","last_clean":"2023-07-14T18:15:49.958189+0000","last_became_active":"2023-07-14T18:14:16.934494+0000","last_became_peered":"2023-07-14T18:14:16.934494+0000","last_unstale":"2023-07-14T18:15:49.958189+0000","last_undegraded":"2023-07-14T18:15:49.958189+0000","last_fullsized":"2023-07-14T18:15:49.958189+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:11:15.794692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.480564+0000","last_change":"2023-07-14T18:14:17.922139+0000","last_active":"2023-07-14T18:16:17.480564+0000","last_peered":"2023-07-14T18:16:17.480564+0000","last_clean":"2023-07-14T18:16:17.480564+0000","last_became_active":"2023-07-14T18:14:17.921829+0000","last_became_peered":"2023-07-14T18:14:17.921829+0000","last_unstale":"2023-07-14T18:16:17.480564+0000","last_undegraded":"2023-07-14T18:16:17.480564+0000","last_fullsized":"2023-07-14T18:16:17.480564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:19:51.159357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.094035+0000","last_change":"2023-07-14T18:14:22.033625+0000","last_active":"2023-07-14T18:15:50.094035+0000","last_peered":"2023-07-14T18:15:50.094035+0000","last_clean":"2023-07-14T18:15:50.094035+0000","last_became_active":"2023-07-14T18:14:22.033253+0000","last_became_peered":"2023-07-14T18:14:22.033253+0000","last_unstale":"2023-07-14T18:15:50.094035+0000","last_undegraded":"2023-07-14T18:15:50.094035+0000","last_fullsized":"2023-07-14T18:15:50.094035+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:36:04.591067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964644+0000","last_change":"2023-07-14T18:14:16.923864+0000","last_active":"2023-07-14T18:15:49.964644+0000","last_peered":"2023-07-14T18:15:49.964644+0000","last_clean":"2023-07-14T18:15:49.964644+0000","last_became_active":"2023-07-14T18:14:16.923546+0000","last_became_peered":"2023-07-14T18:14:16.923546+0000","last_unstale":"2023-07-14T18:15:49.964644+0000","last_undegraded":"2023-07-14T18:15:49.964644+0000","last_fullsized":"2023-07-14T18:15:49.964644+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:07:13.280104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.475115+0000","last_change":"2023-07-14T18:14:17.936473+0000","last_active":"2023-07-14T18:16:17.475115+0000","last_peered":"2023-07-14T18:16:17.475115+0000","last_clean":"2023-07-14T18:16:17.475115+0000","last_became_active":"2023-07-14T18:14:17.935387+0000","last_became_peered":"2023-07-14T18:14:17.935387+0000","last_unstale":"2023-07-14T18:16:17.475115+0000","last_undegraded":"2023-07-14T18:16:17.475115+0000","last_fullsized":"2023-07-14T18:16:17.475115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:05:12.007411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093186+0000","last_change":"2023-07-14T18:14:22.041544+0000","last_active":"2023-07-14T18:15:50.093186+0000","last_peered":"2023-07-14T18:15:50.093186+0000","last_clean":"2023-07-14T18:15:50.093186+0000","last_became_active":"2023-07-14T18:14:22.041183+0000","last_became_peered":"2023-07-14T18:14:22.041183+0000","last_unstale":"2023-07-14T18:15:50.093186+0000","last_undegraded":"2023-07-14T18:15:50.093186+0000","last_fullsized":"2023-07-14T18:15:50.093186+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:08:52.284739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'4","reported_seq":130,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:26.037087+0000","last_change":"2023-07-14T18:14:16.928860+0000","last_active":"2023-07-14T18:16:26.037087+0000","last_peered":"2023-07-14T18:16:26.037087+0000","last_clean":"2023-07-14T18:16:26.037087+0000","last_became_active":"2023-07-14T18:14:16.928567+0000","last_became_peered":"2023-07-14T18:14:16.928567+0000","last_unstale":"2023-07-14T18:16:26.037087+0000","last_undegraded":"2023-07-14T18:16:26.037087+0000","last_fullsized":"2023-07-14T18:16:26.037087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:26:39.917215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.484494+0000","last_change":"2023-07-14T18:14:17.938456+0000","last_active":"2023-07-14T18:16:17.484494+0000","last_peered":"2023-07-14T18:16:17.484494+0000","last_clean":"2023-07-14T18:16:17.484494+0000","last_became_active":"2023-07-14T18:14:17.938086+0000","last_became_peered":"2023-07-14T18:14:17.938086+0000","last_unstale":"2023-07-14T18:16:17.484494+0000","last_undegraded":"2023-07-14T18:16:17.484494+0000","last_fullsized":"2023-07-14T18:16:17.484494+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:19:25.486023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964227+0000","last_change":"2023-07-14T18:14:22.042785+0000","last_active":"2023-07-14T18:15:49.964227+0000","last_peered":"2023-07-14T18:15:49.964227+0000","last_clean":"2023-07-14T18:15:49.964227+0000","last_became_active":"2023-07-14T18:14:22.042346+0000","last_became_peered":"2023-07-14T18:14:22.042346+0000","last_unstale":"2023-07-14T18:15:49.964227+0000","last_undegraded":"2023-07-14T18:15:49.964227+0000","last_fullsized":"2023-07-14T18:15:49.964227+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:49:14.017758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.962653+0000","last_change":"2023-07-14T18:14:16.921168+0000","last_active":"2023-07-14T18:15:49.962653+0000","last_peered":"2023-07-14T18:15:49.962653+0000","last_clean":"2023-07-14T18:15:49.962653+0000","last_became_active":"2023-07-14T18:14:16.920808+0000","last_became_peered":"2023-07-14T18:14:16.920808+0000","last_unstale":"2023-07-14T18:15:49.962653+0000","last_undegraded":"2023-07-14T18:15:49.962653+0000","last_fullsized":"2023-07-14T18:15:49.962653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:13:22.466581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.487015+0000","last_change":"2023-07-14T18:14:17.935100+0000","last_active":"2023-07-14T18:16:17.487015+0000","last_peered":"2023-07-14T18:16:17.487015+0000","last_clean":"2023-07-14T18:16:17.487015+0000","last_became_active":"2023-07-14T18:14:17.934794+0000","last_became_peered":"2023-07-14T18:14:17.934794+0000","last_unstale":"2023-07-14T18:16:17.487015+0000","last_undegraded":"2023-07-14T18:16:17.487015+0000","last_fullsized":"2023-07-14T18:16:17.487015+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:06:24.196022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958851+0000","last_change":"2023-07-14T18:14:22.043495+0000","last_active":"2023-07-14T18:15:49.958851+0000","last_peered":"2023-07-14T18:15:49.958851+0000","last_clean":"2023-07-14T18:15:49.958851+0000","last_became_active":"2023-07-14T18:14:22.042686+0000","last_became_peered":"2023-07-14T18:14:22.042686+0000","last_unstale":"2023-07-14T18:15:49.958851+0000","last_undegraded":"2023-07-14T18:15:49.958851+0000","last_fullsized":"2023-07-14T18:15:49.958851+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:50:08.855428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.965517+0000","last_change":"2023-07-14T18:14:16.926006+0000","last_active":"2023-07-14T18:15:49.965517+0000","last_peered":"2023-07-14T18:15:49.965517+0000","last_clean":"2023-07-14T18:15:49.965517+0000","last_became_active":"2023-07-14T18:14:16.925719+0000","last_became_peered":"2023-07-14T18:14:16.925719+0000","last_unstale":"2023-07-14T18:15:49.965517+0000","last_undegraded":"2023-07-14T18:15:49.965517+0000","last_fullsized":"2023-07-14T18:15:49.965517+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:16:46.148528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.480595+0000","last_change":"2023-07-14T18:14:17.934155+0000","last_active":"2023-07-14T18:16:17.480595+0000","last_peered":"2023-07-14T18:16:17.480595+0000","last_clean":"2023-07-14T18:16:17.480595+0000","last_became_active":"2023-07-14T18:14:17.933756+0000","last_became_peered":"2023-07-14T18:14:17.933756+0000","last_unstale":"2023-07-14T18:16:17.480595+0000","last_undegraded":"2023-07-14T18:16:17.480595+0000","last_fullsized":"2023-07-14T18:16:17.480595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:10.480686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964266+0000","last_change":"2023-07-14T18:14:22.039198+0000","last_active":"2023-07-14T18:15:49.964266+0000","last_peered":"2023-07-14T18:15:49.964266+0000","last_clean":"2023-07-14T18:15:49.964266+0000","last_became_active":"2023-07-14T18:14:22.038842+0000","last_became_peered":"2023-07-14T18:14:22.038842+0000","last_unstale":"2023-07-14T18:15:49.964266+0000","last_undegraded":"2023-07-14T18:15:49.964266+0000","last_fullsized":"2023-07-14T18:15:49.964266+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:31:14.995258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'38","reported_seq":273,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:27.039579+0000","last_change":"2023-07-14T18:14:16.920918+0000","last_active":"2023-07-14T18:16:27.039579+0000","last_peered":"2023-07-14T18:16:27.039579+0000","last_clean":"2023-07-14T18:16:27.039579+0000","last_became_active":"2023-07-14T18:14:16.920572+0000","last_became_peered":"2023-07-14T18:14:16.920572+0000","last_unstale":"2023-07-14T18:16:27.039579+0000","last_undegraded":"2023-07-14T18:16:27.039579+0000","last_fullsized":"2023-07-14T18:16:27.039579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:02:06.484223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65752,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":207,"num_read_kb":168,"num_write":39,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.483336+0000","last_change":"2023-07-14T18:14:17.929413+0000","last_active":"2023-07-14T18:16:17.483336+0000","last_peered":"2023-07-14T18:16:17.483336+0000","last_clean":"2023-07-14T18:16:17.483336+0000","last_became_active":"2023-07-14T18:14:17.929064+0000","last_became_peered":"2023-07-14T18:14:17.929064+0000","last_unstale":"2023-07-14T18:16:17.483336+0000","last_undegraded":"2023-07-14T18:16:17.483336+0000","last_fullsized":"2023-07-14T18:16:17.483336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:56:39.264420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.965827+0000","last_change":"2023-07-14T18:14:22.041983+0000","last_active":"2023-07-14T18:15:49.965827+0000","last_peered":"2023-07-14T18:15:49.965827+0000","last_clean":"2023-07-14T18:15:49.965827+0000","last_became_active":"2023-07-14T18:14:22.041553+0000","last_became_peered":"2023-07-14T18:14:22.041553+0000","last_unstale":"2023-07-14T18:15:49.965827+0000","last_undegraded":"2023-07-14T18:15:49.965827+0000","last_fullsized":"2023-07-14T18:15:49.965827+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:02:11.442175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093843+0000","last_change":"2023-07-14T18:14:16.925444+0000","last_active":"2023-07-14T18:15:50.093843+0000","last_peered":"2023-07-14T18:15:50.093843+0000","last_clean":"2023-07-14T18:15:50.093843+0000","last_became_active":"2023-07-14T18:14:16.925138+0000","last_became_peered":"2023-07-14T18:14:16.925138+0000","last_unstale":"2023-07-14T18:15:50.093843+0000","last_undegraded":"2023-07-14T18:15:50.093843+0000","last_fullsized":"2023-07-14T18:15:50.093843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:19:37.684882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.487136+0000","last_change":"2023-07-14T18:14:17.922351+0000","last_active":"2023-07-14T18:16:17.487136+0000","last_peered":"2023-07-14T18:16:17.487136+0000","last_clean":"2023-07-14T18:16:17.487136+0000","last_became_active":"2023-07-14T18:14:17.921973+0000","last_became_peered":"2023-07-14T18:14:17.921973+0000","last_unstale":"2023-07-14T18:16:17.487136+0000","last_undegraded":"2023-07-14T18:16:17.487136+0000","last_fullsized":"2023-07-14T18:16:17.487136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:27:35.200279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964677+0000","last_change":"2023-07-14T18:14:22.048274+0000","last_active":"2023-07-14T18:15:49.964677+0000","last_peered":"2023-07-14T18:15:49.964677+0000","last_clean":"2023-07-14T18:15:49.964677+0000","last_became_active":"2023-07-14T18:14:22.047946+0000","last_became_peered":"2023-07-14T18:14:22.047946+0000","last_unstale":"2023-07-14T18:15:49.964677+0000","last_undegraded":"2023-07-14T18:15:49.964677+0000","last_fullsized":"2023-07-14T18:15:49.964677+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:43:43.819233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.965500+0000","last_change":"2023-07-14T18:14:16.933566+0000","last_active":"2023-07-14T18:15:49.965500+0000","last_peered":"2023-07-14T18:15:49.965500+0000","last_clean":"2023-07-14T18:15:49.965500+0000","last_became_active":"2023-07-14T18:14:16.933300+0000","last_became_peered":"2023-07-14T18:14:16.933300+0000","last_unstale":"2023-07-14T18:15:49.965500+0000","last_undegraded":"2023-07-14T18:15:49.965500+0000","last_fullsized":"2023-07-14T18:15:49.965500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:37:32.251937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.488953+0000","last_change":"2023-07-14T18:14:17.928521+0000","last_active":"2023-07-14T18:16:17.488953+0000","last_peered":"2023-07-14T18:16:17.488953+0000","last_clean":"2023-07-14T18:16:17.488953+0000","last_became_active":"2023-07-14T18:14:17.928025+0000","last_became_peered":"2023-07-14T18:14:17.928025+0000","last_unstale":"2023-07-14T18:16:17.488953+0000","last_undegraded":"2023-07-14T18:16:17.488953+0000","last_fullsized":"2023-07-14T18:16:17.488953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:58:12.486113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964540+0000","last_change":"2023-07-14T18:14:22.046820+0000","last_active":"2023-07-14T18:15:49.964540+0000","last_peered":"2023-07-14T18:15:49.964540+0000","last_clean":"2023-07-14T18:15:49.964540+0000","last_became_active":"2023-07-14T18:14:22.046440+0000","last_became_peered":"2023-07-14T18:14:22.046440+0000","last_unstale":"2023-07-14T18:15:49.964540+0000","last_undegraded":"2023-07-14T18:15:49.964540+0000","last_fullsized":"2023-07-14T18:15:49.964540+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:40:11.333149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":135,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:29.040866+0000","last_change":"2023-07-14T18:14:16.922075+0000","last_active":"2023-07-14T18:16:29.040866+0000","last_peered":"2023-07-14T18:16:29.040866+0000","last_clean":"2023-07-14T18:16:29.040866+0000","last_became_active":"2023-07-14T18:14:16.921570+0000","last_became_peered":"2023-07-14T18:14:16.921570+0000","last_unstale":"2023-07-14T18:16:29.040866+0000","last_undegraded":"2023-07-14T18:16:29.040866+0000","last_fullsized":"2023-07-14T18:16:29.040866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:57:51.347575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.487451+0000","last_change":"2023-07-14T18:14:17.936439+0000","last_active":"2023-07-14T18:16:17.487451+0000","last_peered":"2023-07-14T18:16:17.487451+0000","last_clean":"2023-07-14T18:16:17.487451+0000","last_became_active":"2023-07-14T18:14:17.936138+0000","last_became_peered":"2023-07-14T18:14:17.936138+0000","last_unstale":"2023-07-14T18:16:17.487451+0000","last_undegraded":"2023-07-14T18:16:17.487451+0000","last_fullsized":"2023-07-14T18:16:17.487451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:32:25.986869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958834+0000","last_change":"2023-07-14T18:14:49.543302+0000","last_active":"2023-07-14T18:15:49.958834+0000","last_peered":"2023-07-14T18:15:49.958834+0000","last_clean":"2023-07-14T18:15:49.958834+0000","last_became_active":"2023-07-14T18:14:49.542768+0000","last_became_peered":"2023-07-14T18:14:49.542768+0000","last_unstale":"2023-07-14T18:15:49.958834+0000","last_undegraded":"2023-07-14T18:15:49.958834+0000","last_fullsized":"2023-07-14T18:15:49.958834+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:05:11.701773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958699+0000","last_change":"2023-07-14T18:14:16.926417+0000","last_active":"2023-07-14T18:15:49.958699+0000","last_peered":"2023-07-14T18:15:49.958699+0000","last_clean":"2023-07-14T18:15:49.958699+0000","last_became_active":"2023-07-14T18:14:16.926101+0000","last_became_peered":"2023-07-14T18:14:16.926101+0000","last_unstale":"2023-07-14T18:15:49.958699+0000","last_undegraded":"2023-07-14T18:15:49.958699+0000","last_fullsized":"2023-07-14T18:15:49.958699+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:16:34.754140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.476341+0000","last_change":"2023-07-14T18:14:48.950649+0000","last_active":"2023-07-14T18:16:17.476341+0000","last_peered":"2023-07-14T18:16:17.476341+0000","last_clean":"2023-07-14T18:16:17.476341+0000","last_became_active":"2023-07-14T18:14:48.950303+0000","last_became_peered":"2023-07-14T18:14:48.950303+0000","last_unstale":"2023-07-14T18:16:17.476341+0000","last_undegraded":"2023-07-14T18:16:17.476341+0000","last_fullsized":"2023-07-14T18:16:17.476341+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:03:15.248073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.965148+0000","last_change":"2023-07-14T18:14:22.042171+0000","last_active":"2023-07-14T18:15:49.965148+0000","last_peered":"2023-07-14T18:15:49.965148+0000","last_clean":"2023-07-14T18:15:49.965148+0000","last_became_active":"2023-07-14T18:14:22.041692+0000","last_became_peered":"2023-07-14T18:14:22.041692+0000","last_unstale":"2023-07-14T18:15:49.965148+0000","last_undegraded":"2023-07-14T18:15:49.965148+0000","last_fullsized":"2023-07-14T18:15:49.965148+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:51:41.773405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.959284+0000","last_change":"2023-07-14T18:14:16.922883+0000","last_active":"2023-07-14T18:15:49.959284+0000","last_peered":"2023-07-14T18:15:49.959284+0000","last_clean":"2023-07-14T18:15:49.959284+0000","last_became_active":"2023-07-14T18:14:16.922501+0000","last_became_peered":"2023-07-14T18:14:16.922501+0000","last_unstale":"2023-07-14T18:15:49.959284+0000","last_undegraded":"2023-07-14T18:15:49.959284+0000","last_fullsized":"2023-07-14T18:15:49.959284+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:53:01.489131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.478108+0000","last_change":"2023-07-14T18:14:17.922899+0000","last_active":"2023-07-14T18:16:17.478108+0000","last_peered":"2023-07-14T18:16:17.478108+0000","last_clean":"2023-07-14T18:16:17.478108+0000","last_became_active":"2023-07-14T18:14:17.922608+0000","last_became_peered":"2023-07-14T18:14:17.922608+0000","last_unstale":"2023-07-14T18:16:17.478108+0000","last_undegraded":"2023-07-14T18:16:17.478108+0000","last_fullsized":"2023-07-14T18:16:17.478108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:07:33.479250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.959164+0000","last_change":"2023-07-14T18:14:22.327484+0000","last_active":"2023-07-14T18:15:49.959164+0000","last_peered":"2023-07-14T18:15:49.959164+0000","last_clean":"2023-07-14T18:15:49.959164+0000","last_became_active":"2023-07-14T18:14:22.327010+0000","last_became_peered":"2023-07-14T18:14:22.327010+0000","last_unstale":"2023-07-14T18:15:49.959164+0000","last_undegraded":"2023-07-14T18:15:49.959164+0000","last_fullsized":"2023-07-14T18:15:49.959164+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:28:55.842693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964720+0000","last_change":"2023-07-14T18:14:16.930606+0000","last_active":"2023-07-14T18:15:49.964720+0000","last_peered":"2023-07-14T18:15:49.964720+0000","last_clean":"2023-07-14T18:15:49.964720+0000","last_became_active":"2023-07-14T18:14:16.930175+0000","last_became_peered":"2023-07-14T18:14:16.930175+0000","last_unstale":"2023-07-14T18:15:49.964720+0000","last_undegraded":"2023-07-14T18:15:49.964720+0000","last_fullsized":"2023-07-14T18:15:49.964720+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:30:16.214141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.471509+0000","last_change":"2023-07-14T18:14:17.931962+0000","last_active":"2023-07-14T18:16:17.471509+0000","last_peered":"2023-07-14T18:16:17.471509+0000","last_clean":"2023-07-14T18:16:17.471509+0000","last_became_active":"2023-07-14T18:14:17.931621+0000","last_became_peered":"2023-07-14T18:14:17.931621+0000","last_unstale":"2023-07-14T18:16:17.471509+0000","last_undegraded":"2023-07-14T18:16:17.471509+0000","last_fullsized":"2023-07-14T18:16:17.471509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:53:21.866251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958462+0000","last_change":"2023-07-14T18:14:22.038977+0000","last_active":"2023-07-14T18:15:49.958462+0000","last_peered":"2023-07-14T18:15:49.958462+0000","last_clean":"2023-07-14T18:15:49.958462+0000","last_became_active":"2023-07-14T18:14:22.038515+0000","last_became_peered":"2023-07-14T18:14:22.038515+0000","last_unstale":"2023-07-14T18:15:49.958462+0000","last_undegraded":"2023-07-14T18:15:49.958462+0000","last_fullsized":"2023-07-14T18:15:49.958462+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:18:48.042709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.962465+0000","last_change":"2023-07-14T18:14:16.925457+0000","last_active":"2023-07-14T18:15:49.962465+0000","last_peered":"2023-07-14T18:15:49.962465+0000","last_clean":"2023-07-14T18:15:49.962465+0000","last_became_active":"2023-07-14T18:14:16.925150+0000","last_became_peered":"2023-07-14T18:14:16.925150+0000","last_unstale":"2023-07-14T18:15:49.962465+0000","last_undegraded":"2023-07-14T18:15:49.962465+0000","last_fullsized":"2023-07-14T18:15:49.962465+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:15:42.815134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.481369+0000","last_change":"2023-07-14T18:14:17.926492+0000","last_active":"2023-07-14T18:16:17.481369+0000","last_peered":"2023-07-14T18:16:17.481369+0000","last_clean":"2023-07-14T18:16:17.481369+0000","last_became_active":"2023-07-14T18:14:17.926218+0000","last_became_peered":"2023-07-14T18:14:17.926218+0000","last_unstale":"2023-07-14T18:16:17.481369+0000","last_undegraded":"2023-07-14T18:16:17.481369+0000","last_fullsized":"2023-07-14T18:16:17.481369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:03:08.979533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958737+0000","last_change":"2023-07-14T18:14:22.327242+0000","last_active":"2023-07-14T18:15:49.958737+0000","last_peered":"2023-07-14T18:15:49.958737+0000","last_clean":"2023-07-14T18:15:49.958737+0000","last_became_active":"2023-07-14T18:14:22.326787+0000","last_became_peered":"2023-07-14T18:14:22.326787+0000","last_unstale":"2023-07-14T18:15:49.958737+0000","last_undegraded":"2023-07-14T18:15:49.958737+0000","last_fullsized":"2023-07-14T18:15:49.958737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:20:16.313522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.469654+0000","last_change":"2023-07-14T18:14:17.925035+0000","last_active":"2023-07-14T18:16:17.469654+0000","last_peered":"2023-07-14T18:16:17.469654+0000","last_clean":"2023-07-14T18:16:17.469654+0000","last_became_active":"2023-07-14T18:14:17.924713+0000","last_became_peered":"2023-07-14T18:14:17.924713+0000","last_unstale":"2023-07-14T18:16:17.469654+0000","last_undegraded":"2023-07-14T18:16:17.469654+0000","last_fullsized":"2023-07-14T18:16:17.469654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:32:01.731363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":132,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:26.035225+0000","last_change":"2023-07-14T18:14:16.935561+0000","last_active":"2023-07-14T18:16:26.035225+0000","last_peered":"2023-07-14T18:16:26.035225+0000","last_clean":"2023-07-14T18:16:26.035225+0000","last_became_active":"2023-07-14T18:14:16.935022+0000","last_became_peered":"2023-07-14T18:14:16.935022+0000","last_unstale":"2023-07-14T18:16:26.035225+0000","last_undegraded":"2023-07-14T18:16:26.035225+0000","last_fullsized":"2023-07-14T18:16:26.035225+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:46:30.440549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.989880+0000","last_change":"2023-07-14T18:15:49.989513+0000","last_active":"2023-07-14T18:15:49.989880+0000","last_peered":"2023-07-14T18:15:49.989880+0000","last_clean":"2023-07-14T18:15:49.989880+0000","last_became_active":"2023-07-14T18:15:49.985094+0000","last_became_peered":"2023-07-14T18:15:49.985094+0000","last_unstale":"2023-07-14T18:15:49.989880+0000","last_undegraded":"2023-07-14T18:15:49.989880+0000","last_fullsized":"2023-07-14T18:15:49.989880+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:12:15.915672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.476140+0000","last_change":"2023-07-14T18:14:17.930211+0000","last_active":"2023-07-14T18:16:17.476140+0000","last_peered":"2023-07-14T18:16:17.476140+0000","last_clean":"2023-07-14T18:16:17.476140+0000","last_became_active":"2023-07-14T18:14:17.929851+0000","last_became_peered":"2023-07-14T18:14:17.929851+0000","last_unstale":"2023-07-14T18:16:17.476140+0000","last_undegraded":"2023-07-14T18:16:17.476140+0000","last_fullsized":"2023-07-14T18:16:17.476140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:03:36.180205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958812+0000","last_change":"2023-07-14T18:14:16.928995+0000","last_active":"2023-07-14T18:15:49.958812+0000","last_peered":"2023-07-14T18:15:49.958812+0000","last_clean":"2023-07-14T18:15:49.958812+0000","last_became_active":"2023-07-14T18:14:16.928612+0000","last_became_peered":"2023-07-14T18:14:16.928612+0000","last_unstale":"2023-07-14T18:15:49.958812+0000","last_undegraded":"2023-07-14T18:15:49.958812+0000","last_fullsized":"2023-07-14T18:15:49.958812+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:01:02.502166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":345,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.962275+0000","last_change":"2023-07-14T18:13:11.614507+0000","last_active":"2023-07-14T18:15:49.962275+0000","last_peered":"2023-07-14T18:15:49.962275+0000","last_clean":"2023-07-14T18:15:49.962275+0000","last_became_active":"2023-07-14T18:13:11.614063+0000","last_became_peered":"2023-07-14T18:13:11.614063+0000","last_unstale":"2023-07-14T18:15:49.962275+0000","last_undegraded":"2023-07-14T18:15:49.962275+0000","last_fullsized":"2023-07-14T18:15:49.962275+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:10:03.995193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:10:03.995193+0000","last_clean_scrub_stamp":"2023-07-14T18:10:03.995193+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:00:53.238854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.959283+0000","last_change":"2023-07-14T18:14:49.542604+0000","last_active":"2023-07-14T18:15:49.959283+0000","last_peered":"2023-07-14T18:15:49.959283+0000","last_clean":"2023-07-14T18:15:49.959283+0000","last_became_active":"2023-07-14T18:14:49.542111+0000","last_became_peered":"2023-07-14T18:14:49.542111+0000","last_unstale":"2023-07-14T18:15:49.959283+0000","last_undegraded":"2023-07-14T18:15:49.959283+0000","last_fullsized":"2023-07-14T18:15:49.959283+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:25:58.915436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.479234+0000","last_change":"2023-07-14T18:14:49.543511+0000","last_active":"2023-07-14T18:16:17.479234+0000","last_peered":"2023-07-14T18:16:17.479234+0000","last_clean":"2023-07-14T18:16:17.479234+0000","last_became_active":"2023-07-14T18:14:49.542407+0000","last_became_peered":"2023-07-14T18:14:49.542407+0000","last_unstale":"2023-07-14T18:16:17.479234+0000","last_undegraded":"2023-07-14T18:16:17.479234+0000","last_fullsized":"2023-07-14T18:16:17.479234+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:18:32.901622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964257+0000","last_change":"2023-07-14T18:14:48.950022+0000","last_active":"2023-07-14T18:15:49.964257+0000","last_peered":"2023-07-14T18:15:49.964257+0000","last_clean":"2023-07-14T18:15:49.964257+0000","last_became_active":"2023-07-14T18:14:48.949149+0000","last_became_peered":"2023-07-14T18:14:48.949149+0000","last_unstale":"2023-07-14T18:15:49.964257+0000","last_undegraded":"2023-07-14T18:15:49.964257+0000","last_fullsized":"2023-07-14T18:15:49.964257+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:10:45.323165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964919+0000","last_change":"2023-07-14T18:14:22.038428+0000","last_active":"2023-07-14T18:15:49.964919+0000","last_peered":"2023-07-14T18:15:49.964919+0000","last_clean":"2023-07-14T18:15:49.964919+0000","last_became_active":"2023-07-14T18:14:22.038038+0000","last_became_peered":"2023-07-14T18:14:22.038038+0000","last_unstale":"2023-07-14T18:15:49.964919+0000","last_undegraded":"2023-07-14T18:15:49.964919+0000","last_fullsized":"2023-07-14T18:15:49.964919+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:16:27.576741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.959318+0000","last_change":"2023-07-14T18:14:16.933110+0000","last_active":"2023-07-14T18:15:49.959318+0000","last_peered":"2023-07-14T18:15:49.959318+0000","last_clean":"2023-07-14T18:15:49.959318+0000","last_became_active":"2023-07-14T18:14:16.932844+0000","last_became_peered":"2023-07-14T18:14:16.932844+0000","last_unstale":"2023-07-14T18:15:49.959318+0000","last_undegraded":"2023-07-14T18:15:49.959318+0000","last_fullsized":"2023-07-14T18:15:49.959318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:08:17.512405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.477926+0000","last_change":"2023-07-14T18:14:17.920168+0000","last_active":"2023-07-14T18:16:17.477926+0000","last_peered":"2023-07-14T18:16:17.477926+0000","last_clean":"2023-07-14T18:16:17.477926+0000","last_became_active":"2023-07-14T18:14:17.919576+0000","last_became_peered":"2023-07-14T18:14:17.919576+0000","last_unstale":"2023-07-14T18:16:17.477926+0000","last_undegraded":"2023-07-14T18:16:17.477926+0000","last_fullsized":"2023-07-14T18:16:17.477926+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:55:49.314127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093284+0000","last_change":"2023-07-14T18:14:22.327206+0000","last_active":"2023-07-14T18:15:50.093284+0000","last_peered":"2023-07-14T18:15:50.093284+0000","last_clean":"2023-07-14T18:15:50.093284+0000","last_became_active":"2023-07-14T18:14:22.326743+0000","last_became_peered":"2023-07-14T18:14:22.326743+0000","last_unstale":"2023-07-14T18:15:50.093284+0000","last_undegraded":"2023-07-14T18:15:50.093284+0000","last_fullsized":"2023-07-14T18:15:50.093284+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:50:43.439856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'8","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:25.919981+0000","last_change":"2023-07-14T18:14:16.928994+0000","last_active":"2023-07-14T18:16:25.919981+0000","last_peered":"2023-07-14T18:16:25.919981+0000","last_clean":"2023-07-14T18:16:25.919981+0000","last_became_active":"2023-07-14T18:14:16.928468+0000","last_became_peered":"2023-07-14T18:14:16.928468+0000","last_unstale":"2023-07-14T18:16:25.919981+0000","last_undegraded":"2023-07-14T18:16:25.919981+0000","last_fullsized":"2023-07-14T18:16:25.919981+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:31:02.519665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.487174+0000","last_change":"2023-07-14T18:14:17.919147+0000","last_active":"2023-07-14T18:16:17.487174+0000","last_peered":"2023-07-14T18:16:17.487174+0000","last_clean":"2023-07-14T18:16:17.487174+0000","last_became_active":"2023-07-14T18:14:17.918811+0000","last_became_peered":"2023-07-14T18:14:17.918811+0000","last_unstale":"2023-07-14T18:16:17.487174+0000","last_undegraded":"2023-07-14T18:16:17.487174+0000","last_fullsized":"2023-07-14T18:16:17.487174+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:48:51.557600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093309+0000","last_change":"2023-07-14T18:14:22.032632+0000","last_active":"2023-07-14T18:15:50.093309+0000","last_peered":"2023-07-14T18:15:50.093309+0000","last_clean":"2023-07-14T18:15:50.093309+0000","last_became_active":"2023-07-14T18:14:22.032233+0000","last_became_peered":"2023-07-14T18:14:22.032233+0000","last_unstale":"2023-07-14T18:15:50.093309+0000","last_undegraded":"2023-07-14T18:15:50.093309+0000","last_fullsized":"2023-07-14T18:15:50.093309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:10:59.598294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.959256+0000","last_change":"2023-07-14T18:14:16.924993+0000","last_active":"2023-07-14T18:15:49.959256+0000","last_peered":"2023-07-14T18:15:49.959256+0000","last_clean":"2023-07-14T18:15:49.959256+0000","last_became_active":"2023-07-14T18:14:16.924702+0000","last_became_peered":"2023-07-14T18:14:16.924702+0000","last_unstale":"2023-07-14T18:15:49.959256+0000","last_undegraded":"2023-07-14T18:15:49.959256+0000","last_fullsized":"2023-07-14T18:15:49.959256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:46:18.671640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.483683+0000","last_change":"2023-07-14T18:14:17.943397+0000","last_active":"2023-07-14T18:16:17.483683+0000","last_peered":"2023-07-14T18:16:17.483683+0000","last_clean":"2023-07-14T18:16:17.483683+0000","last_became_active":"2023-07-14T18:14:17.942114+0000","last_became_peered":"2023-07-14T18:14:17.942114+0000","last_unstale":"2023-07-14T18:16:17.483683+0000","last_undegraded":"2023-07-14T18:16:17.483683+0000","last_fullsized":"2023-07-14T18:16:17.483683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:59:53.996642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958856+0000","last_change":"2023-07-14T18:14:22.328273+0000","last_active":"2023-07-14T18:15:49.958856+0000","last_peered":"2023-07-14T18:15:49.958856+0000","last_clean":"2023-07-14T18:15:49.958856+0000","last_became_active":"2023-07-14T18:14:22.327843+0000","last_became_peered":"2023-07-14T18:14:22.327843+0000","last_unstale":"2023-07-14T18:15:49.958856+0000","last_undegraded":"2023-07-14T18:15:49.958856+0000","last_fullsized":"2023-07-14T18:15:49.958856+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:29:12.613515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.094321+0000","last_change":"2023-07-14T18:14:49.543525+0000","last_active":"2023-07-14T18:15:50.094321+0000","last_peered":"2023-07-14T18:15:50.094321+0000","last_clean":"2023-07-14T18:15:50.094321+0000","last_became_active":"2023-07-14T18:14:49.542962+0000","last_became_peered":"2023-07-14T18:14:49.542962+0000","last_unstale":"2023-07-14T18:15:50.094321+0000","last_undegraded":"2023-07-14T18:15:50.094321+0000","last_fullsized":"2023-07-14T18:15:50.094321+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:43:37.767679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.464347+0000","last_change":"2023-07-14T18:14:17.932181+0000","last_active":"2023-07-14T18:16:17.464347+0000","last_peered":"2023-07-14T18:16:17.464347+0000","last_clean":"2023-07-14T18:16:17.464347+0000","last_became_active":"2023-07-14T18:14:17.931811+0000","last_became_peered":"2023-07-14T18:14:17.931811+0000","last_unstale":"2023-07-14T18:16:17.464347+0000","last_undegraded":"2023-07-14T18:16:17.464347+0000","last_fullsized":"2023-07-14T18:16:17.464347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:59:38.610413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964687+0000","last_change":"2023-07-14T18:14:22.047281+0000","last_active":"2023-07-14T18:15:49.964687+0000","last_peered":"2023-07-14T18:15:49.964687+0000","last_clean":"2023-07-14T18:15:49.964687+0000","last_became_active":"2023-07-14T18:14:22.045531+0000","last_became_peered":"2023-07-14T18:14:22.045531+0000","last_unstale":"2023-07-14T18:15:49.964687+0000","last_undegraded":"2023-07-14T18:15:49.964687+0000","last_fullsized":"2023-07-14T18:15:49.964687+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:53:44.716384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093515+0000","last_change":"2023-07-14T18:14:16.926904+0000","last_active":"2023-07-14T18:15:50.093515+0000","last_peered":"2023-07-14T18:15:50.093515+0000","last_clean":"2023-07-14T18:15:50.093515+0000","last_became_active":"2023-07-14T18:14:16.926579+0000","last_became_peered":"2023-07-14T18:14:16.926579+0000","last_unstale":"2023-07-14T18:15:50.093515+0000","last_undegraded":"2023-07-14T18:15:50.093515+0000","last_fullsized":"2023-07-14T18:15:50.093515+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:55:21.011226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.478290+0000","last_change":"2023-07-14T18:14:17.926005+0000","last_active":"2023-07-14T18:16:17.478290+0000","last_peered":"2023-07-14T18:16:17.478290+0000","last_clean":"2023-07-14T18:16:17.478290+0000","last_became_active":"2023-07-14T18:14:17.925266+0000","last_became_peered":"2023-07-14T18:14:17.925266+0000","last_unstale":"2023-07-14T18:16:17.478290+0000","last_undegraded":"2023-07-14T18:16:17.478290+0000","last_fullsized":"2023-07-14T18:16:17.478290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:18:03.629066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958581+0000","last_change":"2023-07-14T18:14:22.043219+0000","last_active":"2023-07-14T18:15:49.958581+0000","last_peered":"2023-07-14T18:15:49.958581+0000","last_clean":"2023-07-14T18:15:49.958581+0000","last_became_active":"2023-07-14T18:14:22.042654+0000","last_became_peered":"2023-07-14T18:14:22.042654+0000","last_unstale":"2023-07-14T18:15:49.958581+0000","last_undegraded":"2023-07-14T18:15:49.958581+0000","last_fullsized":"2023-07-14T18:15:49.958581+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:20:43.065883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093962+0000","last_change":"2023-07-14T18:14:16.923130+0000","last_active":"2023-07-14T18:15:50.093962+0000","last_peered":"2023-07-14T18:15:50.093962+0000","last_clean":"2023-07-14T18:15:50.093962+0000","last_became_active":"2023-07-14T18:14:16.922835+0000","last_became_peered":"2023-07-14T18:14:16.922835+0000","last_unstale":"2023-07-14T18:15:50.093962+0000","last_undegraded":"2023-07-14T18:15:50.093962+0000","last_fullsized":"2023-07-14T18:15:50.093962+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:29:20.116131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.478830+0000","last_change":"2023-07-14T18:14:17.931024+0000","last_active":"2023-07-14T18:16:17.478830+0000","last_peered":"2023-07-14T18:16:17.478830+0000","last_clean":"2023-07-14T18:16:17.478830+0000","last_became_active":"2023-07-14T18:14:17.929529+0000","last_became_peered":"2023-07-14T18:14:17.929529+0000","last_unstale":"2023-07-14T18:16:17.478830+0000","last_undegraded":"2023-07-14T18:16:17.478830+0000","last_fullsized":"2023-07-14T18:16:17.478830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:06:29.694951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958757+0000","last_change":"2023-07-14T18:14:22.327313+0000","last_active":"2023-07-14T18:15:49.958757+0000","last_peered":"2023-07-14T18:15:49.958757+0000","last_clean":"2023-07-14T18:15:49.958757+0000","last_became_active":"2023-07-14T18:14:22.326922+0000","last_became_peered":"2023-07-14T18:14:22.326922+0000","last_unstale":"2023-07-14T18:15:49.958757+0000","last_undegraded":"2023-07-14T18:15:49.958757+0000","last_fullsized":"2023-07-14T18:15:49.958757+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:27:15.790634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958923+0000","last_change":"2023-07-14T18:14:16.926632+0000","last_active":"2023-07-14T18:15:49.958923+0000","last_peered":"2023-07-14T18:15:49.958923+0000","last_clean":"2023-07-14T18:15:49.958923+0000","last_became_active":"2023-07-14T18:14:16.926279+0000","last_became_peered":"2023-07-14T18:14:16.926279+0000","last_unstale":"2023-07-14T18:15:49.958923+0000","last_undegraded":"2023-07-14T18:15:49.958923+0000","last_fullsized":"2023-07-14T18:15:49.958923+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:55:17.792126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.487420+0000","last_change":"2023-07-14T18:14:17.924416+0000","last_active":"2023-07-14T18:16:17.487420+0000","last_peered":"2023-07-14T18:16:17.487420+0000","last_clean":"2023-07-14T18:16:17.487420+0000","last_became_active":"2023-07-14T18:14:17.924140+0000","last_became_peered":"2023-07-14T18:14:17.924140+0000","last_unstale":"2023-07-14T18:16:17.487420+0000","last_undegraded":"2023-07-14T18:16:17.487420+0000","last_fullsized":"2023-07-14T18:16:17.487420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:43:36.553540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.094104+0000","last_change":"2023-07-14T18:14:22.037127+0000","last_active":"2023-07-14T18:15:50.094104+0000","last_peered":"2023-07-14T18:15:50.094104+0000","last_clean":"2023-07-14T18:15:50.094104+0000","last_became_active":"2023-07-14T18:14:22.036647+0000","last_became_peered":"2023-07-14T18:14:22.036647+0000","last_unstale":"2023-07-14T18:15:50.094104+0000","last_undegraded":"2023-07-14T18:15:50.094104+0000","last_fullsized":"2023-07-14T18:15:50.094104+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:51:25.981240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.094215+0000","last_change":"2023-07-14T18:14:16.921494+0000","last_active":"2023-07-14T18:15:50.094215+0000","last_peered":"2023-07-14T18:15:50.094215+0000","last_clean":"2023-07-14T18:15:50.094215+0000","last_became_active":"2023-07-14T18:14:16.919622+0000","last_became_peered":"2023-07-14T18:14:16.919622+0000","last_unstale":"2023-07-14T18:15:50.094215+0000","last_undegraded":"2023-07-14T18:15:50.094215+0000","last_fullsized":"2023-07-14T18:15:50.094215+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:09:12.990342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.486524+0000","last_change":"2023-07-14T18:14:17.922266+0000","last_active":"2023-07-14T18:16:17.486524+0000","last_peered":"2023-07-14T18:16:17.486524+0000","last_clean":"2023-07-14T18:16:17.486524+0000","last_became_active":"2023-07-14T18:14:17.921461+0000","last_became_peered":"2023-07-14T18:14:17.921461+0000","last_unstale":"2023-07-14T18:16:17.486524+0000","last_undegraded":"2023-07-14T18:16:17.486524+0000","last_fullsized":"2023-07-14T18:16:17.486524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:04:46.997019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:50.093532+0000","last_change":"2023-07-14T18:14:22.044080+0000","last_active":"2023-07-14T18:15:50.093532+0000","last_peered":"2023-07-14T18:15:50.093532+0000","last_clean":"2023-07-14T18:15:50.093532+0000","last_became_active":"2023-07-14T18:14:22.043612+0000","last_became_peered":"2023-07-14T18:14:22.043612+0000","last_unstale":"2023-07-14T18:15:50.093532+0000","last_undegraded":"2023-07-14T18:15:50.093532+0000","last_fullsized":"2023-07-14T18:15:50.093532+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:33:05.435083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.965710+0000","last_change":"2023-07-14T18:14:16.931871+0000","last_active":"2023-07-14T18:15:49.965710+0000","last_peered":"2023-07-14T18:15:49.965710+0000","last_clean":"2023-07-14T18:15:49.965710+0000","last_became_active":"2023-07-14T18:14:16.931549+0000","last_became_peered":"2023-07-14T18:14:16.931549+0000","last_unstale":"2023-07-14T18:15:49.965710+0000","last_undegraded":"2023-07-14T18:15:49.965710+0000","last_fullsized":"2023-07-14T18:15:49.965710+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:41:43.337550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.483151+0000","last_change":"2023-07-14T18:14:17.929550+0000","last_active":"2023-07-14T18:16:17.483151+0000","last_peered":"2023-07-14T18:16:17.483151+0000","last_clean":"2023-07-14T18:16:17.483151+0000","last_became_active":"2023-07-14T18:14:17.929204+0000","last_became_peered":"2023-07-14T18:14:17.929204+0000","last_unstale":"2023-07-14T18:16:17.483151+0000","last_undegraded":"2023-07-14T18:16:17.483151+0000","last_fullsized":"2023-07-14T18:16:17.483151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:15:39.711388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.964416+0000","last_change":"2023-07-14T18:14:22.034899+0000","last_active":"2023-07-14T18:15:49.964416+0000","last_peered":"2023-07-14T18:15:49.964416+0000","last_clean":"2023-07-14T18:15:49.964416+0000","last_became_active":"2023-07-14T18:14:22.034524+0000","last_became_peered":"2023-07-14T18:14:22.034524+0000","last_unstale":"2023-07-14T18:15:49.964416+0000","last_undegraded":"2023-07-14T18:15:49.964416+0000","last_fullsized":"2023-07-14T18:15:49.964416+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:21.005237+0000","last_clean_scrub_stamp":"2023-07-14T18:14:21.005237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:33:24.357185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:15:49.958833+0000","last_change":"2023-07-14T18:14:16.935285+0000","last_active":"2023-07-14T18:15:49.958833+0000","last_peered":"2023-07-14T18:15:49.958833+0000","last_clean":"2023-07-14T18:15:49.958833+0000","last_became_active":"2023-07-14T18:14:16.934888+0000","last_became_peered":"2023-07-14T18:14:16.934888+0000","last_unstale":"2023-07-14T18:15:49.958833+0000","last_undegraded":"2023-07-14T18:15:49.958833+0000","last_fullsized":"2023-07-14T18:15:49.958833+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:15.898429+0000","last_clean_scrub_stamp":"2023-07-14T18:14:15.898429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:24:43.441076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.485394+0000","last_change":"2023-07-14T18:14:17.930982+0000","last_active":"2023-07-14T18:16:17.485394+0000","last_peered":"2023-07-14T18:16:17.485394+0000","last_clean":"2023-07-14T18:16:17.485394+0000","last_became_active":"2023-07-14T18:14:17.930661+0000","last_became_peered":"2023-07-14T18:14:17.930661+0000","last_unstale":"2023-07-14T18:16:17.485394+0000","last_undegraded":"2023-07-14T18:16:17.485394+0000","last_fullsized":"2023-07-14T18:16:17.485394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:02:02.738482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.472552+0000","last_change":"2023-07-14T18:14:17.926204+0000","last_active":"2023-07-14T18:16:17.472552+0000","last_peered":"2023-07-14T18:16:17.472552+0000","last_clean":"2023-07-14T18:16:17.472552+0000","last_became_active":"2023-07-14T18:14:17.925689+0000","last_became_peered":"2023-07-14T18:14:17.925689+0000","last_unstale":"2023-07-14T18:16:17.472552+0000","last_undegraded":"2023-07-14T18:16:17.472552+0000","last_fullsized":"2023-07-14T18:16:17.472552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:24:10.100519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.476582+0000","last_change":"2023-07-14T18:14:17.923146+0000","last_active":"2023-07-14T18:16:17.476582+0000","last_peered":"2023-07-14T18:16:17.476582+0000","last_clean":"2023-07-14T18:16:17.476582+0000","last_became_active":"2023-07-14T18:14:17.922734+0000","last_became_peered":"2023-07-14T18:14:17.922734+0000","last_unstale":"2023-07-14T18:16:17.476582+0000","last_undegraded":"2023-07-14T18:16:17.476582+0000","last_fullsized":"2023-07-14T18:16:17.476582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:36:48.282746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.483489+0000","last_change":"2023-07-14T18:14:17.927397+0000","last_active":"2023-07-14T18:16:17.483489+0000","last_peered":"2023-07-14T18:16:17.483489+0000","last_clean":"2023-07-14T18:16:17.483489+0000","last_became_active":"2023-07-14T18:14:17.927000+0000","last_became_peered":"2023-07-14T18:14:17.927000+0000","last_unstale":"2023-07-14T18:16:17.483489+0000","last_undegraded":"2023-07-14T18:16:17.483489+0000","last_fullsized":"2023-07-14T18:16:17.483489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:55:36.682437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.482209+0000","last_change":"2023-07-14T18:14:17.935266+0000","last_active":"2023-07-14T18:16:17.482209+0000","last_peered":"2023-07-14T18:16:17.482209+0000","last_clean":"2023-07-14T18:16:17.482209+0000","last_became_active":"2023-07-14T18:14:17.934944+0000","last_became_peered":"2023-07-14T18:14:17.934944+0000","last_unstale":"2023-07-14T18:16:17.482209+0000","last_undegraded":"2023-07-14T18:16:17.482209+0000","last_fullsized":"2023-07-14T18:16:17.482209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:18:53.622505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.483800+0000","last_change":"2023-07-14T18:14:49.542810+0000","last_active":"2023-07-14T18:16:17.483800+0000","last_peered":"2023-07-14T18:16:17.483800+0000","last_clean":"2023-07-14T18:16:17.483800+0000","last_became_active":"2023-07-14T18:14:49.542421+0000","last_became_peered":"2023-07-14T18:14:49.542421+0000","last_unstale":"2023-07-14T18:16:17.483800+0000","last_undegraded":"2023-07-14T18:16:17.483800+0000","last_fullsized":"2023-07-14T18:16:17.483800+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:27:08.747661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.466271+0000","last_change":"2023-07-14T18:14:17.930757+0000","last_active":"2023-07-14T18:16:17.466271+0000","last_peered":"2023-07-14T18:16:17.466271+0000","last_clean":"2023-07-14T18:16:17.466271+0000","last_became_active":"2023-07-14T18:14:17.930302+0000","last_became_peered":"2023-07-14T18:14:17.930302+0000","last_unstale":"2023-07-14T18:16:17.466271+0000","last_undegraded":"2023-07-14T18:16:17.466271+0000","last_fullsized":"2023-07-14T18:16:17.466271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:00:25.569289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.475262+0000","last_change":"2023-07-14T18:14:17.930337+0000","last_active":"2023-07-14T18:16:17.475262+0000","last_peered":"2023-07-14T18:16:17.475262+0000","last_clean":"2023-07-14T18:16:17.475262+0000","last_became_active":"2023-07-14T18:14:17.930037+0000","last_became_peered":"2023-07-14T18:14:17.930037+0000","last_unstale":"2023-07-14T18:16:17.475262+0000","last_undegraded":"2023-07-14T18:16:17.475262+0000","last_fullsized":"2023-07-14T18:16:17.475262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:50:04.589039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.478977+0000","last_change":"2023-07-14T18:14:17.912750+0000","last_active":"2023-07-14T18:16:17.478977+0000","last_peered":"2023-07-14T18:16:17.478977+0000","last_clean":"2023-07-14T18:16:17.478977+0000","last_became_active":"2023-07-14T18:14:17.912402+0000","last_became_peered":"2023-07-14T18:14:17.912402+0000","last_unstale":"2023-07-14T18:16:17.478977+0000","last_undegraded":"2023-07-14T18:16:17.478977+0000","last_fullsized":"2023-07-14T18:16:17.478977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:29:09.484261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.471891+0000","last_change":"2023-07-14T18:14:17.930004+0000","last_active":"2023-07-14T18:16:17.471891+0000","last_peered":"2023-07-14T18:16:17.471891+0000","last_clean":"2023-07-14T18:16:17.471891+0000","last_became_active":"2023-07-14T18:14:17.929719+0000","last_became_peered":"2023-07-14T18:14:17.929719+0000","last_unstale":"2023-07-14T18:16:17.471891+0000","last_undegraded":"2023-07-14T18:16:17.471891+0000","last_fullsized":"2023-07-14T18:16:17.471891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:51:52.223909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.475685+0000","last_change":"2023-07-14T18:14:17.919044+0000","last_active":"2023-07-14T18:16:17.475685+0000","last_peered":"2023-07-14T18:16:17.475685+0000","last_clean":"2023-07-14T18:16:17.475685+0000","last_became_active":"2023-07-14T18:14:17.918079+0000","last_became_peered":"2023-07-14T18:14:17.918079+0000","last_unstale":"2023-07-14T18:16:17.475685+0000","last_undegraded":"2023-07-14T18:16:17.475685+0000","last_fullsized":"2023-07-14T18:16:17.475685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:44:33.328875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.488913+0000","last_change":"2023-07-14T18:14:17.921560+0000","last_active":"2023-07-14T18:16:17.488913+0000","last_peered":"2023-07-14T18:16:17.488913+0000","last_clean":"2023-07-14T18:16:17.488913+0000","last_became_active":"2023-07-14T18:14:17.921243+0000","last_became_peered":"2023-07-14T18:14:17.921243+0000","last_unstale":"2023-07-14T18:16:17.488913+0000","last_undegraded":"2023-07-14T18:16:17.488913+0000","last_fullsized":"2023-07-14T18:16:17.488913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:09:32.651260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.487411+0000","last_change":"2023-07-14T18:14:17.927504+0000","last_active":"2023-07-14T18:16:17.487411+0000","last_peered":"2023-07-14T18:16:17.487411+0000","last_clean":"2023-07-14T18:16:17.487411+0000","last_became_active":"2023-07-14T18:14:17.927233+0000","last_became_peered":"2023-07-14T18:14:17.927233+0000","last_unstale":"2023-07-14T18:16:17.487411+0000","last_undegraded":"2023-07-14T18:16:17.487411+0000","last_fullsized":"2023-07-14T18:16:17.487411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:59:31.666727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-07-14T18:16:17.478342+0000","last_change":"2023-07-14T18:14:17.932317+0000","last_active":"2023-07-14T18:16:17.478342+0000","last_peered":"2023-07-14T18:16:17.478342+0000","last_clean":"2023-07-14T18:16:17.478342+0000","last_became_active":"2023-07-14T18:14:17.931973+0000","last_became_peered":"2023-07-14T18:14:17.931973+0000","last_unstale":"2023-07-14T18:16:17.478342+0000","last_undegraded":"2023-07-14T18:16:17.478342+0000","last_fullsized":"2023-07-14T18:16:17.478342+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T18:14:16.902799+0000","last_clean_scrub_stamp":"2023-07-14T18:14:16.902799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:12:59.198777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281026,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70113,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":620,"num_read_kb":384,"num_write":169,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":210339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416423,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":511749,"data_compressed":23584,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.94199999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.53000000000000003}]},{"osd":1,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63900000000000001,"15min":0.63800000000000001},"min":{"1min":0.44500000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.41899999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77700000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.66600000000000004}]},{"osd":2,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72299999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.502,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72699999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.67600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":4,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68600000000000005,"15min":0.68000000000000005},"min":{"1min":0.48099999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89900000000000002,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.66400000000000003,"15min":0.67000000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86799999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.626}]},{"osd":5,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.61099999999999999,"15min":0.625},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68300000000000005,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.38700000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96399999999999997,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.81299999999999994}]},{"osd":6,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.749,"15min":0.75800000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.95999999999999996,"5min":1.351,"15min":1.351},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.79000000000000004,"15min":0.79400000000000004},"min":{"1min":0.48299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":3.2410000000000001,"5min":3.2410000000000001,"15min":3.2410000000000001},"last":0.77700000000000002}]},{"osd":7,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77800000000000002,"15min":0.77900000000000003},"min":{"1min":0.58099999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.026,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.74399999999999999,"15min":0.749},"min":{"1min":0.51600000000000001,"5min":0.505,"15min":0.505},"max":{"1min":0.93100000000000005,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.79800000000000004}]},{"osd":8,"last update":"Fri Jul 14 18:16:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69699999999999995,"15min":0.70599999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.78100000000000003,"5min":0.878,"15min":0.878},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.78800000000000003,"15min":0.79400000000000004},"min":{"1min":0.55400000000000005,"5min":0.504,"15min":0.504},"max":{"1min":0.95299999999999996,"5min":1,"15min":1},"last":0.89700000000000002}]},{"osd":9,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80200000000000005,"15min":0.80600000000000005},"min":{"1min":0.58099999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1279999999999999,"5min":1.635,"15min":1.635},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.84899999999999998,"15min":0.85099999999999998},"min":{"1min":0.59899999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.05,"5min":1.147,"15min":1.147},"last":0.95499999999999996}]},{"osd":10,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80500000000000005,"15min":0.80900000000000005},"min":{"1min":0.57599999999999996,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.97299999999999998,"5min":1.272,"15min":1.272},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.83999999999999997,"15min":0.84999999999999998},"min":{"1min":0.59699999999999998,"5min":0.501,"15min":0.501},"max":{"1min":0.97099999999999997,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.878}]}]},{"osd":10,"up_from":72,"seq":309237645355,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1155072,"data_stored":1168486,"data_compressed":59328,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61499999999999999,"15min":0.623},"min":{"1min":0.44400000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.76100000000000001,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.68000000000000005,"15min":0.68799999999999994},"min":{"1min":0.48499999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.83499999999999996,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.53200000000000003}]},{"osd":1,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.441,"15min":0.44400000000000001},"min":{"1min":0.32600000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61599999999999999,"5min":0.626,"15min":0.626},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47799999999999998,"15min":0.48399999999999999},"min":{"1min":0.29999999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.66900000000000004,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.47699999999999998}]},{"osd":2,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.38600000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.78400000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.42299999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77500000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.71199999999999997}]},{"osd":3,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63900000000000001,"15min":0.63500000000000001},"min":{"1min":0.439,"5min":0.38,"15min":0.38},"max":{"1min":1.4179999999999999,"5min":3.4129999999999998,"15min":3.4129999999999998},"last":0.504},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.70399999999999996,"15min":0.68100000000000005},"min":{"1min":0.45400000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4870000000000001,"5min":5.9939999999999998,"15min":5.9939999999999998},"last":0.58299999999999996}]},{"osd":4,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55200000000000005,"15min":0.56399999999999995},"min":{"1min":0.40500000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68400000000000005,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.68799999999999994,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.62}]},{"osd":5,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.66700000000000004,"15min":0.65800000000000003},"min":{"1min":0.46700000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81000000000000005,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.746},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.47299999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.55400000000000005}]},{"osd":6,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70999999999999996,"15min":0.72399999999999998},"min":{"1min":0.48399999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.747,"15min":0.76500000000000001},"min":{"1min":0.504,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.69299999999999995}]},{"osd":7,"last update":"Fri Jul 14 18:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.42199999999999999}]},{"osd":8,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.73599999999999999,"15min":0.745},"min":{"1min":0.54400000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.84899999999999998,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.75,"15min":0.75800000000000001},"min":{"1min":0.57999999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0149999999999999,"5min":1.026,"15min":1.026},"last":0.79500000000000004}]},{"osd":9,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.81100000000000005,"15min":0.81599999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.3879999999999999,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.80800000000000005,"15min":0.80600000000000005},"min":{"1min":0.60499999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":0.82499999999999996}]},{"osd":11,"last update":"Fri Jul 14 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.752,"15min":0.751},"min":{"1min":0.61499999999999999,"5min":0.499,"15min":0.499},"max":{"1min":0.89300000000000002,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.76500000000000001,"15min":0.76200000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.96699999999999997,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.80800000000000005}]}]},{"osd":9,"up_from":67,"seq":287762808880,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":511985,"data_compressed":23584,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.51100000000000001,"15min":0.53600000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.58699999999999997,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54800000000000004,"15min":0.56799999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Jul 14 18:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64800000000000002,"15min":0.65800000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.90600000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.69899999999999995,"15min":0.72799999999999998},"min":{"1min":0.45000000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88200000000000001,"5min":1.085,"15min":1.085},"last":0.70099999999999996}]},{"osd":2,"last update":"Fri Jul 14 18:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.68700000000000006,"15min":0.69399999999999995},"min":{"1min":0.44800000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91800000000000004,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.73199999999999998,"15min":0.754},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93799999999999994,"5min":1.147,"15min":1.147},"last":0.57399999999999995}]},{"osd":3,"last update":"Fri Jul 14 18:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.63700000000000001,"15min":0.66900000000000004},"min":{"1min":0.40100000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.63500000000000001,"15min":0.66300000000000003},"min":{"1min":0.39400000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.45500000000000002}]},{"osd":4,"last update":"Fri Jul 14 18:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.745,"15min":0.78100000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.878,"5min":1.198,"15min":1.198},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.751,"15min":0.78100000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.99199999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.71599999999999997}]},{"osd":5,"last update":"Fri Jul 14 18:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.77700000000000002,"15min":0.80600000000000005},"min":{"1min":0.53600000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.97199999999999998,"5min":1.224,"15min":1.224},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.73299999999999998,"15min":0.752},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95399999999999996,"5min":1.113,"15min":1.113},"last":0.64400000000000002}]},{"osd":6,"last update":"Fri Jul 14 18:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.75900000000000001}]},{"osd":7,"last update":"Fri Jul 14 18:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.82299999999999995,"15min":0.871},"min":{"1min":0.55800000000000005,"5min":0.503,"15min":0.503},"max":{"1min":0.92300000000000004,"5min":1.323,"15min":1.323},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.80800000000000005,"15min":0.85499999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.88100000000000001,"5min":1.411,"15min":1.411},"last":0.77700000000000002}]},{"osd":8,"last update":"Fri Jul 14 18:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.86599999999999999,"15min":0.91000000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.98799999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.872,"15min":0.91100000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.01,"5min":1.371,"15min":1.371},"last":0.88900000000000001}]},{"osd":10,"last update":"Fri Jul 14 18:16:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.83299999999999996,"15min":0.86899999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0409999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.84199999999999997,"15min":0.86599999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0489999999999999,"5min":1.095,"15min":1.095},"last":0.79300000000000004}]},{"osd":11,"last update":"Fri Jul 14 18:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.80600000000000005,"15min":0.80200000000000005},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.79600000000000004,"15min":0.80000000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.97899999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.83899999999999997}]}]},{"osd":8,"up_from":59,"seq":253403070517,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":511323,"data_compressed":23584,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77000000000000002,"5min":1.238,"15min":1.238},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.622,"15min":0.57999999999999996},"min":{"1min":0.46700000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96199999999999997,"5min":2.5950000000000002,"15min":2.5950000000000002},"last":0.62}]},{"osd":1,"last update":"Fri Jul 14 18:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.60499999999999998,"15min":0.57299999999999995},"min":{"1min":0.47999999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.63,"15min":0.59599999999999997},"min":{"1min":0.56100000000000005,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89800000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.77900000000000003}]},{"osd":2,"last update":"Fri Jul 14 18:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.48299999999999998,"15min":0.47499999999999998},"min":{"1min":0.42499999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.72699999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.49199999999999999,"15min":0.47199999999999998},"min":{"1min":0.42899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.79900000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.75600000000000001}]},{"osd":3,"last update":"Fri Jul 14 18:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.74399999999999999,"15min":0.66200000000000003},"min":{"1min":0.53600000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":3.8090000000000002,"5min":5.9749999999999996,"15min":5.9749999999999996},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.71099999999999997,"15min":0.63100000000000001},"min":{"1min":0.44,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":5.0810000000000004,"5min":5.0810000000000004,"15min":5.0810000000000004},"last":0.67100000000000004}]},{"osd":4,"last update":"Fri Jul 14 18:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.65400000000000003,"15min":0.63200000000000001},"min":{"1min":0.439,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.68700000000000006,"15min":0.66400000000000003},"min":{"1min":0.61699999999999999,"5min":0.434,"15min":0.434},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.92400000000000004}]},{"osd":5,"last update":"Fri Jul 14 18:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.59899999999999998,"15min":0.57099999999999995},"min":{"1min":0.40000000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.86799999999999999,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.57999999999999996,"15min":0.55400000000000005},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.88200000000000001}]},{"osd":6,"last update":"Fri Jul 14 18:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.75800000000000001,"15min":0.73399999999999999},"min":{"1min":0.51000000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.90400000000000003,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.82199999999999995,"15min":0.74399999999999999},"min":{"1min":0.65600000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":3.6339999999999999,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":0.90800000000000003}]},{"osd":7,"last update":"Fri Jul 14 18:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.75900000000000001,"15min":0.73699999999999999},"min":{"1min":0.73999999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.73799999999999999,"15min":0.71099999999999997},"min":{"1min":0.59799999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":1.042}]},{"osd":9,"last update":"Fri Jul 14 18:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":0.88900000000000001,"15min":0.82099999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":5.9370000000000003,"5min":5.9370000000000003,"15min":5.9370000000000003},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.2529999999999999,"5min":0.88,"15min":0.82399999999999995},"min":{"1min":0.57299999999999995,"5min":0.502,"15min":0.502},"max":{"1min":5.9039999999999999,"5min":5.9039999999999999,"15min":5.9039999999999999},"last":1.016}]},{"osd":10,"last update":"Fri Jul 14 18:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.79100000000000004,"15min":0.77000000000000002},"min":{"1min":0.57199999999999995,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":1.03},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.79400000000000004,"15min":0.77900000000000003},"min":{"1min":0.58399999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":1.0649999999999999}]},{"osd":11,"last update":"Fri Jul 14 18:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.79600000000000004,"15min":0.79100000000000004},"min":{"1min":0.57199999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.81899999999999995,"15min":0.81200000000000006},"min":{"1min":0.60399999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.97699999999999998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.995}]}]},{"osd":7,"up_from":51,"seq":219043332152,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":1108174,"data_compressed":59328,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.755,"15min":0.77300000000000002},"min":{"1min":0.38200000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.024,"5min":5.843,"15min":5.843},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.51100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.81299999999999994,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.40899999999999997}]},{"osd":1,"last update":"Fri Jul 14 18:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.42999999999999999,"15min":0.42799999999999999},"min":{"1min":0.32500000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.62,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46500000000000002,"15min":0.46200000000000002},"min":{"1min":0.35599999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.62,"5min":0.752,"15min":0.752},"last":0.51500000000000001}]},{"osd":2,"last update":"Fri Jul 14 18:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.64100000000000001,"15min":0.60399999999999998},"min":{"1min":0.42999999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.92800000000000005,"5min":3.4889999999999999,"15min":3.4889999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.45200000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.84999999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.746}]},{"osd":3,"last update":"Fri Jul 14 18:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.504,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.95099999999999996,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.434,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.78800000000000003,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.57199999999999995}]},{"osd":4,"last update":"Fri Jul 14 18:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.51700000000000002,"15min":0.52900000000000003},"min":{"1min":0.379,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72199999999999998,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.5,"15min":0.498},"min":{"1min":0.379,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.58699999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.54300000000000004}]},{"osd":5,"last update":"Fri Jul 14 18:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.63700000000000001,"15min":0.62},"min":{"1min":0.52300000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.873,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68899999999999995,"15min":0.68600000000000005},"min":{"1min":0.55900000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86199999999999999,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.78500000000000003}]},{"osd":6,"last update":"Fri Jul 14 18:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.92700000000000005,"15min":1.1140000000000001},"min":{"1min":0.47899999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.129,"5min":13.189,"15min":13.189},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.48499999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.92400000000000004,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.68400000000000005}]},{"osd":8,"last update":"Fri Jul 14 18:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.71599999999999997,"15min":0.69799999999999995},"min":{"1min":0.59899999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86899999999999999,"5min":1.04,"15min":1.04},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69799999999999995,"15min":0.69199999999999995},"min":{"1min":0.54700000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85799999999999998,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.72699999999999998}]},{"osd":9,"last update":"Fri Jul 14 18:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.89700000000000002,"15min":0.95899999999999996},"min":{"1min":0.56100000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.96999999999999997,"5min":5.8250000000000002,"15min":5.8250000000000002},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77400000000000002,"15min":0.77100000000000002},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.86799999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.80900000000000005}]},{"osd":10,"last update":"Fri Jul 14 18:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71799999999999997,"15min":0.71699999999999997},"min":{"1min":0.48599999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1,"5min":1,"15min":1},"last":0.629},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76900000000000002,"15min":0.77000000000000002},"min":{"1min":0.59499999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.98499999999999999,"5min":0.995,"15min":0.995},"last":0.85799999999999998}]},{"osd":11,"last update":"Fri Jul 14 18:16:19 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.85499999999999998,"15min":0.85399999999999998},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.875,"15min":0.876},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.88100000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":507161,"data_compressed":23584,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46800000000000003,"15min":0.45600000000000002},"min":{"1min":0.316,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.51600000000000001,"15min":0.48999999999999999},"min":{"1min":0.41499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.39900000000000002}]},{"osd":1,"last update":"Fri Jul 14 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.66600000000000004,"15min":0.64600000000000002},"min":{"1min":0.46700000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.69499999999999995,"15min":0.68000000000000005},"min":{"1min":0.48199999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.76000000000000001}]},{"osd":2,"last update":"Fri Jul 14 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.65500000000000003,"15min":0.63200000000000001},"min":{"1min":0.50900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93799999999999994,"5min":1.087,"15min":1.087},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69399999999999995,"15min":0.68100000000000005},"min":{"1min":0.45500000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.629}]},{"osd":3,"last update":"Fri Jul 14 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.63700000000000001,"15min":0.61699999999999999},"min":{"1min":0.41999999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.504},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.65100000000000002,"15min":0.65200000000000002},"min":{"1min":0.433,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.64900000000000002}]},{"osd":4,"last update":"Fri Jul 14 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.66300000000000003,"15min":0.64900000000000002},"min":{"1min":0.52700000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.72999999999999998,"15min":0.72499999999999998},"min":{"1min":0.57299999999999995,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.79600000000000004}]},{"osd":5,"last update":"Fri Jul 14 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.76200000000000001,"15min":0.76500000000000001},"min":{"1min":0.54700000000000004,"5min":0.435,"15min":0.435},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.75900000000000001,"15min":0.73099999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":4.1449999999999996,"5min":4.1449999999999996,"15min":4.1449999999999996},"last":0.85299999999999998}]},{"osd":7,"last update":"Fri Jul 14 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.78500000000000003,"15min":0.74299999999999999},"min":{"1min":0.58899999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":4.2750000000000004,"5min":4.2750000000000004,"15min":4.2750000000000004},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.747,"15min":0.72699999999999998},"min":{"1min":0.51300000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0860000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.93100000000000005}]},{"osd":8,"last update":"Fri Jul 14 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.79300000000000004,"15min":0.80400000000000005},"min":{"1min":0.54800000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.77300000000000002,"15min":0.75900000000000001},"min":{"1min":0.623,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0980000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.69199999999999995}]},{"osd":9,"last update":"Fri Jul 14 18:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.79300000000000004,"15min":0.81499999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.996,"5min":1.147,"15min":1.147},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83099999999999996,"15min":0.84499999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.133,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.748}]},{"osd":10,"last update":"Fri Jul 14 18:16:13 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86499999999999999,"15min":0.86899999999999999},"min":{"1min":0.67900000000000005,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.028,"5min":0.89000000000000001,"15min":0.86599999999999999},"min":{"1min":0.64200000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":4.1909999999999998,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.97099999999999997}]},{"osd":11,"last update":"Fri Jul 14 18:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.90600000000000003,"15min":0.91200000000000003},"min":{"1min":0.72499999999999998,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.1180000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.84899999999999998,"15min":0.84799999999999998},"min":{"1min":0.73799999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.77300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":573405,"data_compressed":23584,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56999999999999995,"15min":0.52000000000000002},"min":{"1min":0.34100000000000003,"5min":0.30599999999999999,"15min":0.28199999999999997},"max":{"1min":1.3129999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":1.151,"5min":0.70399999999999996,"15min":0.57399999999999995},"min":{"1min":0.44800000000000001,"5min":0.32100000000000001,"15min":0.30299999999999999},"max":{"1min":7.226,"5min":7.226,"15min":7.226},"last":0.36199999999999999}]},{"osd":2,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.626,"15min":0.55400000000000005},"min":{"1min":0.47899999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59999999999999998,"15min":0.53500000000000003},"min":{"1min":0.48999999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84199999999999997,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.752}]},{"osd":3,"last update":"Fri Jul 14 18:15:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.59299999999999997,"15min":0.53300000000000003},"min":{"1min":0.42799999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.125,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.65600000000000003,"15min":0.59999999999999998},"min":{"1min":0.501,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.5009999999999999,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":0.52900000000000003}]},{"osd":4,"last update":"Fri Jul 14 18:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.51000000000000001,"15min":0.46400000000000002},"min":{"1min":0.34599999999999997,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.91100000000000003,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.503,"15min":0.45900000000000002},"min":{"1min":0.36699999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.42699999999999999}]},{"osd":5,"last update":"Fri Jul 14 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.65700000000000003,"15min":0.61399999999999999},"min":{"1min":0.51600000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.65700000000000003,"15min":0.61599999999999999},"min":{"1min":0.61599999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.66300000000000003}]},{"osd":6,"last update":"Fri Jul 14 18:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.68600000000000005,"15min":0.67400000000000004},"min":{"1min":0.443,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.92500000000000004,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68000000000000005,"15min":0.66900000000000004},"min":{"1min":0.51800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.91400000000000003,"5min":1.23,"15min":1.23},"last":0.64700000000000002}]},{"osd":7,"last update":"Fri Jul 14 18:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.59999999999999998,"15min":0.56299999999999994},"min":{"1min":0.438,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.98299999999999998,"5min":1.04,"15min":1.04},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.628,"15min":0.58999999999999997},"min":{"1min":0.45900000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.93799999999999994,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.58199999999999996}]},{"osd":8,"last update":"Fri Jul 14 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.69099999999999995,"15min":0.65100000000000002},"min":{"1min":0.53900000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.008,"5min":1.071,"15min":1.071},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.75,"15min":0.69799999999999995},"min":{"1min":0.68100000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.03,"5min":1.052,"15min":1.052},"last":0.73399999999999999}]},{"osd":9,"last update":"Fri Jul 14 18:15:56 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.81899999999999995,"15min":0.78000000000000003},"min":{"1min":0.63300000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":4.0750000000000002,"5min":4.0750000000000002,"15min":4.0750000000000002},"last":0.69799999999999995},{"interface":"front","average":{"1min":1.054,"5min":0.874,"15min":0.83899999999999997},"min":{"1min":0.69299999999999995,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":4.0369999999999999,"5min":4.0369999999999999,"15min":4.0369999999999999},"last":0.68100000000000005}]},{"osd":10,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.81100000000000005,"15min":0.80500000000000005},"min":{"1min":0.57699999999999996,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.052,"5min":1.167,"15min":1.167},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76800000000000002,"15min":0.75700000000000001},"min":{"1min":0.55900000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0389999999999999,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.71899999999999997}]},{"osd":11,"last update":"Fri Jul 14 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.92000000000000004,"15min":0.92800000000000005},"min":{"1min":0.74099999999999999,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.097,"5min":1.169,"15min":1.169},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.86599999999999999,"15min":0.86799999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.0820000000000001,"5min":1.117,"15min":1.117},"last":0.78500000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":508549,"data_compressed":23584,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 18:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60799999999999998,"15min":0.55500000000000005},"min":{"1min":0.39400000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89800000000000002,"5min":1.698,"15min":1.698},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.61399999999999999,"15min":0.58199999999999996},"min":{"1min":0.40899999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.89600000000000002,"5min":1.179,"15min":1.179},"last":0.63800000000000001}]},{"osd":2,"last update":"Fri Jul 14 18:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.65600000000000003,"15min":0.57499999999999996},"min":{"1min":0.53400000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86799999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68200000000000005,"15min":0.59599999999999997},"min":{"1min":0.499,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89800000000000002,"5min":1.258,"15min":1.258},"last":0.89900000000000002}]},{"osd":3,"last update":"Fri Jul 14 18:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.55900000000000005,"15min":0.56699999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77400000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.64000000000000001,"15min":0.64600000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.85199999999999998}]},{"osd":4,"last update":"Fri Jul 14 18:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.61399999999999999,"15min":0.56899999999999995},"min":{"1min":0.441,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.80600000000000005,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66900000000000004,"15min":0.627},"min":{"1min":0.44,"5min":0.379,"15min":0.379},"max":{"1min":0.83699999999999997,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.58299999999999996}]},{"osd":5,"last update":"Fri Jul 14 18:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.72999999999999998,"15min":0.68899999999999995},"min":{"1min":0.45800000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90100000000000002,"5min":3.3740000000000001,"15min":3.3740000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.71499999999999997,"15min":0.68700000000000006},"min":{"1min":0.48099999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.97899999999999998,"5min":1.228,"15min":1.228},"last":0.97799999999999998}]},{"osd":6,"last update":"Fri Jul 14 18:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.64000000000000001,"15min":0.61599999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93600000000000005,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.65700000000000003,"15min":0.63700000000000001},"min":{"1min":0.41799999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88600000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.751}]},{"osd":7,"last update":"Fri Jul 14 18:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.70199999999999996,"15min":0.70999999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.82299999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.97899999999999998,"5min":1.006,"15min":1.006},"last":0.70999999999999996}]},{"osd":8,"last update":"Fri Jul 14 18:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.80900000000000005,"15min":0.79600000000000004},"min":{"1min":0.60599999999999998,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.028,"5min":1.839,"15min":1.839},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.79700000000000004,"15min":0.77000000000000002},"min":{"1min":0.57599999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0029999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.94499999999999995}]},{"osd":9,"last update":"Fri Jul 14 18:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.79600000000000004,"5min":1.0669999999999999,"15min":1.1850000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1259999999999999,"5min":7.8689999999999998,"15min":7.8689999999999998},"last":0.91000000000000003}]},{"osd":10,"last update":"Fri Jul 14 18:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84999999999999998,"15min":0.85899999999999999},"min":{"1min":0.62,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82199999999999995,"15min":0.82599999999999996},"min":{"1min":0.69199999999999995,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0569999999999999,"5min":1.081,"15min":1.081},"last":0.872}]},{"osd":11,"last update":"Fri Jul 14 18:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.90300000000000002,"15min":0.90700000000000003},"min":{"1min":0.65300000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.093,"5min":1.869,"15min":1.869},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.92900000000000005,"15min":0.93500000000000005},"min":{"1min":0.74399999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.0629999999999999,"5min":1.758,"15min":1.758},"last":1.044}]}]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":573551,"data_compressed":23584,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.629,"15min":0.55700000000000005},"min":{"1min":0.40200000000000002,"5min":0.313,"15min":0.28899999999999998},"max":{"1min":6.5220000000000002,"5min":6.5220000000000002,"15min":6.5220000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.67400000000000004,"15min":0.56399999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":8.984,"5min":8.984,"15min":8.984},"last":0.61399999999999999}]},{"osd":1,"last update":"Fri Jul 14 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.627,"15min":0.53700000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.33400000000000002},"max":{"1min":0.94599999999999995,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.61599999999999999,"15min":0.53400000000000003},"min":{"1min":0.441,"5min":0.36899999999999999,"15min":0.32000000000000001},"max":{"1min":0.89600000000000002,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.66600000000000004}]},{"osd":3,"last update":"Fri Jul 14 18:15:40 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":0.71899999999999997,"15min":0.60999999999999999},"min":{"1min":0.47599999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":9.1110000000000007,"5min":9.1110000000000007,"15min":9.1110000000000007},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.64900000000000002,"15min":0.65700000000000003},"min":{"1min":0.48199999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88100000000000001,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":0.65000000000000002}]},{"osd":4,"last update":"Fri Jul 14 18:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.72899999999999998,"15min":0.74199999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84699999999999998,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.73899999999999999,"15min":0.751},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.89900000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68999999999999995}]},{"osd":5,"last update":"Fri Jul 14 18:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.47799999999999998,"15min":0.46000000000000002},"min":{"1min":0.33200000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.68100000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52300000000000002,"15min":0.49099999999999999},"min":{"1min":0.33100000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.752,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.49099999999999999}]},{"osd":6,"last update":"Fri Jul 14 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.71199999999999997,"15min":0.65600000000000003},"min":{"1min":0.52600000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":6.5389999999999997,"5min":6.5389999999999997,"15min":6.5389999999999997},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67100000000000004,"15min":0.63600000000000001},"min":{"1min":0.46899999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.94599999999999995,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.69799999999999995}]},{"osd":7,"last update":"Fri Jul 14 18:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.79200000000000004,"15min":0.79400000000000004},"min":{"1min":0.59999999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.97299999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.79300000000000004,"15min":0.80400000000000005},"min":{"1min":0.53500000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94599999999999995,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.76200000000000001}]},{"osd":8,"last update":"Fri Jul 14 18:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.65400000000000003,"15min":0.64600000000000002},"min":{"1min":0.50900000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.64800000000000002,"15min":0.63800000000000001},"min":{"1min":0.40500000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.94499999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63300000000000001}]},{"osd":9,"last update":"Fri Jul 14 18:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.78700000000000003,"15min":0.78600000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0149999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.82299999999999995,"15min":0.81399999999999995},"min":{"1min":0.65800000000000003,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.73699999999999999}]},{"osd":10,"last update":"Fri Jul 14 18:16:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.79000000000000004,"15min":0.76200000000000001},"min":{"1min":0.63800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0529999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.79200000000000004,"15min":0.77300000000000002},"min":{"1min":0.54100000000000004,"5min":0.438,"15min":0.438},"max":{"1min":1.0109999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.71199999999999997}]},{"osd":11,"last update":"Fri Jul 14 18:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.82999999999999996,"15min":0.83899999999999997},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.99399999999999999,"5min":1.04,"15min":1.04},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.85799999999999998,"15min":0.86399999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0229999999999999,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.751}]}]},{"osd":3,"up_from":26,"seq":111669149771,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":507355,"data_compressed":23584,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.44900000000000001,"15min":0.39700000000000002},"min":{"1min":0.34300000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.74099999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.47099999999999997,"15min":0.42099999999999999},"min":{"1min":0.311,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.96199999999999997,"5min":0.998,"15min":0.998},"last":0.42099999999999999}]},{"osd":1,"last update":"Fri Jul 14 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.61899999999999999,"15min":0.58299999999999996},"min":{"1min":0.48799999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.64100000000000001,"15min":0.59499999999999997},"min":{"1min":0.51400000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.874,"5min":1.29,"15min":1.29},"last":0.53800000000000003}]},{"osd":2,"last update":"Fri Jul 14 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.64900000000000002,"15min":0.59099999999999997},"min":{"1min":0.55200000000000005,"5min":0.44,"15min":0.44},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.64000000000000001,"15min":0.60699999999999998},"min":{"1min":0.46999999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.81299999999999994}]},{"osd":4,"last update":"Fri Jul 14 18:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.65400000000000003,"15min":0.59799999999999998},"min":{"1min":0.48699999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.123,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.68000000000000005,"15min":0.64000000000000001},"min":{"1min":0.50800000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94999999999999996,"5min":1.153,"15min":1.153},"last":0.68500000000000005}]},{"osd":5,"last update":"Fri Jul 14 18:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.67100000000000004,"15min":0.63800000000000001},"min":{"1min":0.46300000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.86599999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.753},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68200000000000005,"15min":0.627},"min":{"1min":0.53900000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.877,"5min":1.149,"15min":1.149},"last":0.79000000000000004}]},{"osd":6,"last update":"Fri Jul 14 18:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.61299999999999999,"15min":0.59699999999999998},"min":{"1min":0.41799999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85099999999999998,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.66000000000000003,"15min":0.624},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0649999999999999,"5min":1.504,"15min":1.504},"last":0.83399999999999996}]},{"osd":7,"last update":"Fri Jul 14 18:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72299999999999998,"15min":0.69299999999999995},"min":{"1min":0.51600000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.105,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.75,"15min":0.71199999999999997},"min":{"1min":0.60299999999999998,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.96199999999999997,"5min":1.407,"15min":1.407},"last":0.71999999999999997}]},{"osd":8,"last update":"Fri Jul 14 18:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.72799999999999998}]},{"osd":9,"last update":"Fri Jul 14 18:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.79200000000000004,"15min":0.81299999999999994},"min":{"1min":0.51700000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.2450000000000001,"5min":1.619,"15min":1.619},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.86299999999999999,"15min":0.90100000000000002},"min":{"1min":0.57799999999999996,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2689999999999999,"5min":1.573,"15min":1.573},"last":0.80200000000000005}]},{"osd":10,"last update":"Fri Jul 14 18:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.85899999999999999,"15min":0.88100000000000001},"min":{"1min":0.65700000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.94399999999999995,"5min":1.542,"15min":1.542},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.82399999999999995,"15min":0.84999999999999998},"min":{"1min":0.61899999999999999,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.089,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.77900000000000003}]},{"osd":11,"last update":"Fri Jul 14 18:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.84999999999999998,"15min":0.85699999999999998},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":0.93400000000000005,"5min":1.472,"15min":1.472},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.79600000000000004,"15min":0.80700000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.92100000000000004,"5min":1.206,"15min":1.206},"last":0.82599999999999996}]}]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":513184,"data_compressed":23584,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.48999999999999999,"15min":0.46000000000000002},"min":{"1min":0.39700000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.83799999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.50800000000000001,"15min":0.47799999999999998},"min":{"1min":0.46700000000000003,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.747,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.39600000000000002}]},{"osd":1,"last update":"Fri Jul 14 18:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.45000000000000001,"15min":0.38200000000000001},"min":{"1min":0.26400000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.55000000000000004,"5min":3.7890000000000001,"15min":3.7890000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.435,"15min":0.39300000000000002},"min":{"1min":0.23899999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.57799999999999996,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.439}]},{"osd":2,"last update":"Fri Jul 14 18:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60399999999999998,"15min":0.55800000000000005},"min":{"1min":0.34699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.875,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.65000000000000002,"15min":0.56599999999999995},"min":{"1min":0.47099999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.80500000000000005,"5min":3.6989999999999998,"15min":3.6989999999999998},"last":0.57899999999999996}]},{"osd":3,"last update":"Fri Jul 14 18:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.63300000000000001,"15min":0.54600000000000004},"min":{"1min":0.34100000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":3.5190000000000001,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":0.497},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.623,"15min":0.54900000000000004},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":3.593,"5min":3.593,"15min":3.593},"last":0.46600000000000003}]},{"osd":5,"last update":"Fri Jul 14 18:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.67600000000000005,"15min":0.58799999999999997},"min":{"1min":0.41299999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83999999999999997,"5min":3.7490000000000001,"15min":3.7490000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65900000000000003,"15min":0.58599999999999997},"min":{"1min":0.434,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.68700000000000006}]},{"osd":6,"last update":"Fri Jul 14 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.747,"15min":0.68300000000000005},"min":{"1min":0.46700000000000003,"5min":0.441,"15min":0.441},"max":{"1min":4.117,"5min":4.117,"15min":4.117},"last":0.59699999999999998},{"interface":"front","average":{"1min":1.054,"5min":0.76200000000000001,"15min":0.69199999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":4.1440000000000001,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":0.65600000000000003}]},{"osd":7,"last update":"Fri Jul 14 18:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.66900000000000004,"15min":0.59699999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.872,"5min":3.8999999999999999,"15min":3.8999999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67400000000000004,"15min":0.60999999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.95799999999999996,"5min":3.2250000000000001,"15min":3.2250000000000001},"last":0.627}]},{"osd":8,"last update":"Fri Jul 14 18:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.68999999999999995,"15min":0.65400000000000003},"min":{"1min":0.51000000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.874,"5min":2.359,"15min":2.359},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.80400000000000005,"15min":0.76600000000000001},"min":{"1min":0.52000000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.0149999999999999,"5min":3.1739999999999999,"15min":3.1739999999999999},"last":0.76700000000000002}]},{"osd":9,"last update":"Fri Jul 14 18:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.84399999999999997,"15min":0.80800000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":4.3310000000000004,"5min":4.3310000000000004,"15min":4.3310000000000004},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.876,"5min":0.76700000000000002,"15min":0.71899999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":3.766,"5min":3.766,"15min":3.766},"last":0.61299999999999999}]},{"osd":10,"last update":"Fri Jul 14 18:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.80800000000000005,"15min":0.79000000000000004},"min":{"1min":0.39300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91300000000000003,"5min":3.8239999999999998,"15min":3.8239999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.74099999999999999,"15min":0.73599999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91100000000000003,"5min":2.52,"15min":2.52},"last":0.64200000000000002}]},{"osd":11,"last update":"Fri Jul 14 18:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.96099999999999997,"15min":0.99099999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.002,"5min":3.548,"15min":3.548},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.93200000000000005,"15min":0.94799999999999995},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.0269999999999999,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":0.81899999999999995}]}]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":1103479,"data_compressed":59328,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.36799999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68100000000000005,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.55400000000000005,"15min":0.57399999999999995},"min":{"1min":0.379,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.621,"5min":1.645,"15min":1.645},"last":0.52900000000000003}]},{"osd":1,"last update":"Fri Jul 14 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.61299999999999999,"15min":0.56599999999999995},"min":{"1min":0.40999999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.874,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60299999999999998,"15min":0.57699999999999996},"min":{"1min":0.378,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80700000000000005,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.621}]},{"osd":2,"last update":"Fri Jul 14 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.43099999999999999,"15min":0.42699999999999999},"min":{"1min":0.33200000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46000000000000002,"15min":0.48699999999999999},"min":{"1min":0.28699999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.71099999999999997,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.71099999999999997}]},{"osd":3,"last update":"Fri Jul 14 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58199999999999996,"15min":0.61099999999999999},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77300000000000002,"5min":1.591,"15min":1.591},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.46700000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.84699999999999998}]},{"osd":4,"last update":"Fri Jul 14 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.626,"15min":0.62},"min":{"1min":0.47499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82799999999999996,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.63600000000000001,"15min":0.624},"min":{"1min":0.35999999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91200000000000003,"5min":1.504,"15min":1.504},"last":0.68999999999999995}]},{"osd":6,"last update":"Fri Jul 14 18:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64400000000000002,"15min":0.65700000000000003},"min":{"1min":0.45100000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.82699999999999996,"5min":1.627,"15min":1.627},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65900000000000003,"15min":0.63900000000000001},"min":{"1min":0.45800000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.86499999999999999,"5min":1.036,"15min":1.036},"last":0.89600000000000002}]},{"osd":7,"last update":"Fri Jul 14 18:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68400000000000005,"15min":0.68500000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0089999999999999,"5min":1.554,"15min":1.554},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.68799999999999994,"15min":0.66100000000000003},"min":{"1min":0.54000000000000004,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92500000000000004,"5min":1.01,"15min":1.01},"last":0.93899999999999995}]},{"osd":8,"last update":"Fri Jul 14 18:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59399999999999997,"15min":0.57999999999999996},"min":{"1min":0.36599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80600000000000005,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59499999999999997,"15min":0.58299999999999996},"min":{"1min":0.41199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.86299999999999999}]},{"osd":9,"last update":"Fri Jul 14 18:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.57099999999999995,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.88400000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.02,"5min":0.81499999999999995,"15min":0.78500000000000003},"min":{"1min":0.58699999999999997,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":6.0460000000000003,"5min":6.0460000000000003,"15min":6.0460000000000003},"last":0.97399999999999998}]},{"osd":10,"last update":"Fri Jul 14 18:16:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.746,"15min":0.72999999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.77200000000000002,"15min":0.755},"min":{"1min":0.56899999999999995,"5min":0.505,"15min":0.505},"max":{"1min":0.96799999999999997,"5min":1.107,"15min":1.107},"last":1.008}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34102,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34102,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34102,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35744,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":66480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T18:16:32.310 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T18:16:32.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.494+0000 7fa284dff700 1 -- 172.21.15.133:0/1244994124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2801017b0 msgr2=0x7fa280108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:32.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.494+0000 7fa284dff700 1 --2- 172.21.15.133:0/1244994124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2801017b0 0x7fa280108070 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa278009f90 tx=0x7fa27800b330 comp rx=0 tx=0).stop 2023-07-14T18:16:32.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.494+0000 7fa284dff700 1 -- 172.21.15.133:0/1244994124 shutdown_connections 2023-07-14T18:16:32.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.494+0000 7fa284dff700 1 --2- 172.21.15.133:0/1244994124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2801017b0 0x7fa280108070 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:32.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.494+0000 7fa284dff700 1 --2- 172.21.15.133:0/1244994124 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2800ffc30 0x7fa280100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:32.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.494+0000 7fa284dff700 1 --2- 172.21.15.133:0/1244994124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa280100e20 0x7fa280101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:32.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.494+0000 7fa284dff700 1 -- 172.21.15.133:0/1244994124 >> 172.21.15.133:0/1244994124 conn(0x7fa2800fb210 msgr2=0x7fa2800fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:32.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.494+0000 7fa284dff700 1 -- 172.21.15.133:0/1244994124 shutdown_connections 2023-07-14T18:16:32.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.494+0000 7fa284dff700 1 -- 172.21.15.133:0/1244994124 wait complete. 2023-07-14T18:16:32.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.495+0000 7fa284dff700 1 Processor -- start 2023-07-14T18:16:32.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.495+0000 7fa284dff700 1 -- start start 2023-07-14T18:16:32.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.496+0000 7fa284dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2800ffc30 0x7fa2800743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:32.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.496+0000 7fa284dff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa280100e20 0x7fa280072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:32.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.496+0000 7fa284dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2801017b0 0x7fa280072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:32.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.496+0000 7fa284dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa280073710 con 0x7fa2800ffc30 2023-07-14T18:16:32.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.496+0000 7fa284dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa280073880 con 0x7fa2801017b0 2023-07-14T18:16:32.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.496+0000 7fa284dff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28019fa90 con 0x7fa280100e20 2023-07-14T18:16:32.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.496+0000 7fa27e59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2800ffc30 0x7fa2800743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:32.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.496+0000 7fa27e59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2800ffc30 0x7fa2800743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43714/0 (socket says 172.21.15.133:43714) 2023-07-14T18:16:32.545 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.496+0000 7fa27e59c700 1 -- 172.21.15.133:0/3521636818 learned_addr learned my addr 172.21.15.133:0/3521636818 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:32.545 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.496+0000 7fa27dd9b700 1 --2- 172.21.15.133:0/3521636818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa280100e20 0x7fa280072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:32.545 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.496+0000 7fa27ed9d700 1 --2- 172.21.15.133:0/3521636818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2801017b0 0x7fa280072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:32.545 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.496+0000 7fa27e59c700 1 -- 172.21.15.133:0/3521636818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa280100e20 msgr2=0x7fa280072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:32.545 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.496+0000 7fa27e59c700 1 --2- 172.21.15.133:0/3521636818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa280100e20 0x7fa280072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:32.545 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.496+0000 7fa27e59c700 1 -- 172.21.15.133:0/3521636818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2801017b0 msgr2=0x7fa280072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:32.545 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.496+0000 7fa27e59c700 1 --2- 172.21.15.133:0/3521636818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2801017b0 0x7fa280072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:32.546 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.496+0000 7fa27e59c700 1 -- 172.21.15.133:0/3521636818 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa278009c40 con 0x7fa2800ffc30 2023-07-14T18:16:32.546 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.496+0000 7fa27e59c700 1 --2- 172.21.15.133:0/3521636818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2800ffc30 0x7fa2800743f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa27000e9f0 tx=0x7fa27000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:32.546 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.497+0000 7fa26f7fe700 1 -- 172.21.15.133:0/3521636818 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa270004220 con 0x7fa2800ffc30 2023-07-14T18:16:32.546 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.497+0000 7fa284dff700 1 -- 172.21.15.133:0/3521636818 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa28019fd70 con 0x7fa2800ffc30 2023-07-14T18:16:32.546 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.497+0000 7fa26f7fe700 1 -- 172.21.15.133:0/3521636818 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa270004380 con 0x7fa2800ffc30 2023-07-14T18:16:32.546 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.497+0000 7fa26f7fe700 1 -- 172.21.15.133:0/3521636818 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa270003be0 con 0x7fa2800ffc30 2023-07-14T18:16:32.546 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.497+0000 7fa284dff700 1 -- 172.21.15.133:0/3521636818 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2801a0210 con 0x7fa2800ffc30 2023-07-14T18:16:32.547 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.498+0000 7fa284dff700 1 -- 172.21.15.133:0/3521636818 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa28010b460 con 0x7fa2800ffc30 2023-07-14T18:16:32.547 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.500+0000 7fa26f7fe700 1 -- 172.21.15.133:0/3521636818 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fa270003d40 con 0x7fa2800ffc30 2023-07-14T18:16:32.548 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.501+0000 7fa26f7fe700 1 --2- 172.21.15.133:0/3521636818 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa2680a11a0 0x7fa2680a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:32.548 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.501+0000 7fa26f7fe700 1 -- 172.21.15.133:0/3521636818 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa270014070 con 0x7fa2800ffc30 2023-07-14T18:16:32.548 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.501+0000 7fa27dd9b700 1 --2- 172.21.15.133:0/3521636818 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa2680a11a0 0x7fa2680a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:32.548 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.502+0000 7fa27dd9b700 1 --2- 172.21.15.133:0/3521636818 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa2680a11a0 0x7fa2680a3650 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa274009c60 tx=0x7fa274009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:32.549 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.503+0000 7fa26f7fe700 1 -- 172.21.15.133:0/3521636818 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fa2700961c0 con 0x7fa2800ffc30 2023-07-14T18:16:32.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.701+0000 7fa284dff700 1 -- 172.21.15.133:0/3521636818 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2801a0880 con 0x7fa2800ffc30 2023-07-14T18:16:32.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.703+0000 7fa26f7fe700 1 -- 172.21.15.133:0/3521636818 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24520 (secure 0 0 0) 0x7fa2700997e0 con 0x7fa2800ffc30 2023-07-14T18:16:32.749 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:32.749 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":92,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","created":"2023-07-14T18:06:49.561582+0000","modified":"2023-07-14T18:15:49.951050+0000","last_up_change":"2023-07-14T18:13:29.836099+0000","last_in_change":"2023-07-14T18:13:11.606710+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T18:10:03.945039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T18:14:15.451340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T18:14:16.892215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T18:14:20.493175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"85a29de7-d712-422a-b647-d9e90091aaf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6803","nonce":3076510801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6805","nonce":3076510801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6809","nonce":3076510801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6807","nonce":3076510801}]},"public_addr":"172.21.15.133:6803/3076510801","cluster_addr":"172.21.15.133:6805/3076510801","heartbeat_back_addr":"172.21.15.133:6809/3076510801","heartbeat_front_addr":"172.21.15.133:6807/3076510801","state":["exists","up"]},{"osd":1,"uuid":"80ad4e78-7917-495e-b119-99f7c7c36243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6801","nonce":1841901852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6803","nonce":1841901852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6807","nonce":1841901852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6805","nonce":1841901852}]},"public_addr":"172.21.15.161:6801/1841901852","cluster_addr":"172.21.15.161:6803/1841901852","heartbeat_back_addr":"172.21.15.161:6807/1841901852","heartbeat_front_addr":"172.21.15.161:6805/1841901852","state":["exists","up"]},{"osd":2,"uuid":"fc4bfd37-1fdd-47c6-a48e-6d910daf9e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6801","nonce":346103099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6803","nonce":346103099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6807","nonce":346103099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6805","nonce":346103099}]},"public_addr":"172.21.15.171:6801/346103099","cluster_addr":"172.21.15.171:6803/346103099","heartbeat_back_addr":"172.21.15.171:6807/346103099","heartbeat_front_addr":"172.21.15.171:6805/346103099","state":["exists","up"]},{"osd":3,"uuid":"4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6811","nonce":2152994143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6813","nonce":2152994143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6817","nonce":2152994143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6815","nonce":2152994143}]},"public_addr":"172.21.15.133:6811/2152994143","cluster_addr":"172.21.15.133:6813/2152994143","heartbeat_back_addr":"172.21.15.133:6817/2152994143","heartbeat_front_addr":"172.21.15.133:6815/2152994143","state":["exists","up"]},{"osd":4,"uuid":"4601a1c2-254b-4c4d-9b15-8fc952b3d2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6809","nonce":47213291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6811","nonce":47213291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6815","nonce":47213291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6813","nonce":47213291}]},"public_addr":"172.21.15.161:6809/47213291","cluster_addr":"172.21.15.161:6811/47213291","heartbeat_back_addr":"172.21.15.161:6815/47213291","heartbeat_front_addr":"172.21.15.161:6813/47213291","state":["exists","up"]},{"osd":5,"uuid":"6cca180a-9140-4e49-835b-86842dae0574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6809","nonce":777556751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6811","nonce":777556751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6815","nonce":777556751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6813","nonce":777556751}]},"public_addr":"172.21.15.171:6809/777556751","cluster_addr":"172.21.15.171:6811/777556751","heartbeat_back_addr":"172.21.15.171:6815/777556751","heartbeat_front_addr":"172.21.15.171:6813/777556751","state":["exists","up"]},{"osd":6,"uuid":"ea0299ff-7740-4494-a3a9-1503b3fc0ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6819","nonce":2884152358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6821","nonce":2884152358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6825","nonce":2884152358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6823","nonce":2884152358}]},"public_addr":"172.21.15.133:6819/2884152358","cluster_addr":"172.21.15.133:6821/2884152358","heartbeat_back_addr":"172.21.15.133:6825/2884152358","heartbeat_front_addr":"172.21.15.133:6823/2884152358","state":["exists","up"]},{"osd":7,"uuid":"b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6817","nonce":4183705749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6819","nonce":4183705749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6823","nonce":4183705749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6821","nonce":4183705749}]},"public_addr":"172.21.15.161:6817/4183705749","cluster_addr":"172.21.15.161:6819/4183705749","heartbeat_back_addr":"172.21.15.161:6823/4183705749","heartbeat_front_addr":"172.21.15.161:6821/4183705749","state":["exists","up"]},{"osd":8,"uuid":"87dffc07-bf48-48c8-b418-5c4b509686d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6817","nonce":2771494483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6819","nonce":2771494483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6823","nonce":2771494483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6821","nonce":2771494483}]},"public_addr":"172.21.15.171:6817/2771494483","cluster_addr":"172.21.15.171:6819/2771494483","heartbeat_back_addr":"172.21.15.171:6823/2771494483","heartbeat_front_addr":"172.21.15.171:6821/2771494483","state":["exists","up"]},{"osd":9,"uuid":"e0832870-79ce-4bce-8045-58b7a7f6dc69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6827","nonce":2066164149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6829","nonce":2066164149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6833","nonce":2066164149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6831","nonce":2066164149}]},"public_addr":"172.21.15.133:6827/2066164149","cluster_addr":"172.21.15.133:6829/2066164149","heartbeat_back_addr":"172.21.15.133:6833/2066164149","heartbeat_front_addr":"172.21.15.133:6831/2066164149","state":["exists","up"]},{"osd":10,"uuid":"081f0484-d134-4b2e-9c48-ef8e6a646422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6825","nonce":2162333626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6827","nonce":2162333626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6831","nonce":2162333626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6829","nonce":2162333626}]},"public_addr":"172.21.15.161:6825/2162333626","cluster_addr":"172.21.15.161:6827/2162333626","heartbeat_back_addr":"172.21.15.161:6831/2162333626","heartbeat_front_addr":"172.21.15.161:6829/2162333626","state":["exists","up"]},{"osd":11,"uuid":"4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6825","nonce":3403894313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6827","nonce":3403894313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6831","nonce":3403894313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6829","nonce":3403894313}]},"public_addr":"172.21.15.171:6825/3403894313","cluster_addr":"172.21.15.171:6827/3403894313","heartbeat_back_addr":"172.21.15.171:6831/3403894313","heartbeat_front_addr":"172.21.15.171:6829/3403894313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:17.816229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:39.638543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:00.543317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:26.988921+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:48.644694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:10.049860+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:35.986508+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:57.753846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:19.018245+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:44.460450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:06.279960+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:27.955006+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/1832569241":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6801/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:0/4219815304":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:6801/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/432253615":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1840700480":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1807775439":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1089030734":"2023-07-15T18:07:47.346704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T18:16:32.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.705+0000 7fa284dff700 1 -- 172.21.15.133:0/3521636818 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa2680a11a0 msgr2=0x7fa2680a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:32.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.705+0000 7fa284dff700 1 --2- 172.21.15.133:0/3521636818 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa2680a11a0 0x7fa2680a3650 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa274009c60 tx=0x7fa274009260 comp rx=0 tx=0).stop 2023-07-14T18:16:32.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.705+0000 7fa284dff700 1 -- 172.21.15.133:0/3521636818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2800ffc30 msgr2=0x7fa2800743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:32.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.705+0000 7fa284dff700 1 --2- 172.21.15.133:0/3521636818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2800ffc30 0x7fa2800743f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa27000e9f0 tx=0x7fa27000ed00 comp rx=0 tx=0).stop 2023-07-14T18:16:32.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.706+0000 7fa284dff700 1 -- 172.21.15.133:0/3521636818 shutdown_connections 2023-07-14T18:16:32.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.706+0000 7fa284dff700 1 --2- 172.21.15.133:0/3521636818 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa2680a11a0 0x7fa2680a3650 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:32.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.706+0000 7fa284dff700 1 --2- 172.21.15.133:0/3521636818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2801017b0 0x7fa280072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:32.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.706+0000 7fa284dff700 1 --2- 172.21.15.133:0/3521636818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2800ffc30 0x7fa2800743f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:32.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.706+0000 7fa284dff700 1 --2- 172.21.15.133:0/3521636818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa280100e20 0x7fa280072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:32.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.706+0000 7fa284dff700 1 -- 172.21.15.133:0/3521636818 >> 172.21.15.133:0/3521636818 conn(0x7fa2800fb210 msgr2=0x7fa2800fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:32.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.706+0000 7fa284dff700 1 -- 172.21.15.133:0/3521636818 shutdown_connections 2023-07-14T18:16:32.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.706+0000 7fa284dff700 1 -- 172.21.15.133:0/3521636818 wait complete. 2023-07-14T18:16:32.770 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T18:10:03.945039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T18:14:15.451340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T18:14:16.892215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T18:14:20.493175+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T18:16:32.771 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T18:16:32.975 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.930+0000 7f81a03ff700 1 -- 172.21.15.133:0/3873425076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8198107ad0 msgr2=0x7f8198109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:32.975 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.930+0000 7f81a03ff700 1 --2- 172.21.15.133:0/3873425076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8198107ad0 0x7f8198109f50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f819000b0a0 tx=0x7f819000b3b0 comp rx=0 tx=0).stop 2023-07-14T18:16:32.975 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.930+0000 7f81a03ff700 1 -- 172.21.15.133:0/3873425076 shutdown_connections 2023-07-14T18:16:32.976 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.930+0000 7f81a03ff700 1 --2- 172.21.15.133:0/3873425076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8198107ad0 0x7f8198109f50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:32.976 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.930+0000 7f81a03ff700 1 --2- 172.21.15.133:0/3873425076 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8198074680 0x7f8198072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:32.976 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.930+0000 7f81a03ff700 1 --2- 172.21.15.133:0/3873425076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81980730a0 0x7f8198073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:32.976 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.930+0000 7f81a03ff700 1 -- 172.21.15.133:0/3873425076 >> 172.21.15.133:0/3873425076 conn(0x7f81980fb210 msgr2=0x7f81980fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:32.976 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.930+0000 7f81a03ff700 1 -- 172.21.15.133:0/3873425076 shutdown_connections 2023-07-14T18:16:32.976 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.930+0000 7f81a03ff700 1 -- 172.21.15.133:0/3873425076 wait complete. 2023-07-14T18:16:32.978 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.931+0000 7f81a03ff700 1 Processor -- start 2023-07-14T18:16:32.978 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.931+0000 7f81a03ff700 1 -- start start 2023-07-14T18:16:32.978 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.932+0000 7f81a03ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81980730a0 0x7f8198197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:32.978 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.932+0000 7f81a03ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8198074680 0x7f81981983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:32.978 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.932+0000 7f81a03ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8198107ad0 0x7f819819f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:32.978 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.932+0000 7f81a03ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819819fa30 con 0x7f8198074680 2023-07-14T18:16:32.978 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.932+0000 7f81a03ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819819fb70 con 0x7f81980730a0 2023-07-14T18:16:32.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.932+0000 7f81a03ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819819fd00 con 0x7f8198107ad0 2023-07-14T18:16:32.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.932+0000 7f819d99a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8198074680 0x7f81981983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:32.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.932+0000 7f819e99c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8198107ad0 0x7f819819f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:32.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.932+0000 7f819e19b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81980730a0 0x7f8198197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:32.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.932+0000 7f819e19b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81980730a0 0x7f8198197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:33748/0 (socket says 172.21.15.133:33748) 2023-07-14T18:16:32.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.932+0000 7f819d99a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8198074680 0x7f81981983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43726/0 (socket says 172.21.15.133:43726) 2023-07-14T18:16:32.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.932+0000 7f819e99c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8198107ad0 0x7f819819f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:44280/0 (socket says 172.21.15.133:44280) 2023-07-14T18:16:32.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.932+0000 7f819e19b700 1 -- 172.21.15.133:0/1035167011 learned_addr learned my addr 172.21.15.133:0/1035167011 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:32.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.933+0000 7f819e19b700 1 -- 172.21.15.133:0/1035167011 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8198107ad0 msgr2=0x7f819819f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:32.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.933+0000 7f819e19b700 1 --2- 172.21.15.133:0/1035167011 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8198107ad0 0x7f819819f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:32.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.933+0000 7f819e19b700 1 -- 172.21.15.133:0/1035167011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8198074680 msgr2=0x7f81981983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:32.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.933+0000 7f819e19b700 1 --2- 172.21.15.133:0/1035167011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8198074680 0x7f81981983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:32.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.933+0000 7f819e19b700 1 -- 172.21.15.133:0/1035167011 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8190009d00 con 0x7f81980730a0 2023-07-14T18:16:32.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.933+0000 7f819e19b700 1 --2- 172.21.15.133:0/1035167011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81980730a0 0x7f8198197eb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f819400e970 tx=0x7f819400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:32.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.933+0000 7f818f7fe700 1 -- 172.21.15.133:0/1035167011 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194009790 con 0x7f81980730a0 2023-07-14T18:16:32.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.933+0000 7f818f7fe700 1 -- 172.21.15.133:0/1035167011 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f819400c7e0 con 0x7f81980730a0 2023-07-14T18:16:32.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.933+0000 7f81a03ff700 1 -- 172.21.15.133:0/1035167011 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f819819ffe0 con 0x7f81980730a0 2023-07-14T18:16:32.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.933+0000 7f818f7fe700 1 -- 172.21.15.133:0/1035167011 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81940054e0 con 0x7f81980730a0 2023-07-14T18:16:32.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.933+0000 7f81a03ff700 1 -- 172.21.15.133:0/1035167011 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81981a0400 con 0x7f81980730a0 2023-07-14T18:16:32.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.934+0000 7f81a03ff700 1 -- 172.21.15.133:0/1035167011 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f819810d660 con 0x7f81980730a0 2023-07-14T18:16:32.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.937+0000 7f818f7fe700 1 -- 172.21.15.133:0/1035167011 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f819400c950 con 0x7f81980730a0 2023-07-14T18:16:32.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.938+0000 7f818f7fe700 1 --2- 172.21.15.133:0/1035167011 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f81840a11d0 0x7f81840a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:32.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.938+0000 7f818f7fe700 1 -- 172.21.15.133:0/1035167011 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8194014070 con 0x7f81980730a0 2023-07-14T18:16:32.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.938+0000 7f819d99a700 1 --2- 172.21.15.133:0/1035167011 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f81840a11d0 0x7f81840a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:32.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.939+0000 7f819d99a700 1 --2- 172.21.15.133:0/1035167011 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f81840a11d0 0x7f81840a3680 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f81980743a0 tx=0x7f8188009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:32.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:32.940+0000 7f818f7fe700 1 -- 172.21.15.133:0/1035167011 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f81940968f0 con 0x7f81980730a0 2023-07-14T18:16:33.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:32 smithi133 ceph-mon[126384]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T18:16:33.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:32 smithi133 ceph-mon[126384]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 6 op/s 2023-07-14T18:16:33.158 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:32 smithi133 ceph-mon[126384]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T18:16:33.158 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:32 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3521636818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:16:33.186 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.141+0000 7f81a03ff700 1 -- 172.21.15.133:0/1035167011 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f81981a0a70 con 0x7f81980730a0 2023-07-14T18:16:33.187 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.142+0000 7f818f7fe700 1 -- 172.21.15.133:0/1035167011 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8194099f10 con 0x7f81980730a0 2023-07-14T18:16:33.187 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-07-14T18:16:33.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.144+0000 7f81a03ff700 1 -- 172.21.15.133:0/1035167011 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f81840a11d0 msgr2=0x7f81840a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:33.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.144+0000 7f81a03ff700 1 --2- 172.21.15.133:0/1035167011 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f81840a11d0 0x7f81840a3680 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f81980743a0 tx=0x7f8188009260 comp rx=0 tx=0).stop 2023-07-14T18:16:33.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.144+0000 7f81a03ff700 1 -- 172.21.15.133:0/1035167011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81980730a0 msgr2=0x7f8198197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:33.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.144+0000 7f81a03ff700 1 --2- 172.21.15.133:0/1035167011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81980730a0 0x7f8198197eb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f819400e970 tx=0x7f819400ec80 comp rx=0 tx=0).stop 2023-07-14T18:16:33.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.144+0000 7f81a03ff700 1 -- 172.21.15.133:0/1035167011 shutdown_connections 2023-07-14T18:16:33.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.144+0000 7f81a03ff700 1 --2- 172.21.15.133:0/1035167011 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f81840a11d0 0x7f81840a3680 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:33.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.144+0000 7f81a03ff700 1 --2- 172.21.15.133:0/1035167011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81980730a0 0x7f8198197eb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:33.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.144+0000 7f81a03ff700 1 --2- 172.21.15.133:0/1035167011 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8198107ad0 0x7f819819f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:33.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.144+0000 7f81a03ff700 1 --2- 172.21.15.133:0/1035167011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8198074680 0x7f81981983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:33.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.144+0000 7f81a03ff700 1 -- 172.21.15.133:0/1035167011 >> 172.21.15.133:0/1035167011 conn(0x7f81980fb210 msgr2=0x7f8198109cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:33.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.144+0000 7f81a03ff700 1 -- 172.21.15.133:0/1035167011 shutdown_connections 2023-07-14T18:16:33.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.144+0000 7f81a03ff700 1 -- 172.21.15.133:0/1035167011 wait complete. 2023-07-14T18:16:33.208 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T18:16:33.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:32 smithi161 ceph-mon[128993]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T18:16:33.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:32 smithi161 ceph-mon[128993]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 6 op/s 2023-07-14T18:16:33.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:32 smithi161 ceph-mon[128993]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T18:16:33.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:32 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3521636818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:16:33.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:32 smithi171 ceph-mon[129241]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T18:16:33.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:32 smithi171 ceph-mon[129241]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 6 op/s 2023-07-14T18:16:33.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:32 smithi171 ceph-mon[129241]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T18:16:33.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:32 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3521636818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:16:33.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.362+0000 7f6b87fff700 1 -- 172.21.15.133:0/2037270367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b80101790 msgr2=0x7f6b80108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:33.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.362+0000 7f6b87fff700 1 --2- 172.21.15.133:0/2037270367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b80101790 0x7f6b80108050 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6b7800b0a0 tx=0x7f6b7800b3b0 comp rx=0 tx=0).stop 2023-07-14T18:16:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.362+0000 7f6b87fff700 1 -- 172.21.15.133:0/2037270367 shutdown_connections 2023-07-14T18:16:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.362+0000 7f6b87fff700 1 --2- 172.21.15.133:0/2037270367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b80101790 0x7f6b80108050 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.362+0000 7f6b87fff700 1 --2- 172.21.15.133:0/2037270367 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b800ffc10 0x7f6b80100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.362+0000 7f6b87fff700 1 --2- 172.21.15.133:0/2037270367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b80100e00 0x7f6b80101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.362+0000 7f6b87fff700 1 -- 172.21.15.133:0/2037270367 >> 172.21.15.133:0/2037270367 conn(0x7f6b800fb210 msgr2=0x7f6b800fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.362+0000 7f6b87fff700 1 -- 172.21.15.133:0/2037270367 shutdown_connections 2023-07-14T18:16:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.362+0000 7f6b87fff700 1 -- 172.21.15.133:0/2037270367 wait complete. 2023-07-14T18:16:33.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.363+0000 7f6b87fff700 1 Processor -- start 2023-07-14T18:16:33.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.363+0000 7f6b87fff700 1 -- start start 2023-07-14T18:16:33.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.363+0000 7f6b87fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b800ffc10 0x7f6b80197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:33.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.363+0000 7f6b87fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b80100e00 0x7f6b801981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:33.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.363+0000 7f6b87fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b80101790 0x7f6b8019f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:33.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.363+0000 7f6b87fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8019f800 con 0x7f6b800ffc10 2023-07-14T18:16:33.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.363+0000 7f6b87fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8019f970 con 0x7f6b80100e00 2023-07-14T18:16:33.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.363+0000 7f6b87fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8019fb30 con 0x7f6b80101790 2023-07-14T18:16:33.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.363+0000 7f6b85d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b800ffc10 0x7f6b80197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:33.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.363+0000 7f6b85d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b800ffc10 0x7f6b80197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43752/0 (socket says 172.21.15.133:43752) 2023-07-14T18:16:33.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.363+0000 7f6b85d9b700 1 -- 172.21.15.133:0/3268033446 learned_addr learned my addr 172.21.15.133:0/3268033446 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:33.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.363+0000 7f6b8559a700 1 --2- 172.21.15.133:0/3268033446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b80100e00 0x7f6b801981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:33.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.364+0000 7f6b8659c700 1 --2- 172.21.15.133:0/3268033446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b80101790 0x7f6b8019f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:33.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.364+0000 7f6b85d9b700 1 -- 172.21.15.133:0/3268033446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b80101790 msgr2=0x7f6b8019f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:33.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.364+0000 7f6b85d9b700 1 --2- 172.21.15.133:0/3268033446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b80101790 0x7f6b8019f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:33.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.364+0000 7f6b85d9b700 1 -- 172.21.15.133:0/3268033446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b80100e00 msgr2=0x7f6b801981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:33.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.364+0000 7f6b85d9b700 1 --2- 172.21.15.133:0/3268033446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b80100e00 0x7f6b801981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:33.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.364+0000 7f6b85d9b700 1 -- 172.21.15.133:0/3268033446 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b78009d00 con 0x7f6b800ffc10 2023-07-14T18:16:33.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.364+0000 7f6b85d9b700 1 --2- 172.21.15.133:0/3268033446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b800ffc10 0x7f6b80197c90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00e970 tx=0x7f6b7c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:33.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.364+0000 7f6b76ffd700 1 -- 172.21.15.133:0/3268033446 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c009790 con 0x7f6b800ffc10 2023-07-14T18:16:33.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.364+0000 7f6b76ffd700 1 -- 172.21.15.133:0/3268033446 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b7c00c7e0 con 0x7f6b800ffc10 2023-07-14T18:16:33.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.364+0000 7f6b87fff700 1 -- 172.21.15.133:0/3268033446 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b8019fe10 con 0x7f6b800ffc10 2023-07-14T18:16:33.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.364+0000 7f6b76ffd700 1 -- 172.21.15.133:0/3268033446 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c003c70 con 0x7f6b800ffc10 2023-07-14T18:16:33.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.364+0000 7f6b87fff700 1 -- 172.21.15.133:0/3268033446 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b801a0230 con 0x7f6b800ffc10 2023-07-14T18:16:33.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.365+0000 7f6b87fff700 1 -- 172.21.15.133:0/3268033446 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b8010b440 con 0x7f6b800ffc10 2023-07-14T18:16:33.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.367+0000 7f6b76ffd700 1 -- 172.21.15.133:0/3268033446 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6b7c00cc50 con 0x7f6b800ffc10 2023-07-14T18:16:33.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.368+0000 7f6b76ffd700 1 --2- 172.21.15.133:0/3268033446 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6b6c0a1150 0x7f6b6c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:33.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.369+0000 7f6b76ffd700 1 -- 172.21.15.133:0/3268033446 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6b7c014070 con 0x7f6b800ffc10 2023-07-14T18:16:33.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.369+0000 7f6b8559a700 1 --2- 172.21.15.133:0/3268033446 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6b6c0a1150 0x7f6b6c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:33.415 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.369+0000 7f6b8559a700 1 --2- 172.21.15.133:0/3268033446 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6b6c0a1150 0x7f6b6c0a3600 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6b80199190 tx=0x7f6b70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:33.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.372+0000 7f6b76ffd700 1 -- 172.21.15.133:0/3268033446 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f6b7c096210 con 0x7f6b800ffc10 2023-07-14T18:16:33.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.575+0000 7f6b87fff700 1 -- 172.21.15.133:0/3268033446 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6b801a08d0 con 0x7f6b800ffc10 2023-07-14T18:16:33.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.576+0000 7f6b76ffd700 1 -- 172.21.15.133:0/3268033446 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6b7c003dd0 con 0x7f6b800ffc10 2023-07-14T18:16:33.621 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-07-14T18:16:33.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.578+0000 7f6b87fff700 1 -- 172.21.15.133:0/3268033446 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6b6c0a1150 msgr2=0x7f6b6c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:33.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.578+0000 7f6b87fff700 1 --2- 172.21.15.133:0/3268033446 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6b6c0a1150 0x7f6b6c0a3600 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6b80199190 tx=0x7f6b70009260 comp rx=0 tx=0).stop 2023-07-14T18:16:33.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.578+0000 7f6b87fff700 1 -- 172.21.15.133:0/3268033446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b800ffc10 msgr2=0x7f6b80197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:33.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.578+0000 7f6b87fff700 1 --2- 172.21.15.133:0/3268033446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b800ffc10 0x7f6b80197c90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00e970 tx=0x7f6b7c00ec80 comp rx=0 tx=0).stop 2023-07-14T18:16:33.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.578+0000 7f6b87fff700 1 -- 172.21.15.133:0/3268033446 shutdown_connections 2023-07-14T18:16:33.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.578+0000 7f6b87fff700 1 --2- 172.21.15.133:0/3268033446 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6b6c0a1150 0x7f6b6c0a3600 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:33.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.578+0000 7f6b87fff700 1 --2- 172.21.15.133:0/3268033446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b80101790 0x7f6b8019f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:33.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.578+0000 7f6b87fff700 1 --2- 172.21.15.133:0/3268033446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b800ffc10 0x7f6b80197c90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:33.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.578+0000 7f6b87fff700 1 --2- 172.21.15.133:0/3268033446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b80100e00 0x7f6b801981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:33.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.578+0000 7f6b87fff700 1 -- 172.21.15.133:0/3268033446 >> 172.21.15.133:0/3268033446 conn(0x7f6b800fb210 msgr2=0x7f6b80106630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:33.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.578+0000 7f6b87fff700 1 -- 172.21.15.133:0/3268033446 shutdown_connections 2023-07-14T18:16:33.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.578+0000 7f6b87fff700 1 -- 172.21.15.133:0/3268033446 wait complete. 2023-07-14T18:16:33.642 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T18:16:33.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.799+0000 7f27759ff700 1 -- 172.21.15.133:0/3642503166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27701017a0 msgr2=0x7f2770108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:33.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.799+0000 7f27759ff700 1 --2- 172.21.15.133:0/3642503166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27701017a0 0x7f2770108060 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f276800b0a0 tx=0x7f276800b3b0 comp rx=0 tx=0).stop 2023-07-14T18:16:33.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.799+0000 7f27759ff700 1 -- 172.21.15.133:0/3642503166 shutdown_connections 2023-07-14T18:16:33.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.799+0000 7f27759ff700 1 --2- 172.21.15.133:0/3642503166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27700ffc20 0x7f2770100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:33.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.799+0000 7f27759ff700 1 --2- 172.21.15.133:0/3642503166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27701017a0 0x7f2770108060 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:33.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.799+0000 7f27759ff700 1 --2- 172.21.15.133:0/3642503166 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2770100e10 0x7f2770101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:33.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.799+0000 7f27759ff700 1 -- 172.21.15.133:0/3642503166 >> 172.21.15.133:0/3642503166 conn(0x7f27700fb260 msgr2=0x7f27700fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:33.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.799+0000 7f27759ff700 1 -- 172.21.15.133:0/3642503166 shutdown_connections 2023-07-14T18:16:33.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.799+0000 7f27759ff700 1 -- 172.21.15.133:0/3642503166 wait complete. 2023-07-14T18:16:33.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.800+0000 7f27759ff700 1 Processor -- start 2023-07-14T18:16:33.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.800+0000 7f27759ff700 1 -- start start 2023-07-14T18:16:33.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.800+0000 7f27759ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27700ffc20 0x7f2770197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:33.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.800+0000 7f27759ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2770100e10 0x7f27701981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:33.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.800+0000 7f27759ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27701017a0 0x7f277019f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:33.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.800+0000 7f27759ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277019f800 con 0x7f2770100e10 2023-07-14T18:16:33.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.800+0000 7f27759ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277019f970 con 0x7f27701017a0 2023-07-14T18:16:33.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.800+0000 7f27759ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277019fb30 con 0x7f27700ffc20 2023-07-14T18:16:33.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.801+0000 7f276effd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27700ffc20 0x7f2770197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:33.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.801+0000 7f276e7fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2770100e10 0x7f27701981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:33.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.801+0000 7f276effd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27700ffc20 0x7f2770197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:44318/0 (socket says 172.21.15.133:44318) 2023-07-14T18:16:33.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.801+0000 7f276effd700 1 -- 172.21.15.133:0/2479184891 learned_addr learned my addr 172.21.15.133:0/2479184891 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:33.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.801+0000 7f276f7fe700 1 --2- 172.21.15.133:0/2479184891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27701017a0 0x7f277019f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:33.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.801+0000 7f276effd700 1 -- 172.21.15.133:0/2479184891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27701017a0 msgr2=0x7f277019f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:33.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.801+0000 7f276effd700 1 --2- 172.21.15.133:0/2479184891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27701017a0 0x7f277019f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:33.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.801+0000 7f276effd700 1 -- 172.21.15.133:0/2479184891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2770100e10 msgr2=0x7f27701981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:33.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.801+0000 7f276effd700 1 --2- 172.21.15.133:0/2479184891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2770100e10 0x7f27701981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:33.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.801+0000 7f276effd700 1 -- 172.21.15.133:0/2479184891 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2760009520 con 0x7f27700ffc20 2023-07-14T18:16:33.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.801+0000 7f276e7fc700 1 --2- 172.21.15.133:0/2479184891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2770100e10 0x7f27701981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:16:33.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.801+0000 7f276f7fe700 1 --2- 172.21.15.133:0/2479184891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27701017a0 0x7f277019f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:16:33.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.801+0000 7f276effd700 1 --2- 172.21.15.133:0/2479184891 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27700ffc20 0x7f2770197c90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f276000ea50 tx=0x7f276000ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:33.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.801+0000 7f27749fd700 1 -- 172.21.15.133:0/2479184891 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276000cbd0 con 0x7f27700ffc20 2023-07-14T18:16:33.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.801+0000 7f27759ff700 1 -- 172.21.15.133:0/2479184891 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2768009d00 con 0x7f27700ffc20 2023-07-14T18:16:33.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.802+0000 7f27749fd700 1 -- 172.21.15.133:0/2479184891 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f276000cd30 con 0x7f27700ffc20 2023-07-14T18:16:33.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.802+0000 7f27749fd700 1 -- 172.21.15.133:0/2479184891 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2760018680 con 0x7f27700ffc20 2023-07-14T18:16:33.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.802+0000 7f27759ff700 1 -- 172.21.15.133:0/2479184891 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27701a0190 con 0x7f27700ffc20 2023-07-14T18:16:33.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.803+0000 7f27759ff700 1 -- 172.21.15.133:0/2479184891 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f277010b450 con 0x7f27700ffc20 2023-07-14T18:16:33.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.805+0000 7f27749fd700 1 -- 172.21.15.133:0/2479184891 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f27600188a0 con 0x7f27700ffc20 2023-07-14T18:16:33.852 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.806+0000 7f27749fd700 1 --2- 172.21.15.133:0/2479184891 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f27580a1150 0x7f27580a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:33.852 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.806+0000 7f27749fd700 1 -- 172.21.15.133:0/2479184891 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f2760014070 con 0x7f27700ffc20 2023-07-14T18:16:33.852 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.806+0000 7f276e7fc700 1 --2- 172.21.15.133:0/2479184891 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f27580a1150 0x7f27580a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:33.852 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.807+0000 7f276e7fc700 1 --2- 172.21.15.133:0/2479184891 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f27580a1150 0x7f27580a3600 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2770199190 tx=0x7f2764009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:33.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:33.808+0000 7f27749fd700 1 -- 172.21.15.133:0/2479184891 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f2760096b70 con 0x7f27700ffc20 2023-07-14T18:16:34.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.017+0000 7f27759ff700 1 -- 172.21.15.133:0/2479184891 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f27701a08d0 con 0x7f27700ffc20 2023-07-14T18:16:34.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.017+0000 7f27749fd700 1 -- 172.21.15.133:0/2479184891 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f27600b8e60 con 0x7f27700ffc20 2023-07-14T18:16:34.063 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2023-07-14T18:16:34.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.020+0000 7f27759ff700 1 -- 172.21.15.133:0/2479184891 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f27580a1150 msgr2=0x7f27580a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:34.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.020+0000 7f27759ff700 1 --2- 172.21.15.133:0/2479184891 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f27580a1150 0x7f27580a3600 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2770199190 tx=0x7f2764009260 comp rx=0 tx=0).stop 2023-07-14T18:16:34.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.020+0000 7f27759ff700 1 -- 172.21.15.133:0/2479184891 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27700ffc20 msgr2=0x7f2770197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:34.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.020+0000 7f27759ff700 1 --2- 172.21.15.133:0/2479184891 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27700ffc20 0x7f2770197c90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f276000ea50 tx=0x7f276000ed60 comp rx=0 tx=0).stop 2023-07-14T18:16:34.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.020+0000 7f27759ff700 1 -- 172.21.15.133:0/2479184891 shutdown_connections 2023-07-14T18:16:34.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.020+0000 7f27759ff700 1 --2- 172.21.15.133:0/2479184891 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f27580a1150 0x7f27580a3600 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:34.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.020+0000 7f27759ff700 1 --2- 172.21.15.133:0/2479184891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27701017a0 0x7f277019f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:34.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.020+0000 7f27759ff700 1 --2- 172.21.15.133:0/2479184891 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27700ffc20 0x7f2770197c90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:34.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.020+0000 7f27759ff700 1 --2- 172.21.15.133:0/2479184891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2770100e10 0x7f27701981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:34.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.020+0000 7f27759ff700 1 -- 172.21.15.133:0/2479184891 >> 172.21.15.133:0/2479184891 conn(0x7f27700fb260 msgr2=0x7f27700fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:34.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.020+0000 7f27759ff700 1 -- 172.21.15.133:0/2479184891 shutdown_connections 2023-07-14T18:16:34.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.020+0000 7f27759ff700 1 -- 172.21.15.133:0/2479184891 wait complete. 2023-07-14T18:16:34.084 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T18:16:34.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.239+0000 7fe644fff700 1 -- 172.21.15.133:0/2332239246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6401017a0 msgr2=0x7fe640108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:34.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.239+0000 7fe644fff700 1 --2- 172.21.15.133:0/2332239246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6401017a0 0x7fe640108060 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe63800b0a0 tx=0x7fe63800b3b0 comp rx=0 tx=0).stop 2023-07-14T18:16:34.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.239+0000 7fe644fff700 1 -- 172.21.15.133:0/2332239246 shutdown_connections 2023-07-14T18:16:34.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.239+0000 7fe644fff700 1 --2- 172.21.15.133:0/2332239246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6401017a0 0x7fe640108060 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:34.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.239+0000 7fe644fff700 1 --2- 172.21.15.133:0/2332239246 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6400ffc20 0x7fe640100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:34.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.239+0000 7fe644fff700 1 --2- 172.21.15.133:0/2332239246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe640100e10 0x7fe640101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:34.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.239+0000 7fe644fff700 1 -- 172.21.15.133:0/2332239246 >> 172.21.15.133:0/2332239246 conn(0x7fe6400fb260 msgr2=0x7fe6400fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:34.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.239+0000 7fe644fff700 1 -- 172.21.15.133:0/2332239246 shutdown_connections 2023-07-14T18:16:34.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.239+0000 7fe644fff700 1 -- 172.21.15.133:0/2332239246 wait complete. 2023-07-14T18:16:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.240+0000 7fe644fff700 1 Processor -- start 2023-07-14T18:16:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.240+0000 7fe644fff700 1 -- start start 2023-07-14T18:16:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.240+0000 7fe644fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6400ffc20 0x7fe640197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.240+0000 7fe644fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe640100e10 0x7fe6401981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:34.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.240+0000 7fe644fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6401017a0 0x7fe64019f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:34.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.240+0000 7fe644fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64019f7a0 con 0x7fe6401017a0 2023-07-14T18:16:34.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.240+0000 7fe644fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64019f910 con 0x7fe640100e10 2023-07-14T18:16:34.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.240+0000 7fe644fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64019fad0 con 0x7fe6400ffc20 2023-07-14T18:16:34.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.241+0000 7fe63ed9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6401017a0 0x7fe64019f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:34.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.241+0000 7fe63dd9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe640100e10 0x7fe6401981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:34.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.241+0000 7fe63ed9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6401017a0 0x7fe64019f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43790/0 (socket says 172.21.15.133:43790) 2023-07-14T18:16:34.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.241+0000 7fe63ed9d700 1 -- 172.21.15.133:0/2907001666 learned_addr learned my addr 172.21.15.133:0/2907001666 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:34.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.241+0000 7fe63e59c700 1 --2- 172.21.15.133:0/2907001666 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6400ffc20 0x7fe640197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:34.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.241+0000 7fe63ed9d700 1 -- 172.21.15.133:0/2907001666 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6400ffc20 msgr2=0x7fe640197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:34.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.241+0000 7fe63ed9d700 1 --2- 172.21.15.133:0/2907001666 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6400ffc20 0x7fe640197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:34.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.241+0000 7fe63ed9d700 1 -- 172.21.15.133:0/2907001666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe640100e10 msgr2=0x7fe6401981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:34.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.241+0000 7fe63ed9d700 1 --2- 172.21.15.133:0/2907001666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe640100e10 0x7fe6401981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:34.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.241+0000 7fe63ed9d700 1 -- 172.21.15.133:0/2907001666 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe638009d00 con 0x7fe6401017a0 2023-07-14T18:16:34.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.241+0000 7fe63ed9d700 1 --2- 172.21.15.133:0/2907001666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6401017a0 0x7fe64019f1d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe63800b780 tx=0x7fe63800bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:34.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.242+0000 7fe62f7fe700 1 -- 172.21.15.133:0/2907001666 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63800e040 con 0x7fe6401017a0 2023-07-14T18:16:34.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.242+0000 7fe62f7fe700 1 -- 172.21.15.133:0/2907001666 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe638004e60 con 0x7fe6401017a0 2023-07-14T18:16:34.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.242+0000 7fe644fff700 1 -- 172.21.15.133:0/2907001666 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe64019fd50 con 0x7fe6401017a0 2023-07-14T18:16:34.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.242+0000 7fe62f7fe700 1 -- 172.21.15.133:0/2907001666 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63801b6b0 con 0x7fe6401017a0 2023-07-14T18:16:34.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.242+0000 7fe644fff700 1 -- 172.21.15.133:0/2907001666 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6401a01d0 con 0x7fe6401017a0 2023-07-14T18:16:34.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.243+0000 7fe644fff700 1 -- 172.21.15.133:0/2907001666 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe64010b400 con 0x7fe6401017a0 2023-07-14T18:16:34.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.244+0000 7fe62f7fe700 1 -- 172.21.15.133:0/2907001666 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe63801b810 con 0x7fe6401017a0 2023-07-14T18:16:34.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.246+0000 7fe62f7fe700 1 --2- 172.21.15.133:0/2907001666 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe6280a11d0 0x7fe6280a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:34.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.246+0000 7fe62f7fe700 1 -- 172.21.15.133:0/2907001666 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe6380ce5c0 con 0x7fe6401017a0 2023-07-14T18:16:34.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.246+0000 7fe63e59c700 1 --2- 172.21.15.133:0/2907001666 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe6280a11d0 0x7fe6280a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:34.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.247+0000 7fe63e59c700 1 --2- 172.21.15.133:0/2907001666 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe6280a11d0 0x7fe6280a3680 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe640199130 tx=0x7fe630009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:34.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.250+0000 7fe62f7fe700 1 -- 172.21.15.133:0/2907001666 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fe638097a10 con 0x7fe6401017a0 2023-07-14T18:16:34.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:33 smithi161 ceph-mon[128993]: from='client.25456 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T18:16:34.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:33 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1035167011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:16:34.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:33 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3268033446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:16:34.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:33 smithi171 ceph-mon[129241]: from='client.25456 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T18:16:34.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:33 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1035167011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:16:34.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:33 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3268033446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:16:34.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:33 smithi133 ceph-mon[126384]: from='client.25456 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T18:16:34.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:33 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1035167011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:16:34.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:33 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3268033446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:16:34.494 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.449+0000 7fe644fff700 1 -- 172.21.15.133:0/2907001666 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe6401a0780 con 0x7fe6401017a0 2023-07-14T18:16:34.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.449+0000 7fe62f7fe700 1 -- 172.21.15.133:0/2907001666 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe63809b030 con 0x7fe6401017a0 2023-07-14T18:16:34.497 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-07-14T18:16:34.498 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.452+0000 7fe644fff700 1 -- 172.21.15.133:0/2907001666 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe6280a11d0 msgr2=0x7fe6280a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:34.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.452+0000 7fe644fff700 1 --2- 172.21.15.133:0/2907001666 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe6280a11d0 0x7fe6280a3680 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe640199130 tx=0x7fe630009260 comp rx=0 tx=0).stop 2023-07-14T18:16:34.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.452+0000 7fe644fff700 1 -- 172.21.15.133:0/2907001666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6401017a0 msgr2=0x7fe64019f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:34.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.452+0000 7fe644fff700 1 --2- 172.21.15.133:0/2907001666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6401017a0 0x7fe64019f1d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe63800b780 tx=0x7fe63800bd00 comp rx=0 tx=0).stop 2023-07-14T18:16:34.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.452+0000 7fe644fff700 1 -- 172.21.15.133:0/2907001666 shutdown_connections 2023-07-14T18:16:34.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.452+0000 7fe644fff700 1 --2- 172.21.15.133:0/2907001666 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe6280a11d0 0x7fe6280a3680 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:34.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.452+0000 7fe644fff700 1 --2- 172.21.15.133:0/2907001666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6401017a0 0x7fe64019f1d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:34.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.452+0000 7fe644fff700 1 --2- 172.21.15.133:0/2907001666 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6400ffc20 0x7fe640197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:34.501 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.452+0000 7fe644fff700 1 --2- 172.21.15.133:0/2907001666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe640100e10 0x7fe6401981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:34.501 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.452+0000 7fe644fff700 1 -- 172.21.15.133:0/2907001666 >> 172.21.15.133:0/2907001666 conn(0x7fe6400fb260 msgr2=0x7fe6400fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:34.501 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.452+0000 7fe644fff700 1 -- 172.21.15.133:0/2907001666 shutdown_connections 2023-07-14T18:16:34.501 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.452+0000 7fe644fff700 1 -- 172.21.15.133:0/2907001666 wait complete. 2023-07-14T18:16:34.516 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T18:16:34.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.669+0000 7f1d6cfff700 1 -- 172.21.15.133:0/1750943194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d68102fa0 msgr2=0x7f1d68105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:34.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.669+0000 7f1d6cfff700 1 --2- 172.21.15.133:0/1750943194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d68102fa0 0x7f1d68105380 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009990 tx=0x7f1d5c009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:34.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.669+0000 7f1d6cfff700 1 -- 172.21.15.133:0/1750943194 shutdown_connections 2023-07-14T18:16:34.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.669+0000 7f1d6cfff700 1 --2- 172.21.15.133:0/1750943194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d681058c0 0x7f1d68107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:34.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.669+0000 7f1d6cfff700 1 --2- 172.21.15.133:0/1750943194 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1d68100680 0x7f1d68102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:34.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.669+0000 7f1d6cfff700 1 --2- 172.21.15.133:0/1750943194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d68102fa0 0x7f1d68105380 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:34.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.669+0000 7f1d6cfff700 1 -- 172.21.15.133:0/1750943194 >> 172.21.15.133:0/1750943194 conn(0x7f1d680fa100 msgr2=0x7f1d680fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:34.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.670+0000 7f1d6cfff700 1 -- 172.21.15.133:0/1750943194 shutdown_connections 2023-07-14T18:16:34.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.670+0000 7f1d6cfff700 1 -- 172.21.15.133:0/1750943194 wait complete. 2023-07-14T18:16:34.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.671+0000 7f1d6cfff700 1 Processor -- start 2023-07-14T18:16:34.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.671+0000 7f1d6cfff700 1 -- start start 2023-07-14T18:16:34.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.671+0000 7f1d6cfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d68100680 0x7f1d680ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:34.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.671+0000 7f1d6cfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d68102fa0 0x7f1d680fe630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:34.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.671+0000 7f1d6cfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1d681058c0 0x7f1d680feb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:34.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.671+0000 7f1d6cfff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d680ff310 con 0x7f1d68100680 2023-07-14T18:16:34.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.671+0000 7f1d6cfff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d680ff480 con 0x7f1d68102fa0 2023-07-14T18:16:34.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.671+0000 7f1d6cfff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6819f960 con 0x7f1d681058c0 2023-07-14T18:16:34.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.671+0000 7f1d6659c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d68100680 0x7f1d680ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:34.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.671+0000 7f1d6659c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d68100680 0x7f1d680ffff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43810/0 (socket says 172.21.15.133:43810) 2023-07-14T18:16:34.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.671+0000 7f1d6659c700 1 -- 172.21.15.133:0/11483668 learned_addr learned my addr 172.21.15.133:0/11483668 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:34.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.671+0000 7f1d66d9d700 1 --2- 172.21.15.133:0/11483668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1d681058c0 0x7f1d680feb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:34.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.672+0000 7f1d65d9b700 1 --2- 172.21.15.133:0/11483668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d68102fa0 0x7f1d680fe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:34.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.672+0000 7f1d6659c700 1 -- 172.21.15.133:0/11483668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1d681058c0 msgr2=0x7f1d680feb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:34.721 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.672+0000 7f1d6659c700 1 --2- 172.21.15.133:0/11483668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1d681058c0 0x7f1d680feb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:34.721 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.672+0000 7f1d6659c700 1 -- 172.21.15.133:0/11483668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d68102fa0 msgr2=0x7f1d680fe630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:34.721 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.672+0000 7f1d6659c700 1 --2- 172.21.15.133:0/11483668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d68102fa0 0x7f1d680fe630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:34.721 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.672+0000 7f1d6659c700 1 -- 172.21.15.133:0/11483668 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5c0095f0 con 0x7f1d68100680 2023-07-14T18:16:34.721 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.672+0000 7f1d6659c700 1 --2- 172.21.15.133:0/11483668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d68100680 0x7f1d680ffff0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800e9a0 tx=0x7f1d5800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:34.721 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.672+0000 7f1d577fe700 1 -- 172.21.15.133:0/11483668 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d58009790 con 0x7f1d68100680 2023-07-14T18:16:34.721 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.672+0000 7f1d577fe700 1 -- 172.21.15.133:0/11483668 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d5800cbd0 con 0x7f1d68100680 2023-07-14T18:16:34.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.672+0000 7f1d577fe700 1 -- 172.21.15.133:0/11483668 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d58018680 con 0x7f1d68100680 2023-07-14T18:16:34.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.672+0000 7f1d6cfff700 1 -- 172.21.15.133:0/11483668 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d6819fc40 con 0x7f1d68100680 2023-07-14T18:16:34.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.672+0000 7f1d6cfff700 1 -- 172.21.15.133:0/11483668 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d681a0100 con 0x7f1d68100680 2023-07-14T18:16:34.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.673+0000 7f1d6cfff700 1 -- 172.21.15.133:0/11483668 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d680fbc70 con 0x7f1d68100680 2023-07-14T18:16:34.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.675+0000 7f1d577fe700 1 -- 172.21.15.133:0/11483668 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f1d5800cd40 con 0x7f1d68100680 2023-07-14T18:16:34.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.677+0000 7f1d577fe700 1 --2- 172.21.15.133:0/11483668 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f1d500a1220 0x7f1d500a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:34.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.677+0000 7f1d577fe700 1 -- 172.21.15.133:0/11483668 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1d58014070 con 0x7f1d68100680 2023-07-14T18:16:34.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.677+0000 7f1d65d9b700 1 --2- 172.21.15.133:0/11483668 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f1d500a1220 0x7f1d500a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:34.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.678+0000 7f1d65d9b700 1 --2- 172.21.15.133:0/11483668 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f1d500a1220 0x7f1d500a36d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00b2e0 tx=0x7f1d5c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:34.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.680+0000 7f1d577fe700 1 -- 172.21.15.133:0/11483668 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f1d58051780 con 0x7f1d68100680 2023-07-14T18:16:34.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.921+0000 7f1d6cfff700 1 -- 172.21.15.133:0/11483668 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d6802c8e0 con 0x7f1d68100680 2023-07-14T18:16:34.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.922+0000 7f1d577fe700 1 -- 172.21.15.133:0/11483668 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10853 (secure 0 0 0) 0x7f1d58096280 con 0x7f1d68100680 2023-07-14T18:16:34.968 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:34.968 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3168151271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3168151271},{"type":"v1","addr":"172.21.15.133:6837","nonce":3168151271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14544,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/3093144261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3093144261},{"type":"v1","addr":"172.21.15.133:6839","nonce":3093144261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14556,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/3282902068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":3282902068},{"type":"v1","addr":"172.21.15.133:6841","nonce":3282902068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/4023127259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4023127259},{"type":"v1","addr":"172.21.15.161:6833","nonce":4023127259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2722373740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2722373740},{"type":"v1","addr":"172.21.15.171:6833","nonce":2722373740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2399005074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2399005074},{"type":"v1","addr":"172.21.15.133:6835","nonce":2399005074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":26,"flags":178,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"balance_automate":true},"ever_allowed_features":160,"explicitly_allowed_features":160,"created":"2023-07-14T18:14:17.440873+0000","modified":"2023-07-14T18:16:25.929058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24557,"mds_1":24541,"mds_2":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3725502564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3725502564},{"type":"v1","addr":"172.21.15.161:6835","nonce":3725502564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6835/1877362116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1877362116},{"type":"v1","addr":"172.21.15.171:6835","nonce":1877362116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/3458064798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3458064798},{"type":"v1","addr":"172.21.15.161:6837","nonce":3458064798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/106205332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":106205332},{"type":"v1","addr":"172.21.15.171:6837","nonce":106205332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.171:6839/816201198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":816201198},{"type":"v1","addr":"172.21.15.171:6839","nonce":816201198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/2503865871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2503865871},{"type":"v1","addr":"172.21.15.161:6839","nonce":2503865871}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T18:16:34.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.925+0000 7f1d6cfff700 1 -- 172.21.15.133:0/11483668 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f1d500a1220 msgr2=0x7f1d500a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:34.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.925+0000 7f1d6cfff700 1 --2- 172.21.15.133:0/11483668 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f1d500a1220 0x7f1d500a36d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00b2e0 tx=0x7f1d5c005fb0 comp rx=0 tx=0).stop 2023-07-14T18:16:34.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.925+0000 7f1d6cfff700 1 -- 172.21.15.133:0/11483668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d68100680 msgr2=0x7f1d680ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:34.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.925+0000 7f1d6cfff700 1 --2- 172.21.15.133:0/11483668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d68100680 0x7f1d680ffff0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800e9a0 tx=0x7f1d5800ecb0 comp rx=0 tx=0).stop 2023-07-14T18:16:34.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.925+0000 7f1d6cfff700 1 -- 172.21.15.133:0/11483668 shutdown_connections 2023-07-14T18:16:34.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.925+0000 7f1d6cfff700 1 --2- 172.21.15.133:0/11483668 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f1d500a1220 0x7f1d500a36d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:34.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.925+0000 7f1d6cfff700 1 --2- 172.21.15.133:0/11483668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1d681058c0 0x7f1d680feb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:34.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.925+0000 7f1d6cfff700 1 --2- 172.21.15.133:0/11483668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d68100680 0x7f1d680ffff0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:34.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.925+0000 7f1d6cfff700 1 --2- 172.21.15.133:0/11483668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d68102fa0 0x7f1d680fe630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:34.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.925+0000 7f1d6cfff700 1 -- 172.21.15.133:0/11483668 >> 172.21.15.133:0/11483668 conn(0x7f1d680fa100 msgr2=0x7f1d681012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:34.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.925+0000 7f1d6cfff700 1 -- 172.21.15.133:0/11483668 shutdown_connections 2023-07-14T18:16:34.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:34.925+0000 7f1d6cfff700 1 -- 172.21.15.133:0/11483668 wait complete. 2023-07-14T18:16:34.972 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 26 2023-07-14T18:16:34.989 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T18:16:35.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.143+0000 7fb300fff700 1 -- 172.21.15.133:0/3118775538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2fc101790 msgr2=0x7fb2fc108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:35.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.143+0000 7fb300fff700 1 --2- 172.21.15.133:0/3118775538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2fc101790 0x7fb2fc108050 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb2f400b0a0 tx=0x7fb2f400b3b0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.143+0000 7fb300fff700 1 -- 172.21.15.133:0/3118775538 shutdown_connections 2023-07-14T18:16:35.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.143+0000 7fb300fff700 1 --2- 172.21.15.133:0/3118775538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2fc0ffc10 0x7fb2fc100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.143+0000 7fb300fff700 1 --2- 172.21.15.133:0/3118775538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2fc101790 0x7fb2fc108050 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.143+0000 7fb300fff700 1 --2- 172.21.15.133:0/3118775538 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2fc100e00 0x7fb2fc101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.143+0000 7fb300fff700 1 -- 172.21.15.133:0/3118775538 >> 172.21.15.133:0/3118775538 conn(0x7fb2fc0fb230 msgr2=0x7fb2fc0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:35.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.143+0000 7fb300fff700 1 -- 172.21.15.133:0/3118775538 shutdown_connections 2023-07-14T18:16:35.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.144+0000 7fb300fff700 1 -- 172.21.15.133:0/3118775538 wait complete. 2023-07-14T18:16:35.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.145+0000 7fb300fff700 1 Processor -- start 2023-07-14T18:16:35.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.145+0000 7fb300fff700 1 -- start start 2023-07-14T18:16:35.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.145+0000 7fb300fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2fc0ffc10 0x7fb2fc197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:35.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.145+0000 7fb300fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2fc100e00 0x7fb2fc198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:35.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.145+0000 7fb300fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2fc101790 0x7fb2fc19f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:35.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.145+0000 7fb300fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc19f770 con 0x7fb2fc0ffc10 2023-07-14T18:16:35.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.145+0000 7fb300fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc19f8e0 con 0x7fb2fc101790 2023-07-14T18:16:35.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.145+0000 7fb300fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc19faa0 con 0x7fb2fc100e00 2023-07-14T18:16:35.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.145+0000 7fb2fa59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2fc0ffc10 0x7fb2fc197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:35.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.145+0000 7fb2f9d9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2fc100e00 0x7fb2fc198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:35.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.145+0000 7fb2f9d9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2fc100e00 0x7fb2fc198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:44366/0 (socket says 172.21.15.133:44366) 2023-07-14T18:16:35.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.146+0000 7fb2f9d9b700 1 -- 172.21.15.133:0/2099513446 learned_addr learned my addr 172.21.15.133:0/2099513446 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:35.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.146+0000 7fb2fad9d700 1 --2- 172.21.15.133:0/2099513446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2fc101790 0x7fb2fc19f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:35.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.146+0000 7fb2fa59c700 1 -- 172.21.15.133:0/2099513446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2fc100e00 msgr2=0x7fb2fc198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:35.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.146+0000 7fb2fa59c700 1 --2- 172.21.15.133:0/2099513446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2fc100e00 0x7fb2fc198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.146+0000 7fb2fa59c700 1 -- 172.21.15.133:0/2099513446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2fc101790 msgr2=0x7fb2fc19f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:35.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.146+0000 7fb2fa59c700 1 --2- 172.21.15.133:0/2099513446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2fc101790 0x7fb2fc19f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.146+0000 7fb2fa59c700 1 -- 172.21.15.133:0/2099513446 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec009520 con 0x7fb2fc0ffc10 2023-07-14T18:16:35.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.146+0000 7fb2fa59c700 1 --2- 172.21.15.133:0/2099513446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2fc0ffc10 0x7fb2fc197c30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec00e9c0 tx=0x7fb2ec00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:35.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.146+0000 7fb2eb7fe700 1 -- 172.21.15.133:0/2099513446 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec00cb70 con 0x7fb2fc0ffc10 2023-07-14T18:16:35.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.146+0000 7fb2eb7fe700 1 -- 172.21.15.133:0/2099513446 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2ec00ee60 con 0x7fb2fc0ffc10 2023-07-14T18:16:35.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.146+0000 7fb300fff700 1 -- 172.21.15.133:0/2099513446 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f4009d00 con 0x7fb2fc0ffc10 2023-07-14T18:16:35.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.146+0000 7fb2eb7fe700 1 -- 172.21.15.133:0/2099513446 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec018680 con 0x7fb2fc0ffc10 2023-07-14T18:16:35.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.147+0000 7fb300fff700 1 -- 172.21.15.133:0/2099513446 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2fc1a0100 con 0x7fb2fc0ffc10 2023-07-14T18:16:35.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.148+0000 7fb300fff700 1 -- 172.21.15.133:0/2099513446 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2fc10b440 con 0x7fb2fc0ffc10 2023-07-14T18:16:35.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.150+0000 7fb2eb7fe700 1 -- 172.21.15.133:0/2099513446 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fb2ec010430 con 0x7fb2fc0ffc10 2023-07-14T18:16:35.197 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.152+0000 7fb2eb7fe700 1 --2- 172.21.15.133:0/2099513446 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb2e40a11a0 0x7fb2e40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:35.197 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.152+0000 7fb2eb7fe700 1 -- 172.21.15.133:0/2099513446 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb2ec014070 con 0x7fb2fc0ffc10 2023-07-14T18:16:35.198 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.152+0000 7fb2f9d9b700 1 --2- 172.21.15.133:0/2099513446 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb2e40a11a0 0x7fb2e40a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:35.198 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.153+0000 7fb2f9d9b700 1 --2- 172.21.15.133:0/2099513446 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb2e40a11a0 0x7fb2e40a3650 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0009d20 tx=0x7fb2f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:35.200 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.155+0000 7fb2eb7fe700 1 -- 172.21.15.133:0/2099513446 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fb2ec096410 con 0x7fb2fc0ffc10 2023-07-14T18:16:35.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:34 smithi161 ceph-mon[128993]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 853 B/s wr, 8 op/s 2023-07-14T18:16:35.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:34 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2479184891' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:16:35.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:34 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2907001666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:16:35.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:34 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/11483668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:16:35.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:34 smithi171 ceph-mon[129241]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 853 B/s wr, 8 op/s 2023-07-14T18:16:35.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:34 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2479184891' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:16:35.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:34 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2907001666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:16:35.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:34 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/11483668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:16:35.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:34 smithi133 ceph-mon[126384]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 853 B/s wr, 8 op/s 2023-07-14T18:16:35.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:34 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2479184891' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:16:35.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:34 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2907001666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:16:35.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:34 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/11483668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:16:35.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.378+0000 7fb300fff700 1 -- 172.21.15.133:0/2099513446 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2fc1a06b0 con 0x7fb2fc0ffc10 2023-07-14T18:16:35.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.380+0000 7fb2eb7fe700 1 -- 172.21.15.133:0/2099513446 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24520 (secure 0 0 0) 0x7fb2ec099a30 con 0x7fb2fc0ffc10 2023-07-14T18:16:35.425 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:35.426 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":92,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","created":"2023-07-14T18:06:49.561582+0000","modified":"2023-07-14T18:15:49.951050+0000","last_up_change":"2023-07-14T18:13:29.836099+0000","last_in_change":"2023-07-14T18:13:11.606710+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T18:10:03.945039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T18:14:15.451340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T18:14:16.892215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T18:14:20.493175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"85a29de7-d712-422a-b647-d9e90091aaf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6803","nonce":3076510801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6805","nonce":3076510801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6809","nonce":3076510801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6807","nonce":3076510801}]},"public_addr":"172.21.15.133:6803/3076510801","cluster_addr":"172.21.15.133:6805/3076510801","heartbeat_back_addr":"172.21.15.133:6809/3076510801","heartbeat_front_addr":"172.21.15.133:6807/3076510801","state":["exists","up"]},{"osd":1,"uuid":"80ad4e78-7917-495e-b119-99f7c7c36243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6801","nonce":1841901852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6803","nonce":1841901852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6807","nonce":1841901852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6805","nonce":1841901852}]},"public_addr":"172.21.15.161:6801/1841901852","cluster_addr":"172.21.15.161:6803/1841901852","heartbeat_back_addr":"172.21.15.161:6807/1841901852","heartbeat_front_addr":"172.21.15.161:6805/1841901852","state":["exists","up"]},{"osd":2,"uuid":"fc4bfd37-1fdd-47c6-a48e-6d910daf9e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6801","nonce":346103099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6803","nonce":346103099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6807","nonce":346103099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6805","nonce":346103099}]},"public_addr":"172.21.15.171:6801/346103099","cluster_addr":"172.21.15.171:6803/346103099","heartbeat_back_addr":"172.21.15.171:6807/346103099","heartbeat_front_addr":"172.21.15.171:6805/346103099","state":["exists","up"]},{"osd":3,"uuid":"4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6811","nonce":2152994143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6813","nonce":2152994143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6817","nonce":2152994143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6815","nonce":2152994143}]},"public_addr":"172.21.15.133:6811/2152994143","cluster_addr":"172.21.15.133:6813/2152994143","heartbeat_back_addr":"172.21.15.133:6817/2152994143","heartbeat_front_addr":"172.21.15.133:6815/2152994143","state":["exists","up"]},{"osd":4,"uuid":"4601a1c2-254b-4c4d-9b15-8fc952b3d2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6809","nonce":47213291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6811","nonce":47213291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6815","nonce":47213291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6813","nonce":47213291}]},"public_addr":"172.21.15.161:6809/47213291","cluster_addr":"172.21.15.161:6811/47213291","heartbeat_back_addr":"172.21.15.161:6815/47213291","heartbeat_front_addr":"172.21.15.161:6813/47213291","state":["exists","up"]},{"osd":5,"uuid":"6cca180a-9140-4e49-835b-86842dae0574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6809","nonce":777556751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6811","nonce":777556751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6815","nonce":777556751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6813","nonce":777556751}]},"public_addr":"172.21.15.171:6809/777556751","cluster_addr":"172.21.15.171:6811/777556751","heartbeat_back_addr":"172.21.15.171:6815/777556751","heartbeat_front_addr":"172.21.15.171:6813/777556751","state":["exists","up"]},{"osd":6,"uuid":"ea0299ff-7740-4494-a3a9-1503b3fc0ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6819","nonce":2884152358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6821","nonce":2884152358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6825","nonce":2884152358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6823","nonce":2884152358}]},"public_addr":"172.21.15.133:6819/2884152358","cluster_addr":"172.21.15.133:6821/2884152358","heartbeat_back_addr":"172.21.15.133:6825/2884152358","heartbeat_front_addr":"172.21.15.133:6823/2884152358","state":["exists","up"]},{"osd":7,"uuid":"b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6817","nonce":4183705749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6819","nonce":4183705749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6823","nonce":4183705749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6821","nonce":4183705749}]},"public_addr":"172.21.15.161:6817/4183705749","cluster_addr":"172.21.15.161:6819/4183705749","heartbeat_back_addr":"172.21.15.161:6823/4183705749","heartbeat_front_addr":"172.21.15.161:6821/4183705749","state":["exists","up"]},{"osd":8,"uuid":"87dffc07-bf48-48c8-b418-5c4b509686d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6817","nonce":2771494483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6819","nonce":2771494483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6823","nonce":2771494483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6821","nonce":2771494483}]},"public_addr":"172.21.15.171:6817/2771494483","cluster_addr":"172.21.15.171:6819/2771494483","heartbeat_back_addr":"172.21.15.171:6823/2771494483","heartbeat_front_addr":"172.21.15.171:6821/2771494483","state":["exists","up"]},{"osd":9,"uuid":"e0832870-79ce-4bce-8045-58b7a7f6dc69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6827","nonce":2066164149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6829","nonce":2066164149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6833","nonce":2066164149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6831","nonce":2066164149}]},"public_addr":"172.21.15.133:6827/2066164149","cluster_addr":"172.21.15.133:6829/2066164149","heartbeat_back_addr":"172.21.15.133:6833/2066164149","heartbeat_front_addr":"172.21.15.133:6831/2066164149","state":["exists","up"]},{"osd":10,"uuid":"081f0484-d134-4b2e-9c48-ef8e6a646422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6825","nonce":2162333626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6827","nonce":2162333626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6831","nonce":2162333626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6829","nonce":2162333626}]},"public_addr":"172.21.15.161:6825/2162333626","cluster_addr":"172.21.15.161:6827/2162333626","heartbeat_back_addr":"172.21.15.161:6831/2162333626","heartbeat_front_addr":"172.21.15.161:6829/2162333626","state":["exists","up"]},{"osd":11,"uuid":"4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6825","nonce":3403894313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6827","nonce":3403894313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6831","nonce":3403894313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6829","nonce":3403894313}]},"public_addr":"172.21.15.171:6825/3403894313","cluster_addr":"172.21.15.171:6827/3403894313","heartbeat_back_addr":"172.21.15.171:6831/3403894313","heartbeat_front_addr":"172.21.15.171:6829/3403894313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:17.816229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:39.638543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:00.543317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:26.988921+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:48.644694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:10.049860+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:35.986508+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:57.753846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:19.018245+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:44.460450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:06.279960+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:27.955006+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/1832569241":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6801/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:0/4219815304":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:6801/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/432253615":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1840700480":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1807775439":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1089030734":"2023-07-15T18:07:47.346704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T18:16:35.428 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.382+0000 7fb300fff700 1 -- 172.21.15.133:0/2099513446 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb2e40a11a0 msgr2=0x7fb2e40a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:35.428 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.382+0000 7fb300fff700 1 --2- 172.21.15.133:0/2099513446 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb2e40a11a0 0x7fb2e40a3650 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0009d20 tx=0x7fb2f0009260 comp rx=0 tx=0).stop 2023-07-14T18:16:35.428 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.382+0000 7fb300fff700 1 -- 172.21.15.133:0/2099513446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2fc0ffc10 msgr2=0x7fb2fc197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:35.428 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.382+0000 7fb300fff700 1 --2- 172.21.15.133:0/2099513446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2fc0ffc10 0x7fb2fc197c30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec00e9c0 tx=0x7fb2ec00ecd0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.428 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.382+0000 7fb300fff700 1 -- 172.21.15.133:0/2099513446 shutdown_connections 2023-07-14T18:16:35.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.382+0000 7fb300fff700 1 --2- 172.21.15.133:0/2099513446 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fb2e40a11a0 0x7fb2e40a3650 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.382+0000 7fb300fff700 1 --2- 172.21.15.133:0/2099513446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2fc101790 0x7fb2fc19f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.382+0000 7fb300fff700 1 --2- 172.21.15.133:0/2099513446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2fc0ffc10 0x7fb2fc197c30 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.382+0000 7fb300fff700 1 --2- 172.21.15.133:0/2099513446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2fc100e00 0x7fb2fc198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.382+0000 7fb300fff700 1 -- 172.21.15.133:0/2099513446 >> 172.21.15.133:0/2099513446 conn(0x7fb2fc0fb230 msgr2=0x7fb2fc0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:35.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.382+0000 7fb300fff700 1 -- 172.21.15.133:0/2099513446 shutdown_connections 2023-07-14T18:16:35.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.382+0000 7fb300fff700 1 -- 172.21.15.133:0/2099513446 wait complete. 2023-07-14T18:16:35.448 DEBUG:tasks.fwd_scrub:Yielding 2023-07-14T18:16:35.449 INFO:teuthology.run_tasks:Running task exec... 2023-07-14T18:16:35.460 INFO:teuthology.task.exec:Executing custom commands... 2023-07-14T18:16:35.460 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi133.front.sepia.ceph.com 2023-07-14T18:16:35.460 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2023-07-14T18:16:35.461 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-07-14T18:16:35.461 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T18:16:35.461 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T18:16:35.568 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:16:35.641 INFO:teuthology.orchestra.run.smithi133.stdout:Filesystem Size Used Avail Use% Mounted on 2023-07-14T18:16:35.642 INFO:teuthology.orchestra.run.smithi133.stdout:0@06d0f5fa-2271-11ee-9b33-001a4aab830c.cephfs=/volumes/_nogroup/sv_1/63cee60f-94c5-4324-af48-1f2bdf1c218e 1.1T 332M 1.1T 1% /tmp/cephfs 2023-07-14T18:16:35.647 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2023-07-14T18:16:35.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.635+0000 7fdcac9ff700 1 -- 172.21.15.133:0/3639779230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdca8101790 msgr2=0x7fdca8108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:35.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.635+0000 7fdcac9ff700 1 --2- 172.21.15.133:0/3639779230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdca8101790 0x7fdca8108050 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdca0009990 tx=0x7fdca0009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.635+0000 7fdcac9ff700 1 -- 172.21.15.133:0/3639779230 shutdown_connections 2023-07-14T18:16:35.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.635+0000 7fdcac9ff700 1 --2- 172.21.15.133:0/3639779230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdca8101790 0x7fdca8108050 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.635+0000 7fdcac9ff700 1 --2- 172.21.15.133:0/3639779230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdca80ffc10 0x7fdca8100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.635+0000 7fdcac9ff700 1 --2- 172.21.15.133:0/3639779230 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdca8100e00 0x7fdca8101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.635+0000 7fdcac9ff700 1 -- 172.21.15.133:0/3639779230 >> 172.21.15.133:0/3639779230 conn(0x7fdca80fb210 msgr2=0x7fdca80fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:35.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.635+0000 7fdcac9ff700 1 -- 172.21.15.133:0/3639779230 shutdown_connections 2023-07-14T18:16:35.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.635+0000 7fdcac9ff700 1 -- 172.21.15.133:0/3639779230 wait complete. 2023-07-14T18:16:35.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.636+0000 7fdcac9ff700 1 Processor -- start 2023-07-14T18:16:35.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.636+0000 7fdcac9ff700 1 -- start start 2023-07-14T18:16:35.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.637+0000 7fdcac9ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdca80ffc10 0x7fdca8197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:35.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.637+0000 7fdcac9ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdca8100e00 0x7fdca8198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:35.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.637+0000 7fdcac9ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdca8101790 0x7fdca819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:35.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.637+0000 7fdcac9ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca819f660 con 0x7fdca80ffc10 2023-07-14T18:16:35.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.637+0000 7fdcac9ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca819f7a0 con 0x7fdca8101790 2023-07-14T18:16:35.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.637+0000 7fdcac9ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca819f930 con 0x7fdca8100e00 2023-07-14T18:16:35.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.637+0000 7fdca659c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdca80ffc10 0x7fdca8197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:35.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.637+0000 7fdca5d9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdca8100e00 0x7fdca8198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:35.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.637+0000 7fdca659c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdca80ffc10 0x7fdca8197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43840/0 (socket says 172.21.15.133:43840) 2023-07-14T18:16:35.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.637+0000 7fdca659c700 1 -- 172.21.15.133:0/1579852364 learned_addr learned my addr 172.21.15.133:0/1579852364 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:35.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.637+0000 7fdca6d9d700 1 --2- 172.21.15.133:0/1579852364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdca8101790 0x7fdca819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:35.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.638+0000 7fdca5d9b700 1 -- 172.21.15.133:0/1579852364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdca8101790 msgr2=0x7fdca819f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:35.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.638+0000 7fdca5d9b700 1 --2- 172.21.15.133:0/1579852364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdca8101790 0x7fdca819f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.638+0000 7fdca5d9b700 1 -- 172.21.15.133:0/1579852364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdca80ffc10 msgr2=0x7fdca8197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:35.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.638+0000 7fdca5d9b700 1 --2- 172.21.15.133:0/1579852364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdca80ffc10 0x7fdca8197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.638+0000 7fdca5d9b700 1 -- 172.21.15.133:0/1579852364 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc98009c40 con 0x7fdca8100e00 2023-07-14T18:16:35.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.638+0000 7fdca659c700 1 --2- 172.21.15.133:0/1579852364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdca80ffc10 0x7fdca8197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:16:35.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.638+0000 7fdca6d9d700 1 --2- 172.21.15.133:0/1579852364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdca8101790 0x7fdca819f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:16:35.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.638+0000 7fdca5d9b700 1 --2- 172.21.15.133:0/1579852364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdca8100e00 0x7fdca8198020 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00d950 tx=0x7fdc9c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:35.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.638+0000 7fdc977fe700 1 -- 172.21.15.133:0/1579852364 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c00f890 con 0x7fdca8100e00 2023-07-14T18:16:35.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.638+0000 7fdcac9ff700 1 -- 172.21.15.133:0/1579852364 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca00095f0 con 0x7fdca8100e00 2023-07-14T18:16:35.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.638+0000 7fdc977fe700 1 -- 172.21.15.133:0/1579852364 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc9c010460 con 0x7fdca8100e00 2023-07-14T18:16:35.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.638+0000 7fdc977fe700 1 -- 172.21.15.133:0/1579852364 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c00e610 con 0x7fdca8100e00 2023-07-14T18:16:35.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.638+0000 7fdcac9ff700 1 -- 172.21.15.133:0/1579852364 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca819ffe0 con 0x7fdca8100e00 2023-07-14T18:16:35.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.643+0000 7fdc977fe700 1 -- 172.21.15.133:0/1579852364 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fdc9c0105d0 con 0x7fdca8100e00 2023-07-14T18:16:35.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.644+0000 7fdcc0327b80 1 Processor -- start 2023-07-14T18:16:35.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.644+0000 7fdcc0327b80 1 -- start start 2023-07-14T18:16:35.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.644+0000 7fdcc0327b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615ac50bfe0 0x5615ac50c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:35.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.644+0000 7fdcc0327b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615ac50c930 0x5615ac50cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:35.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.644+0000 7fdcc0327b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615ac50d530 0x5615ac50d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:35.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.644+0000 7fdcc0327b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ac410d30 con 0x5615ac50c930 2023-07-14T18:16:35.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.644+0000 7fdcc0327b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ac463ac0 con 0x5615ac50d530 2023-07-14T18:16:35.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.644+0000 7fdcc0327b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ac45cf50 con 0x5615ac50bfe0 2023-07-14T18:16:35.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.645+0000 7fdca6d9d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615ac50bfe0 0x5615ac50c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:35.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.645+0000 7fdca5d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615ac50d530 0x5615ac50d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:35.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.645+0000 7fdca6d9d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615ac50bfe0 0x5615ac50c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:44386/0 (socket says 172.21.15.133:44386) 2023-07-14T18:16:35.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.645+0000 7fdca6d9d700 1 -- 172.21.15.133:0/265572183 learned_addr learned my addr 172.21.15.133:0/265572183 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:35.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.645+0000 7fdca659c700 1 --2- 172.21.15.133:0/265572183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615ac50c930 0x5615ac50cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:35.694 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.645+0000 7fdca6d9d700 1 -- 172.21.15.133:0/265572183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615ac50d530 msgr2=0x5615ac50d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:35.694 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.645+0000 7fdca6d9d700 1 --2- 172.21.15.133:0/265572183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615ac50d530 0x5615ac50d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.694 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.645+0000 7fdca6d9d700 1 -- 172.21.15.133:0/265572183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615ac50c930 msgr2=0x5615ac50cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:35.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.645+0000 7fdca6d9d700 1 --2- 172.21.15.133:0/265572183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615ac50c930 0x5615ac50cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.645+0000 7fdca6d9d700 1 -- 172.21.15.133:0/265572183 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615ac28ae60 con 0x5615ac50bfe0 2023-07-14T18:16:35.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.645+0000 7fdc977fe700 1 --2- 172.21.15.133:0/1579852364 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fdc900a1150 0x7fdc900a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:35.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.645+0000 7fdca5d9b700 1 --2- 172.21.15.133:0/265572183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615ac50d530 0x5615ac50d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:16:35.696 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.646+0000 7fdca6d9d700 1 --2- 172.21.15.133:0/265572183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615ac50bfe0 0x5615ac50c3f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdca0005c90 tx=0x7fdca0005ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4608079a5a6a98b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:35.696 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.646+0000 7fdc977fe700 1 -- 172.21.15.133:0/1579852364 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdc9c0cc910 con 0x7fdca8100e00 2023-07-14T18:16:35.696 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.646+0000 7fdca659c700 1 --2- 172.21.15.133:0/1579852364 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fdc900a1150 0x7fdc900a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:35.696 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.646+0000 7fdc94ff9700 1 -- 172.21.15.133:0/265572183 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca001d070 con 0x5615ac50bfe0 2023-07-14T18:16:35.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.646+0000 7fdc94ff9700 1 -- 172.21.15.133:0/265572183 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdca001e040 con 0x5615ac50bfe0 2023-07-14T18:16:35.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.646+0000 7fdc94ff9700 1 -- 172.21.15.133:0/265572183 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca000f460 con 0x5615ac50bfe0 2023-07-14T18:16:35.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.647+0000 7fdcc0327b80 1 -- 172.21.15.133:0/265572183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615ac50bfe0 msgr2=0x5615ac50c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:35.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.647+0000 7fdca659c700 1 --2- 172.21.15.133:0/1579852364 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fdc900a1150 0x7fdc900a3600 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x5615ac50cd40 tx=0x7fdc9800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:35.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.647+0000 7fdcc0327b80 1 --2- 172.21.15.133:0/265572183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615ac50bfe0 0x5615ac50c3f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdca0005c90 tx=0x7fdca0005ee0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.647+0000 7fdcc0327b80 1 -- 172.21.15.133:0/265572183 shutdown_connections 2023-07-14T18:16:35.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.647+0000 7fdcc0327b80 1 --2- 172.21.15.133:0/265572183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615ac50d530 0x5615ac50d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.647+0000 7fdcc0327b80 1 --2- 172.21.15.133:0/265572183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615ac50bfe0 0x5615ac50c3f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.647+0000 7fdcc0327b80 1 --2- 172.21.15.133:0/265572183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615ac50c930 0x5615ac50cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.647+0000 7fdcc0327b80 1 -- 172.21.15.133:0/265572183 >> 172.21.15.133:0/265572183 conn(0x5615ac501700 msgr2=0x5615ac506a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:35.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.647+0000 7fdcc0327b80 1 -- 172.21.15.133:0/265572183 shutdown_connections 2023-07-14T18:16:35.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.648+0000 7fdcc0327b80 1 -- 172.21.15.133:0/265572183 wait complete. 2023-07-14T18:16:35.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.648+0000 7fdcc0327b80 1 -- start start 2023-07-14T18:16:35.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.648+0000 7fdcc0327b80 1 Processor -- start 2023-07-14T18:16:35.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.649+0000 7fdcc0327b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615ac50bfe0 0x5615ac5159b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:35.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.649+0000 7fdcc0327b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615ac50c930 0x5615ac515ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:35.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.649+0000 7fdcc0327b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615ac50d530 0x5615ac51cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:35.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.649+0000 7fdcc0327b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ac3fcb40 con 0x5615ac50bfe0 2023-07-14T18:16:35.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.649+0000 7fdcc0327b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ac3fc840 con 0x5615ac50c930 2023-07-14T18:16:35.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.649+0000 7fdcc0327b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ac2e09a0 con 0x5615ac50d530 2023-07-14T18:16:35.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.649+0000 7fdcc0327b80 10 client.0 _ll_register_callbacks cb 0x5615ac3fa7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:16:35.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.650+0000 7fdca659c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615ac50bfe0 0x5615ac5159b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:35.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.650+0000 7fdca659c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615ac50bfe0 0x5615ac5159b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43860/0 (socket says 172.21.15.133:43860) 2023-07-14T18:16:35.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.650+0000 7fdca659c700 1 -- 172.21.15.133:0/2461334105 learned_addr learned my addr 172.21.15.133:0/2461334105 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:35.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.650+0000 7fdca6d9d700 1 --2- 172.21.15.133:0/2461334105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615ac50d530 0x5615ac51cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:35.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.650+0000 7fdca5d9b700 1 --2- 172.21.15.133:0/2461334105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615ac50c930 0x5615ac515ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:35.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.650+0000 7fdca659c700 1 -- 172.21.15.133:0/2461334105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615ac50d530 msgr2=0x5615ac51cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:35.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.650+0000 7fdca659c700 1 --2- 172.21.15.133:0/2461334105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615ac50d530 0x5615ac51cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.650+0000 7fdca659c700 1 -- 172.21.15.133:0/2461334105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615ac50c930 msgr2=0x5615ac515ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:35.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.650+0000 7fdca659c700 1 --2- 172.21.15.133:0/2461334105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615ac50c930 0x5615ac515ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.650+0000 7fdca659c700 1 -- 172.21.15.133:0/2461334105 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca819ffe0 con 0x5615ac50bfe0 2023-07-14T18:16:35.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.651+0000 7fdca659c700 1 --2- 172.21.15.133:0/2461334105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615ac50bfe0 0x5615ac5159b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdc980169b0 tx=0x7fdc98004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b1afff4a6c8b947 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:35.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.651+0000 7fdc8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T18:16:35.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.651+0000 7fdc8dffb700 1 -- 172.21.15.133:0/2461334105 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9800e040 con 0x5615ac50bfe0 2023-07-14T18:16:35.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.651+0000 7fdc8dffb700 1 -- 172.21.15.133:0/2461334105 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdc9800e960 con 0x5615ac50bfe0 2023-07-14T18:16:35.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.651+0000 7fdcc0327b80 1 -- 172.21.15.133:0/2461334105 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615ac3b6570 con 0x5615ac50bfe0 2023-07-14T18:16:35.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.651+0000 7fdc8dffb700 1 -- 172.21.15.133:0/2461334105 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc980157a0 con 0x5615ac50bfe0 2023-07-14T18:16:35.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.651+0000 7fdcc0327b80 10 client.15492 fetch_fsmap learned FSMap version 26 2023-07-14T18:16:35.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.651+0000 7fdcc0327b80 1 -- 172.21.15.133:0/2461334105 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fdca8108980 con 0x5615ac50bfe0 2023-07-14T18:16:35.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.651+0000 7fdc8dffb700 1 -- 172.21.15.133:0/2461334105 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5615ac3b6570 con 0x5615ac50bfe0 2023-07-14T18:16:35.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.652+0000 7fdc8dffb700 1 -- 172.21.15.133:0/2461334105 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdc9801e070 con 0x5615ac50bfe0 2023-07-14T18:16:35.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.652+0000 7fdcc0327b80 10 client.15492 fetch_fsmap finished waiting for FSMap version 26 2023-07-14T18:16:35.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.652+0000 7fdcc0327b80 10 client.15492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:16:35.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.652+0000 7fdcc0327b80 20 client.15492 populate_metadata read hostname 'smithi133' 2023-07-14T18:16:35.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.652+0000 7fdcc0327b80 1 --2- 172.21.15.133:0/2461334105 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5615ac5217b0 0x5615ac523bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:35.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.652+0000 7fdcc0327b80 4 client.15492 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:16:35.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.652+0000 7fdcc0327b80 1 -- 172.21.15.133:0/2461334105 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615ac4dae20 con 0x5615ac5217b0 2023-07-14T18:16:35.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.652+0000 7fdca5d9b700 1 --2- 172.21.15.133:0/2461334105 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5615ac5217b0 0x5615ac523bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:35.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.653+0000 7fdca5d9b700 1 --2- 172.21.15.133:0/2461334105 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5615ac5217b0 0x5615ac523bc0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb88c81133ed971d server_cookie=9666787be948ada9 in_seq=0 out_seq=0 2023-07-14T18:16:35.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.653+0000 7fdc8dffb700 10 client.15492 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:16:35.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.653+0000 7fdc8dffb700 1 -- 172.21.15.133:0/2461334105 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5615ac4dae20 con 0x5615ac5217b0 2023-07-14T18:16:35.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.653+0000 7fdc8dffb700 10 client.15492 handle_command_reply: tid=0 2023-07-14T18:16:35.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.653+0000 7fdcc0327b80 1 client.15492 shutdown 2023-07-14T18:16:35.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.654+0000 7fdcc0327b80 1 -- 172.21.15.133:0/2461334105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615ac50bfe0 msgr2=0x5615ac5159b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:35.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.654+0000 7fdcc0327b80 1 --2- 172.21.15.133:0/2461334105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615ac50bfe0 0x5615ac5159b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdc980169b0 tx=0x7fdc98004b40 comp rx=0 tx=0).stop 2023-07-14T18:16:35.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.654+0000 7fdcc0327b80 1 -- 172.21.15.133:0/2461334105 shutdown_connections 2023-07-14T18:16:35.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.654+0000 7fdcc0327b80 1 --2- 172.21.15.133:0/2461334105 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5615ac5217b0 0x5615ac523bc0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.654+0000 7fdcc0327b80 1 --2- 172.21.15.133:0/2461334105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615ac50d530 0x5615ac51cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.654+0000 7fdcc0327b80 1 --2- 172.21.15.133:0/2461334105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615ac50bfe0 0x5615ac5159b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.654+0000 7fdc8dffb700 0 client.15492 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:16:35.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.654+0000 7fdcc0327b80 1 --2- 172.21.15.133:0/2461334105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615ac50c930 0x5615ac515ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.654+0000 7fdcc0327b80 1 -- 172.21.15.133:0/2461334105 >> 172.21.15.133:0/2461334105 conn(0x5615ac501700 msgr2=0x5615ac5056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:35.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.654+0000 7fdcc0327b80 1 -- 172.21.15.133:0/2461334105 shutdown_connections 2023-07-14T18:16:35.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.654+0000 7fdcc0327b80 1 -- 172.21.15.133:0/2461334105 wait complete. 2023-07-14T18:16:35.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.654+0000 7fdcc0327b80 20 client.15492 trim_cache size 0 max 16384 2023-07-14T18:16:35.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.679+0000 7fdcc0327b80 1 Processor -- start 2023-07-14T18:16:35.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.680+0000 7fdcc0327b80 1 -- start start 2023-07-14T18:16:35.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.680+0000 7fdcc0327b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615ac50bfe0 0x5615ac51aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:35.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.680+0000 7fdcc0327b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615ac50c930 0x5615ac510350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:35.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.680+0000 7fdcc0327b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615ac50d530 0x5615ac510890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:35.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.680+0000 7fdcc0327b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ac3b6570 con 0x5615ac50c930 2023-07-14T18:16:35.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.680+0000 7fdcc0327b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ac51ec20 con 0x5615ac50bfe0 2023-07-14T18:16:35.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.680+0000 7fdcc0327b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ac505d50 con 0x5615ac50d530 2023-07-14T18:16:35.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.680+0000 7fdca659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615ac50bfe0 0x5615ac51aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:35.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.680+0000 7fdca659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615ac50bfe0 0x5615ac51aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:33878/0 (socket says 172.21.15.133:33878) 2023-07-14T18:16:35.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.680+0000 7fdca659c700 1 -- 172.21.15.133:0/2451420598 learned_addr learned my addr 172.21.15.133:0/2451420598 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:35.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.680+0000 7fdca5d9b700 1 --2- 172.21.15.133:0/2451420598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615ac50c930 0x5615ac510350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:35.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.680+0000 7fdca6d9d700 1 --2- 172.21.15.133:0/2451420598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615ac50d530 0x5615ac510890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:35.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.681+0000 7fdca659c700 1 -- 172.21.15.133:0/2451420598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615ac50d530 msgr2=0x5615ac510890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:35.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.681+0000 7fdca659c700 1 --2- 172.21.15.133:0/2451420598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615ac50d530 0x5615ac510890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.681+0000 7fdca659c700 1 -- 172.21.15.133:0/2451420598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615ac50c930 msgr2=0x5615ac510350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:35.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.681+0000 7fdca659c700 1 --2- 172.21.15.133:0/2451420598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615ac50c930 0x5615ac510350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.681+0000 7fdca659c700 1 -- 172.21.15.133:0/2451420598 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615ac273ce0 con 0x5615ac50bfe0 2023-07-14T18:16:35.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.681+0000 7fdca659c700 1 --2- 172.21.15.133:0/2451420598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615ac50bfe0 0x5615ac51aeb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdc98016980 tx=0x7fdc980224f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99cbbb74b5d3781f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:35.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.681+0000 7fdc957fa700 1 -- 172.21.15.133:0/2451420598 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9800e040 con 0x5615ac50bfe0 2023-07-14T18:16:35.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.681+0000 7fdc957fa700 1 -- 172.21.15.133:0/2451420598 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc9800f040 con 0x5615ac50bfe0 2023-07-14T18:16:35.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.681+0000 7fdc957fa700 1 -- 172.21.15.133:0/2451420598 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc98015bd0 con 0x5615ac50bfe0 2023-07-14T18:16:35.735 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.682+0000 7fdcc0327b80 1 -- 172.21.15.133:0/2451420598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615ac50bfe0 msgr2=0x5615ac51aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:35.735 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.682+0000 7fdcc0327b80 1 --2- 172.21.15.133:0/2451420598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615ac50bfe0 0x5615ac51aeb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdc98016980 tx=0x7fdc980224f0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.735 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.682+0000 7fdcc0327b80 1 -- 172.21.15.133:0/2451420598 shutdown_connections 2023-07-14T18:16:35.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.682+0000 7fdcc0327b80 1 --2- 172.21.15.133:0/2451420598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615ac50bfe0 0x5615ac51aeb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.682+0000 7fdcc0327b80 1 --2- 172.21.15.133:0/2451420598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615ac50d530 0x5615ac510890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.682+0000 7fdcc0327b80 1 --2- 172.21.15.133:0/2451420598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615ac50c930 0x5615ac510350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.682+0000 7fdcc0327b80 1 -- 172.21.15.133:0/2451420598 >> 172.21.15.133:0/2451420598 conn(0x5615ac501700 msgr2=0x5615ac513820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:35.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.682+0000 7fdcc0327b80 1 -- 172.21.15.133:0/2451420598 shutdown_connections 2023-07-14T18:16:35.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.682+0000 7fdcc0327b80 1 -- 172.21.15.133:0/2451420598 wait complete. 2023-07-14T18:16:35.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.683+0000 7fdcc0327b80 1 -- start start 2023-07-14T18:16:35.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.683+0000 7fdcc0327b80 1 Processor -- start 2023-07-14T18:16:35.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.684+0000 7fdcc0327b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615ac50bfe0 0x5615ac51a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:35.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.684+0000 7fdcc0327b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615ac50c930 0x5615ac51a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:35.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.684+0000 7fdcc0327b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615ac50d530 0x5615ac575640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:35.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.684+0000 7fdcc0327b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ac575b80 con 0x5615ac50c930 2023-07-14T18:16:35.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.684+0000 7fdcc0327b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ac575cc0 con 0x5615ac50bfe0 2023-07-14T18:16:35.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.684+0000 7fdcc0327b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ac575e50 con 0x5615ac50d530 2023-07-14T18:16:35.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.684+0000 7fdcc0327b80 10 client.0 _ll_register_callbacks cb 0x7fdc98014ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:16:35.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.684+0000 7fdca659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615ac50bfe0 0x5615ac51a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:35.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.684+0000 7fdca659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615ac50bfe0 0x5615ac51a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:33882/0 (socket says 172.21.15.133:33882) 2023-07-14T18:16:35.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.684+0000 7fdca659c700 1 -- 172.21.15.133:0/2434057034 learned_addr learned my addr 172.21.15.133:0/2434057034 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:35.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.684+0000 7fdca6d9d700 1 --2- 172.21.15.133:0/2434057034 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615ac50d530 0x5615ac575640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:35.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.684+0000 7fdca5d9b700 1 --2- 172.21.15.133:0/2434057034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615ac50c930 0x5615ac51a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:35.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.685+0000 7fdca659c700 1 -- 172.21.15.133:0/2434057034 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615ac50d530 msgr2=0x5615ac575640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:35.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.685+0000 7fdca659c700 1 --2- 172.21.15.133:0/2434057034 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615ac50d530 0x5615ac575640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.685+0000 7fdca659c700 1 -- 172.21.15.133:0/2434057034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615ac50c930 msgr2=0x5615ac51a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:35.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.685+0000 7fdca659c700 1 --2- 172.21.15.133:0/2434057034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615ac50c930 0x5615ac51a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.685+0000 7fdca659c700 1 -- 172.21.15.133:0/2434057034 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc98030040 con 0x5615ac50bfe0 2023-07-14T18:16:35.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.685+0000 7fdca659c700 1 --2- 172.21.15.133:0/2434057034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615ac50bfe0 0x5615ac51a1e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdc9802bea0 tx=0x7fdc9800e0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d94ee08badb7fbdd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:35.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.685+0000 7fdc8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:16:35.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.685+0000 7fdc8dffb700 1 -- 172.21.15.133:0/2434057034 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc980151f0 con 0x5615ac50bfe0 2023-07-14T18:16:35.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.685+0000 7fdcc0327b80 1 -- 172.21.15.133:0/2434057034 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdca001e040 con 0x5615ac50bfe0 2023-07-14T18:16:35.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.685+0000 7fdc8dffb700 1 -- 172.21.15.133:0/2434057034 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdc98015bd0 con 0x5615ac50bfe0 2023-07-14T18:16:35.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.686+0000 7fdc8dffb700 1 -- 172.21.15.133:0/2434057034 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc98015d90 con 0x5615ac50bfe0 2023-07-14T18:16:35.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.686+0000 7fdcc0327b80 10 client.25484 fetch_fsmap learned FSMap version 26 2023-07-14T18:16:35.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.686+0000 7fdcc0327b80 1 -- 172.21.15.133:0/2434057034 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5615ac2b12d0 con 0x5615ac50bfe0 2023-07-14T18:16:35.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.686+0000 7fdc8dffb700 1 -- 172.21.15.133:0/2434057034 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc9804b080 con 0x5615ac50bfe0 2023-07-14T18:16:35.743 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.687+0000 7fdc8dffb700 1 -- 172.21.15.133:0/2434057034 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdc98040040 con 0x5615ac50bfe0 2023-07-14T18:16:35.743 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.687+0000 7fdcc0327b80 10 client.25484 fetch_fsmap finished waiting for FSMap version 26 2023-07-14T18:16:35.743 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.687+0000 7fdcc0327b80 10 client.25484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:16:35.743 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.687+0000 7fdcc0327b80 20 client.25484 populate_metadata read hostname 'smithi133' 2023-07-14T18:16:35.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.687+0000 7fdcc0327b80 1 --2- 172.21.15.133:0/2434057034 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5615ac5773d0 0x5615ac5797b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:35.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.687+0000 7fdcc0327b80 4 client.25484 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T18:16:35.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.687+0000 7fdcc0327b80 1 -- 172.21.15.133:0/2434057034 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5615ac455f30 con 0x5615ac5773d0 2023-07-14T18:16:35.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.687+0000 7fdca5d9b700 1 --2- 172.21.15.133:0/2434057034 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5615ac5773d0 0x5615ac5797b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:35.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.688+0000 7fdca5d9b700 1 --2- 172.21.15.133:0/2434057034 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5615ac5773d0 0x5615ac5797b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=144a3e9d5b81087e server_cookie=7a9afa093ec0e941 in_seq=0 out_seq=0 2023-07-14T18:16:35.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.688+0000 7fdc8dffb700 10 client.25484 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:16:35.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.689+0000 7fdc8dffb700 1 -- 172.21.15.133:0/2434057034 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5615ac455f30 con 0x5615ac5773d0 2023-07-14T18:16:35.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.689+0000 7fdc8dffb700 10 client.25484 handle_command_reply: tid=0 2023-07-14T18:16:35.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.689+0000 7fdcc0327b80 1 client.25484 shutdown 2023-07-14T18:16:35.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.689+0000 7fdcc0327b80 1 -- 172.21.15.133:0/2434057034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615ac50bfe0 msgr2=0x5615ac51a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:35.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.689+0000 7fdcc0327b80 1 --2- 172.21.15.133:0/2434057034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615ac50bfe0 0x5615ac51a1e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdc9802bea0 tx=0x7fdc9800e0d0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.689+0000 7fdcc0327b80 1 -- 172.21.15.133:0/2434057034 shutdown_connections 2023-07-14T18:16:35.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.690+0000 7fdcc0327b80 1 --2- 172.21.15.133:0/2434057034 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5615ac5773d0 0x5615ac5797b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.690+0000 7fdcc0327b80 1 --2- 172.21.15.133:0/2434057034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615ac50bfe0 0x5615ac51a1e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.690+0000 7fdcc0327b80 1 --2- 172.21.15.133:0/2434057034 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615ac50d530 0x5615ac575640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.690+0000 7fdc8dffb700 0 client.25484 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:16:35.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.690+0000 7fdcc0327b80 1 --2- 172.21.15.133:0/2434057034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615ac50c930 0x5615ac51a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.690+0000 7fdcc0327b80 1 -- 172.21.15.133:0/2434057034 >> 172.21.15.133:0/2434057034 conn(0x5615ac501700 msgr2=0x5615ac510350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:35.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.690+0000 7fdcc0327b80 1 -- 172.21.15.133:0/2434057034 shutdown_connections 2023-07-14T18:16:35.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.690+0000 7fdcc0327b80 1 -- 172.21.15.133:0/2434057034 wait complete. 2023-07-14T18:16:35.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.690+0000 7fdcc0327b80 20 client.25484 trim_cache size 0 max 16384 2023-07-14T18:16:35.747 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:16:35.747 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2023-07-14T18:16:35.747 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "ec0b0c62-b9cb-4251-92c5-5ce8da5879ee", 2023-07-14T18:16:35.747 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2023-07-14T18:16:35.747 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:16:35.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.691+0000 7fdcac9ff700 1 -- 172.21.15.133:0/1579852364 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fdc900a1150 msgr2=0x7fdc900a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:35.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.691+0000 7fdcac9ff700 1 --2- 172.21.15.133:0/1579852364 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fdc900a1150 0x7fdc900a3600 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x5615ac50cd40 tx=0x7fdc9800b040 comp rx=0 tx=0).stop 2023-07-14T18:16:35.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.691+0000 7fdcac9ff700 1 -- 172.21.15.133:0/1579852364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdca8100e00 msgr2=0x7fdca8198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:35.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.691+0000 7fdcac9ff700 1 --2- 172.21.15.133:0/1579852364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdca8100e00 0x7fdca8198020 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00d950 tx=0x7fdc9c00dc60 comp rx=0 tx=0).stop 2023-07-14T18:16:35.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.691+0000 7fdcac9ff700 1 -- 172.21.15.133:0/1579852364 shutdown_connections 2023-07-14T18:16:35.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.691+0000 7fdcac9ff700 1 --2- 172.21.15.133:0/1579852364 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fdc900a1150 0x7fdc900a3600 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.691+0000 7fdcac9ff700 1 --2- 172.21.15.133:0/1579852364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdca8101790 0x7fdca819f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.691+0000 7fdcac9ff700 1 --2- 172.21.15.133:0/1579852364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdca80ffc10 0x7fdca8197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.691+0000 7fdcac9ff700 1 --2- 172.21.15.133:0/1579852364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdca8100e00 0x7fdca8198020 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:35.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.691+0000 7fdcac9ff700 1 -- 172.21.15.133:0/1579852364 >> 172.21.15.133:0/1579852364 conn(0x7fdca80fb210 msgr2=0x7fdca8106650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:35.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.692+0000 7fdcac9ff700 1 -- 172.21.15.133:0/1579852364 shutdown_connections 2023-07-14T18:16:35.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.692+0000 7fdcac9ff700 1 -- 172.21.15.133:0/1579852364 wait complete. 2023-07-14T18:16:35.758 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:16:35.801 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2023-07-14T18:16:35.911 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2023-07-14T18:16:36.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.979+0000 7f6d9d3ff700 1 -- 172.21.15.133:0/3781541849 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d980730a0 msgr2=0x7f6d98073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:36.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.979+0000 7f6d9d3ff700 1 --2- 172.21.15.133:0/3781541849 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d980730a0 0x7f6d98073510 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009990 tx=0x7f6d8c009ca0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.979+0000 7f6d9d3ff700 1 -- 172.21.15.133:0/3781541849 shutdown_connections 2023-07-14T18:16:36.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.979+0000 7f6d9d3ff700 1 --2- 172.21.15.133:0/3781541849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d98074680 0x7f6d98072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.979+0000 7f6d9d3ff700 1 --2- 172.21.15.133:0/3781541849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d98107ae0 0x7f6d98109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.979+0000 7f6d9d3ff700 1 --2- 172.21.15.133:0/3781541849 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d980730a0 0x7f6d98073510 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.979+0000 7f6d9d3ff700 1 -- 172.21.15.133:0/3781541849 >> 172.21.15.133:0/3781541849 conn(0x7f6d980fb260 msgr2=0x7f6d980fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:36.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.979+0000 7f6d9d3ff700 1 -- 172.21.15.133:0/3781541849 shutdown_connections 2023-07-14T18:16:36.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.979+0000 7f6d9d3ff700 1 -- 172.21.15.133:0/3781541849 wait complete. 2023-07-14T18:16:36.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.980+0000 7f6d9d3ff700 1 Processor -- start 2023-07-14T18:16:36.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.980+0000 7f6d9d3ff700 1 -- start start 2023-07-14T18:16:36.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.981+0000 7f6d9d3ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d980730a0 0x7f6d98197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:36.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.981+0000 7f6d9d3ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d98074680 0x7f6d98198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:36.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.981+0000 7f6d9d3ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d98107ae0 0x7f6d9819f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:36.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.981+0000 7f6d9d3ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9819f740 con 0x7f6d980730a0 2023-07-14T18:16:36.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.981+0000 7f6d9d3ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9819f880 con 0x7f6d98107ae0 2023-07-14T18:16:36.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.981+0000 7f6d9d3ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9819fa10 con 0x7f6d98074680 2023-07-14T18:16:36.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.981+0000 7f6d967fc700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d98074680 0x7f6d98198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:36.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.981+0000 7f6d967fc700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d98074680 0x7f6d98198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:44444/0 (socket says 172.21.15.133:44444) 2023-07-14T18:16:36.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.981+0000 7f6d96ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d980730a0 0x7f6d98197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:36.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.981+0000 7f6d967fc700 1 -- 172.21.15.133:0/1972269252 learned_addr learned my addr 172.21.15.133:0/1972269252 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:36.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.981+0000 7f6d977fe700 1 --2- 172.21.15.133:0/1972269252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d98107ae0 0x7f6d9819f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:36.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.981+0000 7f6d967fc700 1 -- 172.21.15.133:0/1972269252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d98107ae0 msgr2=0x7f6d9819f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:36.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.981+0000 7f6d967fc700 1 --2- 172.21.15.133:0/1972269252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d98107ae0 0x7f6d9819f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.981+0000 7f6d967fc700 1 -- 172.21.15.133:0/1972269252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d980730a0 msgr2=0x7f6d98197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:36.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.981+0000 7f6d967fc700 1 --2- 172.21.15.133:0/1972269252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d980730a0 0x7f6d98197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.981+0000 7f6d967fc700 1 -- 172.21.15.133:0/1972269252 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c0095f0 con 0x7f6d98074680 2023-07-14T18:16:36.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.982+0000 7f6d977fe700 1 --2- 172.21.15.133:0/1972269252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d98107ae0 0x7f6d9819f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:16:36.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.982+0000 7f6d967fc700 1 --2- 172.21.15.133:0/1972269252 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d98074680 0x7f6d98198100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c005720 tx=0x7f6d8c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:36.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.982+0000 7f6d7ffff700 1 -- 172.21.15.133:0/1972269252 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c01d070 con 0x7f6d98074680 2023-07-14T18:16:36.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.982+0000 7f6d7ffff700 1 -- 172.21.15.133:0/1972269252 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d8c021410 con 0x7f6d98074680 2023-07-14T18:16:36.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.982+0000 7f6d7ffff700 1 -- 172.21.15.133:0/1972269252 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c017650 con 0x7f6d98074680 2023-07-14T18:16:36.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.982+0000 7f6d9d3ff700 1 -- 172.21.15.133:0/1972269252 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d9819fc90 con 0x7f6d98074680 2023-07-14T18:16:36.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.982+0000 7f6d9d3ff700 1 -- 172.21.15.133:0/1972269252 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d981a00b0 con 0x7f6d98074680 2023-07-14T18:16:36.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.985+0000 7f6d7ffff700 1 -- 172.21.15.133:0/1972269252 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6d8c021580 con 0x7f6d98074680 2023-07-14T18:16:36.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.986+0000 7f6d7ffff700 1 --2- 172.21.15.133:0/1972269252 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6d800a11d0 0x7f6d800a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:36.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.986+0000 7f6d7ffff700 1 -- 172.21.15.133:0/1972269252 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6d8c0cdd80 con 0x7f6d98074680 2023-07-14T18:16:36.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.987+0000 7f6db0cd0b80 1 Processor -- start 2023-07-14T18:16:36.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.987+0000 7f6d96ffd700 1 --2- 172.21.15.133:0/1972269252 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6d800a11d0 0x7f6d800a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:36.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.987+0000 7f6db0cd0b80 1 -- start start 2023-07-14T18:16:36.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.987+0000 7f6db0cd0b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2fee05770 0x55e2fee05b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:36.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.987+0000 7f6db0cd0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e2fee060c0 0x55e2fee066e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:36.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.987+0000 7f6db0cd0b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2fee06cc0 0x55e2fee07130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:36.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.987+0000 7f6db0cd0b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2fed01cb0 con 0x55e2fee06cc0 2023-07-14T18:16:36.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.987+0000 7f6db0cd0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2fed54a40 con 0x55e2fee060c0 2023-07-14T18:16:36.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.987+0000 7f6db0cd0b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2fed4ded0 con 0x55e2fee05770 2023-07-14T18:16:36.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.987+0000 7f6d96ffd700 1 --2- 172.21.15.133:0/1972269252 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6d800a11d0 0x7f6d800a3680 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6d98074a50 tx=0x7f6d88009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:36.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.988+0000 7f6d96ffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2fee05770 0x55e2fee05b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:36.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.988+0000 7f6d967fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e2fee060c0 0x55e2fee066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:36.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.988+0000 7f6d96ffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2fee05770 0x55e2fee05b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:44456/0 (socket says 172.21.15.133:44456) 2023-07-14T18:16:36.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.988+0000 7f6d96ffd700 1 -- 172.21.15.133:0/647190239 learned_addr learned my addr 172.21.15.133:0/647190239 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:36.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.988+0000 7f6d977fe700 1 --2- 172.21.15.133:0/647190239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2fee06cc0 0x55e2fee07130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:36.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.988+0000 7f6d96ffd700 1 -- 172.21.15.133:0/647190239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e2fee060c0 msgr2=0x55e2fee066e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:36.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.988+0000 7f6d96ffd700 1 --2- 172.21.15.133:0/647190239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e2fee060c0 0x55e2fee066e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.988+0000 7f6d96ffd700 1 -- 172.21.15.133:0/647190239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2fee06cc0 msgr2=0x55e2fee07130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:36.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.988+0000 7f6d96ffd700 1 --2- 172.21.15.133:0/647190239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2fee06cc0 0x55e2fee07130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.988+0000 7f6d96ffd700 1 -- 172.21.15.133:0/647190239 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2feb7bde0 con 0x55e2fee05770 2023-07-14T18:16:36.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.988+0000 7f6d967fc700 1 --2- 172.21.15.133:0/647190239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e2fee060c0 0x55e2fee066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:16:36.039 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.988+0000 7f6d96ffd700 1 --2- 172.21.15.133:0/647190239 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2fee05770 0x55e2fee05b80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6d88011d20 tx=0x7f6d88014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8489858c2f1991f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:36.039 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.989+0000 7f6d7d7fa700 1 -- 172.21.15.133:0/647190239 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8800eb20 con 0x55e2fee05770 2023-07-14T18:16:36.039 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.989+0000 7f6d7d7fa700 1 -- 172.21.15.133:0/647190239 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d88021070 con 0x55e2fee05770 2023-07-14T18:16:36.039 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.989+0000 7f6d7d7fa700 1 -- 172.21.15.133:0/647190239 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88025630 con 0x55e2fee05770 2023-07-14T18:16:36.039 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.989+0000 7f6db0cd0b80 1 -- 172.21.15.133:0/647190239 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2fee05770 msgr2=0x55e2fee05b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:36.039 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.989+0000 7f6db0cd0b80 1 --2- 172.21.15.133:0/647190239 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2fee05770 0x55e2fee05b80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6d88011d20 tx=0x7f6d88014000 comp rx=0 tx=0).stop 2023-07-14T18:16:36.039 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.989+0000 7f6db0cd0b80 1 -- 172.21.15.133:0/647190239 shutdown_connections 2023-07-14T18:16:36.040 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.989+0000 7f6db0cd0b80 1 --2- 172.21.15.133:0/647190239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2fee06cc0 0x55e2fee07130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.040 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.989+0000 7f6db0cd0b80 1 --2- 172.21.15.133:0/647190239 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2fee05770 0x55e2fee05b80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.040 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.989+0000 7f6db0cd0b80 1 --2- 172.21.15.133:0/647190239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e2fee060c0 0x55e2fee066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.040 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.989+0000 7f6db0cd0b80 1 -- 172.21.15.133:0/647190239 >> 172.21.15.133:0/647190239 conn(0x55e2fedf2680 msgr2=0x55e2fedf79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:36.040 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.990+0000 7f6db0cd0b80 1 -- 172.21.15.133:0/647190239 shutdown_connections 2023-07-14T18:16:36.040 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.990+0000 7f6db0cd0b80 1 -- 172.21.15.133:0/647190239 wait complete. 2023-07-14T18:16:36.040 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.990+0000 7f6db0cd0b80 1 -- start start 2023-07-14T18:16:36.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.990+0000 7f6db0cd0b80 1 Processor -- start 2023-07-14T18:16:36.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.991+0000 7f6db0cd0b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2fee05770 0x55e2fee0f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:36.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.991+0000 7f6db0cd0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e2fee060c0 0x55e2fee0f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:36.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.991+0000 7f6db0cd0b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2fee06cc0 0x55e2fee166f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:36.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.991+0000 7f6db0cd0b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2fecedac0 con 0x55e2fee05770 2023-07-14T18:16:36.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.991+0000 7f6db0cd0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2feced7c0 con 0x55e2fee060c0 2023-07-14T18:16:36.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.991+0000 7f6db0cd0b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2febd1920 con 0x55e2fee06cc0 2023-07-14T18:16:36.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.991+0000 7f6db0cd0b80 10 client.0 _ll_register_callbacks cb 0x55e2feceb760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:16:36.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.991+0000 7f6d96ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2fee05770 0x55e2fee0f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:36.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.991+0000 7f6d977fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2fee06cc0 0x55e2fee166f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:36.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.991+0000 7f6d96ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2fee05770 0x55e2fee0f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43924/0 (socket says 172.21.15.133:43924) 2023-07-14T18:16:36.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.991+0000 7f6d96ffd700 1 -- 172.21.15.133:0/3183861508 learned_addr learned my addr 172.21.15.133:0/3183861508 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:36.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.991+0000 7f6d967fc700 1 --2- 172.21.15.133:0/3183861508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e2fee060c0 0x55e2fee0f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:36.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.991+0000 7f6d977fe700 1 -- 172.21.15.133:0/3183861508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e2fee060c0 msgr2=0x55e2fee0f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:36.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.991+0000 7f6d977fe700 1 --2- 172.21.15.133:0/3183861508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e2fee060c0 0x55e2fee0f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.991+0000 7f6d977fe700 1 -- 172.21.15.133:0/3183861508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2fee05770 msgr2=0x55e2fee0f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:36.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.991+0000 7f6d977fe700 1 --2- 172.21.15.133:0/3183861508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2fee05770 0x55e2fee0f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.991+0000 7f6d977fe700 1 -- 172.21.15.133:0/3183861508 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d981a00b0 con 0x55e2fee06cc0 2023-07-14T18:16:36.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.992+0000 7f6d977fe700 1 --2- 172.21.15.133:0/3183861508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2fee06cc0 0x55e2fee166f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6d900046a0 tx=0x7f6d900049b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a802e1a98a51b106 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:36.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.992+0000 7f6d7aff5700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:16:36.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.992+0000 7f6d7aff5700 1 -- 172.21.15.133:0/3183861508 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d90004db0 con 0x55e2fee06cc0 2023-07-14T18:16:36.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.992+0000 7f6db0cd0b80 1 -- 172.21.15.133:0/3183861508 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e2feca74f0 con 0x55e2fee06cc0 2023-07-14T18:16:36.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.992+0000 7f6d7aff5700 1 -- 172.21.15.133:0/3183861508 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6d900093a0 con 0x55e2fee06cc0 2023-07-14T18:16:36.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.992+0000 7f6d7aff5700 1 -- 172.21.15.133:0/3183861508 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d9000ea20 con 0x55e2fee06cc0 2023-07-14T18:16:36.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.992+0000 7f6db0cd0b80 10 client.25528 fetch_fsmap learned FSMap version 26 2023-07-14T18:16:36.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.992+0000 7f6db0cd0b80 1 -- 172.21.15.133:0/3183861508 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f6d88017070 con 0x55e2fee06cc0 2023-07-14T18:16:36.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.992+0000 7f6d7aff5700 1 -- 172.21.15.133:0/3183861508 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e2feca74f0 con 0x55e2fee06cc0 2023-07-14T18:16:36.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.992+0000 7f6d7aff5700 1 -- 172.21.15.133:0/3183861508 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6d9001c030 con 0x55e2fee06cc0 2023-07-14T18:16:36.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.993+0000 7f6db0cd0b80 10 client.25528 fetch_fsmap finished waiting for FSMap version 26 2023-07-14T18:16:36.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.993+0000 7f6db0cd0b80 10 client.25528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:16:36.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.993+0000 7f6db0cd0b80 20 client.25528 populate_metadata read hostname 'smithi133' 2023-07-14T18:16:36.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.993+0000 7f6db0cd0b80 1 --2- 172.21.15.133:0/3183861508 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55e2fee1b000 0x55e2fee1d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:36.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.993+0000 7f6db0cd0b80 4 client.25528 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:16:36.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.993+0000 7f6db0cd0b80 1 -- 172.21.15.133:0/3183861508 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2fedcbda0 con 0x55e2fee1b000 2023-07-14T18:16:36.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.993+0000 7f6d96ffd700 1 --2- 172.21.15.133:0/3183861508 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55e2fee1b000 0x55e2fee1d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:36.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.993+0000 7f6d96ffd700 1 --2- 172.21.15.133:0/3183861508 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55e2fee1b000 0x55e2fee1d410 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=578eb94c58151606 server_cookie=a8962ed21561b8d2 in_seq=0 out_seq=0 2023-07-14T18:16:36.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.993+0000 7f6d7aff5700 10 client.25528 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:16:36.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.994+0000 7f6d7aff5700 1 -- 172.21.15.133:0/3183861508 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55e2fedcbda0 con 0x55e2fee1b000 2023-07-14T18:16:36.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.994+0000 7f6d7aff5700 10 client.25528 handle_command_reply: tid=0 2023-07-14T18:16:36.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.994+0000 7f6db0cd0b80 1 client.25528 shutdown 2023-07-14T18:16:36.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.994+0000 7f6db0cd0b80 1 -- 172.21.15.133:0/3183861508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2fee06cc0 msgr2=0x55e2fee166f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:36.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.994+0000 7f6db0cd0b80 1 --2- 172.21.15.133:0/3183861508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2fee06cc0 0x55e2fee166f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6d900046a0 tx=0x7f6d900049b0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.994+0000 7f6db0cd0b80 1 -- 172.21.15.133:0/3183861508 shutdown_connections 2023-07-14T18:16:36.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.994+0000 7f6db0cd0b80 1 --2- 172.21.15.133:0/3183861508 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55e2fee1b000 0x55e2fee1d410 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.994+0000 7f6db0cd0b80 1 --2- 172.21.15.133:0/3183861508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2fee06cc0 0x55e2fee166f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.994+0000 7f6d7aff5700 0 client.25528 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:16:36.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.994+0000 7f6db0cd0b80 1 --2- 172.21.15.133:0/3183861508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2fee05770 0x55e2fee0f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.994+0000 7f6db0cd0b80 1 --2- 172.21.15.133:0/3183861508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e2fee060c0 0x55e2fee0f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.994+0000 7f6db0cd0b80 1 -- 172.21.15.133:0/3183861508 >> 172.21.15.133:0/3183861508 conn(0x55e2fedf2680 msgr2=0x55e2fedf6650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:36.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.995+0000 7f6db0cd0b80 1 -- 172.21.15.133:0/3183861508 shutdown_connections 2023-07-14T18:16:36.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.995+0000 7f6db0cd0b80 1 -- 172.21.15.133:0/3183861508 wait complete. 2023-07-14T18:16:36.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:35.995+0000 7f6db0cd0b80 20 client.25528 trim_cache size 0 max 16384 2023-07-14T18:16:36.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.012+0000 7f6db0cd0b80 1 Processor -- start 2023-07-14T18:16:36.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.012+0000 7f6db0cd0b80 1 -- start start 2023-07-14T18:16:36.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.013+0000 7f6db0cd0b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2fee05770 0x55e2fee21d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:36.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.013+0000 7f6db0cd0b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2fee060c0 0x55e2fee166b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:36.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.013+0000 7f6db0cd0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e2fee06cc0 0x55e2fee07090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:36.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.013+0000 7f6db0cd0b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2fedf94a0 con 0x55e2fee060c0 2023-07-14T18:16:36.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.013+0000 7f6db0cd0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2fee0cde0 con 0x55e2fee06cc0 2023-07-14T18:16:36.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.013+0000 7f6db0cd0b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2fee175d0 con 0x55e2fee05770 2023-07-14T18:16:36.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.013+0000 7f6d96ffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2fee05770 0x55e2fee21d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:36.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.013+0000 7f6d967fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2fee060c0 0x55e2fee166b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:36.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.013+0000 7f6d96ffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2fee05770 0x55e2fee21d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:44486/0 (socket says 172.21.15.133:44486) 2023-07-14T18:16:36.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.013+0000 7f6d96ffd700 1 -- 172.21.15.133:0/140646691 learned_addr learned my addr 172.21.15.133:0/140646691 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:36.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.013+0000 7f6d977fe700 1 --2- 172.21.15.133:0/140646691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e2fee06cc0 0x55e2fee07090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:36.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.014+0000 7f6d96ffd700 1 -- 172.21.15.133:0/140646691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e2fee06cc0 msgr2=0x55e2fee07090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:36.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.014+0000 7f6d96ffd700 1 --2- 172.21.15.133:0/140646691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e2fee06cc0 0x55e2fee07090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.014+0000 7f6d96ffd700 1 -- 172.21.15.133:0/140646691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2fee060c0 msgr2=0x55e2fee166b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.014+0000 7f6d96ffd700 1 --2- 172.21.15.133:0/140646691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2fee060c0 0x55e2fee166b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.014+0000 7f6d96ffd700 1 -- 172.21.15.133:0/140646691 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2feb64c60 con 0x55e2fee05770 2023-07-14T18:16:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.014+0000 7f6d96ffd700 1 --2- 172.21.15.133:0/140646691 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2fee05770 0x55e2fee21d70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800e970 tx=0x7f6d8800ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c8af1135989165 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.014+0000 7f6d7dffb700 1 -- 172.21.15.133:0/140646691 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8801dbc0 con 0x55e2fee05770 2023-07-14T18:16:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.015+0000 7f6d7dffb700 1 -- 172.21.15.133:0/140646691 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d8801dd20 con 0x55e2fee05770 2023-07-14T18:16:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.015+0000 7f6d7dffb700 1 -- 172.21.15.133:0/140646691 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8801d670 con 0x55e2fee05770 2023-07-14T18:16:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.015+0000 7f6db0cd0b80 1 -- 172.21.15.133:0/140646691 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2fee05770 msgr2=0x55e2fee21d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.015+0000 7f6db0cd0b80 1 --2- 172.21.15.133:0/140646691 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2fee05770 0x55e2fee21d70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800e970 tx=0x7f6d8800ef70 comp rx=0 tx=0).stop 2023-07-14T18:16:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.015+0000 7f6db0cd0b80 1 -- 172.21.15.133:0/140646691 shutdown_connections 2023-07-14T18:16:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.015+0000 7f6db0cd0b80 1 --2- 172.21.15.133:0/140646691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e2fee06cc0 0x55e2fee07090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.015+0000 7f6db0cd0b80 1 --2- 172.21.15.133:0/140646691 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2fee05770 0x55e2fee21d70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.015+0000 7f6db0cd0b80 1 --2- 172.21.15.133:0/140646691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2fee060c0 0x55e2fee166b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.015+0000 7f6db0cd0b80 1 -- 172.21.15.133:0/140646691 >> 172.21.15.133:0/140646691 conn(0x55e2fedf2680 msgr2=0x55e2fee16ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.015+0000 7f6db0cd0b80 1 -- 172.21.15.133:0/140646691 shutdown_connections 2023-07-14T18:16:36.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.015+0000 7f6db0cd0b80 1 -- 172.21.15.133:0/140646691 wait complete. 2023-07-14T18:16:36.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.016+0000 7f6db0cd0b80 1 -- start start 2023-07-14T18:16:36.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.016+0000 7f6db0cd0b80 1 Processor -- start 2023-07-14T18:16:36.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.017+0000 7f6db0cd0b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2fee05770 0x55e2fee15f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:36.068 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.017+0000 7f6db0cd0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e2fee060c0 0x55e2fee16460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:36.068 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.017+0000 7f6db0cd0b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2fee06cc0 0x55e2fee129c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:36.068 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.017+0000 7f6db0cd0b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2fee12f90 con 0x55e2fee06cc0 2023-07-14T18:16:36.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.017+0000 7f6db0cd0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2fee13100 con 0x55e2fee060c0 2023-07-14T18:16:36.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.017+0000 7f6db0cd0b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2fee132c0 con 0x55e2fee05770 2023-07-14T18:16:36.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.017+0000 7f6db0cd0b80 10 client.0 _ll_register_callbacks cb 0x7f6d90004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:16:36.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.017+0000 7f6d96ffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2fee05770 0x55e2fee15f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:36.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.017+0000 7f6d96ffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2fee05770 0x55e2fee15f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:44500/0 (socket says 172.21.15.133:44500) 2023-07-14T18:16:36.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.017+0000 7f6d96ffd700 1 -- 172.21.15.133:0/3726073858 learned_addr learned my addr 172.21.15.133:0/3726073858 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:16:36.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.017+0000 7f6d967fc700 1 --2- 172.21.15.133:0/3726073858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e2fee060c0 0x55e2fee16460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:36.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.017+0000 7f6d977fe700 1 --2- 172.21.15.133:0/3726073858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2fee06cc0 0x55e2fee129c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:36.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.018+0000 7f6d967fc700 1 -- 172.21.15.133:0/3726073858 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2fee05770 msgr2=0x55e2fee15f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:36.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.018+0000 7f6d967fc700 1 --2- 172.21.15.133:0/3726073858 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2fee05770 0x55e2fee15f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.018+0000 7f6d967fc700 1 -- 172.21.15.133:0/3726073858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2fee06cc0 msgr2=0x55e2fee129c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:16:36.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.018+0000 7f6d967fc700 1 --2- 172.21.15.133:0/3726073858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2fee06cc0 0x55e2fee129c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.018+0000 7f6d967fc700 1 -- 172.21.15.133:0/3726073858 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8802a040 con 0x55e2fee060c0 2023-07-14T18:16:36.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.018+0000 7f6d977fe700 1 --2- 172.21.15.133:0/3726073858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2fee06cc0 0x55e2fee129c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:16:36.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.018+0000 7f6d967fc700 1 --2- 172.21.15.133:0/3726073858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e2fee060c0 0x55e2fee16460 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0cb4f0 tx=0x7f6d8c021b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b041e62aabf380c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:16:36.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.018+0000 7f6d7aff5700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:16:36.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.018+0000 7f6d7aff5700 1 -- 172.21.15.133:0/3726073858 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c005750 con 0x55e2fee060c0 2023-07-14T18:16:36.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.018+0000 7f6d7aff5700 1 -- 172.21.15.133:0/3726073858 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6d8c09a380 con 0x55e2fee060c0 2023-07-14T18:16:36.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.018+0000 7f6db0cd0b80 1 -- 172.21.15.133:0/3726073858 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d88021070 con 0x55e2fee060c0 2023-07-14T18:16:36.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.018+0000 7f6d7aff5700 1 -- 172.21.15.133:0/3726073858 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d8c09a540 con 0x55e2fee060c0 2023-07-14T18:16:36.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.018+0000 7f6db0cd0b80 10 client.25502 fetch_fsmap learned FSMap version 26 2023-07-14T18:16:36.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.018+0000 7f6db0cd0b80 1 -- 172.21.15.133:0/3726073858 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55e2feba2250 con 0x55e2fee060c0 2023-07-14T18:16:36.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.019+0000 7f6d7aff5700 1 -- 172.21.15.133:0/3726073858 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d8c0cb080 con 0x55e2fee060c0 2023-07-14T18:16:36.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.019+0000 7f6d7aff5700 1 -- 172.21.15.133:0/3726073858 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6d8c0b3280 con 0x55e2fee060c0 2023-07-14T18:16:36.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.019+0000 7f6db0cd0b80 10 client.25502 fetch_fsmap finished waiting for FSMap version 26 2023-07-14T18:16:36.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.019+0000 7f6db0cd0b80 10 client.25502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:16:36.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.019+0000 7f6db0cd0b80 20 client.25502 populate_metadata read hostname 'smithi133' 2023-07-14T18:16:36.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.019+0000 7f6db0cd0b80 1 --2- 172.21.15.133:0/3726073858 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55e2fee6fe90 0x55e2fee722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:16:36.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.019+0000 7f6db0cd0b80 4 client.25502 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:16:36.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.019+0000 7f6db0cd0b80 1 -- 172.21.15.133:0/3726073858 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e2fed46eb0 con 0x55e2fee6fe90 2023-07-14T18:16:36.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.019+0000 7f6d96ffd700 1 --2- 172.21.15.133:0/3726073858 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55e2fee6fe90 0x55e2fee722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:16:36.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.019+0000 7f6d96ffd700 1 --2- 172.21.15.133:0/3726073858 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55e2fee6fe90 0x55e2fee722a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaaab6c130e21ea server_cookie=492208e38e644de0 in_seq=0 out_seq=0 2023-07-14T18:16:36.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.019+0000 7f6d7aff5700 10 client.25502 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:16:36.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.020+0000 7f6d7aff5700 1 -- 172.21.15.133:0/3726073858 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e2fed46eb0 con 0x55e2fee6fe90 2023-07-14T18:16:36.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.020+0000 7f6d7aff5700 10 client.25502 handle_command_reply: tid=0 2023-07-14T18:16:36.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.020+0000 7f6db0cd0b80 1 client.25502 shutdown 2023-07-14T18:16:36.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.020+0000 7f6db0cd0b80 1 -- 172.21.15.133:0/3726073858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e2fee060c0 msgr2=0x55e2fee16460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:36.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.020+0000 7f6db0cd0b80 1 --2- 172.21.15.133:0/3726073858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e2fee060c0 0x55e2fee16460 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0cb4f0 tx=0x7f6d8c021b60 comp rx=0 tx=0).stop 2023-07-14T18:16:36.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.020+0000 7f6db0cd0b80 1 -- 172.21.15.133:0/3726073858 shutdown_connections 2023-07-14T18:16:36.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.020+0000 7f6db0cd0b80 1 --2- 172.21.15.133:0/3726073858 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55e2fee6fe90 0x55e2fee722a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.020+0000 7f6db0cd0b80 1 --2- 172.21.15.133:0/3726073858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2fee06cc0 0x55e2fee129c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.020+0000 7f6db0cd0b80 1 --2- 172.21.15.133:0/3726073858 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2fee05770 0x55e2fee15f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.020+0000 7f6d7aff5700 0 client.25502 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:16:36.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.020+0000 7f6db0cd0b80 1 --2- 172.21.15.133:0/3726073858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e2fee060c0 0x55e2fee16460 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.020+0000 7f6db0cd0b80 1 -- 172.21.15.133:0/3726073858 >> 172.21.15.133:0/3726073858 conn(0x55e2fedf2680 msgr2=0x55e2fee0d030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:36.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.020+0000 7f6db0cd0b80 1 -- 172.21.15.133:0/3726073858 shutdown_connections 2023-07-14T18:16:36.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.020+0000 7f6db0cd0b80 1 -- 172.21.15.133:0/3726073858 wait complete. 2023-07-14T18:16:36.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.020+0000 7f6db0cd0b80 20 client.25502 trim_cache size 0 max 16384 2023-07-14T18:16:36.079 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:16:36.080 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T18:16:36.080 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:16:36.080 INFO:teuthology.orchestra.run.smithi133.stdout: "ec0b0c62-b9cb-4251-92c5-5ce8da5879ee": { 2023-07-14T18:16:36.080 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:16:36.080 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "ec0b0c62-b9cb-4251-92c5-5ce8da5879ee", 2023-07-14T18:16:36.080 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:16:36.081 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:16:36.081 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:16:36.081 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:16:36.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.021+0000 7f6d9d3ff700 1 -- 172.21.15.133:0/1972269252 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6d800a11d0 msgr2=0x7f6d800a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:36.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.021+0000 7f6d9d3ff700 1 --2- 172.21.15.133:0/1972269252 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6d800a11d0 0x7f6d800a3680 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6d98074a50 tx=0x7f6d88009260 comp rx=0 tx=0).stop 2023-07-14T18:16:36.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.021+0000 7f6d9d3ff700 1 -- 172.21.15.133:0/1972269252 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d98074680 msgr2=0x7f6d98198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:16:36.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.021+0000 7f6d9d3ff700 1 --2- 172.21.15.133:0/1972269252 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d98074680 0x7f6d98198100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c005720 tx=0x7f6d8c009580 comp rx=0 tx=0).stop 2023-07-14T18:16:36.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.021+0000 7f6d9d3ff700 1 -- 172.21.15.133:0/1972269252 shutdown_connections 2023-07-14T18:16:36.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.021+0000 7f6d9d3ff700 1 --2- 172.21.15.133:0/1972269252 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6d800a11d0 0x7f6d800a3680 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.021+0000 7f6d9d3ff700 1 --2- 172.21.15.133:0/1972269252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d98107ae0 0x7f6d9819f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.021+0000 7f6d9d3ff700 1 --2- 172.21.15.133:0/1972269252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d980730a0 0x7f6d98197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.021+0000 7f6d9d3ff700 1 --2- 172.21.15.133:0/1972269252 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d98074680 0x7f6d98198100 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:16:36.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.021+0000 7f6d9d3ff700 1 -- 172.21.15.133:0/1972269252 >> 172.21.15.133:0/1972269252 conn(0x7f6d980fb260 msgr2=0x7f6d980fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:16:36.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.021+0000 7f6d9d3ff700 1 -- 172.21.15.133:0/1972269252 shutdown_connections 2023-07-14T18:16:36.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:16:36.021+0000 7f6d9d3ff700 1 -- 172.21.15.133:0/1972269252 wait complete. 2023-07-14T18:16:36.088 INFO:tasks.cephfs.filesystem:scrub status for tag:ec0b0c62-b9cb-4251-92c5-5ce8da5879ee - {'path': '/', 'tag': 'ec0b0c62-b9cb-4251-92c5-5ce8da5879ee', 'options': 'recursive,force'} 2023-07-14T18:16:36.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:35 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:16:36.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:35 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2099513446' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:16:36.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:35 smithi161 ceph-mon[128993]: scrub queued for path: / 2023-07-14T18:16:36.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:35 smithi161 ceph-mon[128993]: scrub summary: idle+waiting paths [/] 2023-07-14T18:16:36.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:35 smithi161 ceph-mon[128993]: scrub summary: active paths [/] 2023-07-14T18:16:36.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:35 smithi161 ceph-mon[128993]: scrub summary: idle+waiting paths [/] 2023-07-14T18:16:36.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:35 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:16:36.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:35 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2099513446' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:16:36.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:35 smithi171 ceph-mon[129241]: scrub queued for path: / 2023-07-14T18:16:36.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:35 smithi171 ceph-mon[129241]: scrub summary: idle+waiting paths [/] 2023-07-14T18:16:36.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:35 smithi171 ceph-mon[129241]: scrub summary: active paths [/] 2023-07-14T18:16:36.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:35 smithi171 ceph-mon[129241]: scrub summary: idle+waiting paths [/] 2023-07-14T18:16:36.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:35 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:16:36.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:35 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2099513446' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:16:36.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:35 smithi133 ceph-mon[126384]: scrub queued for path: / 2023-07-14T18:16:36.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:35 smithi133 ceph-mon[126384]: scrub summary: idle+waiting paths [/] 2023-07-14T18:16:36.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:35 smithi133 ceph-mon[126384]: scrub summary: active paths [/] 2023-07-14T18:16:36.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:35 smithi133 ceph-mon[126384]: scrub summary: idle+waiting paths [/] 2023-07-14T18:16:37.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:36 smithi161 ceph-mon[128993]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2023-07-14T18:16:37.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:36 smithi171 ceph-mon[129241]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2023-07-14T18:16:37.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:36 smithi133 ceph-mon[126384]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2023-07-14T18:16:39.054 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:38 smithi133 ceph-mon[126384]: pgmap v347: 129 pgs: 129 active+clean; 646 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2023-07-14T18:16:39.054 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:38 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:16:39.054 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:38 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:16:39.054 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:38 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:16:39.055 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:38 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:16:39.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:38 smithi161 ceph-mon[128993]: pgmap v347: 129 pgs: 129 active+clean; 646 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2023-07-14T18:16:39.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:38 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:16:39.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:38 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:16:39.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:38 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:16:39.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:38 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:16:39.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:38 smithi171 ceph-mon[129241]: pgmap v347: 129 pgs: 129 active+clean; 646 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2023-07-14T18:16:39.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:38 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:16:39.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:38 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:16:39.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:38 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:16:39.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:38 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:16:39.396 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2023-07-14T18:16:39.494 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2023-07-14T18:16:39.500 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2023-07-14T18:16:39.607 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-07-14T18:16:39.705 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/: 2023-07-14T18:16:39.705 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:16:39.705 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 . 2023-07-14T18:16:39.705 INFO:teuthology.orchestra.run.smithi133.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 .. 2023-07-14T18:16:39.705 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 data 2023-07-14T18:16:39.706 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:39.706 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data: 2023-07-14T18:16:39.706 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:16:39.706 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 . 2023-07-14T18:16:39.706 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 .. 2023-07-14T18:16:39.710 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2023-07-14T18:16:39.836 INFO:teuthology.orchestra.run.smithi133.stdout:[Service] 2023-07-14T18:16:39.836 INFO:teuthology.orchestra.run.smithi133.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2023-07-14T18:16:39.837 INFO:teuthology.orchestra.run.smithi133.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2023-07-14T18:16:39.841 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2023-07-14T18:16:39.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:39 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:16:39.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:39 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:16:39.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:39 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:16:39.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:39 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:16:39.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:39 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:16:40.029 INFO:teuthology.orchestra.run.smithi133.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2023-07-14T18:16:40.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:39 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:16:40.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:39 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:16:40.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:39 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:16:40.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:39 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:16:40.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:39 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:16:40.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:39 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:16:40.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:39 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:16:40.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:39 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:16:40.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:39 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:16:40.118 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:39 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:16:41.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:40 smithi161 ceph-mon[128993]: pgmap v348: 129 pgs: 129 active+clean; 655 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2023-07-14T18:16:41.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:40 smithi171 ceph-mon[129241]: pgmap v348: 129 pgs: 129 active+clean; 655 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2023-07-14T18:16:41.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:40 smithi133 ceph-mon[126384]: pgmap v348: 129 pgs: 129 active+clean; 655 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2023-07-14T18:16:43.304 INFO:teuthology.orchestra.run.smithi133.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2023-07-14T18:16:43.305 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:42 smithi133 ceph-mon[126384]: pgmap v349: 129 pgs: 129 active+clean; 1.1 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 37 KiB/s wr, 7 op/s 2023-07-14T18:16:43.309 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-07-14T18:16:43.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:42 smithi161 ceph-mon[128993]: pgmap v349: 129 pgs: 129 active+clean; 1.1 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 37 KiB/s wr, 7 op/s 2023-07-14T18:16:43.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:42 smithi171 ceph-mon[129241]: pgmap v349: 129 pgs: 129 active+clean; 1.1 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 37 KiB/s wr, 7 op/s 2023-07-14T18:16:43.420 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/: 2023-07-14T18:16:43.420 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:16:43.420 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 . 2023-07-14T18:16:43.420 INFO:teuthology.orchestra.run.smithi133.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 .. 2023-07-14T18:16:43.421 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 14 18:16 data 2023-07-14T18:16:43.421 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:43.421 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data: 2023-07-14T18:16:43.421 INFO:teuthology.orchestra.run.smithi133.stdout:total 30 2023-07-14T18:16:43.421 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 14 18:16 . 2023-07-14T18:16:43.421 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 .. 2023-07-14T18:16:43.421 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 14 18:16 base 2023-07-14T18:16:43.422 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Jul 14 18:16 global 2023-07-14T18:16:43.422 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 log 2023-07-14T18:16:43.422 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 pg_commit_ts 2023-07-14T18:16:43.422 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 pg_dynshmem 2023-07-14T18:16:43.422 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Jul 14 18:16 pg_hba.conf 2023-07-14T18:16:43.422 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Jul 14 18:16 pg_ident.conf 2023-07-14T18:16:43.422 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 14 18:16 pg_logical 2023-07-14T18:16:43.422 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jul 14 18:16 pg_multixact 2023-07-14T18:16:43.423 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 pg_notify 2023-07-14T18:16:43.423 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 pg_replslot 2023-07-14T18:16:43.423 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 pg_serial 2023-07-14T18:16:43.423 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 pg_snapshots 2023-07-14T18:16:43.423 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 pg_stat 2023-07-14T18:16:43.423 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 pg_stat_tmp 2023-07-14T18:16:43.423 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 pg_subtrans 2023-07-14T18:16:43.424 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 pg_tblspc 2023-07-14T18:16:43.424 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 pg_twophase 2023-07-14T18:16:43.424 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 14 18:16 PG_VERSION 2023-07-14T18:16:43.424 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jul 14 18:16 pg_wal 2023-07-14T18:16:43.424 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 pg_xact 2023-07-14T18:16:43.424 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Jul 14 18:16 postgresql.auto.conf 2023-07-14T18:16:43.424 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Jul 14 18:16 postgresql.conf 2023-07-14T18:16:43.425 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:43.425 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/base: 2023-07-14T18:16:43.425 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:16:43.425 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 14 18:16 . 2023-07-14T18:16:43.425 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 14 18:16 .. 2023-07-14T18:16:43.425 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jul 14 18:16 1 2023-07-14T18:16:43.425 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jul 14 18:16 12346 2023-07-14T18:16:43.426 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jul 14 18:16 12347 2023-07-14T18:16:43.426 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:43.426 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/base/1: 2023-07-14T18:16:43.426 INFO:teuthology.orchestra.run.smithi133.stdout:total 7193 2023-07-14T18:16:43.426 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jul 14 18:16 . 2023-07-14T18:16:43.426 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 14 18:16 .. 2023-07-14T18:16:43.426 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 112 2023-07-14T18:16:43.427 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 113 2023-07-14T18:16:43.427 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jul 14 18:16 12182 2023-07-14T18:16:43.427 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12182_fsm 2023-07-14T18:16:43.427 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12182_vm 2023-07-14T18:16:43.427 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12184 2023-07-14T18:16:43.427 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12186 2023-07-14T18:16:43.427 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12187 2023-07-14T18:16:43.427 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12187_fsm 2023-07-14T18:16:43.428 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12187_vm 2023-07-14T18:16:43.428 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12189 2023-07-14T18:16:43.428 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12191 2023-07-14T18:16:43.428 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12192 2023-07-14T18:16:43.428 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12192_fsm 2023-07-14T18:16:43.428 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12192_vm 2023-07-14T18:16:43.428 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12194 2023-07-14T18:16:43.429 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12196 2023-07-14T18:16:43.429 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12197 2023-07-14T18:16:43.429 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12197_fsm 2023-07-14T18:16:43.429 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12197_vm 2023-07-14T18:16:43.429 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12199 2023-07-14T18:16:43.429 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12201 2023-07-14T18:16:43.429 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12202 2023-07-14T18:16:43.430 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12202_fsm 2023-07-14T18:16:43.430 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12202_vm 2023-07-14T18:16:43.430 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12204 2023-07-14T18:16:43.430 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12206 2023-07-14T18:16:43.430 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12207 2023-07-14T18:16:43.430 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12207_fsm 2023-07-14T18:16:43.430 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12207_vm 2023-07-14T18:16:43.430 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12209 2023-07-14T18:16:43.431 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12211 2023-07-14T18:16:43.431 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12212 2023-07-14T18:16:43.431 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12214 2023-07-14T18:16:43.431 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12216 2023-07-14T18:16:43.431 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jul 14 18:16 1247 2023-07-14T18:16:43.431 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1247_fsm 2023-07-14T18:16:43.432 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1247_vm 2023-07-14T18:16:43.432 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jul 14 18:16 1249 2023-07-14T18:16:43.432 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1249_fsm 2023-07-14T18:16:43.432 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1249_vm 2023-07-14T18:16:43.432 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jul 14 18:16 1255 2023-07-14T18:16:43.432 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1255_fsm 2023-07-14T18:16:43.432 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1255_vm 2023-07-14T18:16:43.432 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jul 14 18:16 1259 2023-07-14T18:16:43.433 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1259_fsm 2023-07-14T18:16:43.433 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1259_vm 2023-07-14T18:16:43.433 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 1417 2023-07-14T18:16:43.433 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 1417_vm 2023-07-14T18:16:43.433 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 1418 2023-07-14T18:16:43.433 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 1418_vm 2023-07-14T18:16:43.433 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 174 2023-07-14T18:16:43.434 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 175 2023-07-14T18:16:43.434 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2187 2023-07-14T18:16:43.434 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2224 2023-07-14T18:16:43.434 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2224_vm 2023-07-14T18:16:43.434 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2328 2023-07-14T18:16:43.434 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2328_vm 2023-07-14T18:16:43.434 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2336 2023-07-14T18:16:43.435 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2336_vm 2023-07-14T18:16:43.435 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2337 2023-07-14T18:16:43.435 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2600 2023-07-14T18:16:43.435 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2600_fsm 2023-07-14T18:16:43.435 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2600_vm 2023-07-14T18:16:43.435 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2601 2023-07-14T18:16:43.435 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2601_fsm 2023-07-14T18:16:43.436 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2601_vm 2023-07-14T18:16:43.436 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 14 18:16 2602 2023-07-14T18:16:43.436 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2602_fsm 2023-07-14T18:16:43.436 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2602_vm 2023-07-14T18:16:43.436 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2603 2023-07-14T18:16:43.436 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2603_fsm 2023-07-14T18:16:43.436 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2603_vm 2023-07-14T18:16:43.437 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2604 2023-07-14T18:16:43.437 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2604_vm 2023-07-14T18:16:43.437 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2605 2023-07-14T18:16:43.437 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2605_fsm 2023-07-14T18:16:43.437 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2605_vm 2023-07-14T18:16:43.437 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2606 2023-07-14T18:16:43.437 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2606_fsm 2023-07-14T18:16:43.437 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2606_vm 2023-07-14T18:16:43.438 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2607 2023-07-14T18:16:43.438 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2607_fsm 2023-07-14T18:16:43.438 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2607_vm 2023-07-14T18:16:43.438 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Jul 14 18:16 2608 2023-07-14T18:16:43.438 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2608_fsm 2023-07-14T18:16:43.438 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2608_vm 2023-07-14T18:16:43.438 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jul 14 18:16 2609 2023-07-14T18:16:43.439 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2609_fsm 2023-07-14T18:16:43.439 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2609_vm 2023-07-14T18:16:43.439 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2610 2023-07-14T18:16:43.439 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2610_fsm 2023-07-14T18:16:43.439 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2610_vm 2023-07-14T18:16:43.439 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2611 2023-07-14T18:16:43.439 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2611_vm 2023-07-14T18:16:43.439 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2612 2023-07-14T18:16:43.440 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2612_fsm 2023-07-14T18:16:43.440 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2612_vm 2023-07-14T18:16:43.440 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2613 2023-07-14T18:16:43.440 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2613_vm 2023-07-14T18:16:43.440 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2615 2023-07-14T18:16:43.440 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2615_fsm 2023-07-14T18:16:43.440 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2615_vm 2023-07-14T18:16:43.441 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2616 2023-07-14T18:16:43.441 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2616_fsm 2023-07-14T18:16:43.441 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2616_vm 2023-07-14T18:16:43.441 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jul 14 18:16 2617 2023-07-14T18:16:43.441 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2617_fsm 2023-07-14T18:16:43.441 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2617_vm 2023-07-14T18:16:43.441 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jul 14 18:16 2618 2023-07-14T18:16:43.442 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2618_fsm 2023-07-14T18:16:43.442 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2618_vm 2023-07-14T18:16:43.442 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jul 14 18:16 2619 2023-07-14T18:16:43.442 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2619_fsm 2023-07-14T18:16:43.442 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2619_vm 2023-07-14T18:16:43.442 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2620 2023-07-14T18:16:43.442 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2620_vm 2023-07-14T18:16:43.442 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2650 2023-07-14T18:16:43.443 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2651 2023-07-14T18:16:43.443 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2652 2023-07-14T18:16:43.443 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 2653 2023-07-14T18:16:43.443 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 2654 2023-07-14T18:16:43.443 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2655 2023-07-14T18:16:43.443 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2656 2023-07-14T18:16:43.444 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2657 2023-07-14T18:16:43.444 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jul 14 18:16 2658 2023-07-14T18:16:43.444 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 14 18:16 2659 2023-07-14T18:16:43.444 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2660 2023-07-14T18:16:43.444 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2661 2023-07-14T18:16:43.444 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2662 2023-07-14T18:16:43.444 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 14 18:16 2663 2023-07-14T18:16:43.445 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2664 2023-07-14T18:16:43.445 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2665 2023-07-14T18:16:43.445 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2666 2023-07-14T18:16:43.445 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2667 2023-07-14T18:16:43.445 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2668 2023-07-14T18:16:43.445 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2669 2023-07-14T18:16:43.445 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2670 2023-07-14T18:16:43.446 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 Jul 14 18:16 2673 2023-07-14T18:16:43.446 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Jul 14 18:16 2674 2023-07-14T18:16:43.446 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jul 14 18:16 2675 2023-07-14T18:16:43.446 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2678 2023-07-14T18:16:43.446 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2679 2023-07-14T18:16:43.446 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2680 2023-07-14T18:16:43.446 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2681 2023-07-14T18:16:43.447 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2682 2023-07-14T18:16:43.447 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2683 2023-07-14T18:16:43.447 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2684 2023-07-14T18:16:43.447 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2685 2023-07-14T18:16:43.447 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2686 2023-07-14T18:16:43.447 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2687 2023-07-14T18:16:43.447 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 2688 2023-07-14T18:16:43.447 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 14 18:16 2689 2023-07-14T18:16:43.448 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 14 18:16 2690 2023-07-14T18:16:43.448 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jul 14 18:16 2691 2023-07-14T18:16:43.448 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2692 2023-07-14T18:16:43.448 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2693 2023-07-14T18:16:43.448 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2696 2023-07-14T18:16:43.448 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2699 2023-07-14T18:16:43.448 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2701 2023-07-14T18:16:43.449 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2702 2023-07-14T18:16:43.449 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2703 2023-07-14T18:16:43.449 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 2704 2023-07-14T18:16:43.449 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2753 2023-07-14T18:16:43.449 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2753_fsm 2023-07-14T18:16:43.449 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2753_vm 2023-07-14T18:16:43.449 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2754 2023-07-14T18:16:43.450 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2755 2023-07-14T18:16:43.450 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2756 2023-07-14T18:16:43.450 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2757 2023-07-14T18:16:43.450 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2830 2023-07-14T18:16:43.450 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2830_vm 2023-07-14T18:16:43.450 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2831 2023-07-14T18:16:43.450 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2832 2023-07-14T18:16:43.451 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2832_vm 2023-07-14T18:16:43.451 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2833 2023-07-14T18:16:43.451 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2834 2023-07-14T18:16:43.451 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2834_vm 2023-07-14T18:16:43.451 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2835 2023-07-14T18:16:43.451 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2836 2023-07-14T18:16:43.451 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2836_vm 2023-07-14T18:16:43.452 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2837 2023-07-14T18:16:43.452 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jul 14 18:16 2838 2023-07-14T18:16:43.452 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2838_fsm 2023-07-14T18:16:43.452 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2838_vm 2023-07-14T18:16:43.452 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2839 2023-07-14T18:16:43.452 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2840 2023-07-14T18:16:43.452 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2840_fsm 2023-07-14T18:16:43.453 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2840_vm 2023-07-14T18:16:43.453 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2841 2023-07-14T18:16:43.453 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2995 2023-07-14T18:16:43.453 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2995_vm 2023-07-14T18:16:43.453 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2996 2023-07-14T18:16:43.453 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3079 2023-07-14T18:16:43.453 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3079_fsm 2023-07-14T18:16:43.454 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3079_vm 2023-07-14T18:16:43.454 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3080 2023-07-14T18:16:43.454 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3081 2023-07-14T18:16:43.454 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3085 2023-07-14T18:16:43.454 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3118 2023-07-14T18:16:43.454 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3118_vm 2023-07-14T18:16:43.454 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3119 2023-07-14T18:16:43.455 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3164 2023-07-14T18:16:43.455 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3256 2023-07-14T18:16:43.455 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3256_vm 2023-07-14T18:16:43.455 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3257 2023-07-14T18:16:43.455 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3258 2023-07-14T18:16:43.455 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3350 2023-07-14T18:16:43.455 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3350_vm 2023-07-14T18:16:43.456 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3351 2023-07-14T18:16:43.456 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3379 2023-07-14T18:16:43.456 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3380 2023-07-14T18:16:43.456 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3381 2023-07-14T18:16:43.456 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3381_vm 2023-07-14T18:16:43.456 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3394 2023-07-14T18:16:43.456 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3394_fsm 2023-07-14T18:16:43.456 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3394_vm 2023-07-14T18:16:43.457 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3395 2023-07-14T18:16:43.457 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3439 2023-07-14T18:16:43.457 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3439_vm 2023-07-14T18:16:43.457 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3440 2023-07-14T18:16:43.457 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 3455 2023-07-14T18:16:43.457 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3456 2023-07-14T18:16:43.457 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3456_fsm 2023-07-14T18:16:43.458 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3456_vm 2023-07-14T18:16:43.458 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3466 2023-07-14T18:16:43.458 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3466_vm 2023-07-14T18:16:43.458 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3467 2023-07-14T18:16:43.458 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3468 2023-07-14T18:16:43.458 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3501 2023-07-14T18:16:43.458 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3501_vm 2023-07-14T18:16:43.459 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3502 2023-07-14T18:16:43.459 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3503 2023-07-14T18:16:43.459 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3534 2023-07-14T18:16:43.459 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3541 2023-07-14T18:16:43.459 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3541_fsm 2023-07-14T18:16:43.459 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3541_vm 2023-07-14T18:16:43.459 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3542 2023-07-14T18:16:43.459 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3574 2023-07-14T18:16:43.460 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3575 2023-07-14T18:16:43.460 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3576 2023-07-14T18:16:43.460 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3576_vm 2023-07-14T18:16:43.460 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3596 2023-07-14T18:16:43.460 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3596_vm 2023-07-14T18:16:43.460 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3597 2023-07-14T18:16:43.460 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3598 2023-07-14T18:16:43.461 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3598_vm 2023-07-14T18:16:43.461 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3599 2023-07-14T18:16:43.461 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3600 2023-07-14T18:16:43.461 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3600_fsm 2023-07-14T18:16:43.461 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3600_vm 2023-07-14T18:16:43.461 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3601 2023-07-14T18:16:43.461 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3601_fsm 2023-07-14T18:16:43.462 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3601_vm 2023-07-14T18:16:43.462 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3602 2023-07-14T18:16:43.462 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3602_fsm 2023-07-14T18:16:43.462 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3602_vm 2023-07-14T18:16:43.462 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3603 2023-07-14T18:16:43.462 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3603_fsm 2023-07-14T18:16:43.462 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3603_vm 2023-07-14T18:16:43.463 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3604 2023-07-14T18:16:43.463 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3605 2023-07-14T18:16:43.463 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3606 2023-07-14T18:16:43.465 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3607 2023-07-14T18:16:43.466 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3608 2023-07-14T18:16:43.466 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 3609 2023-07-14T18:16:43.466 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3712 2023-07-14T18:16:43.466 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3764 2023-07-14T18:16:43.466 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3764_fsm 2023-07-14T18:16:43.466 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3764_vm 2023-07-14T18:16:43.466 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3766 2023-07-14T18:16:43.467 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3767 2023-07-14T18:16:43.467 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3997 2023-07-14T18:16:43.467 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 5002 2023-07-14T18:16:43.467 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 548 2023-07-14T18:16:43.467 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 549 2023-07-14T18:16:43.467 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6102 2023-07-14T18:16:43.467 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6102_vm 2023-07-14T18:16:43.468 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6104 2023-07-14T18:16:43.468 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6104_vm 2023-07-14T18:16:43.468 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6106 2023-07-14T18:16:43.468 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6106_vm 2023-07-14T18:16:43.468 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6110 2023-07-14T18:16:43.468 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6111 2023-07-14T18:16:43.468 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6112 2023-07-14T18:16:43.468 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6113 2023-07-14T18:16:43.469 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6117 2023-07-14T18:16:43.469 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 826 2023-07-14T18:16:43.469 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 826_vm 2023-07-14T18:16:43.469 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 827 2023-07-14T18:16:43.469 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 828 2023-07-14T18:16:43.469 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jul 14 18:16 pg_filenode.map 2023-07-14T18:16:43.469 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 14 18:16 PG_VERSION 2023-07-14T18:16:43.470 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:43.470 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-07-14T18:16:43.470 INFO:teuthology.orchestra.run.smithi133.stdout:total 7193 2023-07-14T18:16:43.470 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jul 14 18:16 . 2023-07-14T18:16:43.470 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 14 18:16 .. 2023-07-14T18:16:43.470 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 112 2023-07-14T18:16:43.470 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 113 2023-07-14T18:16:43.471 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jul 14 18:16 12182 2023-07-14T18:16:43.471 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12182_fsm 2023-07-14T18:16:43.471 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12182_vm 2023-07-14T18:16:43.471 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12184 2023-07-14T18:16:43.471 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12186 2023-07-14T18:16:43.471 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12187 2023-07-14T18:16:43.471 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12187_fsm 2023-07-14T18:16:43.471 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12187_vm 2023-07-14T18:16:43.472 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12189 2023-07-14T18:16:43.472 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12191 2023-07-14T18:16:43.472 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12192 2023-07-14T18:16:43.472 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12192_fsm 2023-07-14T18:16:43.472 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12192_vm 2023-07-14T18:16:43.472 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12194 2023-07-14T18:16:43.472 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12196 2023-07-14T18:16:43.473 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12197 2023-07-14T18:16:43.473 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12197_fsm 2023-07-14T18:16:43.473 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12197_vm 2023-07-14T18:16:43.473 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12199 2023-07-14T18:16:43.473 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12201 2023-07-14T18:16:43.473 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12202 2023-07-14T18:16:43.474 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12202_fsm 2023-07-14T18:16:43.474 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12202_vm 2023-07-14T18:16:43.474 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12204 2023-07-14T18:16:43.474 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12206 2023-07-14T18:16:43.474 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12207 2023-07-14T18:16:43.474 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12207_fsm 2023-07-14T18:16:43.474 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12207_vm 2023-07-14T18:16:43.474 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12209 2023-07-14T18:16:43.475 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12211 2023-07-14T18:16:43.475 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12212 2023-07-14T18:16:43.475 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12214 2023-07-14T18:16:43.475 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12216 2023-07-14T18:16:43.475 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jul 14 18:16 1247 2023-07-14T18:16:43.475 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1247_fsm 2023-07-14T18:16:43.475 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1247_vm 2023-07-14T18:16:43.476 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jul 14 18:16 1249 2023-07-14T18:16:43.476 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1249_fsm 2023-07-14T18:16:43.476 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1249_vm 2023-07-14T18:16:43.476 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jul 14 18:16 1255 2023-07-14T18:16:43.476 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1255_fsm 2023-07-14T18:16:43.476 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1255_vm 2023-07-14T18:16:43.476 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jul 14 18:16 1259 2023-07-14T18:16:43.477 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1259_fsm 2023-07-14T18:16:43.477 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1259_vm 2023-07-14T18:16:43.477 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 1417 2023-07-14T18:16:43.477 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 1417_vm 2023-07-14T18:16:43.477 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 1418 2023-07-14T18:16:43.477 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 1418_vm 2023-07-14T18:16:43.477 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 174 2023-07-14T18:16:43.477 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 175 2023-07-14T18:16:43.478 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2187 2023-07-14T18:16:43.478 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2224 2023-07-14T18:16:43.478 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2224_vm 2023-07-14T18:16:43.478 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2328 2023-07-14T18:16:43.478 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2328_vm 2023-07-14T18:16:43.478 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2336 2023-07-14T18:16:43.478 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2336_vm 2023-07-14T18:16:43.479 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2337 2023-07-14T18:16:43.479 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2600 2023-07-14T18:16:43.479 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2600_fsm 2023-07-14T18:16:43.479 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2600_vm 2023-07-14T18:16:43.479 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2601 2023-07-14T18:16:43.479 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2601_fsm 2023-07-14T18:16:43.479 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2601_vm 2023-07-14T18:16:43.480 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 14 18:16 2602 2023-07-14T18:16:43.480 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2602_fsm 2023-07-14T18:16:43.480 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2602_vm 2023-07-14T18:16:43.480 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2603 2023-07-14T18:16:43.480 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2603_fsm 2023-07-14T18:16:43.480 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2603_vm 2023-07-14T18:16:43.480 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2604 2023-07-14T18:16:43.481 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2604_vm 2023-07-14T18:16:43.481 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2605 2023-07-14T18:16:43.481 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2605_fsm 2023-07-14T18:16:43.481 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2605_vm 2023-07-14T18:16:43.481 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2606 2023-07-14T18:16:43.481 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2606_fsm 2023-07-14T18:16:43.481 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2606_vm 2023-07-14T18:16:43.482 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2607 2023-07-14T18:16:43.482 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2607_fsm 2023-07-14T18:16:43.482 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2607_vm 2023-07-14T18:16:43.482 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Jul 14 18:16 2608 2023-07-14T18:16:43.482 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2608_fsm 2023-07-14T18:16:43.482 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2608_vm 2023-07-14T18:16:43.482 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jul 14 18:16 2609 2023-07-14T18:16:43.482 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2609_fsm 2023-07-14T18:16:43.483 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2609_vm 2023-07-14T18:16:43.483 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2610 2023-07-14T18:16:43.483 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2610_fsm 2023-07-14T18:16:43.483 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2610_vm 2023-07-14T18:16:43.483 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2611 2023-07-14T18:16:43.483 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2611_vm 2023-07-14T18:16:43.483 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2612 2023-07-14T18:16:43.484 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2612_fsm 2023-07-14T18:16:43.484 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2612_vm 2023-07-14T18:16:43.484 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2613 2023-07-14T18:16:43.484 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2613_vm 2023-07-14T18:16:43.484 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2615 2023-07-14T18:16:43.484 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2615_fsm 2023-07-14T18:16:43.484 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2615_vm 2023-07-14T18:16:43.485 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2616 2023-07-14T18:16:43.485 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2616_fsm 2023-07-14T18:16:43.485 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2616_vm 2023-07-14T18:16:43.485 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jul 14 18:16 2617 2023-07-14T18:16:43.485 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2617_fsm 2023-07-14T18:16:43.485 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2617_vm 2023-07-14T18:16:43.485 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jul 14 18:16 2618 2023-07-14T18:16:43.485 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2618_fsm 2023-07-14T18:16:43.486 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2618_vm 2023-07-14T18:16:43.486 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jul 14 18:16 2619 2023-07-14T18:16:43.486 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2619_fsm 2023-07-14T18:16:43.486 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2619_vm 2023-07-14T18:16:43.486 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2620 2023-07-14T18:16:43.486 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2620_vm 2023-07-14T18:16:43.486 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2650 2023-07-14T18:16:43.487 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2651 2023-07-14T18:16:43.487 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2652 2023-07-14T18:16:43.487 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 2653 2023-07-14T18:16:43.487 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 2654 2023-07-14T18:16:43.487 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2655 2023-07-14T18:16:43.487 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2656 2023-07-14T18:16:43.487 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2657 2023-07-14T18:16:43.488 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jul 14 18:16 2658 2023-07-14T18:16:43.488 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 14 18:16 2659 2023-07-14T18:16:43.488 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2660 2023-07-14T18:16:43.488 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2661 2023-07-14T18:16:43.488 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2662 2023-07-14T18:16:43.488 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 14 18:16 2663 2023-07-14T18:16:43.488 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2664 2023-07-14T18:16:43.488 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2665 2023-07-14T18:16:43.489 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2666 2023-07-14T18:16:43.489 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2667 2023-07-14T18:16:43.489 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2668 2023-07-14T18:16:43.489 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2669 2023-07-14T18:16:43.489 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2670 2023-07-14T18:16:43.489 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 Jul 14 18:16 2673 2023-07-14T18:16:43.490 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Jul 14 18:16 2674 2023-07-14T18:16:43.490 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jul 14 18:16 2675 2023-07-14T18:16:43.490 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2678 2023-07-14T18:16:43.490 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2679 2023-07-14T18:16:43.490 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2680 2023-07-14T18:16:43.490 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2681 2023-07-14T18:16:43.490 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2682 2023-07-14T18:16:43.491 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2683 2023-07-14T18:16:43.491 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2684 2023-07-14T18:16:43.491 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2685 2023-07-14T18:16:43.491 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2686 2023-07-14T18:16:43.491 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2687 2023-07-14T18:16:43.491 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 2688 2023-07-14T18:16:43.491 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 14 18:16 2689 2023-07-14T18:16:43.491 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 14 18:16 2690 2023-07-14T18:16:43.492 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jul 14 18:16 2691 2023-07-14T18:16:43.492 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2692 2023-07-14T18:16:43.492 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2693 2023-07-14T18:16:43.492 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2696 2023-07-14T18:16:43.492 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2699 2023-07-14T18:16:43.492 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2701 2023-07-14T18:16:43.492 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2702 2023-07-14T18:16:43.493 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2703 2023-07-14T18:16:43.493 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 2704 2023-07-14T18:16:43.493 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2753 2023-07-14T18:16:43.493 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2753_fsm 2023-07-14T18:16:43.493 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2753_vm 2023-07-14T18:16:43.493 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2754 2023-07-14T18:16:43.493 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2755 2023-07-14T18:16:43.494 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2756 2023-07-14T18:16:43.494 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2757 2023-07-14T18:16:43.494 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2830 2023-07-14T18:16:43.494 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2830_vm 2023-07-14T18:16:43.494 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2831 2023-07-14T18:16:43.494 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2832 2023-07-14T18:16:43.494 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2832_vm 2023-07-14T18:16:43.494 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2833 2023-07-14T18:16:43.495 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2834 2023-07-14T18:16:43.495 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2834_vm 2023-07-14T18:16:43.495 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2835 2023-07-14T18:16:43.495 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2836 2023-07-14T18:16:43.495 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2836_vm 2023-07-14T18:16:43.495 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2837 2023-07-14T18:16:43.495 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jul 14 18:16 2838 2023-07-14T18:16:43.496 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2838_fsm 2023-07-14T18:16:43.496 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2838_vm 2023-07-14T18:16:43.496 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2839 2023-07-14T18:16:43.496 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2840 2023-07-14T18:16:43.496 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2840_fsm 2023-07-14T18:16:43.496 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2840_vm 2023-07-14T18:16:43.496 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2841 2023-07-14T18:16:43.497 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2995 2023-07-14T18:16:43.497 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2995_vm 2023-07-14T18:16:43.497 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2996 2023-07-14T18:16:43.497 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3079 2023-07-14T18:16:43.497 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3079_fsm 2023-07-14T18:16:43.497 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3079_vm 2023-07-14T18:16:43.497 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3080 2023-07-14T18:16:43.497 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3081 2023-07-14T18:16:43.498 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3085 2023-07-14T18:16:43.498 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3118 2023-07-14T18:16:43.498 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3118_vm 2023-07-14T18:16:43.498 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3119 2023-07-14T18:16:43.498 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3164 2023-07-14T18:16:43.498 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3256 2023-07-14T18:16:43.498 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3256_vm 2023-07-14T18:16:43.499 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3257 2023-07-14T18:16:43.499 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3258 2023-07-14T18:16:43.499 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3350 2023-07-14T18:16:43.499 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3350_vm 2023-07-14T18:16:43.499 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3351 2023-07-14T18:16:43.499 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3379 2023-07-14T18:16:43.499 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3380 2023-07-14T18:16:43.500 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3381 2023-07-14T18:16:43.500 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3381_vm 2023-07-14T18:16:43.500 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3394 2023-07-14T18:16:43.500 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3394_fsm 2023-07-14T18:16:43.500 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3394_vm 2023-07-14T18:16:43.500 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3395 2023-07-14T18:16:43.500 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3439 2023-07-14T18:16:43.500 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3439_vm 2023-07-14T18:16:43.501 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3440 2023-07-14T18:16:43.501 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 3455 2023-07-14T18:16:43.501 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3456 2023-07-14T18:16:43.501 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3456_fsm 2023-07-14T18:16:43.501 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3456_vm 2023-07-14T18:16:43.501 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3466 2023-07-14T18:16:43.501 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3466_vm 2023-07-14T18:16:43.502 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3467 2023-07-14T18:16:43.502 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3468 2023-07-14T18:16:43.502 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3501 2023-07-14T18:16:43.502 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3501_vm 2023-07-14T18:16:43.502 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3502 2023-07-14T18:16:43.502 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3503 2023-07-14T18:16:43.502 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3534 2023-07-14T18:16:43.503 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3541 2023-07-14T18:16:43.503 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3541_fsm 2023-07-14T18:16:43.503 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3541_vm 2023-07-14T18:16:43.503 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3542 2023-07-14T18:16:43.503 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3574 2023-07-14T18:16:43.503 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3575 2023-07-14T18:16:43.503 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3576 2023-07-14T18:16:43.503 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3576_vm 2023-07-14T18:16:43.504 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3596 2023-07-14T18:16:43.504 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3596_vm 2023-07-14T18:16:43.504 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3597 2023-07-14T18:16:43.504 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3598 2023-07-14T18:16:43.504 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3598_vm 2023-07-14T18:16:43.504 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3599 2023-07-14T18:16:43.504 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3600 2023-07-14T18:16:43.505 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3600_fsm 2023-07-14T18:16:43.505 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3600_vm 2023-07-14T18:16:43.505 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3601 2023-07-14T18:16:43.505 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3601_fsm 2023-07-14T18:16:43.505 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3601_vm 2023-07-14T18:16:43.505 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3602 2023-07-14T18:16:43.505 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3602_fsm 2023-07-14T18:16:43.506 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3602_vm 2023-07-14T18:16:43.506 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3603 2023-07-14T18:16:43.506 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3603_fsm 2023-07-14T18:16:43.506 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3603_vm 2023-07-14T18:16:43.506 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3604 2023-07-14T18:16:43.506 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3605 2023-07-14T18:16:43.506 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3606 2023-07-14T18:16:43.507 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3607 2023-07-14T18:16:43.507 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3608 2023-07-14T18:16:43.507 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 3609 2023-07-14T18:16:43.507 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3712 2023-07-14T18:16:43.507 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3764 2023-07-14T18:16:43.507 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3764_fsm 2023-07-14T18:16:43.507 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3764_vm 2023-07-14T18:16:43.508 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3766 2023-07-14T18:16:43.508 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3767 2023-07-14T18:16:43.508 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3997 2023-07-14T18:16:43.508 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 5002 2023-07-14T18:16:43.508 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 548 2023-07-14T18:16:43.508 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 549 2023-07-14T18:16:43.508 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6102 2023-07-14T18:16:43.509 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6102_vm 2023-07-14T18:16:43.509 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6104 2023-07-14T18:16:43.509 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6104_vm 2023-07-14T18:16:43.509 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6106 2023-07-14T18:16:43.509 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6106_vm 2023-07-14T18:16:43.509 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6110 2023-07-14T18:16:43.509 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6111 2023-07-14T18:16:43.509 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6112 2023-07-14T18:16:43.510 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6113 2023-07-14T18:16:43.510 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6117 2023-07-14T18:16:43.510 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 826 2023-07-14T18:16:43.510 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 826_vm 2023-07-14T18:16:43.510 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 827 2023-07-14T18:16:43.510 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 828 2023-07-14T18:16:43.510 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jul 14 18:16 pg_filenode.map 2023-07-14T18:16:43.511 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 14 18:16 PG_VERSION 2023-07-14T18:16:43.511 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:43.511 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-07-14T18:16:43.511 INFO:teuthology.orchestra.run.smithi133.stdout:total 7193 2023-07-14T18:16:43.511 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jul 14 18:16 . 2023-07-14T18:16:43.511 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 14 18:16 .. 2023-07-14T18:16:43.512 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 112 2023-07-14T18:16:43.512 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 113 2023-07-14T18:16:43.512 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jul 14 18:16 12182 2023-07-14T18:16:43.512 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12182_fsm 2023-07-14T18:16:43.512 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12182_vm 2023-07-14T18:16:43.512 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12184 2023-07-14T18:16:43.512 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12186 2023-07-14T18:16:43.512 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12187 2023-07-14T18:16:43.513 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12187_fsm 2023-07-14T18:16:43.513 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12187_vm 2023-07-14T18:16:43.513 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12189 2023-07-14T18:16:43.513 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12191 2023-07-14T18:16:43.513 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12192 2023-07-14T18:16:43.513 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12192_fsm 2023-07-14T18:16:43.514 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12192_vm 2023-07-14T18:16:43.514 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12194 2023-07-14T18:16:43.514 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12196 2023-07-14T18:16:43.514 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12197 2023-07-14T18:16:43.514 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12197_fsm 2023-07-14T18:16:43.514 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12197_vm 2023-07-14T18:16:43.514 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12199 2023-07-14T18:16:43.515 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12201 2023-07-14T18:16:43.515 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12202 2023-07-14T18:16:43.515 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12202_fsm 2023-07-14T18:16:43.515 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12202_vm 2023-07-14T18:16:43.515 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12204 2023-07-14T18:16:43.515 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12206 2023-07-14T18:16:43.515 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12207 2023-07-14T18:16:43.516 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12207_fsm 2023-07-14T18:16:43.516 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12207_vm 2023-07-14T18:16:43.516 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12209 2023-07-14T18:16:43.516 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12211 2023-07-14T18:16:43.516 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12212 2023-07-14T18:16:43.516 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12214 2023-07-14T18:16:43.516 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12216 2023-07-14T18:16:43.517 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jul 14 18:16 1247 2023-07-14T18:16:43.517 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1247_fsm 2023-07-14T18:16:43.517 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1247_vm 2023-07-14T18:16:43.517 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jul 14 18:16 1249 2023-07-14T18:16:43.517 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1249_fsm 2023-07-14T18:16:43.517 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1249_vm 2023-07-14T18:16:43.517 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jul 14 18:16 1255 2023-07-14T18:16:43.518 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1255_fsm 2023-07-14T18:16:43.518 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1255_vm 2023-07-14T18:16:43.518 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jul 14 18:16 1259 2023-07-14T18:16:43.518 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1259_fsm 2023-07-14T18:16:43.518 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1259_vm 2023-07-14T18:16:43.518 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 1417 2023-07-14T18:16:43.518 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 1417_vm 2023-07-14T18:16:43.518 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 1418 2023-07-14T18:16:43.519 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 1418_vm 2023-07-14T18:16:43.519 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 174 2023-07-14T18:16:43.519 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 175 2023-07-14T18:16:43.519 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2187 2023-07-14T18:16:43.519 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2224 2023-07-14T18:16:43.519 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2224_vm 2023-07-14T18:16:43.519 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2328 2023-07-14T18:16:43.520 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2328_vm 2023-07-14T18:16:43.520 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2336 2023-07-14T18:16:43.520 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2336_vm 2023-07-14T18:16:43.520 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2337 2023-07-14T18:16:43.520 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2600 2023-07-14T18:16:43.520 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2600_fsm 2023-07-14T18:16:43.520 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2600_vm 2023-07-14T18:16:43.521 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2601 2023-07-14T18:16:43.521 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2601_fsm 2023-07-14T18:16:43.521 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2601_vm 2023-07-14T18:16:43.521 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 14 18:16 2602 2023-07-14T18:16:43.521 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2602_fsm 2023-07-14T18:16:43.521 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2602_vm 2023-07-14T18:16:43.521 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2603 2023-07-14T18:16:43.522 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2603_fsm 2023-07-14T18:16:43.522 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2603_vm 2023-07-14T18:16:43.522 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2604 2023-07-14T18:16:43.522 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2604_vm 2023-07-14T18:16:43.522 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2605 2023-07-14T18:16:43.522 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2605_fsm 2023-07-14T18:16:43.522 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2605_vm 2023-07-14T18:16:43.522 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2606 2023-07-14T18:16:43.523 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2606_fsm 2023-07-14T18:16:43.523 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2606_vm 2023-07-14T18:16:43.523 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2607 2023-07-14T18:16:43.523 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2607_fsm 2023-07-14T18:16:43.523 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2607_vm 2023-07-14T18:16:43.523 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Jul 14 18:16 2608 2023-07-14T18:16:43.523 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2608_fsm 2023-07-14T18:16:43.524 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2608_vm 2023-07-14T18:16:43.524 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jul 14 18:16 2609 2023-07-14T18:16:43.524 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2609_fsm 2023-07-14T18:16:43.524 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2609_vm 2023-07-14T18:16:43.524 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2610 2023-07-14T18:16:43.524 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2610_fsm 2023-07-14T18:16:43.524 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2610_vm 2023-07-14T18:16:43.525 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2611 2023-07-14T18:16:43.525 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2611_vm 2023-07-14T18:16:43.525 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2612 2023-07-14T18:16:43.525 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2612_fsm 2023-07-14T18:16:43.525 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2612_vm 2023-07-14T18:16:43.525 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2613 2023-07-14T18:16:43.525 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2613_vm 2023-07-14T18:16:43.526 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2615 2023-07-14T18:16:43.526 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2615_fsm 2023-07-14T18:16:43.526 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2615_vm 2023-07-14T18:16:43.526 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2616 2023-07-14T18:16:43.526 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2616_fsm 2023-07-14T18:16:43.526 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2616_vm 2023-07-14T18:16:43.526 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jul 14 18:16 2617 2023-07-14T18:16:43.527 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2617_fsm 2023-07-14T18:16:43.527 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2617_vm 2023-07-14T18:16:43.527 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jul 14 18:16 2618 2023-07-14T18:16:43.527 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2618_fsm 2023-07-14T18:16:43.527 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2618_vm 2023-07-14T18:16:43.527 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jul 14 18:16 2619 2023-07-14T18:16:43.527 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2619_fsm 2023-07-14T18:16:43.527 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2619_vm 2023-07-14T18:16:43.528 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2620 2023-07-14T18:16:43.528 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2620_vm 2023-07-14T18:16:43.528 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2650 2023-07-14T18:16:43.528 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2651 2023-07-14T18:16:43.528 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2652 2023-07-14T18:16:43.528 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 2653 2023-07-14T18:16:43.528 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 2654 2023-07-14T18:16:43.529 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2655 2023-07-14T18:16:43.529 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2656 2023-07-14T18:16:43.529 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2657 2023-07-14T18:16:43.529 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jul 14 18:16 2658 2023-07-14T18:16:43.529 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 14 18:16 2659 2023-07-14T18:16:43.529 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2660 2023-07-14T18:16:43.529 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2661 2023-07-14T18:16:43.530 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2662 2023-07-14T18:16:43.530 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 14 18:16 2663 2023-07-14T18:16:43.530 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2664 2023-07-14T18:16:43.530 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2665 2023-07-14T18:16:43.530 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2666 2023-07-14T18:16:43.530 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2667 2023-07-14T18:16:43.530 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2668 2023-07-14T18:16:43.531 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2669 2023-07-14T18:16:43.531 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2670 2023-07-14T18:16:43.531 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 Jul 14 18:16 2673 2023-07-14T18:16:43.531 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Jul 14 18:16 2674 2023-07-14T18:16:43.531 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jul 14 18:16 2675 2023-07-14T18:16:43.531 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2678 2023-07-14T18:16:43.531 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2679 2023-07-14T18:16:43.532 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2680 2023-07-14T18:16:43.532 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2681 2023-07-14T18:16:43.532 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2682 2023-07-14T18:16:43.532 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2683 2023-07-14T18:16:43.532 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2684 2023-07-14T18:16:43.532 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2685 2023-07-14T18:16:43.532 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2686 2023-07-14T18:16:43.532 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2687 2023-07-14T18:16:43.533 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 2688 2023-07-14T18:16:43.533 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 14 18:16 2689 2023-07-14T18:16:43.533 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 14 18:16 2690 2023-07-14T18:16:43.533 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jul 14 18:16 2691 2023-07-14T18:16:43.533 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2692 2023-07-14T18:16:43.533 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2693 2023-07-14T18:16:43.533 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2696 2023-07-14T18:16:43.534 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2699 2023-07-14T18:16:43.534 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2701 2023-07-14T18:16:43.534 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2702 2023-07-14T18:16:43.534 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2703 2023-07-14T18:16:43.534 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 2704 2023-07-14T18:16:43.534 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2753 2023-07-14T18:16:43.534 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2753_fsm 2023-07-14T18:16:43.534 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2753_vm 2023-07-14T18:16:43.535 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2754 2023-07-14T18:16:43.535 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2755 2023-07-14T18:16:43.535 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2756 2023-07-14T18:16:43.535 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2757 2023-07-14T18:16:43.535 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2830 2023-07-14T18:16:43.535 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2830_vm 2023-07-14T18:16:43.535 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2831 2023-07-14T18:16:43.536 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2832 2023-07-14T18:16:43.536 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2832_vm 2023-07-14T18:16:43.536 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2833 2023-07-14T18:16:43.536 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2834 2023-07-14T18:16:43.536 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2834_vm 2023-07-14T18:16:43.536 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2835 2023-07-14T18:16:43.536 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2836 2023-07-14T18:16:43.537 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2836_vm 2023-07-14T18:16:43.537 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2837 2023-07-14T18:16:43.537 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jul 14 18:16 2838 2023-07-14T18:16:43.537 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2838_fsm 2023-07-14T18:16:43.537 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2838_vm 2023-07-14T18:16:43.537 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2839 2023-07-14T18:16:43.537 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2840 2023-07-14T18:16:43.538 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2840_fsm 2023-07-14T18:16:43.538 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2840_vm 2023-07-14T18:16:43.538 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2841 2023-07-14T18:16:43.538 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2995 2023-07-14T18:16:43.538 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2995_vm 2023-07-14T18:16:43.538 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2996 2023-07-14T18:16:43.538 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3079 2023-07-14T18:16:43.538 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3079_fsm 2023-07-14T18:16:43.539 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3079_vm 2023-07-14T18:16:43.539 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3080 2023-07-14T18:16:43.539 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3081 2023-07-14T18:16:43.539 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3085 2023-07-14T18:16:43.539 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3118 2023-07-14T18:16:43.539 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3118_vm 2023-07-14T18:16:43.539 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3119 2023-07-14T18:16:43.540 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3164 2023-07-14T18:16:43.540 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3256 2023-07-14T18:16:43.540 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3256_vm 2023-07-14T18:16:43.540 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3257 2023-07-14T18:16:43.540 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3258 2023-07-14T18:16:43.540 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3350 2023-07-14T18:16:43.540 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3350_vm 2023-07-14T18:16:43.541 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3351 2023-07-14T18:16:43.541 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3379 2023-07-14T18:16:43.541 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3380 2023-07-14T18:16:43.541 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3381 2023-07-14T18:16:43.541 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3381_vm 2023-07-14T18:16:43.541 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3394 2023-07-14T18:16:43.541 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3394_fsm 2023-07-14T18:16:43.541 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3394_vm 2023-07-14T18:16:43.542 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3395 2023-07-14T18:16:43.542 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3439 2023-07-14T18:16:43.542 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3439_vm 2023-07-14T18:16:43.542 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3440 2023-07-14T18:16:43.542 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 3455 2023-07-14T18:16:43.542 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3456 2023-07-14T18:16:43.542 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3456_fsm 2023-07-14T18:16:43.543 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3456_vm 2023-07-14T18:16:43.543 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3466 2023-07-14T18:16:43.543 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3466_vm 2023-07-14T18:16:43.543 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3467 2023-07-14T18:16:43.543 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3468 2023-07-14T18:16:43.543 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3501 2023-07-14T18:16:43.543 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3501_vm 2023-07-14T18:16:43.543 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3502 2023-07-14T18:16:43.544 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3503 2023-07-14T18:16:43.544 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3534 2023-07-14T18:16:43.544 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3541 2023-07-14T18:16:43.544 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3541_fsm 2023-07-14T18:16:43.544 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3541_vm 2023-07-14T18:16:43.544 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3542 2023-07-14T18:16:43.544 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3574 2023-07-14T18:16:43.545 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3575 2023-07-14T18:16:43.545 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3576 2023-07-14T18:16:43.545 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3576_vm 2023-07-14T18:16:43.545 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3596 2023-07-14T18:16:43.545 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3596_vm 2023-07-14T18:16:43.545 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3597 2023-07-14T18:16:43.545 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3598 2023-07-14T18:16:43.546 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3598_vm 2023-07-14T18:16:43.546 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3599 2023-07-14T18:16:43.546 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3600 2023-07-14T18:16:43.546 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3600_fsm 2023-07-14T18:16:43.546 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3600_vm 2023-07-14T18:16:43.546 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3601 2023-07-14T18:16:43.546 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3601_fsm 2023-07-14T18:16:43.547 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3601_vm 2023-07-14T18:16:43.547 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3602 2023-07-14T18:16:43.547 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3602_fsm 2023-07-14T18:16:43.547 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3602_vm 2023-07-14T18:16:43.547 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3603 2023-07-14T18:16:43.547 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3603_fsm 2023-07-14T18:16:43.547 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3603_vm 2023-07-14T18:16:43.548 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3604 2023-07-14T18:16:43.548 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3605 2023-07-14T18:16:43.548 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3606 2023-07-14T18:16:43.548 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3607 2023-07-14T18:16:43.548 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3608 2023-07-14T18:16:43.548 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 3609 2023-07-14T18:16:43.548 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3712 2023-07-14T18:16:43.549 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3764 2023-07-14T18:16:43.549 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3764_fsm 2023-07-14T18:16:43.549 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3764_vm 2023-07-14T18:16:43.549 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3766 2023-07-14T18:16:43.549 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3767 2023-07-14T18:16:43.549 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3997 2023-07-14T18:16:43.549 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 5002 2023-07-14T18:16:43.549 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 548 2023-07-14T18:16:43.550 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 549 2023-07-14T18:16:43.550 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6102 2023-07-14T18:16:43.550 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6102_vm 2023-07-14T18:16:43.550 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6104 2023-07-14T18:16:43.550 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6104_vm 2023-07-14T18:16:43.550 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6106 2023-07-14T18:16:43.550 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6106_vm 2023-07-14T18:16:43.551 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6110 2023-07-14T18:16:43.551 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6111 2023-07-14T18:16:43.551 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6112 2023-07-14T18:16:43.551 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6113 2023-07-14T18:16:43.551 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6117 2023-07-14T18:16:43.551 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 826 2023-07-14T18:16:43.551 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 826_vm 2023-07-14T18:16:43.552 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 827 2023-07-14T18:16:43.552 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 828 2023-07-14T18:16:43.552 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jul 14 18:16 pg_filenode.map 2023-07-14T18:16:43.552 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 14 18:16 PG_VERSION 2023-07-14T18:16:43.552 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:43.552 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/global: 2023-07-14T18:16:43.552 INFO:teuthology.orchestra.run.smithi133.stdout:total 553 2023-07-14T18:16:43.553 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Jul 14 18:16 . 2023-07-14T18:16:43.553 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 14 18:16 .. 2023-07-14T18:16:43.553 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1136 2023-07-14T18:16:43.553 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1136_fsm 2023-07-14T18:16:43.553 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1136_vm 2023-07-14T18:16:43.553 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 1137 2023-07-14T18:16:43.553 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1213 2023-07-14T18:16:43.553 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1213_fsm 2023-07-14T18:16:43.554 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1213_vm 2023-07-14T18:16:43.554 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1214 2023-07-14T18:16:43.554 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1214_fsm 2023-07-14T18:16:43.554 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1214_vm 2023-07-14T18:16:43.554 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 1232 2023-07-14T18:16:43.554 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 1233 2023-07-14T18:16:43.554 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1260 2023-07-14T18:16:43.555 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1260_fsm 2023-07-14T18:16:43.555 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1260_vm 2023-07-14T18:16:43.555 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1261 2023-07-14T18:16:43.555 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1261_fsm 2023-07-14T18:16:43.555 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1261_vm 2023-07-14T18:16:43.555 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1262 2023-07-14T18:16:43.555 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1262_fsm 2023-07-14T18:16:43.556 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1262_vm 2023-07-14T18:16:43.556 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2396 2023-07-14T18:16:43.556 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2396_fsm 2023-07-14T18:16:43.556 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2396_vm 2023-07-14T18:16:43.556 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2397 2023-07-14T18:16:43.556 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2671 2023-07-14T18:16:43.556 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2672 2023-07-14T18:16:43.557 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2676 2023-07-14T18:16:43.557 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2677 2023-07-14T18:16:43.557 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2694 2023-07-14T18:16:43.557 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2695 2023-07-14T18:16:43.557 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2697 2023-07-14T18:16:43.557 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2698 2023-07-14T18:16:43.557 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2846 2023-07-14T18:16:43.558 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2846_vm 2023-07-14T18:16:43.558 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2847 2023-07-14T18:16:43.558 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2964 2023-07-14T18:16:43.558 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2964_vm 2023-07-14T18:16:43.558 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2965 2023-07-14T18:16:43.558 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2966 2023-07-14T18:16:43.558 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2966_vm 2023-07-14T18:16:43.558 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2967 2023-07-14T18:16:43.559 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3592 2023-07-14T18:16:43.559 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3592_vm 2023-07-14T18:16:43.559 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3593 2023-07-14T18:16:43.559 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 4060 2023-07-14T18:16:43.559 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 4060_vm 2023-07-14T18:16:43.559 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 4061 2023-07-14T18:16:43.559 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6000 2023-07-14T18:16:43.560 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6000_vm 2023-07-14T18:16:43.560 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6001 2023-07-14T18:16:43.560 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6002 2023-07-14T18:16:43.560 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6100 2023-07-14T18:16:43.560 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6100_vm 2023-07-14T18:16:43.560 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6114 2023-07-14T18:16:43.560 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6115 2023-07-14T18:16:43.561 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 pg_control 2023-07-14T18:16:43.561 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jul 14 18:16 pg_filenode.map 2023-07-14T18:16:43.561 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:43.561 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/log: 2023-07-14T18:16:43.561 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:16:43.561 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 . 2023-07-14T18:16:43.561 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 14 18:16 .. 2023-07-14T18:16:43.562 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:43.562 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-07-14T18:16:43.562 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:16:43.562 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 . 2023-07-14T18:16:43.562 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 14 18:16 .. 2023-07-14T18:16:43.562 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:43.562 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-07-14T18:16:43.562 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:16:43.563 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 . 2023-07-14T18:16:43.563 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 14 18:16 .. 2023-07-14T18:16:43.563 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:43.563 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-07-14T18:16:43.563 INFO:teuthology.orchestra.run.smithi133.stdout:total 1 2023-07-14T18:16:43.563 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 14 18:16 . 2023-07-14T18:16:43.563 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 14 18:16 .. 2023-07-14T18:16:43.564 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 mappings 2023-07-14T18:16:43.564 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jul 14 18:16 replorigin_checkpoint 2023-07-14T18:16:43.564 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 snapshots 2023-07-14T18:16:43.564 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:43.564 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-07-14T18:16:43.564 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:16:43.564 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 . 2023-07-14T18:16:43.565 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 14 18:16 .. 2023-07-14T18:16:43.565 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:43.565 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-07-14T18:16:43.565 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:16:43.565 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 . 2023-07-14T18:16:43.565 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 14 18:16 .. 2023-07-14T18:16:43.565 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:43.566 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-07-14T18:16:43.566 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:16:43.566 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jul 14 18:16 . 2023-07-14T18:16:43.566 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 14 18:16 .. 2023-07-14T18:16:43.566 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 members 2023-07-14T18:16:43.566 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 offsets 2023-07-14T18:16:43.566 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:43.566 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-07-14T18:16:43.567 INFO:teuthology.orchestra.run.smithi133.stdout:total 8 2023-07-14T18:16:43.567 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 . 2023-07-14T18:16:43.567 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jul 14 18:16 .. 2023-07-14T18:16:43.567 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 0000 2023-07-14T18:16:43.567 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:43.567 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-07-14T18:16:43.567 INFO:teuthology.orchestra.run.smithi133.stdout:total 8 2023-07-14T18:16:43.568 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 . 2023-07-14T18:16:43.568 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jul 14 18:16 .. 2023-07-14T18:16:43.568 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 0000 2023-07-14T18:16:43.568 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:43.568 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-07-14T18:16:43.568 INFO:teuthology.orchestra.run.smithi133.stdout:total 8 2023-07-14T18:16:43.568 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 . 2023-07-14T18:16:43.569 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 14 18:16 .. 2023-07-14T18:16:43.569 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 0000 2023-07-14T18:16:43.569 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:43.569 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-07-14T18:16:43.569 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:16:43.569 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 . 2023-07-14T18:16:43.569 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 14 18:16 .. 2023-07-14T18:16:43.569 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:43.570 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-07-14T18:16:43.570 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:16:43.570 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 . 2023-07-14T18:16:43.570 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 14 18:16 .. 2023-07-14T18:16:43.570 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:43.570 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-07-14T18:16:43.570 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:16:43.571 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 . 2023-07-14T18:16:43.571 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 14 18:16 .. 2023-07-14T18:16:43.571 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:43.571 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-07-14T18:16:43.571 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:16:43.571 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 . 2023-07-14T18:16:43.571 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 14 18:16 .. 2023-07-14T18:16:43.571 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:43.572 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-07-14T18:16:43.572 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:16:43.572 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 . 2023-07-14T18:16:43.572 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 14 18:16 .. 2023-07-14T18:16:43.572 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:43.572 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-07-14T18:16:43.572 INFO:teuthology.orchestra.run.smithi133.stdout:total 8 2023-07-14T18:16:43.573 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 . 2023-07-14T18:16:43.573 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 14 18:16 .. 2023-07-14T18:16:43.573 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 0000 2023-07-14T18:16:43.573 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:43.573 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-07-14T18:16:43.573 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:16:43.573 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 . 2023-07-14T18:16:43.574 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 14 18:16 .. 2023-07-14T18:16:43.574 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:43.574 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-07-14T18:16:43.574 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:16:43.574 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 . 2023-07-14T18:16:43.574 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 14 18:16 .. 2023-07-14T18:16:43.574 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:43.575 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-07-14T18:16:43.575 INFO:teuthology.orchestra.run.smithi133.stdout:total 16384 2023-07-14T18:16:43.575 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jul 14 18:16 . 2023-07-14T18:16:43.575 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 14 18:16 .. 2023-07-14T18:16:43.575 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:16 000000010000000000000001 2023-07-14T18:16:43.575 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 archive_status 2023-07-14T18:16:43.575 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:43.576 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-07-14T18:16:43.576 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:16:43.576 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 . 2023-07-14T18:16:43.576 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jul 14 18:16 .. 2023-07-14T18:16:43.576 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:16:43.576 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-07-14T18:16:43.576 INFO:teuthology.orchestra.run.smithi133.stdout:total 8 2023-07-14T18:16:43.577 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 . 2023-07-14T18:16:43.577 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 14 18:16 .. 2023-07-14T18:16:43.577 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 0000 2023-07-14T18:16:43.577 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2023-07-14T18:16:43.805 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 500 -i' 2023-07-14T18:16:43.913 INFO:teuthology.orchestra.run.smithi133.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2023-07-14T18:16:43.920 INFO:teuthology.orchestra.run.smithi133.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2023-07-14T18:16:43.923 INFO:teuthology.orchestra.run.smithi133.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2023-07-14T18:16:43.927 INFO:teuthology.orchestra.run.smithi133.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2023-07-14T18:16:44.212 INFO:teuthology.orchestra.run.smithi133.stderr:creating tables... 2023-07-14T18:16:44.319 INFO:teuthology.orchestra.run.smithi133.stderr:100000 of 50000000 tuples (0%) done (elapsed 0.11 s, remaining 52.96 s) 2023-07-14T18:16:44.580 INFO:teuthology.orchestra.run.smithi133.stderr:200000 of 50000000 tuples (0%) done (elapsed 0.37 s, remaining 91.32 s) 2023-07-14T18:16:44.907 INFO:teuthology.orchestra.run.smithi133.stderr:300000 of 50000000 tuples (0%) done (elapsed 0.69 s, remaining 114.86 s) 2023-07-14T18:16:45.042 INFO:teuthology.orchestra.run.smithi133.stderr:400000 of 50000000 tuples (0%) done (elapsed 0.83 s, remaining 102.72 s) 2023-07-14T18:16:45.330 INFO:teuthology.orchestra.run.smithi133.stderr:500000 of 50000000 tuples (1%) done (elapsed 1.12 s, remaining 110.52 s) 2023-07-14T18:16:45.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:44 smithi161 ceph-mon[128993]: pgmap v350: 129 pgs: 129 active+clean; 19 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 1.8 MiB/s wr, 22 op/s 2023-07-14T18:16:45.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:44 smithi171 ceph-mon[129241]: pgmap v350: 129 pgs: 129 active+clean; 19 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 1.8 MiB/s wr, 22 op/s 2023-07-14T18:16:45.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:44 smithi133 ceph-mon[126384]: pgmap v350: 129 pgs: 129 active+clean; 19 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 1.8 MiB/s wr, 22 op/s 2023-07-14T18:16:45.493 INFO:teuthology.orchestra.run.smithi133.stderr:600000 of 50000000 tuples (1%) done (elapsed 1.28 s, remaining 105.39 s) 2023-07-14T18:16:45.788 INFO:teuthology.orchestra.run.smithi133.stderr:700000 of 50000000 tuples (1%) done (elapsed 1.57 s, remaining 110.90 s) 2023-07-14T18:16:46.151 INFO:teuthology.orchestra.run.smithi133.stderr:800000 of 50000000 tuples (1%) done (elapsed 1.94 s, remaining 119.15 s) 2023-07-14T18:16:46.327 INFO:teuthology.orchestra.run.smithi133.stderr:900000 of 50000000 tuples (1%) done (elapsed 2.11 s, remaining 115.33 s) 2023-07-14T18:16:46.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:45 smithi161 ceph-mon[128993]: scrub summary: idle 2023-07-14T18:16:46.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:45 smithi171 ceph-mon[129241]: scrub summary: idle 2023-07-14T18:16:46.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:45 smithi133 ceph-mon[126384]: scrub summary: idle 2023-07-14T18:16:46.625 INFO:teuthology.orchestra.run.smithi133.stderr:1000000 of 50000000 tuples (2%) done (elapsed 2.41 s, remaining 118.17 s) 2023-07-14T18:16:46.905 INFO:teuthology.orchestra.run.smithi133.stderr:1100000 of 50000000 tuples (2%) done (elapsed 2.69 s, remaining 119.64 s) 2023-07-14T18:16:47.100 INFO:teuthology.orchestra.run.smithi133.stderr:1200000 of 50000000 tuples (2%) done (elapsed 2.89 s, remaining 117.41 s) 2023-07-14T18:16:47.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:46 smithi161 ceph-mon[128993]: pgmap v351: 129 pgs: 129 active+clean; 21 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 2.0 MiB/s wr, 28 op/s 2023-07-14T18:16:47.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:46 smithi161 ceph-mon[128993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:16:47.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:46 smithi171 ceph-mon[129241]: pgmap v351: 129 pgs: 129 active+clean; 21 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 2.0 MiB/s wr, 28 op/s 2023-07-14T18:16:47.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:46 smithi171 ceph-mon[129241]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:16:47.374 INFO:teuthology.orchestra.run.smithi133.stderr:1300000 of 50000000 tuples (2%) done (elapsed 3.16 s, remaining 118.41 s) 2023-07-14T18:16:47.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:46 smithi133 ceph-mon[126384]: pgmap v351: 129 pgs: 129 active+clean; 21 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 2.0 MiB/s wr, 28 op/s 2023-07-14T18:16:47.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:46 smithi133 ceph-mon[126384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:16:47.662 INFO:teuthology.orchestra.run.smithi133.stderr:1400000 of 50000000 tuples (2%) done (elapsed 3.45 s, remaining 119.72 s) 2023-07-14T18:16:47.819 INFO:teuthology.orchestra.run.smithi133.stderr:1500000 of 50000000 tuples (3%) done (elapsed 3.60 s, remaining 116.55 s) 2023-07-14T18:16:48.117 INFO:teuthology.orchestra.run.smithi133.stderr:1600000 of 50000000 tuples (3%) done (elapsed 3.90 s, remaining 118.10 s) 2023-07-14T18:16:48.428 INFO:teuthology.orchestra.run.smithi133.stderr:1700000 of 50000000 tuples (3%) done (elapsed 4.22 s, remaining 119.76 s) 2023-07-14T18:16:48.603 INFO:teuthology.orchestra.run.smithi133.stderr:1800000 of 50000000 tuples (3%) done (elapsed 4.39 s, remaining 117.55 s) 2023-07-14T18:16:48.860 INFO:teuthology.orchestra.run.smithi133.stderr:1900000 of 50000000 tuples (3%) done (elapsed 4.65 s, remaining 117.64 s) 2023-07-14T18:16:49.156 INFO:teuthology.orchestra.run.smithi133.stderr:2000000 of 50000000 tuples (4%) done (elapsed 4.94 s, remaining 118.63 s) 2023-07-14T18:16:49.269 INFO:teuthology.orchestra.run.smithi133.stderr:2100000 of 50000000 tuples (4%) done (elapsed 5.06 s, remaining 115.31 s) 2023-07-14T18:16:49.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:48 smithi161 ceph-mon[128993]: pgmap v352: 129 pgs: 129 active+clean; 103 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 14 MiB/s wr, 83 op/s 2023-07-14T18:16:49.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:48 smithi171 ceph-mon[129241]: pgmap v352: 129 pgs: 129 active+clean; 103 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 14 MiB/s wr, 83 op/s 2023-07-14T18:16:49.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:48 smithi133 ceph-mon[126384]: pgmap v352: 129 pgs: 129 active+clean; 103 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 14 MiB/s wr, 83 op/s 2023-07-14T18:16:49.547 INFO:teuthology.orchestra.run.smithi133.stderr:2200000 of 50000000 tuples (4%) done (elapsed 5.33 s, remaining 115.89 s) 2023-07-14T18:16:49.936 INFO:teuthology.orchestra.run.smithi133.stderr:2300000 of 50000000 tuples (4%) done (elapsed 5.72 s, remaining 118.67 s) 2023-07-14T18:16:50.046 INFO:teuthology.orchestra.run.smithi133.stderr:2400000 of 50000000 tuples (4%) done (elapsed 5.83 s, remaining 115.69 s) 2023-07-14T18:16:50.373 INFO:teuthology.orchestra.run.smithi133.stderr:2500000 of 50000000 tuples (5%) done (elapsed 6.16 s, remaining 117.03 s) 2023-07-14T18:16:50.479 INFO:teuthology.orchestra.run.smithi133.stderr:2600000 of 50000000 tuples (5%) done (elapsed 6.27 s, remaining 114.22 s) 2023-07-14T18:16:50.813 INFO:teuthology.orchestra.run.smithi133.stderr:2700000 of 50000000 tuples (5%) done (elapsed 6.60 s, remaining 115.62 s) 2023-07-14T18:16:50.980 INFO:teuthology.orchestra.run.smithi133.stderr:2800000 of 50000000 tuples (5%) done (elapsed 6.77 s, remaining 114.07 s) 2023-07-14T18:16:51.232 INFO:teuthology.orchestra.run.smithi133.stderr:2900000 of 50000000 tuples (5%) done (elapsed 7.02 s, remaining 114.00 s) 2023-07-14T18:16:51.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:50 smithi161 ceph-mon[128993]: pgmap v353: 129 pgs: 129 active+clean; 138 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 20 MiB/s wr, 119 op/s 2023-07-14T18:16:51.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:50 smithi171 ceph-mon[129241]: pgmap v353: 129 pgs: 129 active+clean; 138 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 20 MiB/s wr, 119 op/s 2023-07-14T18:16:51.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:50 smithi133 ceph-mon[126384]: pgmap v353: 129 pgs: 129 active+clean; 138 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 20 MiB/s wr, 119 op/s 2023-07-14T18:16:51.532 INFO:teuthology.orchestra.run.smithi133.stderr:3000000 of 50000000 tuples (6%) done (elapsed 7.32 s, remaining 114.65 s) 2023-07-14T18:16:51.717 INFO:teuthology.orchestra.run.smithi133.stderr:3100000 of 50000000 tuples (6%) done (elapsed 7.50 s, remaining 113.53 s) 2023-07-14T18:16:51.967 INFO:teuthology.orchestra.run.smithi133.stderr:3200000 of 50000000 tuples (6%) done (elapsed 7.75 s, remaining 113.39 s) 2023-07-14T18:16:52.267 INFO:teuthology.orchestra.run.smithi133.stderr:3300000 of 50000000 tuples (6%) done (elapsed 8.05 s, remaining 113.98 s) 2023-07-14T18:16:52.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:51 smithi161 ceph-mon[128993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:16:52.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:51 smithi171 ceph-mon[129241]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:16:52.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:51 smithi133 ceph-mon[126384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:16:52.446 INFO:teuthology.orchestra.run.smithi133.stderr:3400000 of 50000000 tuples (6%) done (elapsed 8.23 s, remaining 112.83 s) 2023-07-14T18:16:52.727 INFO:teuthology.orchestra.run.smithi133.stderr:3500000 of 50000000 tuples (7%) done (elapsed 8.51 s, remaining 113.11 s) 2023-07-14T18:16:52.999 INFO:teuthology.orchestra.run.smithi133.stderr:3600000 of 50000000 tuples (7%) done (elapsed 8.79 s, remaining 113.24 s) 2023-07-14T18:16:53.168 INFO:teuthology.orchestra.run.smithi133.stderr:3700000 of 50000000 tuples (7%) done (elapsed 8.95 s, remaining 112.06 s) 2023-07-14T18:16:53.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:52 smithi161 ceph-mon[128993]: pgmap v354: 129 pgs: 129 active+clean; 222 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 34 MiB/s wr, 121 op/s 2023-07-14T18:16:53.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:52 smithi171 ceph-mon[129241]: pgmap v354: 129 pgs: 129 active+clean; 222 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 34 MiB/s wr, 121 op/s 2023-07-14T18:16:53.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:52 smithi133 ceph-mon[126384]: pgmap v354: 129 pgs: 129 active+clean; 222 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 34 MiB/s wr, 121 op/s 2023-07-14T18:16:53.423 INFO:teuthology.orchestra.run.smithi133.stderr:3800000 of 50000000 tuples (7%) done (elapsed 9.21 s, remaining 111.97 s) 2023-07-14T18:16:53.785 INFO:teuthology.orchestra.run.smithi133.stderr:3900000 of 50000000 tuples (7%) done (elapsed 9.57 s, remaining 113.13 s) 2023-07-14T18:16:53.998 INFO:teuthology.orchestra.run.smithi133.stderr:4000000 of 50000000 tuples (8%) done (elapsed 9.78 s, remaining 112.53 s) 2023-07-14T18:16:54.258 INFO:teuthology.orchestra.run.smithi133.stderr:4100000 of 50000000 tuples (8%) done (elapsed 10.04 s, remaining 112.45 s) 2023-07-14T18:16:54.554 INFO:teuthology.orchestra.run.smithi133.stderr:4200000 of 50000000 tuples (8%) done (elapsed 10.34 s, remaining 112.76 s) 2023-07-14T18:16:54.660 INFO:teuthology.orchestra.run.smithi133.stderr:4300000 of 50000000 tuples (8%) done (elapsed 10.45 s, remaining 111.02 s) 2023-07-14T18:16:54.931 INFO:teuthology.orchestra.run.smithi133.stderr:4400000 of 50000000 tuples (8%) done (elapsed 10.72 s, remaining 111.08 s) 2023-07-14T18:16:55.097 INFO:teuthology.orchestra.run.smithi133.stderr:4500000 of 50000000 tuples (9%) done (elapsed 10.88 s, remaining 110.05 s) 2023-07-14T18:16:55.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:54 smithi161 ceph-mon[128993]: pgmap v355: 129 pgs: 129 active+clean; 347 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 55 MiB/s wr, 138 op/s 2023-07-14T18:16:55.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:54 smithi171 ceph-mon[129241]: pgmap v355: 129 pgs: 129 active+clean; 347 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 55 MiB/s wr, 138 op/s 2023-07-14T18:16:55.405 INFO:teuthology.orchestra.run.smithi133.stderr:4600000 of 50000000 tuples (9%) done (elapsed 11.19 s, remaining 110.46 s) 2023-07-14T18:16:55.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:54 smithi133 ceph-mon[126384]: pgmap v355: 129 pgs: 129 active+clean; 347 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 55 MiB/s wr, 138 op/s 2023-07-14T18:16:55.688 INFO:teuthology.orchestra.run.smithi133.stderr:4700000 of 50000000 tuples (9%) done (elapsed 11.47 s, remaining 110.60 s) 2023-07-14T18:16:55.842 INFO:teuthology.orchestra.run.smithi133.stderr:4800000 of 50000000 tuples (9%) done (elapsed 11.63 s, remaining 109.51 s) 2023-07-14T18:16:56.139 INFO:teuthology.orchestra.run.smithi133.stderr:4900000 of 50000000 tuples (9%) done (elapsed 11.93 s, remaining 109.76 s) 2023-07-14T18:16:56.411 INFO:teuthology.orchestra.run.smithi133.stderr:5000000 of 50000000 tuples (10%) done (elapsed 12.20 s, remaining 109.78 s) 2023-07-14T18:16:56.585 INFO:teuthology.orchestra.run.smithi133.stderr:5100000 of 50000000 tuples (10%) done (elapsed 12.37 s, remaining 108.92 s) 2023-07-14T18:16:56.856 INFO:teuthology.orchestra.run.smithi133.stderr:5200000 of 50000000 tuples (10%) done (elapsed 12.64 s, remaining 108.92 s) 2023-07-14T18:16:57.161 INFO:teuthology.orchestra.run.smithi133.stderr:5300000 of 50000000 tuples (10%) done (elapsed 12.95 s, remaining 109.20 s) 2023-07-14T18:16:57.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:56 smithi161 ceph-mon[128993]: pgmap v356: 129 pgs: 129 active+clean; 363 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 56 MiB/s wr, 123 op/s 2023-07-14T18:16:57.358 INFO:teuthology.orchestra.run.smithi133.stderr:5400000 of 50000000 tuples (10%) done (elapsed 13.15 s, remaining 108.57 s) 2023-07-14T18:16:57.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:56 smithi171 ceph-mon[129241]: pgmap v356: 129 pgs: 129 active+clean; 363 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 56 MiB/s wr, 123 op/s 2023-07-14T18:16:57.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:56 smithi133 ceph-mon[126384]: pgmap v356: 129 pgs: 129 active+clean; 363 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 56 MiB/s wr, 123 op/s 2023-07-14T18:16:57.672 INFO:teuthology.orchestra.run.smithi133.stderr:5500000 of 50000000 tuples (11%) done (elapsed 13.46 s, remaining 108.89 s) 2023-07-14T18:16:58.024 INFO:teuthology.orchestra.run.smithi133.stderr:5600000 of 50000000 tuples (11%) done (elapsed 13.81 s, remaining 109.50 s) 2023-07-14T18:16:58.131 INFO:teuthology.orchestra.run.smithi133.stderr:5700000 of 50000000 tuples (11%) done (elapsed 13.92 s, remaining 108.17 s) 2023-07-14T18:16:58.491 INFO:teuthology.orchestra.run.smithi133.stderr:5800000 of 50000000 tuples (11%) done (elapsed 14.28 s, remaining 108.80 s) 2023-07-14T18:16:59.194 INFO:teuthology.orchestra.run.smithi133.stderr:5900000 of 50000000 tuples (11%) done (elapsed 14.98 s, remaining 111.97 s) 2023-07-14T18:16:59.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:16:58 smithi161 ceph-mon[128993]: pgmap v357: 129 pgs: 129 active+clean; 507 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 80 MiB/s wr, 136 op/s 2023-07-14T18:16:59.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:16:58 smithi171 ceph-mon[129241]: pgmap v357: 129 pgs: 129 active+clean; 507 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 80 MiB/s wr, 136 op/s 2023-07-14T18:16:59.413 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:16:58 smithi133 ceph-mon[126384]: pgmap v357: 129 pgs: 129 active+clean; 507 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 80 MiB/s wr, 136 op/s 2023-07-14T18:16:59.514 INFO:teuthology.orchestra.run.smithi133.stderr:6000000 of 50000000 tuples (12%) done (elapsed 15.30 s, remaining 112.20 s) 2023-07-14T18:17:00.206 INFO:teuthology.orchestra.run.smithi133.stderr:6100000 of 50000000 tuples (12%) done (elapsed 15.99 s, remaining 115.10 s) 2023-07-14T18:17:00.888 INFO:teuthology.orchestra.run.smithi133.stderr:6200000 of 50000000 tuples (12%) done (elapsed 16.67 s, remaining 117.80 s) 2023-07-14T18:17:01.000 INFO:teuthology.orchestra.run.smithi133.stderr:6300000 of 50000000 tuples (12%) done (elapsed 16.79 s, remaining 116.44 s) 2023-07-14T18:17:01.328 INFO:teuthology.orchestra.run.smithi133.stderr:6400000 of 50000000 tuples (12%) done (elapsed 17.11 s, remaining 116.59 s) 2023-07-14T18:17:01.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:00 smithi161 ceph-mon[128993]: pgmap v358: 129 pgs: 129 active+clean; 563 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 77 MiB/s wr, 86 op/s 2023-07-14T18:17:01.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:00 smithi171 ceph-mon[129241]: pgmap v358: 129 pgs: 129 active+clean; 563 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 77 MiB/s wr, 86 op/s 2023-07-14T18:17:01.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:00 smithi133 ceph-mon[126384]: pgmap v358: 129 pgs: 129 active+clean; 563 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 77 MiB/s wr, 86 op/s 2023-07-14T18:17:01.470 INFO:teuthology.orchestra.run.smithi133.stderr:6500000 of 50000000 tuples (13%) done (elapsed 17.26 s, remaining 115.49 s) 2023-07-14T18:17:01.758 INFO:teuthology.orchestra.run.smithi133.stderr:6600000 of 50000000 tuples (13%) done (elapsed 17.54 s, remaining 115.37 s) 2023-07-14T18:17:01.919 INFO:teuthology.orchestra.run.smithi133.stderr:6700000 of 50000000 tuples (13%) done (elapsed 17.71 s, remaining 114.43 s) 2023-07-14T18:17:02.178 INFO:teuthology.orchestra.run.smithi133.stderr:6800000 of 50000000 tuples (13%) done (elapsed 17.96 s, remaining 114.13 s) 2023-07-14T18:17:02.457 INFO:teuthology.orchestra.run.smithi133.stderr:6900000 of 50000000 tuples (13%) done (elapsed 18.24 s, remaining 113.96 s) 2023-07-14T18:17:02.785 INFO:teuthology.orchestra.run.smithi133.stderr:7000000 of 50000000 tuples (14%) done (elapsed 18.57 s, remaining 114.08 s) 2023-07-14T18:17:02.889 INFO:teuthology.orchestra.run.smithi133.stderr:7100000 of 50000000 tuples (14%) done (elapsed 18.68 s, remaining 112.84 s) 2023-07-14T18:17:03.261 INFO:teuthology.orchestra.run.smithi133.stderr:7200000 of 50000000 tuples (14%) done (elapsed 19.05 s, remaining 113.23 s) 2023-07-14T18:17:03.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:02 smithi161 ceph-mon[128993]: pgmap v359: 129 pgs: 129 active+clean; 976 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 111 MiB/s wr, 62 op/s 2023-07-14T18:17:03.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:02 smithi171 ceph-mon[129241]: pgmap v359: 129 pgs: 129 active+clean; 976 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 111 MiB/s wr, 62 op/s 2023-07-14T18:17:03.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:02 smithi133 ceph-mon[126384]: pgmap v359: 129 pgs: 129 active+clean; 976 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 111 MiB/s wr, 62 op/s 2023-07-14T18:17:03.632 INFO:teuthology.orchestra.run.smithi133.stderr:7300000 of 50000000 tuples (14%) done (elapsed 19.42 s, remaining 113.59 s) 2023-07-14T18:17:03.736 INFO:teuthology.orchestra.run.smithi133.stderr:7400000 of 50000000 tuples (14%) done (elapsed 19.52 s, remaining 112.39 s) 2023-07-14T18:17:04.072 INFO:teuthology.orchestra.run.smithi133.stderr:7500000 of 50000000 tuples (15%) done (elapsed 19.86 s, remaining 112.53 s) 2023-07-14T18:17:04.178 INFO:teuthology.orchestra.run.smithi133.stderr:7600000 of 50000000 tuples (15%) done (elapsed 19.96 s, remaining 111.38 s) 2023-07-14T18:17:04.457 INFO:teuthology.orchestra.run.smithi133.stderr:7700000 of 50000000 tuples (15%) done (elapsed 20.24 s, remaining 111.21 s) 2023-07-14T18:17:04.724 INFO:teuthology.orchestra.run.smithi133.stderr:7800000 of 50000000 tuples (15%) done (elapsed 20.51 s, remaining 110.97 s) 2023-07-14T18:17:04.883 INFO:teuthology.orchestra.run.smithi133.stderr:7900000 of 50000000 tuples (15%) done (elapsed 20.67 s, remaining 110.14 s) 2023-07-14T18:17:05.184 INFO:teuthology.orchestra.run.smithi133.stderr:8000000 of 50000000 tuples (16%) done (elapsed 20.97 s, remaining 110.09 s) 2023-07-14T18:17:05.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:04 smithi161 ceph-mon[128993]: pgmap v360: 129 pgs: 129 active+clean; 1.4 GiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 139 MiB/s wr, 79 op/s 2023-07-14T18:17:05.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:04 smithi171 ceph-mon[129241]: pgmap v360: 129 pgs: 129 active+clean; 1.4 GiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 139 MiB/s wr, 79 op/s 2023-07-14T18:17:05.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:04 smithi133 ceph-mon[126384]: pgmap v360: 129 pgs: 129 active+clean; 1.4 GiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 139 MiB/s wr, 79 op/s 2023-07-14T18:17:05.470 INFO:teuthology.orchestra.run.smithi133.stderr:8100000 of 50000000 tuples (16%) done (elapsed 21.26 s, remaining 109.96 s) 2023-07-14T18:17:05.634 INFO:teuthology.orchestra.run.smithi133.stderr:8200000 of 50000000 tuples (16%) done (elapsed 21.42 s, remaining 109.19 s) 2023-07-14T18:17:05.859 INFO:teuthology.orchestra.run.smithi133.stderr:8300000 of 50000000 tuples (16%) done (elapsed 21.65 s, remaining 108.75 s) 2023-07-14T18:17:06.033 INFO:teuthology.orchestra.run.smithi133.stderr:8400000 of 50000000 tuples (16%) done (elapsed 21.82 s, remaining 108.06 s) 2023-07-14T18:17:06.089 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:17:06.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.265+0000 7f4003bff700 1 -- 172.21.15.133:0/386542766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ffc074680 msgr2=0x7f3ffc072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:06.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.265+0000 7f4003bff700 1 --2- 172.21.15.133:0/386542766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ffc074680 0x7f3ffc072ad0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009990 tx=0x7f3ff8009ca0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.265+0000 7f4003bff700 1 -- 172.21.15.133:0/386542766 shutdown_connections 2023-07-14T18:17:06.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.265+0000 7f4003bff700 1 --2- 172.21.15.133:0/386542766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ffc074680 0x7f3ffc072ad0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.265+0000 7f4003bff700 1 --2- 172.21.15.133:0/386542766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ffc10c1f0 0x7f3ffc10e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.265+0000 7f4003bff700 1 --2- 172.21.15.133:0/386542766 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ffc0730a0 0x7f3ffc073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.265+0000 7f4003bff700 1 -- 172.21.15.133:0/386542766 >> 172.21.15.133:0/386542766 conn(0x7f3ffc0fb260 msgr2=0x7f3ffc0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:06.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.265+0000 7f4003bff700 1 -- 172.21.15.133:0/386542766 shutdown_connections 2023-07-14T18:17:06.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.265+0000 7f4003bff700 1 -- 172.21.15.133:0/386542766 wait complete. 2023-07-14T18:17:06.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.267+0000 7f4003bff700 1 Processor -- start 2023-07-14T18:17:06.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.267+0000 7f4003bff700 1 -- start start 2023-07-14T18:17:06.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.267+0000 7f4003bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ffc0730a0 0x7f3ffc19bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:06.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.268+0000 7f4003bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ffc074680 0x7f3ffc19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:06.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.268+0000 7f4003bff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ffc10c1f0 0x7f3ffc1a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:06.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.268+0000 7f4003bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc1a3b40 con 0x7f3ffc074680 2023-07-14T18:17:06.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.268+0000 7f4003bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc1a3c80 con 0x7f3ffc0730a0 2023-07-14T18:17:06.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.268+0000 7f4003bff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc1a3e10 con 0x7f3ffc10c1f0 2023-07-14T18:17:06.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.268+0000 7f400199b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ffc0730a0 0x7f3ffc19bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:06.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.268+0000 7f400199b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ffc0730a0 0x7f3ffc19bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:58158/0 (socket says 172.21.15.133:58158) 2023-07-14T18:17:06.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.268+0000 7f400119a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ffc074680 0x7f3ffc19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:06.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.268+0000 7f400199b700 1 -- 172.21.15.133:0/1052232662 learned_addr learned my addr 172.21.15.133:0/1052232662 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:06.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.268+0000 7f400219c700 1 --2- 172.21.15.133:0/1052232662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ffc10c1f0 0x7f3ffc1a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:06.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.268+0000 7f400199b700 1 -- 172.21.15.133:0/1052232662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ffc10c1f0 msgr2=0x7f3ffc1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:06.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.268+0000 7f400199b700 1 --2- 172.21.15.133:0/1052232662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ffc10c1f0 0x7f3ffc1a3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.268+0000 7f400199b700 1 -- 172.21.15.133:0/1052232662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ffc074680 msgr2=0x7f3ffc19c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:06.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.268+0000 7f400199b700 1 --2- 172.21.15.133:0/1052232662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ffc074680 0x7f3ffc19c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.268+0000 7f400199b700 1 -- 172.21.15.133:0/1052232662 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff80095f0 con 0x7f3ffc0730a0 2023-07-14T18:17:06.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.268+0000 7f400199b700 1 --2- 172.21.15.133:0/1052232662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ffc0730a0 0x7f3ffc19bfc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8005720 tx=0x7f3ff8009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:06.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.269+0000 7f3ff2ffd700 1 -- 172.21.15.133:0/1052232662 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff801d070 con 0x7f3ffc0730a0 2023-07-14T18:17:06.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.269+0000 7f3ff2ffd700 1 -- 172.21.15.133:0/1052232662 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ff8021410 con 0x7f3ffc0730a0 2023-07-14T18:17:06.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.269+0000 7f4003bff700 1 -- 172.21.15.133:0/1052232662 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ffc1a4090 con 0x7f3ffc0730a0 2023-07-14T18:17:06.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.269+0000 7f3ff2ffd700 1 -- 172.21.15.133:0/1052232662 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8017650 con 0x7f3ffc0730a0 2023-07-14T18:17:06.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.269+0000 7f4003bff700 1 -- 172.21.15.133:0/1052232662 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ffc1a4510 con 0x7f3ffc0730a0 2023-07-14T18:17:06.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.272+0000 7f3ff2ffd700 1 -- 172.21.15.133:0/1052232662 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f3ff8021580 con 0x7f3ffc0730a0 2023-07-14T18:17:06.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.274+0000 7f40175e8b80 1 Processor -- start 2023-07-14T18:17:06.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.274+0000 7f40175e8b80 1 -- start start 2023-07-14T18:17:06.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.274+0000 7f40175e8b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5646ca65cf60 0x5646ca65d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:06.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.274+0000 7f40175e8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646ca65d8b0 0x5646ca65ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:06.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.274+0000 7f40175e8b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5646ca65e4b0 0x5646ca65e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:06.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.274+0000 7f40175e8b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646ca561cb0 con 0x5646ca65e4b0 2023-07-14T18:17:06.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.274+0000 7f40175e8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646ca5b4a40 con 0x5646ca65d8b0 2023-07-14T18:17:06.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.274+0000 7f40175e8b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646ca5aded0 con 0x5646ca65cf60 2023-07-14T18:17:06.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.274+0000 7f3ff2ffd700 1 --2- 172.21.15.133:0/1052232662 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3fe80a11d0 0x7f3fe80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:06.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.275+0000 7f3ff2ffd700 1 -- 172.21.15.133:0/1052232662 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3ff80cdd80 con 0x7f3ffc0730a0 2023-07-14T18:17:06.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.275+0000 7f400199b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646ca65d8b0 0x5646ca65ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:06.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.275+0000 7f400119a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5646ca65e4b0 0x5646ca65e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:06.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.275+0000 7f400199b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646ca65d8b0 0x5646ca65ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:58174/0 (socket says 172.21.15.133:58174) 2023-07-14T18:17:06.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.275+0000 7f400199b700 1 -- 172.21.15.133:0/535682901 learned_addr learned my addr 172.21.15.133:0/535682901 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:06.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.275+0000 7f400199b700 1 -- 172.21.15.133:0/535682901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5646ca65cf60 msgr2=0x5646ca65d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:06.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.275+0000 7f400219c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5646ca65cf60 0x5646ca65d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:06.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.275+0000 7f400199b700 1 --2- 172.21.15.133:0/535682901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5646ca65cf60 0x5646ca65d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.275+0000 7f400199b700 1 -- 172.21.15.133:0/535682901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5646ca65e4b0 msgr2=0x5646ca65e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:06.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.275+0000 7f400199b700 1 --2- 172.21.15.133:0/535682901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5646ca65e4b0 0x5646ca65e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.275+0000 7f400199b700 1 -- 172.21.15.133:0/535682901 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646ca3dbde0 con 0x5646ca65d8b0 2023-07-14T18:17:06.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.275+0000 7f400119a700 1 --2- 172.21.15.133:0/535682901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5646ca65e4b0 0x5646ca65e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:17:06.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.275+0000 7f400219c700 1 --2- 172.21.15.133:0/535682901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5646ca65cf60 0x5646ca65d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T18:17:06.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.275+0000 7f400219c700 1 --2- 172.21.15.133:0/1052232662 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3fe80a11d0 0x7f3fe80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:06.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.276+0000 7f400199b700 1 --2- 172.21.15.133:0/535682901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646ca65d8b0 0x5646ca65ded0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3ff80cb4f0 tx=0x7f3ff809ad90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e616bf83a69764a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:06.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.276+0000 7f3fe7fff700 1 -- 172.21.15.133:0/535682901 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff80aa0b0 con 0x5646ca65d8b0 2023-07-14T18:17:06.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.276+0000 7f3fe7fff700 1 -- 172.21.15.133:0/535682901 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ff8005750 con 0x5646ca65d8b0 2023-07-14T18:17:06.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.276+0000 7f3fe7fff700 1 -- 172.21.15.133:0/535682901 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff80aa6b0 con 0x5646ca65d8b0 2023-07-14T18:17:06.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.277+0000 7f40175e8b80 1 -- 172.21.15.133:0/535682901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646ca65d8b0 msgr2=0x5646ca65ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:06.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.277+0000 7f400219c700 1 --2- 172.21.15.133:0/1052232662 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3fe80a11d0 0x7f3fe80a3680 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009550 tx=0x7f3ff400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:06.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.277+0000 7f40175e8b80 1 --2- 172.21.15.133:0/535682901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646ca65d8b0 0x5646ca65ded0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3ff80cb4f0 tx=0x7f3ff809ad90 comp rx=0 tx=0).stop 2023-07-14T18:17:06.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.277+0000 7f40175e8b80 1 -- 172.21.15.133:0/535682901 shutdown_connections 2023-07-14T18:17:06.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.277+0000 7f40175e8b80 1 --2- 172.21.15.133:0/535682901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5646ca65e4b0 0x5646ca65e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.277+0000 7f40175e8b80 1 --2- 172.21.15.133:0/535682901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5646ca65cf60 0x5646ca65d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.277+0000 7f40175e8b80 1 --2- 172.21.15.133:0/535682901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646ca65d8b0 0x5646ca65ded0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.277+0000 7f40175e8b80 1 -- 172.21.15.133:0/535682901 >> 172.21.15.133:0/535682901 conn(0x5646ca652680 msgr2=0x5646ca6579d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:06.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.277+0000 7f40175e8b80 1 -- 172.21.15.133:0/535682901 shutdown_connections 2023-07-14T18:17:06.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.277+0000 7f40175e8b80 1 -- 172.21.15.133:0/535682901 wait complete. 2023-07-14T18:17:06.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.277+0000 7f40175e8b80 1 -- start start 2023-07-14T18:17:06.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.278+0000 7f40175e8b80 1 Processor -- start 2023-07-14T18:17:06.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.278+0000 7f40175e8b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5646ca65cf60 0x5646ca6665c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:06.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.279+0000 7f40175e8b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5646ca65d8b0 0x5646ca666b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:06.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.279+0000 7f40175e8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646ca65e4b0 0x5646ca66db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:06.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.279+0000 7f40175e8b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646ca54dac0 con 0x5646ca65d8b0 2023-07-14T18:17:06.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.279+0000 7f40175e8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646ca54d7c0 con 0x5646ca65e4b0 2023-07-14T18:17:06.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.279+0000 7f40175e8b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646ca431920 con 0x5646ca65cf60 2023-07-14T18:17:06.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.279+0000 7f40175e8b80 10 client.0 _ll_register_callbacks cb 0x5646ca54b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:17:06.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.279+0000 7f400199b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5646ca65cf60 0x5646ca6665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:06.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.279+0000 7f400199b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5646ca65cf60 0x5646ca6665c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:52288/0 (socket says 172.21.15.133:52288) 2023-07-14T18:17:06.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.279+0000 7f400199b700 1 -- 172.21.15.133:0/962978632 learned_addr learned my addr 172.21.15.133:0/962978632 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:06.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.279+0000 7f400119a700 1 --2- 172.21.15.133:0/962978632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5646ca65d8b0 0x5646ca666b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:06.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.279+0000 7f400219c700 1 --2- 172.21.15.133:0/962978632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646ca65e4b0 0x5646ca66db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:06.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.279+0000 7f400199b700 1 -- 172.21.15.133:0/962978632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646ca65e4b0 msgr2=0x5646ca66db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:06.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.279+0000 7f400199b700 1 --2- 172.21.15.133:0/962978632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646ca65e4b0 0x5646ca66db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.279+0000 7f400199b700 1 -- 172.21.15.133:0/962978632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5646ca65d8b0 msgr2=0x5646ca666b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:06.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.279+0000 7f400199b700 1 --2- 172.21.15.133:0/962978632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5646ca65d8b0 0x5646ca666b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.279+0000 7f400199b700 1 -- 172.21.15.133:0/962978632 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fec0098e0 con 0x5646ca65cf60 2023-07-14T18:17:06.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.279+0000 7f400199b700 1 --2- 172.21.15.133:0/962978632 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5646ca65cf60 0x5646ca6665c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009350 tx=0x7f3ff809a920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c7799414e00eb6a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:06.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.279+0000 7f3fe57fa700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:17:06.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.279+0000 7f3fe57fa700 1 -- 172.21.15.133:0/962978632 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff809eec0 con 0x5646ca65cf60 2023-07-14T18:17:06.333 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.280+0000 7f3fe57fa700 1 -- 172.21.15.133:0/962978632 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3ff809b300 con 0x5646ca65cf60 2023-07-14T18:17:06.333 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.280+0000 7f40175e8b80 1 -- 172.21.15.133:0/962978632 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5646ca5074f0 con 0x5646ca65cf60 2023-07-14T18:17:06.333 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.280+0000 7f3fe57fa700 1 -- 172.21.15.133:0/962978632 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ff80b34d0 con 0x5646ca65cf60 2023-07-14T18:17:06.333 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.280+0000 7f40175e8b80 10 client.25543 fetch_fsmap learned FSMap version 28 2023-07-14T18:17:06.333 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.280+0000 7f40175e8b80 1 -- 172.21.15.133:0/962978632 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f3ffc1a4510 con 0x5646ca65cf60 2023-07-14T18:17:06.333 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.280+0000 7f3fe57fa700 1 -- 172.21.15.133:0/962978632 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5646ca5074f0 con 0x5646ca65cf60 2023-07-14T18:17:06.333 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.280+0000 7f3fe57fa700 1 -- 172.21.15.133:0/962978632 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3ff80a2ea0 con 0x5646ca65cf60 2023-07-14T18:17:06.333 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.280+0000 7f40175e8b80 10 client.25543 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T18:17:06.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.280+0000 7f40175e8b80 10 client.25543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:17:06.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.280+0000 7f40175e8b80 20 client.25543 populate_metadata read hostname 'smithi133' 2023-07-14T18:17:06.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.280+0000 7f40175e8b80 1 --2- 172.21.15.133:0/962978632 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5646ca671cb0 0x5646ca6740c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:06.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.280+0000 7f40175e8b80 4 client.25543 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:17:06.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.280+0000 7f40175e8b80 1 -- 172.21.15.133:0/962978632 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5646ca62bda0 con 0x5646ca671cb0 2023-07-14T18:17:06.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.281+0000 7f400119a700 1 --2- 172.21.15.133:0/962978632 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5646ca671cb0 0x5646ca6740c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:06.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.281+0000 7f3fe57fa700 10 client.25543 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:17:06.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.281+0000 7f400119a700 1 --2- 172.21.15.133:0/962978632 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5646ca671cb0 0x5646ca6740c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=294091fff760e19d server_cookie=bf24b49a890a5710 in_seq=0 out_seq=0 2023-07-14T18:17:06.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.282+0000 7f3fe57fa700 1 -- 172.21.15.133:0/962978632 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5646ca62bda0 con 0x5646ca671cb0 2023-07-14T18:17:06.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.282+0000 7f3fe57fa700 10 client.25543 handle_command_reply: tid=0 2023-07-14T18:17:06.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.282+0000 7f40175e8b80 1 client.25543 shutdown 2023-07-14T18:17:06.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.282+0000 7f40175e8b80 1 -- 172.21.15.133:0/962978632 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5646ca65cf60 msgr2=0x5646ca6665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:06.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.282+0000 7f40175e8b80 1 --2- 172.21.15.133:0/962978632 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5646ca65cf60 0x5646ca6665c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009350 tx=0x7f3ff809a920 comp rx=0 tx=0).stop 2023-07-14T18:17:06.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.282+0000 7f40175e8b80 1 -- 172.21.15.133:0/962978632 shutdown_connections 2023-07-14T18:17:06.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.282+0000 7f40175e8b80 1 --2- 172.21.15.133:0/962978632 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5646ca671cb0 0x5646ca6740c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.282+0000 7f40175e8b80 1 --2- 172.21.15.133:0/962978632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646ca65e4b0 0x5646ca66db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.282+0000 7f40175e8b80 1 --2- 172.21.15.133:0/962978632 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5646ca65cf60 0x5646ca6665c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.282+0000 7f3fe57fa700 0 client.25543 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:17:06.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.282+0000 7f40175e8b80 1 --2- 172.21.15.133:0/962978632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5646ca65d8b0 0x5646ca666b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.282+0000 7f40175e8b80 1 -- 172.21.15.133:0/962978632 >> 172.21.15.133:0/962978632 conn(0x5646ca652680 msgr2=0x5646ca656650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:06.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.282+0000 7f40175e8b80 1 -- 172.21.15.133:0/962978632 shutdown_connections 2023-07-14T18:17:06.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.282+0000 7f40175e8b80 1 -- 172.21.15.133:0/962978632 wait complete. 2023-07-14T18:17:06.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.283+0000 7f40175e8b80 20 client.25543 trim_cache size 0 max 16384 2023-07-14T18:17:06.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.300+0000 7f40175e8b80 1 Processor -- start 2023-07-14T18:17:06.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.300+0000 7f40175e8b80 1 -- start start 2023-07-14T18:17:06.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.300+0000 7f40175e8b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5646ca65cf60 0x5646ca665340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:06.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.300+0000 7f40175e8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646ca65d8b0 0x5646ca66db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:06.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.300+0000 7f40175e8b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5646ca65e4b0 0x5646ca6645c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:06.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.300+0000 7f40175e8b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646ca665a70 con 0x5646ca65cf60 2023-07-14T18:17:06.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.300+0000 7f40175e8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646ca6588b0 con 0x5646ca65d8b0 2023-07-14T18:17:06.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.300+0000 7f40175e8b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646ca659500 con 0x5646ca65e4b0 2023-07-14T18:17:06.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.301+0000 7f400119a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646ca65d8b0 0x5646ca66db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:06.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.301+0000 7f400199b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5646ca65cf60 0x5646ca665340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:06.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.301+0000 7f400119a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646ca65d8b0 0x5646ca66db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:58200/0 (socket says 172.21.15.133:58200) 2023-07-14T18:17:06.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.301+0000 7f400119a700 1 -- 172.21.15.133:0/2025590584 learned_addr learned my addr 172.21.15.133:0/2025590584 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:06.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.301+0000 7f400219c700 1 --2- 172.21.15.133:0/2025590584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5646ca65e4b0 0x5646ca6645c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:06.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.301+0000 7f400119a700 1 -- 172.21.15.133:0/2025590584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5646ca65e4b0 msgr2=0x5646ca6645c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:06.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.301+0000 7f400119a700 1 --2- 172.21.15.133:0/2025590584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5646ca65e4b0 0x5646ca6645c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.301+0000 7f400119a700 1 -- 172.21.15.133:0/2025590584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5646ca65cf60 msgr2=0x5646ca665340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:06.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.301+0000 7f400119a700 1 --2- 172.21.15.133:0/2025590584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5646ca65cf60 0x5646ca665340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.301+0000 7f400119a700 1 -- 172.21.15.133:0/2025590584 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646ca3c4c60 con 0x5646ca65d8b0 2023-07-14T18:17:06.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.301+0000 7f400219c700 1 --2- 172.21.15.133:0/2025590584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5646ca65e4b0 0x5646ca6645c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:17:06.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.301+0000 7f400119a700 1 --2- 172.21.15.133:0/2025590584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646ca65d8b0 0x5646ca66db00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x5646ca674970 tx=0x7f3fec009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7d5403ae3861f86 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:06.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.302+0000 7f3ff0bff700 1 -- 172.21.15.133:0/2025590584 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec01f250 con 0x5646ca65d8b0 2023-07-14T18:17:06.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.302+0000 7f3ff0bff700 1 -- 172.21.15.133:0/2025590584 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fec018040 con 0x5646ca65d8b0 2023-07-14T18:17:06.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.302+0000 7f3ff0bff700 1 -- 172.21.15.133:0/2025590584 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec004830 con 0x5646ca65d8b0 2023-07-14T18:17:06.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.302+0000 7f40175e8b80 1 -- 172.21.15.133:0/2025590584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646ca65d8b0 msgr2=0x5646ca66db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:06.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.302+0000 7f40175e8b80 1 --2- 172.21.15.133:0/2025590584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646ca65d8b0 0x5646ca66db00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x5646ca674970 tx=0x7f3fec009260 comp rx=0 tx=0).stop 2023-07-14T18:17:06.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.302+0000 7f40175e8b80 1 -- 172.21.15.133:0/2025590584 shutdown_connections 2023-07-14T18:17:06.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.302+0000 7f40175e8b80 1 --2- 172.21.15.133:0/2025590584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5646ca65e4b0 0x5646ca6645c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.302+0000 7f40175e8b80 1 --2- 172.21.15.133:0/2025590584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5646ca65cf60 0x5646ca665340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.302+0000 7f40175e8b80 1 --2- 172.21.15.133:0/2025590584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646ca65d8b0 0x5646ca66db00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.302+0000 7f40175e8b80 1 -- 172.21.15.133:0/2025590584 >> 172.21.15.133:0/2025590584 conn(0x5646ca652680 msgr2=0x5646ca661390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:06.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.302+0000 7f40175e8b80 1 -- 172.21.15.133:0/2025590584 shutdown_connections 2023-07-14T18:17:06.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.302+0000 7f40175e8b80 1 -- 172.21.15.133:0/2025590584 wait complete. 2023-07-14T18:17:06.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.303+0000 7f40175e8b80 1 -- start start 2023-07-14T18:17:06.354 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.303+0000 7f40175e8b80 1 Processor -- start 2023-07-14T18:17:06.355 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.304+0000 7f40175e8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646ca65cf60 0x5646ca66d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:06.355 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.304+0000 7f40175e8b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5646ca65d8b0 0x5646ca66a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:06.355 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.304+0000 7f40175e8b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5646ca65e4b0 0x5646ca66a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:06.355 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.304+0000 7f40175e8b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646ca66ade0 con 0x5646ca65e4b0 2023-07-14T18:17:06.355 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.304+0000 7f40175e8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646ca66af50 con 0x5646ca65cf60 2023-07-14T18:17:06.355 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.304+0000 7f40175e8b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646ca6c5990 con 0x5646ca65d8b0 2023-07-14T18:17:06.356 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.304+0000 7f40175e8b80 10 client.0 _ll_register_callbacks cb 0x7f3fec009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:17:06.356 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.304+0000 7f400219c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5646ca65e4b0 0x5646ca66a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:06.356 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.304+0000 7f400119a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5646ca65d8b0 0x5646ca66a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:06.356 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.304+0000 7f400119a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5646ca65d8b0 0x5646ca66a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:52318/0 (socket says 172.21.15.133:52318) 2023-07-14T18:17:06.356 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.304+0000 7f400119a700 1 -- 172.21.15.133:0/1126281481 learned_addr learned my addr 172.21.15.133:0/1126281481 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:06.356 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.304+0000 7f400219c700 1 --2- 172.21.15.133:0/1126281481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5646ca65e4b0 0x5646ca66a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39390/0 (socket says 172.21.15.133:39390) 2023-07-14T18:17:06.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.304+0000 7f400199b700 1 --2- 172.21.15.133:0/1126281481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646ca65cf60 0x5646ca66d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:06.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.304+0000 7f400119a700 1 -- 172.21.15.133:0/1126281481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646ca65cf60 msgr2=0x5646ca66d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:06.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.304+0000 7f400119a700 1 --2- 172.21.15.133:0/1126281481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646ca65cf60 0x5646ca66d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.304+0000 7f400119a700 1 -- 172.21.15.133:0/1126281481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5646ca65e4b0 msgr2=0x5646ca66a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:06.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.304+0000 7f400119a700 1 --2- 172.21.15.133:0/1126281481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5646ca65e4b0 0x5646ca66a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.305+0000 7f400119a700 1 -- 172.21.15.133:0/1126281481 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fec01f930 con 0x5646ca65d8b0 2023-07-14T18:17:06.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.305+0000 7f400199b700 1 --2- 172.21.15.133:0/1126281481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646ca65cf60 0x5646ca66d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:17:06.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.305+0000 7f400119a700 1 --2- 172.21.15.133:0/1126281481 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5646ca65d8b0 0x5646ca66a360 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3fec005c50 tx=0x7f3fec01f280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4920b97ce28964a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:06.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.305+0000 7f3fe57fa700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:17:06.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.305+0000 7f3fe57fa700 1 -- 172.21.15.133:0/1126281481 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec004040 con 0x5646ca65d8b0 2023-07-14T18:17:06.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.305+0000 7f40175e8b80 1 -- 172.21.15.133:0/1126281481 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ff8005750 con 0x5646ca65d8b0 2023-07-14T18:17:06.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.305+0000 7f3fe57fa700 1 -- 172.21.15.133:0/1126281481 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3fec0041a0 con 0x5646ca65d8b0 2023-07-14T18:17:06.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.305+0000 7f3fe57fa700 1 -- 172.21.15.133:0/1126281481 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fec004360 con 0x5646ca65d8b0 2023-07-14T18:17:06.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.305+0000 7f40175e8b80 10 client.25549 fetch_fsmap learned FSMap version 28 2023-07-14T18:17:06.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.305+0000 7f40175e8b80 1 -- 172.21.15.133:0/1126281481 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5646ca402250 con 0x5646ca65d8b0 2023-07-14T18:17:06.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.305+0000 7f3fe57fa700 1 -- 172.21.15.133:0/1126281481 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ff8005750 con 0x5646ca65d8b0 2023-07-14T18:17:06.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.306+0000 7f3fe57fa700 1 -- 172.21.15.133:0/1126281481 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3fec027640 con 0x5646ca65d8b0 2023-07-14T18:17:06.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.306+0000 7f40175e8b80 10 client.25549 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T18:17:06.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.306+0000 7f40175e8b80 10 client.25549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:17:06.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.306+0000 7f40175e8b80 20 client.25549 populate_metadata read hostname 'smithi133' 2023-07-14T18:17:06.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.306+0000 7f40175e8b80 1 --2- 172.21.15.133:0/1126281481 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5646ca6c72d0 0x5646ca6c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:06.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.306+0000 7f40175e8b80 4 client.25549 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:17:06.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.306+0000 7f40175e8b80 1 -- 172.21.15.133:0/1126281481 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5646ca5a6eb0 con 0x5646ca6c72d0 2023-07-14T18:17:06.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.306+0000 7f400199b700 1 --2- 172.21.15.133:0/1126281481 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5646ca6c72d0 0x5646ca6c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:06.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.306+0000 7f3fe57fa700 10 client.25549 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:17:06.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.306+0000 7f400199b700 1 --2- 172.21.15.133:0/1126281481 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5646ca6c72d0 0x5646ca6c96e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a50c1b94e7c03e5 server_cookie=3d8ebed556eb2ca8 in_seq=0 out_seq=0 2023-07-14T18:17:06.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.307+0000 7f3fe57fa700 1 -- 172.21.15.133:0/1126281481 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5646ca5a6eb0 con 0x5646ca6c72d0 2023-07-14T18:17:06.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.307+0000 7f3fe57fa700 10 client.25549 handle_command_reply: tid=0 2023-07-14T18:17:06.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.307+0000 7f40175e8b80 1 client.25549 shutdown 2023-07-14T18:17:06.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.307+0000 7f40175e8b80 1 -- 172.21.15.133:0/1126281481 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5646ca65d8b0 msgr2=0x5646ca66a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:06.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.307+0000 7f40175e8b80 1 --2- 172.21.15.133:0/1126281481 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5646ca65d8b0 0x5646ca66a360 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3fec005c50 tx=0x7f3fec01f280 comp rx=0 tx=0).stop 2023-07-14T18:17:06.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.307+0000 7f40175e8b80 1 -- 172.21.15.133:0/1126281481 shutdown_connections 2023-07-14T18:17:06.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.307+0000 7f40175e8b80 1 --2- 172.21.15.133:0/1126281481 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5646ca6c72d0 0x5646ca6c96e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.307+0000 7f40175e8b80 1 --2- 172.21.15.133:0/1126281481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646ca65cf60 0x5646ca66d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.307+0000 7f40175e8b80 1 --2- 172.21.15.133:0/1126281481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5646ca65e4b0 0x5646ca66a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.307+0000 7f40175e8b80 1 --2- 172.21.15.133:0/1126281481 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5646ca65d8b0 0x5646ca66a360 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.307+0000 7f3fe57fa700 0 client.25549 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:17:06.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.307+0000 7f40175e8b80 1 -- 172.21.15.133:0/1126281481 >> 172.21.15.133:0/1126281481 conn(0x5646ca652680 msgr2=0x5646ca666370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:06.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.307+0000 7f40175e8b80 1 -- 172.21.15.133:0/1126281481 shutdown_connections 2023-07-14T18:17:06.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.307+0000 7f40175e8b80 1 -- 172.21.15.133:0/1126281481 wait complete. 2023-07-14T18:17:06.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.307+0000 7f40175e8b80 20 client.25549 trim_cache size 0 max 16384 2023-07-14T18:17:06.363 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:17:06.363 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2023-07-14T18:17:06.363 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2023-07-14T18:17:06.363 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:17:06.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.308+0000 7f4003bff700 1 -- 172.21.15.133:0/1052232662 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3fe80a11d0 msgr2=0x7f3fe80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:06.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.308+0000 7f4003bff700 1 --2- 172.21.15.133:0/1052232662 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3fe80a11d0 0x7f3fe80a3680 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009550 tx=0x7f3ff400b040 comp rx=0 tx=0).stop 2023-07-14T18:17:06.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.308+0000 7f4003bff700 1 -- 172.21.15.133:0/1052232662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ffc0730a0 msgr2=0x7f3ffc19bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:06.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.308+0000 7f4003bff700 1 --2- 172.21.15.133:0/1052232662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ffc0730a0 0x7f3ffc19bfc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8005720 tx=0x7f3ff8009580 comp rx=0 tx=0).stop 2023-07-14T18:17:06.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.308+0000 7f4003bff700 1 -- 172.21.15.133:0/1052232662 shutdown_connections 2023-07-14T18:17:06.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.308+0000 7f4003bff700 1 --2- 172.21.15.133:0/1052232662 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3fe80a11d0 0x7f3fe80a3680 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.308+0000 7f4003bff700 1 --2- 172.21.15.133:0/1052232662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ffc0730a0 0x7f3ffc19bfc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.308+0000 7f4003bff700 1 --2- 172.21.15.133:0/1052232662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ffc10c1f0 0x7f3ffc1a3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.308+0000 7f4003bff700 1 --2- 172.21.15.133:0/1052232662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ffc074680 0x7f3ffc19c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.308+0000 7f4003bff700 1 -- 172.21.15.133:0/1052232662 >> 172.21.15.133:0/1052232662 conn(0x7f3ffc0fb260 msgr2=0x7f3ffc0fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:06.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.308+0000 7f4003bff700 1 -- 172.21.15.133:0/1052232662 shutdown_connections 2023-07-14T18:17:06.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.308+0000 7f4003bff700 1 -- 172.21.15.133:0/1052232662 wait complete. 2023-07-14T18:17:06.372 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T18:17:06.373 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T18:17:06.377 INFO:teuthology.orchestra.run.smithi133.stderr:8500000 of 50000000 tuples (17%) done (elapsed 22.16 s, remaining 108.21 s) 2023-07-14T18:17:06.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.543+0000 7f4f3adff700 1 -- 172.21.15.133:0/537799415 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f340ffc40 msgr2=0x7f4f34100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:06.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.543+0000 7f4f3adff700 1 --2- 172.21.15.133:0/537799415 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f340ffc40 0x7f4f34100050 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009f90 tx=0x7f4f2800b330 comp rx=0 tx=0).stop 2023-07-14T18:17:06.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.543+0000 7f4f3adff700 1 -- 172.21.15.133:0/537799415 shutdown_connections 2023-07-14T18:17:06.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.543+0000 7f4f3adff700 1 --2- 172.21.15.133:0/537799415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f341017c0 0x7f4f34108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.543+0000 7f4f3adff700 1 --2- 172.21.15.133:0/537799415 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f340ffc40 0x7f4f34100050 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.543+0000 7f4f3adff700 1 --2- 172.21.15.133:0/537799415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f34100e30 0x7f4f34101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.543+0000 7f4f3adff700 1 -- 172.21.15.133:0/537799415 >> 172.21.15.133:0/537799415 conn(0x7f4f340fb260 msgr2=0x7f4f340fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:06.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.543+0000 7f4f3adff700 1 -- 172.21.15.133:0/537799415 shutdown_connections 2023-07-14T18:17:06.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.543+0000 7f4f3adff700 1 -- 172.21.15.133:0/537799415 wait complete. 2023-07-14T18:17:06.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.544+0000 7f4f3adff700 1 Processor -- start 2023-07-14T18:17:06.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.544+0000 7f4f3adff700 1 -- start start 2023-07-14T18:17:06.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.545+0000 7f4f3adff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f340ffc40 0x7f4f34197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:06.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.545+0000 7f4f3adff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f34100e30 0x7f4f34197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:06.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.545+0000 7f4f3adff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f341017c0 0x7f4f3419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:06.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.545+0000 7f4f3adff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3419f600 con 0x7f4f341017c0 2023-07-14T18:17:06.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.545+0000 7f4f3adff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3419f740 con 0x7f4f340ffc40 2023-07-14T18:17:06.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.545+0000 7f4f3adff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3419f8d0 con 0x7f4f34100e30 2023-07-14T18:17:06.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.545+0000 7f4f38b9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f340ffc40 0x7f4f34197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:06.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.545+0000 7f4f38b9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f340ffc40 0x7f4f34197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:58212/0 (socket says 172.21.15.133:58212) 2023-07-14T18:17:06.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.545+0000 7f4f38b9b700 1 -- 172.21.15.133:0/3583937351 learned_addr learned my addr 172.21.15.133:0/3583937351 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:06.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.545+0000 7f4f33fff700 1 --2- 172.21.15.133:0/3583937351 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f34100e30 0x7f4f34197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:06.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.545+0000 7f4f3939c700 1 --2- 172.21.15.133:0/3583937351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f341017c0 0x7f4f3419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:06.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.545+0000 7f4f38b9b700 1 -- 172.21.15.133:0/3583937351 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f34100e30 msgr2=0x7f4f34197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:06.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.545+0000 7f4f38b9b700 1 --2- 172.21.15.133:0/3583937351 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f34100e30 0x7f4f34197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.545+0000 7f4f38b9b700 1 -- 172.21.15.133:0/3583937351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f341017c0 msgr2=0x7f4f3419f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:06.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.545+0000 7f4f38b9b700 1 --2- 172.21.15.133:0/3583937351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f341017c0 0x7f4f3419f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.545+0000 7f4f38b9b700 1 -- 172.21.15.133:0/3583937351 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f28009c40 con 0x7f4f340ffc40 2023-07-14T18:17:06.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.545+0000 7f4f33fff700 1 --2- 172.21.15.133:0/3583937351 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f34100e30 0x7f4f34197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:17:06.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.546+0000 7f4f38b9b700 1 --2- 172.21.15.133:0/3583937351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f340ffc40 0x7f4f34197a80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4f28000f80 tx=0x7f4f28004db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:06.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.546+0000 7f4f31ffb700 1 -- 172.21.15.133:0/3583937351 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2800e040 con 0x7f4f340ffc40 2023-07-14T18:17:06.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.546+0000 7f4f31ffb700 1 -- 172.21.15.133:0/3583937351 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f2800be00 con 0x7f4f340ffc40 2023-07-14T18:17:06.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.546+0000 7f4f3adff700 1 -- 172.21.15.133:0/3583937351 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f3419fb50 con 0x7f4f340ffc40 2023-07-14T18:17:06.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.546+0000 7f4f31ffb700 1 -- 172.21.15.133:0/3583937351 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f28005330 con 0x7f4f340ffc40 2023-07-14T18:17:06.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.546+0000 7f4f3adff700 1 -- 172.21.15.133:0/3583937351 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f3419ffd0 con 0x7f4f340ffc40 2023-07-14T18:17:06.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.547+0000 7f4f3adff700 1 -- 172.21.15.133:0/3583937351 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f3410b3a0 con 0x7f4f340ffc40 2023-07-14T18:17:06.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.549+0000 7f4f31ffb700 1 -- 172.21.15.133:0/3583937351 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4f28005490 con 0x7f4f340ffc40 2023-07-14T18:17:06.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.550+0000 7f4f31ffb700 1 --2- 172.21.15.133:0/3583937351 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4f240a11a0 0x7f4f240a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:06.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.551+0000 7f4f31ffb700 1 -- 172.21.15.133:0/3583937351 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4f280ce440 con 0x7f4f340ffc40 2023-07-14T18:17:06.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.551+0000 7f4f33fff700 1 --2- 172.21.15.133:0/3583937351 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4f240a11a0 0x7f4f240a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:06.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.551+0000 7f4f33fff700 1 --2- 172.21.15.133:0/3583937351 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4f240a11a0 0x7f4f240a3650 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4f20005fd0 tx=0x7f4f20009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:06.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.553+0000 7f4f31ffb700 1 -- 172.21.15.133:0/3583937351 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f4f280978f0 con 0x7f4f340ffc40 2023-07-14T18:17:06.680 INFO:teuthology.orchestra.run.smithi133.stderr:8600000 of 50000000 tuples (17%) done (elapsed 22.47 s, remaining 108.15 s) 2023-07-14T18:17:06.854 INFO:teuthology.orchestra.run.smithi133.stderr:8700000 of 50000000 tuples (17%) done (elapsed 22.64 s, remaining 107.47 s) 2023-07-14T18:17:06.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.812+0000 7f4f3adff700 1 -- 172.21.15.133:0/3583937351 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f3410b590 con 0x7f4f340ffc40 2023-07-14T18:17:06.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.813+0000 7f4f31ffb700 1 -- 172.21.15.133:0/3583937351 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10856 (secure 0 0 0) 0x7f4f2809af10 con 0x7f4f340ffc40 2023-07-14T18:17:06.859 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:17:06.859 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3168151271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3168151271},{"type":"v1","addr":"172.21.15.133:6837","nonce":3168151271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14544,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/3093144261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3093144261},{"type":"v1","addr":"172.21.15.133:6839","nonce":3093144261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14556,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/3282902068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":3282902068},{"type":"v1","addr":"172.21.15.133:6841","nonce":3282902068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/4023127259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4023127259},{"type":"v1","addr":"172.21.15.161:6833","nonce":4023127259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2722373740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2722373740},{"type":"v1","addr":"172.21.15.171:6833","nonce":2722373740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2399005074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2399005074},{"type":"v1","addr":"172.21.15.133:6835","nonce":2399005074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":178,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"balance_automate":true},"ever_allowed_features":160,"explicitly_allowed_features":160,"created":"2023-07-14T18:14:17.440873+0000","modified":"2023-07-14T18:16:50.952764+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24557,"mds_1":24541,"mds_2":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3725502564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3725502564},{"type":"v1","addr":"172.21.15.161:6835","nonce":3725502564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6835/1877362116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1877362116},{"type":"v1","addr":"172.21.15.171:6835","nonce":1877362116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/3458064798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3458064798},{"type":"v1","addr":"172.21.15.161:6837","nonce":3458064798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/106205332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":106205332},{"type":"v1","addr":"172.21.15.171:6837","nonce":106205332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.171:6839/816201198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":816201198},{"type":"v1","addr":"172.21.15.171:6839","nonce":816201198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/2503865871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2503865871},{"type":"v1","addr":"172.21.15.161:6839","nonce":2503865871}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T18:17:06.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.815+0000 7f4f3adff700 1 -- 172.21.15.133:0/3583937351 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4f240a11a0 msgr2=0x7f4f240a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:06.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.815+0000 7f4f3adff700 1 --2- 172.21.15.133:0/3583937351 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4f240a11a0 0x7f4f240a3650 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4f20005fd0 tx=0x7f4f20009310 comp rx=0 tx=0).stop 2023-07-14T18:17:06.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.815+0000 7f4f3adff700 1 -- 172.21.15.133:0/3583937351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f340ffc40 msgr2=0x7f4f34197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:06.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.815+0000 7f4f3adff700 1 --2- 172.21.15.133:0/3583937351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f340ffc40 0x7f4f34197a80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4f28000f80 tx=0x7f4f28004db0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.816+0000 7f4f3adff700 1 -- 172.21.15.133:0/3583937351 shutdown_connections 2023-07-14T18:17:06.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.816+0000 7f4f3adff700 1 --2- 172.21.15.133:0/3583937351 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4f240a11a0 0x7f4f240a3650 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.816+0000 7f4f3adff700 1 --2- 172.21.15.133:0/3583937351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f340ffc40 0x7f4f34197a80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.816+0000 7f4f3adff700 1 --2- 172.21.15.133:0/3583937351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f341017c0 0x7f4f3419f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.863 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.816+0000 7f4f3adff700 1 --2- 172.21.15.133:0/3583937351 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f34100e30 0x7f4f34197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:06.863 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.816+0000 7f4f3adff700 1 -- 172.21.15.133:0/3583937351 >> 172.21.15.133:0/3583937351 conn(0x7f4f340fb260 msgr2=0x7f4f341066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:06.863 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.816+0000 7f4f3adff700 1 -- 172.21.15.133:0/3583937351 shutdown_connections 2023-07-14T18:17:06.863 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:06.816+0000 7f4f3adff700 1 -- 172.21.15.133:0/3583937351 wait complete. 2023-07-14T18:17:06.863 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 28 2023-07-14T18:17:06.881 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T18:17:07.139 INFO:teuthology.orchestra.run.smithi133.stderr:8800000 of 50000000 tuples (17%) done (elapsed 22.93 s, remaining 107.33 s) 2023-07-14T18:17:07.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.101+0000 7f12379ff700 1 -- 172.21.15.133:0/1342448356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12300730a0 msgr2=0x7f1230073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:07.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.101+0000 7f12379ff700 1 --2- 172.21.15.133:0/1342448356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12300730a0 0x7f1230073510 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1220009990 tx=0x7f1220009ca0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.101+0000 7f12379ff700 1 -- 172.21.15.133:0/1342448356 shutdown_connections 2023-07-14T18:17:07.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.101+0000 7f12379ff700 1 --2- 172.21.15.133:0/1342448356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1230107ae0 0x7f1230109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.101+0000 7f12379ff700 1 --2- 172.21.15.133:0/1342448356 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1230074680 0x7f1230072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.101+0000 7f12379ff700 1 --2- 172.21.15.133:0/1342448356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12300730a0 0x7f1230073510 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.101+0000 7f12379ff700 1 -- 172.21.15.133:0/1342448356 >> 172.21.15.133:0/1342448356 conn(0x7f12300fb260 msgr2=0x7f12300fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:07.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.101+0000 7f12379ff700 1 -- 172.21.15.133:0/1342448356 shutdown_connections 2023-07-14T18:17:07.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.101+0000 7f12379ff700 1 -- 172.21.15.133:0/1342448356 wait complete. 2023-07-14T18:17:07.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.102+0000 7f12379ff700 1 Processor -- start 2023-07-14T18:17:07.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.102+0000 7f12379ff700 1 -- start start 2023-07-14T18:17:07.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.102+0000 7f12379ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12300730a0 0x7f1230197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.102+0000 7f12379ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1230074680 0x7f1230198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.102+0000 7f12379ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1230107ae0 0x7f123019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.102+0000 7f12379ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123019f760 con 0x7f1230074680 2023-07-14T18:17:07.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.102+0000 7f12379ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123019f8d0 con 0x7f1230107ae0 2023-07-14T18:17:07.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.102+0000 7f12379ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123019fa90 con 0x7f12300730a0 2023-07-14T18:17:07.151 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.103+0000 7f1234f9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1230074680 0x7f1230198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.151 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.103+0000 7f1235f9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1230107ae0 0x7f123019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.151 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.103+0000 7f1234f9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1230074680 0x7f1230198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48958/0 (socket says 172.21.15.133:48958) 2023-07-14T18:17:07.151 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.103+0000 7f1234f9a700 1 -- 172.21.15.133:0/288167940 learned_addr learned my addr 172.21.15.133:0/288167940 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:07.151 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.103+0000 7f123579b700 1 --2- 172.21.15.133:0/288167940 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12300730a0 0x7f1230197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.151 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.103+0000 7f1235f9c700 1 -- 172.21.15.133:0/288167940 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12300730a0 msgr2=0x7f1230197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:07.151 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.103+0000 7f1235f9c700 1 --2- 172.21.15.133:0/288167940 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12300730a0 0x7f1230197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.152 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.103+0000 7f1235f9c700 1 -- 172.21.15.133:0/288167940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1230074680 msgr2=0x7f1230198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:07.152 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.103+0000 7f1235f9c700 1 --2- 172.21.15.133:0/288167940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1230074680 0x7f1230198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.152 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.103+0000 7f1235f9c700 1 -- 172.21.15.133:0/288167940 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12200095f0 con 0x7f1230107ae0 2023-07-14T18:17:07.152 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.103+0000 7f1234f9a700 1 --2- 172.21.15.133:0/288167940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1230074680 0x7f1230198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:17:07.152 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.103+0000 7f1235f9c700 1 --2- 172.21.15.133:0/288167940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1230107ae0 0x7f123019f190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f122800efc0 tx=0x7f122800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:07.152 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.104+0000 7f12267fc700 1 -- 172.21.15.133:0/288167940 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1228018070 con 0x7f1230107ae0 2023-07-14T18:17:07.152 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.104+0000 7f12379ff700 1 -- 172.21.15.133:0/288167940 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f123019fd10 con 0x7f1230107ae0 2023-07-14T18:17:07.152 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.104+0000 7f12267fc700 1 -- 172.21.15.133:0/288167940 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f122800f040 con 0x7f1230107ae0 2023-07-14T18:17:07.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.104+0000 7f12267fc700 1 -- 172.21.15.133:0/288167940 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122800cd40 con 0x7f1230107ae0 2023-07-14T18:17:07.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.104+0000 7f12379ff700 1 -- 172.21.15.133:0/288167940 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12301a01d0 con 0x7f1230107ae0 2023-07-14T18:17:07.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.107+0000 7f12267fc700 1 -- 172.21.15.133:0/288167940 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f1228003a00 con 0x7f1230107ae0 2023-07-14T18:17:07.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.108+0000 7f124b3dbb80 1 Processor -- start 2023-07-14T18:17:07.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.108+0000 7f124b3dbb80 1 -- start start 2023-07-14T18:17:07.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.109+0000 7f12267fc700 1 --2- 172.21.15.133:0/288167940 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f121c0a11d0 0x7f121c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.155 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.109+0000 7f124b3dbb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c046f9df60 0x55c046f9e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.155 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.109+0000 7f124b3dbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c046f9e8b0 0x55c046f9eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.155 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.109+0000 7f124b3dbb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c046f9f4b0 0x55c046f9f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.155 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.109+0000 7f12267fc700 1 -- 172.21.15.133:0/288167940 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f12280cc7d0 con 0x7f1230107ae0 2023-07-14T18:17:07.155 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.109+0000 7f124b3dbb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c046ea2cb0 con 0x55c046f9f4b0 2023-07-14T18:17:07.156 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.109+0000 7f124b3dbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c046ef5a40 con 0x55c046f9e8b0 2023-07-14T18:17:07.156 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.109+0000 7f124b3dbb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c046eeeed0 con 0x55c046f9df60 2023-07-14T18:17:07.156 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.109+0000 7f123579b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c046f9df60 0x55c046f9e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.156 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.109+0000 7f123579b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c046f9df60 0x55c046f9e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:54780/0 (socket says 172.21.15.133:54780) 2023-07-14T18:17:07.156 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.109+0000 7f123579b700 1 -- 172.21.15.133:0/1761364587 learned_addr learned my addr 172.21.15.133:0/1761364587 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:07.156 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.109+0000 7f1234f9a700 1 --2- 172.21.15.133:0/1761364587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c046f9e8b0 0x55c046f9eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.157 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.109+0000 7f123579b700 1 -- 172.21.15.133:0/1761364587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c046f9e8b0 msgr2=0x55c046f9eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:07.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.109+0000 7f123579b700 1 --2- 172.21.15.133:0/1761364587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c046f9e8b0 0x55c046f9eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.109+0000 7f123579b700 1 -- 172.21.15.133:0/1761364587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c046f9f4b0 msgr2=0x55c046f9f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:07.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.109+0000 7f123579b700 1 --2- 172.21.15.133:0/1761364587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c046f9f4b0 0x55c046f9f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.109+0000 7f123579b700 1 -- 172.21.15.133:0/1761364587 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c046d1cde0 con 0x55c046f9df60 2023-07-14T18:17:07.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.110+0000 7f1234f9a700 1 --2- 172.21.15.133:0/288167940 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f121c0a11d0 0x7f121c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.110+0000 7f123579b700 1 --2- 172.21.15.133:0/1761364587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c046f9df60 0x55c046f9e370 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f122c009cf0 tx=0x7f122c011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6999419d72e24497 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:07.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.110+0000 7f121b7fe700 1 -- 172.21.15.133:0/1761364587 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122c011d80 con 0x55c046f9df60 2023-07-14T18:17:07.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.110+0000 7f121b7fe700 1 -- 172.21.15.133:0/1761364587 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f122c0044f0 con 0x55c046f9df60 2023-07-14T18:17:07.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.110+0000 7f121b7fe700 1 -- 172.21.15.133:0/1761364587 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122c0193e0 con 0x55c046f9df60 2023-07-14T18:17:07.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.111+0000 7f124b3dbb80 1 -- 172.21.15.133:0/1761364587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c046f9df60 msgr2=0x55c046f9e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:07.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.111+0000 7f124b3dbb80 1 --2- 172.21.15.133:0/1761364587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c046f9df60 0x55c046f9e370 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f122c009cf0 tx=0x7f122c011440 comp rx=0 tx=0).stop 2023-07-14T18:17:07.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.111+0000 7f1234f9a700 1 --2- 172.21.15.133:0/288167940 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f121c0a11d0 0x7f121c0a3680 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55c046f9ecc0 tx=0x7f1220009ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:07.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.111+0000 7f124b3dbb80 1 -- 172.21.15.133:0/1761364587 shutdown_connections 2023-07-14T18:17:07.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.111+0000 7f124b3dbb80 1 --2- 172.21.15.133:0/1761364587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c046f9f4b0 0x55c046f9f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.111+0000 7f124b3dbb80 1 --2- 172.21.15.133:0/1761364587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c046f9df60 0x55c046f9e370 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.111+0000 7f124b3dbb80 1 --2- 172.21.15.133:0/1761364587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c046f9e8b0 0x55c046f9eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.111+0000 7f124b3dbb80 1 -- 172.21.15.133:0/1761364587 >> 172.21.15.133:0/1761364587 conn(0x55c046f93680 msgr2=0x55c046f989d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:07.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.111+0000 7f124b3dbb80 1 -- 172.21.15.133:0/1761364587 shutdown_connections 2023-07-14T18:17:07.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.111+0000 7f124b3dbb80 1 -- 172.21.15.133:0/1761364587 wait complete. 2023-07-14T18:17:07.160 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:06 smithi133 ceph-mon[126384]: pgmap v361: 129 pgs: 129 active+clean; 1.4 GiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 125 MiB/s wr, 63 op/s 2023-07-14T18:17:07.160 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:06 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3583937351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:17:07.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.112+0000 7f124b3dbb80 1 -- start start 2023-07-14T18:17:07.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.112+0000 7f124b3dbb80 1 Processor -- start 2023-07-14T18:17:07.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.112+0000 7f124b3dbb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c046f9df60 0x55c046fa79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.112+0000 7f124b3dbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c046f9e8b0 0x55c046fa7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.112+0000 7f124b3dbb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c046f9f4b0 0x55c046faef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.112+0000 7f124b3dbb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c046e8eac0 con 0x55c046f9df60 2023-07-14T18:17:07.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.112+0000 7f124b3dbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c046e8e7c0 con 0x55c046f9e8b0 2023-07-14T18:17:07.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.112+0000 7f124b3dbb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c046d72920 con 0x55c046f9f4b0 2023-07-14T18:17:07.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.113+0000 7f124b3dbb80 10 client.0 _ll_register_callbacks cb 0x55c046e8c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:17:07.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.113+0000 7f123579b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c046f9df60 0x55c046fa79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.113+0000 7f123579b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c046f9df60 0x55c046fa79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48980/0 (socket says 172.21.15.133:48980) 2023-07-14T18:17:07.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.113+0000 7f123579b700 1 -- 172.21.15.133:0/637182566 learned_addr learned my addr 172.21.15.133:0/637182566 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:07.165 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.113+0000 7f1234f9a700 1 --2- 172.21.15.133:0/637182566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c046f9e8b0 0x55c046fa7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.165 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.113+0000 7f1235f9c700 1 --2- 172.21.15.133:0/637182566 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c046f9f4b0 0x55c046faef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.165 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.113+0000 7f123579b700 1 -- 172.21.15.133:0/637182566 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c046f9f4b0 msgr2=0x55c046faef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:07.165 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.113+0000 7f123579b700 1 --2- 172.21.15.133:0/637182566 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c046f9f4b0 0x55c046faef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.165 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.113+0000 7f123579b700 1 -- 172.21.15.133:0/637182566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c046f9e8b0 msgr2=0x55c046fa7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:07.165 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.113+0000 7f123579b700 1 --2- 172.21.15.133:0/637182566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c046f9e8b0 0x55c046fa7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.165 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.113+0000 7f123579b700 1 -- 172.21.15.133:0/637182566 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122c009920 con 0x55c046f9df60 2023-07-14T18:17:07.165 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.114+0000 7f123579b700 1 --2- 172.21.15.133:0/637182566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c046f9df60 0x55c046fa79a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f122c005920 tx=0x7f122c003ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52da93e9cd1a4c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:07.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.114+0000 7f1218ff9700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T18:17:07.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.114+0000 7f1218ff9700 1 -- 172.21.15.133:0/637182566 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122c003dc0 con 0x55c046f9df60 2023-07-14T18:17:07.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.114+0000 7f124b3dbb80 1 -- 172.21.15.133:0/637182566 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c046e484f0 con 0x55c046f9df60 2023-07-14T18:17:07.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.114+0000 7f1218ff9700 1 -- 172.21.15.133:0/637182566 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f122c003f20 con 0x55c046f9df60 2023-07-14T18:17:07.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.114+0000 7f1218ff9700 1 -- 172.21.15.133:0/637182566 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f122c0040e0 con 0x55c046f9df60 2023-07-14T18:17:07.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.114+0000 7f124b3dbb80 10 client.15549 fetch_fsmap learned FSMap version 28 2023-07-14T18:17:07.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.114+0000 7f124b3dbb80 1 -- 172.21.15.133:0/637182566 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c046d05c60 con 0x55c046f9df60 2023-07-14T18:17:07.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.114+0000 7f1218ff9700 1 -- 172.21.15.133:0/637182566 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c046e484f0 con 0x55c046f9df60 2023-07-14T18:17:07.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.115+0000 7f1218ff9700 1 -- 172.21.15.133:0/637182566 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f122c0235a0 con 0x55c046f9df60 2023-07-14T18:17:07.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.115+0000 7f124b3dbb80 10 client.15549 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T18:17:07.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.115+0000 7f124b3dbb80 10 client.15549 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2023-07-14T18:17:07.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.115+0000 7f124b3dbb80 20 client.15549 populate_metadata read hostname 'smithi133' 2023-07-14T18:17:07.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.115+0000 7f124b3dbb80 1 --2- 172.21.15.133:0/637182566 >> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] conn(0x55c046fb35c0 0x55c046fb59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.115+0000 7f124b3dbb80 4 client.15549 mds_command: new command op to 24517 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:17:07.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.115+0000 7f124b3dbb80 1 -- 172.21.15.133:0/637182566 --> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c046f6cda0 con 0x55c046fb35c0 2023-07-14T18:17:07.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.115+0000 7f1234f9a700 1 --2- 172.21.15.133:0/637182566 >> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] conn(0x55c046fb35c0 0x55c046fb59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.168 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.116+0000 7f1218ff9700 10 client.15549 ms_handle_connect on v2:172.21.15.171:6834/1877362116 2023-07-14T18:17:07.168 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.116+0000 7f1234f9a700 1 --2- 172.21.15.133:0/637182566 >> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] conn(0x55c046fb35c0 0x55c046fb59d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d2c77469fe3d4d56 server_cookie=d5b67407a420cc5f in_seq=0 out_seq=0 2023-07-14T18:17:07.168 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.116+0000 7f1218ff9700 1 -- 172.21.15.133:0/637182566 <== mds.2 v2:172.21.15.171:6834/1877362116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55c046f6cda0 con 0x55c046fb35c0 2023-07-14T18:17:07.168 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.116+0000 7f1218ff9700 10 client.15549 handle_command_reply: tid=0 2023-07-14T18:17:07.168 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.116+0000 7f124b3dbb80 1 client.15549 shutdown 2023-07-14T18:17:07.168 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.117+0000 7f124b3dbb80 1 -- 172.21.15.133:0/637182566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c046f9df60 msgr2=0x55c046fa79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:07.168 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.117+0000 7f124b3dbb80 1 --2- 172.21.15.133:0/637182566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c046f9df60 0x55c046fa79a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f122c005920 tx=0x7f122c003ad0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.169 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.117+0000 7f124b3dbb80 1 -- 172.21.15.133:0/637182566 shutdown_connections 2023-07-14T18:17:07.169 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.117+0000 7f124b3dbb80 1 --2- 172.21.15.133:0/637182566 >> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] conn(0x55c046fb35c0 0x55c046fb59d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.169 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.117+0000 7f124b3dbb80 1 --2- 172.21.15.133:0/637182566 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c046f9f4b0 0x55c046faef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.169 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.117+0000 7f124b3dbb80 1 --2- 172.21.15.133:0/637182566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c046f9df60 0x55c046fa79a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.169 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.117+0000 7f124b3dbb80 1 --2- 172.21.15.133:0/637182566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c046f9e8b0 0x55c046fa7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.169 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.117+0000 7f124b3dbb80 1 -- 172.21.15.133:0/637182566 >> 172.21.15.133:0/637182566 conn(0x55c046f93680 msgr2=0x55c046f97650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:07.169 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.117+0000 7f1218ff9700 0 client.15549 ms_handle_reset on v2:172.21.15.171:6834/1877362116 2023-07-14T18:17:07.169 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.117+0000 7f124b3dbb80 1 -- 172.21.15.133:0/637182566 shutdown_connections 2023-07-14T18:17:07.170 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.117+0000 7f124b3dbb80 1 -- 172.21.15.133:0/637182566 wait complete. 2023-07-14T18:17:07.170 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.117+0000 7f124b3dbb80 20 client.15549 trim_cache size 0 max 16384 2023-07-14T18:17:07.176 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.130+0000 7f124b3dbb80 1 Processor -- start 2023-07-14T18:17:07.176 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.130+0000 7f124b3dbb80 1 -- start start 2023-07-14T18:17:07.176 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.131+0000 7f124b3dbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c046f9df60 0x55c046f994c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.177 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.131+0000 7f124b3dbb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c046f9e8b0 0x55c046faeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.177 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.131+0000 7f124b3dbb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c046f9f4b0 0x55c046fba2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.177 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.131+0000 7f124b3dbb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c046f9a450 con 0x55c046f9f4b0 2023-07-14T18:17:07.177 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.131+0000 7f124b3dbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c046f9f8c0 con 0x55c046f9df60 2023-07-14T18:17:07.177 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.131+0000 7f124b3dbb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c046f9fa80 con 0x55c046f9e8b0 2023-07-14T18:17:07.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.131+0000 7f1235f9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c046f9f4b0 0x55c046fba2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.131+0000 7f1234f9a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c046f9e8b0 0x55c046faeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.131+0000 7f1234f9a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c046f9e8b0 0x55c046faeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:54794/0 (socket says 172.21.15.133:54794) 2023-07-14T18:17:07.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.131+0000 7f1234f9a700 1 -- 172.21.15.133:0/3319700690 learned_addr learned my addr 172.21.15.133:0/3319700690 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:07.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.131+0000 7f123579b700 1 --2- 172.21.15.133:0/3319700690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c046f9df60 0x55c046f994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.131+0000 7f1234f9a700 1 -- 172.21.15.133:0/3319700690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c046f9df60 msgr2=0x55c046f994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:07.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.131+0000 7f1234f9a700 1 --2- 172.21.15.133:0/3319700690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c046f9df60 0x55c046f994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.131+0000 7f1234f9a700 1 -- 172.21.15.133:0/3319700690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c046f9f4b0 msgr2=0x55c046fba2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:07.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.131+0000 7f1234f9a700 1 --2- 172.21.15.133:0/3319700690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c046f9f4b0 0x55c046fba2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.131+0000 7f1234f9a700 1 -- 172.21.15.133:0/3319700690 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c046d43250 con 0x55c046f9e8b0 2023-07-14T18:17:07.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.131+0000 7f123579b700 1 --2- 172.21.15.133:0/3319700690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c046f9df60 0x55c046f994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:17:07.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.131+0000 7f1235f9c700 1 --2- 172.21.15.133:0/3319700690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c046f9f4b0 0x55c046fba2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:17:07.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.132+0000 7f1234f9a700 1 --2- 172.21.15.133:0/3319700690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c046f9e8b0 0x55c046faeeb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f122000b2e0 tx=0x7f1220022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a55f98a338c03ef0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:07.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.132+0000 7f121bfff700 1 -- 172.21.15.133:0/3319700690 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1220036070 con 0x55c046f9e8b0 2023-07-14T18:17:07.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.132+0000 7f121bfff700 1 -- 172.21.15.133:0/3319700690 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1220004bc0 con 0x55c046f9e8b0 2023-07-14T18:17:07.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.132+0000 7f121bfff700 1 -- 172.21.15.133:0/3319700690 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1220031430 con 0x55c046f9e8b0 2023-07-14T18:17:07.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.133+0000 7f124b3dbb80 1 -- 172.21.15.133:0/3319700690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c046f9e8b0 msgr2=0x55c046faeeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:07.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.133+0000 7f124b3dbb80 1 --2- 172.21.15.133:0/3319700690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c046f9e8b0 0x55c046faeeb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f122000b2e0 tx=0x7f1220022000 comp rx=0 tx=0).stop 2023-07-14T18:17:07.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.133+0000 7f124b3dbb80 1 -- 172.21.15.133:0/3319700690 shutdown_connections 2023-07-14T18:17:07.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.133+0000 7f124b3dbb80 1 --2- 172.21.15.133:0/3319700690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c046f9df60 0x55c046f994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.133+0000 7f124b3dbb80 1 --2- 172.21.15.133:0/3319700690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c046f9f4b0 0x55c046fba2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.133+0000 7f124b3dbb80 1 --2- 172.21.15.133:0/3319700690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c046f9e8b0 0x55c046faeeb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.133+0000 7f124b3dbb80 1 -- 172.21.15.133:0/3319700690 >> 172.21.15.133:0/3319700690 conn(0x55c046f93680 msgr2=0x55c046fa66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:07.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.133+0000 7f124b3dbb80 1 -- 172.21.15.133:0/3319700690 shutdown_connections 2023-07-14T18:17:07.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.133+0000 7f124b3dbb80 1 -- 172.21.15.133:0/3319700690 wait complete. 2023-07-14T18:17:07.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.133+0000 7f124b3dbb80 1 -- start start 2023-07-14T18:17:07.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.133+0000 7f124b3dbb80 1 Processor -- start 2023-07-14T18:17:07.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.134+0000 7f124b3dbb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c046f9df60 0x55c046facea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.134+0000 7f124b3dbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c046f9e8b0 0x55c0470031f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.134+0000 7f124b3dbb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c046f9f4b0 0x55c047005740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.134+0000 7f124b3dbb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c047005c80 con 0x55c046f9f4b0 2023-07-14T18:17:07.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.134+0000 7f124b3dbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c047005dc0 con 0x55c046f9e8b0 2023-07-14T18:17:07.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.134+0000 7f124b3dbb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c047005f80 con 0x55c046f9df60 2023-07-14T18:17:07.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.134+0000 7f124b3dbb80 10 client.0 _ll_register_callbacks cb 0x7f12280097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:17:07.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.134+0000 7f123579b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c046f9df60 0x55c046facea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.186 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.134+0000 7f123579b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c046f9df60 0x55c046facea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:54810/0 (socket says 172.21.15.133:54810) 2023-07-14T18:17:07.186 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.134+0000 7f123579b700 1 -- 172.21.15.133:0/4037375800 learned_addr learned my addr 172.21.15.133:0/4037375800 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:07.186 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.134+0000 7f1235f9c700 1 --2- 172.21.15.133:0/4037375800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c046f9f4b0 0x55c047005740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.186 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.134+0000 7f1234f9a700 1 --2- 172.21.15.133:0/4037375800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c046f9e8b0 0x55c0470031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.186 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.134+0000 7f123579b700 1 -- 172.21.15.133:0/4037375800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c046f9e8b0 msgr2=0x55c0470031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:07.186 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.135+0000 7f123579b700 1 --2- 172.21.15.133:0/4037375800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c046f9e8b0 0x55c0470031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.186 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.135+0000 7f123579b700 1 -- 172.21.15.133:0/4037375800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c046f9f4b0 msgr2=0x55c047005740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:07.187 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.135+0000 7f123579b700 1 --2- 172.21.15.133:0/4037375800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c046f9f4b0 0x55c047005740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.187 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.135+0000 7f123579b700 1 -- 172.21.15.133:0/4037375800 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c046d05c60 con 0x55c046f9df60 2023-07-14T18:17:07.187 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.135+0000 7f1235f9c700 1 --2- 172.21.15.133:0/4037375800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c046f9f4b0 0x55c047005740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:17:07.187 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.135+0000 7f123579b700 1 --2- 172.21.15.133:0/4037375800 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c046f9df60 0x55c046facea0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f122c005a80 tx=0x7f122c005e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b763dfcf1e8cd24 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:07.187 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.135+0000 7f1218ff9700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:17:07.187 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.135+0000 7f1218ff9700 1 -- 172.21.15.133:0/4037375800 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122c023c20 con 0x55c046f9df60 2023-07-14T18:17:07.187 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.135+0000 7f124b3dbb80 1 -- 172.21.15.133:0/4037375800 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f122c0044f0 con 0x55c046f9df60 2023-07-14T18:17:07.187 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.135+0000 7f1218ff9700 1 -- 172.21.15.133:0/4037375800 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f122c000e40 con 0x55c046f9df60 2023-07-14T18:17:07.188 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.135+0000 7f1218ff9700 1 -- 172.21.15.133:0/4037375800 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f122c023610 con 0x55c046f9df60 2023-07-14T18:17:07.188 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.135+0000 7f124b3dbb80 10 client.25573 fetch_fsmap learned FSMap version 28 2023-07-14T18:17:07.188 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.135+0000 7f124b3dbb80 1 -- 172.21.15.133:0/4037375800 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f12301a01d0 con 0x55c046f9df60 2023-07-14T18:17:07.188 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.135+0000 7f1218ff9700 1 -- 172.21.15.133:0/4037375800 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f122c0044f0 con 0x55c046f9df60 2023-07-14T18:17:07.188 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.136+0000 7f1218ff9700 1 -- 172.21.15.133:0/4037375800 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f122c005340 con 0x55c046f9df60 2023-07-14T18:17:07.188 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.136+0000 7f124b3dbb80 10 client.25573 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T18:17:07.188 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.136+0000 7f124b3dbb80 10 client.25573 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2023-07-14T18:17:07.188 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.136+0000 7f124b3dbb80 20 client.25573 populate_metadata read hostname 'smithi133' 2023-07-14T18:17:07.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.136+0000 7f124b3dbb80 1 --2- 172.21.15.133:0/4037375800 >> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] conn(0x55c047007660 0x55c047009a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.136+0000 7f124b3dbb80 4 client.25573 mds_command: new command op to 24517 tid=0[{"prefix": "damage ls"}] 2023-07-14T18:17:07.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.136+0000 7f124b3dbb80 1 -- 172.21.15.133:0/4037375800 --> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c046ee7eb0 con 0x55c047007660 2023-07-14T18:17:07.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.136+0000 7f1234f9a700 1 --2- 172.21.15.133:0/4037375800 >> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] conn(0x55c047007660 0x55c047009a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.136+0000 7f1234f9a700 1 --2- 172.21.15.133:0/4037375800 >> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] conn(0x55c047007660 0x55c047009a70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b24e31058df195 server_cookie=c40bc70e32470da1 in_seq=0 out_seq=0 2023-07-14T18:17:07.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.136+0000 7f1218ff9700 10 client.25573 ms_handle_connect on v2:172.21.15.171:6834/1877362116 2023-07-14T18:17:07.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.137+0000 7f1218ff9700 1 -- 172.21.15.133:0/4037375800 <== mds.2 v2:172.21.15.171:6834/1877362116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c046ee7eb0 con 0x55c047007660 2023-07-14T18:17:07.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.137+0000 7f1218ff9700 10 client.25573 handle_command_reply: tid=0 2023-07-14T18:17:07.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.137+0000 7f124b3dbb80 1 client.25573 shutdown 2023-07-14T18:17:07.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.137+0000 7f124b3dbb80 1 -- 172.21.15.133:0/4037375800 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c046f9df60 msgr2=0x55c046facea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:07.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.137+0000 7f124b3dbb80 1 --2- 172.21.15.133:0/4037375800 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c046f9df60 0x55c046facea0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f122c005a80 tx=0x7f122c005e30 comp rx=0 tx=0).stop 2023-07-14T18:17:07.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.137+0000 7f124b3dbb80 1 -- 172.21.15.133:0/4037375800 shutdown_connections 2023-07-14T18:17:07.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.137+0000 7f124b3dbb80 1 --2- 172.21.15.133:0/4037375800 >> [v2:172.21.15.171:6834/1877362116,v1:172.21.15.171:6835/1877362116] conn(0x55c047007660 0x55c047009a70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.137+0000 7f124b3dbb80 1 --2- 172.21.15.133:0/4037375800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c046f9f4b0 0x55c047005740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.137+0000 7f124b3dbb80 1 --2- 172.21.15.133:0/4037375800 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c046f9df60 0x55c046facea0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.137+0000 7f1218ff9700 0 client.25573 ms_handle_reset on v2:172.21.15.171:6834/1877362116 2023-07-14T18:17:07.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.137+0000 7f124b3dbb80 1 --2- 172.21.15.133:0/4037375800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c046f9e8b0 0x55c0470031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.137+0000 7f124b3dbb80 1 -- 172.21.15.133:0/4037375800 >> 172.21.15.133:0/4037375800 conn(0x55c046f93680 msgr2=0x55c046faf810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:07.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.137+0000 7f124b3dbb80 1 -- 172.21.15.133:0/4037375800 shutdown_connections 2023-07-14T18:17:07.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.137+0000 7f124b3dbb80 1 -- 172.21.15.133:0/4037375800 wait complete. 2023-07-14T18:17:07.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.137+0000 7f124b3dbb80 20 client.25573 trim_cache size 0 max 16384 2023-07-14T18:17:07.192 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2023-07-14T18:17:07.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.138+0000 7f12379ff700 1 -- 172.21.15.133:0/288167940 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f121c0a11d0 msgr2=0x7f121c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:07.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.138+0000 7f12379ff700 1 --2- 172.21.15.133:0/288167940 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f121c0a11d0 0x7f121c0a3680 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55c046f9ecc0 tx=0x7f1220009ca0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.138+0000 7f12379ff700 1 -- 172.21.15.133:0/288167940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1230107ae0 msgr2=0x7f123019f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:07.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.138+0000 7f12379ff700 1 --2- 172.21.15.133:0/288167940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1230107ae0 0x7f123019f190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f122800efc0 tx=0x7f122800c3a0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.138+0000 7f12379ff700 1 -- 172.21.15.133:0/288167940 shutdown_connections 2023-07-14T18:17:07.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.138+0000 7f12379ff700 1 --2- 172.21.15.133:0/288167940 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f121c0a11d0 0x7f121c0a3680 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.138+0000 7f12379ff700 1 --2- 172.21.15.133:0/288167940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1230107ae0 0x7f123019f190 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.138+0000 7f12379ff700 1 --2- 172.21.15.133:0/288167940 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12300730a0 0x7f1230197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.138+0000 7f12379ff700 1 --2- 172.21.15.133:0/288167940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1230074680 0x7f1230198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.138+0000 7f12379ff700 1 -- 172.21.15.133:0/288167940 >> 172.21.15.133:0/288167940 conn(0x7f12300fb260 msgr2=0x7f12300fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:07.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.138+0000 7f12379ff700 1 -- 172.21.15.133:0/288167940 shutdown_connections 2023-07-14T18:17:07.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.138+0000 7f12379ff700 1 -- 172.21.15.133:0/288167940 wait complete. 2023-07-14T18:17:07.201 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T18:17:07.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:06 smithi161 ceph-mon[128993]: pgmap v361: 129 pgs: 129 active+clean; 1.4 GiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 125 MiB/s wr, 63 op/s 2023-07-14T18:17:07.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:06 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3583937351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:17:07.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:06 smithi171 ceph-mon[129241]: pgmap v361: 129 pgs: 129 active+clean; 1.4 GiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 125 MiB/s wr, 63 op/s 2023-07-14T18:17:07.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:06 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3583937351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:17:07.430 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.384+0000 7f2fb2bff700 1 -- 172.21.15.133:0/426234704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fac100690 msgr2=0x7f2fac102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:07.430 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.384+0000 7f2fb2bff700 1 --2- 172.21.15.133:0/426234704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fac100690 0x7f2fac102a70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0009990 tx=0x7f2fa0009ca0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.430 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.384+0000 7f2fb2bff700 1 -- 172.21.15.133:0/426234704 shutdown_connections 2023-07-14T18:17:07.431 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.384+0000 7f2fb2bff700 1 --2- 172.21.15.133:0/426234704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fac1058d0 0x7f2fac107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.431 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.384+0000 7f2fb2bff700 1 --2- 172.21.15.133:0/426234704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fac100690 0x7f2fac102a70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.431 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.384+0000 7f2fb2bff700 1 --2- 172.21.15.133:0/426234704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fac102fb0 0x7f2fac105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.431 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.384+0000 7f2fb2bff700 1 -- 172.21.15.133:0/426234704 >> 172.21.15.133:0/426234704 conn(0x7f2fac0fa130 msgr2=0x7f2fac0fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:07.431 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.384+0000 7f2fb2bff700 1 -- 172.21.15.133:0/426234704 shutdown_connections 2023-07-14T18:17:07.431 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.384+0000 7f2fb2bff700 1 -- 172.21.15.133:0/426234704 wait complete. 2023-07-14T18:17:07.433 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.385+0000 7f2fb2bff700 1 Processor -- start 2023-07-14T18:17:07.433 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.385+0000 7f2fb2bff700 1 -- start start 2023-07-14T18:17:07.433 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.385+0000 7f2fb2bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fac100690 0x7f2fac1959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.433 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.385+0000 7f2fb2bff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fac102fb0 0x7f2fac195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.385+0000 7f2fb2bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fac1058d0 0x7f2fac19cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.385+0000 7f2fb2bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac19d520 con 0x7f2fac100690 2023-07-14T18:17:07.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.385+0000 7f2fb2bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac19d660 con 0x7f2fac1058d0 2023-07-14T18:17:07.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.385+0000 7f2fb2bff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac19d7f0 con 0x7f2fac102fb0 2023-07-14T18:17:07.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.386+0000 7f2fb099b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fac100690 0x7f2fac1959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.386+0000 7f2fabfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fac102fb0 0x7f2fac195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.386+0000 7f2fb099b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fac100690 0x7f2fac1959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49020/0 (socket says 172.21.15.133:49020) 2023-07-14T18:17:07.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.386+0000 7f2fb099b700 1 -- 172.21.15.133:0/2819630335 learned_addr learned my addr 172.21.15.133:0/2819630335 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:07.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.386+0000 7f2fabfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fac102fb0 0x7f2fac195ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:54828/0 (socket says 172.21.15.133:54828) 2023-07-14T18:17:07.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.386+0000 7f2fb119c700 1 --2- 172.21.15.133:0/2819630335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fac1058d0 0x7f2fac19cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.386+0000 7f2fabfff700 1 -- 172.21.15.133:0/2819630335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fac1058d0 msgr2=0x7f2fac19cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:07.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.386+0000 7f2fabfff700 1 --2- 172.21.15.133:0/2819630335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fac1058d0 0x7f2fac19cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.386+0000 7f2fabfff700 1 -- 172.21.15.133:0/2819630335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fac100690 msgr2=0x7f2fac1959a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:07.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.386+0000 7f2fabfff700 1 --2- 172.21.15.133:0/2819630335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fac100690 0x7f2fac1959a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.386+0000 7f2fabfff700 1 -- 172.21.15.133:0/2819630335 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa00095f0 con 0x7f2fac102fb0 2023-07-14T18:17:07.436 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.386+0000 7f2fabfff700 1 --2- 172.21.15.133:0/2819630335 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fac102fb0 0x7f2fac195ee0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800d950 tx=0x7f2f9800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:07.436 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.386+0000 7f2fa9ffb700 1 -- 172.21.15.133:0/2819630335 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9800f890 con 0x7f2fac102fb0 2023-07-14T18:17:07.436 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.386+0000 7f2fa9ffb700 1 -- 172.21.15.133:0/2819630335 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f98010460 con 0x7f2fac102fb0 2023-07-14T18:17:07.436 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.387+0000 7f2fa9ffb700 1 -- 172.21.15.133:0/2819630335 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9800e610 con 0x7f2fac102fb0 2023-07-14T18:17:07.436 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.387+0000 7f2fb2bff700 1 -- 172.21.15.133:0/2819630335 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fac19dad0 con 0x7f2fac102fb0 2023-07-14T18:17:07.441 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.387+0000 7f2fb2bff700 1 -- 172.21.15.133:0/2819630335 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fac19dfe0 con 0x7f2fac102fb0 2023-07-14T18:17:07.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.390+0000 7f2fa9ffb700 1 -- 172.21.15.133:0/2819630335 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f2f980105d0 con 0x7f2fac102fb0 2023-07-14T18:17:07.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.391+0000 7f2fa9ffb700 1 --2- 172.21.15.133:0/2819630335 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f2f9c0a11d0 0x7f2f9c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.391+0000 7f2fa9ffb700 1 -- 172.21.15.133:0/2819630335 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f2f980cc890 con 0x7f2fac102fb0 2023-07-14T18:17:07.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.392+0000 7f2fb099b700 1 --2- 172.21.15.133:0/2819630335 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f2f9c0a11d0 0x7f2f9c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.392+0000 7f2fb099b700 1 --2- 172.21.15.133:0/2819630335 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f2f9c0a11d0 0x7f2f9c0a3680 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0006010 tx=0x7f2fa000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:07.443 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.393+0000 7f2fc65d7b80 1 Processor -- start 2023-07-14T18:17:07.443 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.393+0000 7f2fc65d7b80 1 -- start start 2023-07-14T18:17:07.443 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.394+0000 7f2fc65d7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a40f1cb320 0x55a40f1cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.443 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.394+0000 7f2fc65d7b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a40f1cbc70 0x55a40f1cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.443 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.394+0000 7f2fc65d7b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a40f1cc890 0x55a40f1ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.443 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.394+0000 7f2fc65d7b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40f0cbcb0 con 0x55a40f1cbc70 2023-07-14T18:17:07.443 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.394+0000 7f2fc65d7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40f11ea40 con 0x55a40f1cb320 2023-07-14T18:17:07.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.394+0000 7f2fc65d7b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40f117ed0 con 0x55a40f1cc890 2023-07-14T18:17:07.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.394+0000 7f2fb119c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a40f1cc890 0x55a40f1ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.394+0000 7f2fabfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a40f1cbc70 0x55a40f1cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.394+0000 7f2fb119c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a40f1cc890 0x55a40f1ccce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:54832/0 (socket says 172.21.15.133:54832) 2023-07-14T18:17:07.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.394+0000 7f2fb119c700 1 -- 172.21.15.133:0/4210468501 learned_addr learned my addr 172.21.15.133:0/4210468501 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:07.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.394+0000 7f2fb099b700 1 --2- 172.21.15.133:0/4210468501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a40f1cb320 0x55a40f1cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.394+0000 7f2fb119c700 1 -- 172.21.15.133:0/4210468501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a40f1cb320 msgr2=0x55a40f1cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:07.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.394+0000 7f2fb119c700 1 --2- 172.21.15.133:0/4210468501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a40f1cb320 0x55a40f1cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.394+0000 7f2fb119c700 1 -- 172.21.15.133:0/4210468501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a40f1cbc70 msgr2=0x55a40f1cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:07.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.394+0000 7f2fb119c700 1 --2- 172.21.15.133:0/4210468501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a40f1cbc70 0x55a40f1cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.394+0000 7f2fb119c700 1 -- 172.21.15.133:0/4210468501 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a40ef45de0 con 0x55a40f1cc890 2023-07-14T18:17:07.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.395+0000 7f2fb119c700 1 --2- 172.21.15.133:0/4210468501 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a40f1cc890 0x55a40f1ccce0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4011680 tx=0x7f2fa4009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf27a365aa1590d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:07.446 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.395+0000 7f2f96ffd700 1 -- 172.21.15.133:0/4210468501 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa400e040 con 0x55a40f1cc890 2023-07-14T18:17:07.446 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.395+0000 7f2f96ffd700 1 -- 172.21.15.133:0/4210468501 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fa400f040 con 0x55a40f1cc890 2023-07-14T18:17:07.446 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.395+0000 7f2f96ffd700 1 -- 172.21.15.133:0/4210468501 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa40048c0 con 0x55a40f1cc890 2023-07-14T18:17:07.446 INFO:teuthology.orchestra.run.smithi133.stderr:8900000 of 50000000 tuples (17%) done (elapsed 23.23 s, remaining 107.26 s) 2023-07-14T18:17:07.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.396+0000 7f2fc65d7b80 1 -- 172.21.15.133:0/4210468501 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a40f1cc890 msgr2=0x55a40f1ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:07.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.396+0000 7f2fc65d7b80 1 --2- 172.21.15.133:0/4210468501 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a40f1cc890 0x55a40f1ccce0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4011680 tx=0x7f2fa4009920 comp rx=0 tx=0).stop 2023-07-14T18:17:07.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.396+0000 7f2fc65d7b80 1 -- 172.21.15.133:0/4210468501 shutdown_connections 2023-07-14T18:17:07.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.396+0000 7f2fc65d7b80 1 --2- 172.21.15.133:0/4210468501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a40f1cb320 0x55a40f1cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.396+0000 7f2fc65d7b80 1 --2- 172.21.15.133:0/4210468501 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a40f1cc890 0x55a40f1ccce0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.396+0000 7f2fc65d7b80 1 --2- 172.21.15.133:0/4210468501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a40f1cbc70 0x55a40f1cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.396+0000 7f2fc65d7b80 1 -- 172.21.15.133:0/4210468501 >> 172.21.15.133:0/4210468501 conn(0x55a40f1bc680 msgr2=0x55a40f1c19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:07.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.396+0000 7f2fc65d7b80 1 -- 172.21.15.133:0/4210468501 shutdown_connections 2023-07-14T18:17:07.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.396+0000 7f2fc65d7b80 1 -- 172.21.15.133:0/4210468501 wait complete. 2023-07-14T18:17:07.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.396+0000 7f2fc65d7b80 1 -- start start 2023-07-14T18:17:07.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.397+0000 7f2fc65d7b80 1 Processor -- start 2023-07-14T18:17:07.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.397+0000 7f2fc65d7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a40f1cb320 0x55a40f1d4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.397+0000 7f2fc65d7b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a40f1cbc70 0x55a40f1d52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.397+0000 7f2fc65d7b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a40f1cc890 0x55a40f1dc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.397+0000 7f2fc65d7b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40f0b7ac0 con 0x55a40f1cbc70 2023-07-14T18:17:07.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.397+0000 7f2fc65d7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40f0b77c0 con 0x55a40f1cb320 2023-07-14T18:17:07.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.397+0000 7f2fc65d7b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40ef9b920 con 0x55a40f1cc890 2023-07-14T18:17:07.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.397+0000 7f2fc65d7b80 10 client.0 _ll_register_callbacks cb 0x55a40f0b5760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:17:07.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.397+0000 7f2fb099b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a40f1cb320 0x55a40f1d4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.452 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.397+0000 7f2fb119c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a40f1cc890 0x55a40f1dc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.452 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.397+0000 7f2fabfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a40f1cbc70 0x55a40f1d52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.452 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.397+0000 7f2fb099b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a40f1cb320 0x55a40f1d4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:40310/0 (socket says 172.21.15.133:40310) 2023-07-14T18:17:07.452 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.397+0000 7f2fb099b700 1 -- 172.21.15.133:0/526779901 learned_addr learned my addr 172.21.15.133:0/526779901 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:07.452 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.398+0000 7f2fb119c700 1 -- 172.21.15.133:0/526779901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a40f1cb320 msgr2=0x55a40f1d4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:07.452 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.398+0000 7f2fb119c700 1 --2- 172.21.15.133:0/526779901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a40f1cb320 0x55a40f1d4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.452 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.398+0000 7f2fb119c700 1 -- 172.21.15.133:0/526779901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a40f1cbc70 msgr2=0x55a40f1d52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:07.453 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.398+0000 7f2fb119c700 1 --2- 172.21.15.133:0/526779901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a40f1cbc70 0x55a40f1d52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.453 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.398+0000 7f2fb119c700 1 -- 172.21.15.133:0/526779901 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa4011330 con 0x55a40f1cc890 2023-07-14T18:17:07.453 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.398+0000 7f2fb119c700 1 --2- 172.21.15.133:0/526779901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a40f1cc890 0x55a40f1dc2e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400b010 tx=0x7f2fa4005ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ffb1fe0ffe81ef7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:07.453 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.398+0000 7f2f8bfff700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:17:07.453 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.398+0000 7f2f8bfff700 1 -- 172.21.15.133:0/526779901 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa400e070 con 0x55a40f1cc890 2023-07-14T18:17:07.453 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.398+0000 7f2fc65d7b80 1 -- 172.21.15.133:0/526779901 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a40f0714f0 con 0x55a40f1cc890 2023-07-14T18:17:07.453 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.398+0000 7f2f8bfff700 1 -- 172.21.15.133:0/526779901 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f2fa4003c10 con 0x55a40f1cc890 2023-07-14T18:17:07.453 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.399+0000 7f2f8bfff700 1 -- 172.21.15.133:0/526779901 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fa4003dd0 con 0x55a40f1cc890 2023-07-14T18:17:07.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.399+0000 7f2fc65d7b80 10 client.25585 fetch_fsmap learned FSMap version 28 2023-07-14T18:17:07.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.399+0000 7f2fc65d7b80 1 -- 172.21.15.133:0/526779901 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a40ef2ec60 con 0x55a40f1cc890 2023-07-14T18:17:07.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.399+0000 7f2f8bfff700 1 -- 172.21.15.133:0/526779901 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a40f0714f0 con 0x55a40f1cc890 2023-07-14T18:17:07.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.399+0000 7f2f8bfff700 1 -- 172.21.15.133:0/526779901 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f2fa4027460 con 0x55a40f1cc890 2023-07-14T18:17:07.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.399+0000 7f2fc65d7b80 10 client.25585 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T18:17:07.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.399+0000 7f2fc65d7b80 10 client.25585 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T18:17:07.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.399+0000 7f2fc65d7b80 20 client.25585 populate_metadata read hostname 'smithi133' 2023-07-14T18:17:07.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.399+0000 7f2fc65d7b80 1 --2- 172.21.15.133:0/526779901 >> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] conn(0x55a40f1e0a60 0x55a40f1e2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.399+0000 7f2fc65d7b80 4 client.25585 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:17:07.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.399+0000 7f2fc65d7b80 1 -- 172.21.15.133:0/526779901 --> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a40f195da0 con 0x55a40f1e0a60 2023-07-14T18:17:07.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.399+0000 7f2fb099b700 1 --2- 172.21.15.133:0/526779901 >> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] conn(0x55a40f1e0a60 0x55a40f1e2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.400+0000 7f2fb099b700 1 --2- 172.21.15.133:0/526779901 >> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] conn(0x55a40f1e0a60 0x55a40f1e2e70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=64955f8e83153a1c server_cookie=b5d26e89bd675996 in_seq=0 out_seq=0 2023-07-14T18:17:07.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.400+0000 7f2f8bfff700 10 client.25585 ms_handle_connect on v2:172.21.15.171:6838/816201198 2023-07-14T18:17:07.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.400+0000 7f2f8bfff700 1 -- 172.21.15.133:0/526779901 <== mds.1 v2:172.21.15.171:6838/816201198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55a40f195da0 con 0x55a40f1e0a60 2023-07-14T18:17:07.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.400+0000 7f2f8bfff700 10 client.25585 handle_command_reply: tid=0 2023-07-14T18:17:07.456 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.400+0000 7f2fc65d7b80 1 client.25585 shutdown 2023-07-14T18:17:07.456 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.401+0000 7f2fc65d7b80 1 -- 172.21.15.133:0/526779901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a40f1cc890 msgr2=0x55a40f1dc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:07.456 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.401+0000 7f2fc65d7b80 1 --2- 172.21.15.133:0/526779901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a40f1cc890 0x55a40f1dc2e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400b010 tx=0x7f2fa4005ae0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.456 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.401+0000 7f2fc65d7b80 1 -- 172.21.15.133:0/526779901 shutdown_connections 2023-07-14T18:17:07.456 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.401+0000 7f2fc65d7b80 1 --2- 172.21.15.133:0/526779901 >> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] conn(0x55a40f1e0a60 0x55a40f1e2e70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.456 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.401+0000 7f2fc65d7b80 1 --2- 172.21.15.133:0/526779901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a40f1cb320 0x55a40f1d4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.456 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.401+0000 7f2fc65d7b80 1 --2- 172.21.15.133:0/526779901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a40f1cc890 0x55a40f1dc2e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.456 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.401+0000 7f2fc65d7b80 1 --2- 172.21.15.133:0/526779901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a40f1cbc70 0x55a40f1d52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.457 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.401+0000 7f2f8bfff700 0 client.25585 ms_handle_reset on v2:172.21.15.171:6838/816201198 2023-07-14T18:17:07.457 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.401+0000 7f2fc65d7b80 1 -- 172.21.15.133:0/526779901 >> 172.21.15.133:0/526779901 conn(0x55a40f1bc680 msgr2=0x55a40f1c0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:07.457 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.401+0000 7f2fc65d7b80 1 -- 172.21.15.133:0/526779901 shutdown_connections 2023-07-14T18:17:07.457 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.401+0000 7f2fc65d7b80 1 -- 172.21.15.133:0/526779901 wait complete. 2023-07-14T18:17:07.457 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.401+0000 7f2fc65d7b80 20 client.25585 trim_cache size 0 max 16384 2023-07-14T18:17:07.460 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.415+0000 7f2fc65d7b80 1 Processor -- start 2023-07-14T18:17:07.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.415+0000 7f2fc65d7b80 1 -- start start 2023-07-14T18:17:07.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.415+0000 7f2fc65d7b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a40f1cb320 0x55a40f1ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.415+0000 7f2fc65d7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a40f1cbc70 0x55a40f1dc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.415+0000 7f2fc65d7b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a40f1cc890 0x55a40f1d2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.415+0000 7f2fc65d7b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40f1d4210 con 0x55a40f1cc890 2023-07-14T18:17:07.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.415+0000 7f2fc65d7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40f1c2e10 con 0x55a40f1cbc70 2023-07-14T18:17:07.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.415+0000 7f2fc65d7b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40f1e34d0 con 0x55a40f1cb320 2023-07-14T18:17:07.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.415+0000 7f2fb119c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a40f1cc890 0x55a40f1d2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.463 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.415+0000 7f2fb099b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a40f1cb320 0x55a40f1ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.463 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.416+0000 7f2fb099b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a40f1cb320 0x55a40f1ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:54848/0 (socket says 172.21.15.133:54848) 2023-07-14T18:17:07.463 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.416+0000 7f2fabfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a40f1cbc70 0x55a40f1dc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.463 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.416+0000 7f2fb099b700 1 -- 172.21.15.133:0/823393709 learned_addr learned my addr 172.21.15.133:0/823393709 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:07.463 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.416+0000 7f2fb119c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a40f1cc890 0x55a40f1d2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49042/0 (socket says 172.21.15.133:49042) 2023-07-14T18:17:07.463 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.416+0000 7f2fb099b700 1 -- 172.21.15.133:0/823393709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a40f1cbc70 msgr2=0x55a40f1dc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:07.463 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.416+0000 7f2fb099b700 1 --2- 172.21.15.133:0/823393709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a40f1cbc70 0x55a40f1dc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.416+0000 7f2fb099b700 1 -- 172.21.15.133:0/823393709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a40f1cc890 msgr2=0x55a40f1d2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:07.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.416+0000 7f2fb099b700 1 --2- 172.21.15.133:0/823393709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a40f1cc890 0x55a40f1d2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.416+0000 7f2fb099b700 1 -- 172.21.15.133:0/823393709 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a40ef6c250 con 0x55a40f1cb320 2023-07-14T18:17:07.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.416+0000 7f2fb099b700 1 --2- 172.21.15.133:0/823393709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a40f1cb320 0x55a40f1ccc60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0005910 tx=0x7f2fa0023000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b78033162536116c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:07.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.416+0000 7f2f977fe700 1 -- 172.21.15.133:0/823393709 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0034250 con 0x55a40f1cb320 2023-07-14T18:17:07.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.416+0000 7f2f977fe700 1 -- 172.21.15.133:0/823393709 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fa002c040 con 0x55a40f1cb320 2023-07-14T18:17:07.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.416+0000 7f2f977fe700 1 -- 172.21.15.133:0/823393709 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa00049c0 con 0x55a40f1cb320 2023-07-14T18:17:07.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.417+0000 7f2fc65d7b80 1 -- 172.21.15.133:0/823393709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a40f1cb320 msgr2=0x55a40f1ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:07.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.417+0000 7f2fc65d7b80 1 --2- 172.21.15.133:0/823393709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a40f1cb320 0x55a40f1ccc60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0005910 tx=0x7f2fa0023000 comp rx=0 tx=0).stop 2023-07-14T18:17:07.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.417+0000 7f2fc65d7b80 1 -- 172.21.15.133:0/823393709 shutdown_connections 2023-07-14T18:17:07.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.417+0000 7f2fc65d7b80 1 --2- 172.21.15.133:0/823393709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a40f1cc890 0x55a40f1d2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.417+0000 7f2fc65d7b80 1 --2- 172.21.15.133:0/823393709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a40f1cb320 0x55a40f1ccc60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.417+0000 7f2fc65d7b80 1 --2- 172.21.15.133:0/823393709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a40f1cbc70 0x55a40f1dc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.417+0000 7f2fc65d7b80 1 -- 172.21.15.133:0/823393709 >> 172.21.15.133:0/823393709 conn(0x55a40f1bc680 msgr2=0x55a40f1d3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:07.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.417+0000 7f2fc65d7b80 1 -- 172.21.15.133:0/823393709 shutdown_connections 2023-07-14T18:17:07.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.417+0000 7f2fc65d7b80 1 -- 172.21.15.133:0/823393709 wait complete. 2023-07-14T18:17:07.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.418+0000 7f2fc65d7b80 1 -- start start 2023-07-14T18:17:07.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.418+0000 7f2fc65d7b80 1 Processor -- start 2023-07-14T18:17:07.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.418+0000 7f2fc65d7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a40f1cb320 0x55a40f1dbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.418+0000 7f2fc65d7b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a40f1cbc70 0x55a40f1dc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.418+0000 7f2fc65d7b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a40f1cc890 0x55a40f1d8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.418+0000 7f2fc65d7b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40f1d9070 con 0x55a40f1cbc70 2023-07-14T18:17:07.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.418+0000 7f2fc65d7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40f1d91e0 con 0x55a40f1cb320 2023-07-14T18:17:07.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.418+0000 7f2fc65d7b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40f1d93a0 con 0x55a40f1cc890 2023-07-14T18:17:07.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.418+0000 7f2fc65d7b80 10 client.0 _ll_register_callbacks cb 0x7f2f98095cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:17:07.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.419+0000 7f2fabfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a40f1cbc70 0x55a40f1dc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.419+0000 7f2fb099b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a40f1cb320 0x55a40f1dbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.419+0000 7f2fb099b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a40f1cb320 0x55a40f1dbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:40316/0 (socket says 172.21.15.133:40316) 2023-07-14T18:17:07.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.419+0000 7f2fabfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a40f1cbc70 0x55a40f1dc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49048/0 (socket says 172.21.15.133:49048) 2023-07-14T18:17:07.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.419+0000 7f2fb099b700 1 -- 172.21.15.133:0/3395075242 learned_addr learned my addr 172.21.15.133:0/3395075242 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:07.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.419+0000 7f2fb119c700 1 --2- 172.21.15.133:0/3395075242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a40f1cc890 0x55a40f1d8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.419+0000 7f2fb099b700 1 -- 172.21.15.133:0/3395075242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a40f1cc890 msgr2=0x55a40f1d8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:07.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.419+0000 7f2fb099b700 1 --2- 172.21.15.133:0/3395075242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a40f1cc890 0x55a40f1d8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.419+0000 7f2fb099b700 1 -- 172.21.15.133:0/3395075242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a40f1cbc70 msgr2=0x55a40f1dc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:07.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.419+0000 7f2fb099b700 1 --2- 172.21.15.133:0/3395075242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a40f1cbc70 0x55a40f1dc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.419+0000 7f2fb099b700 1 -- 172.21.15.133:0/3395075242 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fac108920 con 0x55a40f1cb320 2023-07-14T18:17:07.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.419+0000 7f2fb099b700 1 --2- 172.21.15.133:0/3395075242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a40f1cb320 0x55a40f1dbd00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0005300 tx=0x7f2fa00050b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8795c890d6f24f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:07.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.419+0000 7f2f8bfff700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:17:07.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.420+0000 7f2f8bfff700 1 -- 172.21.15.133:0/3395075242 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0034b40 con 0x55a40f1cb320 2023-07-14T18:17:07.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.420+0000 7f2fc65d7b80 1 -- 172.21.15.133:0/3395075242 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2fa400f040 con 0x55a40f1cb320 2023-07-14T18:17:07.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.420+0000 7f2f8bfff700 1 -- 172.21.15.133:0/3395075242 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f2fa0034420 con 0x55a40f1cb320 2023-07-14T18:17:07.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.420+0000 7f2f8bfff700 1 -- 172.21.15.133:0/3395075242 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fa0034ca0 con 0x55a40f1cb320 2023-07-14T18:17:07.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.420+0000 7f2fc65d7b80 10 client.25562 fetch_fsmap learned FSMap version 28 2023-07-14T18:17:07.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.420+0000 7f2fc65d7b80 1 -- 172.21.15.133:0/3395075242 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a40ef442b0 con 0x55a40f1cb320 2023-07-14T18:17:07.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.420+0000 7f2f8bfff700 1 -- 172.21.15.133:0/3395075242 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fa400f040 con 0x55a40f1cb320 2023-07-14T18:17:07.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.420+0000 7f2f8bfff700 1 -- 172.21.15.133:0/3395075242 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f2fa000f760 con 0x55a40f1cb320 2023-07-14T18:17:07.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.420+0000 7f2fc65d7b80 10 client.25562 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T18:17:07.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.420+0000 7f2fc65d7b80 10 client.25562 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T18:17:07.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.420+0000 7f2fc65d7b80 20 client.25562 populate_metadata read hostname 'smithi133' 2023-07-14T18:17:07.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.420+0000 7f2fc65d7b80 1 --2- 172.21.15.133:0/3395075242 >> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] conn(0x55a40f235930 0x55a40f237d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.420+0000 7f2fc65d7b80 4 client.25562 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-14T18:17:07.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.420+0000 7f2fc65d7b80 1 -- 172.21.15.133:0/3395075242 --> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a40f110eb0 con 0x55a40f235930 2023-07-14T18:17:07.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.421+0000 7f2fabfff700 1 --2- 172.21.15.133:0/3395075242 >> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] conn(0x55a40f235930 0x55a40f237d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.421+0000 7f2f8bfff700 10 client.25562 ms_handle_connect on v2:172.21.15.171:6838/816201198 2023-07-14T18:17:07.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.421+0000 7f2fabfff700 1 --2- 172.21.15.133:0/3395075242 >> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] conn(0x55a40f235930 0x55a40f237d10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=78ccef5dc9bc0f54 server_cookie=83f0100d7b9ddf48 in_seq=0 out_seq=0 2023-07-14T18:17:07.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.421+0000 7f2f8bfff700 1 -- 172.21.15.133:0/3395075242 <== mds.1 v2:172.21.15.171:6838/816201198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a40f110eb0 con 0x55a40f235930 2023-07-14T18:17:07.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.421+0000 7f2f8bfff700 10 client.25562 handle_command_reply: tid=0 2023-07-14T18:17:07.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.421+0000 7f2fc65d7b80 1 client.25562 shutdown 2023-07-14T18:17:07.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.421+0000 7f2fc65d7b80 1 -- 172.21.15.133:0/3395075242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a40f1cb320 msgr2=0x55a40f1dbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:07.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.421+0000 7f2fc65d7b80 1 --2- 172.21.15.133:0/3395075242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a40f1cb320 0x55a40f1dbd00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0005300 tx=0x7f2fa00050b0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.421+0000 7f2fc65d7b80 1 -- 172.21.15.133:0/3395075242 shutdown_connections 2023-07-14T18:17:07.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.421+0000 7f2fc65d7b80 1 --2- 172.21.15.133:0/3395075242 >> [v2:172.21.15.171:6838/816201198,v1:172.21.15.171:6839/816201198] conn(0x55a40f235930 0x55a40f237d10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.421+0000 7f2fc65d7b80 1 --2- 172.21.15.133:0/3395075242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a40f1cb320 0x55a40f1dbd00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.421+0000 7f2fc65d7b80 1 --2- 172.21.15.133:0/3395075242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a40f1cc890 0x55a40f1d8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.476 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.421+0000 7f2f8bfff700 0 client.25562 ms_handle_reset on v2:172.21.15.171:6838/816201198 2023-07-14T18:17:07.476 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.421+0000 7f2fc65d7b80 1 --2- 172.21.15.133:0/3395075242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a40f1cbc70 0x55a40f1dc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.476 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.421+0000 7f2fc65d7b80 1 -- 172.21.15.133:0/3395075242 >> 172.21.15.133:0/3395075242 conn(0x55a40f1bc680 msgr2=0x55a40f1d2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:07.476 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.422+0000 7f2fc65d7b80 1 -- 172.21.15.133:0/3395075242 shutdown_connections 2023-07-14T18:17:07.476 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.422+0000 7f2fc65d7b80 1 -- 172.21.15.133:0/3395075242 wait complete. 2023-07-14T18:17:07.476 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.422+0000 7f2fc65d7b80 20 client.25562 trim_cache size 0 max 16384 2023-07-14T18:17:07.477 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2023-07-14T18:17:07.477 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.422+0000 7f2fb2bff700 1 -- 172.21.15.133:0/2819630335 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f2f9c0a11d0 msgr2=0x7f2f9c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:07.477 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.422+0000 7f2fb2bff700 1 --2- 172.21.15.133:0/2819630335 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f2f9c0a11d0 0x7f2f9c0a3680 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0006010 tx=0x7f2fa000b260 comp rx=0 tx=0).stop 2023-07-14T18:17:07.477 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.422+0000 7f2fb2bff700 1 -- 172.21.15.133:0/2819630335 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fac102fb0 msgr2=0x7f2fac195ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:07.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.422+0000 7f2fb2bff700 1 --2- 172.21.15.133:0/2819630335 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fac102fb0 0x7f2fac195ee0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800d950 tx=0x7f2f9800dc60 comp rx=0 tx=0).stop 2023-07-14T18:17:07.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.423+0000 7f2fb2bff700 1 -- 172.21.15.133:0/2819630335 shutdown_connections 2023-07-14T18:17:07.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.423+0000 7f2fb2bff700 1 --2- 172.21.15.133:0/2819630335 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f2f9c0a11d0 0x7f2f9c0a3680 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.423+0000 7f2fb2bff700 1 --2- 172.21.15.133:0/2819630335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fac1058d0 0x7f2fac19cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.423+0000 7f2fb2bff700 1 --2- 172.21.15.133:0/2819630335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fac100690 0x7f2fac1959a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.423+0000 7f2fb2bff700 1 --2- 172.21.15.133:0/2819630335 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fac102fb0 0x7f2fac195ee0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.423+0000 7f2fb2bff700 1 -- 172.21.15.133:0/2819630335 >> 172.21.15.133:0/2819630335 conn(0x7f2fac0fa130 msgr2=0x7f2fac1012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:07.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.423+0000 7f2fb2bff700 1 -- 172.21.15.133:0/2819630335 shutdown_connections 2023-07-14T18:17:07.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.423+0000 7f2fb2bff700 1 -- 172.21.15.133:0/2819630335 wait complete. 2023-07-14T18:17:07.489 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T18:17:07.646 INFO:teuthology.orchestra.run.smithi133.stderr:9000000 of 50000000 tuples (18%) done (elapsed 23.43 s, remaining 106.75 s) 2023-07-14T18:17:07.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.665+0000 7f4a89dff700 1 -- 172.21.15.133:0/598777670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a840ffc10 msgr2=0x7f4a84100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:07.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.665+0000 7f4a89dff700 1 --2- 172.21.15.133:0/598777670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a840ffc10 0x7f4a84100020 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4a74009990 tx=0x7f4a74009ca0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.665+0000 7f4a89dff700 1 -- 172.21.15.133:0/598777670 shutdown_connections 2023-07-14T18:17:07.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.665+0000 7f4a89dff700 1 --2- 172.21.15.133:0/598777670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a840ffc10 0x7f4a84100020 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.665+0000 7f4a89dff700 1 --2- 172.21.15.133:0/598777670 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a84101790 0x7f4a84108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.665+0000 7f4a89dff700 1 --2- 172.21.15.133:0/598777670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a84100e00 0x7f4a84101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.665+0000 7f4a89dff700 1 -- 172.21.15.133:0/598777670 >> 172.21.15.133:0/598777670 conn(0x7f4a840fb210 msgr2=0x7f4a840fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:07.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.666+0000 7f4a89dff700 1 -- 172.21.15.133:0/598777670 shutdown_connections 2023-07-14T18:17:07.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.666+0000 7f4a89dff700 1 -- 172.21.15.133:0/598777670 wait complete. 2023-07-14T18:17:07.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.667+0000 7f4a89dff700 1 Processor -- start 2023-07-14T18:17:07.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.667+0000 7f4a89dff700 1 -- start start 2023-07-14T18:17:07.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.667+0000 7f4a89dff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a840ffc10 0x7f4a84197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.667+0000 7f4a89dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a84100e00 0x7f4a841981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.667+0000 7f4a89dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a84101790 0x7f4a8419f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.667+0000 7f4a89dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8419f7f0 con 0x7f4a84101790 2023-07-14T18:17:07.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.667+0000 7f4a89dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8419f960 con 0x7f4a84100e00 2023-07-14T18:17:07.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.667+0000 7f4a89dff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8419fb20 con 0x7f4a840ffc10 2023-07-14T18:17:07.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.667+0000 7f4a837fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a840ffc10 0x7f4a84197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.667+0000 7f4a82ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a84100e00 0x7f4a841981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.667+0000 7f4a837fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a840ffc10 0x7f4a84197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:54870/0 (socket says 172.21.15.133:54870) 2023-07-14T18:17:07.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.667+0000 7f4a837fe700 1 -- 172.21.15.133:0/4191636274 learned_addr learned my addr 172.21.15.133:0/4191636274 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:07.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.667+0000 7f4a83fff700 1 --2- 172.21.15.133:0/4191636274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a84101790 0x7f4a8419f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.668+0000 7f4a837fe700 1 -- 172.21.15.133:0/4191636274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a84100e00 msgr2=0x7f4a841981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:07.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.668+0000 7f4a837fe700 1 --2- 172.21.15.133:0/4191636274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a84100e00 0x7f4a841981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.668+0000 7f4a837fe700 1 -- 172.21.15.133:0/4191636274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a84101790 msgr2=0x7f4a8419f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:07.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.668+0000 7f4a837fe700 1 --2- 172.21.15.133:0/4191636274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a84101790 0x7f4a8419f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.668+0000 7f4a837fe700 1 -- 172.21.15.133:0/4191636274 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a7c009c40 con 0x7f4a840ffc10 2023-07-14T18:17:07.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.668+0000 7f4a82ffd700 1 --2- 172.21.15.133:0/4191636274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a84100e00 0x7f4a841981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:17:07.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.668+0000 7f4a837fe700 1 --2- 172.21.15.133:0/4191636274 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a840ffc10 0x7f4a84197c80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4a74004730 tx=0x7f4a74004760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:07.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.668+0000 7f4a80ff9700 1 -- 172.21.15.133:0/4191636274 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7401d070 con 0x7f4a840ffc10 2023-07-14T18:17:07.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.668+0000 7f4a89dff700 1 -- 172.21.15.133:0/4191636274 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a740095f0 con 0x7f4a840ffc10 2023-07-14T18:17:07.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.668+0000 7f4a80ff9700 1 -- 172.21.15.133:0/4191636274 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a74021410 con 0x7f4a840ffc10 2023-07-14T18:17:07.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.668+0000 7f4a80ff9700 1 -- 172.21.15.133:0/4191636274 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a74017650 con 0x7f4a840ffc10 2023-07-14T18:17:07.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.668+0000 7f4a89dff700 1 -- 172.21.15.133:0/4191636274 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a841a0090 con 0x7f4a840ffc10 2023-07-14T18:17:07.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.672+0000 7f4a9d732b80 1 Processor -- start 2023-07-14T18:17:07.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.673+0000 7f4a9d732b80 1 -- start start 2023-07-14T18:17:07.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.673+0000 7f4a9d732b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a31c55af60 0x55a31c55b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.673+0000 7f4a9d732b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a31c55b8b0 0x55a31c55bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.673+0000 7f4a9d732b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a31c55c4b0 0x55a31c55c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.673+0000 7f4a9d732b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a31c45fcb0 con 0x55a31c55af60 2023-07-14T18:17:07.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.673+0000 7f4a9d732b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a31c4b2a40 con 0x55a31c55b8b0 2023-07-14T18:17:07.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.673+0000 7f4a9d732b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a31c4abed0 con 0x55a31c55c4b0 2023-07-14T18:17:07.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.673+0000 7f4a80ff9700 1 -- 172.21.15.133:0/4191636274 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4a74021810 con 0x7f4a840ffc10 2023-07-14T18:17:07.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.673+0000 7f4a83fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a31c55af60 0x55a31c55b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.673+0000 7f4a83fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a31c55af60 0x55a31c55b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49086/0 (socket says 172.21.15.133:49086) 2023-07-14T18:17:07.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.673+0000 7f4a83fff700 1 -- 172.21.15.133:0/707966983 learned_addr learned my addr 172.21.15.133:0/707966983 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:07.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.673+0000 7f4a82ffd700 1 --2- 172.21.15.133:0/707966983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a31c55c4b0 0x55a31c55c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.674+0000 7f4a837fe700 1 --2- 172.21.15.133:0/707966983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a31c55b8b0 0x55a31c55bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.674+0000 7f4a83fff700 1 -- 172.21.15.133:0/707966983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a31c55c4b0 msgr2=0x55a31c55c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:07.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.674+0000 7f4a83fff700 1 --2- 172.21.15.133:0/707966983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a31c55c4b0 0x55a31c55c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.674+0000 7f4a83fff700 1 -- 172.21.15.133:0/707966983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a31c55b8b0 msgr2=0x55a31c55bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:07.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.674+0000 7f4a83fff700 1 --2- 172.21.15.133:0/707966983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a31c55b8b0 0x55a31c55bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.674+0000 7f4a83fff700 1 -- 172.21.15.133:0/707966983 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a31c2d9de0 con 0x55a31c55af60 2023-07-14T18:17:07.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.674+0000 7f4a837fe700 1 --2- 172.21.15.133:0/707966983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a31c55b8b0 0x55a31c55bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:17:07.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.674+0000 7f4a83fff700 1 --2- 172.21.15.133:0/707966983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a31c55af60 0x55a31c55b370 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00e730 tx=0x7f4a7c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416c1e893ce2384e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:07.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.674+0000 7f4a69ffb700 1 -- 172.21.15.133:0/707966983 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c00e040 con 0x55a31c55af60 2023-07-14T18:17:07.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.674+0000 7f4a69ffb700 1 -- 172.21.15.133:0/707966983 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a7c00f040 con 0x55a31c55af60 2023-07-14T18:17:07.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.674+0000 7f4a69ffb700 1 -- 172.21.15.133:0/707966983 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c007500 con 0x55a31c55af60 2023-07-14T18:17:07.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.675+0000 7f4a80ff9700 1 --2- 172.21.15.133:0/4191636274 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4a6c0a1150 0x7f4a6c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.675+0000 7f4a9d732b80 1 -- 172.21.15.133:0/707966983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a31c55af60 msgr2=0x55a31c55b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:07.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.675+0000 7f4a9d732b80 1 --2- 172.21.15.133:0/707966983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a31c55af60 0x55a31c55b370 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00e730 tx=0x7f4a7c009920 comp rx=0 tx=0).stop 2023-07-14T18:17:07.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.675+0000 7f4a80ff9700 1 -- 172.21.15.133:0/4191636274 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4a740ce0b0 con 0x7f4a840ffc10 2023-07-14T18:17:07.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.675+0000 7f4a9d732b80 1 -- 172.21.15.133:0/707966983 shutdown_connections 2023-07-14T18:17:07.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.675+0000 7f4a9d732b80 1 --2- 172.21.15.133:0/707966983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a31c55c4b0 0x55a31c55c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.675+0000 7f4a9d732b80 1 --2- 172.21.15.133:0/707966983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a31c55af60 0x55a31c55b370 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.675+0000 7f4a9d732b80 1 --2- 172.21.15.133:0/707966983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a31c55b8b0 0x55a31c55bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.675+0000 7f4a9d732b80 1 -- 172.21.15.133:0/707966983 >> 172.21.15.133:0/707966983 conn(0x55a31c550680 msgr2=0x55a31c5559d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:07.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.675+0000 7f4a9d732b80 1 -- 172.21.15.133:0/707966983 shutdown_connections 2023-07-14T18:17:07.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.675+0000 7f4a837fe700 1 --2- 172.21.15.133:0/4191636274 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4a6c0a1150 0x7f4a6c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.676+0000 7f4a9d732b80 1 -- 172.21.15.133:0/707966983 wait complete. 2023-07-14T18:17:07.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.676+0000 7f4a837fe700 1 --2- 172.21.15.133:0/4191636274 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4a6c0a1150 0x7f4a6c0a3600 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4a740976c0 tx=0x7f4a74009ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:07.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.676+0000 7f4a9d732b80 1 -- start start 2023-07-14T18:17:07.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.676+0000 7f4a9d732b80 1 Processor -- start 2023-07-14T18:17:07.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.677+0000 7f4a9d732b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a31c55af60 0x55a31c5648a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.677+0000 7f4a9d732b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a31c55b8b0 0x55a31c564de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.677+0000 7f4a9d732b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a31c55c4b0 0x55a31c56be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.677+0000 7f4a9d732b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a31c44bac0 con 0x55a31c55b8b0 2023-07-14T18:17:07.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.677+0000 7f4a9d732b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a31c44b7c0 con 0x55a31c55c4b0 2023-07-14T18:17:07.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.677+0000 7f4a9d732b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a31c32f920 con 0x55a31c55af60 2023-07-14T18:17:07.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.677+0000 7f4a9d732b80 10 client.0 _ll_register_callbacks cb 0x55a31c449760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:17:07.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.677+0000 7f4a82ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a31c55b8b0 0x55a31c564de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.677+0000 7f4a82ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a31c55b8b0 0x55a31c564de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49092/0 (socket says 172.21.15.133:49092) 2023-07-14T18:17:07.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.677+0000 7f4a82ffd700 1 -- 172.21.15.133:0/3253725500 learned_addr learned my addr 172.21.15.133:0/3253725500 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:07.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.677+0000 7f4a83fff700 1 --2- 172.21.15.133:0/3253725500 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a31c55af60 0x55a31c5648a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.677+0000 7f4a82ffd700 1 -- 172.21.15.133:0/3253725500 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a31c55af60 msgr2=0x55a31c5648a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:07.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.677+0000 7f4a82ffd700 1 --2- 172.21.15.133:0/3253725500 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a31c55af60 0x55a31c5648a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.677+0000 7f4a82ffd700 1 -- 172.21.15.133:0/3253725500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a31c55c4b0 msgr2=0x55a31c56be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:07.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.677+0000 7f4a83fff700 1 --2- 172.21.15.133:0/3253725500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a31c55c4b0 0x55a31c56be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.678+0000 7f4a82ffd700 1 --2- 172.21.15.133:0/3253725500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a31c55c4b0 0x55a31c56be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.678+0000 7f4a82ffd700 1 -- 172.21.15.133:0/3253725500 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a7c00e410 con 0x55a31c55b8b0 2023-07-14T18:17:07.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.678+0000 7f4a83fff700 1 --2- 172.21.15.133:0/3253725500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a31c55c4b0 0x55a31c56be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T18:17:07.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.678+0000 7f4a82ffd700 1 --2- 172.21.15.133:0/3253725500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a31c55b8b0 0x55a31c564de0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4a78011d10 tx=0x7f4a7800f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=403870d2f20d9cd1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:07.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.678+0000 7f4a62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T18:17:07.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.678+0000 7f4a62ffd700 1 -- 172.21.15.133:0/3253725500 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a78016070 con 0x55a31c55b8b0 2023-07-14T18:17:07.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.678+0000 7f4a9d732b80 1 -- 172.21.15.133:0/3253725500 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a31c4054f0 con 0x55a31c55b8b0 2023-07-14T18:17:07.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.678+0000 7f4a62ffd700 1 -- 172.21.15.133:0/3253725500 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4a7800fcf0 con 0x55a31c55b8b0 2023-07-14T18:17:07.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.678+0000 7f4a62ffd700 1 -- 172.21.15.133:0/3253725500 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a78004050 con 0x55a31c55b8b0 2023-07-14T18:17:07.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.678+0000 7f4a9d732b80 10 client.15585 fetch_fsmap learned FSMap version 28 2023-07-14T18:17:07.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.678+0000 7f4a9d732b80 1 -- 172.21.15.133:0/3253725500 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a31c2c2c60 con 0x55a31c55b8b0 2023-07-14T18:17:07.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.679+0000 7f4a62ffd700 1 -- 172.21.15.133:0/3253725500 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a31c4054f0 con 0x55a31c55b8b0 2023-07-14T18:17:07.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.679+0000 7f4a62ffd700 1 -- 172.21.15.133:0/3253725500 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4a78003ca0 con 0x55a31c55b8b0 2023-07-14T18:17:07.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.679+0000 7f4a9d732b80 10 client.15585 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T18:17:07.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.679+0000 7f4a9d732b80 10 client.15585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:17:07.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.679+0000 7f4a9d732b80 20 client.15585 populate_metadata read hostname 'smithi133' 2023-07-14T18:17:07.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.679+0000 7f4a9d732b80 1 --2- 172.21.15.133:0/3253725500 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55a31c570780 0x55a31c572b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.679+0000 7f4a9d732b80 4 client.15585 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:17:07.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.679+0000 7f4a9d732b80 1 -- 172.21.15.133:0/3253725500 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a31c529da0 con 0x55a31c570780 2023-07-14T18:17:07.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.679+0000 7f4a83fff700 1 --2- 172.21.15.133:0/3253725500 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55a31c570780 0x55a31c572b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.680+0000 7f4a83fff700 1 --2- 172.21.15.133:0/3253725500 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55a31c570780 0x55a31c572b90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ff73fa3830b1cea server_cookie=2010c627da40221b in_seq=0 out_seq=0 2023-07-14T18:17:07.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.680+0000 7f4a62ffd700 10 client.15585 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:17:07.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.681+0000 7f4a62ffd700 1 -- 172.21.15.133:0/3253725500 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55a31c529da0 con 0x55a31c570780 2023-07-14T18:17:07.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.681+0000 7f4a62ffd700 10 client.15585 handle_command_reply: tid=0 2023-07-14T18:17:07.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.681+0000 7f4a9d732b80 1 client.15585 shutdown 2023-07-14T18:17:07.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.681+0000 7f4a9d732b80 1 -- 172.21.15.133:0/3253725500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a31c55b8b0 msgr2=0x55a31c564de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:07.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.681+0000 7f4a9d732b80 1 --2- 172.21.15.133:0/3253725500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a31c55b8b0 0x55a31c564de0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4a78011d10 tx=0x7f4a7800f6a0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.681+0000 7f4a9d732b80 1 -- 172.21.15.133:0/3253725500 shutdown_connections 2023-07-14T18:17:07.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.681+0000 7f4a9d732b80 1 --2- 172.21.15.133:0/3253725500 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55a31c570780 0x55a31c572b90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.681+0000 7f4a9d732b80 1 --2- 172.21.15.133:0/3253725500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a31c55c4b0 0x55a31c56be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.681+0000 7f4a9d732b80 1 --2- 172.21.15.133:0/3253725500 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a31c55af60 0x55a31c5648a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.681+0000 7f4a9d732b80 1 --2- 172.21.15.133:0/3253725500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a31c55b8b0 0x55a31c564de0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.681+0000 7f4a62ffd700 0 client.15585 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:17:07.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.681+0000 7f4a9d732b80 1 -- 172.21.15.133:0/3253725500 >> 172.21.15.133:0/3253725500 conn(0x55a31c550680 msgr2=0x55a31c554650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:07.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.681+0000 7f4a9d732b80 1 -- 172.21.15.133:0/3253725500 shutdown_connections 2023-07-14T18:17:07.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.681+0000 7f4a9d732b80 1 -- 172.21.15.133:0/3253725500 wait complete. 2023-07-14T18:17:07.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.681+0000 7f4a9d732b80 20 client.15585 trim_cache size 0 max 16384 2023-07-14T18:17:07.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.695+0000 7f4a9d732b80 1 Processor -- start 2023-07-14T18:17:07.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.695+0000 7f4a9d732b80 1 -- start start 2023-07-14T18:17:07.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.695+0000 7f4a9d732b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a31c55af60 0x55a31c55c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.695+0000 7f4a9d732b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a31c55b8b0 0x55a31c56bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.695+0000 7f4a9d732b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a31c55c4b0 0x55a31c562830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.695+0000 7f4a9d732b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a31c563340 con 0x55a31c55af60 2023-07-14T18:17:07.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.695+0000 7f4a9d732b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a31c56da90 con 0x55a31c55c4b0 2023-07-14T18:17:07.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.695+0000 7f4a9d732b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a31c550250 con 0x55a31c55b8b0 2023-07-14T18:17:07.743 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.695+0000 7f4a82ffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a31c55b8b0 0x55a31c56bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.743 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.695+0000 7f4a82ffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a31c55b8b0 0x55a31c56bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:54884/0 (socket says 172.21.15.133:54884) 2023-07-14T18:17:07.743 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.695+0000 7f4a83fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a31c55af60 0x55a31c55c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.695+0000 7f4a82ffd700 1 -- 172.21.15.133:0/3113930839 learned_addr learned my addr 172.21.15.133:0/3113930839 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:07.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.696+0000 7f4a82ffd700 1 -- 172.21.15.133:0/3113930839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a31c55c4b0 msgr2=0x55a31c562830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:07.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.696+0000 7f4a83fff700 1 --2- 172.21.15.133:0/3113930839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a31c55c4b0 0x55a31c562830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.696+0000 7f4a82ffd700 1 --2- 172.21.15.133:0/3113930839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a31c55c4b0 0x55a31c562830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.696+0000 7f4a82ffd700 1 -- 172.21.15.133:0/3113930839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a31c55af60 msgr2=0x55a31c55c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:07.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.696+0000 7f4a82ffd700 1 --2- 172.21.15.133:0/3113930839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a31c55af60 0x55a31c55c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.696+0000 7f4a82ffd700 1 -- 172.21.15.133:0/3113930839 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a31c300250 con 0x55a31c55b8b0 2023-07-14T18:17:07.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.696+0000 7f4a83fff700 1 --2- 172.21.15.133:0/3113930839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a31c55c4b0 0x55a31c562830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T18:17:07.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.696+0000 7f4a82ffd700 1 --2- 172.21.15.133:0/3113930839 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a31c55b8b0 0x55a31c56bde0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800ffd0 tx=0x7f4a78019590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad37e4172be27284 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:07.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.696+0000 7f4a6a7fc700 1 -- 172.21.15.133:0/3113930839 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a78016070 con 0x55a31c55b8b0 2023-07-14T18:17:07.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.696+0000 7f4a6a7fc700 1 -- 172.21.15.133:0/3113930839 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a78019660 con 0x55a31c55b8b0 2023-07-14T18:17:07.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.696+0000 7f4a6a7fc700 1 -- 172.21.15.133:0/3113930839 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a78005450 con 0x55a31c55b8b0 2023-07-14T18:17:07.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.697+0000 7f4a9d732b80 1 -- 172.21.15.133:0/3113930839 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a31c55b8b0 msgr2=0x55a31c56bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:07.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.697+0000 7f4a9d732b80 1 --2- 172.21.15.133:0/3113930839 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a31c55b8b0 0x55a31c56bde0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800ffd0 tx=0x7f4a78019590 comp rx=0 tx=0).stop 2023-07-14T18:17:07.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.697+0000 7f4a9d732b80 1 -- 172.21.15.133:0/3113930839 shutdown_connections 2023-07-14T18:17:07.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.697+0000 7f4a9d732b80 1 --2- 172.21.15.133:0/3113930839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a31c55c4b0 0x55a31c562830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.697+0000 7f4a9d732b80 1 --2- 172.21.15.133:0/3113930839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a31c55af60 0x55a31c55c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.697+0000 7f4a9d732b80 1 --2- 172.21.15.133:0/3113930839 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a31c55b8b0 0x55a31c56bde0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.697+0000 7f4a9d732b80 1 -- 172.21.15.133:0/3113930839 >> 172.21.15.133:0/3113930839 conn(0x55a31c550680 msgr2=0x55a31c563600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:07.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.697+0000 7f4a9d732b80 1 -- 172.21.15.133:0/3113930839 shutdown_connections 2023-07-14T18:17:07.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.697+0000 7f4a9d732b80 1 -- 172.21.15.133:0/3113930839 wait complete. 2023-07-14T18:17:07.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.697+0000 7f4a9d732b80 1 -- start start 2023-07-14T18:17:07.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.698+0000 7f4a9d732b80 1 Processor -- start 2023-07-14T18:17:07.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.698+0000 7f4a9d732b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a31c55af60 0x55a31c56b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.698+0000 7f4a9d732b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a31c55b8b0 0x55a31c56bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.698+0000 7f4a9d732b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a31c55c4b0 0x55a31c568120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.698+0000 7f4a9d732b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a31c568660 con 0x55a31c55c4b0 2023-07-14T18:17:07.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.698+0000 7f4a9d732b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a31c5687d0 con 0x55a31c55af60 2023-07-14T18:17:07.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.698+0000 7f4a9d732b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a31c568940 con 0x55a31c55b8b0 2023-07-14T18:17:07.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.698+0000 7f4a9d732b80 10 client.0 _ll_register_callbacks cb 0x7f4a780117e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:17:07.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.698+0000 7f4a83fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a31c55af60 0x55a31c56b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.699+0000 7f4a83fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a31c55af60 0x55a31c56b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:40388/0 (socket says 172.21.15.133:40388) 2023-07-14T18:17:07.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.699+0000 7f4a83fff700 1 -- 172.21.15.133:0/3988344489 learned_addr learned my addr 172.21.15.133:0/3988344489 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:07.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.699+0000 7f4a82ffd700 1 --2- 172.21.15.133:0/3988344489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a31c55b8b0 0x55a31c56bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.699+0000 7f4a83fff700 1 --2- 172.21.15.133:0/3988344489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a31c55c4b0 0x55a31c568120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.699+0000 7f4a83fff700 1 -- 172.21.15.133:0/3988344489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a31c55b8b0 msgr2=0x55a31c56bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:07.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.699+0000 7f4a83fff700 1 --2- 172.21.15.133:0/3988344489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a31c55b8b0 0x55a31c56bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.699+0000 7f4a83fff700 1 -- 172.21.15.133:0/3988344489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a31c55c4b0 msgr2=0x55a31c568120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:07.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.699+0000 7f4a83fff700 1 --2- 172.21.15.133:0/3988344489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a31c55c4b0 0x55a31c568120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.699+0000 7f4a83fff700 1 -- 172.21.15.133:0/3988344489 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a31c2c2c60 con 0x55a31c55af60 2023-07-14T18:17:07.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.699+0000 7f4a83fff700 1 --2- 172.21.15.133:0/3988344489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a31c55af60 0x55a31c56b680 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c004630 tx=0x7f4a7c00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7420d8ede0823e29 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:07.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.699+0000 7f4a62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:17:07.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.699+0000 7f4a62ffd700 1 -- 172.21.15.133:0/3988344489 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c004260 con 0x55a31c55af60 2023-07-14T18:17:07.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.699+0000 7f4a9d732b80 1 -- 172.21.15.133:0/3988344489 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a7c00f040 con 0x55a31c55af60 2023-07-14T18:17:07.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.699+0000 7f4a62ffd700 1 -- 172.21.15.133:0/3988344489 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4a7c027150 con 0x55a31c55af60 2023-07-14T18:17:07.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.700+0000 7f4a62ffd700 1 -- 172.21.15.133:0/3988344489 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a7c027310 con 0x55a31c55af60 2023-07-14T18:17:07.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.700+0000 7f4a9d732b80 10 client.25574 fetch_fsmap learned FSMap version 28 2023-07-14T18:17:07.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.700+0000 7f4a9d732b80 1 -- 172.21.15.133:0/3988344489 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a31c2d82b0 con 0x55a31c55af60 2023-07-14T18:17:07.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.700+0000 7f4a62ffd700 1 -- 172.21.15.133:0/3988344489 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a7c038080 con 0x55a31c55af60 2023-07-14T18:17:07.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.700+0000 7f4a62ffd700 1 -- 172.21.15.133:0/3988344489 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4a7c0276e0 con 0x55a31c55af60 2023-07-14T18:17:07.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.700+0000 7f4a9d732b80 10 client.25574 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T18:17:07.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.700+0000 7f4a9d732b80 10 client.25574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:17:07.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.700+0000 7f4a9d732b80 20 client.25574 populate_metadata read hostname 'smithi133' 2023-07-14T18:17:07.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.700+0000 7f4a9d732b80 1 --2- 172.21.15.133:0/3988344489 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55a31c5c5810 0x55a31c5c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:07.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.700+0000 7f4a9d732b80 4 client.25574 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-14T18:17:07.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.700+0000 7f4a9d732b80 1 -- 172.21.15.133:0/3988344489 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a31c4a4eb0 con 0x55a31c5c5810 2023-07-14T18:17:07.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.700+0000 7f4a82ffd700 1 --2- 172.21.15.133:0/3988344489 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55a31c5c5810 0x55a31c5c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:07.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.701+0000 7f4a82ffd700 1 --2- 172.21.15.133:0/3988344489 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55a31c5c5810 0x55a31c5c7c20 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e78b3a992d65b3d server_cookie=e28946343a9ea189 in_seq=0 out_seq=0 2023-07-14T18:17:07.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.701+0000 7f4a62ffd700 10 client.25574 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:17:07.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.701+0000 7f4a62ffd700 1 -- 172.21.15.133:0/3988344489 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a31c4a4eb0 con 0x55a31c5c5810 2023-07-14T18:17:07.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.701+0000 7f4a62ffd700 10 client.25574 handle_command_reply: tid=0 2023-07-14T18:17:07.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.701+0000 7f4a9d732b80 1 client.25574 shutdown 2023-07-14T18:17:07.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.701+0000 7f4a9d732b80 1 -- 172.21.15.133:0/3988344489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a31c55af60 msgr2=0x55a31c56b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:07.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.701+0000 7f4a9d732b80 1 --2- 172.21.15.133:0/3988344489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a31c55af60 0x55a31c56b680 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c004630 tx=0x7f4a7c00b040 comp rx=0 tx=0).stop 2023-07-14T18:17:07.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.701+0000 7f4a9d732b80 1 -- 172.21.15.133:0/3988344489 shutdown_connections 2023-07-14T18:17:07.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.701+0000 7f4a9d732b80 1 --2- 172.21.15.133:0/3988344489 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55a31c5c5810 0x55a31c5c7c20 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.701+0000 7f4a9d732b80 1 --2- 172.21.15.133:0/3988344489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a31c55af60 0x55a31c56b680 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.701+0000 7f4a9d732b80 1 --2- 172.21.15.133:0/3988344489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a31c55c4b0 0x55a31c568120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.701+0000 7f4a62ffd700 0 client.25574 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:17:07.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.701+0000 7f4a9d732b80 1 --2- 172.21.15.133:0/3988344489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a31c55b8b0 0x55a31c56bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.701+0000 7f4a9d732b80 1 -- 172.21.15.133:0/3988344489 >> 172.21.15.133:0/3988344489 conn(0x55a31c550680 msgr2=0x55a31c55f390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:07.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.701+0000 7f4a9d732b80 1 -- 172.21.15.133:0/3988344489 shutdown_connections 2023-07-14T18:17:07.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.701+0000 7f4a9d732b80 1 -- 172.21.15.133:0/3988344489 wait complete. 2023-07-14T18:17:07.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.702+0000 7f4a9d732b80 20 client.25574 trim_cache size 0 max 16384 2023-07-14T18:17:07.756 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2023-07-14T18:17:07.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.702+0000 7f4a89dff700 1 -- 172.21.15.133:0/4191636274 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4a6c0a1150 msgr2=0x7f4a6c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:07.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.702+0000 7f4a89dff700 1 --2- 172.21.15.133:0/4191636274 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4a6c0a1150 0x7f4a6c0a3600 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4a740976c0 tx=0x7f4a74009ca0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.702+0000 7f4a89dff700 1 -- 172.21.15.133:0/4191636274 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a840ffc10 msgr2=0x7f4a84197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:07.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.702+0000 7f4a89dff700 1 --2- 172.21.15.133:0/4191636274 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a840ffc10 0x7f4a84197c80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4a74004730 tx=0x7f4a74004760 comp rx=0 tx=0).stop 2023-07-14T18:17:07.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.702+0000 7f4a89dff700 1 -- 172.21.15.133:0/4191636274 shutdown_connections 2023-07-14T18:17:07.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.702+0000 7f4a89dff700 1 --2- 172.21.15.133:0/4191636274 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f4a6c0a1150 0x7f4a6c0a3600 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.702+0000 7f4a89dff700 1 --2- 172.21.15.133:0/4191636274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a84101790 0x7f4a8419f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.702+0000 7f4a89dff700 1 --2- 172.21.15.133:0/4191636274 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a840ffc10 0x7f4a84197c80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.702+0000 7f4a89dff700 1 --2- 172.21.15.133:0/4191636274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a84100e00 0x7f4a841981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:07.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.702+0000 7f4a89dff700 1 -- 172.21.15.133:0/4191636274 >> 172.21.15.133:0/4191636274 conn(0x7f4a840fb210 msgr2=0x7f4a84106790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:07.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.702+0000 7f4a89dff700 1 -- 172.21.15.133:0/4191636274 shutdown_connections 2023-07-14T18:17:07.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:07.702+0000 7f4a89dff700 1 -- 172.21.15.133:0/4191636274 wait complete. 2023-07-14T18:17:07.927 INFO:teuthology.orchestra.run.smithi133.stderr:9100000 of 50000000 tuples (18%) done (elapsed 23.71 s, remaining 106.58 s) 2023-07-14T18:17:08.203 INFO:teuthology.orchestra.run.smithi133.stderr:9200000 of 50000000 tuples (18%) done (elapsed 23.99 s, remaining 106.39 s) 2023-07-14T18:17:08.379 INFO:teuthology.orchestra.run.smithi133.stderr:9300000 of 50000000 tuples (18%) done (elapsed 24.17 s, remaining 105.76 s) 2023-07-14T18:17:08.665 INFO:teuthology.orchestra.run.smithi133.stderr:9400000 of 50000000 tuples (18%) done (elapsed 24.45 s, remaining 105.61 s) 2023-07-14T18:17:08.768 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T18:17:08.768 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T18:17:08.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.951+0000 7f70df7ff700 1 -- 172.21.15.133:0/193942486 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70d80730a0 msgr2=0x7f70d8073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:08.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.951+0000 7f70df7ff700 1 --2- 172.21.15.133:0/193942486 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70d80730a0 0x7f70d8073510 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009990 tx=0x7f70c8009ca0 comp rx=0 tx=0).stop 2023-07-14T18:17:08.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.951+0000 7f70df7ff700 1 -- 172.21.15.133:0/193942486 shutdown_connections 2023-07-14T18:17:08.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.951+0000 7f70df7ff700 1 --2- 172.21.15.133:0/193942486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70d8074680 0x7f70d8072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:08.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.951+0000 7f70df7ff700 1 --2- 172.21.15.133:0/193942486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70d810c1f0 0x7f70d810e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:08.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.951+0000 7f70df7ff700 1 --2- 172.21.15.133:0/193942486 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70d80730a0 0x7f70d8073510 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:08.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.951+0000 7f70df7ff700 1 -- 172.21.15.133:0/193942486 >> 172.21.15.133:0/193942486 conn(0x7f70d80fb260 msgr2=0x7f70d80fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:08.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.951+0000 7f70df7ff700 1 -- 172.21.15.133:0/193942486 shutdown_connections 2023-07-14T18:17:08.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.951+0000 7f70df7ff700 1 -- 172.21.15.133:0/193942486 wait complete. 2023-07-14T18:17:09.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.952+0000 7f70df7ff700 1 Processor -- start 2023-07-14T18:17:09.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.952+0000 7f70df7ff700 1 -- start start 2023-07-14T18:17:09.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.953+0000 7f70df7ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70d80730a0 0x7f70d819c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.953+0000 7f70df7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70d8074680 0x7f70d819c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.953+0000 7f70df7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70d810c1f0 0x7f70d81a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.953+0000 7f70df7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d81a3b90 con 0x7f70d810c1f0 2023-07-14T18:17:09.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.953+0000 7f70df7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d81a3cd0 con 0x7f70d8074680 2023-07-14T18:17:09.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.953+0000 7f70df7ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d81a3e60 con 0x7f70d80730a0 2023-07-14T18:17:09.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.953+0000 7f70dd59b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70d80730a0 0x7f70d819c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.953+0000 7f70dd59b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70d80730a0 0x7f70d819c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:54914/0 (socket says 172.21.15.133:54914) 2023-07-14T18:17:09.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.953+0000 7f70dd59b700 1 -- 172.21.15.133:0/1762843185 learned_addr learned my addr 172.21.15.133:0/1762843185 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:09.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.953+0000 7f70dcd9a700 1 --2- 172.21.15.133:0/1762843185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70d8074680 0x7f70d819c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.953+0000 7f70ddd9c700 1 --2- 172.21.15.133:0/1762843185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70d810c1f0 0x7f70d81a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.953+0000 7f70dd59b700 1 -- 172.21.15.133:0/1762843185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70d8074680 msgr2=0x7f70d819c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:09.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.953+0000 7f70dd59b700 1 --2- 172.21.15.133:0/1762843185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70d8074680 0x7f70d819c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.953+0000 7f70dd59b700 1 -- 172.21.15.133:0/1762843185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70d810c1f0 msgr2=0x7f70d81a35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:09.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.954+0000 7f70dd59b700 1 --2- 172.21.15.133:0/1762843185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70d810c1f0 0x7f70d81a35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.954+0000 7f70dd59b700 1 -- 172.21.15.133:0/1762843185 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d4009520 con 0x7f70d80730a0 2023-07-14T18:17:09.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.954+0000 7f70ddd9c700 1 --2- 172.21.15.133:0/1762843185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70d810c1f0 0x7f70d81a35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:17:09.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.954+0000 7f70dd59b700 1 --2- 172.21.15.133:0/1762843185 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70d80730a0 0x7f70d819c010 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f70d400ea30 tx=0x7f70d400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:09.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.954+0000 7f70ce7fc700 1 -- 172.21.15.133:0/1762843185 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4009820 con 0x7f70d80730a0 2023-07-14T18:17:09.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.954+0000 7f70ce7fc700 1 -- 172.21.15.133:0/1762843185 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70d40044e0 con 0x7f70d80730a0 2023-07-14T18:17:09.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.954+0000 7f70df7ff700 1 -- 172.21.15.133:0/1762843185 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70c80095f0 con 0x7f70d80730a0 2023-07-14T18:17:09.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.954+0000 7f70ce7fc700 1 -- 172.21.15.133:0/1762843185 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4003e80 con 0x7f70d80730a0 2023-07-14T18:17:09.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.954+0000 7f70df7ff700 1 -- 172.21.15.133:0/1762843185 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d81a44c0 con 0x7f70d80730a0 2023-07-14T18:17:09.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.958+0000 7f70ce7fc700 1 -- 172.21.15.133:0/1762843185 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f70d4004000 con 0x7f70d80730a0 2023-07-14T18:17:09.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.959+0000 7f70f30a2b80 1 Processor -- start 2023-07-14T18:17:09.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.959+0000 7f70ce7fc700 1 --2- 172.21.15.133:0/1762843185 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f70c40a1150 0x7f70c40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.960+0000 7f70ce7fc700 1 -- 172.21.15.133:0/1762843185 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f70d4014070 con 0x7f70d80730a0 2023-07-14T18:17:09.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.960+0000 7f70f30a2b80 1 -- start start 2023-07-14T18:17:09.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.960+0000 7f70ddd9c700 1 --2- 172.21.15.133:0/1762843185 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f70c40a1150 0x7f70c40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.960+0000 7f70f30a2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563037aeefe0 0x563037aef3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.960+0000 7f70f30a2b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563037aef930 0x563037aeff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.960+0000 7f70f30a2b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563037af0530 0x563037af09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.960+0000 7f70f30a2b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630379f3d30 con 0x563037af0530 2023-07-14T18:17:09.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.960+0000 7f70f30a2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563037a46ac0 con 0x563037aeefe0 2023-07-14T18:17:09.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.960+0000 7f70f30a2b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x563037a3ff50 con 0x563037aef930 2023-07-14T18:17:09.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.960+0000 7f70dd59b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563037aeefe0 0x563037aef3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.960+0000 7f70dd59b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563037aeefe0 0x563037aef3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:40404/0 (socket says 172.21.15.133:40404) 2023-07-14T18:17:09.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.960+0000 7f70dd59b700 1 -- 172.21.15.133:0/3003701798 learned_addr learned my addr 172.21.15.133:0/3003701798 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:09.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.960+0000 7f70ddd9c700 1 --2- 172.21.15.133:0/3003701798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563037af0530 0x563037af09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.961+0000 7f70dd59b700 1 -- 172.21.15.133:0/3003701798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563037aef930 msgr2=0x563037aeff50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T18:17:09.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.961+0000 7f70dd59b700 1 --2- 172.21.15.133:0/3003701798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563037aef930 0x563037aeff50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.010 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.961+0000 7f70dd59b700 1 -- 172.21.15.133:0/3003701798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563037af0530 msgr2=0x563037af09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:09.010 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.961+0000 7f70dd59b700 1 --2- 172.21.15.133:0/3003701798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563037af0530 0x563037af09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.010 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.961+0000 7f70ddd9c700 1 --2- 172.21.15.133:0/1762843185 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f70c40a1150 0x7f70c40a3600 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009410 tx=0x7f70d0009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:09.010 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.961+0000 7f70dd59b700 1 -- 172.21.15.133:0/3003701798 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56303786de60 con 0x563037aeefe0 2023-07-14T18:17:09.010 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.961+0000 7f70dd59b700 1 --2- 172.21.15.133:0/3003701798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563037aeefe0 0x563037aef3f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f70d4096cf0 tx=0x7f70d409a0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=390b037e9ff9bd86 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:09.010 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.961+0000 7f70c37fe700 1 -- 172.21.15.133:0/3003701798 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d409a280 con 0x563037aeefe0 2023-07-14T18:17:09.010 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.961+0000 7f70c37fe700 1 -- 172.21.15.133:0/3003701798 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70d409e2b0 con 0x563037aeefe0 2023-07-14T18:17:09.011 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.961+0000 7f70c37fe700 1 -- 172.21.15.133:0/3003701798 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d409a740 con 0x563037aeefe0 2023-07-14T18:17:09.011 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.962+0000 7f70f30a2b80 1 -- 172.21.15.133:0/3003701798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563037aeefe0 msgr2=0x563037aef3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:09.011 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.962+0000 7f70f30a2b80 1 --2- 172.21.15.133:0/3003701798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563037aeefe0 0x563037aef3f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f70d4096cf0 tx=0x7f70d409a0d0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.011 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.962+0000 7f70f30a2b80 1 -- 172.21.15.133:0/3003701798 shutdown_connections 2023-07-14T18:17:09.011 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.962+0000 7f70f30a2b80 1 --2- 172.21.15.133:0/3003701798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563037aeefe0 0x563037aef3f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.011 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.962+0000 7f70f30a2b80 1 --2- 172.21.15.133:0/3003701798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563037af0530 0x563037af09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.011 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.962+0000 7f70f30a2b80 1 --2- 172.21.15.133:0/3003701798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563037aef930 0x563037aeff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.962+0000 7f70f30a2b80 1 -- 172.21.15.133:0/3003701798 >> 172.21.15.133:0/3003701798 conn(0x563037ae4700 msgr2=0x563037ae9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:09.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.962+0000 7f70f30a2b80 1 -- 172.21.15.133:0/3003701798 shutdown_connections 2023-07-14T18:17:09.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.962+0000 7f70f30a2b80 1 -- 172.21.15.133:0/3003701798 wait complete. 2023-07-14T18:17:09.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.963+0000 7f70f30a2b80 1 -- start start 2023-07-14T18:17:09.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.963+0000 7f70f30a2b80 1 Processor -- start 2023-07-14T18:17:09.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.963+0000 7f70f30a2b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563037aeefe0 0x563037af89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.963+0000 7f70f30a2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563037aef930 0x563037af8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.963+0000 7f70f30a2b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563037af0530 0x563037afff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.963+0000 7f70f30a2b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630379dfb40 con 0x563037af0530 2023-07-14T18:17:09.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.963+0000 7f70f30a2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630379df840 con 0x563037aef930 2023-07-14T18:17:09.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.963+0000 7f70f30a2b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630378c39a0 con 0x563037aeefe0 2023-07-14T18:17:09.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.963+0000 7f70f30a2b80 10 client.0 _ll_register_callbacks cb 0x5630379dd7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:17:09.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.963+0000 7f70dcd9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563037aef930 0x563037af8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.964+0000 7f70dcd9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563037aef930 0x563037af8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:40408/0 (socket says 172.21.15.133:40408) 2023-07-14T18:17:09.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.964+0000 7f70dcd9a700 1 -- 172.21.15.133:0/3315568877 learned_addr learned my addr 172.21.15.133:0/3315568877 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:09.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.964+0000 7f70ddd9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563037af0530 0x563037afff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.964+0000 7f70dd59b700 1 --2- 172.21.15.133:0/3315568877 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563037aeefe0 0x563037af89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.964+0000 7f70dcd9a700 1 -- 172.21.15.133:0/3315568877 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563037aeefe0 msgr2=0x563037af89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:09.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.964+0000 7f70dcd9a700 1 --2- 172.21.15.133:0/3315568877 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563037aeefe0 0x563037af89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.964+0000 7f70dcd9a700 1 -- 172.21.15.133:0/3315568877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563037af0530 msgr2=0x563037afff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:09.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.964+0000 7f70dcd9a700 1 --2- 172.21.15.133:0/3315568877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563037af0530 0x563037afff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.964+0000 7f70dcd9a700 1 -- 172.21.15.133:0/3315568877 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d81a44c0 con 0x563037aef930 2023-07-14T18:17:09.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.964+0000 7f70dcd9a700 1 --2- 172.21.15.133:0/3315568877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563037aef930 0x563037af8ef0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f70c8006010 tx=0x7f70c800beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbf41059b8ee9c98 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:09.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.964+0000 7f70c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:17:09.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.964+0000 7f70c0ff9700 1 -- 172.21.15.133:0/3315568877 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c8004500 con 0x563037aef930 2023-07-14T18:17:09.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.964+0000 7f70f30a2b80 1 -- 172.21.15.133:0/3315568877 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563037999570 con 0x563037aef930 2023-07-14T18:17:09.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.965+0000 7f70c0ff9700 1 -- 172.21.15.133:0/3315568877 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f70c8004660 con 0x563037aef930 2023-07-14T18:17:09.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.965+0000 7f70c0ff9700 1 -- 172.21.15.133:0/3315568877 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70c8004820 con 0x563037aef930 2023-07-14T18:17:09.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.965+0000 7f70f30a2b80 10 client.25586 fetch_fsmap learned FSMap version 28 2023-07-14T18:17:09.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.965+0000 7f70f30a2b80 1 -- 172.21.15.133:0/3315568877 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563037856ce0 con 0x563037aef930 2023-07-14T18:17:09.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.965+0000 7f70c0ff9700 1 -- 172.21.15.133:0/3315568877 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563037999570 con 0x563037aef930 2023-07-14T18:17:09.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.965+0000 7f70c0ff9700 1 -- 172.21.15.133:0/3315568877 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f70c800f920 con 0x563037aef930 2023-07-14T18:17:09.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.965+0000 7f70f30a2b80 10 client.25586 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T18:17:09.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.965+0000 7f70f30a2b80 10 client.25586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:17:09.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.965+0000 7f70f30a2b80 20 client.25586 populate_metadata read hostname 'smithi133' 2023-07-14T18:17:09.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.965+0000 7f70f30a2b80 1 --2- 172.21.15.133:0/3315568877 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x563037b04780 0x563037b06b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.965+0000 7f70f30a2b80 4 client.25586 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:17:09.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.965+0000 7f70f30a2b80 1 -- 172.21.15.133:0/3315568877 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563037abde20 con 0x563037b04780 2023-07-14T18:17:09.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.966+0000 7f70dd59b700 1 --2- 172.21.15.133:0/3315568877 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x563037b04780 0x563037b06b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.966+0000 7f70dd59b700 1 --2- 172.21.15.133:0/3315568877 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x563037b04780 0x563037b06b90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=931332c35689196b server_cookie=2fd1af939f75ef7c in_seq=0 out_seq=0 2023-07-14T18:17:09.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.966+0000 7f70c0ff9700 10 client.25586 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:17:09.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.967+0000 7f70c0ff9700 1 -- 172.21.15.133:0/3315568877 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x563037abde20 con 0x563037b04780 2023-07-14T18:17:09.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.967+0000 7f70c0ff9700 10 client.25586 handle_command_reply: tid=0 2023-07-14T18:17:09.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.967+0000 7f70f30a2b80 1 client.25586 shutdown 2023-07-14T18:17:09.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.967+0000 7f70f30a2b80 1 -- 172.21.15.133:0/3315568877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563037aef930 msgr2=0x563037af8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:09.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.967+0000 7f70f30a2b80 1 --2- 172.21.15.133:0/3315568877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563037aef930 0x563037af8ef0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f70c8006010 tx=0x7f70c800beb0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.967+0000 7f70f30a2b80 1 -- 172.21.15.133:0/3315568877 shutdown_connections 2023-07-14T18:17:09.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.967+0000 7f70f30a2b80 1 --2- 172.21.15.133:0/3315568877 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x563037b04780 0x563037b06b90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.967+0000 7f70f30a2b80 1 --2- 172.21.15.133:0/3315568877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563037af0530 0x563037afff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.967+0000 7f70f30a2b80 1 --2- 172.21.15.133:0/3315568877 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563037aeefe0 0x563037af89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.967+0000 7f70c0ff9700 0 client.25586 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:17:09.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.967+0000 7f70f30a2b80 1 --2- 172.21.15.133:0/3315568877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563037aef930 0x563037af8ef0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.967+0000 7f70f30a2b80 1 -- 172.21.15.133:0/3315568877 >> 172.21.15.133:0/3315568877 conn(0x563037ae4700 msgr2=0x563037ae86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:09.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.967+0000 7f70f30a2b80 1 -- 172.21.15.133:0/3315568877 shutdown_connections 2023-07-14T18:17:09.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.967+0000 7f70f30a2b80 1 -- 172.21.15.133:0/3315568877 wait complete. 2023-07-14T18:17:09.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.967+0000 7f70f30a2b80 20 client.25586 trim_cache size 0 max 16384 2023-07-14T18:17:09.030 INFO:teuthology.orchestra.run.smithi133.stderr:9500000 of 50000000 tuples (19%) done (elapsed 24.82 s, remaining 105.80 s) 2023-07-14T18:17:09.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.999+0000 7f70f30a2b80 1 Processor -- start 2023-07-14T18:17:09.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.999+0000 7f70f30a2b80 1 -- start start 2023-07-14T18:17:09.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.999+0000 7f70f30a2b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563037aeefe0 0x563037af89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.999+0000 7f70f30a2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563037aef930 0x563037affef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.999+0000 7f70f30a2b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563037af0530 0x563037af8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.999+0000 7f70f30a2b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563037af03f0 con 0x563037aeefe0 2023-07-14T18:17:09.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.999+0000 7f70f30a2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563037aea890 con 0x563037aef930 2023-07-14T18:17:09.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.999+0000 7f70f30a2b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x563037aeb560 con 0x563037af0530 2023-07-14T18:17:09.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:08.999+0000 7f70dd59b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563037aeefe0 0x563037af89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.000+0000 7f70dd59b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563037aeefe0 0x563037af89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49176/0 (socket says 172.21.15.133:49176) 2023-07-14T18:17:09.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.000+0000 7f70dd59b700 1 -- 172.21.15.133:0/2692409668 learned_addr learned my addr 172.21.15.133:0/2692409668 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:09.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.000+0000 7f70dcd9a700 1 --2- 172.21.15.133:0/2692409668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563037aef930 0x563037affef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.000+0000 7f70ddd9c700 1 --2- 172.21.15.133:0/2692409668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563037af0530 0x563037af8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.000+0000 7f70dcd9a700 1 -- 172.21.15.133:0/2692409668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563037af0530 msgr2=0x563037af8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:09.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.000+0000 7f70dcd9a700 1 --2- 172.21.15.133:0/2692409668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563037af0530 0x563037af8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.000+0000 7f70dcd9a700 1 -- 172.21.15.133:0/2692409668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563037aeefe0 msgr2=0x563037af89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:09.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.000+0000 7f70dcd9a700 1 --2- 172.21.15.133:0/2692409668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563037aeefe0 0x563037af89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.000+0000 7f70dcd9a700 1 -- 172.21.15.133:0/2692409668 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630378942d0 con 0x563037aef930 2023-07-14T18:17:09.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.000+0000 7f70dcd9a700 1 --2- 172.21.15.133:0/2692409668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563037aef930 0x563037affef0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f70c8005850 tx=0x7f70c8009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f805ff3fc35d590c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:09.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.000+0000 7f70c3fff700 1 -- 172.21.15.133:0/2692409668 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c80059b0 con 0x563037aef930 2023-07-14T18:17:09.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.000+0000 7f70c3fff700 1 -- 172.21.15.133:0/2692409668 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70c801e040 con 0x563037aef930 2023-07-14T18:17:09.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.000+0000 7f70c3fff700 1 -- 172.21.15.133:0/2692409668 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c80179b0 con 0x563037aef930 2023-07-14T18:17:09.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.001+0000 7f70f30a2b80 1 -- 172.21.15.133:0/2692409668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563037aef930 msgr2=0x563037affef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:09.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.001+0000 7f70f30a2b80 1 --2- 172.21.15.133:0/2692409668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563037aef930 0x563037affef0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f70c8005850 tx=0x7f70c8009ca0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.001+0000 7f70f30a2b80 1 -- 172.21.15.133:0/2692409668 shutdown_connections 2023-07-14T18:17:09.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.001+0000 7f70f30a2b80 1 --2- 172.21.15.133:0/2692409668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563037af0530 0x563037af8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.001+0000 7f70f30a2b80 1 --2- 172.21.15.133:0/2692409668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563037aeefe0 0x563037af89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.001+0000 7f70f30a2b80 1 --2- 172.21.15.133:0/2692409668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563037aef930 0x563037affef0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.001+0000 7f70f30a2b80 1 -- 172.21.15.133:0/2692409668 >> 172.21.15.133:0/2692409668 conn(0x563037ae4700 msgr2=0x563037af3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:09.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.001+0000 7f70f30a2b80 1 -- 172.21.15.133:0/2692409668 shutdown_connections 2023-07-14T18:17:09.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.001+0000 7f70f30a2b80 1 -- 172.21.15.133:0/2692409668 wait complete. 2023-07-14T18:17:09.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.002+0000 7f70f30a2b80 1 -- start start 2023-07-14T18:17:09.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.002+0000 7f70f30a2b80 1 Processor -- start 2023-07-14T18:17:09.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.002+0000 7f70f30a2b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563037aeefe0 0x563037afcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.002+0000 7f70f30a2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563037aef930 0x563037afd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.002+0000 7f70f30a2b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563037af0530 0x563037b58410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.002+0000 7f70f30a2b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563037b000f0 con 0x563037aeefe0 2023-07-14T18:17:09.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.002+0000 7f70f30a2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563037b00260 con 0x563037aef930 2023-07-14T18:17:09.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.003+0000 7f70f30a2b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x563037b58950 con 0x563037af0530 2023-07-14T18:17:09.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.003+0000 7f70f30a2b80 10 client.0 _ll_register_callbacks cb 0x7f70d00097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:17:09.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.003+0000 7f70ddd9c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563037af0530 0x563037b58410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.003+0000 7f70dd59b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563037aeefe0 0x563037afcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.003+0000 7f70ddd9c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563037af0530 0x563037b58410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:54936/0 (socket says 172.21.15.133:54936) 2023-07-14T18:17:09.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.003+0000 7f70ddd9c700 1 -- 172.21.15.133:0/245047686 learned_addr learned my addr 172.21.15.133:0/245047686 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:09.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.003+0000 7f70dcd9a700 1 --2- 172.21.15.133:0/245047686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563037aef930 0x563037afd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.003+0000 7f70ddd9c700 1 -- 172.21.15.133:0/245047686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563037aef930 msgr2=0x563037afd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:09.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.003+0000 7f70ddd9c700 1 --2- 172.21.15.133:0/245047686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563037aef930 0x563037afd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.003+0000 7f70ddd9c700 1 -- 172.21.15.133:0/245047686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563037aeefe0 msgr2=0x563037afcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:09.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.003+0000 7f70ddd9c700 1 --2- 172.21.15.133:0/245047686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563037aeefe0 0x563037afcb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.003+0000 7f70ddd9c700 1 -- 172.21.15.133:0/245047686 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563037856ce0 con 0x563037af0530 2023-07-14T18:17:09.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.003+0000 7f70dcd9a700 1 --2- 172.21.15.133:0/245047686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563037aef930 0x563037afd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:17:09.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.003+0000 7f70ddd9c700 1 --2- 172.21.15.133:0/245047686 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563037af0530 0x563037b58410 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f70d0017e50 tx=0x7f70d0004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b02167ff81e29036 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:09.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.003+0000 7f70c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:17:09.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.004+0000 7f70c0ff9700 1 -- 172.21.15.133:0/245047686 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d000e050 con 0x563037af0530 2023-07-14T18:17:09.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.004+0000 7f70f30a2b80 1 -- 172.21.15.133:0/245047686 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70d409e2b0 con 0x563037af0530 2023-07-14T18:17:09.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.004+0000 7f70c0ff9700 1 -- 172.21.15.133:0/245047686 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f70d0018db0 con 0x563037af0530 2023-07-14T18:17:09.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.004+0000 7f70c0ff9700 1 -- 172.21.15.133:0/245047686 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70d0017e80 con 0x563037af0530 2023-07-14T18:17:09.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.004+0000 7f70f30a2b80 10 client.25624 fetch_fsmap learned FSMap version 28 2023-07-14T18:17:09.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.004+0000 7f70f30a2b80 1 -- 172.21.15.133:0/245047686 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56303786c330 con 0x563037af0530 2023-07-14T18:17:09.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.004+0000 7f70c0ff9700 1 -- 172.21.15.133:0/245047686 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70d409e2b0 con 0x563037af0530 2023-07-14T18:17:09.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.004+0000 7f70c0ff9700 1 -- 172.21.15.133:0/245047686 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f70d002a020 con 0x563037af0530 2023-07-14T18:17:09.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.004+0000 7f70f30a2b80 10 client.25624 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T18:17:09.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.004+0000 7f70f30a2b80 10 client.25624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:17:09.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.004+0000 7f70f30a2b80 20 client.25624 populate_metadata read hostname 'smithi133' 2023-07-14T18:17:09.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.004+0000 7f70f30a2b80 1 --2- 172.21.15.133:0/245047686 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x563037b5a2f0 0x563037b5c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.004+0000 7f70f30a2b80 4 client.25624 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T18:17:09.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.005+0000 7f70f30a2b80 1 -- 172.21.15.133:0/245047686 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563037a38f30 con 0x563037b5a2f0 2023-07-14T18:17:09.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.005+0000 7f70dd59b700 1 --2- 172.21.15.133:0/245047686 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x563037b5a2f0 0x563037b5c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.005+0000 7f70c0ff9700 10 client.25624 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:17:09.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.005+0000 7f70dd59b700 1 --2- 172.21.15.133:0/245047686 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x563037b5a2f0 0x563037b5c700 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=352625f26272c96a server_cookie=81fefce78f21e24e in_seq=0 out_seq=0 2023-07-14T18:17:09.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.006+0000 7f70c0ff9700 1 -- 172.21.15.133:0/245047686 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563037a38f30 con 0x563037b5a2f0 2023-07-14T18:17:09.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.006+0000 7f70c0ff9700 10 client.25624 handle_command_reply: tid=0 2023-07-14T18:17:09.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.006+0000 7f70f30a2b80 1 client.25624 shutdown 2023-07-14T18:17:09.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.006+0000 7f70f30a2b80 1 -- 172.21.15.133:0/245047686 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563037af0530 msgr2=0x563037b58410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:09.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.006+0000 7f70f30a2b80 1 --2- 172.21.15.133:0/245047686 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563037af0530 0x563037b58410 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f70d0017e50 tx=0x7f70d0004b40 comp rx=0 tx=0).stop 2023-07-14T18:17:09.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.006+0000 7f70f30a2b80 1 -- 172.21.15.133:0/245047686 shutdown_connections 2023-07-14T18:17:09.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.006+0000 7f70f30a2b80 1 --2- 172.21.15.133:0/245047686 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x563037b5a2f0 0x563037b5c700 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.006+0000 7f70f30a2b80 1 --2- 172.21.15.133:0/245047686 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563037af0530 0x563037b58410 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.007+0000 7f70f30a2b80 1 --2- 172.21.15.133:0/245047686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563037aeefe0 0x563037afcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.007+0000 7f70c0ff9700 0 client.25624 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:17:09.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.007+0000 7f70f30a2b80 1 --2- 172.21.15.133:0/245047686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563037aef930 0x563037afd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.007+0000 7f70f30a2b80 1 -- 172.21.15.133:0/245047686 >> 172.21.15.133:0/245047686 conn(0x563037ae4700 msgr2=0x563037af8950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:09.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.007+0000 7f70f30a2b80 1 -- 172.21.15.133:0/245047686 shutdown_connections 2023-07-14T18:17:09.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.007+0000 7f70f30a2b80 1 -- 172.21.15.133:0/245047686 wait complete. 2023-07-14T18:17:09.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.007+0000 7f70f30a2b80 20 client.25624 trim_cache size 0 max 16384 2023-07-14T18:17:09.060 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:17:09.060 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2023-07-14T18:17:09.061 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:17:09.061 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2023-07-14T18:17:09.061 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:17:09.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.007+0000 7f70df7ff700 1 -- 172.21.15.133:0/1762843185 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f70c40a1150 msgr2=0x7f70c40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:09.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.007+0000 7f70df7ff700 1 --2- 172.21.15.133:0/1762843185 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f70c40a1150 0x7f70c40a3600 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009410 tx=0x7f70d0009920 comp rx=0 tx=0).stop 2023-07-14T18:17:09.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.007+0000 7f70df7ff700 1 -- 172.21.15.133:0/1762843185 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70d80730a0 msgr2=0x7f70d819c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:09.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.007+0000 7f70df7ff700 1 --2- 172.21.15.133:0/1762843185 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70d80730a0 0x7f70d819c010 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f70d400ea30 tx=0x7f70d400edf0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.008+0000 7f70df7ff700 1 -- 172.21.15.133:0/1762843185 shutdown_connections 2023-07-14T18:17:09.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.008+0000 7f70df7ff700 1 --2- 172.21.15.133:0/1762843185 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f70c40a1150 0x7f70c40a3600 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.008+0000 7f70df7ff700 1 --2- 172.21.15.133:0/1762843185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70d810c1f0 0x7f70d81a35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.008+0000 7f70df7ff700 1 --2- 172.21.15.133:0/1762843185 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70d80730a0 0x7f70d819c010 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.008+0000 7f70df7ff700 1 --2- 172.21.15.133:0/1762843185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70d8074680 0x7f70d819c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.008+0000 7f70df7ff700 1 -- 172.21.15.133:0/1762843185 >> 172.21.15.133:0/1762843185 conn(0x7f70d80fb260 msgr2=0x7f70d80fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:09.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.008+0000 7f70df7ff700 1 -- 172.21.15.133:0/1762843185 shutdown_connections 2023-07-14T18:17:09.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.008+0000 7f70df7ff700 1 -- 172.21.15.133:0/1762843185 wait complete. 2023-07-14T18:17:09.073 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:17:09.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:08 smithi133 ceph-mon[126384]: pgmap v362: 129 pgs: 129 active+clean; 1.6 GiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 146 MiB/s wr, 80 op/s 2023-07-14T18:17:09.176 INFO:teuthology.orchestra.run.smithi133.stderr:9600000 of 50000000 tuples (19%) done (elapsed 24.96 s, remaining 105.05 s) 2023-07-14T18:17:09.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.284+0000 7f602adff700 1 -- 172.21.15.133:0/2034621473 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6024074680 msgr2=0x7f6024072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:09.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.284+0000 7f602adff700 1 --2- 172.21.15.133:0/2034621473 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6024074680 0x7f6024072ad0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f60200098f0 tx=0x7f6020009c00 comp rx=0 tx=0).stop 2023-07-14T18:17:09.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.284+0000 7f602adff700 1 -- 172.21.15.133:0/2034621473 shutdown_connections 2023-07-14T18:17:09.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.284+0000 7f602adff700 1 --2- 172.21.15.133:0/2034621473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6024107ad0 0x7f6024109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.284+0000 7f602adff700 1 --2- 172.21.15.133:0/2034621473 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6024074680 0x7f6024072ad0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.284+0000 7f602adff700 1 --2- 172.21.15.133:0/2034621473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60240730a0 0x7f6024073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.284+0000 7f602adff700 1 -- 172.21.15.133:0/2034621473 >> 172.21.15.133:0/2034621473 conn(0x7f60240fb210 msgr2=0x7f60240fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:09.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.284+0000 7f602adff700 1 -- 172.21.15.133:0/2034621473 shutdown_connections 2023-07-14T18:17:09.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.284+0000 7f602adff700 1 -- 172.21.15.133:0/2034621473 wait complete. 2023-07-14T18:17:09.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.286+0000 7f602adff700 1 Processor -- start 2023-07-14T18:17:09.333 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.286+0000 7f602adff700 1 -- start start 2023-07-14T18:17:09.333 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.286+0000 7f602adff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60240730a0 0x7f6024197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.333 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.286+0000 7f602adff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6024074680 0x7f6024198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.333 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.286+0000 7f602adff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6024107ad0 0x7f602419f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.333 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.286+0000 7f602adff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602419f970 con 0x7f6024107ad0 2023-07-14T18:17:09.333 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.286+0000 7f602adff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602419fab0 con 0x7f60240730a0 2023-07-14T18:17:09.333 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.286+0000 7f602adff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602419fc40 con 0x7f6024074680 2023-07-14T18:17:09.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.286+0000 7f6028b9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60240730a0 0x7f6024197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.286+0000 7f6028b9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60240730a0 0x7f6024197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:40440/0 (socket says 172.21.15.133:40440) 2023-07-14T18:17:09.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.286+0000 7f601bfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6024074680 0x7f6024198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.287+0000 7f6028b9b700 1 -- 172.21.15.133:0/252256673 learned_addr learned my addr 172.21.15.133:0/252256673 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:09.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.287+0000 7f602939c700 1 --2- 172.21.15.133:0/252256673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6024107ad0 0x7f602419f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.287+0000 7f6028b9b700 1 -- 172.21.15.133:0/252256673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6024074680 msgr2=0x7f6024198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:09.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.287+0000 7f6028b9b700 1 --2- 172.21.15.133:0/252256673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6024074680 0x7f6024198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.287+0000 7f6028b9b700 1 -- 172.21.15.133:0/252256673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6024107ad0 msgr2=0x7f602419f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:09.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.287+0000 7f6028b9b700 1 --2- 172.21.15.133:0/252256673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6024107ad0 0x7f602419f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.287+0000 7f6028b9b700 1 -- 172.21.15.133:0/252256673 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6020009520 con 0x7f60240730a0 2023-07-14T18:17:09.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.287+0000 7f602939c700 1 --2- 172.21.15.133:0/252256673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6024107ad0 0x7f602419f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:17:09.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.287+0000 7f6028b9b700 1 --2- 172.21.15.133:0/252256673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60240730a0 0x7f6024197df0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6020009fd0 tx=0x7f602000b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:09.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.287+0000 7f6019ffb700 1 -- 172.21.15.133:0/252256673 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602001c070 con 0x7f60240730a0 2023-07-14T18:17:09.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.287+0000 7f6019ffb700 1 -- 172.21.15.133:0/252256673 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f602000f7d0 con 0x7f60240730a0 2023-07-14T18:17:09.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.287+0000 7f602adff700 1 -- 172.21.15.133:0/252256673 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f602419fec0 con 0x7f60240730a0 2023-07-14T18:17:09.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.288+0000 7f6019ffb700 1 -- 172.21.15.133:0/252256673 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020003a80 con 0x7f60240730a0 2023-07-14T18:17:09.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.288+0000 7f602adff700 1 -- 172.21.15.133:0/252256673 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60241a0280 con 0x7f60240730a0 2023-07-14T18:17:09.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.291+0000 7f6019ffb700 1 -- 172.21.15.133:0/252256673 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6020020030 con 0x7f60240730a0 2023-07-14T18:17:09.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.292+0000 7f603e78bb80 1 Processor -- start 2023-07-14T18:17:09.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.292+0000 7f603e78bb80 1 -- start start 2023-07-14T18:17:09.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.292+0000 7f603e78bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8746aff60 0x55d8746b0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.292+0000 7f603e78bb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8746b08b0 0x55d8746b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.292+0000 7f603e78bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8746b14b0 0x55d8746b1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.292+0000 7f603e78bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8745b4cb0 con 0x55d8746b14b0 2023-07-14T18:17:09.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.292+0000 7f603e78bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d874607a40 con 0x55d8746aff60 2023-07-14T18:17:09.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.292+0000 7f603e78bb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d874600ed0 con 0x55d8746b08b0 2023-07-14T18:17:09.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.293+0000 7f6028b9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8746b08b0 0x55d8746b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.293+0000 7f602939c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8746aff60 0x55d8746b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.293+0000 7f602939c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8746aff60 0x55d8746b0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:40454/0 (socket says 172.21.15.133:40454) 2023-07-14T18:17:09.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.293+0000 7f602939c700 1 -- 172.21.15.133:0/2744406670 learned_addr learned my addr 172.21.15.133:0/2744406670 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:09.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.293+0000 7f6028b9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8746b08b0 0x55d8746b0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:54968/0 (socket says 172.21.15.133:54968) 2023-07-14T18:17:09.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.293+0000 7f6019ffb700 1 --2- 172.21.15.133:0/252256673 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f600c0a11d0 0x7f600c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.293+0000 7f6019ffb700 1 -- 172.21.15.133:0/252256673 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f60200cdc00 con 0x7f60240730a0 2023-07-14T18:17:09.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.293+0000 7f601bfff700 1 --2- 172.21.15.133:0/2744406670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8746b14b0 0x55d8746b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.293+0000 7f6028b9b700 1 -- 172.21.15.133:0/2744406670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8746aff60 msgr2=0x55d8746b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:09.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.293+0000 7f6028b9b700 1 --2- 172.21.15.133:0/2744406670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8746aff60 0x55d8746b0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.293+0000 7f602939c700 1 --2- 172.21.15.133:0/252256673 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f600c0a11d0 0x7f600c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.293+0000 7f6028b9b700 1 -- 172.21.15.133:0/2744406670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8746b14b0 msgr2=0x55d8746b1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:09.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.293+0000 7f6028b9b700 1 --2- 172.21.15.133:0/2744406670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8746b14b0 0x55d8746b1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.293+0000 7f6028b9b700 1 -- 172.21.15.133:0/2744406670 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d87442ede0 con 0x55d8746b08b0 2023-07-14T18:17:09.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.293+0000 7f6028b9b700 1 --2- 172.21.15.133:0/2744406670 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8746b08b0 0x55d8746b0ed0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6020097210 tx=0x7f6020005750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38c88f118b300d1a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:09.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.293+0000 7f600affd700 1 -- 172.21.15.133:0/2744406670 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60200d2040 con 0x55d8746b08b0 2023-07-14T18:17:09.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.294+0000 7f600affd700 1 -- 172.21.15.133:0/2744406670 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f602009a850 con 0x55d8746b08b0 2023-07-14T18:17:09.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.294+0000 7f600affd700 1 -- 172.21.15.133:0/2744406670 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020003be0 con 0x55d8746b08b0 2023-07-14T18:17:09.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.294+0000 7f603e78bb80 1 -- 172.21.15.133:0/2744406670 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8746b08b0 msgr2=0x55d8746b0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:09.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.294+0000 7f603e78bb80 1 --2- 172.21.15.133:0/2744406670 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8746b08b0 0x55d8746b0ed0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6020097210 tx=0x7f6020005750 comp rx=0 tx=0).stop 2023-07-14T18:17:09.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.294+0000 7f602939c700 1 --2- 172.21.15.133:0/252256673 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f600c0a11d0 0x7f600c0a3680 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f601c0094f0 tx=0x7f601c004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:09.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.294+0000 7f603e78bb80 1 -- 172.21.15.133:0/2744406670 shutdown_connections 2023-07-14T18:17:09.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.294+0000 7f603e78bb80 1 --2- 172.21.15.133:0/2744406670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8746aff60 0x55d8746b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.294+0000 7f603e78bb80 1 --2- 172.21.15.133:0/2744406670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8746b14b0 0x55d8746b1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.294+0000 7f603e78bb80 1 --2- 172.21.15.133:0/2744406670 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8746b08b0 0x55d8746b0ed0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.294+0000 7f603e78bb80 1 -- 172.21.15.133:0/2744406670 >> 172.21.15.133:0/2744406670 conn(0x55d8746a5680 msgr2=0x55d8746aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:09.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.294+0000 7f603e78bb80 1 -- 172.21.15.133:0/2744406670 shutdown_connections 2023-07-14T18:17:09.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.294+0000 7f603e78bb80 1 -- 172.21.15.133:0/2744406670 wait complete. 2023-07-14T18:17:09.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.295+0000 7f603e78bb80 1 -- start start 2023-07-14T18:17:09.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.295+0000 7f603e78bb80 1 Processor -- start 2023-07-14T18:17:09.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.295+0000 7f603e78bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8746aff60 0x55d8746b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.295+0000 7f603e78bb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8746b08b0 0x55d8746b9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.295+0000 7f603e78bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8746b14b0 0x55d8746c0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.296+0000 7f603e78bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8745a0ac0 con 0x55d8746b14b0 2023-07-14T18:17:09.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.296+0000 7f603e78bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8745a07c0 con 0x55d8746aff60 2023-07-14T18:17:09.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.296+0000 7f603e78bb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d874484920 con 0x55d8746b08b0 2023-07-14T18:17:09.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.296+0000 7f603e78bb80 10 client.0 _ll_register_callbacks cb 0x55d87459e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:17:09.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.296+0000 7f601bfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8746b08b0 0x55d8746b9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.296+0000 7f6028b9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8746aff60 0x55d8746b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.296+0000 7f601bfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8746b08b0 0x55d8746b9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:54974/0 (socket says 172.21.15.133:54974) 2023-07-14T18:17:09.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.296+0000 7f601bfff700 1 -- 172.21.15.133:0/3309430626 learned_addr learned my addr 172.21.15.133:0/3309430626 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:09.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.296+0000 7f602939c700 1 --2- 172.21.15.133:0/3309430626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8746b14b0 0x55d8746c0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.296+0000 7f601bfff700 1 -- 172.21.15.133:0/3309430626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8746aff60 msgr2=0x55d8746b9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:09.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.296+0000 7f601bfff700 1 --2- 172.21.15.133:0/3309430626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8746aff60 0x55d8746b9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.296+0000 7f601bfff700 1 -- 172.21.15.133:0/3309430626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8746b14b0 msgr2=0x55d8746c0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:09.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.296+0000 7f601bfff700 1 --2- 172.21.15.133:0/3309430626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8746b14b0 0x55d8746c0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.296+0000 7f601bfff700 1 -- 172.21.15.133:0/3309430626 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60241a0280 con 0x55d8746b08b0 2023-07-14T18:17:09.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.296+0000 7f6028b9b700 1 --2- 172.21.15.133:0/3309430626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8746aff60 0x55d8746b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:17:09.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.296+0000 7f601bfff700 1 --2- 172.21.15.133:0/3309430626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8746b08b0 0x55d8746b9ac0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6014013710 tx=0x7f6014013a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3411a04e1257da1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:09.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.296+0000 7f6003fff700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:17:09.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.297+0000 7f6003fff700 1 -- 172.21.15.133:0/3309430626 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6014004020 con 0x55d8746b08b0 2023-07-14T18:17:09.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.297+0000 7f603e78bb80 1 -- 172.21.15.133:0/3309430626 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d87455a4f0 con 0x55d8746b08b0 2023-07-14T18:17:09.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.297+0000 7f6003fff700 1 -- 172.21.15.133:0/3309430626 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6014004180 con 0x55d8746b08b0 2023-07-14T18:17:09.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.297+0000 7f6003fff700 1 -- 172.21.15.133:0/3309430626 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6014004340 con 0x55d8746b08b0 2023-07-14T18:17:09.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.297+0000 7f603e78bb80 10 client.25636 fetch_fsmap learned FSMap version 28 2023-07-14T18:17:09.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.297+0000 7f603e78bb80 1 -- 172.21.15.133:0/3309430626 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d874417c60 con 0x55d8746b08b0 2023-07-14T18:17:09.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.297+0000 7f6003fff700 1 -- 172.21.15.133:0/3309430626 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d87455a4f0 con 0x55d8746b08b0 2023-07-14T18:17:09.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.297+0000 7f6003fff700 1 -- 172.21.15.133:0/3309430626 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6014010830 con 0x55d8746b08b0 2023-07-14T18:17:09.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.298+0000 7f603e78bb80 10 client.25636 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T18:17:09.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.298+0000 7f603e78bb80 10 client.25636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:17:09.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.298+0000 7f603e78bb80 20 client.25636 populate_metadata read hostname 'smithi133' 2023-07-14T18:17:09.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.298+0000 7f603e78bb80 1 --2- 172.21.15.133:0/3309430626 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55d8746c53e0 0x55d8746c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.298+0000 7f603e78bb80 4 client.25636 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:17:09.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.298+0000 7f603e78bb80 1 -- 172.21.15.133:0/3309430626 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d87467eda0 con 0x55d8746c53e0 2023-07-14T18:17:09.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.298+0000 7f6028b9b700 1 --2- 172.21.15.133:0/3309430626 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55d8746c53e0 0x55d8746c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.298+0000 7f6028b9b700 1 --2- 172.21.15.133:0/3309430626 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55d8746c53e0 0x55d8746c77f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=936371307dacec3f server_cookie=51490ef9e1c99c3e in_seq=0 out_seq=0 2023-07-14T18:17:09.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.298+0000 7f6003fff700 10 client.25636 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:17:09.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.299+0000 7f6003fff700 1 -- 172.21.15.133:0/3309430626 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55d87467eda0 con 0x55d8746c53e0 2023-07-14T18:17:09.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.299+0000 7f6003fff700 10 client.25636 handle_command_reply: tid=0 2023-07-14T18:17:09.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.299+0000 7f603e78bb80 1 client.25636 shutdown 2023-07-14T18:17:09.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.299+0000 7f603e78bb80 1 -- 172.21.15.133:0/3309430626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8746b08b0 msgr2=0x55d8746b9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:09.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.299+0000 7f603e78bb80 1 --2- 172.21.15.133:0/3309430626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8746b08b0 0x55d8746b9ac0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6014013710 tx=0x7f6014013a20 comp rx=0 tx=0).stop 2023-07-14T18:17:09.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.299+0000 7f603e78bb80 1 -- 172.21.15.133:0/3309430626 shutdown_connections 2023-07-14T18:17:09.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.299+0000 7f603e78bb80 1 --2- 172.21.15.133:0/3309430626 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55d8746c53e0 0x55d8746c77f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.299+0000 7f603e78bb80 1 --2- 172.21.15.133:0/3309430626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8746aff60 0x55d8746b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.299+0000 7f603e78bb80 1 --2- 172.21.15.133:0/3309430626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8746b14b0 0x55d8746c0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.299+0000 7f6003fff700 0 client.25636 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:17:09.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.299+0000 7f603e78bb80 1 --2- 172.21.15.133:0/3309430626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8746b08b0 0x55d8746b9ac0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.299+0000 7f603e78bb80 1 -- 172.21.15.133:0/3309430626 >> 172.21.15.133:0/3309430626 conn(0x55d8746a5680 msgr2=0x55d8746a9650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:09.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.299+0000 7f603e78bb80 1 -- 172.21.15.133:0/3309430626 shutdown_connections 2023-07-14T18:17:09.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.299+0000 7f603e78bb80 1 -- 172.21.15.133:0/3309430626 wait complete. 2023-07-14T18:17:09.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.300+0000 7f603e78bb80 20 client.25636 trim_cache size 0 max 16384 2023-07-14T18:17:09.354 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:08 smithi161 ceph-mon[128993]: pgmap v362: 129 pgs: 129 active+clean; 1.6 GiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 146 MiB/s wr, 80 op/s 2023-07-14T18:17:09.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.319+0000 7f603e78bb80 1 Processor -- start 2023-07-14T18:17:09.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.319+0000 7f603e78bb80 1 -- start start 2023-07-14T18:17:09.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.319+0000 7f603e78bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8746aff60 0x55d8746bea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.319+0000 7f603e78bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8746b08b0 0x55d8746b7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.319+0000 7f603e78bb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8746b14b0 0x55d8746b7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.319+0000 7f603e78bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8746b1330 con 0x55d8746b08b0 2023-07-14T18:17:09.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.319+0000 7f603e78bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8746b8a30 con 0x55d8746aff60 2023-07-14T18:17:09.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.319+0000 7f603e78bb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8746b90f0 con 0x55d8746b14b0 2023-07-14T18:17:09.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.319+0000 7f6028b9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8746aff60 0x55d8746bea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.319+0000 7f6028b9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8746aff60 0x55d8746bea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:40462/0 (socket says 172.21.15.133:40462) 2023-07-14T18:17:09.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.319+0000 7f6028b9b700 1 -- 172.21.15.133:0/1067234965 learned_addr learned my addr 172.21.15.133:0/1067234965 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:09.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.320+0000 7f602939c700 1 --2- 172.21.15.133:0/1067234965 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8746b14b0 0x55d8746b7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.320+0000 7f601bfff700 1 --2- 172.21.15.133:0/1067234965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8746b08b0 0x55d8746b7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.320+0000 7f6028b9b700 1 -- 172.21.15.133:0/1067234965 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8746b14b0 msgr2=0x55d8746b7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:09.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.320+0000 7f6028b9b700 1 --2- 172.21.15.133:0/1067234965 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8746b14b0 0x55d8746b7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.320+0000 7f6028b9b700 1 -- 172.21.15.133:0/1067234965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8746b08b0 msgr2=0x55d8746b7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:09.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.320+0000 7f6028b9b700 1 --2- 172.21.15.133:0/1067234965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8746b08b0 0x55d8746b7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.320+0000 7f6028b9b700 1 -- 172.21.15.133:0/1067234965 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d874455250 con 0x55d8746aff60 2023-07-14T18:17:09.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.320+0000 7f6028b9b700 1 --2- 172.21.15.133:0/1067234965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8746aff60 0x55d8746bea10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f602009aca0 tx=0x7f6020097100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dc12c3b737ba90f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:09.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:08 smithi171 ceph-mon[129241]: pgmap v362: 129 pgs: 129 active+clean; 1.6 GiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 146 MiB/s wr, 80 op/s 2023-07-14T18:17:09.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.320+0000 7f600b7fe700 1 -- 172.21.15.133:0/1067234965 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60200d2070 con 0x55d8746aff60 2023-07-14T18:17:09.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.320+0000 7f600b7fe700 1 -- 172.21.15.133:0/1067234965 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f602009a670 con 0x55d8746aff60 2023-07-14T18:17:09.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.320+0000 7f600b7fe700 1 -- 172.21.15.133:0/1067234965 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020003be0 con 0x55d8746aff60 2023-07-14T18:17:09.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.321+0000 7f603e78bb80 1 -- 172.21.15.133:0/1067234965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8746aff60 msgr2=0x55d8746bea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:09.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.321+0000 7f603e78bb80 1 --2- 172.21.15.133:0/1067234965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8746aff60 0x55d8746bea10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f602009aca0 tx=0x7f6020097100 comp rx=0 tx=0).stop 2023-07-14T18:17:09.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.321+0000 7f603e78bb80 1 -- 172.21.15.133:0/1067234965 shutdown_connections 2023-07-14T18:17:09.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.321+0000 7f603e78bb80 1 --2- 172.21.15.133:0/1067234965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8746aff60 0x55d8746bea10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.321+0000 7f603e78bb80 1 --2- 172.21.15.133:0/1067234965 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8746b14b0 0x55d8746b7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.321+0000 7f603e78bb80 1 --2- 172.21.15.133:0/1067234965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8746b08b0 0x55d8746b7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.372 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.321+0000 7f603e78bb80 1 -- 172.21.15.133:0/1067234965 >> 172.21.15.133:0/1067234965 conn(0x55d8746a5680 msgr2=0x55d8746c13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:09.372 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.321+0000 7f603e78bb80 1 -- 172.21.15.133:0/1067234965 shutdown_connections 2023-07-14T18:17:09.372 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.321+0000 7f603e78bb80 1 -- 172.21.15.133:0/1067234965 wait complete. 2023-07-14T18:17:09.372 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.322+0000 7f603e78bb80 1 -- start start 2023-07-14T18:17:09.372 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.322+0000 7f603e78bb80 1 Processor -- start 2023-07-14T18:17:09.372 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.322+0000 7f603e78bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8746b08b0 0x55d8746be180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.372 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.322+0000 7f603e78bb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8746b14b0 0x55d8746be760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.322+0000 7f603e78bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8746bcb30 0x55d8746bcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.322+0000 7f603e78bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8746bd4e0 con 0x55d8746b08b0 2023-07-14T18:17:09.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.322+0000 7f603e78bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8746bd650 con 0x55d8746bcb30 2023-07-14T18:17:09.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.322+0000 7f603e78bb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8746bd810 con 0x55d8746b14b0 2023-07-14T18:17:09.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.322+0000 7f603e78bb80 10 client.0 _ll_register_callbacks cb 0x7f6014009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:17:09.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.322+0000 7f601bfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8746b14b0 0x55d8746be760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.322+0000 7f6028b9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8746b08b0 0x55d8746be180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.323+0000 7f602939c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8746bcb30 0x55d8746bcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.323+0000 7f6028b9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8746b08b0 0x55d8746be180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49234/0 (socket says 172.21.15.133:49234) 2023-07-14T18:17:09.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.323+0000 7f6028b9b700 1 -- 172.21.15.133:0/2261296163 learned_addr learned my addr 172.21.15.133:0/2261296163 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:09.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.323+0000 7f601bfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8746b14b0 0x55d8746be760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:55000/0 (socket says 172.21.15.133:55000) 2023-07-14T18:17:09.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.323+0000 7f6028b9b700 1 -- 172.21.15.133:0/2261296163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8746b14b0 msgr2=0x55d8746be760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:09.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.323+0000 7f6028b9b700 1 --2- 172.21.15.133:0/2261296163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8746b14b0 0x55d8746be760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.323+0000 7f6028b9b700 1 -- 172.21.15.133:0/2261296163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8746bcb30 msgr2=0x55d8746bcfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:09.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.323+0000 7f6028b9b700 1 --2- 172.21.15.133:0/2261296163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8746bcb30 0x55d8746bcfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.323+0000 7f6028b9b700 1 -- 172.21.15.133:0/2261296163 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d87442ede0 con 0x55d8746b08b0 2023-07-14T18:17:09.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.323+0000 7f601bfff700 1 --2- 172.21.15.133:0/2261296163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8746b14b0 0x55d8746be760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:17:09.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.323+0000 7f6028b9b700 1 --2- 172.21.15.133:0/2261296163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8746b08b0 0x55d8746be180 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f602009a9c0 tx=0x7f6020009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ab2f21449c8fd3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:09.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.323+0000 7f6003fff700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T18:17:09.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.323+0000 7f6003fff700 1 -- 172.21.15.133:0/2261296163 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60200d2070 con 0x55d8746b08b0 2023-07-14T18:17:09.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.323+0000 7f603e78bb80 1 -- 172.21.15.133:0/2261296163 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f602009a850 con 0x55d8746b08b0 2023-07-14T18:17:09.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.323+0000 7f6003fff700 1 -- 172.21.15.133:0/2261296163 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f602009ad10 con 0x55d8746b08b0 2023-07-14T18:17:09.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.323+0000 7f6003fff700 1 -- 172.21.15.133:0/2261296163 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60200cb0a0 con 0x55d8746b08b0 2023-07-14T18:17:09.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.324+0000 7f603e78bb80 10 client.15624 fetch_fsmap learned FSMap version 28 2023-07-14T18:17:09.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.324+0000 7f603e78bb80 1 -- 172.21.15.133:0/2261296163 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d87442d2b0 con 0x55d8746b08b0 2023-07-14T18:17:09.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.324+0000 7f6003fff700 1 -- 172.21.15.133:0/2261296163 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f602009a850 con 0x55d8746b08b0 2023-07-14T18:17:09.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.324+0000 7f6003fff700 1 -- 172.21.15.133:0/2261296163 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6020015020 con 0x55d8746b08b0 2023-07-14T18:17:09.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.324+0000 7f603e78bb80 10 client.15624 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T18:17:09.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.324+0000 7f603e78bb80 10 client.15624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:17:09.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.324+0000 7f603e78bb80 20 client.15624 populate_metadata read hostname 'smithi133' 2023-07-14T18:17:09.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.324+0000 7f603e78bb80 1 --2- 172.21.15.133:0/2261296163 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55d87471a0f0 0x55d87471c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:09.377 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.324+0000 7f603e78bb80 4 client.15624 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:17:09.377 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.324+0000 7f603e78bb80 1 -- 172.21.15.133:0/2261296163 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d8745f9eb0 con 0x55d87471a0f0 2023-07-14T18:17:09.377 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.324+0000 7f601bfff700 1 --2- 172.21.15.133:0/2261296163 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55d87471a0f0 0x55d87471c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:09.377 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.325+0000 7f6003fff700 10 client.15624 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:17:09.377 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.325+0000 7f601bfff700 1 --2- 172.21.15.133:0/2261296163 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55d87471a0f0 0x55d87471c4d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0d0bfb0ce716caa server_cookie=b08e730b6e0ee61e in_seq=0 out_seq=0 2023-07-14T18:17:09.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.325+0000 7f6003fff700 1 -- 172.21.15.133:0/2261296163 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55d8745f9eb0 con 0x55d87471a0f0 2023-07-14T18:17:09.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.325+0000 7f6003fff700 10 client.15624 handle_command_reply: tid=0 2023-07-14T18:17:09.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.325+0000 7f603e78bb80 1 client.15624 shutdown 2023-07-14T18:17:09.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.325+0000 7f603e78bb80 1 -- 172.21.15.133:0/2261296163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8746b08b0 msgr2=0x55d8746be180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:09.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.325+0000 7f603e78bb80 1 --2- 172.21.15.133:0/2261296163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8746b08b0 0x55d8746be180 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f602009a9c0 tx=0x7f6020009c00 comp rx=0 tx=0).stop 2023-07-14T18:17:09.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.325+0000 7f603e78bb80 1 -- 172.21.15.133:0/2261296163 shutdown_connections 2023-07-14T18:17:09.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.325+0000 7f603e78bb80 1 --2- 172.21.15.133:0/2261296163 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55d87471a0f0 0x55d87471c4d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.325+0000 7f603e78bb80 1 --2- 172.21.15.133:0/2261296163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8746bcb30 0x55d8746bcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.325+0000 7f603e78bb80 1 --2- 172.21.15.133:0/2261296163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8746b08b0 0x55d8746be180 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.325+0000 7f603e78bb80 1 --2- 172.21.15.133:0/2261296163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8746b14b0 0x55d8746be760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.325+0000 7f603e78bb80 1 -- 172.21.15.133:0/2261296163 >> 172.21.15.133:0/2261296163 conn(0x55d8746a5680 msgr2=0x55d8746b7770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:09.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.325+0000 7f6003fff700 0 client.15624 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:17:09.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.325+0000 7f603e78bb80 1 -- 172.21.15.133:0/2261296163 shutdown_connections 2023-07-14T18:17:09.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.325+0000 7f603e78bb80 1 -- 172.21.15.133:0/2261296163 wait complete. 2023-07-14T18:17:09.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.326+0000 7f603e78bb80 20 client.15624 trim_cache size 0 max 16384 2023-07-14T18:17:09.380 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:17:09.380 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-14T18:17:09.380 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:17:09.381 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:17:09.381 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:17:09.381 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:17:09.381 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:17:09.381 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:17:09.381 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:17:09.381 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:17:09.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.326+0000 7f602adff700 1 -- 172.21.15.133:0/252256673 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f600c0a11d0 msgr2=0x7f600c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:09.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.326+0000 7f602adff700 1 --2- 172.21.15.133:0/252256673 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f600c0a11d0 0x7f600c0a3680 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f601c0094f0 tx=0x7f601c004b40 comp rx=0 tx=0).stop 2023-07-14T18:17:09.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.326+0000 7f602adff700 1 -- 172.21.15.133:0/252256673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60240730a0 msgr2=0x7f6024197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:09.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.326+0000 7f602adff700 1 --2- 172.21.15.133:0/252256673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60240730a0 0x7f6024197df0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6020009fd0 tx=0x7f602000b840 comp rx=0 tx=0).stop 2023-07-14T18:17:09.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.326+0000 7f602adff700 1 -- 172.21.15.133:0/252256673 shutdown_connections 2023-07-14T18:17:09.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.326+0000 7f602adff700 1 --2- 172.21.15.133:0/252256673 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f600c0a11d0 0x7f600c0a3680 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.326+0000 7f602adff700 1 --2- 172.21.15.133:0/252256673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60240730a0 0x7f6024197df0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.326+0000 7f602adff700 1 --2- 172.21.15.133:0/252256673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6024107ad0 0x7f602419f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.326+0000 7f602adff700 1 --2- 172.21.15.133:0/252256673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6024074680 0x7f6024198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:09.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.326+0000 7f602adff700 1 -- 172.21.15.133:0/252256673 >> 172.21.15.133:0/252256673 conn(0x7f60240fb210 msgr2=0x7f6024109e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:09.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.326+0000 7f602adff700 1 -- 172.21.15.133:0/252256673 shutdown_connections 2023-07-14T18:17:09.384 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:09.326+0000 7f602adff700 1 -- 172.21.15.133:0/252256673 wait complete. 2023-07-14T18:17:09.390 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:17:09.576 INFO:teuthology.orchestra.run.smithi133.stderr:9700000 of 50000000 tuples (19%) done (elapsed 25.36 s, remaining 105.37 s) 2023-07-14T18:17:10.117 INFO:teuthology.orchestra.run.smithi133.stderr:9800000 of 50000000 tuples (19%) done (elapsed 25.90 s, remaining 106.26 s) 2023-07-14T18:17:10.270 INFO:teuthology.orchestra.run.smithi133.stderr:9900000 of 50000000 tuples (19%) done (elapsed 26.06 s, remaining 105.54 s) 2023-07-14T18:17:10.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:10 smithi161 ceph-mon[128993]: scrub queued for path: / 2023-07-14T18:17:10.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:10 smithi161 ceph-mon[128993]: scrub summary: idle+waiting paths [/] 2023-07-14T18:17:10.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:10 smithi161 ceph-mon[128993]: scrub summary: active paths [/] 2023-07-14T18:17:10.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:10 smithi161 ceph-mon[128993]: scrub summary: active paths [/] 2023-07-14T18:17:10.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:10 smithi161 ceph-mon[128993]: scrub summary: active paths [/] 2023-07-14T18:17:10.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:10 smithi161 ceph-mon[128993]: scrub summary: idle+waiting paths [/] 2023-07-14T18:17:10.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:10 smithi161 ceph-mon[128993]: scrub summary: idle+waiting paths [/] 2023-07-14T18:17:10.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:10 smithi161 ceph-mon[128993]: scrub summary: active paths [/] 2023-07-14T18:17:10.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:10 smithi161 ceph-mon[128993]: scrub summary: active paths [/] 2023-07-14T18:17:10.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:10 smithi161 ceph-mon[128993]: scrub summary: idle+waiting paths [/] 2023-07-14T18:17:10.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:10 smithi161 ceph-mon[128993]: scrub summary: active paths [/] 2023-07-14T18:17:10.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:10 smithi161 ceph-mon[128993]: scrub summary: idle+waiting paths [/] 2023-07-14T18:17:10.354 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:10 smithi161 ceph-mon[128993]: scrub summary: idle+waiting paths [/] 2023-07-14T18:17:10.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:10 smithi171 ceph-mon[129241]: scrub queued for path: / 2023-07-14T18:17:10.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:10 smithi171 ceph-mon[129241]: scrub summary: idle+waiting paths [/] 2023-07-14T18:17:10.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:10 smithi171 ceph-mon[129241]: scrub summary: active paths [/] 2023-07-14T18:17:10.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:10 smithi171 ceph-mon[129241]: scrub summary: active paths [/] 2023-07-14T18:17:10.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:10 smithi171 ceph-mon[129241]: scrub summary: active paths [/] 2023-07-14T18:17:10.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:10 smithi171 ceph-mon[129241]: scrub summary: idle+waiting paths [/] 2023-07-14T18:17:10.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:10 smithi171 ceph-mon[129241]: scrub summary: idle+waiting paths [/] 2023-07-14T18:17:10.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:10 smithi171 ceph-mon[129241]: scrub summary: active paths [/] 2023-07-14T18:17:10.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:10 smithi171 ceph-mon[129241]: scrub summary: active paths [/] 2023-07-14T18:17:10.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:10 smithi171 ceph-mon[129241]: scrub summary: idle+waiting paths [/] 2023-07-14T18:17:10.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:10 smithi171 ceph-mon[129241]: scrub summary: active paths [/] 2023-07-14T18:17:10.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:10 smithi171 ceph-mon[129241]: scrub summary: idle+waiting paths [/] 2023-07-14T18:17:10.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:10 smithi171 ceph-mon[129241]: scrub summary: idle+waiting paths [/] 2023-07-14T18:17:10.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:10 smithi133 ceph-mon[126384]: scrub queued for path: / 2023-07-14T18:17:10.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:10 smithi133 ceph-mon[126384]: scrub summary: idle+waiting paths [/] 2023-07-14T18:17:10.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:10 smithi133 ceph-mon[126384]: scrub summary: active paths [/] 2023-07-14T18:17:10.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:10 smithi133 ceph-mon[126384]: scrub summary: active paths [/] 2023-07-14T18:17:10.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:10 smithi133 ceph-mon[126384]: scrub summary: active paths [/] 2023-07-14T18:17:10.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:10 smithi133 ceph-mon[126384]: scrub summary: idle+waiting paths [/] 2023-07-14T18:17:10.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:10 smithi133 ceph-mon[126384]: scrub summary: idle+waiting paths [/] 2023-07-14T18:17:10.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:10 smithi133 ceph-mon[126384]: scrub summary: active paths [/] 2023-07-14T18:17:10.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:10 smithi133 ceph-mon[126384]: scrub summary: active paths [/] 2023-07-14T18:17:10.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:10 smithi133 ceph-mon[126384]: scrub summary: idle+waiting paths [/] 2023-07-14T18:17:10.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:10 smithi133 ceph-mon[126384]: scrub summary: active paths [/] 2023-07-14T18:17:10.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:10 smithi133 ceph-mon[126384]: scrub summary: idle+waiting paths [/] 2023-07-14T18:17:10.410 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:10 smithi133 ceph-mon[126384]: scrub summary: idle+waiting paths [/] 2023-07-14T18:17:10.911 INFO:teuthology.orchestra.run.smithi133.stderr:10000000 of 50000000 tuples (20%) done (elapsed 26.70 s, remaining 106.79 s) 2023-07-14T18:17:11.039 INFO:teuthology.orchestra.run.smithi133.stderr:10100000 of 50000000 tuples (20%) done (elapsed 26.82 s, remaining 105.97 s) 2023-07-14T18:17:11.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:11 smithi161 ceph-mon[128993]: pgmap v363: 129 pgs: 129 active+clean; 1.6 GiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 MiB/s wr, 64 op/s 2023-07-14T18:17:11.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:11 smithi171 ceph-mon[129241]: pgmap v363: 129 pgs: 129 active+clean; 1.6 GiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 MiB/s wr, 64 op/s 2023-07-14T18:17:11.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:11 smithi133 ceph-mon[126384]: pgmap v363: 129 pgs: 129 active+clean; 1.6 GiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 MiB/s wr, 64 op/s 2023-07-14T18:17:11.659 INFO:teuthology.orchestra.run.smithi133.stderr:10200000 of 50000000 tuples (20%) done (elapsed 27.45 s, remaining 107.09 s) 2023-07-14T18:17:11.870 INFO:teuthology.orchestra.run.smithi133.stderr:10300000 of 50000000 tuples (20%) done (elapsed 27.66 s, remaining 106.60 s) 2023-07-14T18:17:12.038 INFO:teuthology.orchestra.run.smithi133.stderr:10400000 of 50000000 tuples (20%) done (elapsed 27.82 s, remaining 105.95 s) 2023-07-14T18:17:12.252 INFO:teuthology.orchestra.run.smithi133.stderr:10500000 of 50000000 tuples (21%) done (elapsed 28.04 s, remaining 105.48 s) 2023-07-14T18:17:12.490 INFO:teuthology.orchestra.run.smithi133.stderr:10600000 of 50000000 tuples (21%) done (elapsed 28.28 s, remaining 105.10 s) 2023-07-14T18:17:12.599 INFO:teuthology.orchestra.run.smithi133.stderr:10700000 of 50000000 tuples (21%) done (elapsed 28.39 s, remaining 104.26 s) 2023-07-14T18:17:12.785 INFO:teuthology.orchestra.run.smithi133.stderr:10800000 of 50000000 tuples (21%) done (elapsed 28.57 s, remaining 103.70 s) 2023-07-14T18:17:12.947 INFO:teuthology.orchestra.run.smithi133.stderr:10900000 of 50000000 tuples (21%) done (elapsed 28.73 s, remaining 103.07 s) 2023-07-14T18:17:13.089 INFO:teuthology.orchestra.run.smithi133.stderr:11000000 of 50000000 tuples (22%) done (elapsed 28.88 s, remaining 102.38 s) 2023-07-14T18:17:13.312 INFO:teuthology.orchestra.run.smithi133.stderr:11100000 of 50000000 tuples (22%) done (elapsed 29.10 s, remaining 101.97 s) 2023-07-14T18:17:13.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:13 smithi161 ceph-mon[128993]: pgmap v364: 129 pgs: 129 active+clean; 1.9 GiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 147 MiB/s wr, 67 op/s 2023-07-14T18:17:13.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:13 smithi171 ceph-mon[129241]: pgmap v364: 129 pgs: 129 active+clean; 1.9 GiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 147 MiB/s wr, 67 op/s 2023-07-14T18:17:13.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:13 smithi133 ceph-mon[126384]: pgmap v364: 129 pgs: 129 active+clean; 1.9 GiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 147 MiB/s wr, 67 op/s 2023-07-14T18:17:13.475 INFO:teuthology.orchestra.run.smithi133.stderr:11200000 of 50000000 tuples (22%) done (elapsed 29.26 s, remaining 101.37 s) 2023-07-14T18:17:13.662 INFO:teuthology.orchestra.run.smithi133.stderr:11300000 of 50000000 tuples (22%) done (elapsed 29.45 s, remaining 100.85 s) 2023-07-14T18:17:13.858 INFO:teuthology.orchestra.run.smithi133.stderr:11400000 of 50000000 tuples (22%) done (elapsed 29.64 s, remaining 100.38 s) 2023-07-14T18:17:13.984 INFO:teuthology.orchestra.run.smithi133.stderr:11500000 of 50000000 tuples (23%) done (elapsed 29.77 s, remaining 99.67 s) 2023-07-14T18:17:14.160 INFO:teuthology.orchestra.run.smithi133.stderr:11600000 of 50000000 tuples (23%) done (elapsed 29.95 s, remaining 99.13 s) 2023-07-14T18:17:14.364 INFO:teuthology.orchestra.run.smithi133.stderr:11700000 of 50000000 tuples (23%) done (elapsed 30.15 s, remaining 98.70 s) 2023-07-14T18:17:14.551 INFO:teuthology.orchestra.run.smithi133.stderr:11800000 of 50000000 tuples (23%) done (elapsed 30.34 s, remaining 98.21 s) 2023-07-14T18:17:14.680 INFO:teuthology.orchestra.run.smithi133.stderr:11900000 of 50000000 tuples (23%) done (elapsed 30.47 s, remaining 97.54 s) 2023-07-14T18:17:14.836 INFO:teuthology.orchestra.run.smithi133.stderr:12000000 of 50000000 tuples (24%) done (elapsed 30.62 s, remaining 96.97 s) 2023-07-14T18:17:15.046 INFO:teuthology.orchestra.run.smithi133.stderr:12100000 of 50000000 tuples (24%) done (elapsed 30.83 s, remaining 96.57 s) 2023-07-14T18:17:15.339 INFO:teuthology.orchestra.run.smithi133.stderr:12200000 of 50000000 tuples (24%) done (elapsed 31.13 s, remaining 96.44 s) 2023-07-14T18:17:15.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:15 smithi161 ceph-mon[128993]: pgmap v365: 129 pgs: 129 active+clean; 2.1 GiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 139 MiB/s wr, 74 op/s 2023-07-14T18:17:15.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:15 smithi171 ceph-mon[129241]: pgmap v365: 129 pgs: 129 active+clean; 2.1 GiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 139 MiB/s wr, 74 op/s 2023-07-14T18:17:15.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:15 smithi133 ceph-mon[126384]: pgmap v365: 129 pgs: 129 active+clean; 2.1 GiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 139 MiB/s wr, 74 op/s 2023-07-14T18:17:16.004 INFO:teuthology.orchestra.run.smithi133.stderr:12300000 of 50000000 tuples (24%) done (elapsed 31.79 s, remaining 97.44 s) 2023-07-14T18:17:16.140 INFO:teuthology.orchestra.run.smithi133.stderr:12400000 of 50000000 tuples (24%) done (elapsed 31.93 s, remaining 96.81 s) 2023-07-14T18:17:16.358 INFO:teuthology.orchestra.run.smithi133.stderr:12500000 of 50000000 tuples (25%) done (elapsed 32.14 s, remaining 96.43 s) 2023-07-14T18:17:16.556 INFO:teuthology.orchestra.run.smithi133.stderr:12600000 of 50000000 tuples (25%) done (elapsed 32.34 s, remaining 96.00 s) 2023-07-14T18:17:16.706 INFO:teuthology.orchestra.run.smithi133.stderr:12700000 of 50000000 tuples (25%) done (elapsed 32.49 s, remaining 95.43 s) 2023-07-14T18:17:16.936 INFO:teuthology.orchestra.run.smithi133.stderr:12800000 of 50000000 tuples (25%) done (elapsed 32.72 s, remaining 95.10 s) 2023-07-14T18:17:17.076 INFO:teuthology.orchestra.run.smithi133.stderr:12900000 of 50000000 tuples (25%) done (elapsed 32.86 s, remaining 94.51 s) 2023-07-14T18:17:17.266 INFO:teuthology.orchestra.run.smithi133.stderr:13000000 of 50000000 tuples (26%) done (elapsed 33.05 s, remaining 94.07 s) 2023-07-14T18:17:17.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:17 smithi161 ceph-mon[128993]: pgmap v366: 129 pgs: 129 active+clean; 2.2 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 105 MiB/s wr, 55 op/s 2023-07-14T18:17:17.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:17 smithi171 ceph-mon[129241]: pgmap v366: 129 pgs: 129 active+clean; 2.2 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 105 MiB/s wr, 55 op/s 2023-07-14T18:17:17.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:17 smithi133 ceph-mon[126384]: pgmap v366: 129 pgs: 129 active+clean; 2.2 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 105 MiB/s wr, 55 op/s 2023-07-14T18:17:17.419 INFO:teuthology.orchestra.run.smithi133.stderr:13100000 of 50000000 tuples (26%) done (elapsed 33.21 s, remaining 93.53 s) 2023-07-14T18:17:17.594 INFO:teuthology.orchestra.run.smithi133.stderr:13200000 of 50000000 tuples (26%) done (elapsed 33.38 s, remaining 93.06 s) 2023-07-14T18:17:17.716 INFO:teuthology.orchestra.run.smithi133.stderr:13300000 of 50000000 tuples (26%) done (elapsed 33.50 s, remaining 92.44 s) 2023-07-14T18:17:17.896 INFO:teuthology.orchestra.run.smithi133.stderr:13400000 of 50000000 tuples (26%) done (elapsed 33.68 s, remaining 92.00 s) 2023-07-14T18:17:18.037 INFO:teuthology.orchestra.run.smithi133.stderr:13500000 of 50000000 tuples (27%) done (elapsed 33.82 s, remaining 91.45 s) 2023-07-14T18:17:18.251 INFO:teuthology.orchestra.run.smithi133.stderr:13600000 of 50000000 tuples (27%) done (elapsed 34.04 s, remaining 91.10 s) 2023-07-14T18:17:18.429 INFO:teuthology.orchestra.run.smithi133.stderr:13700000 of 50000000 tuples (27%) done (elapsed 34.22 s, remaining 90.66 s) 2023-07-14T18:17:18.765 INFO:teuthology.orchestra.run.smithi133.stderr:13800000 of 50000000 tuples (27%) done (elapsed 34.55 s, remaining 90.63 s) 2023-07-14T18:17:19.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:19 smithi161 ceph-mon[128993]: pgmap v367: 129 pgs: 129 active+clean; 2.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 120 MiB/s wr, 87 op/s 2023-07-14T18:17:19.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:19 smithi171 ceph-mon[129241]: pgmap v367: 129 pgs: 129 active+clean; 2.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 120 MiB/s wr, 87 op/s 2023-07-14T18:17:19.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:19 smithi133 ceph-mon[126384]: pgmap v367: 129 pgs: 129 active+clean; 2.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 120 MiB/s wr, 87 op/s 2023-07-14T18:17:19.816 INFO:teuthology.orchestra.run.smithi133.stderr:13900000 of 50000000 tuples (27%) done (elapsed 35.60 s, remaining 92.46 s) 2023-07-14T18:17:19.965 INFO:teuthology.orchestra.run.smithi133.stderr:14000000 of 50000000 tuples (28%) done (elapsed 35.75 s, remaining 91.93 s) 2023-07-14T18:17:20.187 INFO:teuthology.orchestra.run.smithi133.stderr:14100000 of 50000000 tuples (28%) done (elapsed 35.97 s, remaining 91.59 s) 2023-07-14T18:17:20.363 INFO:teuthology.orchestra.run.smithi133.stderr:14200000 of 50000000 tuples (28%) done (elapsed 36.15 s, remaining 91.14 s) 2023-07-14T18:17:20.482 INFO:teuthology.orchestra.run.smithi133.stderr:14300000 of 50000000 tuples (28%) done (elapsed 36.27 s, remaining 90.54 s) 2023-07-14T18:17:20.642 INFO:teuthology.orchestra.run.smithi133.stderr:14400000 of 50000000 tuples (28%) done (elapsed 36.43 s, remaining 90.06 s) 2023-07-14T18:17:20.833 INFO:teuthology.orchestra.run.smithi133.stderr:14500000 of 50000000 tuples (29%) done (elapsed 36.62 s, remaining 89.65 s) 2023-07-14T18:17:21.037 INFO:teuthology.orchestra.run.smithi133.stderr:14600000 of 50000000 tuples (29%) done (elapsed 36.82 s, remaining 89.28 s) 2023-07-14T18:17:21.173 INFO:teuthology.orchestra.run.smithi133.stderr:14700000 of 50000000 tuples (29%) done (elapsed 36.96 s, remaining 88.75 s) 2023-07-14T18:17:21.345 INFO:teuthology.orchestra.run.smithi133.stderr:14800000 of 50000000 tuples (29%) done (elapsed 37.13 s, remaining 88.31 s) 2023-07-14T18:17:21.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:21 smithi161 ceph-mon[128993]: pgmap v368: 129 pgs: 129 active+clean; 2.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 104 MiB/s wr, 81 op/s 2023-07-14T18:17:21.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:21 smithi171 ceph-mon[129241]: pgmap v368: 129 pgs: 129 active+clean; 2.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 104 MiB/s wr, 81 op/s 2023-07-14T18:17:21.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:21 smithi133 ceph-mon[126384]: pgmap v368: 129 pgs: 129 active+clean; 2.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 104 MiB/s wr, 81 op/s 2023-07-14T18:17:21.606 INFO:teuthology.orchestra.run.smithi133.stderr:14900000 of 50000000 tuples (29%) done (elapsed 37.39 s, remaining 88.09 s) 2023-07-14T18:17:21.782 INFO:teuthology.orchestra.run.smithi133.stderr:15000000 of 50000000 tuples (30%) done (elapsed 37.57 s, remaining 87.66 s) 2023-07-14T18:17:21.934 INFO:teuthology.orchestra.run.smithi133.stderr:15100000 of 50000000 tuples (30%) done (elapsed 37.72 s, remaining 87.18 s) 2023-07-14T18:17:22.115 INFO:teuthology.orchestra.run.smithi133.stderr:15200000 of 50000000 tuples (30%) done (elapsed 37.90 s, remaining 86.77 s) 2023-07-14T18:17:22.304 INFO:teuthology.orchestra.run.smithi133.stderr:15300000 of 50000000 tuples (30%) done (elapsed 38.09 s, remaining 86.39 s) 2023-07-14T18:17:22.522 INFO:teuthology.orchestra.run.smithi133.stderr:15400000 of 50000000 tuples (30%) done (elapsed 38.31 s, remaining 86.07 s) 2023-07-14T18:17:22.670 INFO:teuthology.orchestra.run.smithi133.stderr:15500000 of 50000000 tuples (31%) done (elapsed 38.46 s, remaining 85.60 s) 2023-07-14T18:17:22.833 INFO:teuthology.orchestra.run.smithi133.stderr:15600000 of 50000000 tuples (31%) done (elapsed 38.62 s, remaining 85.16 s) 2023-07-14T18:17:23.006 INFO:teuthology.orchestra.run.smithi133.stderr:15700000 of 50000000 tuples (31%) done (elapsed 38.79 s, remaining 84.75 s) 2023-07-14T18:17:23.189 INFO:teuthology.orchestra.run.smithi133.stderr:15800000 of 50000000 tuples (31%) done (elapsed 38.97 s, remaining 84.36 s) 2023-07-14T18:17:23.343 INFO:teuthology.orchestra.run.smithi133.stderr:15900000 of 50000000 tuples (31%) done (elapsed 39.13 s, remaining 83.92 s) 2023-07-14T18:17:23.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:23 smithi161 ceph-mon[128993]: pgmap v369: 129 pgs: 129 active+clean; 2.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 118 MiB/s wr, 84 op/s 2023-07-14T18:17:23.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:23 smithi171 ceph-mon[129241]: pgmap v369: 129 pgs: 129 active+clean; 2.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 118 MiB/s wr, 84 op/s 2023-07-14T18:17:23.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:23 smithi133 ceph-mon[126384]: pgmap v369: 129 pgs: 129 active+clean; 2.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 118 MiB/s wr, 84 op/s 2023-07-14T18:17:23.463 INFO:teuthology.orchestra.run.smithi133.stderr:16000000 of 50000000 tuples (32%) done (elapsed 39.25 s, remaining 83.41 s) 2023-07-14T18:17:23.657 INFO:teuthology.orchestra.run.smithi133.stderr:16100000 of 50000000 tuples (32%) done (elapsed 39.44 s, remaining 83.05 s) 2023-07-14T18:17:23.817 INFO:teuthology.orchestra.run.smithi133.stderr:16200000 of 50000000 tuples (32%) done (elapsed 39.60 s, remaining 82.63 s) 2023-07-14T18:17:23.952 INFO:teuthology.orchestra.run.smithi133.stderr:16300000 of 50000000 tuples (32%) done (elapsed 39.74 s, remaining 82.16 s) 2023-07-14T18:17:24.135 INFO:teuthology.orchestra.run.smithi133.stderr:16400000 of 50000000 tuples (32%) done (elapsed 39.92 s, remaining 81.79 s) 2023-07-14T18:17:24.333 INFO:teuthology.orchestra.run.smithi133.stderr:16500000 of 50000000 tuples (33%) done (elapsed 40.12 s, remaining 81.45 s) 2023-07-14T18:17:24.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:24 smithi161 ceph-mon[128993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:17:24.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:24 smithi161 ceph-mon[128993]: pgmap v370: 129 pgs: 129 active+clean; 2.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 125 MiB/s wr, 97 op/s 2023-07-14T18:17:24.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:24 smithi171 ceph-mon[129241]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:17:24.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:24 smithi171 ceph-mon[129241]: pgmap v370: 129 pgs: 129 active+clean; 2.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 125 MiB/s wr, 97 op/s 2023-07-14T18:17:24.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:24 smithi133 ceph-mon[126384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:17:24.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:24 smithi133 ceph-mon[126384]: pgmap v370: 129 pgs: 129 active+clean; 2.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 125 MiB/s wr, 97 op/s 2023-07-14T18:17:24.466 INFO:teuthology.orchestra.run.smithi133.stderr:16600000 of 50000000 tuples (33%) done (elapsed 40.25 s, remaining 80.99 s) 2023-07-14T18:17:24.689 INFO:teuthology.orchestra.run.smithi133.stderr:16700000 of 50000000 tuples (33%) done (elapsed 40.48 s, remaining 80.71 s) 2023-07-14T18:17:24.858 INFO:teuthology.orchestra.run.smithi133.stderr:16800000 of 50000000 tuples (33%) done (elapsed 40.64 s, remaining 80.32 s) 2023-07-14T18:17:24.997 INFO:teuthology.orchestra.run.smithi133.stderr:16900000 of 50000000 tuples (33%) done (elapsed 40.78 s, remaining 79.88 s) 2023-07-14T18:17:25.152 INFO:teuthology.orchestra.run.smithi133.stderr:17000000 of 50000000 tuples (34%) done (elapsed 40.94 s, remaining 79.47 s) 2023-07-14T18:17:25.345 INFO:teuthology.orchestra.run.smithi133.stderr:17100000 of 50000000 tuples (34%) done (elapsed 41.13 s, remaining 79.14 s) 2023-07-14T18:17:25.482 INFO:teuthology.orchestra.run.smithi133.stderr:17200000 of 50000000 tuples (34%) done (elapsed 41.27 s, remaining 78.70 s) 2023-07-14T18:17:25.655 INFO:teuthology.orchestra.run.smithi133.stderr:17300000 of 50000000 tuples (34%) done (elapsed 41.44 s, remaining 78.33 s) 2023-07-14T18:17:25.854 INFO:teuthology.orchestra.run.smithi133.stderr:17400000 of 50000000 tuples (34%) done (elapsed 41.64 s, remaining 78.02 s) 2023-07-14T18:17:26.016 INFO:teuthology.orchestra.run.smithi133.stderr:17500000 of 50000000 tuples (35%) done (elapsed 41.80 s, remaining 77.63 s) 2023-07-14T18:17:26.180 INFO:teuthology.orchestra.run.smithi133.stderr:17600000 of 50000000 tuples (35%) done (elapsed 41.97 s, remaining 77.26 s) 2023-07-14T18:17:26.896 INFO:teuthology.orchestra.run.smithi133.stderr:17700000 of 50000000 tuples (35%) done (elapsed 42.68 s, remaining 77.89 s) 2023-07-14T18:17:27.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:26 smithi161 ceph-mon[128993]: pgmap v371: 129 pgs: 129 active+clean; 2.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 96 MiB/s wr, 78 op/s 2023-07-14T18:17:27.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:26 smithi171 ceph-mon[129241]: pgmap v371: 129 pgs: 129 active+clean; 2.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 96 MiB/s wr, 78 op/s 2023-07-14T18:17:27.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:26 smithi133 ceph-mon[126384]: pgmap v371: 129 pgs: 129 active+clean; 2.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 96 MiB/s wr, 78 op/s 2023-07-14T18:17:27.706 INFO:teuthology.orchestra.run.smithi133.stderr:17800000 of 50000000 tuples (35%) done (elapsed 43.49 s, remaining 78.68 s) 2023-07-14T18:17:27.829 INFO:teuthology.orchestra.run.smithi133.stderr:17900000 of 50000000 tuples (35%) done (elapsed 43.62 s, remaining 78.22 s) 2023-07-14T18:17:28.026 INFO:teuthology.orchestra.run.smithi133.stderr:18000000 of 50000000 tuples (36%) done (elapsed 43.81 s, remaining 77.89 s) 2023-07-14T18:17:28.215 INFO:teuthology.orchestra.run.smithi133.stderr:18100000 of 50000000 tuples (36%) done (elapsed 44.00 s, remaining 77.55 s) 2023-07-14T18:17:28.404 INFO:teuthology.orchestra.run.smithi133.stderr:18200000 of 50000000 tuples (36%) done (elapsed 44.19 s, remaining 77.21 s) 2023-07-14T18:17:28.536 INFO:teuthology.orchestra.run.smithi133.stderr:18300000 of 50000000 tuples (36%) done (elapsed 44.32 s, remaining 76.78 s) 2023-07-14T18:17:28.687 INFO:teuthology.orchestra.run.smithi133.stderr:18400000 of 50000000 tuples (36%) done (elapsed 44.47 s, remaining 76.38 s) 2023-07-14T18:17:28.901 INFO:teuthology.orchestra.run.smithi133.stderr:18500000 of 50000000 tuples (37%) done (elapsed 44.69 s, remaining 76.09 s) 2023-07-14T18:17:29.053 INFO:teuthology.orchestra.run.smithi133.stderr:18600000 of 50000000 tuples (37%) done (elapsed 44.84 s, remaining 75.70 s) 2023-07-14T18:17:29.232 INFO:teuthology.orchestra.run.smithi133.stderr:18700000 of 50000000 tuples (37%) done (elapsed 45.02 s, remaining 75.35 s) 2023-07-14T18:17:29.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:28 smithi161 ceph-mon[128993]: pgmap v372: 129 pgs: 129 active+clean; 2.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 112 MiB/s wr, 87 op/s 2023-07-14T18:17:29.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:28 smithi161 ceph-mon[128993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:17:29.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:28 smithi171 ceph-mon[129241]: pgmap v372: 129 pgs: 129 active+clean; 2.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 112 MiB/s wr, 87 op/s 2023-07-14T18:17:29.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:28 smithi171 ceph-mon[129241]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:17:29.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:28 smithi133 ceph-mon[126384]: pgmap v372: 129 pgs: 129 active+clean; 2.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 112 MiB/s wr, 87 op/s 2023-07-14T18:17:29.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:28 smithi133 ceph-mon[126384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:17:29.424 INFO:teuthology.orchestra.run.smithi133.stderr:18800000 of 50000000 tuples (37%) done (elapsed 45.21 s, remaining 75.03 s) 2023-07-14T18:17:29.619 INFO:teuthology.orchestra.run.smithi133.stderr:18900000 of 50000000 tuples (37%) done (elapsed 45.40 s, remaining 74.71 s) 2023-07-14T18:17:29.726 INFO:teuthology.orchestra.run.smithi133.stderr:19000000 of 50000000 tuples (38%) done (elapsed 45.51 s, remaining 74.26 s) 2023-07-14T18:17:29.954 INFO:teuthology.orchestra.run.smithi133.stderr:19100000 of 50000000 tuples (38%) done (elapsed 45.74 s, remaining 74.00 s) 2023-07-14T18:17:30.124 INFO:teuthology.orchestra.run.smithi133.stderr:19200000 of 50000000 tuples (38%) done (elapsed 45.91 s, remaining 73.65 s) 2023-07-14T18:17:30.332 INFO:teuthology.orchestra.run.smithi133.stderr:19300000 of 50000000 tuples (38%) done (elapsed 46.12 s, remaining 73.36 s) 2023-07-14T18:17:30.473 INFO:teuthology.orchestra.run.smithi133.stderr:19400000 of 50000000 tuples (38%) done (elapsed 46.26 s, remaining 72.97 s) 2023-07-14T18:17:30.662 INFO:teuthology.orchestra.run.smithi133.stderr:19500000 of 50000000 tuples (39%) done (elapsed 46.45 s, remaining 72.65 s) 2023-07-14T18:17:30.857 INFO:teuthology.orchestra.run.smithi133.stderr:19600000 of 50000000 tuples (39%) done (elapsed 46.64 s, remaining 72.34 s) 2023-07-14T18:17:31.026 INFO:teuthology.orchestra.run.smithi133.stderr:19700000 of 50000000 tuples (39%) done (elapsed 46.81 s, remaining 72.00 s) 2023-07-14T18:17:31.266 INFO:teuthology.orchestra.run.smithi133.stderr:19800000 of 50000000 tuples (39%) done (elapsed 47.05 s, remaining 71.77 s) 2023-07-14T18:17:31.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:30 smithi161 ceph-mon[128993]: pgmap v373: 129 pgs: 129 active+clean; 2.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 106 MiB/s wr, 59 op/s 2023-07-14T18:17:31.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:30 smithi171 ceph-mon[129241]: pgmap v373: 129 pgs: 129 active+clean; 2.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 106 MiB/s wr, 59 op/s 2023-07-14T18:17:31.375 INFO:teuthology.orchestra.run.smithi133.stderr:19900000 of 50000000 tuples (39%) done (elapsed 47.16 s, remaining 71.33 s) 2023-07-14T18:17:31.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:30 smithi133 ceph-mon[126384]: pgmap v373: 129 pgs: 129 active+clean; 2.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 106 MiB/s wr, 59 op/s 2023-07-14T18:17:31.551 INFO:teuthology.orchestra.run.smithi133.stderr:20000000 of 50000000 tuples (40%) done (elapsed 47.34 s, remaining 71.01 s) 2023-07-14T18:17:31.731 INFO:teuthology.orchestra.run.smithi133.stderr:20100000 of 50000000 tuples (40%) done (elapsed 47.52 s, remaining 70.68 s) 2023-07-14T18:17:31.868 INFO:teuthology.orchestra.run.smithi133.stderr:20200000 of 50000000 tuples (40%) done (elapsed 47.65 s, remaining 70.30 s) 2023-07-14T18:17:32.075 INFO:teuthology.orchestra.run.smithi133.stderr:20300000 of 50000000 tuples (40%) done (elapsed 47.86 s, remaining 70.02 s) 2023-07-14T18:17:32.241 INFO:teuthology.orchestra.run.smithi133.stderr:20400000 of 50000000 tuples (40%) done (elapsed 48.03 s, remaining 69.69 s) 2023-07-14T18:17:32.401 INFO:teuthology.orchestra.run.smithi133.stderr:20500000 of 50000000 tuples (41%) done (elapsed 48.19 s, remaining 69.34 s) 2023-07-14T18:17:32.541 INFO:teuthology.orchestra.run.smithi133.stderr:20600000 of 50000000 tuples (41%) done (elapsed 48.33 s, remaining 68.97 s) 2023-07-14T18:17:32.736 INFO:teuthology.orchestra.run.smithi133.stderr:20700000 of 50000000 tuples (41%) done (elapsed 48.52 s, remaining 68.68 s) 2023-07-14T18:17:32.926 INFO:teuthology.orchestra.run.smithi133.stderr:20800000 of 50000000 tuples (41%) done (elapsed 48.71 s, remaining 68.38 s) 2023-07-14T18:17:33.095 INFO:teuthology.orchestra.run.smithi133.stderr:20900000 of 50000000 tuples (41%) done (elapsed 48.88 s, remaining 68.06 s) 2023-07-14T18:17:33.218 INFO:teuthology.orchestra.run.smithi133.stderr:21000000 of 50000000 tuples (42%) done (elapsed 49.00 s, remaining 67.67 s) 2023-07-14T18:17:33.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:32 smithi161 ceph-mon[128993]: pgmap v374: 129 pgs: 129 active+clean; 3.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 128 MiB/s wr, 55 op/s 2023-07-14T18:17:33.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:32 smithi171 ceph-mon[129241]: pgmap v374: 129 pgs: 129 active+clean; 3.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 128 MiB/s wr, 55 op/s 2023-07-14T18:17:33.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:32 smithi133 ceph-mon[126384]: pgmap v374: 129 pgs: 129 active+clean; 3.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 128 MiB/s wr, 55 op/s 2023-07-14T18:17:33.416 INFO:teuthology.orchestra.run.smithi133.stderr:21100000 of 50000000 tuples (42%) done (elapsed 49.20 s, remaining 67.39 s) 2023-07-14T18:17:33.581 INFO:teuthology.orchestra.run.smithi133.stderr:21200000 of 50000000 tuples (42%) done (elapsed 49.37 s, remaining 67.06 s) 2023-07-14T18:17:33.718 INFO:teuthology.orchestra.run.smithi133.stderr:21300000 of 50000000 tuples (42%) done (elapsed 49.50 s, remaining 66.70 s) 2023-07-14T18:17:33.911 INFO:teuthology.orchestra.run.smithi133.stderr:21400000 of 50000000 tuples (42%) done (elapsed 49.70 s, remaining 66.42 s) 2023-07-14T18:17:34.079 INFO:teuthology.orchestra.run.smithi133.stderr:21500000 of 50000000 tuples (43%) done (elapsed 49.87 s, remaining 66.10 s) 2023-07-14T18:17:34.561 INFO:teuthology.orchestra.run.smithi133.stderr:21600000 of 50000000 tuples (43%) done (elapsed 50.35 s, remaining 66.20 s) 2023-07-14T18:17:35.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:34 smithi161 ceph-mon[128993]: pgmap v375: 129 pgs: 129 active+clean; 3.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 121 MiB/s wr, 64 op/s 2023-07-14T18:17:35.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:34 smithi171 ceph-mon[129241]: pgmap v375: 129 pgs: 129 active+clean; 3.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 121 MiB/s wr, 64 op/s 2023-07-14T18:17:35.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:34 smithi133 ceph-mon[126384]: pgmap v375: 129 pgs: 129 active+clean; 3.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 121 MiB/s wr, 64 op/s 2023-07-14T18:17:35.448 INFO:teuthology.orchestra.run.smithi133.stderr:21700000 of 50000000 tuples (43%) done (elapsed 51.23 s, remaining 66.82 s) 2023-07-14T18:17:35.562 INFO:teuthology.orchestra.run.smithi133.stderr:21800000 of 50000000 tuples (43%) done (elapsed 51.35 s, remaining 66.42 s) 2023-07-14T18:17:35.760 INFO:teuthology.orchestra.run.smithi133.stderr:21900000 of 50000000 tuples (43%) done (elapsed 51.55 s, remaining 66.14 s) 2023-07-14T18:17:35.961 INFO:teuthology.orchestra.run.smithi133.stderr:22000000 of 50000000 tuples (44%) done (elapsed 51.75 s, remaining 65.86 s) 2023-07-14T18:17:36.148 INFO:teuthology.orchestra.run.smithi133.stderr:22100000 of 50000000 tuples (44%) done (elapsed 51.93 s, remaining 65.56 s) 2023-07-14T18:17:36.283 INFO:teuthology.orchestra.run.smithi133.stderr:22200000 of 50000000 tuples (44%) done (elapsed 52.07 s, remaining 65.20 s) 2023-07-14T18:17:36.449 INFO:teuthology.orchestra.run.smithi133.stderr:22300000 of 50000000 tuples (44%) done (elapsed 52.24 s, remaining 64.88 s) 2023-07-14T18:17:36.591 INFO:teuthology.orchestra.run.smithi133.stderr:22400000 of 50000000 tuples (44%) done (elapsed 52.38 s, remaining 64.54 s) 2023-07-14T18:17:36.775 INFO:teuthology.orchestra.run.smithi133.stderr:22500000 of 50000000 tuples (45%) done (elapsed 52.56 s, remaining 64.24 s) 2023-07-14T18:17:36.949 INFO:teuthology.orchestra.run.smithi133.stderr:22600000 of 50000000 tuples (45%) done (elapsed 52.73 s, remaining 63.93 s) 2023-07-14T18:17:37.095 INFO:teuthology.orchestra.run.smithi133.stderr:22700000 of 50000000 tuples (45%) done (elapsed 52.88 s, remaining 63.60 s) 2023-07-14T18:17:37.328 INFO:teuthology.orchestra.run.smithi133.stderr:22800000 of 50000000 tuples (45%) done (elapsed 53.11 s, remaining 63.36 s) 2023-07-14T18:17:37.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:36 smithi161 ceph-mon[128993]: pgmap v376: 129 pgs: 129 active+clean; 3.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 90 MiB/s wr, 44 op/s 2023-07-14T18:17:37.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:36 smithi171 ceph-mon[129241]: pgmap v376: 129 pgs: 129 active+clean; 3.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 90 MiB/s wr, 44 op/s 2023-07-14T18:17:37.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:36 smithi133 ceph-mon[126384]: pgmap v376: 129 pgs: 129 active+clean; 3.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 90 MiB/s wr, 44 op/s 2023-07-14T18:17:37.485 INFO:teuthology.orchestra.run.smithi133.stderr:22900000 of 50000000 tuples (45%) done (elapsed 53.27 s, remaining 63.04 s) 2023-07-14T18:17:37.682 INFO:teuthology.orchestra.run.smithi133.stderr:23000000 of 50000000 tuples (46%) done (elapsed 53.47 s, remaining 62.77 s) 2023-07-14T18:17:37.804 INFO:teuthology.orchestra.run.smithi133.stderr:23100000 of 50000000 tuples (46%) done (elapsed 53.59 s, remaining 62.41 s) 2023-07-14T18:17:38.027 INFO:teuthology.orchestra.run.smithi133.stderr:23200000 of 50000000 tuples (46%) done (elapsed 53.81 s, remaining 62.16 s) 2023-07-14T18:17:38.214 INFO:teuthology.orchestra.run.smithi133.stderr:23300000 of 50000000 tuples (46%) done (elapsed 54.00 s, remaining 61.88 s) 2023-07-14T18:17:38.389 INFO:teuthology.orchestra.run.smithi133.stderr:23400000 of 50000000 tuples (46%) done (elapsed 54.18 s, remaining 61.58 s) 2023-07-14T18:17:38.566 INFO:teuthology.orchestra.run.smithi133.stderr:23500000 of 50000000 tuples (47%) done (elapsed 54.35 s, remaining 61.29 s) 2023-07-14T18:17:38.789 INFO:teuthology.orchestra.run.smithi133.stderr:23600000 of 50000000 tuples (47%) done (elapsed 54.57 s, remaining 61.05 s) 2023-07-14T18:17:38.960 INFO:teuthology.orchestra.run.smithi133.stderr:23700000 of 50000000 tuples (47%) done (elapsed 54.75 s, remaining 60.75 s) 2023-07-14T18:17:39.084 INFO:teuthology.orchestra.run.smithi133.stderr:23800000 of 50000000 tuples (47%) done (elapsed 54.87 s, remaining 60.40 s) 2023-07-14T18:17:39.299 INFO:teuthology.orchestra.run.smithi133.stderr:23900000 of 50000000 tuples (47%) done (elapsed 55.08 s, remaining 60.16 s) 2023-07-14T18:17:39.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:38 smithi161 ceph-mon[128993]: pgmap v377: 129 pgs: 129 active+clean; 3.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 134 MiB/s wr, 66 op/s 2023-07-14T18:17:39.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:38 smithi171 ceph-mon[129241]: pgmap v377: 129 pgs: 129 active+clean; 3.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 134 MiB/s wr, 66 op/s 2023-07-14T18:17:39.391 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:17:39.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:38 smithi133 ceph-mon[126384]: pgmap v377: 129 pgs: 129 active+clean; 3.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 134 MiB/s wr, 66 op/s 2023-07-14T18:17:39.480 INFO:teuthology.orchestra.run.smithi133.stderr:24000000 of 50000000 tuples (48%) done (elapsed 55.27 s, remaining 59.87 s) 2023-07-14T18:17:39.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.559+0000 7f957a1ff700 1 -- 172.21.15.133:0/1834470240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95740730a0 msgr2=0x7f9574073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:39.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.559+0000 7f957a1ff700 1 --2- 172.21.15.133:0/1834470240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95740730a0 0x7f9574073510 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9568009990 tx=0x7f9568009ca0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.560+0000 7f957a1ff700 1 -- 172.21.15.133:0/1834470240 shutdown_connections 2023-07-14T18:17:39.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.560+0000 7f957a1ff700 1 --2- 172.21.15.133:0/1834470240 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9574107ae0 0x7f9574109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.560+0000 7f957a1ff700 1 --2- 172.21.15.133:0/1834470240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9574074680 0x7f9574072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.560+0000 7f957a1ff700 1 --2- 172.21.15.133:0/1834470240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95740730a0 0x7f9574073510 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.560+0000 7f957a1ff700 1 -- 172.21.15.133:0/1834470240 >> 172.21.15.133:0/1834470240 conn(0x7f95740fb260 msgr2=0x7f95740fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:39.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.560+0000 7f957a1ff700 1 -- 172.21.15.133:0/1834470240 shutdown_connections 2023-07-14T18:17:39.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.560+0000 7f957a1ff700 1 -- 172.21.15.133:0/1834470240 wait complete. 2023-07-14T18:17:39.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.561+0000 7f957a1ff700 1 Processor -- start 2023-07-14T18:17:39.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.561+0000 7f957a1ff700 1 -- start start 2023-07-14T18:17:39.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.562+0000 7f957a1ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95740730a0 0x7f9574197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:39.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.562+0000 7f957a1ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9574074680 0x7f9574198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:39.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.562+0000 7f957a1ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9574107ae0 0x7f957419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:39.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.562+0000 7f957a1ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957419f760 con 0x7f95740730a0 2023-07-14T18:17:39.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.562+0000 7f957a1ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957419f8a0 con 0x7f9574107ae0 2023-07-14T18:17:39.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.562+0000 7f957a1ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957419fa30 con 0x7f9574074680 2023-07-14T18:17:39.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.562+0000 7f95737fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95740730a0 0x7f9574197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:39.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.562+0000 7f9572ffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9574074680 0x7f9574198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:39.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.562+0000 7f95737fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95740730a0 0x7f9574197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57628/0 (socket says 172.21.15.133:57628) 2023-07-14T18:17:39.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.562+0000 7f95737fe700 1 -- 172.21.15.133:0/1042783628 learned_addr learned my addr 172.21.15.133:0/1042783628 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:39.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.562+0000 7f9573fff700 1 --2- 172.21.15.133:0/1042783628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9574107ae0 0x7f957419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:39.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.563+0000 7f95737fe700 1 -- 172.21.15.133:0/1042783628 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9574074680 msgr2=0x7f9574198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:39.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.563+0000 7f95737fe700 1 --2- 172.21.15.133:0/1042783628 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9574074680 0x7f9574198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.563+0000 7f95737fe700 1 -- 172.21.15.133:0/1042783628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9574107ae0 msgr2=0x7f957419f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:39.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.563+0000 7f95737fe700 1 --2- 172.21.15.133:0/1042783628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9574107ae0 0x7f957419f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.563+0000 7f95737fe700 1 -- 172.21.15.133:0/1042783628 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95680095f0 con 0x7f95740730a0 2023-07-14T18:17:39.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.563+0000 7f95737fe700 1 --2- 172.21.15.133:0/1042783628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95740730a0 0x7f9574197be0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f956400e970 tx=0x7f956400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:39.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.563+0000 7f9570ff9700 1 -- 172.21.15.133:0/1042783628 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9564009790 con 0x7f95740730a0 2023-07-14T18:17:39.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.563+0000 7f957a1ff700 1 -- 172.21.15.133:0/1042783628 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f957419fd10 con 0x7f95740730a0 2023-07-14T18:17:39.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.563+0000 7f9570ff9700 1 -- 172.21.15.133:0/1042783628 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f956400c7e0 con 0x7f95740730a0 2023-07-14T18:17:39.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.563+0000 7f9570ff9700 1 -- 172.21.15.133:0/1042783628 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95640054e0 con 0x7f95740730a0 2023-07-14T18:17:39.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.563+0000 7f957a1ff700 1 -- 172.21.15.133:0/1042783628 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95741a01c0 con 0x7f95740730a0 2023-07-14T18:17:39.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.568+0000 7f9570ff9700 1 -- 172.21.15.133:0/1042783628 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f956400c950 con 0x7f95740730a0 2023-07-14T18:17:39.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.569+0000 7f958daffb80 1 Processor -- start 2023-07-14T18:17:39.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.569+0000 7f958daffb80 1 -- start start 2023-07-14T18:17:39.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.570+0000 7f958daffb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55880560ef60 0x55880560f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:39.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.570+0000 7f9570ff9700 1 --2- 172.21.15.133:0/1042783628 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f955c0a1150 0x7f955c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:39.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.570+0000 7f958daffb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55880560f8b0 0x55880560fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:39.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.570+0000 7f958daffb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588056104b0 0x558805610920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:39.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.570+0000 7f958daffb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558805513cb0 con 0x55880560f8b0 2023-07-14T18:17:39.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.570+0000 7f958daffb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558805566a40 con 0x55880560ef60 2023-07-14T18:17:39.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.570+0000 7f958daffb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880555fed0 con 0x5588056104b0 2023-07-14T18:17:39.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.570+0000 7f9570ff9700 1 -- 172.21.15.133:0/1042783628 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9564014070 con 0x7f95740730a0 2023-07-14T18:17:39.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.570+0000 7f9573fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55880560ef60 0x55880560f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:39.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.570+0000 7f9573fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55880560ef60 0x55880560f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:60362/0 (socket says 172.21.15.133:60362) 2023-07-14T18:17:39.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.570+0000 7f9573fff700 1 -- 172.21.15.133:0/3673275918 learned_addr learned my addr 172.21.15.133:0/3673275918 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:39.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.570+0000 7f9573fff700 1 --2- 172.21.15.133:0/3673275918 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588056104b0 0x558805610920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:39.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.570+0000 7f9572ffd700 1 --2- 172.21.15.133:0/3673275918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55880560f8b0 0x55880560fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:39.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.570+0000 7f9573fff700 1 -- 172.21.15.133:0/3673275918 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588056104b0 msgr2=0x558805610920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:39.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.570+0000 7f95737fe700 1 --2- 172.21.15.133:0/1042783628 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f955c0a1150 0x7f955c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:39.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.570+0000 7f9573fff700 1 --2- 172.21.15.133:0/3673275918 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588056104b0 0x558805610920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.570+0000 7f9573fff700 1 -- 172.21.15.133:0/3673275918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55880560f8b0 msgr2=0x55880560fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:39.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.570+0000 7f9573fff700 1 --2- 172.21.15.133:0/3673275918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55880560f8b0 0x55880560fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.570+0000 7f9573fff700 1 -- 172.21.15.133:0/3673275918 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55880538dde0 con 0x55880560ef60 2023-07-14T18:17:39.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.571+0000 7f9573fff700 1 --2- 172.21.15.133:0/3673275918 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55880560ef60 0x55880560f370 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f956c008cd0 tx=0x7f956c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1652979837733422 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:39.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.571+0000 7f95737fe700 1 --2- 172.21.15.133:0/1042783628 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f955c0a1150 0x7f955c0a3600 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9564005950 tx=0x7f9564004750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:39.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.571+0000 7f9559ffb700 1 -- 172.21.15.133:0/3673275918 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c01f070 con 0x55880560ef60 2023-07-14T18:17:39.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.571+0000 7f9559ffb700 1 -- 172.21.15.133:0/3673275918 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f956c00f040 con 0x55880560ef60 2023-07-14T18:17:39.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.571+0000 7f9559ffb700 1 -- 172.21.15.133:0/3673275918 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c00e050 con 0x55880560ef60 2023-07-14T18:17:39.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.572+0000 7f958daffb80 1 -- 172.21.15.133:0/3673275918 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55880560ef60 msgr2=0x55880560f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:39.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.572+0000 7f958daffb80 1 --2- 172.21.15.133:0/3673275918 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55880560ef60 0x55880560f370 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f956c008cd0 tx=0x7f956c009920 comp rx=0 tx=0).stop 2023-07-14T18:17:39.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.572+0000 7f958daffb80 1 -- 172.21.15.133:0/3673275918 shutdown_connections 2023-07-14T18:17:39.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.572+0000 7f958daffb80 1 --2- 172.21.15.133:0/3673275918 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55880560ef60 0x55880560f370 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.572+0000 7f958daffb80 1 --2- 172.21.15.133:0/3673275918 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588056104b0 0x558805610920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.572+0000 7f958daffb80 1 --2- 172.21.15.133:0/3673275918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55880560f8b0 0x55880560fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.572+0000 7f958daffb80 1 -- 172.21.15.133:0/3673275918 >> 172.21.15.133:0/3673275918 conn(0x558805604680 msgr2=0x5588056099d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:39.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.572+0000 7f958daffb80 1 -- 172.21.15.133:0/3673275918 shutdown_connections 2023-07-14T18:17:39.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.572+0000 7f958daffb80 1 -- 172.21.15.133:0/3673275918 wait complete. 2023-07-14T18:17:39.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.572+0000 7f958daffb80 1 -- start start 2023-07-14T18:17:39.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.573+0000 7f958daffb80 1 Processor -- start 2023-07-14T18:17:39.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.573+0000 7f958daffb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55880560ef60 0x5588056189d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:39.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.573+0000 7f958daffb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55880560f8b0 0x558805618f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:39.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.573+0000 7f958daffb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588056104b0 0x55880561ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:39.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.573+0000 7f958daffb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588054ffac0 con 0x5588056104b0 2023-07-14T18:17:39.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.573+0000 7f958daffb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588054ff7c0 con 0x55880560ef60 2023-07-14T18:17:39.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.573+0000 7f958daffb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588053e3920 con 0x55880560f8b0 2023-07-14T18:17:39.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.573+0000 7f958daffb80 10 client.0 _ll_register_callbacks cb 0x5588054fd760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:17:39.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.573+0000 7f9573fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55880560ef60 0x5588056189d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:39.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.573+0000 7f9572ffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55880560f8b0 0x558805618f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:39.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.573+0000 7f9572ffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55880560f8b0 0x558805618f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:47718/0 (socket says 172.21.15.133:47718) 2023-07-14T18:17:39.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.573+0000 7f9572ffd700 1 -- 172.21.15.133:0/3096588322 learned_addr learned my addr 172.21.15.133:0/3096588322 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:39.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.573+0000 7f9573fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55880560ef60 0x5588056189d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:60368/0 (socket says 172.21.15.133:60368) 2023-07-14T18:17:39.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.573+0000 7f9573fff700 1 --2- 172.21.15.133:0/3096588322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588056104b0 0x55880561ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:39.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.574+0000 7f9572ffd700 1 -- 172.21.15.133:0/3096588322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55880560ef60 msgr2=0x5588056189d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:39.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.574+0000 7f9572ffd700 1 --2- 172.21.15.133:0/3096588322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55880560ef60 0x5588056189d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.574+0000 7f9572ffd700 1 -- 172.21.15.133:0/3096588322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588056104b0 msgr2=0x55880561ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:39.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.574+0000 7f9572ffd700 1 --2- 172.21.15.133:0/3096588322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588056104b0 0x55880561ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.574+0000 7f9572ffd700 1 -- 172.21.15.133:0/3096588322 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f956c0089b0 con 0x55880560f8b0 2023-07-14T18:17:39.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.574+0000 7f9573fff700 1 --2- 172.21.15.133:0/3096588322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588056104b0 0x55880561ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:17:39.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.574+0000 7f9572ffd700 1 --2- 172.21.15.133:0/3096588322 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55880560f8b0 0x558805618f10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f95740743a0 tx=0x7f956800bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1024fe7218e6352a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:39.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.574+0000 7f9552ffd700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:17:39.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.574+0000 7f9552ffd700 1 -- 172.21.15.133:0/3096588322 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956801d030 con 0x55880560f8b0 2023-07-14T18:17:39.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.574+0000 7f958daffb80 1 -- 172.21.15.133:0/3096588322 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5588054b94f0 con 0x55880560f8b0 2023-07-14T18:17:39.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.574+0000 7f9552ffd700 1 -- 172.21.15.133:0/3096588322 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9568004500 con 0x55880560f8b0 2023-07-14T18:17:39.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.574+0000 7f9552ffd700 1 -- 172.21.15.133:0/3096588322 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95680046c0 con 0x55880560f8b0 2023-07-14T18:17:39.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.574+0000 7f958daffb80 10 client.25654 fetch_fsmap learned FSMap version 30 2023-07-14T18:17:39.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.574+0000 7f958daffb80 1 -- 172.21.15.133:0/3096588322 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558805376c60 con 0x55880560f8b0 2023-07-14T18:17:39.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.575+0000 7f9552ffd700 1 -- 172.21.15.133:0/3096588322 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5588054b94f0 con 0x55880560f8b0 2023-07-14T18:17:39.629 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.575+0000 7f9552ffd700 1 -- 172.21.15.133:0/3096588322 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f956800f880 con 0x55880560f8b0 2023-07-14T18:17:39.629 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.575+0000 7f958daffb80 10 client.25654 fetch_fsmap finished waiting for FSMap version 30 2023-07-14T18:17:39.629 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.575+0000 7f958daffb80 10 client.25654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:17:39.629 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.575+0000 7f958daffb80 20 client.25654 populate_metadata read hostname 'smithi133' 2023-07-14T18:17:39.629 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.575+0000 7f958daffb80 1 --2- 172.21.15.133:0/3096588322 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x558805624850 0x558805626c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:39.629 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.575+0000 7f958daffb80 4 client.25654 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:17:39.629 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.575+0000 7f958daffb80 1 -- 172.21.15.133:0/3096588322 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588055ddda0 con 0x558805624850 2023-07-14T18:17:39.629 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.575+0000 7f9573fff700 1 --2- 172.21.15.133:0/3096588322 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x558805624850 0x558805626c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:39.630 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.576+0000 7f9552ffd700 10 client.25654 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:17:39.630 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.576+0000 7f9573fff700 1 --2- 172.21.15.133:0/3096588322 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x558805624850 0x558805626c60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5417fcb5954149d2 server_cookie=7e9e4323161c5ce3 in_seq=0 out_seq=0 2023-07-14T18:17:39.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.576+0000 7f9552ffd700 1 -- 172.21.15.133:0/3096588322 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5588055ddda0 con 0x558805624850 2023-07-14T18:17:39.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.576+0000 7f9552ffd700 10 client.25654 handle_command_reply: tid=0 2023-07-14T18:17:39.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.576+0000 7f958daffb80 1 client.25654 shutdown 2023-07-14T18:17:39.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.577+0000 7f958daffb80 1 -- 172.21.15.133:0/3096588322 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55880560f8b0 msgr2=0x558805618f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:39.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.577+0000 7f958daffb80 1 --2- 172.21.15.133:0/3096588322 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55880560f8b0 0x558805618f10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f95740743a0 tx=0x7f956800bf60 comp rx=0 tx=0).stop 2023-07-14T18:17:39.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.577+0000 7f958daffb80 1 -- 172.21.15.133:0/3096588322 shutdown_connections 2023-07-14T18:17:39.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.577+0000 7f958daffb80 1 --2- 172.21.15.133:0/3096588322 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x558805624850 0x558805626c60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.577+0000 7f958daffb80 1 --2- 172.21.15.133:0/3096588322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55880560ef60 0x5588056189d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.577+0000 7f9552ffd700 0 client.25654 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:17:39.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.577+0000 7f958daffb80 1 --2- 172.21.15.133:0/3096588322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588056104b0 0x55880561ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.577+0000 7f958daffb80 1 --2- 172.21.15.133:0/3096588322 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55880560f8b0 0x558805618f10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.577+0000 7f958daffb80 1 -- 172.21.15.133:0/3096588322 >> 172.21.15.133:0/3096588322 conn(0x558805604680 msgr2=0x558805608650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:39.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.577+0000 7f958daffb80 1 -- 172.21.15.133:0/3096588322 shutdown_connections 2023-07-14T18:17:39.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.577+0000 7f958daffb80 1 -- 172.21.15.133:0/3096588322 wait complete. 2023-07-14T18:17:39.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.577+0000 7f958daffb80 20 client.25654 trim_cache size 0 max 16384 2023-07-14T18:17:39.633 INFO:teuthology.orchestra.run.smithi133.stderr:24100000 of 50000000 tuples (48%) done (elapsed 55.42 s, remaining 59.56 s) 2023-07-14T18:17:39.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.594+0000 7f958daffb80 1 Processor -- start 2023-07-14T18:17:39.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.595+0000 7f958daffb80 1 -- start start 2023-07-14T18:17:39.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.595+0000 7f958daffb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55880560ef60 0x5588056168f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:39.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.595+0000 7f958daffb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55880560f8b0 0x55880561fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:39.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.595+0000 7f958daffb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588056104b0 0x5588056189d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:39.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.595+0000 7f958daffb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880560b500 con 0x5588056104b0 2023-07-14T18:17:39.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.595+0000 7f958daffb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558805617e80 con 0x55880560f8b0 2023-07-14T18:17:39.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.595+0000 7f958daffb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x558805618540 con 0x55880560ef60 2023-07-14T18:17:39.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.595+0000 7f9572ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55880560f8b0 0x55880561fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:39.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.595+0000 7f9572ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55880560f8b0 0x55880561fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:60382/0 (socket says 172.21.15.133:60382) 2023-07-14T18:17:39.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.595+0000 7f9572ffd700 1 -- 172.21.15.133:0/3565901201 learned_addr learned my addr 172.21.15.133:0/3565901201 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:39.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.595+0000 7f9573fff700 1 --2- 172.21.15.133:0/3565901201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55880560ef60 0x5588056168f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:39.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.595+0000 7f9573fff700 1 --2- 172.21.15.133:0/3565901201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588056104b0 0x5588056189d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:39.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.595+0000 7f9572ffd700 1 -- 172.21.15.133:0/3565901201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55880560ef60 msgr2=0x5588056168f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:39.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.595+0000 7f9572ffd700 1 --2- 172.21.15.133:0/3565901201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55880560ef60 0x5588056168f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.595+0000 7f9572ffd700 1 -- 172.21.15.133:0/3565901201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588056104b0 msgr2=0x5588056189d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:39.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.595+0000 7f9572ffd700 1 --2- 172.21.15.133:0/3565901201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588056104b0 0x5588056189d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.596+0000 7f9572ffd700 1 -- 172.21.15.133:0/3565901201 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588053b4250 con 0x55880560f8b0 2023-07-14T18:17:39.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.596+0000 7f9573fff700 1 --2- 172.21.15.133:0/3565901201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588056104b0 0x5588056189d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T18:17:39.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.596+0000 7f9572ffd700 1 --2- 172.21.15.133:0/3565901201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55880560f8b0 0x55880561fe70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9568009990 tx=0x7f9568005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dcc3f702181483d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:39.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.596+0000 7f955a7fc700 1 -- 172.21.15.133:0/3565901201 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956801d040 con 0x55880560f8b0 2023-07-14T18:17:39.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.596+0000 7f955a7fc700 1 -- 172.21.15.133:0/3565901201 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f956801e040 con 0x55880560f8b0 2023-07-14T18:17:39.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.596+0000 7f955a7fc700 1 -- 172.21.15.133:0/3565901201 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9568016950 con 0x55880560f8b0 2023-07-14T18:17:39.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.597+0000 7f958daffb80 1 -- 172.21.15.133:0/3565901201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55880560f8b0 msgr2=0x55880561fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:39.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.597+0000 7f958daffb80 1 --2- 172.21.15.133:0/3565901201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55880560f8b0 0x55880561fe70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9568009990 tx=0x7f9568005f00 comp rx=0 tx=0).stop 2023-07-14T18:17:39.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.597+0000 7f958daffb80 1 -- 172.21.15.133:0/3565901201 shutdown_connections 2023-07-14T18:17:39.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.597+0000 7f958daffb80 1 --2- 172.21.15.133:0/3565901201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588056104b0 0x5588056189d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.597+0000 7f958daffb80 1 --2- 172.21.15.133:0/3565901201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55880560ef60 0x5588056168f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.597+0000 7f958daffb80 1 --2- 172.21.15.133:0/3565901201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55880560f8b0 0x55880561fe70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.597+0000 7f958daffb80 1 -- 172.21.15.133:0/3565901201 >> 172.21.15.133:0/3565901201 conn(0x558805604680 msgr2=0x558805617670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:39.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.597+0000 7f958daffb80 1 -- 172.21.15.133:0/3565901201 shutdown_connections 2023-07-14T18:17:39.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.597+0000 7f958daffb80 1 -- 172.21.15.133:0/3565901201 wait complete. 2023-07-14T18:17:39.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.598+0000 7f958daffb80 1 -- start start 2023-07-14T18:17:39.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.598+0000 7f958daffb80 1 Processor -- start 2023-07-14T18:17:39.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.598+0000 7f958daffb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55880560ef60 0x55880561fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:39.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.598+0000 7f958daffb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55880560f8b0 0x55880561e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:39.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.598+0000 7f958daffb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588056104b0 0x55880561e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:39.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.598+0000 7f958daffb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880561ebd0 con 0x55880560ef60 2023-07-14T18:17:39.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.598+0000 7f958daffb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880561ed40 con 0x55880560f8b0 2023-07-14T18:17:39.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.598+0000 7f958daffb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880561eeb0 con 0x5588056104b0 2023-07-14T18:17:39.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.598+0000 7f958daffb80 10 client.0 _ll_register_callbacks cb 0x7f9568009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:17:39.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.599+0000 7f9573fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55880560ef60 0x55880561fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:39.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.599+0000 7f9573fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55880560ef60 0x55880561fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57670/0 (socket says 172.21.15.133:57670) 2023-07-14T18:17:39.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.599+0000 7f9573fff700 1 -- 172.21.15.133:0/4026187510 learned_addr learned my addr 172.21.15.133:0/4026187510 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:17:39.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.599+0000 7f9572ffd700 1 --2- 172.21.15.133:0/4026187510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55880560f8b0 0x55880561e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:39.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.599+0000 7f9573fff700 1 --2- 172.21.15.133:0/4026187510 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588056104b0 0x55880561e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:39.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.599+0000 7f9572ffd700 1 -- 172.21.15.133:0/4026187510 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588056104b0 msgr2=0x55880561e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:39.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.599+0000 7f9572ffd700 1 --2- 172.21.15.133:0/4026187510 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588056104b0 0x55880561e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.599+0000 7f9572ffd700 1 -- 172.21.15.133:0/4026187510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55880560ef60 msgr2=0x55880561fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:17:39.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.599+0000 7f9572ffd700 1 --2- 172.21.15.133:0/4026187510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55880560ef60 0x55880561fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.599+0000 7f9572ffd700 1 -- 172.21.15.133:0/4026187510 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558805376c60 con 0x55880560f8b0 2023-07-14T18:17:39.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.599+0000 7f9573fff700 1 --2- 172.21.15.133:0/4026187510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55880560ef60 0x55880561fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:17:39.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.599+0000 7f9572ffd700 1 --2- 172.21.15.133:0/4026187510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55880560f8b0 0x55880561e150 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f956800fc10 tx=0x7f9568016450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94c410a55533e47b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:17:39.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.599+0000 7f9552ffd700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:17:39.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.600+0000 7f9552ffd700 1 -- 172.21.15.133:0/4026187510 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9568043070 con 0x55880560f8b0 2023-07-14T18:17:39.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.600+0000 7f958daffb80 1 -- 172.21.15.133:0/4026187510 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f956c00f040 con 0x55880560f8b0 2023-07-14T18:17:39.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.600+0000 7f9552ffd700 1 -- 172.21.15.133:0/4026187510 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f956800f460 con 0x55880560f8b0 2023-07-14T18:17:39.653 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.600+0000 7f9552ffd700 1 -- 172.21.15.133:0/4026187510 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f956800f620 con 0x55880560f8b0 2023-07-14T18:17:39.653 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.600+0000 7f958daffb80 10 client.25628 fetch_fsmap learned FSMap version 30 2023-07-14T18:17:39.653 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.600+0000 7f958daffb80 1 -- 172.21.15.133:0/4026187510 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55880538c2b0 con 0x55880560f8b0 2023-07-14T18:17:39.653 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.600+0000 7f9552ffd700 1 -- 172.21.15.133:0/4026187510 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f956c00f040 con 0x55880560f8b0 2023-07-14T18:17:39.653 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.600+0000 7f9552ffd700 1 -- 172.21.15.133:0/4026187510 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9568046030 con 0x55880560f8b0 2023-07-14T18:17:39.653 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.600+0000 7f958daffb80 10 client.25628 fetch_fsmap finished waiting for FSMap version 30 2023-07-14T18:17:39.653 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.601+0000 7f958daffb80 10 client.25628 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:17:39.653 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.601+0000 7f958daffb80 20 client.25628 populate_metadata read hostname 'smithi133' 2023-07-14T18:17:39.654 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.601+0000 7f958daffb80 1 --2- 172.21.15.133:0/4026187510 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x558805679770 0x55880567bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:17:39.654 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.601+0000 7f958daffb80 4 client.25628 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:17:39.654 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.601+0000 7f958daffb80 1 -- 172.21.15.133:0/4026187510 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558805558eb0 con 0x558805679770 2023-07-14T18:17:39.654 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.601+0000 7f9573fff700 1 --2- 172.21.15.133:0/4026187510 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x558805679770 0x55880567bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:17:39.655 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.601+0000 7f9552ffd700 10 client.25628 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:17:39.655 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.601+0000 7f9573fff700 1 --2- 172.21.15.133:0/4026187510 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x558805679770 0x55880567bb80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3d0080b4e43affe server_cookie=891ad6cdddafd9ba in_seq=0 out_seq=0 2023-07-14T18:17:39.655 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.601+0000 7f9552ffd700 1 -- 172.21.15.133:0/4026187510 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x558805558eb0 con 0x558805679770 2023-07-14T18:17:39.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.601+0000 7f9552ffd700 10 client.25628 handle_command_reply: tid=0 2023-07-14T18:17:39.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.601+0000 7f958daffb80 1 client.25628 shutdown 2023-07-14T18:17:39.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.602+0000 7f958daffb80 1 -- 172.21.15.133:0/4026187510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55880560f8b0 msgr2=0x55880561e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:39.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.602+0000 7f958daffb80 1 --2- 172.21.15.133:0/4026187510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55880560f8b0 0x55880561e150 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f956800fc10 tx=0x7f9568016450 comp rx=0 tx=0).stop 2023-07-14T18:17:39.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.602+0000 7f958daffb80 1 -- 172.21.15.133:0/4026187510 shutdown_connections 2023-07-14T18:17:39.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.602+0000 7f958daffb80 1 --2- 172.21.15.133:0/4026187510 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x558805679770 0x55880567bb80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.602+0000 7f958daffb80 1 --2- 172.21.15.133:0/4026187510 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588056104b0 0x55880561e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.602+0000 7f958daffb80 1 --2- 172.21.15.133:0/4026187510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55880560ef60 0x55880561fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.602+0000 7f9552ffd700 0 client.25628 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:17:39.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.602+0000 7f958daffb80 1 --2- 172.21.15.133:0/4026187510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55880560f8b0 0x55880561e150 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.602+0000 7f958daffb80 1 -- 172.21.15.133:0/4026187510 >> 172.21.15.133:0/4026187510 conn(0x558805604680 msgr2=0x5588056168f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:39.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.602+0000 7f958daffb80 1 -- 172.21.15.133:0/4026187510 shutdown_connections 2023-07-14T18:17:39.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.602+0000 7f958daffb80 1 -- 172.21.15.133:0/4026187510 wait complete. 2023-07-14T18:17:39.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.602+0000 7f958daffb80 20 client.25628 trim_cache size 0 max 16384 2023-07-14T18:17:39.658 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:17:39.658 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-14T18:17:39.658 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:17:39.658 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:17:39.658 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:17:39.658 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:17:39.658 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:17:39.659 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:17:39.659 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:17:39.659 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:17:39.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.603+0000 7f957a1ff700 1 -- 172.21.15.133:0/1042783628 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f955c0a1150 msgr2=0x7f955c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:39.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.603+0000 7f957a1ff700 1 --2- 172.21.15.133:0/1042783628 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f955c0a1150 0x7f955c0a3600 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9564005950 tx=0x7f9564004750 comp rx=0 tx=0).stop 2023-07-14T18:17:39.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.603+0000 7f957a1ff700 1 -- 172.21.15.133:0/1042783628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95740730a0 msgr2=0x7f9574197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:17:39.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.603+0000 7f957a1ff700 1 --2- 172.21.15.133:0/1042783628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95740730a0 0x7f9574197be0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f956400e970 tx=0x7f956400ec80 comp rx=0 tx=0).stop 2023-07-14T18:17:39.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.603+0000 7f957a1ff700 1 -- 172.21.15.133:0/1042783628 shutdown_connections 2023-07-14T18:17:39.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.603+0000 7f957a1ff700 1 --2- 172.21.15.133:0/1042783628 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f955c0a1150 0x7f955c0a3600 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.603+0000 7f957a1ff700 1 --2- 172.21.15.133:0/1042783628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9574107ae0 0x7f957419f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.603+0000 7f957a1ff700 1 --2- 172.21.15.133:0/1042783628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95740730a0 0x7f9574197be0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.603+0000 7f957a1ff700 1 --2- 172.21.15.133:0/1042783628 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9574074680 0x7f9574198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:17:39.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.603+0000 7f957a1ff700 1 -- 172.21.15.133:0/1042783628 >> 172.21.15.133:0/1042783628 conn(0x7f95740fb260 msgr2=0x7f95741039e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:17:39.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.603+0000 7f957a1ff700 1 -- 172.21.15.133:0/1042783628 shutdown_connections 2023-07-14T18:17:39.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:17:39.603+0000 7f957a1ff700 1 -- 172.21.15.133:0/1042783628 wait complete. 2023-07-14T18:17:39.669 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:17:39.852 INFO:teuthology.orchestra.run.smithi133.stderr:24200000 of 50000000 tuples (48%) done (elapsed 55.64 s, remaining 59.32 s) 2023-07-14T18:17:40.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:39 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:17:40.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:39 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:17:40.405 INFO:teuthology.orchestra.run.smithi133.stderr:24300000 of 50000000 tuples (48%) done (elapsed 56.19 s, remaining 59.43 s) 2023-07-14T18:17:40.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:39 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:17:40.702 INFO:teuthology.orchestra.run.smithi133.stderr:24400000 of 50000000 tuples (48%) done (elapsed 56.49 s, remaining 59.27 s) 2023-07-14T18:17:40.887 INFO:teuthology.orchestra.run.smithi133.stderr:24500000 of 50000000 tuples (49%) done (elapsed 56.67 s, remaining 58.99 s) 2023-07-14T18:17:41.114 INFO:teuthology.orchestra.run.smithi133.stderr:24600000 of 50000000 tuples (49%) done (elapsed 56.90 s, remaining 58.75 s) 2023-07-14T18:17:41.265 INFO:teuthology.orchestra.run.smithi133.stderr:24700000 of 50000000 tuples (49%) done (elapsed 57.05 s, remaining 58.44 s) 2023-07-14T18:17:41.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:40 smithi161 ceph-mon[128993]: pgmap v378: 129 pgs: 129 active+clean; 3.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 126 MiB/s wr, 62 op/s 2023-07-14T18:17:41.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:40 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:17:41.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:40 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:17:41.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:40 smithi171 ceph-mon[129241]: pgmap v378: 129 pgs: 129 active+clean; 3.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 126 MiB/s wr, 62 op/s 2023-07-14T18:17:41.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:40 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:17:41.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:40 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:17:41.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:40 smithi133 ceph-mon[126384]: pgmap v378: 129 pgs: 129 active+clean; 3.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 126 MiB/s wr, 62 op/s 2023-07-14T18:17:41.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:40 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:17:41.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:40 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:17:41.468 INFO:teuthology.orchestra.run.smithi133.stderr:24800000 of 50000000 tuples (49%) done (elapsed 57.25 s, remaining 58.18 s) 2023-07-14T18:17:41.670 INFO:teuthology.orchestra.run.smithi133.stderr:24900000 of 50000000 tuples (49%) done (elapsed 57.46 s, remaining 57.92 s) 2023-07-14T18:17:41.855 INFO:teuthology.orchestra.run.smithi133.stderr:25000000 of 50000000 tuples (50%) done (elapsed 57.64 s, remaining 57.64 s) 2023-07-14T18:17:42.039 INFO:teuthology.orchestra.run.smithi133.stderr:25100000 of 50000000 tuples (50%) done (elapsed 57.83 s, remaining 57.36 s) 2023-07-14T18:17:42.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:41 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:17:42.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:41 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:17:42.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:41 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:17:42.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:41 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:17:42.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:41 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:17:42.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:41 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:17:42.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:41 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:17:42.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:41 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:17:42.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:41 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:17:42.515 INFO:teuthology.orchestra.run.smithi133.stderr:25200000 of 50000000 tuples (50%) done (elapsed 58.30 s, remaining 57.38 s) 2023-07-14T18:17:42.989 INFO:teuthology.orchestra.run.smithi133.stderr:25300000 of 50000000 tuples (50%) done (elapsed 58.78 s, remaining 57.38 s) 2023-07-14T18:17:43.197 INFO:teuthology.orchestra.run.smithi133.stderr:25400000 of 50000000 tuples (50%) done (elapsed 58.98 s, remaining 57.12 s) 2023-07-14T18:17:43.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:42 smithi161 ceph-mon[128993]: pgmap v379: 129 pgs: 129 active+clean; 3.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 119 MiB/s wr, 79 op/s 2023-07-14T18:17:43.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:42 smithi171 ceph-mon[129241]: pgmap v379: 129 pgs: 129 active+clean; 3.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 119 MiB/s wr, 79 op/s 2023-07-14T18:17:43.411 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:42 smithi133 ceph-mon[126384]: pgmap v379: 129 pgs: 129 active+clean; 3.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 119 MiB/s wr, 79 op/s 2023-07-14T18:17:43.543 INFO:teuthology.orchestra.run.smithi133.stderr:25500000 of 50000000 tuples (51%) done (elapsed 59.33 s, remaining 57.00 s) 2023-07-14T18:17:43.911 INFO:teuthology.orchestra.run.smithi133.stderr:25600000 of 50000000 tuples (51%) done (elapsed 59.70 s, remaining 56.90 s) 2023-07-14T18:17:44.700 INFO:teuthology.orchestra.run.smithi133.stderr:25700000 of 50000000 tuples (51%) done (elapsed 60.49 s, remaining 57.19 s) 2023-07-14T18:17:45.283 INFO:teuthology.orchestra.run.smithi133.stderr:25800000 of 50000000 tuples (51%) done (elapsed 61.07 s, remaining 57.28 s) 2023-07-14T18:17:45.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:44 smithi161 ceph-mon[128993]: pgmap v380: 129 pgs: 129 active+clean; 3.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 102 MiB/s wr, 78 op/s 2023-07-14T18:17:45.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:44 smithi171 ceph-mon[129241]: pgmap v380: 129 pgs: 129 active+clean; 3.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 102 MiB/s wr, 78 op/s 2023-07-14T18:17:45.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:44 smithi133 ceph-mon[126384]: pgmap v380: 129 pgs: 129 active+clean; 3.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 102 MiB/s wr, 78 op/s 2023-07-14T18:17:45.477 INFO:teuthology.orchestra.run.smithi133.stderr:25900000 of 50000000 tuples (51%) done (elapsed 61.26 s, remaining 57.01 s) 2023-07-14T18:17:45.622 INFO:teuthology.orchestra.run.smithi133.stderr:26000000 of 50000000 tuples (52%) done (elapsed 61.41 s, remaining 56.68 s) 2023-07-14T18:17:45.830 INFO:teuthology.orchestra.run.smithi133.stderr:26100000 of 50000000 tuples (52%) done (elapsed 61.62 s, remaining 56.42 s) 2023-07-14T18:17:45.979 INFO:teuthology.orchestra.run.smithi133.stderr:26200000 of 50000000 tuples (52%) done (elapsed 61.76 s, remaining 56.11 s) 2023-07-14T18:17:46.124 INFO:teuthology.orchestra.run.smithi133.stderr:26300000 of 50000000 tuples (52%) done (elapsed 61.91 s, remaining 55.79 s) 2023-07-14T18:17:46.333 INFO:teuthology.orchestra.run.smithi133.stderr:26400000 of 50000000 tuples (52%) done (elapsed 62.12 s, remaining 55.53 s) 2023-07-14T18:17:46.548 INFO:teuthology.orchestra.run.smithi133.stderr:26500000 of 50000000 tuples (53%) done (elapsed 62.33 s, remaining 55.28 s) 2023-07-14T18:17:46.698 INFO:teuthology.orchestra.run.smithi133.stderr:26600000 of 50000000 tuples (53%) done (elapsed 62.48 s, remaining 54.97 s) 2023-07-14T18:17:46.937 INFO:teuthology.orchestra.run.smithi133.stderr:26700000 of 50000000 tuples (53%) done (elapsed 62.72 s, remaining 54.74 s) 2023-07-14T18:17:47.106 INFO:teuthology.orchestra.run.smithi133.stderr:26800000 of 50000000 tuples (53%) done (elapsed 62.89 s, remaining 54.44 s) 2023-07-14T18:17:47.316 INFO:teuthology.orchestra.run.smithi133.stderr:26900000 of 50000000 tuples (53%) done (elapsed 63.10 s, remaining 54.19 s) 2023-07-14T18:17:47.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:46 smithi161 ceph-mon[128993]: pgmap v381: 129 pgs: 129 active+clean; 3.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 89 MiB/s wr, 63 op/s 2023-07-14T18:17:47.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:46 smithi171 ceph-mon[129241]: pgmap v381: 129 pgs: 129 active+clean; 3.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 89 MiB/s wr, 63 op/s 2023-07-14T18:17:47.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:46 smithi133 ceph-mon[126384]: pgmap v381: 129 pgs: 129 active+clean; 3.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 89 MiB/s wr, 63 op/s 2023-07-14T18:17:47.521 INFO:teuthology.orchestra.run.smithi133.stderr:27000000 of 50000000 tuples (54%) done (elapsed 63.31 s, remaining 53.93 s) 2023-07-14T18:17:47.629 INFO:teuthology.orchestra.run.smithi133.stderr:27100000 of 50000000 tuples (54%) done (elapsed 63.41 s, remaining 53.59 s) 2023-07-14T18:17:47.823 INFO:teuthology.orchestra.run.smithi133.stderr:27200000 of 50000000 tuples (54%) done (elapsed 63.61 s, remaining 53.32 s) 2023-07-14T18:17:48.024 INFO:teuthology.orchestra.run.smithi133.stderr:27300000 of 50000000 tuples (54%) done (elapsed 63.81 s, remaining 53.06 s) 2023-07-14T18:17:48.171 INFO:teuthology.orchestra.run.smithi133.stderr:27400000 of 50000000 tuples (54%) done (elapsed 63.96 s, remaining 52.75 s) 2023-07-14T18:17:48.370 INFO:teuthology.orchestra.run.smithi133.stderr:27500000 of 50000000 tuples (55%) done (elapsed 64.16 s, remaining 52.49 s) 2023-07-14T18:17:48.535 INFO:teuthology.orchestra.run.smithi133.stderr:27600000 of 50000000 tuples (55%) done (elapsed 64.32 s, remaining 52.20 s) 2023-07-14T18:17:48.679 INFO:teuthology.orchestra.run.smithi133.stderr:27700000 of 50000000 tuples (55%) done (elapsed 64.47 s, remaining 51.90 s) 2023-07-14T18:17:48.869 INFO:teuthology.orchestra.run.smithi133.stderr:27800000 of 50000000 tuples (55%) done (elapsed 64.65 s, remaining 51.63 s) 2023-07-14T18:17:49.025 INFO:teuthology.orchestra.run.smithi133.stderr:27900000 of 50000000 tuples (55%) done (elapsed 64.81 s, remaining 51.34 s) 2023-07-14T18:17:49.162 INFO:teuthology.orchestra.run.smithi133.stderr:28000000 of 50000000 tuples (56%) done (elapsed 64.95 s, remaining 51.03 s) 2023-07-14T18:17:49.325 INFO:teuthology.orchestra.run.smithi133.stderr:28100000 of 50000000 tuples (56%) done (elapsed 65.11 s, remaining 50.74 s) 2023-07-14T18:17:49.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:48 smithi161 ceph-mon[128993]: pgmap v382: 129 pgs: 129 active+clean; 4.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 126 MiB/s wr, 108 op/s 2023-07-14T18:17:49.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:48 smithi171 ceph-mon[129241]: pgmap v382: 129 pgs: 129 active+clean; 4.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 126 MiB/s wr, 108 op/s 2023-07-14T18:17:49.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:48 smithi133 ceph-mon[126384]: pgmap v382: 129 pgs: 129 active+clean; 4.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 126 MiB/s wr, 108 op/s 2023-07-14T18:17:49.528 INFO:teuthology.orchestra.run.smithi133.stderr:28200000 of 50000000 tuples (56%) done (elapsed 65.31 s, remaining 50.49 s) 2023-07-14T18:17:49.669 INFO:teuthology.orchestra.run.smithi133.stderr:28300000 of 50000000 tuples (56%) done (elapsed 65.45 s, remaining 50.19 s) 2023-07-14T18:17:49.848 INFO:teuthology.orchestra.run.smithi133.stderr:28400000 of 50000000 tuples (56%) done (elapsed 65.63 s, remaining 49.92 s) 2023-07-14T18:17:50.058 INFO:teuthology.orchestra.run.smithi133.stderr:28500000 of 50000000 tuples (57%) done (elapsed 65.84 s, remaining 49.67 s) 2023-07-14T18:17:50.202 INFO:teuthology.orchestra.run.smithi133.stderr:28600000 of 50000000 tuples (57%) done (elapsed 65.99 s, remaining 49.38 s) 2023-07-14T18:17:50.361 INFO:teuthology.orchestra.run.smithi133.stderr:28700000 of 50000000 tuples (57%) done (elapsed 66.15 s, remaining 49.09 s) 2023-07-14T18:17:50.559 INFO:teuthology.orchestra.run.smithi133.stderr:28800000 of 50000000 tuples (57%) done (elapsed 66.34 s, remaining 48.84 s) 2023-07-14T18:17:50.752 INFO:teuthology.orchestra.run.smithi133.stderr:28900000 of 50000000 tuples (57%) done (elapsed 66.54 s, remaining 48.58 s) 2023-07-14T18:17:50.880 INFO:teuthology.orchestra.run.smithi133.stderr:29000000 of 50000000 tuples (58%) done (elapsed 66.67 s, remaining 48.28 s) 2023-07-14T18:17:51.076 INFO:teuthology.orchestra.run.smithi133.stderr:29100000 of 50000000 tuples (58%) done (elapsed 66.86 s, remaining 48.02 s) 2023-07-14T18:17:51.249 INFO:teuthology.orchestra.run.smithi133.stderr:29200000 of 50000000 tuples (58%) done (elapsed 67.03 s, remaining 47.75 s) 2023-07-14T18:17:51.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:50 smithi161 ceph-mon[128993]: pgmap v383: 129 pgs: 129 active+clean; 4.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 97 MiB/s wr, 98 op/s 2023-07-14T18:17:51.367 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:50 smithi171 ceph-mon[129241]: pgmap v383: 129 pgs: 129 active+clean; 4.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 97 MiB/s wr, 98 op/s 2023-07-14T18:17:51.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:50 smithi133 ceph-mon[126384]: pgmap v383: 129 pgs: 129 active+clean; 4.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 97 MiB/s wr, 98 op/s 2023-07-14T18:17:51.472 INFO:teuthology.orchestra.run.smithi133.stderr:29300000 of 50000000 tuples (58%) done (elapsed 67.26 s, remaining 47.52 s) 2023-07-14T18:17:51.792 INFO:teuthology.orchestra.run.smithi133.stderr:29400000 of 50000000 tuples (58%) done (elapsed 67.58 s, remaining 47.35 s) 2023-07-14T18:17:52.286 INFO:teuthology.orchestra.run.smithi133.stderr:29500000 of 50000000 tuples (59%) done (elapsed 68.07 s, remaining 47.30 s) 2023-07-14T18:17:52.787 INFO:teuthology.orchestra.run.smithi133.stderr:29600000 of 50000000 tuples (59%) done (elapsed 68.57 s, remaining 47.26 s) 2023-07-14T18:17:52.973 INFO:teuthology.orchestra.run.smithi133.stderr:29700000 of 50000000 tuples (59%) done (elapsed 68.76 s, remaining 47.00 s) 2023-07-14T18:17:53.185 INFO:teuthology.orchestra.run.smithi133.stderr:29800000 of 50000000 tuples (59%) done (elapsed 68.97 s, remaining 46.75 s) 2023-07-14T18:17:53.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:52 smithi161 ceph-mon[128993]: pgmap v384: 129 pgs: 129 active+clean; 4.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 91 MiB/s wr, 95 op/s 2023-07-14T18:17:53.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:52 smithi171 ceph-mon[129241]: pgmap v384: 129 pgs: 129 active+clean; 4.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 91 MiB/s wr, 95 op/s 2023-07-14T18:17:53.387 INFO:teuthology.orchestra.run.smithi133.stderr:29900000 of 50000000 tuples (59%) done (elapsed 69.17 s, remaining 46.50 s) 2023-07-14T18:17:53.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:52 smithi133 ceph-mon[126384]: pgmap v384: 129 pgs: 129 active+clean; 4.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 91 MiB/s wr, 95 op/s 2023-07-14T18:17:53.544 INFO:teuthology.orchestra.run.smithi133.stderr:30000000 of 50000000 tuples (60%) done (elapsed 69.33 s, remaining 46.22 s) 2023-07-14T18:17:53.727 INFO:teuthology.orchestra.run.smithi133.stderr:30100000 of 50000000 tuples (60%) done (elapsed 69.51 s, remaining 45.96 s) 2023-07-14T18:17:53.918 INFO:teuthology.orchestra.run.smithi133.stderr:30200000 of 50000000 tuples (60%) done (elapsed 69.70 s, remaining 45.70 s) 2023-07-14T18:17:54.107 INFO:teuthology.orchestra.run.smithi133.stderr:30300000 of 50000000 tuples (60%) done (elapsed 69.89 s, remaining 45.44 s) 2023-07-14T18:17:54.286 INFO:teuthology.orchestra.run.smithi133.stderr:30400000 of 50000000 tuples (60%) done (elapsed 70.07 s, remaining 45.18 s) 2023-07-14T18:17:54.412 INFO:teuthology.orchestra.run.smithi133.stderr:30500000 of 50000000 tuples (61%) done (elapsed 70.20 s, remaining 44.88 s) 2023-07-14T18:17:54.595 INFO:teuthology.orchestra.run.smithi133.stderr:30600000 of 50000000 tuples (61%) done (elapsed 70.38 s, remaining 44.62 s) 2023-07-14T18:17:54.807 INFO:teuthology.orchestra.run.smithi133.stderr:30700000 of 50000000 tuples (61%) done (elapsed 70.59 s, remaining 44.38 s) 2023-07-14T18:17:55.015 INFO:teuthology.orchestra.run.smithi133.stderr:30800000 of 50000000 tuples (61%) done (elapsed 70.80 s, remaining 44.14 s) 2023-07-14T18:17:55.120 INFO:teuthology.orchestra.run.smithi133.stderr:30900000 of 50000000 tuples (61%) done (elapsed 70.91 s, remaining 43.83 s) 2023-07-14T18:17:55.340 INFO:teuthology.orchestra.run.smithi133.stderr:31000000 of 50000000 tuples (62%) done (elapsed 71.13 s, remaining 43.59 s) 2023-07-14T18:17:55.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:55 smithi161 ceph-mon[128993]: pgmap v385: 129 pgs: 129 active+clean; 4.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 101 MiB/s wr, 81 op/s 2023-07-14T18:17:55.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:55 smithi171 ceph-mon[129241]: pgmap v385: 129 pgs: 129 active+clean; 4.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 101 MiB/s wr, 81 op/s 2023-07-14T18:17:55.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:55 smithi133 ceph-mon[126384]: pgmap v385: 129 pgs: 129 active+clean; 4.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 101 MiB/s wr, 81 op/s 2023-07-14T18:17:55.513 INFO:teuthology.orchestra.run.smithi133.stderr:31100000 of 50000000 tuples (62%) done (elapsed 71.30 s, remaining 43.33 s) 2023-07-14T18:17:55.688 INFO:teuthology.orchestra.run.smithi133.stderr:31200000 of 50000000 tuples (62%) done (elapsed 71.47 s, remaining 43.07 s) 2023-07-14T18:17:55.818 INFO:teuthology.orchestra.run.smithi133.stderr:31300000 of 50000000 tuples (62%) done (elapsed 71.60 s, remaining 42.78 s) 2023-07-14T18:17:56.027 INFO:teuthology.orchestra.run.smithi133.stderr:31400000 of 50000000 tuples (62%) done (elapsed 71.81 s, remaining 42.54 s) 2023-07-14T18:17:56.197 INFO:teuthology.orchestra.run.smithi133.stderr:31500000 of 50000000 tuples (63%) done (elapsed 71.98 s, remaining 42.28 s) 2023-07-14T18:17:56.382 INFO:teuthology.orchestra.run.smithi133.stderr:31600000 of 50000000 tuples (63%) done (elapsed 72.17 s, remaining 42.02 s) 2023-07-14T18:17:56.555 INFO:teuthology.orchestra.run.smithi133.stderr:31700000 of 50000000 tuples (63%) done (elapsed 72.34 s, remaining 41.76 s) 2023-07-14T18:17:56.709 INFO:teuthology.orchestra.run.smithi133.stderr:31800000 of 50000000 tuples (63%) done (elapsed 72.49 s, remaining 41.49 s) 2023-07-14T18:17:56.877 INFO:teuthology.orchestra.run.smithi133.stderr:31900000 of 50000000 tuples (63%) done (elapsed 72.66 s, remaining 41.23 s) 2023-07-14T18:17:57.059 INFO:teuthology.orchestra.run.smithi133.stderr:32000000 of 50000000 tuples (64%) done (elapsed 72.84 s, remaining 40.97 s) 2023-07-14T18:17:57.237 INFO:teuthology.orchestra.run.smithi133.stderr:32100000 of 50000000 tuples (64%) done (elapsed 73.02 s, remaining 40.72 s) 2023-07-14T18:17:57.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:57 smithi161 ceph-mon[128993]: pgmap v386: 129 pgs: 129 active+clean; 4.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 99 MiB/s wr, 76 op/s 2023-07-14T18:17:57.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:57 smithi171 ceph-mon[129241]: pgmap v386: 129 pgs: 129 active+clean; 4.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 99 MiB/s wr, 76 op/s 2023-07-14T18:17:57.373 INFO:teuthology.orchestra.run.smithi133.stderr:32200000 of 50000000 tuples (64%) done (elapsed 73.16 s, remaining 40.44 s) 2023-07-14T18:17:57.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:57 smithi133 ceph-mon[126384]: pgmap v386: 129 pgs: 129 active+clean; 4.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 99 MiB/s wr, 76 op/s 2023-07-14T18:17:57.556 INFO:teuthology.orchestra.run.smithi133.stderr:32300000 of 50000000 tuples (64%) done (elapsed 73.34 s, remaining 40.19 s) 2023-07-14T18:17:57.752 INFO:teuthology.orchestra.run.smithi133.stderr:32400000 of 50000000 tuples (64%) done (elapsed 73.54 s, remaining 39.95 s) 2023-07-14T18:17:57.913 INFO:teuthology.orchestra.run.smithi133.stderr:32500000 of 50000000 tuples (65%) done (elapsed 73.70 s, remaining 39.68 s) 2023-07-14T18:17:58.063 INFO:teuthology.orchestra.run.smithi133.stderr:32600000 of 50000000 tuples (65%) done (elapsed 73.85 s, remaining 39.42 s) 2023-07-14T18:17:58.249 INFO:teuthology.orchestra.run.smithi133.stderr:32700000 of 50000000 tuples (65%) done (elapsed 74.03 s, remaining 39.17 s) 2023-07-14T18:17:58.481 INFO:teuthology.orchestra.run.smithi133.stderr:32800000 of 50000000 tuples (65%) done (elapsed 74.27 s, remaining 38.94 s) 2023-07-14T18:17:58.636 INFO:teuthology.orchestra.run.smithi133.stderr:32900000 of 50000000 tuples (65%) done (elapsed 74.42 s, remaining 38.68 s) 2023-07-14T18:17:58.758 INFO:teuthology.orchestra.run.smithi133.stderr:33000000 of 50000000 tuples (66%) done (elapsed 74.54 s, remaining 38.40 s) 2023-07-14T18:17:58.954 INFO:teuthology.orchestra.run.smithi133.stderr:33100000 of 50000000 tuples (66%) done (elapsed 74.74 s, remaining 38.16 s) 2023-07-14T18:17:59.326 INFO:teuthology.orchestra.run.smithi133.stderr:33200000 of 50000000 tuples (66%) done (elapsed 75.11 s, remaining 38.01 s) 2023-07-14T18:17:59.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:17:59 smithi161 ceph-mon[128993]: pgmap v387: 129 pgs: 129 active+clean; 4.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 134 MiB/s wr, 96 op/s 2023-07-14T18:17:59.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:17:59 smithi171 ceph-mon[129241]: pgmap v387: 129 pgs: 129 active+clean; 4.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 134 MiB/s wr, 96 op/s 2023-07-14T18:17:59.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:17:59 smithi133 ceph-mon[126384]: pgmap v387: 129 pgs: 129 active+clean; 4.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 134 MiB/s wr, 96 op/s 2023-07-14T18:17:59.769 INFO:teuthology.orchestra.run.smithi133.stderr:33300000 of 50000000 tuples (66%) done (elapsed 75.55 s, remaining 37.89 s) 2023-07-14T18:18:00.234 INFO:teuthology.orchestra.run.smithi133.stderr:33400000 of 50000000 tuples (66%) done (elapsed 76.02 s, remaining 37.78 s) 2023-07-14T18:18:00.968 INFO:teuthology.orchestra.run.smithi133.stderr:33500000 of 50000000 tuples (67%) done (elapsed 76.75 s, remaining 37.80 s) 2023-07-14T18:18:01.129 INFO:teuthology.orchestra.run.smithi133.stderr:33600000 of 50000000 tuples (67%) done (elapsed 76.91 s, remaining 37.54 s) 2023-07-14T18:18:01.313 INFO:teuthology.orchestra.run.smithi133.stderr:33700000 of 50000000 tuples (67%) done (elapsed 77.10 s, remaining 37.29 s) 2023-07-14T18:18:01.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:01 smithi161 ceph-mon[128993]: pgmap v388: 129 pgs: 129 active+clean; 4.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 102 MiB/s wr, 55 op/s 2023-07-14T18:18:01.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:01 smithi171 ceph-mon[129241]: pgmap v388: 129 pgs: 129 active+clean; 4.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 102 MiB/s wr, 55 op/s 2023-07-14T18:18:01.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:01 smithi133 ceph-mon[126384]: pgmap v388: 129 pgs: 129 active+clean; 4.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 102 MiB/s wr, 55 op/s 2023-07-14T18:18:01.509 INFO:teuthology.orchestra.run.smithi133.stderr:33800000 of 50000000 tuples (67%) done (elapsed 77.29 s, remaining 37.05 s) 2023-07-14T18:18:01.681 INFO:teuthology.orchestra.run.smithi133.stderr:33900000 of 50000000 tuples (67%) done (elapsed 77.47 s, remaining 36.79 s) 2023-07-14T18:18:01.865 INFO:teuthology.orchestra.run.smithi133.stderr:34000000 of 50000000 tuples (68%) done (elapsed 77.65 s, remaining 36.54 s) 2023-07-14T18:18:01.999 INFO:teuthology.orchestra.run.smithi133.stderr:34100000 of 50000000 tuples (68%) done (elapsed 77.78 s, remaining 36.27 s) 2023-07-14T18:18:02.201 INFO:teuthology.orchestra.run.smithi133.stderr:34200000 of 50000000 tuples (68%) done (elapsed 77.99 s, remaining 36.03 s) 2023-07-14T18:18:02.364 INFO:teuthology.orchestra.run.smithi133.stderr:34300000 of 50000000 tuples (68%) done (elapsed 78.15 s, remaining 35.77 s) 2023-07-14T18:18:02.497 INFO:teuthology.orchestra.run.smithi133.stderr:34400000 of 50000000 tuples (68%) done (elapsed 78.28 s, remaining 35.50 s) 2023-07-14T18:18:02.673 INFO:teuthology.orchestra.run.smithi133.stderr:34500000 of 50000000 tuples (69%) done (elapsed 78.46 s, remaining 35.25 s) 2023-07-14T18:18:02.866 INFO:teuthology.orchestra.run.smithi133.stderr:34600000 of 50000000 tuples (69%) done (elapsed 78.65 s, remaining 35.01 s) 2023-07-14T18:18:03.040 INFO:teuthology.orchestra.run.smithi133.stderr:34700000 of 50000000 tuples (69%) done (elapsed 78.83 s, remaining 34.76 s) 2023-07-14T18:18:03.188 INFO:teuthology.orchestra.run.smithi133.stderr:34800000 of 50000000 tuples (69%) done (elapsed 78.97 s, remaining 34.49 s) 2023-07-14T18:18:03.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:03 smithi161 ceph-mon[128993]: pgmap v389: 129 pgs: 129 active+clean; 4.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 116 MiB/s wr, 51 op/s 2023-07-14T18:18:03.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:03 smithi171 ceph-mon[129241]: pgmap v389: 129 pgs: 129 active+clean; 4.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 116 MiB/s wr, 51 op/s 2023-07-14T18:18:03.369 INFO:teuthology.orchestra.run.smithi133.stderr:34900000 of 50000000 tuples (69%) done (elapsed 79.15 s, remaining 34.25 s) 2023-07-14T18:18:03.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:03 smithi133 ceph-mon[126384]: pgmap v389: 129 pgs: 129 active+clean; 4.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 116 MiB/s wr, 51 op/s 2023-07-14T18:18:03.552 INFO:teuthology.orchestra.run.smithi133.stderr:35000000 of 50000000 tuples (70%) done (elapsed 79.34 s, remaining 34.00 s) 2023-07-14T18:18:03.738 INFO:teuthology.orchestra.run.smithi133.stderr:35100000 of 50000000 tuples (70%) done (elapsed 79.52 s, remaining 33.76 s) 2023-07-14T18:18:03.843 INFO:teuthology.orchestra.run.smithi133.stderr:35200000 of 50000000 tuples (70%) done (elapsed 79.63 s, remaining 33.48 s) 2023-07-14T18:18:04.042 INFO:teuthology.orchestra.run.smithi133.stderr:35300000 of 50000000 tuples (70%) done (elapsed 79.83 s, remaining 33.24 s) 2023-07-14T18:18:04.243 INFO:teuthology.orchestra.run.smithi133.stderr:35400000 of 50000000 tuples (70%) done (elapsed 80.03 s, remaining 33.01 s) 2023-07-14T18:18:04.386 INFO:teuthology.orchestra.run.smithi133.stderr:35500000 of 50000000 tuples (71%) done (elapsed 80.17 s, remaining 32.75 s) 2023-07-14T18:18:04.542 INFO:teuthology.orchestra.run.smithi133.stderr:35600000 of 50000000 tuples (71%) done (elapsed 80.33 s, remaining 32.49 s) 2023-07-14T18:18:04.730 INFO:teuthology.orchestra.run.smithi133.stderr:35700000 of 50000000 tuples (71%) done (elapsed 80.52 s, remaining 32.25 s) 2023-07-14T18:18:04.861 INFO:teuthology.orchestra.run.smithi133.stderr:35800000 of 50000000 tuples (71%) done (elapsed 80.65 s, remaining 31.99 s) 2023-07-14T18:18:05.040 INFO:teuthology.orchestra.run.smithi133.stderr:35900000 of 50000000 tuples (71%) done (elapsed 80.83 s, remaining 31.75 s) 2023-07-14T18:18:05.250 INFO:teuthology.orchestra.run.smithi133.stderr:36000000 of 50000000 tuples (72%) done (elapsed 81.04 s, remaining 31.51 s) 2023-07-14T18:18:05.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:05 smithi161 ceph-mon[128993]: pgmap v390: 129 pgs: 129 active+clean; 5.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 135 MiB/s wr, 68 op/s 2023-07-14T18:18:05.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:05 smithi171 ceph-mon[129241]: pgmap v390: 129 pgs: 129 active+clean; 5.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 135 MiB/s wr, 68 op/s 2023-07-14T18:18:05.403 INFO:teuthology.orchestra.run.smithi133.stderr:36100000 of 50000000 tuples (72%) done (elapsed 81.19 s, remaining 31.26 s) 2023-07-14T18:18:05.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:05 smithi133 ceph-mon[126384]: pgmap v390: 129 pgs: 129 active+clean; 5.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 135 MiB/s wr, 68 op/s 2023-07-14T18:18:05.565 INFO:teuthology.orchestra.run.smithi133.stderr:36200000 of 50000000 tuples (72%) done (elapsed 81.35 s, remaining 31.01 s) 2023-07-14T18:18:05.758 INFO:teuthology.orchestra.run.smithi133.stderr:36300000 of 50000000 tuples (72%) done (elapsed 81.54 s, remaining 30.78 s) 2023-07-14T18:18:05.947 INFO:teuthology.orchestra.run.smithi133.stderr:36400000 of 50000000 tuples (72%) done (elapsed 81.73 s, remaining 30.54 s) 2023-07-14T18:18:06.051 INFO:teuthology.orchestra.run.smithi133.stderr:36500000 of 50000000 tuples (73%) done (elapsed 81.84 s, remaining 30.27 s) 2023-07-14T18:18:06.230 INFO:teuthology.orchestra.run.smithi133.stderr:36600000 of 50000000 tuples (73%) done (elapsed 82.02 s, remaining 30.03 s) 2023-07-14T18:18:06.411 INFO:teuthology.orchestra.run.smithi133.stderr:36700000 of 50000000 tuples (73%) done (elapsed 82.20 s, remaining 29.79 s) 2023-07-14T18:18:06.580 INFO:teuthology.orchestra.run.smithi133.stderr:36800000 of 50000000 tuples (73%) done (elapsed 82.37 s, remaining 29.54 s) 2023-07-14T18:18:06.714 INFO:teuthology.orchestra.run.smithi133.stderr:36900000 of 50000000 tuples (73%) done (elapsed 82.50 s, remaining 29.29 s) 2023-07-14T18:18:06.877 INFO:teuthology.orchestra.run.smithi133.stderr:37000000 of 50000000 tuples (74%) done (elapsed 82.66 s, remaining 29.04 s) 2023-07-14T18:18:07.213 INFO:teuthology.orchestra.run.smithi133.stderr:37100000 of 50000000 tuples (74%) done (elapsed 83.00 s, remaining 28.86 s) 2023-07-14T18:18:07.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:07 smithi161 ceph-mon[128993]: pgmap v391: 129 pgs: 129 active+clean; 5.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 121 MiB/s wr, 60 op/s 2023-07-14T18:18:07.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:07 smithi171 ceph-mon[129241]: pgmap v391: 129 pgs: 129 active+clean; 5.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 121 MiB/s wr, 60 op/s 2023-07-14T18:18:07.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:07 smithi133 ceph-mon[126384]: pgmap v391: 129 pgs: 129 active+clean; 5.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 121 MiB/s wr, 60 op/s 2023-07-14T18:18:07.802 INFO:teuthology.orchestra.run.smithi133.stderr:37200000 of 50000000 tuples (74%) done (elapsed 83.59 s, remaining 28.76 s) 2023-07-14T18:18:08.436 INFO:teuthology.orchestra.run.smithi133.stderr:37300000 of 50000000 tuples (74%) done (elapsed 84.22 s, remaining 28.68 s) 2023-07-14T18:18:08.556 INFO:teuthology.orchestra.run.smithi133.stderr:37400000 of 50000000 tuples (74%) done (elapsed 84.34 s, remaining 28.41 s) 2023-07-14T18:18:08.757 INFO:teuthology.orchestra.run.smithi133.stderr:37500000 of 50000000 tuples (75%) done (elapsed 84.54 s, remaining 28.18 s) 2023-07-14T18:18:08.952 INFO:teuthology.orchestra.run.smithi133.stderr:37600000 of 50000000 tuples (75%) done (elapsed 84.74 s, remaining 27.95 s) 2023-07-14T18:18:09.091 INFO:teuthology.orchestra.run.smithi133.stderr:37700000 of 50000000 tuples (75%) done (elapsed 84.88 s, remaining 27.69 s) 2023-07-14T18:18:09.308 INFO:teuthology.orchestra.run.smithi133.stderr:37800000 of 50000000 tuples (75%) done (elapsed 85.09 s, remaining 27.46 s) 2023-07-14T18:18:09.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:09 smithi161 ceph-mon[128993]: pgmap v392: 129 pgs: 129 active+clean; 5.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 130 MiB/s wr, 66 op/s 2023-07-14T18:18:09.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:09 smithi171 ceph-mon[129241]: pgmap v392: 129 pgs: 129 active+clean; 5.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 130 MiB/s wr, 66 op/s 2023-07-14T18:18:09.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:09 smithi133 ceph-mon[126384]: pgmap v392: 129 pgs: 129 active+clean; 5.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 130 MiB/s wr, 66 op/s 2023-07-14T18:18:09.468 INFO:teuthology.orchestra.run.smithi133.stderr:37900000 of 50000000 tuples (75%) done (elapsed 85.25 s, remaining 27.22 s) 2023-07-14T18:18:09.606 INFO:teuthology.orchestra.run.smithi133.stderr:38000000 of 50000000 tuples (76%) done (elapsed 85.39 s, remaining 26.97 s) 2023-07-14T18:18:09.670 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:18:09.775 INFO:teuthology.orchestra.run.smithi133.stderr:38100000 of 50000000 tuples (76%) done (elapsed 85.56 s, remaining 26.72 s) 2023-07-14T18:18:09.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.844+0000 7fc07a5ff700 1 -- 172.21.15.133:0/4289238451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0741058d0 msgr2=0x7fc074107d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:18:09.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.844+0000 7fc07a5ff700 1 --2- 172.21.15.133:0/4289238451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0741058d0 0x7fc074107d30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009f90 tx=0x7fc06c00b330 comp rx=0 tx=0).stop 2023-07-14T18:18:09.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.844+0000 7fc07a5ff700 1 -- 172.21.15.133:0/4289238451 shutdown_connections 2023-07-14T18:18:09.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.844+0000 7fc07a5ff700 1 --2- 172.21.15.133:0/4289238451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0741058d0 0x7fc074107d30 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.844+0000 7fc07a5ff700 1 --2- 172.21.15.133:0/4289238451 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc074100690 0x7fc074102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.844+0000 7fc07a5ff700 1 --2- 172.21.15.133:0/4289238451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc074102fb0 0x7fc074105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.844+0000 7fc07a5ff700 1 -- 172.21.15.133:0/4289238451 >> 172.21.15.133:0/4289238451 conn(0x7fc0740fa130 msgr2=0x7fc0740fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:18:09.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.844+0000 7fc07a5ff700 1 -- 172.21.15.133:0/4289238451 shutdown_connections 2023-07-14T18:18:09.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.844+0000 7fc07a5ff700 1 -- 172.21.15.133:0/4289238451 wait complete. 2023-07-14T18:18:09.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.846+0000 7fc07a5ff700 1 Processor -- start 2023-07-14T18:18:09.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.846+0000 7fc07a5ff700 1 -- start start 2023-07-14T18:18:09.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.846+0000 7fc07a5ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc074100690 0x7fc074195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:09.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.846+0000 7fc07a5ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc074102fb0 0x7fc074196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:09.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.847+0000 7fc07a5ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc0741058d0 0x7fc07419d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:09.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.847+0000 7fc07a5ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07419d650 con 0x7fc074102fb0 2023-07-14T18:18:09.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.847+0000 7fc07a5ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07419d790 con 0x7fc074100690 2023-07-14T18:18:09.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.847+0000 7fc07a5ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07419d920 con 0x7fc0741058d0 2023-07-14T18:18:09.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.847+0000 7fc073fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc074100690 0x7fc074195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:09.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.847+0000 7fc073fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc074100690 0x7fc074195ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:54838/0 (socket says 172.21.15.133:54838) 2023-07-14T18:18:09.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.847+0000 7fc073fff700 1 -- 172.21.15.133:0/3638562432 learned_addr learned my addr 172.21.15.133:0/3638562432 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:18:09.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.847+0000 7fc0737fe700 1 --2- 172.21.15.133:0/3638562432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc074102fb0 0x7fc074196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:09.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.847+0000 7fc078b9c700 1 --2- 172.21.15.133:0/3638562432 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc0741058d0 0x7fc07419d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:09.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.847+0000 7fc073fff700 1 -- 172.21.15.133:0/3638562432 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc0741058d0 msgr2=0x7fc07419d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:18:09.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.847+0000 7fc073fff700 1 --2- 172.21.15.133:0/3638562432 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc0741058d0 0x7fc07419d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.847+0000 7fc073fff700 1 -- 172.21.15.133:0/3638562432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc074102fb0 msgr2=0x7fc074196010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:18:09.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.847+0000 7fc073fff700 1 --2- 172.21.15.133:0/3638562432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc074102fb0 0x7fc074196010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.847+0000 7fc073fff700 1 -- 172.21.15.133:0/3638562432 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0680095f0 con 0x7fc074100690 2023-07-14T18:18:09.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.848+0000 7fc073fff700 1 --2- 172.21.15.133:0/3638562432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc074100690 0x7fc074195ad0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc06400e9c0 tx=0x7fc06400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:18:09.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.848+0000 7fc0717fa700 1 -- 172.21.15.133:0/3638562432 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0640097b0 con 0x7fc074100690 2023-07-14T18:18:09.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.848+0000 7fc07a5ff700 1 -- 172.21.15.133:0/3638562432 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc06c009c40 con 0x7fc074100690 2023-07-14T18:18:09.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.848+0000 7fc0717fa700 1 -- 172.21.15.133:0/3638562432 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc06400ee60 con 0x7fc074100690 2023-07-14T18:18:09.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.848+0000 7fc0717fa700 1 -- 172.21.15.133:0/3638562432 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0640054f0 con 0x7fc074100690 2023-07-14T18:18:09.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.848+0000 7fc07a5ff700 1 -- 172.21.15.133:0/3638562432 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc07419df80 con 0x7fc074100690 2023-07-14T18:18:09.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.852+0000 7fc0717fa700 1 -- 172.21.15.133:0/3638562432 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fc064004750 con 0x7fc074100690 2023-07-14T18:18:09.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.854+0000 7fc08df22b80 1 Processor -- start 2023-07-14T18:18:09.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.854+0000 7fc08df22b80 1 -- start start 2023-07-14T18:18:09.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.854+0000 7fc08df22b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1b4848f60 0x55e1b4849370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:09.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.854+0000 7fc08df22b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1b48498b0 0x55e1b4849ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:09.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.854+0000 7fc08df22b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1b484a4b0 0x55e1b484a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:09.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.854+0000 7fc08df22b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b474dcb0 con 0x55e1b484a4b0 2023-07-14T18:18:09.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.854+0000 7fc08df22b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b47a0a40 con 0x55e1b4848f60 2023-07-14T18:18:09.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.854+0000 7fc08df22b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b4799ed0 con 0x55e1b48498b0 2023-07-14T18:18:09.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.854+0000 7fc078b9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1b4848f60 0x55e1b4849370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:09.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.855+0000 7fc073fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1b48498b0 0x55e1b4849ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:09.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.855+0000 7fc078b9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1b4848f60 0x55e1b4849370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:54846/0 (socket says 172.21.15.133:54846) 2023-07-14T18:18:09.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.855+0000 7fc078b9c700 1 -- 172.21.15.133:0/2768686580 learned_addr learned my addr 172.21.15.133:0/2768686580 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:18:09.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.855+0000 7fc0737fe700 1 --2- 172.21.15.133:0/2768686580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1b484a4b0 0x55e1b484a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:09.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.855+0000 7fc078b9c700 1 -- 172.21.15.133:0/2768686580 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1b48498b0 msgr2=0x55e1b4849ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:18:09.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.855+0000 7fc0717fa700 1 --2- 172.21.15.133:0/3638562432 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc05c0a1150 0x7fc05c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:09.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.855+0000 7fc078b9c700 1 --2- 172.21.15.133:0/2768686580 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1b48498b0 0x55e1b4849ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.855+0000 7fc078b9c700 1 -- 172.21.15.133:0/2768686580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1b484a4b0 msgr2=0x55e1b484a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:18:09.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.855+0000 7fc078b9c700 1 --2- 172.21.15.133:0/2768686580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1b484a4b0 0x55e1b484a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.855+0000 7fc078b9c700 1 -- 172.21.15.133:0/2768686580 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1b45c7de0 con 0x55e1b4848f60 2023-07-14T18:18:09.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.855+0000 7fc0737fe700 1 --2- 172.21.15.133:0/2768686580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1b484a4b0 0x55e1b484a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:18:09.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.855+0000 7fc073fff700 1 --2- 172.21.15.133:0/2768686580 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1b48498b0 0x55e1b4849ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T18:18:09.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.855+0000 7fc0717fa700 1 -- 172.21.15.133:0/3638562432 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc064014070 con 0x7fc074100690 2023-07-14T18:18:09.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.856+0000 7fc078b9c700 1 --2- 172.21.15.133:0/3638562432 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc05c0a1150 0x7fc05c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:09.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.856+0000 7fc078b9c700 1 --2- 172.21.15.133:0/2768686580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1b4848f60 0x55e1b4849370 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009b40 tx=0x7fc06c009b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b82a94630661ae1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:18:09.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.856+0000 7fc05a7fc700 1 -- 172.21.15.133:0/2768686580 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c00e040 con 0x55e1b4848f60 2023-07-14T18:18:09.906 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.856+0000 7fc05a7fc700 1 -- 172.21.15.133:0/2768686580 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc06c007d10 con 0x55e1b4848f60 2023-07-14T18:18:09.906 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.856+0000 7fc05a7fc700 1 -- 172.21.15.133:0/2768686580 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c005040 con 0x55e1b4848f60 2023-07-14T18:18:09.906 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.857+0000 7fc08df22b80 1 -- 172.21.15.133:0/2768686580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1b4848f60 msgr2=0x55e1b4849370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:18:09.906 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.857+0000 7fc078b9c700 1 --2- 172.21.15.133:0/3638562432 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc05c0a1150 0x7fc05c0a3600 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0061e0 tx=0x7fc06c005400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:18:09.906 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.857+0000 7fc08df22b80 1 --2- 172.21.15.133:0/2768686580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1b4848f60 0x55e1b4849370 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009b40 tx=0x7fc06c009b70 comp rx=0 tx=0).stop 2023-07-14T18:18:09.906 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.857+0000 7fc08df22b80 1 -- 172.21.15.133:0/2768686580 shutdown_connections 2023-07-14T18:18:09.906 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.857+0000 7fc08df22b80 1 --2- 172.21.15.133:0/2768686580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1b4848f60 0x55e1b4849370 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.906 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.857+0000 7fc08df22b80 1 --2- 172.21.15.133:0/2768686580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1b484a4b0 0x55e1b484a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.857+0000 7fc08df22b80 1 --2- 172.21.15.133:0/2768686580 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1b48498b0 0x55e1b4849ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.857+0000 7fc08df22b80 1 -- 172.21.15.133:0/2768686580 >> 172.21.15.133:0/2768686580 conn(0x55e1b483e680 msgr2=0x55e1b48439d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:18:09.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.857+0000 7fc08df22b80 1 -- 172.21.15.133:0/2768686580 shutdown_connections 2023-07-14T18:18:09.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.857+0000 7fc08df22b80 1 -- 172.21.15.133:0/2768686580 wait complete. 2023-07-14T18:18:09.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.858+0000 7fc08df22b80 1 -- start start 2023-07-14T18:18:09.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.858+0000 7fc08df22b80 1 Processor -- start 2023-07-14T18:18:09.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.858+0000 7fc08df22b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1b4848f60 0x55e1b4852880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:09.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.858+0000 7fc08df22b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1b48498b0 0x55e1b4852dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:09.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.858+0000 7fc08df22b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1b484a4b0 0x55e1b4859e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:09.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.858+0000 7fc08df22b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b4739ac0 con 0x55e1b48498b0 2023-07-14T18:18:09.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.858+0000 7fc08df22b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b47397c0 con 0x55e1b4848f60 2023-07-14T18:18:09.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.858+0000 7fc08df22b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b461d920 con 0x55e1b484a4b0 2023-07-14T18:18:09.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.858+0000 7fc08df22b80 10 client.0 _ll_register_callbacks cb 0x55e1b4737760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:18:09.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.859+0000 7fc073fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1b4848f60 0x55e1b4852880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:09.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.859+0000 7fc073fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1b4848f60 0x55e1b4852880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:54848/0 (socket says 172.21.15.133:54848) 2023-07-14T18:18:09.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.859+0000 7fc073fff700 1 -- 172.21.15.133:0/482344055 learned_addr learned my addr 172.21.15.133:0/482344055 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:18:09.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.859+0000 7fc078b9c700 1 --2- 172.21.15.133:0/482344055 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1b484a4b0 0x55e1b4859e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:09.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.859+0000 7fc0737fe700 1 --2- 172.21.15.133:0/482344055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1b48498b0 0x55e1b4852dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:09.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.859+0000 7fc073fff700 1 -- 172.21.15.133:0/482344055 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1b484a4b0 msgr2=0x55e1b4859e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:18:09.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.859+0000 7fc073fff700 1 --2- 172.21.15.133:0/482344055 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1b484a4b0 0x55e1b4859e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.859+0000 7fc073fff700 1 -- 172.21.15.133:0/482344055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1b48498b0 msgr2=0x55e1b4852dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:18:09.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.859+0000 7fc073fff700 1 --2- 172.21.15.133:0/482344055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1b48498b0 0x55e1b4852dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.859+0000 7fc073fff700 1 -- 172.21.15.133:0/482344055 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc074108920 con 0x55e1b4848f60 2023-07-14T18:18:09.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.859+0000 7fc0737fe700 1 --2- 172.21.15.133:0/482344055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1b48498b0 0x55e1b4852dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T18:18:09.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.859+0000 7fc073fff700 1 --2- 172.21.15.133:0/482344055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1b4848f60 0x55e1b4852880 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc06400c5b0 tx=0x7fc0640a7af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34efceae0416054 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:18:09.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.859+0000 7fc0537fe700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:18:09.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.859+0000 7fc0537fe700 1 -- 172.21.15.133:0/482344055 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc064099f40 con 0x55e1b4848f60 2023-07-14T18:18:09.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.859+0000 7fc08df22b80 1 -- 172.21.15.133:0/482344055 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e1b46f34f0 con 0x55e1b4848f60 2023-07-14T18:18:09.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.859+0000 7fc0537fe700 1 -- 172.21.15.133:0/482344055 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc06409df70 con 0x55e1b4848f60 2023-07-14T18:18:09.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.860+0000 7fc0537fe700 1 -- 172.21.15.133:0/482344055 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0640a2390 con 0x55e1b4848f60 2023-07-14T18:18:09.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.860+0000 7fc08df22b80 10 client.25640 fetch_fsmap learned FSMap version 30 2023-07-14T18:18:09.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.860+0000 7fc08df22b80 1 -- 172.21.15.133:0/482344055 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e1b45b0c60 con 0x55e1b4848f60 2023-07-14T18:18:09.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.860+0000 7fc0537fe700 1 -- 172.21.15.133:0/482344055 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e1b46f34f0 con 0x55e1b4848f60 2023-07-14T18:18:09.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.860+0000 7fc0537fe700 1 -- 172.21.15.133:0/482344055 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc0640a8040 con 0x55e1b4848f60 2023-07-14T18:18:09.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.860+0000 7fc08df22b80 10 client.25640 fetch_fsmap finished waiting for FSMap version 30 2023-07-14T18:18:09.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.860+0000 7fc08df22b80 10 client.25640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:18:09.914 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.860+0000 7fc08df22b80 20 client.25640 populate_metadata read hostname 'smithi133' 2023-07-14T18:18:09.914 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.860+0000 7fc08df22b80 1 --2- 172.21.15.133:0/482344055 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55e1b485e5a0 0x55e1b48609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:09.914 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.860+0000 7fc08df22b80 4 client.25640 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:18:09.914 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.860+0000 7fc08df22b80 1 -- 172.21.15.133:0/482344055 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1b4817da0 con 0x55e1b485e5a0 2023-07-14T18:18:09.914 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.861+0000 7fc0737fe700 1 --2- 172.21.15.133:0/482344055 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55e1b485e5a0 0x55e1b48609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:09.914 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.862+0000 7fc0737fe700 1 --2- 172.21.15.133:0/482344055 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55e1b485e5a0 0x55e1b48609b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5073f6f6990d98d server_cookie=8770383293346c91 in_seq=0 out_seq=0 2023-07-14T18:18:09.914 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.862+0000 7fc0537fe700 10 client.25640 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:18:09.914 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.863+0000 7fc0537fe700 1 -- 172.21.15.133:0/482344055 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55e1b4817da0 con 0x55e1b485e5a0 2023-07-14T18:18:09.915 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.863+0000 7fc0537fe700 10 client.25640 handle_command_reply: tid=0 2023-07-14T18:18:09.915 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.863+0000 7fc08df22b80 1 client.25640 shutdown 2023-07-14T18:18:09.915 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.863+0000 7fc08df22b80 1 -- 172.21.15.133:0/482344055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1b4848f60 msgr2=0x55e1b4852880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:18:09.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.863+0000 7fc08df22b80 1 --2- 172.21.15.133:0/482344055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1b4848f60 0x55e1b4852880 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc06400c5b0 tx=0x7fc0640a7af0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.863+0000 7fc08df22b80 1 -- 172.21.15.133:0/482344055 shutdown_connections 2023-07-14T18:18:09.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.863+0000 7fc08df22b80 1 --2- 172.21.15.133:0/482344055 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55e1b485e5a0 0x55e1b48609b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.863+0000 7fc08df22b80 1 --2- 172.21.15.133:0/482344055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1b4848f60 0x55e1b4852880 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.863+0000 7fc08df22b80 1 --2- 172.21.15.133:0/482344055 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1b484a4b0 0x55e1b4859e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.863+0000 7fc0537fe700 0 client.25640 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:18:09.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.863+0000 7fc08df22b80 1 --2- 172.21.15.133:0/482344055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1b48498b0 0x55e1b4852dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.863+0000 7fc08df22b80 1 -- 172.21.15.133:0/482344055 >> 172.21.15.133:0/482344055 conn(0x55e1b483e680 msgr2=0x55e1b4842650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:18:09.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.863+0000 7fc08df22b80 1 -- 172.21.15.133:0/482344055 shutdown_connections 2023-07-14T18:18:09.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.863+0000 7fc08df22b80 1 -- 172.21.15.133:0/482344055 wait complete. 2023-07-14T18:18:09.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.863+0000 7fc08df22b80 20 client.25640 trim_cache size 0 max 16384 2023-07-14T18:18:09.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.881+0000 7fc08df22b80 1 Processor -- start 2023-07-14T18:18:09.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.881+0000 7fc08df22b80 1 -- start start 2023-07-14T18:18:09.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.882+0000 7fc08df22b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1b4848f60 0x55e1b4851f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:09.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.882+0000 7fc08df22b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1b48498b0 0x55e1b4850780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:09.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.882+0000 7fc08df22b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1b484a4b0 0x55e1b4850cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:09.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.882+0000 7fc08df22b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b485ba00 con 0x55e1b4848f60 2023-07-14T18:18:09.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.882+0000 7fc08df22b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b4844cc0 con 0x55e1b484a4b0 2023-07-14T18:18:09.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.882+0000 7fc08df22b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b4851d30 con 0x55e1b48498b0 2023-07-14T18:18:09.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.882+0000 7fc0737fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1b48498b0 0x55e1b4850780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:09.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.882+0000 7fc078b9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1b484a4b0 0x55e1b4850cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:09.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.882+0000 7fc073fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1b4848f60 0x55e1b4851f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:09.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.882+0000 7fc078b9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1b484a4b0 0x55e1b4850cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:54850/0 (socket says 172.21.15.133:54850) 2023-07-14T18:18:09.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.882+0000 7fc078b9c700 1 -- 172.21.15.133:0/1713469848 learned_addr learned my addr 172.21.15.133:0/1713469848 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:18:09.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.882+0000 7fc0737fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1b48498b0 0x55e1b4850780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:49498/0 (socket says 172.21.15.133:49498) 2023-07-14T18:18:09.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.882+0000 7fc078b9c700 1 -- 172.21.15.133:0/1713469848 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1b48498b0 msgr2=0x55e1b4850780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:18:09.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.882+0000 7fc078b9c700 1 --2- 172.21.15.133:0/1713469848 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1b48498b0 0x55e1b4850780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.882+0000 7fc078b9c700 1 -- 172.21.15.133:0/1713469848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1b4848f60 msgr2=0x55e1b4851f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:18:09.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.882+0000 7fc078b9c700 1 --2- 172.21.15.133:0/1713469848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1b4848f60 0x55e1b4851f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.882+0000 7fc078b9c700 1 -- 172.21.15.133:0/1713469848 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1b45ee250 con 0x55e1b484a4b0 2023-07-14T18:18:09.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.883+0000 7fc078b9c700 1 --2- 172.21.15.133:0/1713469848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1b484a4b0 0x55e1b4850cc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009ba0 tx=0x7fc06c02e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20ee4e96f71a6014 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:18:09.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.883+0000 7fc05affd700 1 -- 172.21.15.133:0/1713469848 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c00e040 con 0x55e1b484a4b0 2023-07-14T18:18:09.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.883+0000 7fc05affd700 1 -- 172.21.15.133:0/1713469848 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc06c00b680 con 0x55e1b484a4b0 2023-07-14T18:18:09.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.883+0000 7fc05affd700 1 -- 172.21.15.133:0/1713469848 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c003e00 con 0x55e1b484a4b0 2023-07-14T18:18:09.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.884+0000 7fc08df22b80 1 -- 172.21.15.133:0/1713469848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1b484a4b0 msgr2=0x55e1b4850cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:18:09.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.884+0000 7fc08df22b80 1 --2- 172.21.15.133:0/1713469848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1b484a4b0 0x55e1b4850cc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009ba0 tx=0x7fc06c02e000 comp rx=0 tx=0).stop 2023-07-14T18:18:09.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.884+0000 7fc08df22b80 1 -- 172.21.15.133:0/1713469848 shutdown_connections 2023-07-14T18:18:09.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.884+0000 7fc08df22b80 1 --2- 172.21.15.133:0/1713469848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1b484a4b0 0x55e1b4850cc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.884+0000 7fc08df22b80 1 --2- 172.21.15.133:0/1713469848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1b4848f60 0x55e1b4851f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.884+0000 7fc08df22b80 1 --2- 172.21.15.133:0/1713469848 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1b48498b0 0x55e1b4850780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.884+0000 7fc08df22b80 1 -- 172.21.15.133:0/1713469848 >> 172.21.15.133:0/1713469848 conn(0x55e1b483e680 msgr2=0x55e1b484d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:18:09.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.884+0000 7fc08df22b80 1 -- 172.21.15.133:0/1713469848 shutdown_connections 2023-07-14T18:18:09.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.884+0000 7fc08df22b80 1 -- 172.21.15.133:0/1713469848 wait complete. 2023-07-14T18:18:09.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.884+0000 7fc08df22b80 1 -- start start 2023-07-14T18:18:09.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.884+0000 7fc08df22b80 1 Processor -- start 2023-07-14T18:18:09.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.885+0000 7fc08df22b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1b4848f60 0x55e1b48ae480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:09.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.885+0000 7fc08df22b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1b48498b0 0x55e1b48ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:09.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.885+0000 7fc08df22b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1b484a4b0 0x55e1b48b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:09.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.885+0000 7fc08df22b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b48b5e30 con 0x55e1b4848f60 2023-07-14T18:18:09.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.885+0000 7fc08df22b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b48b5f70 con 0x55e1b48498b0 2023-07-14T18:18:09.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.885+0000 7fc08df22b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b48b6100 con 0x55e1b484a4b0 2023-07-14T18:18:09.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.885+0000 7fc08df22b80 10 client.0 _ll_register_callbacks cb 0x7fc0640998a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:18:09.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.885+0000 7fc078b9c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1b484a4b0 0x55e1b48b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:09.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.885+0000 7fc0737fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1b48498b0 0x55e1b48ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:09.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.885+0000 7fc073fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1b4848f60 0x55e1b48ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:09.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.885+0000 7fc078b9c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1b484a4b0 0x55e1b48b58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:49506/0 (socket says 172.21.15.133:49506) 2023-07-14T18:18:09.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.885+0000 7fc078b9c700 1 -- 172.21.15.133:0/601891042 learned_addr learned my addr 172.21.15.133:0/601891042 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:18:09.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.886+0000 7fc078b9c700 1 -- 172.21.15.133:0/601891042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1b48498b0 msgr2=0x55e1b48ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:18:09.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.886+0000 7fc078b9c700 1 --2- 172.21.15.133:0/601891042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1b48498b0 0x55e1b48ae9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.886+0000 7fc078b9c700 1 -- 172.21.15.133:0/601891042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1b4848f60 msgr2=0x55e1b48ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:18:09.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.886+0000 7fc078b9c700 1 --2- 172.21.15.133:0/601891042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1b4848f60 0x55e1b48ae480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.886+0000 7fc078b9c700 1 -- 172.21.15.133:0/601891042 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc06c02b070 con 0x55e1b484a4b0 2023-07-14T18:18:09.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.886+0000 7fc078b9c700 1 --2- 172.21.15.133:0/601891042 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1b484a4b0 0x55e1b48b58f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc06c007bf0 tx=0x7fc06c005050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9dd4dfb38e3ad2a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:18:09.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.886+0000 7fc0537fe700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:18:09.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.886+0000 7fc0537fe700 1 -- 172.21.15.133:0/601891042 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c00e040 con 0x55e1b484a4b0 2023-07-14T18:18:09.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.886+0000 7fc08df22b80 1 -- 172.21.15.133:0/601891042 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc06c007d10 con 0x55e1b484a4b0 2023-07-14T18:18:09.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.886+0000 7fc0537fe700 1 -- 172.21.15.133:0/601891042 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc06c0075e0 con 0x55e1b484a4b0 2023-07-14T18:18:09.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.886+0000 7fc0537fe700 1 -- 172.21.15.133:0/601891042 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc06c003b80 con 0x55e1b484a4b0 2023-07-14T18:18:09.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.886+0000 7fc08df22b80 10 client.25675 fetch_fsmap learned FSMap version 30 2023-07-14T18:18:09.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.886+0000 7fc08df22b80 1 -- 172.21.15.133:0/601891042 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e1b45c62b0 con 0x55e1b484a4b0 2023-07-14T18:18:09.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.887+0000 7fc0537fe700 1 -- 172.21.15.133:0/601891042 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc06c007d10 con 0x55e1b484a4b0 2023-07-14T18:18:09.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.887+0000 7fc0537fe700 1 -- 172.21.15.133:0/601891042 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc06c02f070 con 0x55e1b484a4b0 2023-07-14T18:18:09.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.887+0000 7fc08df22b80 10 client.25675 fetch_fsmap finished waiting for FSMap version 30 2023-07-14T18:18:09.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.887+0000 7fc08df22b80 10 client.25675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:18:09.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.887+0000 7fc08df22b80 20 client.25675 populate_metadata read hostname 'smithi133' 2023-07-14T18:18:09.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.887+0000 7fc08df22b80 1 --2- 172.21.15.133:0/601891042 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55e1b48b77f0 0x55e1b48b9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:09.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.887+0000 7fc08df22b80 4 client.25675 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:18:09.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.887+0000 7fc08df22b80 1 -- 172.21.15.133:0/601891042 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e1b4792eb0 con 0x55e1b48b77f0 2023-07-14T18:18:09.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.887+0000 7fc073fff700 1 --2- 172.21.15.133:0/601891042 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55e1b48b77f0 0x55e1b48b9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:09.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.887+0000 7fc0537fe700 10 client.25675 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:18:09.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.887+0000 7fc073fff700 1 --2- 172.21.15.133:0/601891042 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55e1b48b77f0 0x55e1b48b9c00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=158885de95f7e58e server_cookie=88d4713fc35e92c0 in_seq=0 out_seq=0 2023-07-14T18:18:09.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.888+0000 7fc0537fe700 1 -- 172.21.15.133:0/601891042 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55e1b4792eb0 con 0x55e1b48b77f0 2023-07-14T18:18:09.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.888+0000 7fc0537fe700 10 client.25675 handle_command_reply: tid=0 2023-07-14T18:18:09.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.888+0000 7fc08df22b80 1 client.25675 shutdown 2023-07-14T18:18:09.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.888+0000 7fc08df22b80 1 -- 172.21.15.133:0/601891042 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1b484a4b0 msgr2=0x55e1b48b58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:18:09.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.888+0000 7fc08df22b80 1 --2- 172.21.15.133:0/601891042 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1b484a4b0 0x55e1b48b58f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc06c007bf0 tx=0x7fc06c005050 comp rx=0 tx=0).stop 2023-07-14T18:18:09.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.888+0000 7fc08df22b80 1 -- 172.21.15.133:0/601891042 shutdown_connections 2023-07-14T18:18:09.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.888+0000 7fc08df22b80 1 --2- 172.21.15.133:0/601891042 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55e1b48b77f0 0x55e1b48b9c00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.888+0000 7fc08df22b80 1 --2- 172.21.15.133:0/601891042 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1b484a4b0 0x55e1b48b58f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.888+0000 7fc0537fe700 0 client.25675 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:18:09.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.888+0000 7fc08df22b80 1 --2- 172.21.15.133:0/601891042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1b4848f60 0x55e1b48ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.888+0000 7fc08df22b80 1 --2- 172.21.15.133:0/601891042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1b48498b0 0x55e1b48ae9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.888+0000 7fc08df22b80 1 -- 172.21.15.133:0/601891042 >> 172.21.15.133:0/601891042 conn(0x55e1b483e680 msgr2=0x55e1b484d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:18:09.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.888+0000 7fc08df22b80 1 -- 172.21.15.133:0/601891042 shutdown_connections 2023-07-14T18:18:09.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.888+0000 7fc08df22b80 1 -- 172.21.15.133:0/601891042 wait complete. 2023-07-14T18:18:09.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.888+0000 7fc08df22b80 20 client.25675 trim_cache size 0 max 16384 2023-07-14T18:18:09.944 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:18:09.944 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-14T18:18:09.944 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:18:09.944 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:18:09.944 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:18:09.944 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:18:09.945 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:18:09.945 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:18:09.945 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:18:09.945 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:18:09.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.889+0000 7fc07a5ff700 1 -- 172.21.15.133:0/3638562432 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc05c0a1150 msgr2=0x7fc05c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:18:09.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.889+0000 7fc07a5ff700 1 --2- 172.21.15.133:0/3638562432 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc05c0a1150 0x7fc05c0a3600 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0061e0 tx=0x7fc06c005400 comp rx=0 tx=0).stop 2023-07-14T18:18:09.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.889+0000 7fc07a5ff700 1 -- 172.21.15.133:0/3638562432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc074100690 msgr2=0x7fc074195ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:18:09.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.889+0000 7fc07a5ff700 1 --2- 172.21.15.133:0/3638562432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc074100690 0x7fc074195ad0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc06400e9c0 tx=0x7fc06400ecd0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.889+0000 7fc07a5ff700 1 -- 172.21.15.133:0/3638562432 shutdown_connections 2023-07-14T18:18:09.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.889+0000 7fc07a5ff700 1 --2- 172.21.15.133:0/3638562432 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc05c0a1150 0x7fc05c0a3600 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.889+0000 7fc07a5ff700 1 --2- 172.21.15.133:0/3638562432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc074100690 0x7fc074195ad0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.889+0000 7fc07a5ff700 1 --2- 172.21.15.133:0/3638562432 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc0741058d0 0x7fc07419d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.889+0000 7fc07a5ff700 1 --2- 172.21.15.133:0/3638562432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc074102fb0 0x7fc074196010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:09.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.889+0000 7fc07a5ff700 1 -- 172.21.15.133:0/3638562432 >> 172.21.15.133:0/3638562432 conn(0x7fc0740fa130 msgr2=0x7fc0741012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:18:09.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.889+0000 7fc07a5ff700 1 -- 172.21.15.133:0/3638562432 shutdown_connections 2023-07-14T18:18:09.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:09.889+0000 7fc07a5ff700 1 -- 172.21.15.133:0/3638562432 wait complete. 2023-07-14T18:18:09.957 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:18:09.989 INFO:teuthology.orchestra.run.smithi133.stderr:38200000 of 50000000 tuples (76%) done (elapsed 85.77 s, remaining 26.50 s) 2023-07-14T18:18:10.144 INFO:teuthology.orchestra.run.smithi133.stderr:38300000 of 50000000 tuples (76%) done (elapsed 85.93 s, remaining 26.25 s) 2023-07-14T18:18:10.304 INFO:teuthology.orchestra.run.smithi133.stderr:38400000 of 50000000 tuples (76%) done (elapsed 86.09 s, remaining 26.01 s) 2023-07-14T18:18:10.509 INFO:teuthology.orchestra.run.smithi133.stderr:38500000 of 50000000 tuples (77%) done (elapsed 86.29 s, remaining 25.78 s) 2023-07-14T18:18:10.703 INFO:teuthology.orchestra.run.smithi133.stderr:38600000 of 50000000 tuples (77%) done (elapsed 86.49 s, remaining 25.54 s) 2023-07-14T18:18:10.884 INFO:teuthology.orchestra.run.smithi133.stderr:38700000 of 50000000 tuples (77%) done (elapsed 86.67 s, remaining 25.31 s) 2023-07-14T18:18:10.999 INFO:teuthology.orchestra.run.smithi133.stderr:38800000 of 50000000 tuples (77%) done (elapsed 86.78 s, remaining 25.05 s) 2023-07-14T18:18:11.165 INFO:teuthology.orchestra.run.smithi133.stderr:38900000 of 50000000 tuples (77%) done (elapsed 86.95 s, remaining 24.81 s) 2023-07-14T18:18:11.339 INFO:teuthology.orchestra.run.smithi133.stderr:39000000 of 50000000 tuples (78%) done (elapsed 87.12 s, remaining 24.57 s) 2023-07-14T18:18:11.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:11 smithi161 ceph-mon[128993]: pgmap v393: 129 pgs: 129 active+clean; 5.3 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 103 MiB/s wr, 49 op/s 2023-07-14T18:18:11.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:11 smithi171 ceph-mon[129241]: pgmap v393: 129 pgs: 129 active+clean; 5.3 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 103 MiB/s wr, 49 op/s 2023-07-14T18:18:11.411 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:11 smithi133 ceph-mon[126384]: pgmap v393: 129 pgs: 129 active+clean; 5.3 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 103 MiB/s wr, 49 op/s 2023-07-14T18:18:11.479 INFO:teuthology.orchestra.run.smithi133.stderr:39100000 of 50000000 tuples (78%) done (elapsed 87.26 s, remaining 24.33 s) 2023-07-14T18:18:11.690 INFO:teuthology.orchestra.run.smithi133.stderr:39200000 of 50000000 tuples (78%) done (elapsed 87.48 s, remaining 24.10 s) 2023-07-14T18:18:11.851 INFO:teuthology.orchestra.run.smithi133.stderr:39300000 of 50000000 tuples (78%) done (elapsed 87.64 s, remaining 23.86 s) 2023-07-14T18:18:12.079 INFO:teuthology.orchestra.run.smithi133.stderr:39400000 of 50000000 tuples (78%) done (elapsed 87.87 s, remaining 23.64 s) 2023-07-14T18:18:12.187 INFO:teuthology.orchestra.run.smithi133.stderr:39500000 of 50000000 tuples (79%) done (elapsed 87.97 s, remaining 23.38 s) 2023-07-14T18:18:12.383 INFO:teuthology.orchestra.run.smithi133.stderr:39600000 of 50000000 tuples (79%) done (elapsed 88.17 s, remaining 23.16 s) 2023-07-14T18:18:12.584 INFO:teuthology.orchestra.run.smithi133.stderr:39700000 of 50000000 tuples (79%) done (elapsed 88.37 s, remaining 22.93 s) 2023-07-14T18:18:12.764 INFO:teuthology.orchestra.run.smithi133.stderr:39800000 of 50000000 tuples (79%) done (elapsed 88.55 s, remaining 22.69 s) 2023-07-14T18:18:12.880 INFO:teuthology.orchestra.run.smithi133.stderr:39900000 of 50000000 tuples (79%) done (elapsed 88.67 s, remaining 22.44 s) 2023-07-14T18:18:13.069 INFO:teuthology.orchestra.run.smithi133.stderr:40000000 of 50000000 tuples (80%) done (elapsed 88.85 s, remaining 22.21 s) 2023-07-14T18:18:13.247 INFO:teuthology.orchestra.run.smithi133.stderr:40100000 of 50000000 tuples (80%) done (elapsed 89.03 s, remaining 21.98 s) 2023-07-14T18:18:13.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:13 smithi161 ceph-mon[128993]: pgmap v394: 129 pgs: 129 active+clean; 5.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 121 MiB/s wr, 53 op/s 2023-07-14T18:18:13.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:13 smithi171 ceph-mon[129241]: pgmap v394: 129 pgs: 129 active+clean; 5.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 121 MiB/s wr, 53 op/s 2023-07-14T18:18:13.385 INFO:teuthology.orchestra.run.smithi133.stderr:40200000 of 50000000 tuples (80%) done (elapsed 89.17 s, remaining 21.74 s) 2023-07-14T18:18:13.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:13 smithi133 ceph-mon[126384]: pgmap v394: 129 pgs: 129 active+clean; 5.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 121 MiB/s wr, 53 op/s 2023-07-14T18:18:13.590 INFO:teuthology.orchestra.run.smithi133.stderr:40300000 of 50000000 tuples (80%) done (elapsed 89.38 s, remaining 21.51 s) 2023-07-14T18:18:13.774 INFO:teuthology.orchestra.run.smithi133.stderr:40400000 of 50000000 tuples (80%) done (elapsed 89.56 s, remaining 21.28 s) 2023-07-14T18:18:13.914 INFO:teuthology.orchestra.run.smithi133.stderr:40500000 of 50000000 tuples (81%) done (elapsed 89.70 s, remaining 21.04 s) 2023-07-14T18:18:14.133 INFO:teuthology.orchestra.run.smithi133.stderr:40600000 of 50000000 tuples (81%) done (elapsed 89.92 s, remaining 20.82 s) 2023-07-14T18:18:14.298 INFO:teuthology.orchestra.run.smithi133.stderr:40700000 of 50000000 tuples (81%) done (elapsed 90.08 s, remaining 20.58 s) 2023-07-14T18:18:14.491 INFO:teuthology.orchestra.run.smithi133.stderr:40800000 of 50000000 tuples (81%) done (elapsed 90.28 s, remaining 20.36 s) 2023-07-14T18:18:14.691 INFO:teuthology.orchestra.run.smithi133.stderr:40900000 of 50000000 tuples (81%) done (elapsed 90.48 s, remaining 20.13 s) 2023-07-14T18:18:14.907 INFO:teuthology.orchestra.run.smithi133.stderr:41000000 of 50000000 tuples (82%) done (elapsed 90.69 s, remaining 19.91 s) 2023-07-14T18:18:15.301 INFO:teuthology.orchestra.run.smithi133.stderr:41100000 of 50000000 tuples (82%) done (elapsed 91.09 s, remaining 19.72 s) 2023-07-14T18:18:15.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:15 smithi171 ceph-mon[129241]: pgmap v395: 129 pgs: 129 active+clean; 5.7 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 119 MiB/s wr, 63 op/s 2023-07-14T18:18:15.408 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:15 smithi133 ceph-mon[126384]: pgmap v395: 129 pgs: 129 active+clean; 5.7 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 119 MiB/s wr, 63 op/s 2023-07-14T18:18:15.603 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:15 smithi161 ceph-mon[128993]: pgmap v395: 129 pgs: 129 active+clean; 5.7 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 119 MiB/s wr, 63 op/s 2023-07-14T18:18:15.733 INFO:teuthology.orchestra.run.smithi133.stderr:41200000 of 50000000 tuples (82%) done (elapsed 91.52 s, remaining 19.55 s) 2023-07-14T18:18:16.341 INFO:teuthology.orchestra.run.smithi133.stderr:41300000 of 50000000 tuples (82%) done (elapsed 92.13 s, remaining 19.41 s) 2023-07-14T18:18:16.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:16 smithi171 ceph-mon[129241]: pgmap v396: 129 pgs: 129 active+clean; 5.7 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 93 MiB/s wr, 43 op/s 2023-07-14T18:18:16.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:16 smithi133 ceph-mon[126384]: pgmap v396: 129 pgs: 129 active+clean; 5.7 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 93 MiB/s wr, 43 op/s 2023-07-14T18:18:16.603 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:16 smithi161 ceph-mon[128993]: pgmap v396: 129 pgs: 129 active+clean; 5.7 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 93 MiB/s wr, 43 op/s 2023-07-14T18:18:16.756 INFO:teuthology.orchestra.run.smithi133.stderr:41400000 of 50000000 tuples (82%) done (elapsed 92.54 s, remaining 19.22 s) 2023-07-14T18:18:16.888 INFO:teuthology.orchestra.run.smithi133.stderr:41500000 of 50000000 tuples (83%) done (elapsed 92.67 s, remaining 18.98 s) 2023-07-14T18:18:17.073 INFO:teuthology.orchestra.run.smithi133.stderr:41600000 of 50000000 tuples (83%) done (elapsed 92.86 s, remaining 18.75 s) 2023-07-14T18:18:17.274 INFO:teuthology.orchestra.run.smithi133.stderr:41700000 of 50000000 tuples (83%) done (elapsed 93.06 s, remaining 18.52 s) 2023-07-14T18:18:17.451 INFO:teuthology.orchestra.run.smithi133.stderr:41800000 of 50000000 tuples (83%) done (elapsed 93.24 s, remaining 18.29 s) 2023-07-14T18:18:17.593 INFO:teuthology.orchestra.run.smithi133.stderr:41900000 of 50000000 tuples (83%) done (elapsed 93.38 s, remaining 18.05 s) 2023-07-14T18:18:17.771 INFO:teuthology.orchestra.run.smithi133.stderr:42000000 of 50000000 tuples (84%) done (elapsed 93.56 s, remaining 17.82 s) 2023-07-14T18:18:17.994 INFO:teuthology.orchestra.run.smithi133.stderr:42100000 of 50000000 tuples (84%) done (elapsed 93.78 s, remaining 17.60 s) 2023-07-14T18:18:18.150 INFO:teuthology.orchestra.run.smithi133.stderr:42200000 of 50000000 tuples (84%) done (elapsed 93.93 s, remaining 17.36 s) 2023-07-14T18:18:18.361 INFO:teuthology.orchestra.run.smithi133.stderr:42300000 of 50000000 tuples (84%) done (elapsed 94.15 s, remaining 17.14 s) 2023-07-14T18:18:18.544 INFO:teuthology.orchestra.run.smithi133.stderr:42400000 of 50000000 tuples (84%) done (elapsed 94.33 s, remaining 16.91 s) 2023-07-14T18:18:18.740 INFO:teuthology.orchestra.run.smithi133.stderr:42500000 of 50000000 tuples (85%) done (elapsed 94.53 s, remaining 16.68 s) 2023-07-14T18:18:18.853 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:18 smithi161 ceph-mon[128993]: pgmap v397: 129 pgs: 129 active+clean; 6.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 125 MiB/s wr, 57 op/s 2023-07-14T18:18:18.868 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:18 smithi171 ceph-mon[129241]: pgmap v397: 129 pgs: 129 active+clean; 6.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 125 MiB/s wr, 57 op/s 2023-07-14T18:18:18.894 INFO:teuthology.orchestra.run.smithi133.stderr:42600000 of 50000000 tuples (85%) done (elapsed 94.68 s, remaining 16.45 s) 2023-07-14T18:18:18.908 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:18 smithi133 ceph-mon[126384]: pgmap v397: 129 pgs: 129 active+clean; 6.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 125 MiB/s wr, 57 op/s 2023-07-14T18:18:19.016 INFO:teuthology.orchestra.run.smithi133.stderr:42700000 of 50000000 tuples (85%) done (elapsed 94.80 s, remaining 16.21 s) 2023-07-14T18:18:19.196 INFO:teuthology.orchestra.run.smithi133.stderr:42800000 of 50000000 tuples (85%) done (elapsed 94.98 s, remaining 15.98 s) 2023-07-14T18:18:19.360 INFO:teuthology.orchestra.run.smithi133.stderr:42900000 of 50000000 tuples (85%) done (elapsed 95.14 s, remaining 15.75 s) 2023-07-14T18:18:19.491 INFO:teuthology.orchestra.run.smithi133.stderr:43000000 of 50000000 tuples (86%) done (elapsed 95.28 s, remaining 15.51 s) 2023-07-14T18:18:19.714 INFO:teuthology.orchestra.run.smithi133.stderr:43100000 of 50000000 tuples (86%) done (elapsed 95.50 s, remaining 15.29 s) 2023-07-14T18:18:19.889 INFO:teuthology.orchestra.run.smithi133.stderr:43200000 of 50000000 tuples (86%) done (elapsed 95.67 s, remaining 15.06 s) 2023-07-14T18:18:20.034 INFO:teuthology.orchestra.run.smithi133.stderr:43300000 of 50000000 tuples (86%) done (elapsed 95.82 s, remaining 14.83 s) 2023-07-14T18:18:20.200 INFO:teuthology.orchestra.run.smithi133.stderr:43400000 of 50000000 tuples (86%) done (elapsed 95.99 s, remaining 14.60 s) 2023-07-14T18:18:20.427 INFO:teuthology.orchestra.run.smithi133.stderr:43500000 of 50000000 tuples (87%) done (elapsed 96.21 s, remaining 14.38 s) 2023-07-14T18:18:20.581 INFO:teuthology.orchestra.run.smithi133.stderr:43600000 of 50000000 tuples (87%) done (elapsed 96.37 s, remaining 14.15 s) 2023-07-14T18:18:20.749 INFO:teuthology.orchestra.run.smithi133.stderr:43700000 of 50000000 tuples (87%) done (elapsed 96.53 s, remaining 13.92 s) 2023-07-14T18:18:20.926 INFO:teuthology.orchestra.run.smithi133.stderr:43800000 of 50000000 tuples (87%) done (elapsed 96.71 s, remaining 13.69 s) 2023-07-14T18:18:21.130 INFO:teuthology.orchestra.run.smithi133.stderr:43900000 of 50000000 tuples (87%) done (elapsed 96.91 s, remaining 13.47 s) 2023-07-14T18:18:21.257 INFO:teuthology.orchestra.run.smithi133.stderr:44000000 of 50000000 tuples (88%) done (elapsed 97.04 s, remaining 13.23 s) 2023-07-14T18:18:21.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:20 smithi161 ceph-mon[128993]: pgmap v398: 129 pgs: 129 active+clean; 6.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 126 MiB/s wr, 55 op/s 2023-07-14T18:18:21.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:20 smithi171 ceph-mon[129241]: pgmap v398: 129 pgs: 129 active+clean; 6.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 126 MiB/s wr, 55 op/s 2023-07-14T18:18:21.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:20 smithi133 ceph-mon[126384]: pgmap v398: 129 pgs: 129 active+clean; 6.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 126 MiB/s wr, 55 op/s 2023-07-14T18:18:21.796 INFO:teuthology.orchestra.run.smithi133.stderr:44100000 of 50000000 tuples (88%) done (elapsed 97.58 s, remaining 13.06 s) 2023-07-14T18:18:22.274 INFO:teuthology.orchestra.run.smithi133.stderr:44200000 of 50000000 tuples (88%) done (elapsed 98.06 s, remaining 12.87 s) 2023-07-14T18:18:22.451 INFO:teuthology.orchestra.run.smithi133.stderr:44300000 of 50000000 tuples (88%) done (elapsed 98.24 s, remaining 12.64 s) 2023-07-14T18:18:22.585 INFO:teuthology.orchestra.run.smithi133.stderr:44400000 of 50000000 tuples (88%) done (elapsed 98.37 s, remaining 12.41 s) 2023-07-14T18:18:22.786 INFO:teuthology.orchestra.run.smithi133.stderr:44500000 of 50000000 tuples (89%) done (elapsed 98.57 s, remaining 12.18 s) 2023-07-14T18:18:22.956 INFO:teuthology.orchestra.run.smithi133.stderr:44600000 of 50000000 tuples (89%) done (elapsed 98.74 s, remaining 11.96 s) 2023-07-14T18:18:23.130 INFO:teuthology.orchestra.run.smithi133.stderr:44700000 of 50000000 tuples (89%) done (elapsed 98.92 s, remaining 11.73 s) 2023-07-14T18:18:23.297 INFO:teuthology.orchestra.run.smithi133.stderr:44800000 of 50000000 tuples (89%) done (elapsed 99.08 s, remaining 11.50 s) 2023-07-14T18:18:23.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:22 smithi161 ceph-mon[128993]: pgmap v399: 129 pgs: 129 active+clean; 6.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 129 MiB/s wr, 56 op/s 2023-07-14T18:18:23.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:22 smithi171 ceph-mon[129241]: pgmap v399: 129 pgs: 129 active+clean; 6.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 129 MiB/s wr, 56 op/s 2023-07-14T18:18:23.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:22 smithi133 ceph-mon[126384]: pgmap v399: 129 pgs: 129 active+clean; 6.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 129 MiB/s wr, 56 op/s 2023-07-14T18:18:23.494 INFO:teuthology.orchestra.run.smithi133.stderr:44900000 of 50000000 tuples (89%) done (elapsed 99.28 s, remaining 11.28 s) 2023-07-14T18:18:23.770 INFO:teuthology.orchestra.run.smithi133.stderr:45000000 of 50000000 tuples (90%) done (elapsed 99.55 s, remaining 11.06 s) 2023-07-14T18:18:24.546 INFO:teuthology.orchestra.run.smithi133.stderr:45100000 of 50000000 tuples (90%) done (elapsed 100.33 s, remaining 10.90 s) 2023-07-14T18:18:24.845 INFO:teuthology.orchestra.run.smithi133.stderr:45200000 of 50000000 tuples (90%) done (elapsed 100.63 s, remaining 10.69 s) 2023-07-14T18:18:25.025 INFO:teuthology.orchestra.run.smithi133.stderr:45300000 of 50000000 tuples (90%) done (elapsed 100.81 s, remaining 10.46 s) 2023-07-14T18:18:25.205 INFO:teuthology.orchestra.run.smithi133.stderr:45400000 of 50000000 tuples (90%) done (elapsed 100.99 s, remaining 10.23 s) 2023-07-14T18:18:25.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:25 smithi161 ceph-mon[128993]: pgmap v400: 129 pgs: 129 active+clean; 6.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 118 MiB/s wr, 76 op/s 2023-07-14T18:18:25.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:25 smithi171 ceph-mon[129241]: pgmap v400: 129 pgs: 129 active+clean; 6.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 118 MiB/s wr, 76 op/s 2023-07-14T18:18:25.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:25 smithi133 ceph-mon[126384]: pgmap v400: 129 pgs: 129 active+clean; 6.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 118 MiB/s wr, 76 op/s 2023-07-14T18:18:25.421 INFO:teuthology.orchestra.run.smithi133.stderr:45500000 of 50000000 tuples (91%) done (elapsed 101.21 s, remaining 10.01 s) 2023-07-14T18:18:25.566 INFO:teuthology.orchestra.run.smithi133.stderr:45600000 of 50000000 tuples (91%) done (elapsed 101.35 s, remaining 9.78 s) 2023-07-14T18:18:25.759 INFO:teuthology.orchestra.run.smithi133.stderr:45700000 of 50000000 tuples (91%) done (elapsed 101.54 s, remaining 9.55 s) 2023-07-14T18:18:25.954 INFO:teuthology.orchestra.run.smithi133.stderr:45800000 of 50000000 tuples (91%) done (elapsed 101.74 s, remaining 9.33 s) 2023-07-14T18:18:26.159 INFO:teuthology.orchestra.run.smithi133.stderr:45900000 of 50000000 tuples (91%) done (elapsed 101.94 s, remaining 9.11 s) 2023-07-14T18:18:26.321 INFO:teuthology.orchestra.run.smithi133.stderr:46000000 of 50000000 tuples (92%) done (elapsed 102.11 s, remaining 8.88 s) 2023-07-14T18:18:26.435 INFO:teuthology.orchestra.run.smithi133.stderr:46100000 of 50000000 tuples (92%) done (elapsed 102.22 s, remaining 8.65 s) 2023-07-14T18:18:26.590 INFO:teuthology.orchestra.run.smithi133.stderr:46200000 of 50000000 tuples (92%) done (elapsed 102.38 s, remaining 8.42 s) 2023-07-14T18:18:26.711 INFO:teuthology.orchestra.run.smithi133.stderr:46300000 of 50000000 tuples (92%) done (elapsed 102.50 s, remaining 8.19 s) 2023-07-14T18:18:26.900 INFO:teuthology.orchestra.run.smithi133.stderr:46400000 of 50000000 tuples (92%) done (elapsed 102.68 s, remaining 7.97 s) 2023-07-14T18:18:27.068 INFO:teuthology.orchestra.run.smithi133.stderr:46500000 of 50000000 tuples (93%) done (elapsed 102.85 s, remaining 7.74 s) 2023-07-14T18:18:27.211 INFO:teuthology.orchestra.run.smithi133.stderr:46600000 of 50000000 tuples (93%) done (elapsed 103.00 s, remaining 7.51 s) 2023-07-14T18:18:27.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:27 smithi161 ceph-mon[128993]: pgmap v401: 129 pgs: 129 active+clean; 6.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 91 MiB/s wr, 57 op/s 2023-07-14T18:18:27.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:27 smithi171 ceph-mon[129241]: pgmap v401: 129 pgs: 129 active+clean; 6.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 91 MiB/s wr, 57 op/s 2023-07-14T18:18:27.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:27 smithi133 ceph-mon[126384]: pgmap v401: 129 pgs: 129 active+clean; 6.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 91 MiB/s wr, 57 op/s 2023-07-14T18:18:27.430 INFO:teuthology.orchestra.run.smithi133.stderr:46700000 of 50000000 tuples (93%) done (elapsed 103.22 s, remaining 7.29 s) 2023-07-14T18:18:27.616 INFO:teuthology.orchestra.run.smithi133.stderr:46800000 of 50000000 tuples (93%) done (elapsed 103.40 s, remaining 7.07 s) 2023-07-14T18:18:27.761 INFO:teuthology.orchestra.run.smithi133.stderr:46900000 of 50000000 tuples (93%) done (elapsed 103.55 s, remaining 6.84 s) 2023-07-14T18:18:27.922 INFO:teuthology.orchestra.run.smithi133.stderr:47000000 of 50000000 tuples (94%) done (elapsed 103.71 s, remaining 6.62 s) 2023-07-14T18:18:28.133 INFO:teuthology.orchestra.run.smithi133.stderr:47100000 of 50000000 tuples (94%) done (elapsed 103.92 s, remaining 6.40 s) 2023-07-14T18:18:28.274 INFO:teuthology.orchestra.run.smithi133.stderr:47200000 of 50000000 tuples (94%) done (elapsed 104.06 s, remaining 6.17 s) 2023-07-14T18:18:28.432 INFO:teuthology.orchestra.run.smithi133.stderr:47300000 of 50000000 tuples (94%) done (elapsed 104.22 s, remaining 5.95 s) 2023-07-14T18:18:28.634 INFO:teuthology.orchestra.run.smithi133.stderr:47400000 of 50000000 tuples (94%) done (elapsed 104.42 s, remaining 5.73 s) 2023-07-14T18:18:28.827 INFO:teuthology.orchestra.run.smithi133.stderr:47500000 of 50000000 tuples (95%) done (elapsed 104.61 s, remaining 5.51 s) 2023-07-14T18:18:29.066 INFO:teuthology.orchestra.run.smithi133.stderr:47600000 of 50000000 tuples (95%) done (elapsed 104.85 s, remaining 5.29 s) 2023-07-14T18:18:29.172 INFO:teuthology.orchestra.run.smithi133.stderr:47700000 of 50000000 tuples (95%) done (elapsed 104.96 s, remaining 5.06 s) 2023-07-14T18:18:29.344 INFO:teuthology.orchestra.run.smithi133.stderr:47800000 of 50000000 tuples (95%) done (elapsed 105.13 s, remaining 4.84 s) 2023-07-14T18:18:29.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:29 smithi161 ceph-mon[128993]: pgmap v402: 129 pgs: 129 active+clean; 6.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 132 MiB/s wr, 92 op/s 2023-07-14T18:18:29.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:29 smithi171 ceph-mon[129241]: pgmap v402: 129 pgs: 129 active+clean; 6.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 132 MiB/s wr, 92 op/s 2023-07-14T18:18:29.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:29 smithi133 ceph-mon[126384]: pgmap v402: 129 pgs: 129 active+clean; 6.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 132 MiB/s wr, 92 op/s 2023-07-14T18:18:29.557 INFO:teuthology.orchestra.run.smithi133.stderr:47900000 of 50000000 tuples (95%) done (elapsed 105.34 s, remaining 4.62 s) 2023-07-14T18:18:29.715 INFO:teuthology.orchestra.run.smithi133.stderr:48000000 of 50000000 tuples (96%) done (elapsed 105.50 s, remaining 4.40 s) 2023-07-14T18:18:29.918 INFO:teuthology.orchestra.run.smithi133.stderr:48100000 of 50000000 tuples (96%) done (elapsed 105.70 s, remaining 4.18 s) 2023-07-14T18:18:30.061 INFO:teuthology.orchestra.run.smithi133.stderr:48200000 of 50000000 tuples (96%) done (elapsed 105.85 s, remaining 3.95 s) 2023-07-14T18:18:30.257 INFO:teuthology.orchestra.run.smithi133.stderr:48300000 of 50000000 tuples (96%) done (elapsed 106.04 s, remaining 3.73 s) 2023-07-14T18:18:30.442 INFO:teuthology.orchestra.run.smithi133.stderr:48400000 of 50000000 tuples (96%) done (elapsed 106.23 s, remaining 3.51 s) 2023-07-14T18:18:30.550 INFO:teuthology.orchestra.run.smithi133.stderr:48500000 of 50000000 tuples (97%) done (elapsed 106.34 s, remaining 3.29 s) 2023-07-14T18:18:30.768 INFO:teuthology.orchestra.run.smithi133.stderr:48600000 of 50000000 tuples (97%) done (elapsed 106.55 s, remaining 3.07 s) 2023-07-14T18:18:30.918 INFO:teuthology.orchestra.run.smithi133.stderr:48700000 of 50000000 tuples (97%) done (elapsed 106.70 s, remaining 2.85 s) 2023-07-14T18:18:31.056 INFO:teuthology.orchestra.run.smithi133.stderr:48800000 of 50000000 tuples (97%) done (elapsed 106.84 s, remaining 2.63 s) 2023-07-14T18:18:31.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:31 smithi161 ceph-mon[128993]: pgmap v403: 129 pgs: 129 active+clean; 6.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 108 MiB/s wr, 82 op/s 2023-07-14T18:18:31.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:31 smithi171 ceph-mon[129241]: pgmap v403: 129 pgs: 129 active+clean; 6.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 108 MiB/s wr, 82 op/s 2023-07-14T18:18:31.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:31 smithi133 ceph-mon[126384]: pgmap v403: 129 pgs: 129 active+clean; 6.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 108 MiB/s wr, 82 op/s 2023-07-14T18:18:31.560 INFO:teuthology.orchestra.run.smithi133.stderr:48900000 of 50000000 tuples (97%) done (elapsed 107.34 s, remaining 2.41 s) 2023-07-14T18:18:31.866 INFO:teuthology.orchestra.run.smithi133.stderr:49000000 of 50000000 tuples (98%) done (elapsed 107.65 s, remaining 2.20 s) 2023-07-14T18:18:32.264 INFO:teuthology.orchestra.run.smithi133.stderr:49100000 of 50000000 tuples (98%) done (elapsed 108.05 s, remaining 1.98 s) 2023-07-14T18:18:33.099 INFO:teuthology.orchestra.run.smithi133.stderr:49200000 of 50000000 tuples (98%) done (elapsed 108.88 s, remaining 1.77 s) 2023-07-14T18:18:33.226 INFO:teuthology.orchestra.run.smithi133.stderr:49300000 of 50000000 tuples (98%) done (elapsed 109.01 s, remaining 1.55 s) 2023-07-14T18:18:33.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:33 smithi161 ceph-mon[128993]: pgmap v404: 129 pgs: 129 active+clean; 6.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 100 MiB/s wr, 80 op/s 2023-07-14T18:18:33.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:33 smithi171 ceph-mon[129241]: pgmap v404: 129 pgs: 129 active+clean; 6.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 100 MiB/s wr, 80 op/s 2023-07-14T18:18:33.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:33 smithi133 ceph-mon[126384]: pgmap v404: 129 pgs: 129 active+clean; 6.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 100 MiB/s wr, 80 op/s 2023-07-14T18:18:33.414 INFO:teuthology.orchestra.run.smithi133.stderr:49400000 of 50000000 tuples (98%) done (elapsed 109.20 s, remaining 1.33 s) 2023-07-14T18:18:33.615 INFO:teuthology.orchestra.run.smithi133.stderr:49500000 of 50000000 tuples (99%) done (elapsed 109.40 s, remaining 1.11 s) 2023-07-14T18:18:33.807 INFO:teuthology.orchestra.run.smithi133.stderr:49600000 of 50000000 tuples (99%) done (elapsed 109.59 s, remaining 0.88 s) 2023-07-14T18:18:33.945 INFO:teuthology.orchestra.run.smithi133.stderr:49700000 of 50000000 tuples (99%) done (elapsed 109.73 s, remaining 0.66 s) 2023-07-14T18:18:34.113 INFO:teuthology.orchestra.run.smithi133.stderr:49800000 of 50000000 tuples (99%) done (elapsed 109.90 s, remaining 0.44 s) 2023-07-14T18:18:34.335 INFO:teuthology.orchestra.run.smithi133.stderr:49900000 of 50000000 tuples (99%) done (elapsed 110.12 s, remaining 0.22 s) 2023-07-14T18:18:34.474 INFO:teuthology.orchestra.run.smithi133.stderr:50000000 of 50000000 tuples (100%) done (elapsed 110.26 s, remaining 0.00 s) 2023-07-14T18:18:34.539 INFO:teuthology.orchestra.run.smithi133.stderr:vacuum... 2023-07-14T18:18:35.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:35 smithi161 ceph-mon[128993]: pgmap v405: 129 pgs: 129 active+clean; 6.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 109 MiB/s wr, 86 op/s 2023-07-14T18:18:35.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:35 smithi171 ceph-mon[129241]: pgmap v405: 129 pgs: 129 active+clean; 6.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 109 MiB/s wr, 86 op/s 2023-07-14T18:18:35.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:35 smithi133 ceph-mon[126384]: pgmap v405: 129 pgs: 129 active+clean; 6.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 109 MiB/s wr, 86 op/s 2023-07-14T18:18:37.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:37 smithi161 ceph-mon[128993]: pgmap v406: 129 pgs: 129 active+clean; 6.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 102 MiB/s wr, 59 op/s 2023-07-14T18:18:37.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:37 smithi171 ceph-mon[129241]: pgmap v406: 129 pgs: 129 active+clean; 6.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 102 MiB/s wr, 59 op/s 2023-07-14T18:18:37.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:37 smithi133 ceph-mon[126384]: pgmap v406: 129 pgs: 129 active+clean; 6.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 102 MiB/s wr, 59 op/s 2023-07-14T18:18:39.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:39 smithi161 ceph-mon[128993]: pgmap v407: 129 pgs: 129 active+clean; 7.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 135 MiB/s wr, 83 op/s 2023-07-14T18:18:39.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:39 smithi171 ceph-mon[129241]: pgmap v407: 129 pgs: 129 active+clean; 7.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 135 MiB/s wr, 83 op/s 2023-07-14T18:18:39.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:39 smithi133 ceph-mon[126384]: pgmap v407: 129 pgs: 129 active+clean; 7.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 135 MiB/s wr, 83 op/s 2023-07-14T18:18:39.958 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:18:40.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.298+0000 7f7831bff700 1 -- 172.21.15.133:0/1914548528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f782c107ad0 msgr2=0x7f782c109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:18:40.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.298+0000 7f7831bff700 1 --2- 172.21.15.133:0/1914548528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f782c107ad0 0x7f782c109f50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f782400b0a0 tx=0x7f782400b3b0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.299+0000 7f7831bff700 1 -- 172.21.15.133:0/1914548528 shutdown_connections 2023-07-14T18:18:40.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.299+0000 7f7831bff700 1 --2- 172.21.15.133:0/1914548528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f782c107ad0 0x7f782c109f50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.299+0000 7f7831bff700 1 --2- 172.21.15.133:0/1914548528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f782c074680 0x7f782c072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.299+0000 7f7831bff700 1 --2- 172.21.15.133:0/1914548528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f782c0730a0 0x7f782c073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.299+0000 7f7831bff700 1 -- 172.21.15.133:0/1914548528 >> 172.21.15.133:0/1914548528 conn(0x7f782c0fb210 msgr2=0x7f782c0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:18:40.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.299+0000 7f7831bff700 1 -- 172.21.15.133:0/1914548528 shutdown_connections 2023-07-14T18:18:40.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.299+0000 7f7831bff700 1 -- 172.21.15.133:0/1914548528 wait complete. 2023-07-14T18:18:40.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.300+0000 7f7831bff700 1 Processor -- start 2023-07-14T18:18:40.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.301+0000 7f7831bff700 1 -- start start 2023-07-14T18:18:40.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.301+0000 7f7831bff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f782c0730a0 0x7f782c102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:40.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.301+0000 7f7831bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f782c074680 0x7f782c103470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:40.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.301+0000 7f7831bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f782c107ad0 0x7f782c0ffac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:40.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.301+0000 7f7831bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782c100260 con 0x7f782c074680 2023-07-14T18:18:40.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.301+0000 7f7831bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782c1003d0 con 0x7f782c107ad0 2023-07-14T18:18:40.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.301+0000 7f7831bff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782c100590 con 0x7f782c0730a0 2023-07-14T18:18:40.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.301+0000 7f782b7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f782c0730a0 0x7f782c102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:40.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.301+0000 7f782b7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f782c0730a0 0x7f782c102f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:43380/0 (socket says 172.21.15.133:43380) 2023-07-14T18:18:40.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.301+0000 7f782b7fe700 1 -- 172.21.15.133:0/3820356411 learned_addr learned my addr 172.21.15.133:0/3820356411 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:18:40.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.302+0000 7f782bfff700 1 --2- 172.21.15.133:0/3820356411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f782c107ad0 0x7f782c0ffac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:40.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.302+0000 7f782affd700 1 --2- 172.21.15.133:0/3820356411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f782c074680 0x7f782c103470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:40.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.302+0000 7f782b7fe700 1 -- 172.21.15.133:0/3820356411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f782c107ad0 msgr2=0x7f782c0ffac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:18:40.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.302+0000 7f782b7fe700 1 --2- 172.21.15.133:0/3820356411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f782c107ad0 0x7f782c0ffac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.302+0000 7f782b7fe700 1 -- 172.21.15.133:0/3820356411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f782c074680 msgr2=0x7f782c103470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:18:40.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.302+0000 7f782b7fe700 1 --2- 172.21.15.133:0/3820356411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f782c074680 0x7f782c103470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.302+0000 7f782b7fe700 1 -- 172.21.15.133:0/3820356411 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7824009d00 con 0x7f782c0730a0 2023-07-14T18:18:40.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.302+0000 7f782bfff700 1 --2- 172.21.15.133:0/3820356411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f782c107ad0 0x7f782c0ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:18:40.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.302+0000 7f782affd700 1 --2- 172.21.15.133:0/3820356411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f782c074680 0x7f782c103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T18:18:40.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.303+0000 7f782b7fe700 1 --2- 172.21.15.133:0/3820356411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f782c0730a0 0x7f782c102f30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f781c00d980 tx=0x7f781c00dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:18:40.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.303+0000 7f7828ff9700 1 -- 172.21.15.133:0/3820356411 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c009b40 con 0x7f782c0730a0 2023-07-14T18:18:40.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.303+0000 7f7828ff9700 1 -- 172.21.15.133:0/3820356411 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f781c00de60 con 0x7f782c0730a0 2023-07-14T18:18:40.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.303+0000 7f7828ff9700 1 -- 172.21.15.133:0/3820356411 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c00f620 con 0x7f782c0730a0 2023-07-14T18:18:40.354 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.303+0000 7f7831bff700 1 -- 172.21.15.133:0/3820356411 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f782c197a40 con 0x7f782c0730a0 2023-07-14T18:18:40.354 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.303+0000 7f7831bff700 1 -- 172.21.15.133:0/3820356411 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f782c197f00 con 0x7f782c0730a0 2023-07-14T18:18:40.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.311+0000 7f78454e8b80 1 Processor -- start 2023-07-14T18:18:40.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.311+0000 7f78454e8b80 1 -- start start 2023-07-14T18:18:40.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.311+0000 7f78454e8b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e519fe0f60 0x55e519fe1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:40.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.311+0000 7f78454e8b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e519fe18b0 0x55e519fe1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:40.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.311+0000 7f78454e8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e519fe24b0 0x55e519fe2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:40.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.311+0000 7f78454e8b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e519ee5cb0 con 0x55e519fe0f60 2023-07-14T18:18:40.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.311+0000 7f78454e8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e519f38a40 con 0x55e519fe24b0 2023-07-14T18:18:40.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.311+0000 7f78454e8b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e519f31ed0 con 0x55e519fe18b0 2023-07-14T18:18:40.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.312+0000 7f7828ff9700 1 -- 172.21.15.133:0/3820356411 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f781c010460 con 0x7f782c0730a0 2023-07-14T18:18:40.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.312+0000 7f782b7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e519fe18b0 0x55e519fe1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:40.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.312+0000 7f782bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e519fe0f60 0x55e519fe1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:40.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.312+0000 7f782b7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e519fe18b0 0x55e519fe1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:43386/0 (socket says 172.21.15.133:43386) 2023-07-14T18:18:40.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.312+0000 7f782b7fe700 1 -- 172.21.15.133:0/3779504015 learned_addr learned my addr 172.21.15.133:0/3779504015 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:18:40.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.312+0000 7f782affd700 1 --2- 172.21.15.133:0/3779504015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e519fe24b0 0x55e519fe2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:40.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.312+0000 7f782b7fe700 1 -- 172.21.15.133:0/3779504015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e519fe24b0 msgr2=0x55e519fe2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:18:40.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.312+0000 7f782b7fe700 1 --2- 172.21.15.133:0/3779504015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e519fe24b0 0x55e519fe2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.312+0000 7f782b7fe700 1 -- 172.21.15.133:0/3779504015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e519fe0f60 msgr2=0x55e519fe1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:18:40.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.312+0000 7f782b7fe700 1 --2- 172.21.15.133:0/3779504015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e519fe0f60 0x55e519fe1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.312+0000 7f782b7fe700 1 -- 172.21.15.133:0/3779504015 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e519d5fde0 con 0x55e519fe18b0 2023-07-14T18:18:40.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.313+0000 7f782b7fe700 1 --2- 172.21.15.133:0/3779504015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e519fe18b0 0x55e519fe1ed0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f781c0d2a90 tx=0x7f781c0d2da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6d4260948cc51ea server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:18:40.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.313+0000 7f7811ffb700 1 -- 172.21.15.133:0/3779504015 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c0d4c20 con 0x55e519fe18b0 2023-07-14T18:18:40.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.313+0000 7f7811ffb700 1 -- 172.21.15.133:0/3779504015 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f781c0d26d0 con 0x55e519fe18b0 2023-07-14T18:18:40.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.313+0000 7f7811ffb700 1 -- 172.21.15.133:0/3779504015 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c0d7680 con 0x55e519fe18b0 2023-07-14T18:18:40.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.314+0000 7f78454e8b80 1 -- 172.21.15.133:0/3779504015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e519fe18b0 msgr2=0x55e519fe1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:18:40.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.314+0000 7f78454e8b80 1 --2- 172.21.15.133:0/3779504015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e519fe18b0 0x55e519fe1ed0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f781c0d2a90 tx=0x7f781c0d2da0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.314+0000 7f78454e8b80 1 -- 172.21.15.133:0/3779504015 shutdown_connections 2023-07-14T18:18:40.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.314+0000 7f78454e8b80 1 --2- 172.21.15.133:0/3779504015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e519fe24b0 0x55e519fe2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.314+0000 7f78454e8b80 1 --2- 172.21.15.133:0/3779504015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e519fe0f60 0x55e519fe1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.314+0000 7f78454e8b80 1 --2- 172.21.15.133:0/3779504015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e519fe18b0 0x55e519fe1ed0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.314+0000 7f78454e8b80 1 -- 172.21.15.133:0/3779504015 >> 172.21.15.133:0/3779504015 conn(0x55e519fd6680 msgr2=0x55e519fdb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:18:40.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.314+0000 7f7828ff9700 1 --2- 172.21.15.133:0/3820356411 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f78140a11d0 0x7f78140a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:40.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.314+0000 7f7828ff9700 1 -- 172.21.15.133:0/3820356411 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f781c0cc880 con 0x7f782c0730a0 2023-07-14T18:18:40.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.314+0000 7f782bfff700 1 --2- 172.21.15.133:0/3820356411 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f78140a11d0 0x7f78140a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:40.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.315+0000 7f78454e8b80 1 -- 172.21.15.133:0/3779504015 shutdown_connections 2023-07-14T18:18:40.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.315+0000 7f78454e8b80 1 -- 172.21.15.133:0/3779504015 wait complete. 2023-07-14T18:18:40.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.315+0000 7f782bfff700 1 --2- 172.21.15.133:0/3820356411 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f78140a11d0 0x7f78140a3680 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7824009610 tx=0x7f7824006040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:18:40.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.315+0000 7f78454e8b80 1 -- start start 2023-07-14T18:18:40.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.316+0000 7f78454e8b80 1 Processor -- start 2023-07-14T18:18:40.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.316+0000 7f78454e8b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e519fe0f60 0x55e519fea7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:40.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.316+0000 7f78454e8b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e519fe18b0 0x55e519fead10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:40.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.316+0000 7f78454e8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e519fe24b0 0x55e519ff1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:40.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.316+0000 7f78454e8b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e519ed1ac0 con 0x55e519fe18b0 2023-07-14T18:18:40.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.316+0000 7f78454e8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e519ed17c0 con 0x55e519fe24b0 2023-07-14T18:18:40.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.316+0000 7f78454e8b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e519db5920 con 0x55e519fe0f60 2023-07-14T18:18:40.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.317+0000 7f782b7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e519fe0f60 0x55e519fea7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:40.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.317+0000 7f78454e8b80 10 client.0 _ll_register_callbacks cb 0x55e519ecf760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:18:40.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.317+0000 7f782b7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e519fe0f60 0x55e519fea7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:43398/0 (socket says 172.21.15.133:43398) 2023-07-14T18:18:40.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.317+0000 7f782b7fe700 1 -- 172.21.15.133:0/1411791976 learned_addr learned my addr 172.21.15.133:0/1411791976 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:18:40.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.317+0000 7f782b7fe700 1 -- 172.21.15.133:0/1411791976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e519fe24b0 msgr2=0x55e519ff1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:18:40.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.317+0000 7f782b7fe700 1 --2- 172.21.15.133:0/1411791976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e519fe24b0 0x55e519ff1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.317+0000 7f782b7fe700 1 -- 172.21.15.133:0/1411791976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e519fe18b0 msgr2=0x55e519fead10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:18:40.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.317+0000 7f782b7fe700 1 --2- 172.21.15.133:0/1411791976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e519fe18b0 0x55e519fead10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.317+0000 7f782b7fe700 1 -- 172.21.15.133:0/1411791976 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f782c197f00 con 0x55e519fe0f60 2023-07-14T18:18:40.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.317+0000 7f782b7fe700 1 --2- 172.21.15.133:0/1411791976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e519fe0f60 0x55e519fea7d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f781c095e00 tx=0x7f781c0ce710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14b64408c1809c80 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:18:40.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.317+0000 7f780affd700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:18:40.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.317+0000 7f780affd700 1 -- 172.21.15.133:0/1411791976 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c0d26d0 con 0x55e519fe0f60 2023-07-14T18:18:40.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.317+0000 7f780affd700 1 -- 172.21.15.133:0/1411791976 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f781c099900 con 0x55e519fe0f60 2023-07-14T18:18:40.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.317+0000 7f78454e8b80 1 -- 172.21.15.133:0/1411791976 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e519fafda0 con 0x55e519fe0f60 2023-07-14T18:18:40.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.317+0000 7f780affd700 1 -- 172.21.15.133:0/1411791976 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f781c095c40 con 0x55e519fe0f60 2023-07-14T18:18:40.372 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.318+0000 7f78454e8b80 10 client.25687 fetch_fsmap learned FSMap version 30 2023-07-14T18:18:40.372 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.318+0000 7f78454e8b80 1 -- 172.21.15.133:0/1411791976 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e519d48c60 con 0x55e519fe0f60 2023-07-14T18:18:40.372 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.318+0000 7f780affd700 1 -- 172.21.15.133:0/1411791976 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e519fafda0 con 0x55e519fe0f60 2023-07-14T18:18:40.372 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.318+0000 7f780affd700 1 -- 172.21.15.133:0/1411791976 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f781c096010 con 0x55e519fe0f60 2023-07-14T18:18:40.372 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.318+0000 7f78454e8b80 10 client.25687 fetch_fsmap finished waiting for FSMap version 30 2023-07-14T18:18:40.372 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.318+0000 7f78454e8b80 10 client.25687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:18:40.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.318+0000 7f78454e8b80 20 client.25687 populate_metadata read hostname 'smithi133' 2023-07-14T18:18:40.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.318+0000 7f78454e8b80 1 --2- 172.21.15.133:0/1411791976 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55e519ff61a0 0x55e519ff85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:40.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.318+0000 7f78454e8b80 4 client.25687 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:18:40.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.318+0000 7f78454e8b80 1 -- 172.21.15.133:0/1411791976 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e519f2aeb0 con 0x55e519ff61a0 2023-07-14T18:18:40.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.318+0000 7f782affd700 1 --2- 172.21.15.133:0/1411791976 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55e519ff61a0 0x55e519ff85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:40.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.319+0000 7f780affd700 10 client.25687 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:18:40.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.319+0000 7f782affd700 1 --2- 172.21.15.133:0/1411791976 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55e519ff61a0 0x55e519ff85b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ce0f356158d10c9 server_cookie=b675b337f8be3e89 in_seq=0 out_seq=0 2023-07-14T18:18:40.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.320+0000 7f780affd700 1 -- 172.21.15.133:0/1411791976 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55e519f2aeb0 con 0x55e519ff61a0 2023-07-14T18:18:40.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.320+0000 7f780affd700 10 client.25687 handle_command_reply: tid=0 2023-07-14T18:18:40.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.320+0000 7f78454e8b80 1 client.25687 shutdown 2023-07-14T18:18:40.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.320+0000 7f78454e8b80 1 -- 172.21.15.133:0/1411791976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e519fe0f60 msgr2=0x55e519fea7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:18:40.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.320+0000 7f78454e8b80 1 --2- 172.21.15.133:0/1411791976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e519fe0f60 0x55e519fea7d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f781c095e00 tx=0x7f781c0ce710 comp rx=0 tx=0).stop 2023-07-14T18:18:40.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.320+0000 7f78454e8b80 1 -- 172.21.15.133:0/1411791976 shutdown_connections 2023-07-14T18:18:40.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.320+0000 7f78454e8b80 1 --2- 172.21.15.133:0/1411791976 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55e519ff61a0 0x55e519ff85b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.320+0000 7f78454e8b80 1 --2- 172.21.15.133:0/1411791976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e519fe24b0 0x55e519ff1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.320+0000 7f78454e8b80 1 --2- 172.21.15.133:0/1411791976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e519fe0f60 0x55e519fea7d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.320+0000 7f780affd700 0 client.25687 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:18:40.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.320+0000 7f78454e8b80 1 --2- 172.21.15.133:0/1411791976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e519fe18b0 0x55e519fead10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.320+0000 7f78454e8b80 1 -- 172.21.15.133:0/1411791976 >> 172.21.15.133:0/1411791976 conn(0x55e519fd6680 msgr2=0x55e519fda650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:18:40.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.321+0000 7f78454e8b80 1 -- 172.21.15.133:0/1411791976 shutdown_connections 2023-07-14T18:18:40.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.321+0000 7f78454e8b80 1 -- 172.21.15.133:0/1411791976 wait complete. 2023-07-14T18:18:40.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.321+0000 7f78454e8b80 20 client.25687 trim_cache size 0 max 16384 2023-07-14T18:18:40.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.357+0000 7f78454e8b80 1 Processor -- start 2023-07-14T18:18:40.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.357+0000 7f78454e8b80 1 -- start start 2023-07-14T18:18:40.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.358+0000 7f78454e8b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e519fe0f60 0x55e519fe2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:40.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.358+0000 7f78454e8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e519fe18b0 0x55e519ff1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:40.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.358+0000 7f78454e8b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e519fe24b0 0x55e519fe95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:40.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.358+0000 7f78454e8b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e519fe2330 con 0x55e519fe0f60 2023-07-14T18:18:40.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.358+0000 7f78454e8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e519fe93a0 con 0x55e519fe18b0 2023-07-14T18:18:40.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.358+0000 7f78454e8b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e519fe9c80 con 0x55e519fe24b0 2023-07-14T18:18:40.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.358+0000 7f782affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e519fe18b0 0x55e519ff1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:40.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.358+0000 7f782affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e519fe18b0 0x55e519ff1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:57444/0 (socket says 172.21.15.133:57444) 2023-07-14T18:18:40.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.358+0000 7f782affd700 1 -- 172.21.15.133:0/3358155014 learned_addr learned my addr 172.21.15.133:0/3358155014 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:18:40.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.358+0000 7f782bfff700 1 --2- 172.21.15.133:0/3358155014 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e519fe24b0 0x55e519fe95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:40.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.359+0000 7f782b7fe700 1 --2- 172.21.15.133:0/3358155014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e519fe0f60 0x55e519fe2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:40.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.359+0000 7f782affd700 1 -- 172.21.15.133:0/3358155014 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e519fe24b0 msgr2=0x55e519fe95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:18:40.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.359+0000 7f782affd700 1 --2- 172.21.15.133:0/3358155014 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e519fe24b0 0x55e519fe95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.359+0000 7f782affd700 1 -- 172.21.15.133:0/3358155014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e519fe0f60 msgr2=0x55e519fe2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:18:40.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.359+0000 7f782affd700 1 --2- 172.21.15.133:0/3358155014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e519fe0f60 0x55e519fe2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.359+0000 7f782affd700 1 -- 172.21.15.133:0/3358155014 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e519d86250 con 0x55e519fe18b0 2023-07-14T18:18:40.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.359+0000 7f782b7fe700 1 --2- 172.21.15.133:0/3358155014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e519fe0f60 0x55e519fe2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T18:18:40.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.359+0000 7f782affd700 1 --2- 172.21.15.133:0/3358155014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e519fe18b0 0x55e519ff1d10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55e519ff8ec0 tx=0x7f7820005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=716d39b8ae1dd56c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:18:40.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.359+0000 7f78127fc700 1 -- 172.21.15.133:0/3358155014 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7820012b10 con 0x55e519fe18b0 2023-07-14T18:18:40.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.359+0000 7f78127fc700 1 -- 172.21.15.133:0/3358155014 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f782001ac80 con 0x55e519fe18b0 2023-07-14T18:18:40.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.359+0000 7f78127fc700 1 -- 172.21.15.133:0/3358155014 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7820011860 con 0x55e519fe18b0 2023-07-14T18:18:40.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.360+0000 7f78454e8b80 1 -- 172.21.15.133:0/3358155014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e519fe18b0 msgr2=0x55e519ff1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:18:40.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.360+0000 7f78454e8b80 1 --2- 172.21.15.133:0/3358155014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e519fe18b0 0x55e519ff1d10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55e519ff8ec0 tx=0x7f7820005ce0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.361+0000 7f78454e8b80 1 -- 172.21.15.133:0/3358155014 shutdown_connections 2023-07-14T18:18:40.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.361+0000 7f78454e8b80 1 --2- 172.21.15.133:0/3358155014 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e519fe24b0 0x55e519fe95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.361+0000 7f78454e8b80 1 --2- 172.21.15.133:0/3358155014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e519fe0f60 0x55e519fe2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.361+0000 7f78454e8b80 1 --2- 172.21.15.133:0/3358155014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e519fe18b0 0x55e519ff1d10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.361+0000 7f78454e8b80 1 -- 172.21.15.133:0/3358155014 >> 172.21.15.133:0/3358155014 conn(0x55e519fd6680 msgr2=0x55e519fe52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:18:40.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.361+0000 7f78454e8b80 1 -- 172.21.15.133:0/3358155014 shutdown_connections 2023-07-14T18:18:40.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.361+0000 7f78454e8b80 1 -- 172.21.15.133:0/3358155014 wait complete. 2023-07-14T18:18:40.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.362+0000 7f78454e8b80 1 -- start start 2023-07-14T18:18:40.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.362+0000 7f78454e8b80 1 Processor -- start 2023-07-14T18:18:40.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.362+0000 7f78454e8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e519fe0f60 0x55e519ff1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:40.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.362+0000 7f78454e8b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e519fe18b0 0x55e519fee310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:40.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.363+0000 7f78454e8b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e519fe24b0 0x55e519fee850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:40.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.363+0000 7f78454e8b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e519feef30 con 0x55e519fe24b0 2023-07-14T18:18:40.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.363+0000 7f78454e8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e519ff1ea0 con 0x55e519fe0f60 2023-07-14T18:18:40.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.363+0000 7f78454e8b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e519ff2030 con 0x55e519fe18b0 2023-07-14T18:18:40.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.363+0000 7f78454e8b80 10 client.0 _ll_register_callbacks cb 0x7f7824009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:18:40.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.363+0000 7f782b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e519fe0f60 0x55e519ff1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:40.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.363+0000 7f782b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e519fe0f60 0x55e519ff1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:57448/0 (socket says 172.21.15.133:57448) 2023-07-14T18:18:40.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.363+0000 7f782b7fe700 1 -- 172.21.15.133:0/2244333160 learned_addr learned my addr 172.21.15.133:0/2244333160 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:18:40.415 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.363+0000 7f782affd700 1 --2- 172.21.15.133:0/2244333160 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e519fe18b0 0x55e519fee310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:40.415 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.363+0000 7f782bfff700 1 --2- 172.21.15.133:0/2244333160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e519fe24b0 0x55e519fee850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:40.416 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.363+0000 7f782b7fe700 1 -- 172.21.15.133:0/2244333160 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e519fe18b0 msgr2=0x55e519fee310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:18:40.416 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.363+0000 7f782b7fe700 1 --2- 172.21.15.133:0/2244333160 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e519fe18b0 0x55e519fee310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.416 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.363+0000 7f782b7fe700 1 -- 172.21.15.133:0/2244333160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e519fe24b0 msgr2=0x55e519fee850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:18:40.416 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.363+0000 7f782b7fe700 1 --2- 172.21.15.133:0/2244333160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e519fe24b0 0x55e519fee850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.416 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.363+0000 7f782b7fe700 1 -- 172.21.15.133:0/2244333160 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7820009bb0 con 0x55e519fe0f60 2023-07-14T18:18:40.416 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.364+0000 7f782bfff700 1 --2- 172.21.15.133:0/2244333160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e519fe24b0 0x55e519fee850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T18:18:40.416 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.364+0000 7f782b7fe700 1 --2- 172.21.15.133:0/2244333160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e519fe0f60 0x55e519ff1820 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f781c09ce50 tx=0x7f781c0d2da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b17393b970257ea server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:18:40.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.364+0000 7f780affd700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:18:40.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.364+0000 7f780affd700 1 -- 172.21.15.133:0/2244333160 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c0ceae0 con 0x55e519fe0f60 2023-07-14T18:18:40.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.364+0000 7f780affd700 1 -- 172.21.15.133:0/2244333160 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f781c099900 con 0x55e519fe0f60 2023-07-14T18:18:40.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.364+0000 7f78454e8b80 1 -- 172.21.15.133:0/2244333160 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e519e8b4f0 con 0x55e519fe0f60 2023-07-14T18:18:40.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.364+0000 7f780affd700 1 -- 172.21.15.133:0/2244333160 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f781c0d26c0 con 0x55e519fe0f60 2023-07-14T18:18:40.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.364+0000 7f78454e8b80 10 client.25658 fetch_fsmap learned FSMap version 30 2023-07-14T18:18:40.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.364+0000 7f78454e8b80 1 -- 172.21.15.133:0/2244333160 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e519d5e2b0 con 0x55e519fe0f60 2023-07-14T18:18:40.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.365+0000 7f780affd700 1 -- 172.21.15.133:0/2244333160 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e519e8b4f0 con 0x55e519fe0f60 2023-07-14T18:18:40.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.365+0000 7f780affd700 1 -- 172.21.15.133:0/2244333160 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f781c098e80 con 0x55e519fe0f60 2023-07-14T18:18:40.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.365+0000 7f78454e8b80 10 client.25658 fetch_fsmap finished waiting for FSMap version 30 2023-07-14T18:18:40.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.365+0000 7f78454e8b80 10 client.25658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:18:40.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.365+0000 7f78454e8b80 20 client.25658 populate_metadata read hostname 'smithi133' 2023-07-14T18:18:40.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.365+0000 7f78454e8b80 1 --2- 172.21.15.133:0/2244333160 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55e51a04b0d0 0x55e51a04d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:18:40.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.365+0000 7f78454e8b80 4 client.25658 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:18:40.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.365+0000 7f78454e8b80 1 -- 172.21.15.133:0/2244333160 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e519eed050 con 0x55e51a04b0d0 2023-07-14T18:18:40.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.365+0000 7f782affd700 1 --2- 172.21.15.133:0/2244333160 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55e51a04b0d0 0x55e51a04d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:18:40.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.366+0000 7f780affd700 10 client.25658 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:18:40.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.366+0000 7f782affd700 1 --2- 172.21.15.133:0/2244333160 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55e51a04b0d0 0x55e51a04d4e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87b64fb2857db7d9 server_cookie=a7f356ee91d5e5e8 in_seq=0 out_seq=0 2023-07-14T18:18:40.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.367+0000 7f780affd700 1 -- 172.21.15.133:0/2244333160 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55e519eed050 con 0x55e51a04b0d0 2023-07-14T18:18:40.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.367+0000 7f780affd700 10 client.25658 handle_command_reply: tid=0 2023-07-14T18:18:40.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.367+0000 7f78454e8b80 1 client.25658 shutdown 2023-07-14T18:18:40.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.367+0000 7f78454e8b80 1 -- 172.21.15.133:0/2244333160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e519fe0f60 msgr2=0x55e519ff1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:18:40.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.367+0000 7f78454e8b80 1 --2- 172.21.15.133:0/2244333160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e519fe0f60 0x55e519ff1820 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f781c09ce50 tx=0x7f781c0d2da0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.368+0000 7f78454e8b80 1 -- 172.21.15.133:0/2244333160 shutdown_connections 2023-07-14T18:18:40.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.368+0000 7f78454e8b80 1 --2- 172.21.15.133:0/2244333160 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55e51a04b0d0 0x55e51a04d4e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.368+0000 7f78454e8b80 1 --2- 172.21.15.133:0/2244333160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e519fe0f60 0x55e519ff1820 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.421 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.368+0000 7f78454e8b80 1 --2- 172.21.15.133:0/2244333160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e519fe24b0 0x55e519fee850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.421 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.368+0000 7f780affd700 0 client.25658 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:18:40.421 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.368+0000 7f78454e8b80 1 --2- 172.21.15.133:0/2244333160 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e519fe18b0 0x55e519fee310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.421 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.368+0000 7f78454e8b80 1 -- 172.21.15.133:0/2244333160 >> 172.21.15.133:0/2244333160 conn(0x55e519fd6680 msgr2=0x55e519fe8780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:18:40.421 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.368+0000 7f78454e8b80 1 -- 172.21.15.133:0/2244333160 shutdown_connections 2023-07-14T18:18:40.421 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.368+0000 7f78454e8b80 1 -- 172.21.15.133:0/2244333160 wait complete. 2023-07-14T18:18:40.421 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.368+0000 7f78454e8b80 20 client.25658 trim_cache size 0 max 16384 2023-07-14T18:18:40.422 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:18:40.422 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-14T18:18:40.422 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:18:40.422 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:18:40.422 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:18:40.422 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:18:40.423 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:18:40.423 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:18:40.423 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:18:40.423 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:18:40.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.369+0000 7f7831bff700 1 -- 172.21.15.133:0/3820356411 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f78140a11d0 msgr2=0x7f78140a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:18:40.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.369+0000 7f7831bff700 1 --2- 172.21.15.133:0/3820356411 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f78140a11d0 0x7f78140a3680 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7824009610 tx=0x7f7824006040 comp rx=0 tx=0).stop 2023-07-14T18:18:40.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.369+0000 7f7831bff700 1 -- 172.21.15.133:0/3820356411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f782c0730a0 msgr2=0x7f782c102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:18:40.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.369+0000 7f7831bff700 1 --2- 172.21.15.133:0/3820356411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f782c0730a0 0x7f782c102f30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f781c00d980 tx=0x7f781c00dd40 comp rx=0 tx=0).stop 2023-07-14T18:18:40.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.370+0000 7f7831bff700 1 -- 172.21.15.133:0/3820356411 shutdown_connections 2023-07-14T18:18:40.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.370+0000 7f7831bff700 1 --2- 172.21.15.133:0/3820356411 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f78140a11d0 0x7f78140a3680 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.370+0000 7f7831bff700 1 --2- 172.21.15.133:0/3820356411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f782c107ad0 0x7f782c0ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.370+0000 7f7831bff700 1 --2- 172.21.15.133:0/3820356411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f782c0730a0 0x7f782c102f30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.370+0000 7f7831bff700 1 --2- 172.21.15.133:0/3820356411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f782c074680 0x7f782c103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:18:40.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.370+0000 7f7831bff700 1 -- 172.21.15.133:0/3820356411 >> 172.21.15.133:0/3820356411 conn(0x7f782c0fb210 msgr2=0x7f782c109c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:18:40.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.370+0000 7f7831bff700 1 -- 172.21.15.133:0/3820356411 shutdown_connections 2023-07-14T18:18:40.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:18:40.370+0000 7f7831bff700 1 -- 172.21.15.133:0/3820356411 wait complete. 2023-07-14T18:18:40.473 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:18:41.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:41 smithi171 ceph-mon[129241]: pgmap v408: 129 pgs: 129 active+clean; 7.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 95 MiB/s wr, 50 op/s 2023-07-14T18:18:41.414 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:41 smithi133 ceph-mon[126384]: pgmap v408: 129 pgs: 129 active+clean; 7.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 95 MiB/s wr, 50 op/s 2023-07-14T18:18:41.603 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:41 smithi161 ceph-mon[128993]: pgmap v408: 129 pgs: 129 active+clean; 7.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 95 MiB/s wr, 50 op/s 2023-07-14T18:18:42.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:42 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:18:42.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:42 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:18:42.603 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:42 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:18:43.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:43 smithi171 ceph-mon[129241]: pgmap v409: 129 pgs: 129 active+clean; 7.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 98 MiB/s wr, 51 op/s 2023-07-14T18:18:43.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:43 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:18:43.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:43 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:18:43.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:43 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:18:43.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:43 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:18:43.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:43 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:18:43.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:43 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:18:43.370 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:43 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:18:43.424 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:43 smithi133 ceph-mon[126384]: pgmap v409: 129 pgs: 129 active+clean; 7.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 98 MiB/s wr, 51 op/s 2023-07-14T18:18:43.424 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:43 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:18:43.424 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:43 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:18:43.425 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:43 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:18:43.425 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:43 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:18:43.425 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:43 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:18:43.425 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:43 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:18:43.425 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:43 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:18:43.603 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:43 smithi161 ceph-mon[128993]: pgmap v409: 129 pgs: 129 active+clean; 7.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 98 MiB/s wr, 51 op/s 2023-07-14T18:18:43.603 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:43 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:18:43.603 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:43 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:18:43.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:43 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:18:43.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:43 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:18:43.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:43 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:18:43.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:43 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:18:43.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:43 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:18:44.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:44 smithi133 ceph-mon[126384]: pgmap v410: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 117 MiB/s wr, 61 op/s 2023-07-14T18:18:44.603 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:44 smithi161 ceph-mon[128993]: pgmap v410: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 117 MiB/s wr, 61 op/s 2023-07-14T18:18:44.618 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:44 smithi171 ceph-mon[129241]: pgmap v410: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 117 MiB/s wr, 61 op/s 2023-07-14T18:18:47.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:46 smithi161 ceph-mon[128993]: pgmap v411: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 104 MiB/s wr, 52 op/s 2023-07-14T18:18:47.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:46 smithi171 ceph-mon[129241]: pgmap v411: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 104 MiB/s wr, 52 op/s 2023-07-14T18:18:47.411 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:46 smithi133 ceph-mon[126384]: pgmap v411: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 104 MiB/s wr, 52 op/s 2023-07-14T18:18:49.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:49 smithi161 ceph-mon[128993]: pgmap v412: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 188 MiB/s wr, 78 op/s 2023-07-14T18:18:49.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:49 smithi171 ceph-mon[129241]: pgmap v412: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 188 MiB/s wr, 78 op/s 2023-07-14T18:18:49.412 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:49 smithi133 ceph-mon[126384]: pgmap v412: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 188 MiB/s wr, 78 op/s 2023-07-14T18:18:51.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:51 smithi161 ceph-mon[128993]: pgmap v413: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 189 MiB/s wr, 65 op/s 2023-07-14T18:18:51.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:51 smithi171 ceph-mon[129241]: pgmap v413: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 189 MiB/s wr, 65 op/s 2023-07-14T18:18:51.410 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:51 smithi133 ceph-mon[126384]: pgmap v413: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 189 MiB/s wr, 65 op/s 2023-07-14T18:18:51.620 INFO:teuthology.orchestra.run.smithi133.stderr:set primary keys... 2023-07-14T18:18:53.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:53 smithi161 ceph-mon[128993]: pgmap v414: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 248 MiB/s wr, 79 op/s 2023-07-14T18:18:53.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:53 smithi171 ceph-mon[129241]: pgmap v414: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 248 MiB/s wr, 79 op/s 2023-07-14T18:18:53.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:53 smithi133 ceph-mon[126384]: pgmap v414: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 248 MiB/s wr, 79 op/s 2023-07-14T18:18:55.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:55 smithi161 ceph-mon[128993]: pgmap v415: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 286 MiB/s wr, 92 op/s 2023-07-14T18:18:55.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:55 smithi171 ceph-mon[129241]: pgmap v415: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 286 MiB/s wr, 92 op/s 2023-07-14T18:18:55.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:55 smithi133 ceph-mon[126384]: pgmap v415: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 286 MiB/s wr, 92 op/s 2023-07-14T18:18:57.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:57 smithi161 ceph-mon[128993]: pgmap v416: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 263 MiB/s wr, 79 op/s 2023-07-14T18:18:57.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:57 smithi171 ceph-mon[129241]: pgmap v416: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 263 MiB/s wr, 79 op/s 2023-07-14T18:18:57.410 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:57 smithi133 ceph-mon[126384]: pgmap v416: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 263 MiB/s wr, 79 op/s 2023-07-14T18:18:58.603 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:18:58 smithi161 ceph-mon[128993]: pgmap v417: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 298 MiB/s wr, 94 op/s 2023-07-14T18:18:58.619 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:18:58 smithi171 ceph-mon[129241]: pgmap v417: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 298 MiB/s wr, 94 op/s 2023-07-14T18:18:58.663 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:18:58 smithi133 ceph-mon[126384]: pgmap v417: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 298 MiB/s wr, 94 op/s 2023-07-14T18:19:01.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:00 smithi161 ceph-mon[128993]: pgmap v418: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 235 MiB/s wr, 76 op/s 2023-07-14T18:19:01.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:00 smithi171 ceph-mon[129241]: pgmap v418: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 235 MiB/s wr, 76 op/s 2023-07-14T18:19:01.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:00 smithi133 ceph-mon[126384]: pgmap v418: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 235 MiB/s wr, 76 op/s 2023-07-14T18:19:03.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:03 smithi161 ceph-mon[128993]: pgmap v419: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 258 MiB/s wr, 79 op/s 2023-07-14T18:19:03.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:03 smithi171 ceph-mon[129241]: pgmap v419: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 258 MiB/s wr, 79 op/s 2023-07-14T18:19:03.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:03 smithi133 ceph-mon[126384]: pgmap v419: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 258 MiB/s wr, 79 op/s 2023-07-14T18:19:05.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:05 smithi161 ceph-mon[128993]: pgmap v420: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 262 MiB/s wr, 83 op/s 2023-07-14T18:19:05.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:05 smithi171 ceph-mon[129241]: pgmap v420: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 262 MiB/s wr, 83 op/s 2023-07-14T18:19:05.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:05 smithi133 ceph-mon[126384]: pgmap v420: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 262 MiB/s wr, 83 op/s 2023-07-14T18:19:07.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:07 smithi161 ceph-mon[128993]: pgmap v421: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 213 MiB/s wr, 66 op/s 2023-07-14T18:19:07.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:07 smithi171 ceph-mon[129241]: pgmap v421: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 213 MiB/s wr, 66 op/s 2023-07-14T18:19:07.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:07 smithi133 ceph-mon[126384]: pgmap v421: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 213 MiB/s wr, 66 op/s 2023-07-14T18:19:09.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:09 smithi161 ceph-mon[128993]: pgmap v422: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 215 MiB/s wr, 68 op/s 2023-07-14T18:19:09.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:09 smithi171 ceph-mon[129241]: pgmap v422: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 215 MiB/s wr, 68 op/s 2023-07-14T18:19:09.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:09 smithi133 ceph-mon[126384]: pgmap v422: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 215 MiB/s wr, 68 op/s 2023-07-14T18:19:10.474 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:19:10.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.658+0000 7ff782fff700 1 -- 172.21.15.133:0/2792058180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff77c103070 msgr2=0x7ff77c105450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:19:10.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.658+0000 7ff782fff700 1 --2- 172.21.15.133:0/2792058180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff77c103070 0x7ff77c105450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff768009990 tx=0x7ff768009ca0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.658+0000 7ff782fff700 1 -- 172.21.15.133:0/2792058180 shutdown_connections 2023-07-14T18:19:10.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.658+0000 7ff782fff700 1 --2- 172.21.15.133:0/2792058180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff77c105990 0x7ff77c107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.658+0000 7ff782fff700 1 --2- 172.21.15.133:0/2792058180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff77c100750 0x7ff77c102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.658+0000 7ff782fff700 1 --2- 172.21.15.133:0/2792058180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff77c103070 0x7ff77c105450 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.658+0000 7ff782fff700 1 -- 172.21.15.133:0/2792058180 >> 172.21.15.133:0/2792058180 conn(0x7ff77c0fa130 msgr2=0x7ff77c0fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:19:10.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.658+0000 7ff782fff700 1 -- 172.21.15.133:0/2792058180 shutdown_connections 2023-07-14T18:19:10.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.658+0000 7ff782fff700 1 -- 172.21.15.133:0/2792058180 wait complete. 2023-07-14T18:19:10.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.659+0000 7ff782fff700 1 Processor -- start 2023-07-14T18:19:10.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.659+0000 7ff782fff700 1 -- start start 2023-07-14T18:19:10.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.660+0000 7ff782fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff77c100750 0x7ff77c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:10.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.660+0000 7ff782fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff77c103070 0x7ff77c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:10.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.660+0000 7ff782fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff77c105990 0x7ff77c19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:10.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.660+0000 7ff782fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c19f710 con 0x7ff77c100750 2023-07-14T18:19:10.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.660+0000 7ff782fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c19f850 con 0x7ff77c105990 2023-07-14T18:19:10.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.660+0000 7ff782fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c19f9e0 con 0x7ff77c103070 2023-07-14T18:19:10.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.660+0000 7ff77bfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff77c103070 0x7ff77c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:10.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.660+0000 7ff77bfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff77c103070 0x7ff77c1980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:54946/0 (socket says 172.21.15.133:54946) 2023-07-14T18:19:10.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.660+0000 7ff77bfff700 1 -- 172.21.15.133:0/327590352 learned_addr learned my addr 172.21.15.133:0/327590352 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:19:10.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.660+0000 7ff780d9b700 1 --2- 172.21.15.133:0/327590352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff77c100750 0x7ff77c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:10.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.660+0000 7ff78159c700 1 --2- 172.21.15.133:0/327590352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff77c105990 0x7ff77c19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:10.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.660+0000 7ff77bfff700 1 -- 172.21.15.133:0/327590352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff77c105990 msgr2=0x7ff77c19f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:19:10.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.660+0000 7ff77bfff700 1 --2- 172.21.15.133:0/327590352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff77c105990 0x7ff77c19f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.660+0000 7ff77bfff700 1 -- 172.21.15.133:0/327590352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff77c100750 msgr2=0x7ff77c197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:19:10.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.660+0000 7ff77bfff700 1 --2- 172.21.15.133:0/327590352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff77c100750 0x7ff77c197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.660+0000 7ff77bfff700 1 -- 172.21.15.133:0/327590352 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7680095f0 con 0x7ff77c103070 2023-07-14T18:19:10.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.661+0000 7ff77bfff700 1 --2- 172.21.15.133:0/327590352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff77c103070 0x7ff77c1980d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff76800b680 tx=0x7ff768005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:19:10.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.661+0000 7ff779ffb700 1 -- 172.21.15.133:0/327590352 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76801d070 con 0x7ff77c103070 2023-07-14T18:19:10.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.661+0000 7ff779ffb700 1 -- 172.21.15.133:0/327590352 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff768022470 con 0x7ff77c103070 2023-07-14T18:19:10.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.661+0000 7ff782fff700 1 -- 172.21.15.133:0/327590352 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff77c19fc60 con 0x7ff77c103070 2023-07-14T18:19:10.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.661+0000 7ff779ffb700 1 -- 172.21.15.133:0/327590352 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff768017650 con 0x7ff77c103070 2023-07-14T18:19:10.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.661+0000 7ff782fff700 1 -- 172.21.15.133:0/327590352 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77c1a00e0 con 0x7ff77c103070 2023-07-14T18:19:10.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.664+0000 7ff779ffb700 1 -- 172.21.15.133:0/327590352 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff7680225e0 con 0x7ff77c103070 2023-07-14T18:19:10.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.665+0000 7ff796a16b80 1 Processor -- start 2023-07-14T18:19:10.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.666+0000 7ff779ffb700 1 --2- 172.21.15.133:0/327590352 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff76c0a11d0 0x7ff76c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:10.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.666+0000 7ff796a16b80 1 -- start start 2023-07-14T18:19:10.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.666+0000 7ff779ffb700 1 -- 172.21.15.133:0/327590352 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff7680ce0a0 con 0x7ff77c103070 2023-07-14T18:19:10.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.666+0000 7ff78159c700 1 --2- 172.21.15.133:0/327590352 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff76c0a11d0 0x7ff76c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:10.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.666+0000 7ff796a16b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577cee9af60 0x5577cee9b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:10.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.666+0000 7ff796a16b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577cee9b8b0 0x5577cee9bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:10.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.666+0000 7ff796a16b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577cee9c4b0 0x5577cee9c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:10.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.666+0000 7ff796a16b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ced9fcb0 con 0x5577cee9b8b0 2023-07-14T18:19:10.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.666+0000 7ff796a16b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577cedf2a40 con 0x5577cee9c4b0 2023-07-14T18:19:10.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.666+0000 7ff796a16b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577cedebed0 con 0x5577cee9af60 2023-07-14T18:19:10.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.667+0000 7ff780d9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577cee9af60 0x5577cee9b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:10.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.667+0000 7ff780d9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577cee9af60 0x5577cee9b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:54956/0 (socket says 172.21.15.133:54956) 2023-07-14T18:19:10.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.667+0000 7ff780d9b700 1 -- 172.21.15.133:0/3161511564 learned_addr learned my addr 172.21.15.133:0/3161511564 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:19:10.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.667+0000 7ff78159c700 1 --2- 172.21.15.133:0/3161511564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577cee9c4b0 0x5577cee9c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:10.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.667+0000 7ff77bfff700 1 --2- 172.21.15.133:0/3161511564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577cee9b8b0 0x5577cee9bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:10.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.667+0000 7ff780d9b700 1 -- 172.21.15.133:0/3161511564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577cee9c4b0 msgr2=0x5577cee9c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:19:10.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.667+0000 7ff780d9b700 1 --2- 172.21.15.133:0/3161511564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577cee9c4b0 0x5577cee9c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.667+0000 7ff780d9b700 1 -- 172.21.15.133:0/3161511564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577cee9b8b0 msgr2=0x5577cee9bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:19:10.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.667+0000 7ff780d9b700 1 --2- 172.21.15.133:0/3161511564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577cee9b8b0 0x5577cee9bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.667+0000 7ff780d9b700 1 -- 172.21.15.133:0/3161511564 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577cec19de0 con 0x5577cee9af60 2023-07-14T18:19:10.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.667+0000 7ff78159c700 1 --2- 172.21.15.133:0/3161511564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577cee9c4b0 0x5577cee9c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:19:10.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.667+0000 7ff78159c700 1 --2- 172.21.15.133:0/327590352 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff76c0a11d0 0x7ff76c0a3680 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff77400f590 tx=0x7ff77400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:19:10.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.667+0000 7ff780d9b700 1 --2- 172.21.15.133:0/3161511564 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577cee9af60 0x5577cee9b370 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff770009d80 tx=0x7ff770011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aebca5130b0efcc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:19:10.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.667+0000 7ff766ffd700 1 -- 172.21.15.133:0/3161511564 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff770004230 con 0x5577cee9af60 2023-07-14T18:19:10.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.668+0000 7ff766ffd700 1 -- 172.21.15.133:0/3161511564 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff770004390 con 0x5577cee9af60 2023-07-14T18:19:10.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.668+0000 7ff766ffd700 1 -- 172.21.15.133:0/3161511564 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff770019840 con 0x5577cee9af60 2023-07-14T18:19:10.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.668+0000 7ff796a16b80 1 -- 172.21.15.133:0/3161511564 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577cee9af60 msgr2=0x5577cee9b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:19:10.735 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.668+0000 7ff796a16b80 1 --2- 172.21.15.133:0/3161511564 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577cee9af60 0x5577cee9b370 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff770009d80 tx=0x7ff770011720 comp rx=0 tx=0).stop 2023-07-14T18:19:10.735 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.668+0000 7ff796a16b80 1 -- 172.21.15.133:0/3161511564 shutdown_connections 2023-07-14T18:19:10.735 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.668+0000 7ff796a16b80 1 --2- 172.21.15.133:0/3161511564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577cee9c4b0 0x5577cee9c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.735 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.668+0000 7ff796a16b80 1 --2- 172.21.15.133:0/3161511564 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577cee9af60 0x5577cee9b370 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.735 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.668+0000 7ff796a16b80 1 --2- 172.21.15.133:0/3161511564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577cee9b8b0 0x5577cee9bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.735 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.668+0000 7ff796a16b80 1 -- 172.21.15.133:0/3161511564 >> 172.21.15.133:0/3161511564 conn(0x5577cee90680 msgr2=0x5577cee959d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:19:10.735 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.668+0000 7ff796a16b80 1 -- 172.21.15.133:0/3161511564 shutdown_connections 2023-07-14T18:19:10.735 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.669+0000 7ff796a16b80 1 -- 172.21.15.133:0/3161511564 wait complete. 2023-07-14T18:19:10.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.669+0000 7ff796a16b80 1 -- start start 2023-07-14T18:19:10.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.669+0000 7ff796a16b80 1 Processor -- start 2023-07-14T18:19:10.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.670+0000 7ff796a16b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577cee9af60 0x5577ceea48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:10.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.670+0000 7ff796a16b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577cee9b8b0 0x5577ceea4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:10.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.670+0000 7ff796a16b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577cee9c4b0 0x5577ceeabe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:10.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.670+0000 7ff796a16b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ced8bac0 con 0x5577cee9c4b0 2023-07-14T18:19:10.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.670+0000 7ff796a16b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ced8b7c0 con 0x5577cee9b8b0 2023-07-14T18:19:10.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.670+0000 7ff796a16b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577cec6f920 con 0x5577cee9af60 2023-07-14T18:19:10.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.670+0000 7ff796a16b80 10 client.0 _ll_register_callbacks cb 0x5577ced89760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:19:10.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.670+0000 7ff780d9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577cee9af60 0x5577ceea48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:10.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.670+0000 7ff77bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577cee9b8b0 0x5577ceea4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:10.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.670+0000 7ff780d9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577cee9af60 0x5577ceea48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:54966/0 (socket says 172.21.15.133:54966) 2023-07-14T18:19:10.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.670+0000 7ff780d9b700 1 -- 172.21.15.133:0/1969504763 learned_addr learned my addr 172.21.15.133:0/1969504763 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:19:10.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.670+0000 7ff78159c700 1 --2- 172.21.15.133:0/1969504763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577cee9c4b0 0x5577ceeabe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:10.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.670+0000 7ff780d9b700 1 -- 172.21.15.133:0/1969504763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577cee9b8b0 msgr2=0x5577ceea4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:19:10.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.670+0000 7ff780d9b700 1 --2- 172.21.15.133:0/1969504763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577cee9b8b0 0x5577ceea4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.670+0000 7ff780d9b700 1 -- 172.21.15.133:0/1969504763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577cee9c4b0 msgr2=0x5577ceeabe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:19:10.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.670+0000 7ff780d9b700 1 --2- 172.21.15.133:0/1969504763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577cee9c4b0 0x5577ceeabe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.670+0000 7ff780d9b700 1 -- 172.21.15.133:0/1969504763 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7700099e0 con 0x5577cee9af60 2023-07-14T18:19:10.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.671+0000 7ff780d9b700 1 --2- 172.21.15.133:0/1969504763 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577cee9af60 0x5577ceea48b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff77001b040 tx=0x7ff770003930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dadbfeb556f1f1f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:19:10.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.671+0000 7ff75bfff700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:19:10.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.671+0000 7ff75bfff700 1 -- 172.21.15.133:0/1969504763 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff770005450 con 0x5577cee9af60 2023-07-14T18:19:10.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.671+0000 7ff796a16b80 1 -- 172.21.15.133:0/1969504763 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577ced454f0 con 0x5577cee9af60 2023-07-14T18:19:10.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.671+0000 7ff75bfff700 1 -- 172.21.15.133:0/1969504763 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff7700055b0 con 0x5577cee9af60 2023-07-14T18:19:10.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.671+0000 7ff75bfff700 1 -- 172.21.15.133:0/1969504763 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff770009ed0 con 0x5577cee9af60 2023-07-14T18:19:10.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.671+0000 7ff796a16b80 10 client.25705 fetch_fsmap learned FSMap version 30 2023-07-14T18:19:10.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.671+0000 7ff796a16b80 1 -- 172.21.15.133:0/1969504763 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5577cec02c60 con 0x5577cee9af60 2023-07-14T18:19:10.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.671+0000 7ff75bfff700 1 -- 172.21.15.133:0/1969504763 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5577ced454f0 con 0x5577cee9af60 2023-07-14T18:19:10.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.671+0000 7ff75bfff700 1 -- 172.21.15.133:0/1969504763 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff770021040 con 0x5577cee9af60 2023-07-14T18:19:10.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.672+0000 7ff796a16b80 10 client.25705 fetch_fsmap finished waiting for FSMap version 30 2023-07-14T18:19:10.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.672+0000 7ff796a16b80 10 client.25705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:19:10.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.672+0000 7ff796a16b80 20 client.25705 populate_metadata read hostname 'smithi133' 2023-07-14T18:19:10.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.672+0000 7ff796a16b80 1 --2- 172.21.15.133:0/1969504763 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5577ceeb05e0 0x5577ceeb29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:10.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.672+0000 7ff796a16b80 4 client.25705 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:19:10.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.672+0000 7ff796a16b80 1 -- 172.21.15.133:0/1969504763 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577cee69da0 con 0x5577ceeb05e0 2023-07-14T18:19:10.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.672+0000 7ff77bfff700 1 --2- 172.21.15.133:0/1969504763 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5577ceeb05e0 0x5577ceeb29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:10.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.672+0000 7ff77bfff700 1 --2- 172.21.15.133:0/1969504763 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5577ceeb05e0 0x5577ceeb29f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1c91032e76bdd5c server_cookie=f1f0a370133558ae in_seq=0 out_seq=0 2023-07-14T18:19:10.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.672+0000 7ff75bfff700 10 client.25705 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:19:10.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.673+0000 7ff75bfff700 1 -- 172.21.15.133:0/1969504763 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5577cee69da0 con 0x5577ceeb05e0 2023-07-14T18:19:10.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.673+0000 7ff75bfff700 10 client.25705 handle_command_reply: tid=0 2023-07-14T18:19:10.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.673+0000 7ff796a16b80 1 client.25705 shutdown 2023-07-14T18:19:10.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.673+0000 7ff796a16b80 1 -- 172.21.15.133:0/1969504763 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577cee9af60 msgr2=0x5577ceea48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:19:10.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.673+0000 7ff796a16b80 1 --2- 172.21.15.133:0/1969504763 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577cee9af60 0x5577ceea48b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff77001b040 tx=0x7ff770003930 comp rx=0 tx=0).stop 2023-07-14T18:19:10.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.673+0000 7ff796a16b80 1 -- 172.21.15.133:0/1969504763 shutdown_connections 2023-07-14T18:19:10.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.673+0000 7ff796a16b80 1 --2- 172.21.15.133:0/1969504763 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5577ceeb05e0 0x5577ceeb29f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.673+0000 7ff796a16b80 1 --2- 172.21.15.133:0/1969504763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577cee9c4b0 0x5577ceeabe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.673+0000 7ff796a16b80 1 --2- 172.21.15.133:0/1969504763 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577cee9af60 0x5577ceea48b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.673+0000 7ff75bfff700 0 client.25705 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:19:10.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.673+0000 7ff796a16b80 1 --2- 172.21.15.133:0/1969504763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577cee9b8b0 0x5577ceea4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.673+0000 7ff796a16b80 1 -- 172.21.15.133:0/1969504763 >> 172.21.15.133:0/1969504763 conn(0x5577cee90680 msgr2=0x5577cee94650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:19:10.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.674+0000 7ff796a16b80 1 -- 172.21.15.133:0/1969504763 shutdown_connections 2023-07-14T18:19:10.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.674+0000 7ff796a16b80 1 -- 172.21.15.133:0/1969504763 wait complete. 2023-07-14T18:19:10.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.674+0000 7ff796a16b80 20 client.25705 trim_cache size 0 max 16384 2023-07-14T18:19:10.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.691+0000 7ff796a16b80 1 Processor -- start 2023-07-14T18:19:10.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.691+0000 7ff796a16b80 1 -- start start 2023-07-14T18:19:10.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.691+0000 7ff796a16b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577cee9af60 0x5577ceea36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:10.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.691+0000 7ff796a16b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577cee9b8b0 0x5577ceeabdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:10.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.692+0000 7ff796a16b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577cee9c4b0 0x5577ceea27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:10.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.692+0000 7ff796a16b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ceea5aa0 con 0x5577cee9c4b0 2023-07-14T18:19:10.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.692+0000 7ff796a16b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577cee9c330 con 0x5577cee9b8b0 2023-07-14T18:19:10.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.692+0000 7ff796a16b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ceeadaa0 con 0x5577cee9af60 2023-07-14T18:19:10.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.692+0000 7ff77bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577cee9b8b0 0x5577ceeabdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:10.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.692+0000 7ff780d9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577cee9af60 0x5577ceea36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:10.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.692+0000 7ff780d9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577cee9af60 0x5577ceea36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:54970/0 (socket says 172.21.15.133:54970) 2023-07-14T18:19:10.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.692+0000 7ff77bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577cee9b8b0 0x5577ceeabdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:47672/0 (socket says 172.21.15.133:47672) 2023-07-14T18:19:10.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.692+0000 7ff780d9b700 1 -- 172.21.15.133:0/1598697620 learned_addr learned my addr 172.21.15.133:0/1598697620 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:19:10.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.692+0000 7ff78159c700 1 --2- 172.21.15.133:0/1598697620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577cee9c4b0 0x5577ceea27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:10.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.692+0000 7ff780d9b700 1 -- 172.21.15.133:0/1598697620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577cee9b8b0 msgr2=0x5577ceeabdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:19:10.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.692+0000 7ff780d9b700 1 --2- 172.21.15.133:0/1598697620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577cee9b8b0 0x5577ceeabdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.692+0000 7ff780d9b700 1 -- 172.21.15.133:0/1598697620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577cee9c4b0 msgr2=0x5577ceea27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:19:10.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.692+0000 7ff780d9b700 1 --2- 172.21.15.133:0/1598697620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577cee9c4b0 0x5577ceea27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.692+0000 7ff780d9b700 1 -- 172.21.15.133:0/1598697620 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577cec40250 con 0x5577cee9af60 2023-07-14T18:19:10.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.692+0000 7ff780d9b700 1 --2- 172.21.15.133:0/1598697620 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577cee9af60 0x5577ceea36a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff770004360 tx=0x7ff770005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e114aacaa362b70c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:19:10.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.693+0000 7ff7677fe700 1 -- 172.21.15.133:0/1598697620 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff770009ed0 con 0x5577cee9af60 2023-07-14T18:19:10.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.693+0000 7ff7677fe700 1 -- 172.21.15.133:0/1598697620 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff770003a50 con 0x5577cee9af60 2023-07-14T18:19:10.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.693+0000 7ff7677fe700 1 -- 172.21.15.133:0/1598697620 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff770005520 con 0x5577cee9af60 2023-07-14T18:19:10.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.693+0000 7ff796a16b80 1 -- 172.21.15.133:0/1598697620 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577cee9af60 msgr2=0x5577ceea36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:19:10.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.693+0000 7ff796a16b80 1 --2- 172.21.15.133:0/1598697620 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577cee9af60 0x5577ceea36a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff770004360 tx=0x7ff770005dc0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.693+0000 7ff796a16b80 1 -- 172.21.15.133:0/1598697620 shutdown_connections 2023-07-14T18:19:10.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.693+0000 7ff796a16b80 1 --2- 172.21.15.133:0/1598697620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577cee9c4b0 0x5577ceea27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.693+0000 7ff796a16b80 1 --2- 172.21.15.133:0/1598697620 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577cee9af60 0x5577ceea36a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.693+0000 7ff796a16b80 1 --2- 172.21.15.133:0/1598697620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577cee9b8b0 0x5577ceeabdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.693+0000 7ff796a16b80 1 -- 172.21.15.133:0/1598697620 >> 172.21.15.133:0/1598697620 conn(0x5577cee90680 msgr2=0x5577ceeac720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:19:10.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.693+0000 7ff796a16b80 1 -- 172.21.15.133:0/1598697620 shutdown_connections 2023-07-14T18:19:10.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.694+0000 7ff796a16b80 1 -- 172.21.15.133:0/1598697620 wait complete. 2023-07-14T18:19:10.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.694+0000 7ff796a16b80 1 -- start start 2023-07-14T18:19:10.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.694+0000 7ff796a16b80 1 Processor -- start 2023-07-14T18:19:10.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.694+0000 7ff796a16b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577cee9af60 0x5577ceeab770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:10.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.694+0000 7ff796a16b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577cee9b8b0 0x5577ceeabcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:10.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.694+0000 7ff796a16b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577cee9c4b0 0x5577ceea83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:10.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.694+0000 7ff796a16b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ceea8b30 con 0x5577cee9af60 2023-07-14T18:19:10.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.694+0000 7ff796a16b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ceea8ca0 con 0x5577cee9b8b0 2023-07-14T18:19:10.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.695+0000 7ff796a16b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ceea8e60 con 0x5577cee9c4b0 2023-07-14T18:19:10.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.695+0000 7ff796a16b80 10 client.0 _ll_register_callbacks cb 0x7ff770009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:19:10.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.695+0000 7ff77bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577cee9b8b0 0x5577ceeabcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:10.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.695+0000 7ff780d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577cee9af60 0x5577ceeab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:10.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.695+0000 7ff78159c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577cee9c4b0 0x5577ceea83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:10.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.695+0000 7ff77bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577cee9b8b0 0x5577ceeabcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:47680/0 (socket says 172.21.15.133:47680) 2023-07-14T18:19:10.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.695+0000 7ff77bfff700 1 -- 172.21.15.133:0/1543827853 learned_addr learned my addr 172.21.15.133:0/1543827853 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:19:10.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.695+0000 7ff77bfff700 1 -- 172.21.15.133:0/1543827853 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577cee9c4b0 msgr2=0x5577ceea83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:19:10.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.695+0000 7ff77bfff700 1 --2- 172.21.15.133:0/1543827853 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577cee9c4b0 0x5577ceea83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.695+0000 7ff77bfff700 1 -- 172.21.15.133:0/1543827853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577cee9af60 msgr2=0x5577ceeab770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:19:10.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.695+0000 7ff77bfff700 1 --2- 172.21.15.133:0/1543827853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577cee9af60 0x5577ceeab770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.695+0000 7ff77bfff700 1 -- 172.21.15.133:0/1543827853 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577cec02c60 con 0x5577cee9b8b0 2023-07-14T18:19:10.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.695+0000 7ff77bfff700 1 --2- 172.21.15.133:0/1543827853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577cee9b8b0 0x5577ceeabcb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff7680cb810 tx=0x7ff7680a1040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2eeb312d6a273314 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:19:10.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.695+0000 7ff75bfff700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:19:10.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.696+0000 7ff75bfff700 1 -- 172.21.15.133:0/1543827853 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7680b7070 con 0x5577cee9b8b0 2023-07-14T18:19:10.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.696+0000 7ff75bfff700 1 -- 172.21.15.133:0/1543827853 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff76809b1b0 con 0x5577cee9b8b0 2023-07-14T18:19:10.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.696+0000 7ff796a16b80 1 -- 172.21.15.133:0/1543827853 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577cede4eb0 con 0x5577cee9b8b0 2023-07-14T18:19:10.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.696+0000 7ff75bfff700 1 -- 172.21.15.133:0/1543827853 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff76809f1e0 con 0x5577cee9b8b0 2023-07-14T18:19:10.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.696+0000 7ff796a16b80 10 client.25676 fetch_fsmap learned FSMap version 30 2023-07-14T18:19:10.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.696+0000 7ff796a16b80 1 -- 172.21.15.133:0/1543827853 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5577cec182b0 con 0x5577cee9b8b0 2023-07-14T18:19:10.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.696+0000 7ff75bfff700 1 -- 172.21.15.133:0/1543827853 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5577cede4eb0 con 0x5577cee9b8b0 2023-07-14T18:19:10.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.696+0000 7ff75bfff700 1 -- 172.21.15.133:0/1543827853 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff76809b410 con 0x5577cee9b8b0 2023-07-14T18:19:10.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.696+0000 7ff796a16b80 10 client.25676 fetch_fsmap finished waiting for FSMap version 30 2023-07-14T18:19:10.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.696+0000 7ff796a16b80 10 client.25676 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:19:10.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.696+0000 7ff796a16b80 20 client.25676 populate_metadata read hostname 'smithi133' 2023-07-14T18:19:10.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.696+0000 7ff796a16b80 1 --2- 172.21.15.133:0/1543827853 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5577cef05650 0x5577cef07a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:10.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.696+0000 7ff796a16b80 4 client.25676 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:19:10.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.696+0000 7ff796a16b80 1 -- 172.21.15.133:0/1543827853 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5577ceda7050 con 0x5577cef05650 2023-07-14T18:19:10.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.697+0000 7ff780d9b700 1 --2- 172.21.15.133:0/1543827853 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5577cef05650 0x5577cef07a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:10.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.697+0000 7ff780d9b700 1 --2- 172.21.15.133:0/1543827853 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5577cef05650 0x5577cef07a30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=915f6dabdab71861 server_cookie=a29ba7ed27ef857e in_seq=0 out_seq=0 2023-07-14T18:19:10.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.697+0000 7ff75bfff700 10 client.25676 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:19:10.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.697+0000 7ff75bfff700 1 -- 172.21.15.133:0/1543827853 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5577ceda7050 con 0x5577cef05650 2023-07-14T18:19:10.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.697+0000 7ff75bfff700 10 client.25676 handle_command_reply: tid=0 2023-07-14T18:19:10.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.697+0000 7ff796a16b80 1 client.25676 shutdown 2023-07-14T18:19:10.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.697+0000 7ff796a16b80 1 -- 172.21.15.133:0/1543827853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577cee9b8b0 msgr2=0x5577ceeabcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:19:10.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.697+0000 7ff796a16b80 1 --2- 172.21.15.133:0/1543827853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577cee9b8b0 0x5577ceeabcb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff7680cb810 tx=0x7ff7680a1040 comp rx=0 tx=0).stop 2023-07-14T18:19:10.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.698+0000 7ff796a16b80 1 -- 172.21.15.133:0/1543827853 shutdown_connections 2023-07-14T18:19:10.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.698+0000 7ff796a16b80 1 --2- 172.21.15.133:0/1543827853 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5577cef05650 0x5577cef07a30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.698+0000 7ff796a16b80 1 --2- 172.21.15.133:0/1543827853 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5577cee9c4b0 0x5577ceea83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.698+0000 7ff796a16b80 1 --2- 172.21.15.133:0/1543827853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577cee9af60 0x5577ceeab770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.698+0000 7ff796a16b80 1 --2- 172.21.15.133:0/1543827853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5577cee9b8b0 0x5577ceeabcb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.698+0000 7ff796a16b80 1 -- 172.21.15.133:0/1543827853 >> 172.21.15.133:0/1543827853 conn(0x5577cee90680 msgr2=0x5577ceeac720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:19:10.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.698+0000 7ff75bfff700 0 client.25676 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:19:10.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.698+0000 7ff796a16b80 1 -- 172.21.15.133:0/1543827853 shutdown_connections 2023-07-14T18:19:10.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.698+0000 7ff796a16b80 1 -- 172.21.15.133:0/1543827853 wait complete. 2023-07-14T18:19:10.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.698+0000 7ff796a16b80 20 client.25676 trim_cache size 0 max 16384 2023-07-14T18:19:10.757 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:19:10.757 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-14T18:19:10.758 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:19:10.758 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:19:10.758 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:19:10.758 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:19:10.758 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:19:10.758 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:19:10.758 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:19:10.759 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:19:10.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.698+0000 7ff782fff700 1 -- 172.21.15.133:0/327590352 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff76c0a11d0 msgr2=0x7ff76c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:19:10.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.698+0000 7ff782fff700 1 --2- 172.21.15.133:0/327590352 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff76c0a11d0 0x7ff76c0a3680 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff77400f590 tx=0x7ff77400b040 comp rx=0 tx=0).stop 2023-07-14T18:19:10.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.698+0000 7ff782fff700 1 -- 172.21.15.133:0/327590352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff77c103070 msgr2=0x7ff77c1980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:19:10.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.698+0000 7ff782fff700 1 --2- 172.21.15.133:0/327590352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff77c103070 0x7ff77c1980d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff76800b680 tx=0x7ff768005740 comp rx=0 tx=0).stop 2023-07-14T18:19:10.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.698+0000 7ff782fff700 1 -- 172.21.15.133:0/327590352 shutdown_connections 2023-07-14T18:19:10.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.698+0000 7ff782fff700 1 --2- 172.21.15.133:0/327590352 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff76c0a11d0 0x7ff76c0a3680 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.698+0000 7ff782fff700 1 --2- 172.21.15.133:0/327590352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff77c105990 0x7ff77c19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.698+0000 7ff782fff700 1 --2- 172.21.15.133:0/327590352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff77c100750 0x7ff77c197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.699+0000 7ff782fff700 1 --2- 172.21.15.133:0/327590352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff77c103070 0x7ff77c1980d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:10.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.699+0000 7ff782fff700 1 -- 172.21.15.133:0/327590352 >> 172.21.15.133:0/327590352 conn(0x7ff77c0fa130 msgr2=0x7ff77c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:19:10.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.699+0000 7ff782fff700 1 -- 172.21.15.133:0/327590352 shutdown_connections 2023-07-14T18:19:10.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:10.699+0000 7ff782fff700 1 -- 172.21.15.133:0/327590352 wait complete. 2023-07-14T18:19:10.766 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:19:11.357 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:11 smithi161 ceph-mon[128993]: pgmap v423: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 169 MiB/s wr, 52 op/s 2023-07-14T18:19:11.370 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:11 smithi171 ceph-mon[129241]: pgmap v423: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 169 MiB/s wr, 52 op/s 2023-07-14T18:19:11.410 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:11 smithi133 ceph-mon[126384]: pgmap v423: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 169 MiB/s wr, 52 op/s 2023-07-14T18:19:13.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:13 smithi161 ceph-mon[128993]: pgmap v424: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 149 MiB/s wr, 52 op/s 2023-07-14T18:19:13.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:13 smithi171 ceph-mon[129241]: pgmap v424: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 149 MiB/s wr, 52 op/s 2023-07-14T18:19:13.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:13 smithi133 ceph-mon[126384]: pgmap v424: 129 pgs: 129 active+clean; 7.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 149 MiB/s wr, 52 op/s 2023-07-14T18:19:15.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:15 smithi161 ceph-mon[128993]: pgmap v425: 129 pgs: 129 active+clean; 7.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 113 MiB/s wr, 53 op/s 2023-07-14T18:19:15.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:15 smithi171 ceph-mon[129241]: pgmap v425: 129 pgs: 129 active+clean; 7.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 113 MiB/s wr, 53 op/s 2023-07-14T18:19:15.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:15 smithi133 ceph-mon[126384]: pgmap v425: 129 pgs: 129 active+clean; 7.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 113 MiB/s wr, 53 op/s 2023-07-14T18:19:17.354 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:17 smithi161 ceph-mon[128993]: pgmap v426: 129 pgs: 129 active+clean; 7.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 51 MiB/s wr, 34 op/s 2023-07-14T18:19:17.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:17 smithi171 ceph-mon[129241]: pgmap v426: 129 pgs: 129 active+clean; 7.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 51 MiB/s wr, 34 op/s 2023-07-14T18:19:17.410 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:17 smithi133 ceph-mon[126384]: pgmap v426: 129 pgs: 129 active+clean; 7.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 51 MiB/s wr, 34 op/s 2023-07-14T18:19:19.354 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:19 smithi161 ceph-mon[128993]: pgmap v427: 129 pgs: 129 active+clean; 7.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 76 MiB/s wr, 76 op/s 2023-07-14T18:19:19.354 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:19 smithi161 ceph-mon[128993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:19:19.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:19 smithi171 ceph-mon[129241]: pgmap v427: 129 pgs: 129 active+clean; 7.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 76 MiB/s wr, 76 op/s 2023-07-14T18:19:19.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:19 smithi171 ceph-mon[129241]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:19:19.410 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:19 smithi133 ceph-mon[126384]: pgmap v427: 129 pgs: 129 active+clean; 7.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 76 MiB/s wr, 76 op/s 2023-07-14T18:19:19.410 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:19 smithi133 ceph-mon[126384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:19:21.354 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:21 smithi161 ceph-mon[128993]: pgmap v428: 129 pgs: 129 active+clean; 7.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 74 MiB/s wr, 82 op/s 2023-07-14T18:19:21.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:21 smithi171 ceph-mon[129241]: pgmap v428: 129 pgs: 129 active+clean; 7.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 74 MiB/s wr, 82 op/s 2023-07-14T18:19:21.410 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:21 smithi133 ceph-mon[126384]: pgmap v428: 129 pgs: 129 active+clean; 7.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 74 MiB/s wr, 82 op/s 2023-07-14T18:19:23.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:23 smithi171 ceph-mon[129241]: pgmap v429: 129 pgs: 129 active+clean; 7.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 85 MiB/s wr, 92 op/s 2023-07-14T18:19:23.411 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:23 smithi133 ceph-mon[126384]: pgmap v429: 129 pgs: 129 active+clean; 7.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 85 MiB/s wr, 92 op/s 2023-07-14T18:19:23.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:23 smithi161 ceph-mon[128993]: pgmap v429: 129 pgs: 129 active+clean; 7.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 85 MiB/s wr, 92 op/s 2023-07-14T18:19:25.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:25 smithi171 ceph-mon[129241]: pgmap v430: 129 pgs: 129 active+clean; 7.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 99 MiB/s wr, 100 op/s 2023-07-14T18:19:25.410 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:25 smithi133 ceph-mon[126384]: pgmap v430: 129 pgs: 129 active+clean; 7.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 99 MiB/s wr, 100 op/s 2023-07-14T18:19:25.593 INFO:teuthology.orchestra.run.smithi133.stderr:done. 2023-07-14T18:19:25.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:25 smithi161 ceph-mon[128993]: pgmap v430: 129 pgs: 129 active+clean; 7.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 99 MiB/s wr, 100 op/s 2023-07-14T18:19:25.607 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2023-07-14T18:19:25.878 INFO:teuthology.orchestra.run.smithi133.stderr:starting vacuum...end. 2023-07-14T18:19:26.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:26 smithi161 ceph-mon[128993]: pgmap v431: 129 pgs: 129 active+clean; 7.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 89 MiB/s wr, 87 op/s 2023-07-14T18:19:26.619 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:26 smithi171 ceph-mon[129241]: pgmap v431: 129 pgs: 129 active+clean; 7.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 89 MiB/s wr, 87 op/s 2023-07-14T18:19:26.669 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:26 smithi133 ceph-mon[126384]: pgmap v431: 129 pgs: 129 active+clean; 7.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 89 MiB/s wr, 87 op/s 2023-07-14T18:19:28.910 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:28 smithi133 ceph-mon[126384]: pgmap v432: 129 pgs: 129 active+clean; 8.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 206 MiB/s wr, 126 op/s 2023-07-14T18:19:29.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:28 smithi161 ceph-mon[128993]: pgmap v432: 129 pgs: 129 active+clean; 8.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 206 MiB/s wr, 126 op/s 2023-07-14T18:19:29.119 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:28 smithi171 ceph-mon[129241]: pgmap v432: 129 pgs: 129 active+clean; 8.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 206 MiB/s wr, 126 op/s 2023-07-14T18:19:30.878 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 5.0 s, 2426.4 tps, lat 37.742 ms stddev 50.303 2023-07-14T18:19:31.354 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:31 smithi161 ceph-mon[128993]: pgmap v433: 129 pgs: 129 active+clean; 9.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 204 MiB/s wr, 104 op/s 2023-07-14T18:19:31.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:31 smithi171 ceph-mon[129241]: pgmap v433: 129 pgs: 129 active+clean; 9.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 204 MiB/s wr, 104 op/s 2023-07-14T18:19:31.411 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:31 smithi133 ceph-mon[126384]: pgmap v433: 129 pgs: 129 active+clean; 9.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 204 MiB/s wr, 104 op/s 2023-07-14T18:19:32.356 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:32 smithi161 ceph-mon[128993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:19:32.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:32 smithi171 ceph-mon[129241]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:19:32.413 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:32 smithi133 ceph-mon[126384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:19:33.354 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:33 smithi161 ceph-mon[128993]: pgmap v434: 129 pgs: 129 active+clean; 8.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 73 MiB/s rd, 206 MiB/s wr, 151 op/s 2023-07-14T18:19:33.370 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:33 smithi171 ceph-mon[129241]: pgmap v434: 129 pgs: 129 active+clean; 8.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 73 MiB/s rd, 206 MiB/s wr, 151 op/s 2023-07-14T18:19:33.410 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:33 smithi133 ceph-mon[126384]: pgmap v434: 129 pgs: 129 active+clean; 8.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 73 MiB/s rd, 206 MiB/s wr, 151 op/s 2023-07-14T18:19:35.354 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:35 smithi161 ceph-mon[128993]: pgmap v435: 129 pgs: 129 active+clean; 8.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 MiB/s rd, 205 MiB/s wr, 227 op/s 2023-07-14T18:19:35.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:35 smithi171 ceph-mon[129241]: pgmap v435: 129 pgs: 129 active+clean; 8.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 MiB/s rd, 205 MiB/s wr, 227 op/s 2023-07-14T18:19:35.414 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:35 smithi133 ceph-mon[126384]: pgmap v435: 129 pgs: 129 active+clean; 8.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 MiB/s rd, 205 MiB/s wr, 227 op/s 2023-07-14T18:19:35.878 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 10.0 s, 2281.3 tps, lat 43.747 ms stddev 55.928 2023-07-14T18:19:37.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:37 smithi171 ceph-mon[129241]: pgmap v436: 129 pgs: 129 active+clean; 8.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 MiB/s rd, 185 MiB/s wr, 255 op/s 2023-07-14T18:19:37.412 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:37 smithi133 ceph-mon[126384]: pgmap v436: 129 pgs: 129 active+clean; 8.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 MiB/s rd, 185 MiB/s wr, 255 op/s 2023-07-14T18:19:37.606 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:37 smithi161 ceph-mon[128993]: pgmap v436: 129 pgs: 129 active+clean; 8.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 MiB/s rd, 185 MiB/s wr, 255 op/s 2023-07-14T18:19:39.371 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:39 smithi171 ceph-mon[129241]: pgmap v437: 129 pgs: 129 active+clean; 8.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 MiB/s rd, 183 MiB/s wr, 616 op/s 2023-07-14T18:19:39.415 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:39 smithi133 ceph-mon[126384]: pgmap v437: 129 pgs: 129 active+clean; 8.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 MiB/s rd, 183 MiB/s wr, 616 op/s 2023-07-14T18:19:39.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:39 smithi161 ceph-mon[128993]: pgmap v437: 129 pgs: 129 active+clean; 8.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 MiB/s rd, 183 MiB/s wr, 616 op/s 2023-07-14T18:19:40.767 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:19:40.879 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 15.0 s, 1758.8 tps, lat 54.904 ms stddev 108.579 2023-07-14T18:19:41.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.217+0000 7fbc041ff700 1 -- 172.21.15.133:0/1866856523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbfc071010 msgr2=0x7fbbfc071420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:19:41.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.217+0000 7fbc041ff700 1 --2- 172.21.15.133:0/1866856523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbfc071010 0x7fbbfc071420 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80098f0 tx=0x7fbbf8009c00 comp rx=0 tx=0).stop 2023-07-14T18:19:41.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.240+0000 7fbc041ff700 1 -- 172.21.15.133:0/1866856523 shutdown_connections 2023-07-14T18:19:41.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.240+0000 7fbc041ff700 1 --2- 172.21.15.133:0/1866856523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbfc071010 0x7fbbfc071420 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.240+0000 7fbc041ff700 1 --2- 172.21.15.133:0/1866856523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbfc10be90 0x7fbbfc10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.240+0000 7fbc041ff700 1 --2- 172.21.15.133:0/1866856523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbbfc071960 0x7fbbfc10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.240+0000 7fbc041ff700 1 -- 172.21.15.133:0/1866856523 >> 172.21.15.133:0/1866856523 conn(0x7fbbfc06cbc0 msgr2=0x7fbbfc06eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:19:41.301 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.251+0000 7fbc041ff700 1 -- 172.21.15.133:0/1866856523 shutdown_connections 2023-07-14T18:19:41.302 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.253+0000 7fbc041ff700 1 -- 172.21.15.133:0/1866856523 wait complete. 2023-07-14T18:19:41.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.254+0000 7fbc041ff700 1 Processor -- start 2023-07-14T18:19:41.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.255+0000 7fbc041ff700 1 -- start start 2023-07-14T18:19:41.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.255+0000 7fbc041ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbbfc071010 0x7fbbfc113c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:41.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.255+0000 7fbc041ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbfc071960 0x7fbbfc114150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:41.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.255+0000 7fbc041ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbfc10be90 0x7fbbfc1b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:41.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.255+0000 7fbc041ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc117230 con 0x7fbbfc071960 2023-07-14T18:19:41.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.255+0000 7fbc041ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc1173a0 con 0x7fbbfc10be90 2023-07-14T18:19:41.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.255+0000 7fbc041ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc1b4f80 con 0x7fbbfc071010 2023-07-14T18:19:41.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.255+0000 7fbc01f9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbbfc071010 0x7fbbfc113c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:41.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.255+0000 7fbc01f9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbbfc071010 0x7fbbfc113c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:33018/0 (socket says 172.21.15.133:33018) 2023-07-14T18:19:41.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.255+0000 7fbc01f9b700 1 -- 172.21.15.133:0/2093452857 learned_addr learned my addr 172.21.15.133:0/2093452857 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:19:41.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.256+0000 7fbc0279c700 1 --2- 172.21.15.133:0/2093452857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbfc10be90 0x7fbbfc1b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:41.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.256+0000 7fbc01f9b700 1 -- 172.21.15.133:0/2093452857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbfc10be90 msgr2=0x7fbbfc1b4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:19:41.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.256+0000 7fbc01f9b700 1 --2- 172.21.15.133:0/2093452857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbfc10be90 0x7fbbfc1b4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.256+0000 7fbc01f9b700 1 -- 172.21.15.133:0/2093452857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbfc071960 msgr2=0x7fbbfc114150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:19:41.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.256+0000 7fbc01f9b700 1 --2- 172.21.15.133:0/2093452857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbfc071960 0x7fbbfc114150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.256+0000 7fbc01f9b700 1 -- 172.21.15.133:0/2093452857 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf8009520 con 0x7fbbfc071010 2023-07-14T18:19:41.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.257+0000 7fbc01f9b700 1 --2- 172.21.15.133:0/2093452857 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbbfc071010 0x7fbbfc113c10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800b790 tx=0x7fbbf800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:19:41.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.258+0000 7fbbf2ffd700 1 -- 172.21.15.133:0/2093452857 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf801c070 con 0x7fbbfc071010 2023-07-14T18:19:41.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.258+0000 7fbbf2ffd700 1 -- 172.21.15.133:0/2093452857 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbf800f840 con 0x7fbbfc071010 2023-07-14T18:19:41.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.258+0000 7fbbf2ffd700 1 -- 172.21.15.133:0/2093452857 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8004c40 con 0x7fbbfc071010 2023-07-14T18:19:41.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.258+0000 7fbc041ff700 1 -- 172.21.15.133:0/2093452857 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbfc1b51b0 con 0x7fbbfc071010 2023-07-14T18:19:41.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.259+0000 7fbc041ff700 1 -- 172.21.15.133:0/2093452857 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbfc1b5520 con 0x7fbbfc071010 2023-07-14T18:19:41.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.271+0000 7fbc17adfb80 1 Processor -- start 2023-07-14T18:19:41.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.271+0000 7fbc17adfb80 1 -- start start 2023-07-14T18:19:41.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.272+0000 7fbc17adfb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b68c1cff60 0x55b68c1d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:41.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.272+0000 7fbc17adfb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b68c1d08b0 0x55b68c1d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:41.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.272+0000 7fbc17adfb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b68c1d14b0 0x55b68c1d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:41.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.272+0000 7fbc17adfb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68c0d4cb0 con 0x55b68c1cff60 2023-07-14T18:19:41.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.272+0000 7fbc17adfb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68c127a40 con 0x55b68c1d14b0 2023-07-14T18:19:41.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.272+0000 7fbc17adfb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68c120ed0 con 0x55b68c1d08b0 2023-07-14T18:19:41.333 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.282+0000 7fbc0179a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b68c1d14b0 0x55b68c1d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:41.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.283+0000 7fbc0179a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b68c1d14b0 0x55b68c1d1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:50412/0 (socket says 172.21.15.133:50412) 2023-07-14T18:19:41.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.283+0000 7fbc0179a700 1 -- 172.21.15.133:0/112169659 learned_addr learned my addr 172.21.15.133:0/112169659 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:19:41.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.283+0000 7fbc0279c700 1 --2- 172.21.15.133:0/112169659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b68c1cff60 0x55b68c1d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:41.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.283+0000 7fbc0179a700 1 -- 172.21.15.133:0/112169659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b68c1d08b0 msgr2=0x55b68c1d0ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T18:19:41.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.283+0000 7fbc0179a700 1 --2- 172.21.15.133:0/112169659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b68c1d08b0 0x55b68c1d0ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.283+0000 7fbc0179a700 1 -- 172.21.15.133:0/112169659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b68c1cff60 msgr2=0x55b68c1d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:19:41.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.283+0000 7fbc0179a700 1 --2- 172.21.15.133:0/112169659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b68c1cff60 0x55b68c1d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.283+0000 7fbc0179a700 1 -- 172.21.15.133:0/112169659 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b68bf4ede0 con 0x55b68c1d14b0 2023-07-14T18:19:41.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.285+0000 7fbbf2ffd700 1 -- 172.21.15.133:0/2093452857 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fbbf8020070 con 0x7fbbfc071010 2023-07-14T18:19:41.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.286+0000 7fbc0179a700 1 --2- 172.21.15.133:0/112169659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b68c1d14b0 0x55b68c1d1920 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400e460 tx=0x7fbbf400e770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c50fd96b92da3633 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:19:41.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.286+0000 7fbbe7fff700 1 -- 172.21.15.133:0/112169659 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf40092d0 con 0x55b68c1d14b0 2023-07-14T18:19:41.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.286+0000 7fbbe7fff700 1 -- 172.21.15.133:0/112169659 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbf400f040 con 0x55b68c1d14b0 2023-07-14T18:19:41.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.286+0000 7fbbe7fff700 1 -- 172.21.15.133:0/112169659 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4014870 con 0x55b68c1d14b0 2023-07-14T18:19:41.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.287+0000 7fbc17adfb80 1 -- 172.21.15.133:0/112169659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b68c1d14b0 msgr2=0x55b68c1d1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:19:41.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.287+0000 7fbc17adfb80 1 --2- 172.21.15.133:0/112169659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b68c1d14b0 0x55b68c1d1920 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400e460 tx=0x7fbbf400e770 comp rx=0 tx=0).stop 2023-07-14T18:19:41.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.287+0000 7fbbf2ffd700 1 --2- 172.21.15.133:0/2093452857 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fbbe80a1350 0x7fbbe80a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:41.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.288+0000 7fbbf2ffd700 1 -- 172.21.15.133:0/2093452857 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbbf80cd990 con 0x7fbbfc071010 2023-07-14T18:19:41.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.291+0000 7fbc17adfb80 1 -- 172.21.15.133:0/112169659 shutdown_connections 2023-07-14T18:19:41.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.291+0000 7fbc17adfb80 1 --2- 172.21.15.133:0/112169659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b68c1d14b0 0x55b68c1d1920 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.291+0000 7fbc17adfb80 1 --2- 172.21.15.133:0/112169659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b68c1cff60 0x55b68c1d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.291+0000 7fbc17adfb80 1 --2- 172.21.15.133:0/112169659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b68c1d08b0 0x55b68c1d0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.291+0000 7fbc17adfb80 1 -- 172.21.15.133:0/112169659 >> 172.21.15.133:0/112169659 conn(0x55b68c1c5680 msgr2=0x55b68c1ca9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:19:41.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.291+0000 7fbc0279c700 1 --2- 172.21.15.133:0/2093452857 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fbbe80a1350 0x7fbbe80a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:41.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.292+0000 7fbc0279c700 1 --2- 172.21.15.133:0/2093452857 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fbbe80a1350 0x7fbbe80a3800 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbbec006fd0 tx=0x7fbbec008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:19:41.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.294+0000 7fbc17adfb80 1 -- 172.21.15.133:0/112169659 shutdown_connections 2023-07-14T18:19:41.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.294+0000 7fbc17adfb80 1 -- 172.21.15.133:0/112169659 wait complete. 2023-07-14T18:19:41.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.295+0000 7fbc17adfb80 1 -- start start 2023-07-14T18:19:41.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.295+0000 7fbc17adfb80 1 Processor -- start 2023-07-14T18:19:41.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.296+0000 7fbc17adfb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b68c1cff60 0x55b68c1d99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:41.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.296+0000 7fbc17adfb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b68c1d08b0 0x55b68c1d9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:41.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.296+0000 7fbc17adfb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b68c1d14b0 0x55b68c1e0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:41.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.296+0000 7fbc17adfb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68c0c0ac0 con 0x55b68c1cff60 2023-07-14T18:19:41.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.296+0000 7fbc17adfb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68c0c07c0 con 0x55b68c1d08b0 2023-07-14T18:19:41.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.296+0000 7fbc17adfb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68bfa4920 con 0x55b68c1d14b0 2023-07-14T18:19:41.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.296+0000 7fbc17adfb80 10 client.0 _ll_register_callbacks cb 0x55b68c0be760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:19:41.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.296+0000 7fbc0179a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b68c1d08b0 0x55b68c1d9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:41.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.297+0000 7fbc0179a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b68c1d08b0 0x55b68c1d9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:50422/0 (socket says 172.21.15.133:50422) 2023-07-14T18:19:41.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.297+0000 7fbc0179a700 1 -- 172.21.15.133:0/1087341465 learned_addr learned my addr 172.21.15.133:0/1087341465 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:19:41.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.297+0000 7fbc0279c700 1 --2- 172.21.15.133:0/1087341465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b68c1d14b0 0x55b68c1e0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:41.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.297+0000 7fbc0179a700 1 -- 172.21.15.133:0/1087341465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b68c1d14b0 msgr2=0x55b68c1e0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:19:41.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.297+0000 7fbc0179a700 1 --2- 172.21.15.133:0/1087341465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b68c1d14b0 0x55b68c1e0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.297+0000 7fbc0179a700 1 -- 172.21.15.133:0/1087341465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b68c1cff60 msgr2=0x55b68c1d99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:19:41.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.297+0000 7fbc0179a700 1 --2- 172.21.15.133:0/1087341465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b68c1cff60 0x55b68c1d99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.297+0000 7fbc0179a700 1 -- 172.21.15.133:0/1087341465 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf400e140 con 0x55b68c1d08b0 2023-07-14T18:19:41.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.298+0000 7fbc0179a700 1 --2- 172.21.15.133:0/1087341465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b68c1d08b0 0x55b68c1d9ee0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009a20 tx=0x7fbbf4007c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4554b203e1948d39 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:19:41.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.298+0000 7fbbe57fa700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:19:41.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.298+0000 7fbbe57fa700 1 -- 172.21.15.133:0/1087341465 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4014a00 con 0x55b68c1d08b0 2023-07-14T18:19:41.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.299+0000 7fbbe57fa700 1 -- 172.21.15.133:0/1087341465 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbbf4014b60 con 0x55b68c1d08b0 2023-07-14T18:19:41.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.299+0000 7fbc17adfb80 1 -- 172.21.15.133:0/1087341465 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b68c07a4f0 con 0x55b68c1d08b0 2023-07-14T18:19:41.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.299+0000 7fbbe57fa700 1 -- 172.21.15.133:0/1087341465 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbf4009450 con 0x55b68c1d08b0 2023-07-14T18:19:41.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.299+0000 7fbc17adfb80 10 client.25688 fetch_fsmap learned FSMap version 33 2023-07-14T18:19:41.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.299+0000 7fbc17adfb80 1 -- 172.21.15.133:0/1087341465 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55b68bf37c60 con 0x55b68c1d08b0 2023-07-14T18:19:41.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.301+0000 7fbbe57fa700 1 -- 172.21.15.133:0/1087341465 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b68c07a4f0 con 0x55b68c1d08b0 2023-07-14T18:19:41.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.301+0000 7fbbe57fa700 1 -- 172.21.15.133:0/1087341465 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 33) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fbbf401d430 con 0x55b68c1d08b0 2023-07-14T18:19:41.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.301+0000 7fbc17adfb80 10 client.25688 fetch_fsmap finished waiting for FSMap version 33 2023-07-14T18:19:41.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.301+0000 7fbc17adfb80 10 client.25688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:19:41.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.301+0000 7fbc17adfb80 20 client.25688 populate_metadata read hostname 'smithi133' 2023-07-14T18:19:41.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.301+0000 7fbc17adfb80 1 --2- 172.21.15.133:0/1087341465 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55b68c1e5750 0x55b68c1e7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:41.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.301+0000 7fbc17adfb80 4 client.25688 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:19:41.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.301+0000 7fbc17adfb80 1 -- 172.21.15.133:0/1087341465 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b68c19eda0 con 0x55b68c1e5750 2023-07-14T18:19:41.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.303+0000 7fbc01f9b700 1 --2- 172.21.15.133:0/1087341465 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55b68c1e5750 0x55b68c1e7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:41.356 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.308+0000 7fbc01f9b700 1 --2- 172.21.15.133:0/1087341465 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55b68c1e5750 0x55b68c1e7b60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40ea55732d77d56d server_cookie=92ec5b58eb1354fa in_seq=0 out_seq=0 2023-07-14T18:19:41.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.312+0000 7fbbe57fa700 10 client.25688 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:19:41.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.312+0000 7fbbe57fa700 1 -- 172.21.15.133:0/1087341465 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55b68c19eda0 con 0x55b68c1e5750 2023-07-14T18:19:41.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.312+0000 7fbbe57fa700 10 client.25688 handle_command_reply: tid=0 2023-07-14T18:19:41.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.312+0000 7fbc17adfb80 1 client.25688 shutdown 2023-07-14T18:19:41.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.313+0000 7fbc17adfb80 1 -- 172.21.15.133:0/1087341465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b68c1d08b0 msgr2=0x55b68c1d9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:19:41.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.313+0000 7fbc17adfb80 1 --2- 172.21.15.133:0/1087341465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b68c1d08b0 0x55b68c1d9ee0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009a20 tx=0x7fbbf4007c10 comp rx=0 tx=0).stop 2023-07-14T18:19:41.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.313+0000 7fbc17adfb80 1 -- 172.21.15.133:0/1087341465 shutdown_connections 2023-07-14T18:19:41.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.313+0000 7fbc17adfb80 1 --2- 172.21.15.133:0/1087341465 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55b68c1e5750 0x55b68c1e7b60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.313+0000 7fbc17adfb80 1 --2- 172.21.15.133:0/1087341465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b68c1d14b0 0x55b68c1e0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.313+0000 7fbc17adfb80 1 --2- 172.21.15.133:0/1087341465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b68c1cff60 0x55b68c1d99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.313+0000 7fbc17adfb80 1 --2- 172.21.15.133:0/1087341465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b68c1d08b0 0x55b68c1d9ee0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.313+0000 7fbc17adfb80 1 -- 172.21.15.133:0/1087341465 >> 172.21.15.133:0/1087341465 conn(0x55b68c1c5680 msgr2=0x55b68c1c9650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:19:41.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.313+0000 7fbbe57fa700 0 client.25688 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:19:41.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.313+0000 7fbc17adfb80 1 -- 172.21.15.133:0/1087341465 shutdown_connections 2023-07-14T18:19:41.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.313+0000 7fbc17adfb80 1 -- 172.21.15.133:0/1087341465 wait complete. 2023-07-14T18:19:41.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.314+0000 7fbc17adfb80 20 client.25688 trim_cache size 0 max 16384 2023-07-14T18:19:41.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.351+0000 7fbc17adfb80 1 Processor -- start 2023-07-14T18:19:41.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.351+0000 7fbc17adfb80 1 -- start start 2023-07-14T18:19:41.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.351+0000 7fbc17adfb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b68c1cff60 0x55b68c1dee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:41.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.351+0000 7fbc17adfb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b68c1d08b0 0x55b68c1d78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:41.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.351+0000 7fbc17adfb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b68c1d14b0 0x55b68c1d43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:41.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.352+0000 7fbc17adfb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68c1d1da0 con 0x55b68c1d14b0 2023-07-14T18:19:41.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.352+0000 7fbc17adfb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68c1d8e50 con 0x55b68c1d08b0 2023-07-14T18:19:41.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.352+0000 7fbc17adfb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68c1cb4c0 con 0x55b68c1cff60 2023-07-14T18:19:41.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.352+0000 7fbc0179a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b68c1d08b0 0x55b68c1d78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:41.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.352+0000 7fbc0179a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b68c1d08b0 0x55b68c1d78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:50428/0 (socket says 172.21.15.133:50428) 2023-07-14T18:19:41.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.352+0000 7fbc0179a700 1 -- 172.21.15.133:0/3046999996 learned_addr learned my addr 172.21.15.133:0/3046999996 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:19:41.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.353+0000 7fbc0279c700 1 --2- 172.21.15.133:0/3046999996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b68c1d14b0 0x55b68c1d43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:41.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.353+0000 7fbc0179a700 1 -- 172.21.15.133:0/3046999996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b68c1cff60 msgr2=0x55b68c1dee30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T18:19:41.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.353+0000 7fbc0179a700 1 --2- 172.21.15.133:0/3046999996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b68c1cff60 0x55b68c1dee30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.353+0000 7fbc0179a700 1 -- 172.21.15.133:0/3046999996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b68c1d14b0 msgr2=0x55b68c1d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:19:41.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.353+0000 7fbc0179a700 1 --2- 172.21.15.133:0/3046999996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b68c1d14b0 0x55b68c1d43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.353+0000 7fbc0179a700 1 -- 172.21.15.133:0/3046999996 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b68bf75250 con 0x55b68c1d08b0 2023-07-14T18:19:41.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.354+0000 7fbc0179a700 1 --2- 172.21.15.133:0/3046999996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b68c1d08b0 0x55b68c1d78a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400e7b0 tx=0x7fbbf4009b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df1300af34db768a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:19:41.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.354+0000 7fbbf0ff9700 1 -- 172.21.15.133:0/3046999996 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4009450 con 0x55b68c1d08b0 2023-07-14T18:19:41.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.354+0000 7fbbf0ff9700 1 -- 172.21.15.133:0/3046999996 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbf401dab0 con 0x55b68c1d08b0 2023-07-14T18:19:41.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.354+0000 7fbbf0ff9700 1 -- 172.21.15.133:0/3046999996 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4004e10 con 0x55b68c1d08b0 2023-07-14T18:19:41.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.355+0000 7fbc17adfb80 1 -- 172.21.15.133:0/3046999996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b68c1d08b0 msgr2=0x55b68c1d78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:19:41.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.355+0000 7fbc17adfb80 1 --2- 172.21.15.133:0/3046999996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b68c1d08b0 0x55b68c1d78a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400e7b0 tx=0x7fbbf4009b10 comp rx=0 tx=0).stop 2023-07-14T18:19:41.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.356+0000 7fbc17adfb80 1 -- 172.21.15.133:0/3046999996 shutdown_connections 2023-07-14T18:19:41.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.356+0000 7fbc17adfb80 1 --2- 172.21.15.133:0/3046999996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b68c1d14b0 0x55b68c1d43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.356+0000 7fbc17adfb80 1 --2- 172.21.15.133:0/3046999996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b68c1cff60 0x55b68c1dee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.356+0000 7fbc17adfb80 1 --2- 172.21.15.133:0/3046999996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b68c1d08b0 0x55b68c1d78a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.356+0000 7fbc17adfb80 1 -- 172.21.15.133:0/3046999996 >> 172.21.15.133:0/3046999996 conn(0x55b68c1c5680 msgr2=0x55b68c1e7760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:19:41.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.358+0000 7fbc17adfb80 1 -- 172.21.15.133:0/3046999996 shutdown_connections 2023-07-14T18:19:41.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.358+0000 7fbc17adfb80 1 -- 172.21.15.133:0/3046999996 wait complete. 2023-07-14T18:19:41.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.358+0000 7fbc17adfb80 1 -- start start 2023-07-14T18:19:41.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.359+0000 7fbc17adfb80 1 Processor -- start 2023-07-14T18:19:41.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.359+0000 7fbc17adfb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b68c1d08b0 0x55b68c1dcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:41.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.359+0000 7fbc17adfb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b68c1d14b0 0x55b68c1dd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:41.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.359+0000 7fbc17adfb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b68c1de2a0 0x55b68c1de710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:41.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.359+0000 7fbc17adfb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68c1dec50 con 0x55b68c1d14b0 2023-07-14T18:19:41.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.359+0000 7fbc17adfb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68c1dedc0 con 0x55b68c1d08b0 2023-07-14T18:19:41.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.360+0000 7fbc17adfb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68c1def80 con 0x55b68c1de2a0 2023-07-14T18:19:41.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.360+0000 7fbc17adfb80 10 client.0 _ll_register_callbacks cb 0x7fbbf400e360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:19:41.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.360+0000 7fbc0279c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b68c1de2a0 0x55b68c1de710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:41.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.360+0000 7fbc0279c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b68c1de2a0 0x55b68c1de710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:33032/0 (socket says 172.21.15.133:33032) 2023-07-14T18:19:41.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.360+0000 7fbc0279c700 1 -- 172.21.15.133:0/2945843180 learned_addr learned my addr 172.21.15.133:0/2945843180 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:19:41.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.361+0000 7fbc0179a700 1 --2- 172.21.15.133:0/2945843180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b68c1d14b0 0x55b68c1dd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:41.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.361+0000 7fbc0279c700 1 -- 172.21.15.133:0/2945843180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b68c1d08b0 msgr2=0x55b68c1dcd50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T18:19:41.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.361+0000 7fbc0279c700 1 --2- 172.21.15.133:0/2945843180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b68c1d08b0 0x55b68c1dcd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.361+0000 7fbc0279c700 1 -- 172.21.15.133:0/2945843180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b68c1d14b0 msgr2=0x55b68c1dd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:19:41.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.361+0000 7fbc0279c700 1 --2- 172.21.15.133:0/2945843180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b68c1d14b0 0x55b68c1dd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.361+0000 7fbc0279c700 1 -- 172.21.15.133:0/2945843180 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b68bf37c60 con 0x55b68c1de2a0 2023-07-14T18:19:41.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.362+0000 7fbc0279c700 1 --2- 172.21.15.133:0/2945843180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b68c1de2a0 0x55b68c1de710 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00ff50 tx=0x7fbbec004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5675ca98c4ba3a83 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:19:41.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.362+0000 7fbbe57fa700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:19:41.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.362+0000 7fbbe57fa700 1 -- 172.21.15.133:0/2945843180 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec015610 con 0x55b68c1de2a0 2023-07-14T18:19:41.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.362+0000 7fbbe57fa700 1 -- 172.21.15.133:0/2945843180 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbbec015770 con 0x55b68c1de2a0 2023-07-14T18:19:41.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.362+0000 7fbc17adfb80 1 -- 172.21.15.133:0/2945843180 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbbf400f040 con 0x55b68c1de2a0 2023-07-14T18:19:41.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.363+0000 7fbbe57fa700 1 -- 172.21.15.133:0/2945843180 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbec014e90 con 0x55b68c1de2a0 2023-07-14T18:19:41.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.363+0000 7fbc17adfb80 10 client.25723 fetch_fsmap learned FSMap version 33 2023-07-14T18:19:41.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.363+0000 7fbc17adfb80 1 -- 172.21.15.133:0/2945843180 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55b68bf4d2b0 con 0x55b68c1de2a0 2023-07-14T18:19:41.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.363+0000 7fbbe57fa700 1 -- 172.21.15.133:0/2945843180 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbf400f040 con 0x55b68c1de2a0 2023-07-14T18:19:41.413 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:41 smithi133 ceph-mon[126384]: pgmap v438: 129 pgs: 129 active+clean; 8.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 73 MiB/s rd, 67 MiB/s wr, 690 op/s 2023-07-14T18:19:41.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.363+0000 7fbbe57fa700 1 -- 172.21.15.133:0/2945843180 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 33) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fbbec02bc30 con 0x55b68c1de2a0 2023-07-14T18:19:41.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.364+0000 7fbc17adfb80 10 client.25723 fetch_fsmap finished waiting for FSMap version 33 2023-07-14T18:19:41.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.364+0000 7fbc17adfb80 10 client.25723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:19:41.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.364+0000 7fbc17adfb80 20 client.25723 populate_metadata read hostname 'smithi133' 2023-07-14T18:19:41.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.364+0000 7fbc17adfb80 1 --2- 172.21.15.133:0/2945843180 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55b68c23a920 0x55b68c23cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:19:41.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.364+0000 7fbc17adfb80 4 client.25723 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:19:41.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.364+0000 7fbc17adfb80 1 -- 172.21.15.133:0/2945843180 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b68c119eb0 con 0x55b68c23a920 2023-07-14T18:19:41.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.365+0000 7fbc01f9b700 1 --2- 172.21.15.133:0/2945843180 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55b68c23a920 0x55b68c23cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:19:41.439 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.391+0000 7fbbe57fa700 10 client.25723 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:19:41.439 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.391+0000 7fbc01f9b700 1 --2- 172.21.15.133:0/2945843180 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55b68c23a920 0x55b68c23cd30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc10bc8f8d5751ca server_cookie=45df303de834b88b in_seq=0 out_seq=0 2023-07-14T18:19:41.440 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.391+0000 7fbbe57fa700 1 -- 172.21.15.133:0/2945843180 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55b68c119eb0 con 0x55b68c23a920 2023-07-14T18:19:41.440 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.391+0000 7fbbe57fa700 10 client.25723 handle_command_reply: tid=0 2023-07-14T18:19:41.440 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.391+0000 7fbc17adfb80 1 client.25723 shutdown 2023-07-14T18:19:41.440 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.392+0000 7fbc17adfb80 1 -- 172.21.15.133:0/2945843180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b68c1de2a0 msgr2=0x55b68c1de710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:19:41.441 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.392+0000 7fbc17adfb80 1 --2- 172.21.15.133:0/2945843180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b68c1de2a0 0x55b68c1de710 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00ff50 tx=0x7fbbec004750 comp rx=0 tx=0).stop 2023-07-14T18:19:41.441 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.392+0000 7fbc17adfb80 1 -- 172.21.15.133:0/2945843180 shutdown_connections 2023-07-14T18:19:41.441 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.392+0000 7fbc17adfb80 1 --2- 172.21.15.133:0/2945843180 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55b68c23a920 0x55b68c23cd30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.441 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.392+0000 7fbc17adfb80 1 --2- 172.21.15.133:0/2945843180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b68c1d08b0 0x55b68c1dcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.441 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.392+0000 7fbc17adfb80 1 --2- 172.21.15.133:0/2945843180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b68c1de2a0 0x55b68c1de710 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.441 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.392+0000 7fbc17adfb80 1 --2- 172.21.15.133:0/2945843180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b68c1d14b0 0x55b68c1dd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.441 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.392+0000 7fbc17adfb80 1 -- 172.21.15.133:0/2945843180 >> 172.21.15.133:0/2945843180 conn(0x55b68c1c5680 msgr2=0x55b68c1d78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:19:41.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.392+0000 7fbbe57fa700 0 client.25723 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:19:41.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.394+0000 7fbc17adfb80 1 -- 172.21.15.133:0/2945843180 shutdown_connections 2023-07-14T18:19:41.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.394+0000 7fbc17adfb80 1 -- 172.21.15.133:0/2945843180 wait complete. 2023-07-14T18:19:41.443 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.395+0000 7fbc17adfb80 20 client.25723 trim_cache size 0 max 16384 2023-07-14T18:19:41.443 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:19:41.443 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-14T18:19:41.444 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:19:41.444 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:19:41.444 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:19:41.444 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:19:41.444 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:19:41.444 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:19:41.444 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:19:41.445 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:19:41.447 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.399+0000 7fbc041ff700 1 -- 172.21.15.133:0/2093452857 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fbbe80a1350 msgr2=0x7fbbe80a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:19:41.447 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.399+0000 7fbc041ff700 1 --2- 172.21.15.133:0/2093452857 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fbbe80a1350 0x7fbbe80a3800 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbbec006fd0 tx=0x7fbbec008040 comp rx=0 tx=0).stop 2023-07-14T18:19:41.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.399+0000 7fbc041ff700 1 -- 172.21.15.133:0/2093452857 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbbfc071010 msgr2=0x7fbbfc113c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:19:41.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.399+0000 7fbc041ff700 1 --2- 172.21.15.133:0/2093452857 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbbfc071010 0x7fbbfc113c10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800b790 tx=0x7fbbf800b870 comp rx=0 tx=0).stop 2023-07-14T18:19:41.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.400+0000 7fbc041ff700 1 -- 172.21.15.133:0/2093452857 shutdown_connections 2023-07-14T18:19:41.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.400+0000 7fbc041ff700 1 --2- 172.21.15.133:0/2093452857 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fbbe80a1350 0x7fbbe80a3800 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.400+0000 7fbc041ff700 1 --2- 172.21.15.133:0/2093452857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbfc10be90 0x7fbbfc1b4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.400+0000 7fbc041ff700 1 --2- 172.21.15.133:0/2093452857 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbbfc071010 0x7fbbfc113c10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.400+0000 7fbc041ff700 1 --2- 172.21.15.133:0/2093452857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbfc071960 0x7fbbfc114150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:19:41.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.400+0000 7fbc041ff700 1 -- 172.21.15.133:0/2093452857 >> 172.21.15.133:0/2093452857 conn(0x7fbbfc06cbc0 msgr2=0x7fbbfc06dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:19:41.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.400+0000 7fbc041ff700 1 -- 172.21.15.133:0/2093452857 shutdown_connections 2023-07-14T18:19:41.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:19:41.400+0000 7fbc041ff700 1 -- 172.21.15.133:0/2093452857 wait complete. 2023-07-14T18:19:41.485 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:19:41.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:41 smithi161 ceph-mon[128993]: pgmap v438: 129 pgs: 129 active+clean; 8.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 73 MiB/s rd, 67 MiB/s wr, 690 op/s 2023-07-14T18:19:41.622 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:41 smithi171 ceph-mon[129241]: pgmap v438: 129 pgs: 129 active+clean; 8.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 73 MiB/s rd, 67 MiB/s wr, 690 op/s 2023-07-14T18:19:42.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:42 smithi161 ceph-mon[128993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:19:42.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:42 smithi161 ceph-mon[128993]: pgmap v439: 129 pgs: 129 active+clean; 8.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 50 MiB/s wr, 1.55k op/s 2023-07-14T18:19:42.621 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:42 smithi171 ceph-mon[129241]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:19:42.621 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:42 smithi171 ceph-mon[129241]: pgmap v439: 129 pgs: 129 active+clean; 8.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 50 MiB/s wr, 1.55k op/s 2023-07-14T18:19:42.661 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:42 smithi133 ceph-mon[126384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:19:42.661 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:42 smithi133 ceph-mon[126384]: pgmap v439: 129 pgs: 129 active+clean; 8.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 50 MiB/s wr, 1.55k op/s 2023-07-14T18:19:43.856 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:43 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:19:43.869 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:43 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:19:43.911 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:43 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:19:44.854 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:44 smithi161 ceph-mon[128993]: pgmap v440: 129 pgs: 129 active+clean; 8.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 55 MiB/s wr, 2.74k op/s 2023-07-14T18:19:44.869 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:44 smithi171 ceph-mon[129241]: pgmap v440: 129 pgs: 129 active+clean; 8.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 55 MiB/s wr, 2.74k op/s 2023-07-14T18:19:44.910 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:44 smithi133 ceph-mon[126384]: pgmap v440: 129 pgs: 129 active+clean; 8.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 55 MiB/s wr, 2.74k op/s 2023-07-14T18:19:45.854 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:45 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:19:45.855 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:45 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:19:45.855 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:45 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:19:45.869 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:45 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:19:45.870 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:45 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:19:45.870 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:45 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:19:45.878 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 20.0 s, 1391.4 tps, lat 72.027 ms stddev 122.711 2023-07-14T18:19:45.910 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:45 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:19:45.910 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:45 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:19:45.911 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:45 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:19:46.854 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:46 smithi161 ceph-mon[128993]: pgmap v441: 129 pgs: 129 active+clean; 8.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 48 MiB/s wr, 2.87k op/s 2023-07-14T18:19:46.869 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:46 smithi171 ceph-mon[129241]: pgmap v441: 129 pgs: 129 active+clean; 8.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 48 MiB/s wr, 2.87k op/s 2023-07-14T18:19:46.913 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:46 smithi133 ceph-mon[126384]: pgmap v441: 129 pgs: 129 active+clean; 8.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 48 MiB/s wr, 2.87k op/s 2023-07-14T18:19:49.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:48 smithi161 ceph-mon[128993]: pgmap v442: 129 pgs: 129 active+clean; 8.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 MiB/s wr, 3.67k op/s 2023-07-14T18:19:49.119 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:48 smithi171 ceph-mon[129241]: pgmap v442: 129 pgs: 129 active+clean; 8.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 MiB/s wr, 3.67k op/s 2023-07-14T18:19:49.161 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:48 smithi133 ceph-mon[126384]: pgmap v442: 129 pgs: 129 active+clean; 8.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 MiB/s wr, 3.67k op/s 2023-07-14T18:19:50.880 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 25.0 s, 2078.2 tps, lat 49.776 ms stddev 67.418 2023-07-14T18:19:51.354 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:51 smithi161 ceph-mon[128993]: pgmap v443: 129 pgs: 129 active+clean; 8.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 53 MiB/s wr, 3.46k op/s 2023-07-14T18:19:51.371 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:51 smithi171 ceph-mon[129241]: pgmap v443: 129 pgs: 129 active+clean; 8.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 53 MiB/s wr, 3.46k op/s 2023-07-14T18:19:51.411 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:51 smithi133 ceph-mon[126384]: pgmap v443: 129 pgs: 129 active+clean; 8.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 53 MiB/s wr, 3.46k op/s 2023-07-14T18:19:53.355 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:53 smithi161 ceph-mon[128993]: pgmap v444: 129 pgs: 129 active+clean; 8.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 56 MiB/s wr, 3.57k op/s 2023-07-14T18:19:53.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:53 smithi171 ceph-mon[129241]: pgmap v444: 129 pgs: 129 active+clean; 8.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 56 MiB/s wr, 3.57k op/s 2023-07-14T18:19:53.411 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:53 smithi133 ceph-mon[126384]: pgmap v444: 129 pgs: 129 active+clean; 8.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 56 MiB/s wr, 3.57k op/s 2023-07-14T18:19:55.356 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:55 smithi161 ceph-mon[128993]: pgmap v445: 129 pgs: 129 active+clean; 8.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 52 MiB/s wr, 2.94k op/s 2023-07-14T18:19:55.370 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:55 smithi171 ceph-mon[129241]: pgmap v445: 129 pgs: 129 active+clean; 8.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 52 MiB/s wr, 2.94k op/s 2023-07-14T18:19:55.410 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:55 smithi133 ceph-mon[126384]: pgmap v445: 129 pgs: 129 active+clean; 8.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 52 MiB/s wr, 2.94k op/s 2023-07-14T18:19:55.880 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 30.0 s, 1919.6 tps, lat 48.318 ms stddev 42.006 2023-07-14T18:19:57.370 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:57 smithi171 ceph-mon[129241]: pgmap v446: 129 pgs: 129 active+clean; 8.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 39 MiB/s wr, 1.75k op/s 2023-07-14T18:19:57.410 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:57 smithi133 ceph-mon[126384]: pgmap v446: 129 pgs: 129 active+clean; 8.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 39 MiB/s wr, 1.75k op/s 2023-07-14T18:19:57.606 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:57 smithi161 ceph-mon[128993]: pgmap v446: 129 pgs: 129 active+clean; 8.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 39 MiB/s wr, 1.75k op/s 2023-07-14T18:19:59.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:59 smithi161 ceph-mon[128993]: pgmap v447: 129 pgs: 129 active+clean; 8.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 50 MiB/s wr, 2.37k op/s 2023-07-14T18:19:59.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:19:59 smithi161 ceph-mon[128993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:19:59.619 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:59 smithi171 ceph-mon[129241]: pgmap v447: 129 pgs: 129 active+clean; 8.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 50 MiB/s wr, 2.37k op/s 2023-07-14T18:19:59.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:19:59 smithi171 ceph-mon[129241]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:19:59.662 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:59 smithi133 ceph-mon[126384]: pgmap v447: 129 pgs: 129 active+clean; 8.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 50 MiB/s wr, 2.37k op/s 2023-07-14T18:19:59.662 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:19:59 smithi133 ceph-mon[126384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:20:00.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:00 smithi161 ceph-mon[128993]: overall HEALTH_OK 2023-07-14T18:20:00.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:00 smithi171 ceph-mon[129241]: overall HEALTH_OK 2023-07-14T18:20:00.661 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:00 smithi133 ceph-mon[126384]: overall HEALTH_OK 2023-07-14T18:20:00.883 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 35.0 s, 690.0 tps, lat 154.745 ms stddev 253.288 2023-07-14T18:20:01.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:01 smithi161 ceph-mon[128993]: pgmap v448: 129 pgs: 129 active+clean; 8.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 45 MiB/s wr, 2.09k op/s 2023-07-14T18:20:01.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:01 smithi171 ceph-mon[129241]: pgmap v448: 129 pgs: 129 active+clean; 8.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 45 MiB/s wr, 2.09k op/s 2023-07-14T18:20:01.660 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:01 smithi133 ceph-mon[126384]: pgmap v448: 129 pgs: 129 active+clean; 8.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 45 MiB/s wr, 2.09k op/s 2023-07-14T18:20:03.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:03 smithi161 ceph-mon[128993]: pgmap v449: 129 pgs: 129 active+clean; 8.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 52 MiB/s wr, 3.24k op/s 2023-07-14T18:20:03.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:03 smithi171 ceph-mon[129241]: pgmap v449: 129 pgs: 129 active+clean; 8.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 52 MiB/s wr, 3.24k op/s 2023-07-14T18:20:03.660 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:03 smithi133 ceph-mon[126384]: pgmap v449: 129 pgs: 129 active+clean; 8.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 52 MiB/s wr, 3.24k op/s 2023-07-14T18:20:05.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:05 smithi161 ceph-mon[128993]: pgmap v450: 129 pgs: 129 active+clean; 8.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 58 MiB/s wr, 3.90k op/s 2023-07-14T18:20:05.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:05 smithi171 ceph-mon[129241]: pgmap v450: 129 pgs: 129 active+clean; 8.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 58 MiB/s wr, 3.90k op/s 2023-07-14T18:20:05.661 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:05 smithi133 ceph-mon[126384]: pgmap v450: 129 pgs: 129 active+clean; 8.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 58 MiB/s wr, 3.90k op/s 2023-07-14T18:20:05.878 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 40.0 s, 2409.6 tps, lat 41.498 ms stddev 50.483 2023-07-14T18:20:06.562 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:06 smithi133 ceph-mon[126384]: pgmap v451: 129 pgs: 129 active+clean; 8.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 52 MiB/s wr, 3.71k op/s 2023-07-14T18:20:06.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:06 smithi161 ceph-mon[128993]: pgmap v451: 129 pgs: 129 active+clean; 8.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 52 MiB/s wr, 3.71k op/s 2023-07-14T18:20:06.621 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:06 smithi171 ceph-mon[129241]: pgmap v451: 129 pgs: 129 active+clean; 8.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 52 MiB/s wr, 3.71k op/s 2023-07-14T18:20:09.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:08 smithi161 ceph-mon[128993]: pgmap v452: 129 pgs: 129 active+clean; 8.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 60 MiB/s wr, 4.02k op/s 2023-07-14T18:20:09.120 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:08 smithi171 ceph-mon[129241]: pgmap v452: 129 pgs: 129 active+clean; 8.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 60 MiB/s wr, 4.02k op/s 2023-07-14T18:20:09.161 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:08 smithi133 ceph-mon[126384]: pgmap v452: 129 pgs: 129 active+clean; 8.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 60 MiB/s wr, 4.02k op/s 2023-07-14T18:20:10.889 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 45.0 s, 1625.3 tps, lat 61.639 ms stddev 93.377 2023-07-14T18:20:11.370 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:11 smithi171 ceph-mon[129241]: pgmap v453: 129 pgs: 129 active+clean; 8.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 51 MiB/s wr, 3.32k op/s 2023-07-14T18:20:11.411 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:11 smithi133 ceph-mon[126384]: pgmap v453: 129 pgs: 129 active+clean; 8.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 51 MiB/s wr, 3.32k op/s 2023-07-14T18:20:11.486 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:20:11.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:11 smithi161 ceph-mon[128993]: pgmap v453: 129 pgs: 129 active+clean; 8.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 51 MiB/s wr, 3.32k op/s 2023-07-14T18:20:11.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.870+0000 7f15645ff700 1 -- 172.21.15.133:0/4243000443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f155c108200 msgr2=0x7f155c10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:20:11.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.870+0000 7f15645ff700 1 --2- 172.21.15.133:0/4243000443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f155c108200 0x7f155c10eaa0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f155400c310 tx=0x7f155400c620 comp rx=0 tx=0).stop 2023-07-14T18:20:11.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.871+0000 7f15645ff700 1 -- 172.21.15.133:0/4243000443 shutdown_connections 2023-07-14T18:20:11.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.871+0000 7f15645ff700 1 --2- 172.21.15.133:0/4243000443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f155c108200 0x7f155c10eaa0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:11.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.871+0000 7f15645ff700 1 --2- 172.21.15.133:0/4243000443 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f155c071820 0x7f155c071c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:11.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.871+0000 7f15645ff700 1 --2- 172.21.15.133:0/4243000443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f155c107850 0x7f155c107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:11.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.871+0000 7f15645ff700 1 -- 172.21.15.133:0/4243000443 >> 172.21.15.133:0/4243000443 conn(0x7f155c06ceb0 msgr2=0x7f155c06f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:20:11.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.871+0000 7f15645ff700 1 -- 172.21.15.133:0/4243000443 shutdown_connections 2023-07-14T18:20:11.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.871+0000 7f15645ff700 1 -- 172.21.15.133:0/4243000443 wait complete. 2023-07-14T18:20:11.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.872+0000 7f15645ff700 1 Processor -- start 2023-07-14T18:20:11.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.872+0000 7f15645ff700 1 -- start start 2023-07-14T18:20:11.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.873+0000 7f15645ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f155c071820 0x7f155c10a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:11.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.873+0000 7f15645ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f155c107850 0x7f155c108c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:11.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.873+0000 7f15645ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f155c108200 0x7f155c109170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:11.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.873+0000 7f15645ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155c1a2110 con 0x7f155c107850 2023-07-14T18:20:11.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.873+0000 7f15645ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155c1096b0 con 0x7f155c108200 2023-07-14T18:20:11.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.873+0000 7f15645ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155c109870 con 0x7f155c071820 2023-07-14T18:20:11.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.873+0000 7f156239b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f155c071820 0x7f155c10a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:20:11.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.874+0000 7f156239b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f155c071820 0x7f155c10a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:51264/0 (socket says 172.21.15.133:51264) 2023-07-14T18:20:11.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.874+0000 7f156239b700 1 -- 172.21.15.133:0/3162392766 learned_addr learned my addr 172.21.15.133:0/3162392766 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:20:11.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.874+0000 7f1562b9c700 1 --2- 172.21.15.133:0/3162392766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f155c108200 0x7f155c109170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:20:11.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.874+0000 7f156239b700 1 -- 172.21.15.133:0/3162392766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f155c108200 msgr2=0x7f155c109170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:20:11.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.874+0000 7f156239b700 1 --2- 172.21.15.133:0/3162392766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f155c108200 0x7f155c109170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:11.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.874+0000 7f156239b700 1 -- 172.21.15.133:0/3162392766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f155c107850 msgr2=0x7f155c108c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:20:11.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.874+0000 7f156239b700 1 --2- 172.21.15.133:0/3162392766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f155c107850 0x7f155c108c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:11.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.874+0000 7f156239b700 1 -- 172.21.15.133:0/3162392766 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155400c050 con 0x7f155c071820 2023-07-14T18:20:11.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.875+0000 7f156239b700 1 --2- 172.21.15.133:0/3162392766 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f155c071820 0x7f155c10a5f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f155800d950 tx=0x7f155800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:20:11.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.876+0000 7f15537fe700 1 -- 172.21.15.133:0/3162392766 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1558009940 con 0x7f155c071820 2023-07-14T18:20:11.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.876+0000 7f15537fe700 1 -- 172.21.15.133:0/3162392766 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1558010460 con 0x7f155c071820 2023-07-14T18:20:11.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.876+0000 7f15537fe700 1 -- 172.21.15.133:0/3162392766 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155800f620 con 0x7f155c071820 2023-07-14T18:20:11.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.876+0000 7f15645ff700 1 -- 172.21.15.133:0/3162392766 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f155c1acac0 con 0x7f155c071820 2023-07-14T18:20:11.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.876+0000 7f15645ff700 1 -- 172.21.15.133:0/3162392766 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f155c1ace90 con 0x7f155c071820 2023-07-14T18:20:11.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.882+0000 7f1577e54b80 1 Processor -- start 2023-07-14T18:20:11.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.885+0000 7f1577e54b80 1 -- start start 2023-07-14T18:20:11.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.886+0000 7f1577e54b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac07c5bf60 0x55ac07c5c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:11.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.886+0000 7f1577e54b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac07c5c8b0 0x55ac07c5ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:11.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.886+0000 7f1577e54b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac07c5d4b0 0x55ac07c5d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:11.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.886+0000 7f1577e54b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac07b60cb0 con 0x55ac07c5c8b0 2023-07-14T18:20:11.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.886+0000 7f1577e54b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac07bb3a40 con 0x55ac07c5d4b0 2023-07-14T18:20:11.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.886+0000 7f1577e54b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac07baced0 con 0x55ac07c5bf60 2023-07-14T18:20:11.938 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.889+0000 7f1561b9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac07c5d4b0 0x55ac07c5d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:20:11.938 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.889+0000 7f1561b9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac07c5d4b0 0x55ac07c5d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:40136/0 (socket says 172.21.15.133:40136) 2023-07-14T18:20:11.938 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.889+0000 7f1561b9a700 1 -- 172.21.15.133:0/3252372818 learned_addr learned my addr 172.21.15.133:0/3252372818 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:20:11.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.893+0000 7f1561b9a700 1 -- 172.21.15.133:0/3252372818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac07c5bf60 msgr2=0x55ac07c5c370 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T18:20:11.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.893+0000 7f1561b9a700 1 --2- 172.21.15.133:0/3252372818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac07c5bf60 0x55ac07c5c370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:11.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.894+0000 7f1561b9a700 1 -- 172.21.15.133:0/3252372818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac07c5c8b0 msgr2=0x55ac07c5ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:20:11.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.894+0000 7f1561b9a700 1 --2- 172.21.15.133:0/3252372818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac07c5c8b0 0x55ac07c5ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:11.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.894+0000 7f1561b9a700 1 -- 172.21.15.133:0/3252372818 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac079dade0 con 0x55ac07c5d4b0 2023-07-14T18:20:11.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.897+0000 7f15537fe700 1 -- 172.21.15.133:0/3162392766 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f155800f780 con 0x7f155c071820 2023-07-14T18:20:11.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.899+0000 7f15537fe700 1 --2- 172.21.15.133:0/3162392766 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f15480a1350 0x7f15480a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:11.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.899+0000 7f15537fe700 1 -- 172.21.15.133:0/3162392766 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f15580cd9f0 con 0x7f155c071820 2023-07-14T18:20:11.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.913+0000 7f1561b9a700 1 --2- 172.21.15.133:0/3252372818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac07c5d4b0 0x55ac07c5d920 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f154c00bb90 tx=0x7f154c00bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9525ea4d840e80b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:20:11.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.916+0000 7f1562b9c700 1 --2- 172.21.15.133:0/3162392766 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f15480a1350 0x7f15480a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:20:11.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.916+0000 7f1550bfe700 1 -- 172.21.15.133:0/3252372818 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c00d870 con 0x55ac07c5d4b0 2023-07-14T18:20:11.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.916+0000 7f1550bfe700 1 -- 172.21.15.133:0/3252372818 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f154c00de90 con 0x55ac07c5d4b0 2023-07-14T18:20:11.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.916+0000 7f1550bfe700 1 -- 172.21.15.133:0/3252372818 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c005440 con 0x55ac07c5d4b0 2023-07-14T18:20:11.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.917+0000 7f1577e54b80 1 -- 172.21.15.133:0/3252372818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac07c5d4b0 msgr2=0x55ac07c5d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:20:11.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.917+0000 7f1577e54b80 1 --2- 172.21.15.133:0/3252372818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac07c5d4b0 0x55ac07c5d920 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f154c00bb90 tx=0x7f154c00bea0 comp rx=0 tx=0).stop 2023-07-14T18:20:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.926+0000 7f1577e54b80 1 -- 172.21.15.133:0/3252372818 shutdown_connections 2023-07-14T18:20:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.926+0000 7f1577e54b80 1 --2- 172.21.15.133:0/3252372818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac07c5d4b0 0x55ac07c5d920 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.926+0000 7f1577e54b80 1 --2- 172.21.15.133:0/3252372818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac07c5bf60 0x55ac07c5c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.926+0000 7f1577e54b80 1 --2- 172.21.15.133:0/3252372818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac07c5c8b0 0x55ac07c5ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.926+0000 7f1577e54b80 1 -- 172.21.15.133:0/3252372818 >> 172.21.15.133:0/3252372818 conn(0x55ac07c51680 msgr2=0x55ac07c569d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:20:11.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.939+0000 7f1577e54b80 1 -- 172.21.15.133:0/3252372818 shutdown_connections 2023-07-14T18:20:11.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.940+0000 7f1577e54b80 1 -- 172.21.15.133:0/3252372818 wait complete. 2023-07-14T18:20:11.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.940+0000 7f1577e54b80 1 -- start start 2023-07-14T18:20:11.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.941+0000 7f1577e54b80 1 Processor -- start 2023-07-14T18:20:11.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.942+0000 7f1577e54b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac07c5bf60 0x55ac07c65a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:11.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.942+0000 7f1577e54b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac07c5c8b0 0x55ac07c65f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:11.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.942+0000 7f1577e54b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac07c5d4b0 0x55ac07c6cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:11.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.942+0000 7f1577e54b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac07b4cac0 con 0x55ac07c5d4b0 2023-07-14T18:20:11.994 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.942+0000 7f1577e54b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac07b4c7c0 con 0x55ac07c5c8b0 2023-07-14T18:20:11.994 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.942+0000 7f1577e54b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac07a30920 con 0x55ac07c5bf60 2023-07-14T18:20:11.994 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.942+0000 7f1577e54b80 10 client.0 _ll_register_callbacks cb 0x55ac07b4a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:20:11.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.945+0000 7f1562b9c700 1 --2- 172.21.15.133:0/3162392766 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f15480a1350 0x7f15480a3800 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f155c10ac50 tx=0x7f1554006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:20:11.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.946+0000 7f1561b9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac07c5c8b0 0x55ac07c65f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:20:11.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.946+0000 7f1561b9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac07c5c8b0 0x55ac07c65f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:40144/0 (socket says 172.21.15.133:40144) 2023-07-14T18:20:11.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.946+0000 7f1561b9a700 1 -- 172.21.15.133:0/3766731219 learned_addr learned my addr 172.21.15.133:0/3766731219 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:20:12.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.949+0000 7f156239b700 1 --2- 172.21.15.133:0/3766731219 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac07c5bf60 0x55ac07c65a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:20:12.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.950+0000 7f1561b9a700 1 -- 172.21.15.133:0/3766731219 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac07c5bf60 msgr2=0x55ac07c65a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:20:12.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.950+0000 7f1561b9a700 1 --2- 172.21.15.133:0/3766731219 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac07c5bf60 0x55ac07c65a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:12.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.950+0000 7f1561b9a700 1 -- 172.21.15.133:0/3766731219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac07c5d4b0 msgr2=0x55ac07c6cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:20:12.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.950+0000 7f1561b9a700 1 --2- 172.21.15.133:0/3766731219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac07c5d4b0 0x55ac07c6cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:12.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.950+0000 7f1561b9a700 1 -- 172.21.15.133:0/3766731219 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154c00b7c0 con 0x55ac07c5c8b0 2023-07-14T18:20:12.011 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.963+0000 7f1561b9a700 1 --2- 172.21.15.133:0/3766731219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac07c5c8b0 0x55ac07c65f50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f154c000c00 tx=0x7f154c004b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db041811e7839c59 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:20:12.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.963+0000 7f1545ffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:20:12.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.963+0000 7f1545ffb700 1 -- 172.21.15.133:0/3766731219 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c00dbc0 con 0x55ac07c5c8b0 2023-07-14T18:20:12.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.964+0000 7f1545ffb700 1 -- 172.21.15.133:0/3766731219 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f154c0051c0 con 0x55ac07c5c8b0 2023-07-14T18:20:12.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.965+0000 7f1577e54b80 1 -- 172.21.15.133:0/3766731219 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac07b064f0 con 0x55ac07c5c8b0 2023-07-14T18:20:12.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.966+0000 7f1545ffb700 1 -- 172.21.15.133:0/3766731219 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f154c00dd20 con 0x55ac07c5c8b0 2023-07-14T18:20:12.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.971+0000 7f1577e54b80 10 client.25703 fetch_fsmap learned FSMap version 34 2023-07-14T18:20:12.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.972+0000 7f1577e54b80 1 -- 172.21.15.133:0/3766731219 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ac079c3c60 con 0x55ac07c5c8b0 2023-07-14T18:20:12.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.972+0000 7f1545ffb700 1 -- 172.21.15.133:0/3766731219 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ac07b064f0 con 0x55ac07c5c8b0 2023-07-14T18:20:12.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.972+0000 7f1545ffb700 1 -- 172.21.15.133:0/3766731219 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f154c01c450 con 0x55ac07c5c8b0 2023-07-14T18:20:12.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.972+0000 7f1577e54b80 10 client.25703 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T18:20:12.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.972+0000 7f1577e54b80 10 client.25703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:20:12.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.972+0000 7f1577e54b80 20 client.25703 populate_metadata read hostname 'smithi133' 2023-07-14T18:20:12.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.972+0000 7f1577e54b80 1 --2- 172.21.15.133:0/3766731219 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55ac07c82630 0x55ac07c84a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:12.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.973+0000 7f1577e54b80 4 client.25703 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:20:12.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.973+0000 7f1577e54b80 1 -- 172.21.15.133:0/3766731219 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac07c2ada0 con 0x55ac07c82630 2023-07-14T18:20:12.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.973+0000 7f156239b700 1 --2- 172.21.15.133:0/3766731219 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55ac07c82630 0x55ac07c84a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:20:12.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.976+0000 7f1545ffb700 10 client.25703 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:20:12.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.976+0000 7f156239b700 1 --2- 172.21.15.133:0/3766731219 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55ac07c82630 0x55ac07c84a40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=933a76777987c79b server_cookie=f0cab76d5679d748 in_seq=0 out_seq=0 2023-07-14T18:20:12.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.978+0000 7f1545ffb700 1 -- 172.21.15.133:0/3766731219 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55ac07c2ada0 con 0x55ac07c82630 2023-07-14T18:20:12.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.978+0000 7f1545ffb700 10 client.25703 handle_command_reply: tid=0 2023-07-14T18:20:12.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.978+0000 7f1577e54b80 1 client.25703 shutdown 2023-07-14T18:20:12.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.980+0000 7f1577e54b80 1 -- 172.21.15.133:0/3766731219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac07c5c8b0 msgr2=0x55ac07c65f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:20:12.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.980+0000 7f1577e54b80 1 --2- 172.21.15.133:0/3766731219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac07c5c8b0 0x55ac07c65f50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f154c000c00 tx=0x7f154c004b80 comp rx=0 tx=0).stop 2023-07-14T18:20:12.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.980+0000 7f1577e54b80 1 -- 172.21.15.133:0/3766731219 shutdown_connections 2023-07-14T18:20:12.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.980+0000 7f1577e54b80 1 --2- 172.21.15.133:0/3766731219 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55ac07c82630 0x55ac07c84a40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:12.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.980+0000 7f1577e54b80 1 --2- 172.21.15.133:0/3766731219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac07c5d4b0 0x55ac07c6cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:12.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.980+0000 7f1577e54b80 1 --2- 172.21.15.133:0/3766731219 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac07c5bf60 0x55ac07c65a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:12.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.980+0000 7f1577e54b80 1 --2- 172.21.15.133:0/3766731219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac07c5c8b0 0x55ac07c65f50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:12.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.980+0000 7f1577e54b80 1 -- 172.21.15.133:0/3766731219 >> 172.21.15.133:0/3766731219 conn(0x55ac07c51680 msgr2=0x55ac07c55650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:20:12.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.980+0000 7f1545ffb700 0 client.25703 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:20:12.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.981+0000 7f1577e54b80 1 -- 172.21.15.133:0/3766731219 shutdown_connections 2023-07-14T18:20:12.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.982+0000 7f1577e54b80 1 -- 172.21.15.133:0/3766731219 wait complete. 2023-07-14T18:20:12.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:11.983+0000 7f1577e54b80 20 client.25703 trim_cache size 0 max 16384 2023-07-14T18:20:12.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.012+0000 7f1577e54b80 1 Processor -- start 2023-07-14T18:20:12.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.013+0000 7f1577e54b80 1 -- start start 2023-07-14T18:20:12.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.013+0000 7f1577e54b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac07c5bf60 0x55ac07c63900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:12.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.013+0000 7f1577e54b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac07c5c8b0 0x55ac07c6cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:12.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.013+0000 7f1577e54b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac07c5d4b0 0x55ac07c65a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:12.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.013+0000 7f1577e54b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac07c5d370 con 0x55ac07c5c8b0 2023-07-14T18:20:12.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.013+0000 7f1577e54b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac07c655b0 con 0x55ac07c5bf60 2023-07-14T18:20:12.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.013+0000 7f1577e54b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac07c58510 con 0x55ac07c5d4b0 2023-07-14T18:20:12.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.014+0000 7f156239b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac07c5bf60 0x55ac07c63900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:20:12.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.014+0000 7f156239b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac07c5bf60 0x55ac07c63900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:40146/0 (socket says 172.21.15.133:40146) 2023-07-14T18:20:12.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.014+0000 7f156239b700 1 -- 172.21.15.133:0/713316714 learned_addr learned my addr 172.21.15.133:0/713316714 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:20:12.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.014+0000 7f1562b9c700 1 --2- 172.21.15.133:0/713316714 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac07c5d4b0 0x55ac07c65a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:20:12.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.015+0000 7f156239b700 1 -- 172.21.15.133:0/713316714 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac07c5d4b0 msgr2=0x55ac07c65a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:20:12.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.015+0000 7f156239b700 1 --2- 172.21.15.133:0/713316714 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac07c5d4b0 0x55ac07c65a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:12.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.015+0000 7f156239b700 1 -- 172.21.15.133:0/713316714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac07c5c8b0 msgr2=0x55ac07c6cf50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T18:20:12.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.015+0000 7f156239b700 1 --2- 172.21.15.133:0/713316714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac07c5c8b0 0x55ac07c6cf50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:12.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.015+0000 7f156239b700 1 -- 172.21.15.133:0/713316714 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac07a01250 con 0x55ac07c5bf60 2023-07-14T18:20:12.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.015+0000 7f156239b700 1 --2- 172.21.15.133:0/713316714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac07c5bf60 0x55ac07c63900 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1558005950 tx=0x7f1558000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94259ab7110cc167 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:20:12.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.015+0000 7f15513ff700 1 -- 172.21.15.133:0/713316714 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15580b3210 con 0x55ac07c5bf60 2023-07-14T18:20:12.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.015+0000 7f15513ff700 1 -- 172.21.15.133:0/713316714 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15580a2ec0 con 0x55ac07c5bf60 2023-07-14T18:20:12.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.016+0000 7f15513ff700 1 -- 172.21.15.133:0/713316714 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15580a2a30 con 0x55ac07c5bf60 2023-07-14T18:20:12.068 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.017+0000 7f1577e54b80 1 -- 172.21.15.133:0/713316714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac07c5bf60 msgr2=0x55ac07c63900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:20:12.068 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.017+0000 7f1577e54b80 1 --2- 172.21.15.133:0/713316714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac07c5bf60 0x55ac07c63900 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1558005950 tx=0x7f1558000bc0 comp rx=0 tx=0).stop 2023-07-14T18:20:12.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.019+0000 7f1577e54b80 1 -- 172.21.15.133:0/713316714 shutdown_connections 2023-07-14T18:20:12.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.019+0000 7f1577e54b80 1 --2- 172.21.15.133:0/713316714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac07c5bf60 0x55ac07c63900 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:12.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.019+0000 7f1577e54b80 1 --2- 172.21.15.133:0/713316714 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac07c5d4b0 0x55ac07c65a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:12.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.019+0000 7f1577e54b80 1 --2- 172.21.15.133:0/713316714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac07c5c8b0 0x55ac07c6cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:12.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.019+0000 7f1577e54b80 1 -- 172.21.15.133:0/713316714 >> 172.21.15.133:0/713316714 conn(0x55ac07c51680 msgr2=0x55ac07c64700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:20:12.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.020+0000 7f1577e54b80 1 -- 172.21.15.133:0/713316714 shutdown_connections 2023-07-14T18:20:12.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.020+0000 7f1577e54b80 1 -- 172.21.15.133:0/713316714 wait complete. 2023-07-14T18:20:12.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.020+0000 7f1577e54b80 1 -- start start 2023-07-14T18:20:12.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.020+0000 7f1577e54b80 1 Processor -- start 2023-07-14T18:20:12.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.021+0000 7f1577e54b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac07c5bf60 0x55ac07c6cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:12.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.021+0000 7f1577e54b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac07c5c8b0 0x55ac07c6b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:12.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.021+0000 7f1577e54b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac07c5d4b0 0x55ac07c6bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:12.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.021+0000 7f1577e54b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac07c6c270 con 0x55ac07c5c8b0 2023-07-14T18:20:12.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.021+0000 7f1577e54b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac07c6c3e0 con 0x55ac07c5d4b0 2023-07-14T18:20:12.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.021+0000 7f1577e54b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac07cd6340 con 0x55ac07c5bf60 2023-07-14T18:20:12.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.021+0000 7f1577e54b80 10 client.0 _ll_register_callbacks cb 0x7f154c00b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:20:12.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.021+0000 7f1561b9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac07c5c8b0 0x55ac07c6b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:20:12.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.021+0000 7f1561b9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac07c5c8b0 0x55ac07c6b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58958/0 (socket says 172.21.15.133:58958) 2023-07-14T18:20:12.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.021+0000 7f1561b9a700 1 -- 172.21.15.133:0/2046593036 learned_addr learned my addr 172.21.15.133:0/2046593036 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:20:12.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.021+0000 7f1562b9c700 1 --2- 172.21.15.133:0/2046593036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac07c5d4b0 0x55ac07c6bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:20:12.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.022+0000 7f1561b9a700 1 -- 172.21.15.133:0/2046593036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac07c5bf60 msgr2=0x55ac07c6cf10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T18:20:12.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.022+0000 7f1561b9a700 1 --2- 172.21.15.133:0/2046593036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac07c5bf60 0x55ac07c6cf10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:12.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.022+0000 7f1561b9a700 1 -- 172.21.15.133:0/2046593036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac07c5d4b0 msgr2=0x55ac07c6bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:20:12.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.022+0000 7f1561b9a700 1 --2- 172.21.15.133:0/2046593036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac07c5d4b0 0x55ac07c6bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:12.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.022+0000 7f1561b9a700 1 -- 172.21.15.133:0/2046593036 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f155c1ace90 con 0x55ac07c5c8b0 2023-07-14T18:20:12.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.022+0000 7f1562b9c700 1 --2- 172.21.15.133:0/2046593036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac07c5d4b0 0x55ac07c6bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:20:12.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.022+0000 7f1561b9a700 1 --2- 172.21.15.133:0/2046593036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac07c5c8b0 0x55ac07c6b520 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f154c00d5c0 tx=0x7f154c005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3507c4ca4f7320 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:20:12.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.022+0000 7f1545ffb700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T18:20:12.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.022+0000 7f1545ffb700 1 -- 172.21.15.133:0/2046593036 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c033030 con 0x55ac07c5c8b0 2023-07-14T18:20:12.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.022+0000 7f1545ffb700 1 -- 172.21.15.133:0/2046593036 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f154c034070 con 0x55ac07c5c8b0 2023-07-14T18:20:12.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.022+0000 7f1577e54b80 1 -- 172.21.15.133:0/2046593036 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f154c00de90 con 0x55ac07c5c8b0 2023-07-14T18:20:12.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.023+0000 7f1545ffb700 1 -- 172.21.15.133:0/2046593036 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f154c01c660 con 0x55ac07c5c8b0 2023-07-14T18:20:12.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.023+0000 7f1577e54b80 10 client.15693 fetch_fsmap learned FSMap version 34 2023-07-14T18:20:12.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.023+0000 7f1577e54b80 1 -- 172.21.15.133:0/2046593036 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ac079d92b0 con 0x55ac07c5c8b0 2023-07-14T18:20:12.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.023+0000 7f1545ffb700 1 -- 172.21.15.133:0/2046593036 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f154c00de90 con 0x55ac07c5c8b0 2023-07-14T18:20:12.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.024+0000 7f1545ffb700 1 -- 172.21.15.133:0/2046593036 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f154c004040 con 0x55ac07c5c8b0 2023-07-14T18:20:12.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.024+0000 7f1577e54b80 10 client.15693 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T18:20:12.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.024+0000 7f1577e54b80 10 client.15693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:20:12.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.025+0000 7f1577e54b80 20 client.15693 populate_metadata read hostname 'smithi133' 2023-07-14T18:20:12.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.025+0000 7f1577e54b80 1 --2- 172.21.15.133:0/2046593036 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55ac07cd7a00 0x55ac07cd9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:12.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.025+0000 7f1577e54b80 4 client.15693 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:20:12.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.025+0000 7f1577e54b80 1 -- 172.21.15.133:0/2046593036 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ac07ba5eb0 con 0x55ac07cd7a00 2023-07-14T18:20:12.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.025+0000 7f156239b700 1 --2- 172.21.15.133:0/2046593036 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55ac07cd7a00 0x55ac07cd9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:20:12.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.025+0000 7f156239b700 1 --2- 172.21.15.133:0/2046593036 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55ac07cd7a00 0x55ac07cd9e10 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5317bff1ac176c23 server_cookie=83d550d444086dc in_seq=0 out_seq=0 2023-07-14T18:20:12.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.025+0000 7f1545ffb700 10 client.15693 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:20:12.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.026+0000 7f1545ffb700 1 -- 172.21.15.133:0/2046593036 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55ac07ba5eb0 con 0x55ac07cd7a00 2023-07-14T18:20:12.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.026+0000 7f1545ffb700 10 client.15693 handle_command_reply: tid=0 2023-07-14T18:20:12.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.026+0000 7f1577e54b80 1 client.15693 shutdown 2023-07-14T18:20:12.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.026+0000 7f1577e54b80 1 -- 172.21.15.133:0/2046593036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac07c5c8b0 msgr2=0x55ac07c6b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:20:12.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.026+0000 7f1577e54b80 1 --2- 172.21.15.133:0/2046593036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac07c5c8b0 0x55ac07c6b520 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f154c00d5c0 tx=0x7f154c005a10 comp rx=0 tx=0).stop 2023-07-14T18:20:12.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.028+0000 7f1577e54b80 1 -- 172.21.15.133:0/2046593036 shutdown_connections 2023-07-14T18:20:12.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.028+0000 7f1577e54b80 1 --2- 172.21.15.133:0/2046593036 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55ac07cd7a00 0x55ac07cd9e10 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:12.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.028+0000 7f1577e54b80 1 --2- 172.21.15.133:0/2046593036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac07c5d4b0 0x55ac07c6bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:12.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.028+0000 7f1577e54b80 1 --2- 172.21.15.133:0/2046593036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac07c5bf60 0x55ac07c6cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:12.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.028+0000 7f1577e54b80 1 --2- 172.21.15.133:0/2046593036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac07c5c8b0 0x55ac07c6b520 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:12.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.028+0000 7f1577e54b80 1 -- 172.21.15.133:0/2046593036 >> 172.21.15.133:0/2046593036 conn(0x55ac07c51680 msgr2=0x55ac07c63900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:20:12.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.028+0000 7f1545ffb700 0 client.15693 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:20:12.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.028+0000 7f1577e54b80 1 -- 172.21.15.133:0/2046593036 shutdown_connections 2023-07-14T18:20:12.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.028+0000 7f1577e54b80 1 -- 172.21.15.133:0/2046593036 wait complete. 2023-07-14T18:20:12.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.029+0000 7f1577e54b80 20 client.15693 trim_cache size 0 max 16384 2023-07-14T18:20:12.081 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:20:12.081 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-14T18:20:12.081 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:20:12.082 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:20:12.082 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:20:12.082 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:20:12.082 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:20:12.082 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:20:12.082 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:20:12.082 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:20:12.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.030+0000 7f15645ff700 1 -- 172.21.15.133:0/3162392766 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f15480a1350 msgr2=0x7f15480a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:20:12.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.030+0000 7f15645ff700 1 --2- 172.21.15.133:0/3162392766 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f15480a1350 0x7f15480a3800 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f155c10ac50 tx=0x7f1554006210 comp rx=0 tx=0).stop 2023-07-14T18:20:12.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.030+0000 7f15645ff700 1 -- 172.21.15.133:0/3162392766 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f155c071820 msgr2=0x7f155c10a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:20:12.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.030+0000 7f15645ff700 1 --2- 172.21.15.133:0/3162392766 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f155c071820 0x7f155c10a5f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f155800d950 tx=0x7f155800dc60 comp rx=0 tx=0).stop 2023-07-14T18:20:12.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.030+0000 7f15645ff700 1 -- 172.21.15.133:0/3162392766 shutdown_connections 2023-07-14T18:20:12.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.030+0000 7f15645ff700 1 --2- 172.21.15.133:0/3162392766 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f15480a1350 0x7f15480a3800 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:12.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.030+0000 7f15645ff700 1 --2- 172.21.15.133:0/3162392766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f155c108200 0x7f155c109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:12.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.030+0000 7f15645ff700 1 --2- 172.21.15.133:0/3162392766 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f155c071820 0x7f155c10a5f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:12.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.030+0000 7f15645ff700 1 --2- 172.21.15.133:0/3162392766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f155c107850 0x7f155c108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:12.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.030+0000 7f15645ff700 1 -- 172.21.15.133:0/3162392766 >> 172.21.15.133:0/3162392766 conn(0x7f155c06ceb0 msgr2=0x7f155c06f160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:20:12.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.034+0000 7f15645ff700 1 -- 172.21.15.133:0/3162392766 shutdown_connections 2023-07-14T18:20:12.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:12.034+0000 7f15645ff700 1 -- 172.21.15.133:0/3162392766 wait complete. 2023-07-14T18:20:12.113 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:20:13.412 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:13 smithi133 ceph-mon[126384]: pgmap v454: 129 pgs: 129 active+clean; 8.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 50 MiB/s wr, 3.00k op/s 2023-07-14T18:20:13.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:13 smithi161 ceph-mon[128993]: pgmap v454: 129 pgs: 129 active+clean; 8.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 50 MiB/s wr, 3.00k op/s 2023-07-14T18:20:13.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:13 smithi171 ceph-mon[129241]: pgmap v454: 129 pgs: 129 active+clean; 8.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 50 MiB/s wr, 3.00k op/s 2023-07-14T18:20:15.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:15 smithi161 ceph-mon[128993]: pgmap v455: 129 pgs: 129 active+clean; 8.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 41 MiB/s wr, 1.82k op/s 2023-07-14T18:20:15.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:15 smithi171 ceph-mon[129241]: pgmap v455: 129 pgs: 129 active+clean; 8.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 41 MiB/s wr, 1.82k op/s 2023-07-14T18:20:15.664 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:15 smithi133 ceph-mon[126384]: pgmap v455: 129 pgs: 129 active+clean; 8.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 41 MiB/s wr, 1.82k op/s 2023-07-14T18:20:15.884 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 50.0 s, 943.7 tps, lat 103.816 ms stddev 221.639 2023-07-14T18:20:17.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:17 smithi161 ceph-mon[128993]: pgmap v456: 129 pgs: 129 active+clean; 8.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 32 MiB/s wr, 1.27k op/s 2023-07-14T18:20:17.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:17 smithi171 ceph-mon[129241]: pgmap v456: 129 pgs: 129 active+clean; 8.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 32 MiB/s wr, 1.27k op/s 2023-07-14T18:20:17.665 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:17 smithi133 ceph-mon[126384]: pgmap v456: 129 pgs: 129 active+clean; 8.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 32 MiB/s wr, 1.27k op/s 2023-07-14T18:20:18.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:18 smithi161 ceph-mon[128993]: pgmap v457: 129 pgs: 129 active+clean; 8.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 46 MiB/s wr, 2.71k op/s 2023-07-14T18:20:18.621 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:18 smithi171 ceph-mon[129241]: pgmap v457: 129 pgs: 129 active+clean; 8.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 46 MiB/s wr, 2.71k op/s 2023-07-14T18:20:18.660 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:18 smithi133 ceph-mon[126384]: pgmap v457: 129 pgs: 129 active+clean; 8.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 46 MiB/s wr, 2.71k op/s 2023-07-14T18:20:20.880 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 55.0 s, 1408.4 tps, lat 72.657 ms stddev 99.689 2023-07-14T18:20:21.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:21 smithi161 ceph-mon[128993]: pgmap v458: 129 pgs: 129 active+clean; 8.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 46 MiB/s wr, 3.13k op/s 2023-07-14T18:20:21.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:21 smithi171 ceph-mon[129241]: pgmap v458: 129 pgs: 129 active+clean; 8.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 46 MiB/s wr, 3.13k op/s 2023-07-14T18:20:21.660 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:21 smithi133 ceph-mon[126384]: pgmap v458: 129 pgs: 129 active+clean; 8.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 46 MiB/s wr, 3.13k op/s 2023-07-14T18:20:23.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:23 smithi161 ceph-mon[128993]: pgmap v459: 129 pgs: 129 active+clean; 8.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 48 MiB/s wr, 3.37k op/s 2023-07-14T18:20:23.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:23 smithi171 ceph-mon[129241]: pgmap v459: 129 pgs: 129 active+clean; 8.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 48 MiB/s wr, 3.37k op/s 2023-07-14T18:20:23.661 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:23 smithi133 ceph-mon[126384]: pgmap v459: 129 pgs: 129 active+clean; 8.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 48 MiB/s wr, 3.37k op/s 2023-07-14T18:20:25.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:25 smithi161 ceph-mon[128993]: pgmap v460: 129 pgs: 129 active+clean; 8.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 48 MiB/s wr, 3.34k op/s 2023-07-14T18:20:25.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:25 smithi171 ceph-mon[129241]: pgmap v460: 129 pgs: 129 active+clean; 8.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 48 MiB/s wr, 3.34k op/s 2023-07-14T18:20:25.664 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:25 smithi133 ceph-mon[126384]: pgmap v460: 129 pgs: 129 active+clean; 8.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 48 MiB/s wr, 3.34k op/s 2023-07-14T18:20:25.879 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 60.0 s, 1998.6 tps, lat 46.832 ms stddev 84.778 2023-07-14T18:20:27.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:27 smithi161 ceph-mon[128993]: pgmap v461: 129 pgs: 129 active+clean; 8.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 48 MiB/s wr, 3.31k op/s 2023-07-14T18:20:27.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:27 smithi171 ceph-mon[129241]: pgmap v461: 129 pgs: 129 active+clean; 8.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 48 MiB/s wr, 3.31k op/s 2023-07-14T18:20:27.664 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:27 smithi133 ceph-mon[126384]: pgmap v461: 129 pgs: 129 active+clean; 8.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 48 MiB/s wr, 3.31k op/s 2023-07-14T18:20:29.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:29 smithi161 ceph-mon[128993]: pgmap v462: 129 pgs: 129 active+clean; 8.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 52 MiB/s wr, 3.26k op/s 2023-07-14T18:20:29.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:29 smithi171 ceph-mon[129241]: pgmap v462: 129 pgs: 129 active+clean; 8.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 52 MiB/s wr, 3.26k op/s 2023-07-14T18:20:29.665 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:29 smithi133 ceph-mon[126384]: pgmap v462: 129 pgs: 129 active+clean; 8.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 52 MiB/s wr, 3.26k op/s 2023-07-14T18:20:30.879 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 65.0 s, 1655.0 tps, lat 64.160 ms stddev 71.027 2023-07-14T18:20:31.411 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:31 smithi133 ceph-mon[126384]: pgmap v463: 129 pgs: 129 active+clean; 8.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 41 MiB/s wr, 1.92k op/s 2023-07-14T18:20:31.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:31 smithi161 ceph-mon[128993]: pgmap v463: 129 pgs: 129 active+clean; 8.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 41 MiB/s wr, 1.92k op/s 2023-07-14T18:20:31.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:31 smithi171 ceph-mon[129241]: pgmap v463: 129 pgs: 129 active+clean; 8.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 41 MiB/s wr, 1.92k op/s 2023-07-14T18:20:33.606 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:33 smithi161 ceph-mon[128993]: pgmap v464: 129 pgs: 129 active+clean; 8.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 36 MiB/s wr, 1.33k op/s 2023-07-14T18:20:33.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:33 smithi171 ceph-mon[129241]: pgmap v464: 129 pgs: 129 active+clean; 8.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 36 MiB/s wr, 1.33k op/s 2023-07-14T18:20:33.663 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:33 smithi133 ceph-mon[126384]: pgmap v464: 129 pgs: 129 active+clean; 8.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 36 MiB/s wr, 1.33k op/s 2023-07-14T18:20:35.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:35 smithi161 ceph-mon[128993]: pgmap v465: 129 pgs: 129 active+clean; 8.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 MiB/s wr, 1.12k op/s 2023-07-14T18:20:35.625 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:35 smithi171 ceph-mon[129241]: pgmap v465: 129 pgs: 129 active+clean; 8.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 MiB/s wr, 1.12k op/s 2023-07-14T18:20:35.664 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:35 smithi133 ceph-mon[126384]: pgmap v465: 129 pgs: 129 active+clean; 8.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 MiB/s wr, 1.12k op/s 2023-07-14T18:20:35.880 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 70.0 s, 800.0 tps, lat 115.596 ms stddev 343.601 2023-07-14T18:20:37.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:37 smithi161 ceph-mon[128993]: pgmap v466: 129 pgs: 129 active+clean; 8.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 33 MiB/s wr, 1.14k op/s 2023-07-14T18:20:37.621 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:37 smithi171 ceph-mon[129241]: pgmap v466: 129 pgs: 129 active+clean; 8.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 33 MiB/s wr, 1.14k op/s 2023-07-14T18:20:37.662 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:37 smithi133 ceph-mon[126384]: pgmap v466: 129 pgs: 129 active+clean; 8.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 33 MiB/s wr, 1.14k op/s 2023-07-14T18:20:39.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:39 smithi161 ceph-mon[128993]: pgmap v467: 129 pgs: 129 active+clean; 8.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 MiB/s wr, 2.74k op/s 2023-07-14T18:20:39.622 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:39 smithi171 ceph-mon[129241]: pgmap v467: 129 pgs: 129 active+clean; 8.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 MiB/s wr, 2.74k op/s 2023-07-14T18:20:39.661 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:39 smithi133 ceph-mon[126384]: pgmap v467: 129 pgs: 129 active+clean; 8.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 MiB/s wr, 2.74k op/s 2023-07-14T18:20:40.882 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 75.0 s, 1402.3 tps, lat 76.498 ms stddev 124.306 2023-07-14T18:20:41.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:41 smithi161 ceph-mon[128993]: pgmap v468: 129 pgs: 129 active+clean; 8.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 MiB/s wr, 3.19k op/s 2023-07-14T18:20:41.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:41 smithi171 ceph-mon[129241]: pgmap v468: 129 pgs: 129 active+clean; 8.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 MiB/s wr, 3.19k op/s 2023-07-14T18:20:41.661 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:41 smithi133 ceph-mon[126384]: pgmap v468: 129 pgs: 129 active+clean; 8.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 MiB/s wr, 3.19k op/s 2023-07-14T18:20:42.115 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:20:42.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.430+0000 7f0a9d7ff700 1 -- 172.21.15.133:0/4219996442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a980feb10 msgr2=0x7f0a980fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:20:42.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.430+0000 7f0a9d7ff700 1 --2- 172.21.15.133:0/4219996442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a980feb10 0x7f0a980fef20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0a880131e0 tx=0x7f0a88011f90 comp rx=0 tx=0).stop 2023-07-14T18:20:42.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.430+0000 7f0a9d7ff700 1 -- 172.21.15.133:0/4219996442 shutdown_connections 2023-07-14T18:20:42.480 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.430+0000 7f0a9d7ff700 1 --2- 172.21.15.133:0/4219996442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a980feb10 0x7f0a980fef20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.480 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.430+0000 7f0a9d7ff700 1 --2- 172.21.15.133:0/4219996442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a98100690 0x7f0a980677a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.480 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.430+0000 7f0a9d7ff700 1 --2- 172.21.15.133:0/4219996442 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a980ffd00 0x7f0a98100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.480 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.430+0000 7f0a9d7ff700 1 -- 172.21.15.133:0/4219996442 >> 172.21.15.133:0/4219996442 conn(0x7f0a980fa130 msgr2=0x7f0a980fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:20:42.480 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.430+0000 7f0a9d7ff700 1 -- 172.21.15.133:0/4219996442 shutdown_connections 2023-07-14T18:20:42.480 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.430+0000 7f0a9d7ff700 1 -- 172.21.15.133:0/4219996442 wait complete. 2023-07-14T18:20:42.483 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.435+0000 7f0a9d7ff700 1 Processor -- start 2023-07-14T18:20:42.484 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.435+0000 7f0a9d7ff700 1 -- start start 2023-07-14T18:20:42.484 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.435+0000 7f0a9d7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a980feb10 0x7f0a98193830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:42.485 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.435+0000 7f0a9d7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a980ffd00 0x7f0a98193d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:42.485 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.435+0000 7f0a9d7ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a98100690 0x7f0a9819ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:42.485 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.435+0000 7f0a9d7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9819b3b0 con 0x7f0a980feb10 2023-07-14T18:20:42.485 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.435+0000 7f0a9d7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9819b4f0 con 0x7f0a980ffd00 2023-07-14T18:20:42.485 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.435+0000 7f0a9d7ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9819b680 con 0x7f0a98100690 2023-07-14T18:20:42.486 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.436+0000 7f0a967fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a980ffd00 0x7f0a98193d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:20:42.486 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.436+0000 7f0a967fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a980ffd00 0x7f0a98193d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:38374/0 (socket says 172.21.15.133:38374) 2023-07-14T18:20:42.487 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.436+0000 7f0a967fc700 1 -- 172.21.15.133:0/2570285735 learned_addr learned my addr 172.21.15.133:0/2570285735 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:20:42.487 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.436+0000 7f0a977fe700 1 --2- 172.21.15.133:0/2570285735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a98100690 0x7f0a9819ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:20:42.487 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.437+0000 7f0a96ffd700 1 --2- 172.21.15.133:0/2570285735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a980feb10 0x7f0a98193830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:20:42.487 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.437+0000 7f0a977fe700 1 -- 172.21.15.133:0/2570285735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a980ffd00 msgr2=0x7f0a98193d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:20:42.487 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.437+0000 7f0a977fe700 1 --2- 172.21.15.133:0/2570285735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a980ffd00 0x7f0a98193d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.487 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.437+0000 7f0a977fe700 1 -- 172.21.15.133:0/2570285735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a980feb10 msgr2=0x7f0a98193830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:20:42.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.437+0000 7f0a977fe700 1 --2- 172.21.15.133:0/2570285735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a980feb10 0x7f0a98193830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.437+0000 7f0a977fe700 1 -- 172.21.15.133:0/2570285735 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a88011e00 con 0x7f0a98100690 2023-07-14T18:20:42.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.437+0000 7f0a967fc700 1 --2- 172.21.15.133:0/2570285735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a980ffd00 0x7f0a98193d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:20:42.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.437+0000 7f0a977fe700 1 --2- 172.21.15.133:0/2570285735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a98100690 0x7f0a9819ade0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000be20 tx=0x7f0a90009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:20:42.489 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.438+0000 7f0a7ffff700 1 -- 172.21.15.133:0/2570285735 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9000cc10 con 0x7f0a98100690 2023-07-14T18:20:42.489 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.438+0000 7f0a9d7ff700 1 -- 172.21.15.133:0/2570285735 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a9819b960 con 0x7f0a98100690 2023-07-14T18:20:42.489 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.438+0000 7f0a9d7ff700 1 -- 172.21.15.133:0/2570285735 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9819bef0 con 0x7f0a98100690 2023-07-14T18:20:42.489 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.438+0000 7f0a7ffff700 1 -- 172.21.15.133:0/2570285735 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a9000cd70 con 0x7f0a98100690 2023-07-14T18:20:42.489 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.439+0000 7f0a7ffff700 1 -- 172.21.15.133:0/2570285735 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a900126a0 con 0x7f0a98100690 2023-07-14T18:20:42.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.443+0000 7f0a7ffff700 1 -- 172.21.15.133:0/2570285735 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f0a90012940 con 0x7f0a98100690 2023-07-14T18:20:42.493 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.444+0000 7f0a7ffff700 1 --2- 172.21.15.133:0/2570285735 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0a800a1320 0x7f0a800a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:42.494 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.444+0000 7f0a7ffff700 1 -- 172.21.15.133:0/2570285735 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f0a90006e40 con 0x7f0a98100690 2023-07-14T18:20:42.494 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.444+0000 7f0a96ffd700 1 --2- 172.21.15.133:0/2570285735 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0a800a1320 0x7f0a800a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:20:42.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.445+0000 7f0ab10c0b80 1 Processor -- start 2023-07-14T18:20:42.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.445+0000 7f0ab10c0b80 1 -- start start 2023-07-14T18:20:42.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.445+0000 7f0a96ffd700 1 --2- 172.21.15.133:0/2570285735 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0a800a1320 0x7f0a800a37d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0a880045a0 tx=0x7f0a88013880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:20:42.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.446+0000 7f0ab10c0b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c0f9a2f60 0x557c0f9a3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:42.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.446+0000 7f0ab10c0b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c0f9a38b0 0x557c0f9a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:42.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.446+0000 7f0ab10c0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0f9a44b0 0x557c0f9a4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:42.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.446+0000 7f0ab10c0b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0f8a7cb0 con 0x557c0f9a2f60 2023-07-14T18:20:42.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.446+0000 7f0ab10c0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0f8faa40 con 0x557c0f9a44b0 2023-07-14T18:20:42.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.446+0000 7f0ab10c0b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0f8f3ed0 con 0x557c0f9a38b0 2023-07-14T18:20:42.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.446+0000 7f0a96ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c0f9a2f60 0x557c0f9a3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:20:42.497 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.446+0000 7f0a96ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c0f9a2f60 0x557c0f9a3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52138/0 (socket says 172.21.15.133:52138) 2023-07-14T18:20:42.497 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.446+0000 7f0a96ffd700 1 -- 172.21.15.133:0/3779440780 learned_addr learned my addr 172.21.15.133:0/3779440780 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:20:42.497 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.447+0000 7f0a977fe700 1 --2- 172.21.15.133:0/3779440780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0f9a44b0 0x557c0f9a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:20:42.497 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.447+0000 7f0a96ffd700 1 -- 172.21.15.133:0/3779440780 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c0f9a38b0 msgr2=0x557c0f9a3ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T18:20:42.497 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.447+0000 7f0a96ffd700 1 --2- 172.21.15.133:0/3779440780 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c0f9a38b0 0x557c0f9a3ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.497 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.447+0000 7f0a96ffd700 1 -- 172.21.15.133:0/3779440780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0f9a44b0 msgr2=0x557c0f9a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:20:42.497 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.447+0000 7f0a96ffd700 1 --2- 172.21.15.133:0/3779440780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0f9a44b0 0x557c0f9a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.498 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.447+0000 7f0a96ffd700 1 -- 172.21.15.133:0/3779440780 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c0f721de0 con 0x557c0f9a2f60 2023-07-14T18:20:42.501 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.451+0000 7f0a96ffd700 1 --2- 172.21.15.133:0/3779440780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c0f9a2f60 0x557c0f9a3370 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0a980ffb60 tx=0x7f0a88022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe9a10686f1f312 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:20:42.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.453+0000 7f0a7d7fa700 1 -- 172.21.15.133:0/3779440780 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a88033070 con 0x557c0f9a2f60 2023-07-14T18:20:42.504 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.453+0000 7f0a7d7fa700 1 -- 172.21.15.133:0/3779440780 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a88005500 con 0x557c0f9a2f60 2023-07-14T18:20:42.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.453+0000 7f0a7d7fa700 1 -- 172.21.15.133:0/3779440780 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8801a430 con 0x557c0f9a2f60 2023-07-14T18:20:42.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.454+0000 7f0ab10c0b80 1 -- 172.21.15.133:0/3779440780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c0f9a2f60 msgr2=0x557c0f9a3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:20:42.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.454+0000 7f0ab10c0b80 1 --2- 172.21.15.133:0/3779440780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c0f9a2f60 0x557c0f9a3370 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0a980ffb60 tx=0x7f0a88022000 comp rx=0 tx=0).stop 2023-07-14T18:20:42.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.455+0000 7f0ab10c0b80 1 -- 172.21.15.133:0/3779440780 shutdown_connections 2023-07-14T18:20:42.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.455+0000 7f0ab10c0b80 1 --2- 172.21.15.133:0/3779440780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0f9a44b0 0x557c0f9a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.455+0000 7f0ab10c0b80 1 --2- 172.21.15.133:0/3779440780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c0f9a2f60 0x557c0f9a3370 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.455+0000 7f0ab10c0b80 1 --2- 172.21.15.133:0/3779440780 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c0f9a38b0 0x557c0f9a3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.455+0000 7f0ab10c0b80 1 -- 172.21.15.133:0/3779440780 >> 172.21.15.133:0/3779440780 conn(0x557c0f998680 msgr2=0x557c0f99d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:20:42.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.456+0000 7f0ab10c0b80 1 -- 172.21.15.133:0/3779440780 shutdown_connections 2023-07-14T18:20:42.508 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.456+0000 7f0ab10c0b80 1 -- 172.21.15.133:0/3779440780 wait complete. 2023-07-14T18:20:42.508 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.457+0000 7f0ab10c0b80 1 -- start start 2023-07-14T18:20:42.508 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.457+0000 7f0ab10c0b80 1 Processor -- start 2023-07-14T18:20:42.509 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.458+0000 7f0ab10c0b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c0f9a2f60 0x557c0f9ac990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:42.509 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.458+0000 7f0ab10c0b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c0f9a38b0 0x557c0f9aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:42.509 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.458+0000 7f0ab10c0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0f9a44b0 0x557c0f9b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:42.509 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.458+0000 7f0ab10c0b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0f893ac0 con 0x557c0f9a2f60 2023-07-14T18:20:42.510 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.458+0000 7f0ab10c0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0f8937c0 con 0x557c0f9a44b0 2023-07-14T18:20:42.510 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.458+0000 7f0ab10c0b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0f777920 con 0x557c0f9a38b0 2023-07-14T18:20:42.510 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.458+0000 7f0ab10c0b80 10 client.0 _ll_register_callbacks cb 0x557c0f891760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:20:42.510 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.458+0000 7f0a967fc700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c0f9a38b0 0x557c0f9aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:20:42.510 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.458+0000 7f0a977fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0f9a44b0 0x557c0f9b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:20:42.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.458+0000 7f0a967fc700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c0f9a38b0 0x557c0f9aced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:57772/0 (socket says 172.21.15.133:57772) 2023-07-14T18:20:42.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.458+0000 7f0a967fc700 1 -- 172.21.15.133:0/524211755 learned_addr learned my addr 172.21.15.133:0/524211755 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:20:42.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.459+0000 7f0a967fc700 1 -- 172.21.15.133:0/524211755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0f9a44b0 msgr2=0x557c0f9b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:20:42.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.459+0000 7f0a967fc700 1 --2- 172.21.15.133:0/524211755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0f9a44b0 0x557c0f9b3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.459+0000 7f0a967fc700 1 -- 172.21.15.133:0/524211755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c0f9a2f60 msgr2=0x557c0f9ac990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T18:20:42.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.459+0000 7f0a967fc700 1 --2- 172.21.15.133:0/524211755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c0f9a2f60 0x557c0f9ac990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.459+0000 7f0a967fc700 1 -- 172.21.15.133:0/524211755 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a980681d0 con 0x557c0f9a38b0 2023-07-14T18:20:42.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.459+0000 7f0a967fc700 1 --2- 172.21.15.133:0/524211755 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c0f9a38b0 0x557c0f9aced0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00fea0 tx=0x7f0a8c00d620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5858cbc779f35f63 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:20:42.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.459+0000 7f0a7aff5700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:20:42.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.475+0000 7f0a7aff5700 1 -- 172.21.15.133:0/524211755 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c00ae40 con 0x557c0f9a38b0 2023-07-14T18:20:42.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.475+0000 7f0a7aff5700 1 -- 172.21.15.133:0/524211755 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f0a8c018440 con 0x557c0f9a38b0 2023-07-14T18:20:42.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.475+0000 7f0ab10c0b80 1 -- 172.21.15.133:0/524211755 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c0f84d4f0 con 0x557c0f9a38b0 2023-07-14T18:20:42.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.485+0000 7f0a7aff5700 1 -- 172.21.15.133:0/524211755 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a8c018600 con 0x557c0f9a38b0 2023-07-14T18:20:42.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.485+0000 7f0ab10c0b80 10 client.25741 fetch_fsmap learned FSMap version 34 2023-07-14T18:20:42.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.485+0000 7f0ab10c0b80 1 -- 172.21.15.133:0/524211755 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x557c0f70ac60 con 0x557c0f9a38b0 2023-07-14T18:20:42.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.489+0000 7f0a7aff5700 1 -- 172.21.15.133:0/524211755 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557c0f84d4f0 con 0x557c0f9a38b0 2023-07-14T18:20:42.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.489+0000 7f0a7aff5700 1 -- 172.21.15.133:0/524211755 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0a8c020070 con 0x557c0f9a38b0 2023-07-14T18:20:42.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.490+0000 7f0ab10c0b80 10 client.25741 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T18:20:42.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.490+0000 7f0ab10c0b80 10 client.25741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:20:42.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.490+0000 7f0ab10c0b80 20 client.25741 populate_metadata read hostname 'smithi133' 2023-07-14T18:20:42.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.490+0000 7f0ab10c0b80 1 --2- 172.21.15.133:0/524211755 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x557c0f9b87b0 0x557c0f9babc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:42.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.490+0000 7f0ab10c0b80 4 client.25741 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:20:42.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.490+0000 7f0ab10c0b80 1 -- 172.21.15.133:0/524211755 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c0f971da0 con 0x557c0f9b87b0 2023-07-14T18:20:42.549 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.500+0000 7f0a96ffd700 1 --2- 172.21.15.133:0/524211755 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x557c0f9b87b0 0x557c0f9babc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:20:42.556 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.504+0000 7f0a96ffd700 1 --2- 172.21.15.133:0/524211755 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x557c0f9b87b0 0x557c0f9babc0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=351c824670e875a8 server_cookie=54027d090eb51c5f in_seq=0 out_seq=0 2023-07-14T18:20:42.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.506+0000 7f0a7aff5700 10 client.25741 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:20:42.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.507+0000 7f0a7aff5700 1 -- 172.21.15.133:0/524211755 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x557c0f971da0 con 0x557c0f9b87b0 2023-07-14T18:20:42.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.507+0000 7f0a7aff5700 10 client.25741 handle_command_reply: tid=0 2023-07-14T18:20:42.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.507+0000 7f0ab10c0b80 1 client.25741 shutdown 2023-07-14T18:20:42.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.511+0000 7f0ab10c0b80 1 -- 172.21.15.133:0/524211755 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c0f9a38b0 msgr2=0x557c0f9aced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:20:42.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.511+0000 7f0ab10c0b80 1 --2- 172.21.15.133:0/524211755 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c0f9a38b0 0x557c0f9aced0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00fea0 tx=0x7f0a8c00d620 comp rx=0 tx=0).stop 2023-07-14T18:20:42.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.511+0000 7f0ab10c0b80 1 -- 172.21.15.133:0/524211755 shutdown_connections 2023-07-14T18:20:42.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.511+0000 7f0ab10c0b80 1 --2- 172.21.15.133:0/524211755 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x557c0f9b87b0 0x557c0f9babc0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.511+0000 7f0ab10c0b80 1 --2- 172.21.15.133:0/524211755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0f9a44b0 0x557c0f9b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.511+0000 7f0ab10c0b80 1 --2- 172.21.15.133:0/524211755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c0f9a2f60 0x557c0f9ac990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.511+0000 7f0ab10c0b80 1 --2- 172.21.15.133:0/524211755 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c0f9a38b0 0x557c0f9aced0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.511+0000 7f0ab10c0b80 1 -- 172.21.15.133:0/524211755 >> 172.21.15.133:0/524211755 conn(0x557c0f998680 msgr2=0x557c0f99c650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:20:42.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.511+0000 7f0a7aff5700 0 client.25741 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:20:42.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.511+0000 7f0ab10c0b80 1 -- 172.21.15.133:0/524211755 shutdown_connections 2023-07-14T18:20:42.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.511+0000 7f0ab10c0b80 1 -- 172.21.15.133:0/524211755 wait complete. 2023-07-14T18:20:42.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.511+0000 7f0ab10c0b80 20 client.25741 trim_cache size 0 max 16384 2023-07-14T18:20:42.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.539+0000 7f0ab10c0b80 1 Processor -- start 2023-07-14T18:20:42.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.544+0000 7f0ab10c0b80 1 -- start start 2023-07-14T18:20:42.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.545+0000 7f0ab10c0b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c0f9a2f60 0x557c0f9ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:42.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.545+0000 7f0ab10c0b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c0f9a38b0 0x557c0f9b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:42.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.545+0000 7f0ab10c0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0f9a44b0 0x557c0f99d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:42.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.545+0000 7f0ab10c0b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0f99f500 con 0x557c0f9a2f60 2023-07-14T18:20:42.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.545+0000 7f0ab10c0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0f9b5c00 con 0x557c0f9a44b0 2023-07-14T18:20:42.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.545+0000 7f0ab10c0b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0f9ad980 con 0x557c0f9a38b0 2023-07-14T18:20:42.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.546+0000 7f0a967fc700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c0f9a38b0 0x557c0f9b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:20:42.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.546+0000 7f0a967fc700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c0f9a38b0 0x557c0f9b3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:57778/0 (socket says 172.21.15.133:57778) 2023-07-14T18:20:42.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.546+0000 7f0a96ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c0f9a2f60 0x557c0f9ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:20:42.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.546+0000 7f0a967fc700 1 -- 172.21.15.133:0/2603458066 learned_addr learned my addr 172.21.15.133:0/2603458066 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:20:42.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.546+0000 7f0a977fe700 1 --2- 172.21.15.133:0/2603458066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0f9a44b0 0x557c0f99d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:20:42.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.546+0000 7f0a977fe700 1 -- 172.21.15.133:0/2603458066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c0f9a38b0 msgr2=0x557c0f9b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:20:42.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.546+0000 7f0a977fe700 1 --2- 172.21.15.133:0/2603458066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c0f9a38b0 0x557c0f9b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.546+0000 7f0a977fe700 1 -- 172.21.15.133:0/2603458066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c0f9a2f60 msgr2=0x557c0f9ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:20:42.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.546+0000 7f0a977fe700 1 --2- 172.21.15.133:0/2603458066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c0f9a2f60 0x557c0f9ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.546+0000 7f0a977fe700 1 -- 172.21.15.133:0/2603458066 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c0f748250 con 0x557c0f9a44b0 2023-07-14T18:20:42.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.547+0000 7f0a977fe700 1 --2- 172.21.15.133:0/2603458066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0f9a44b0 0x557c0f99d0f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0a900993d0 tx=0x7f0a900b9040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b5b3576f4bbb3e3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:20:42.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.547+0000 7f0a7dffb700 1 -- 172.21.15.133:0/2603458066 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9009d470 con 0x557c0f9a44b0 2023-07-14T18:20:42.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.547+0000 7f0a7dffb700 1 -- 172.21.15.133:0/2603458066 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a900ae070 con 0x557c0f9a44b0 2023-07-14T18:20:42.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.547+0000 7f0a7dffb700 1 -- 172.21.15.133:0/2603458066 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9009d160 con 0x557c0f9a44b0 2023-07-14T18:20:42.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.548+0000 7f0ab10c0b80 1 -- 172.21.15.133:0/2603458066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0f9a44b0 msgr2=0x557c0f99d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:20:42.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.548+0000 7f0ab10c0b80 1 --2- 172.21.15.133:0/2603458066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0f9a44b0 0x557c0f99d0f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0a900993d0 tx=0x7f0a900b9040 comp rx=0 tx=0).stop 2023-07-14T18:20:42.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.548+0000 7f0ab10c0b80 1 -- 172.21.15.133:0/2603458066 shutdown_connections 2023-07-14T18:20:42.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.548+0000 7f0ab10c0b80 1 --2- 172.21.15.133:0/2603458066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0f9a44b0 0x557c0f99d0f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.548+0000 7f0ab10c0b80 1 --2- 172.21.15.133:0/2603458066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c0f9a2f60 0x557c0f9ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.548+0000 7f0ab10c0b80 1 --2- 172.21.15.133:0/2603458066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c0f9a38b0 0x557c0f9b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.548+0000 7f0ab10c0b80 1 -- 172.21.15.133:0/2603458066 >> 172.21.15.133:0/2603458066 conn(0x557c0f998680 msgr2=0x557c0f99d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:20:42.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.548+0000 7f0ab10c0b80 1 -- 172.21.15.133:0/2603458066 shutdown_connections 2023-07-14T18:20:42.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.548+0000 7f0ab10c0b80 1 -- 172.21.15.133:0/2603458066 wait complete. 2023-07-14T18:20:42.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.548+0000 7f0ab10c0b80 1 -- start start 2023-07-14T18:20:42.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.549+0000 7f0ab10c0b80 1 Processor -- start 2023-07-14T18:20:42.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.549+0000 7f0ab10c0b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c0f9a2f60 0x557c0f9b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:42.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.549+0000 7f0ab10c0b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c0f9a38b0 0x557c0f9b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:42.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.549+0000 7f0ab10c0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0f9a44b0 0x557c0f9b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:42.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.549+0000 7f0ab10c0b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0f9b2c50 con 0x557c0f9a38b0 2023-07-14T18:20:42.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.549+0000 7f0ab10c0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0f9b2dc0 con 0x557c0f9a44b0 2023-07-14T18:20:42.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.549+0000 7f0ab10c0b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0f9b2f80 con 0x557c0f9a2f60 2023-07-14T18:20:42.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.549+0000 7f0ab10c0b80 10 client.0 _ll_register_callbacks cb 0x7f0a8c00f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:20:42.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.549+0000 7f0a977fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0f9a44b0 0x557c0f9b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:20:42.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.549+0000 7f0a977fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0f9a44b0 0x557c0f9b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:38412/0 (socket says 172.21.15.133:38412) 2023-07-14T18:20:42.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.549+0000 7f0a977fe700 1 -- 172.21.15.133:0/1056456355 learned_addr learned my addr 172.21.15.133:0/1056456355 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:20:42.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.550+0000 7f0a977fe700 1 -- 172.21.15.133:0/1056456355 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c0f9a2f60 msgr2=0x557c0f9b3b80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T18:20:42.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.550+0000 7f0a977fe700 1 --2- 172.21.15.133:0/1056456355 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c0f9a2f60 0x557c0f9b3b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.550+0000 7f0a977fe700 1 -- 172.21.15.133:0/1056456355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c0f9a38b0 msgr2=0x557c0f9b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:20:42.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.550+0000 7f0a977fe700 1 --2- 172.21.15.133:0/1056456355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c0f9a38b0 0x557c0f9b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.550+0000 7f0a977fe700 1 -- 172.21.15.133:0/1056456355 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9819bef0 con 0x557c0f9a44b0 2023-07-14T18:20:42.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.551+0000 7f0a977fe700 1 --2- 172.21.15.133:0/1056456355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0f9a44b0 0x557c0f9b26d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0a900a1360 tx=0x7f0a900995b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=102e82ddfa69c06c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:20:42.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.551+0000 7f0a7aff5700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:20:42.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.551+0000 7f0a7aff5700 1 -- 172.21.15.133:0/1056456355 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a900c9070 con 0x557c0f9a44b0 2023-07-14T18:20:42.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.551+0000 7f0a7aff5700 1 -- 172.21.15.133:0/1056456355 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f0a9009d470 con 0x557c0f9a44b0 2023-07-14T18:20:42.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.551+0000 7f0ab10c0b80 1 -- 172.21.15.133:0/1056456355 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a88005500 con 0x557c0f9a44b0 2023-07-14T18:20:42.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.552+0000 7f0a7aff5700 1 -- 172.21.15.133:0/1056456355 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a9009d160 con 0x557c0f9a44b0 2023-07-14T18:20:42.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.552+0000 7f0ab10c0b80 10 client.25727 fetch_fsmap learned FSMap version 34 2023-07-14T18:20:42.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.552+0000 7f0ab10c0b80 1 -- 172.21.15.133:0/1056456355 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x557c0f7202b0 con 0x557c0f9a44b0 2023-07-14T18:20:42.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.552+0000 7f0a7aff5700 1 -- 172.21.15.133:0/1056456355 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a88005500 con 0x557c0f9a44b0 2023-07-14T18:20:42.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.552+0000 7f0a7aff5700 1 -- 172.21.15.133:0/1056456355 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0a900a1400 con 0x557c0f9a44b0 2023-07-14T18:20:42.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.553+0000 7f0ab10c0b80 10 client.25727 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T18:20:42.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.553+0000 7f0ab10c0b80 10 client.25727 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:20:42.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.553+0000 7f0ab10c0b80 20 client.25727 populate_metadata read hostname 'smithi133' 2023-07-14T18:20:42.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.553+0000 7f0ab10c0b80 1 --2- 172.21.15.133:0/1056456355 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x557c0fa1e970 0x557c0fa20d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:20:42.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.553+0000 7f0ab10c0b80 4 client.25727 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:20:42.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.553+0000 7f0ab10c0b80 1 -- 172.21.15.133:0/1056456355 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557c0f8eceb0 con 0x557c0fa1e970 2023-07-14T18:20:42.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.556+0000 7f0a96ffd700 1 --2- 172.21.15.133:0/1056456355 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x557c0fa1e970 0x557c0fa20d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:20:42.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.558+0000 7f0a96ffd700 1 --2- 172.21.15.133:0/1056456355 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x557c0fa1e970 0x557c0fa20d80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85f090c7d1a24519 server_cookie=e476cd15554cc38d in_seq=0 out_seq=0 2023-07-14T18:20:42.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.558+0000 7f0a7aff5700 10 client.25727 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:20:42.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.559+0000 7f0a7aff5700 1 -- 172.21.15.133:0/1056456355 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x557c0f8eceb0 con 0x557c0fa1e970 2023-07-14T18:20:42.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.559+0000 7f0a7aff5700 10 client.25727 handle_command_reply: tid=0 2023-07-14T18:20:42.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.559+0000 7f0ab10c0b80 1 client.25727 shutdown 2023-07-14T18:20:42.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.559+0000 7f0ab10c0b80 1 -- 172.21.15.133:0/1056456355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0f9a44b0 msgr2=0x557c0f9b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:20:42.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.559+0000 7f0ab10c0b80 1 --2- 172.21.15.133:0/1056456355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0f9a44b0 0x557c0f9b26d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0a900a1360 tx=0x7f0a900995b0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.559+0000 7f0ab10c0b80 1 -- 172.21.15.133:0/1056456355 shutdown_connections 2023-07-14T18:20:42.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.559+0000 7f0ab10c0b80 1 --2- 172.21.15.133:0/1056456355 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x557c0fa1e970 0x557c0fa20d80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.559+0000 7f0ab10c0b80 1 --2- 172.21.15.133:0/1056456355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557c0f9a44b0 0x557c0f9b26d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.559+0000 7f0ab10c0b80 1 --2- 172.21.15.133:0/1056456355 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c0f9a2f60 0x557c0f9b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.559+0000 7f0ab10c0b80 1 --2- 172.21.15.133:0/1056456355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c0f9a38b0 0x557c0f9b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.559+0000 7f0ab10c0b80 1 -- 172.21.15.133:0/1056456355 >> 172.21.15.133:0/1056456355 conn(0x557c0f998680 msgr2=0x557c0f9aa8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:20:42.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.559+0000 7f0a7aff5700 0 client.25727 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:20:42.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.559+0000 7f0ab10c0b80 1 -- 172.21.15.133:0/1056456355 shutdown_connections 2023-07-14T18:20:42.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.559+0000 7f0ab10c0b80 1 -- 172.21.15.133:0/1056456355 wait complete. 2023-07-14T18:20:42.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.560+0000 7f0ab10c0b80 20 client.25727 trim_cache size 0 max 16384 2023-07-14T18:20:42.611 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:20:42.611 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-14T18:20:42.611 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:20:42.611 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:20:42.611 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:20:42.612 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:20:42.612 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:20:42.612 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:20:42.612 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:20:42.612 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:20:42.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.561+0000 7f0a9d7ff700 1 -- 172.21.15.133:0/2570285735 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0a800a1320 msgr2=0x7f0a800a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:20:42.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.561+0000 7f0a9d7ff700 1 --2- 172.21.15.133:0/2570285735 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0a800a1320 0x7f0a800a37d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0a880045a0 tx=0x7f0a88013880 comp rx=0 tx=0).stop 2023-07-14T18:20:42.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.561+0000 7f0a9d7ff700 1 -- 172.21.15.133:0/2570285735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a98100690 msgr2=0x7f0a9819ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:20:42.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.561+0000 7f0a9d7ff700 1 --2- 172.21.15.133:0/2570285735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a98100690 0x7f0a9819ade0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000be20 tx=0x7f0a90009ed0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.561+0000 7f0a9d7ff700 1 -- 172.21.15.133:0/2570285735 shutdown_connections 2023-07-14T18:20:42.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.561+0000 7f0a9d7ff700 1 --2- 172.21.15.133:0/2570285735 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f0a800a1320 0x7f0a800a37d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.561+0000 7f0a9d7ff700 1 --2- 172.21.15.133:0/2570285735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a98100690 0x7f0a9819ade0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.561+0000 7f0a9d7ff700 1 --2- 172.21.15.133:0/2570285735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a980feb10 0x7f0a98193830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.561+0000 7f0a9d7ff700 1 --2- 172.21.15.133:0/2570285735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a980ffd00 0x7f0a98193d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:20:42.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.561+0000 7f0a9d7ff700 1 -- 172.21.15.133:0/2570285735 >> 172.21.15.133:0/2570285735 conn(0x7f0a980fa130 msgr2=0x7f0a980fbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:20:42.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.561+0000 7f0a9d7ff700 1 -- 172.21.15.133:0/2570285735 shutdown_connections 2023-07-14T18:20:42.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:20:42.561+0000 7f0a9d7ff700 1 -- 172.21.15.133:0/2570285735 wait complete. 2023-07-14T18:20:42.643 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:20:43.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:43 smithi161 ceph-mon[128993]: pgmap v469: 129 pgs: 129 active+clean; 8.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 MiB/s wr, 3.33k op/s 2023-07-14T18:20:43.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:43 smithi171 ceph-mon[129241]: pgmap v469: 129 pgs: 129 active+clean; 8.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 MiB/s wr, 3.33k op/s 2023-07-14T18:20:43.661 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:43 smithi133 ceph-mon[126384]: pgmap v469: 129 pgs: 129 active+clean; 8.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 MiB/s wr, 3.33k op/s 2023-07-14T18:20:45.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:45 smithi161 ceph-mon[128993]: pgmap v470: 129 pgs: 129 active+clean; 8.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 47 MiB/s wr, 3.51k op/s 2023-07-14T18:20:45.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:45 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:20:45.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:45 smithi171 ceph-mon[129241]: pgmap v470: 129 pgs: 129 active+clean; 8.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 47 MiB/s wr, 3.51k op/s 2023-07-14T18:20:45.621 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:45 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:20:45.663 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:45 smithi133 ceph-mon[126384]: pgmap v470: 129 pgs: 129 active+clean; 8.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 47 MiB/s wr, 3.51k op/s 2023-07-14T18:20:45.663 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:45 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:20:45.879 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 80.0 s, 1579.4 tps, lat 55.210 ms stddev 139.585 2023-07-14T18:20:46.661 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:46 smithi133 ceph-mon[126384]: pgmap v471: 129 pgs: 129 active+clean; 8.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 46 MiB/s wr, 3.45k op/s 2023-07-14T18:20:46.855 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:46 smithi161 ceph-mon[128993]: pgmap v471: 129 pgs: 129 active+clean; 8.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 46 MiB/s wr, 3.45k op/s 2023-07-14T18:20:46.870 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:46 smithi171 ceph-mon[129241]: pgmap v471: 129 pgs: 129 active+clean; 8.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 46 MiB/s wr, 3.45k op/s 2023-07-14T18:20:47.665 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:47 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:20:47.665 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:47 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:20:47.665 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:47 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:20:47.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:47 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:20:47.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:47 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:20:47.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:47 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:20:47.870 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:47 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:20:47.870 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:47 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:20:47.871 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:47 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:20:48.856 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:48 smithi161 ceph-mon[128993]: pgmap v472: 129 pgs: 129 active+clean; 8.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 53 MiB/s wr, 4.01k op/s 2023-07-14T18:20:48.871 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:48 smithi171 ceph-mon[129241]: pgmap v472: 129 pgs: 129 active+clean; 8.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 53 MiB/s wr, 4.01k op/s 2023-07-14T18:20:48.911 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:48 smithi133 ceph-mon[126384]: pgmap v472: 129 pgs: 129 active+clean; 8.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 53 MiB/s wr, 4.01k op/s 2023-07-14T18:20:50.883 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 85.0 s, 789.4 tps, lat 141.716 ms stddev 262.709 2023-07-14T18:20:51.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:51 smithi161 ceph-mon[128993]: pgmap v473: 129 pgs: 129 active+clean; 8.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 39 MiB/s wr, 2.48k op/s 2023-07-14T18:20:51.621 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:51 smithi171 ceph-mon[129241]: pgmap v473: 129 pgs: 129 active+clean; 8.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 39 MiB/s wr, 2.48k op/s 2023-07-14T18:20:51.664 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:51 smithi133 ceph-mon[126384]: pgmap v473: 129 pgs: 129 active+clean; 8.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 39 MiB/s wr, 2.48k op/s 2023-07-14T18:20:53.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:53 smithi161 ceph-mon[128993]: pgmap v474: 129 pgs: 129 active+clean; 8.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 MiB/s wr, 2.08k op/s 2023-07-14T18:20:53.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:53 smithi171 ceph-mon[129241]: pgmap v474: 129 pgs: 129 active+clean; 8.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 MiB/s wr, 2.08k op/s 2023-07-14T18:20:53.662 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:53 smithi133 ceph-mon[126384]: pgmap v474: 129 pgs: 129 active+clean; 8.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 MiB/s wr, 2.08k op/s 2023-07-14T18:20:55.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:55 smithi161 ceph-mon[128993]: pgmap v475: 129 pgs: 129 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 MiB/s wr, 2.02k op/s 2023-07-14T18:20:55.630 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:55 smithi171 ceph-mon[129241]: pgmap v475: 129 pgs: 129 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 MiB/s wr, 2.02k op/s 2023-07-14T18:20:55.662 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:55 smithi133 ceph-mon[126384]: pgmap v475: 129 pgs: 129 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 MiB/s wr, 2.02k op/s 2023-07-14T18:20:55.882 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 90.0 s, 1628.8 tps, lat 61.943 ms stddev 56.311 2023-07-14T18:20:57.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:57 smithi161 ceph-mon[128993]: pgmap v476: 129 pgs: 129 active+clean; 8.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 1.73k op/s 2023-07-14T18:20:57.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:57 smithi171 ceph-mon[129241]: pgmap v476: 129 pgs: 129 active+clean; 8.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 1.73k op/s 2023-07-14T18:20:57.662 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:57 smithi133 ceph-mon[126384]: pgmap v476: 129 pgs: 129 active+clean; 8.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 1.73k op/s 2023-07-14T18:20:59.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:20:59 smithi161 ceph-mon[128993]: pgmap v477: 129 pgs: 129 active+clean; 8.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 34 MiB/s wr, 1.96k op/s 2023-07-14T18:20:59.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:20:59 smithi171 ceph-mon[129241]: pgmap v477: 129 pgs: 129 active+clean; 8.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 34 MiB/s wr, 1.96k op/s 2023-07-14T18:20:59.662 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:20:59 smithi133 ceph-mon[126384]: pgmap v477: 129 pgs: 129 active+clean; 8.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 34 MiB/s wr, 1.96k op/s 2023-07-14T18:21:00.881 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 95.0 s, 580.4 tps, lat 162.552 ms stddev 330.963 2023-07-14T18:21:01.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:01 smithi161 ceph-mon[128993]: pgmap v478: 129 pgs: 129 active+clean; 8.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 28 MiB/s wr, 1.59k op/s 2023-07-14T18:21:01.624 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:01 smithi171 ceph-mon[129241]: pgmap v478: 129 pgs: 129 active+clean; 8.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 28 MiB/s wr, 1.59k op/s 2023-07-14T18:21:01.662 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:01 smithi133 ceph-mon[126384]: pgmap v478: 129 pgs: 129 active+clean; 8.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 28 MiB/s wr, 1.59k op/s 2023-07-14T18:21:03.606 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:03 smithi161 ceph-mon[128993]: pgmap v479: 129 pgs: 129 active+clean; 8.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 36 MiB/s wr, 2.28k op/s 2023-07-14T18:21:03.621 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:03 smithi171 ceph-mon[129241]: pgmap v479: 129 pgs: 129 active+clean; 8.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 36 MiB/s wr, 2.28k op/s 2023-07-14T18:21:03.663 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:03 smithi133 ceph-mon[126384]: pgmap v479: 129 pgs: 129 active+clean; 8.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 36 MiB/s wr, 2.28k op/s 2023-07-14T18:21:05.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:05 smithi161 ceph-mon[128993]: pgmap v480: 129 pgs: 129 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 39 MiB/s wr, 2.61k op/s 2023-07-14T18:21:05.621 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:05 smithi171 ceph-mon[129241]: pgmap v480: 129 pgs: 129 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 39 MiB/s wr, 2.61k op/s 2023-07-14T18:21:05.663 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:05 smithi133 ceph-mon[126384]: pgmap v480: 129 pgs: 129 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 39 MiB/s wr, 2.61k op/s 2023-07-14T18:21:05.879 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 100.0 s, 1768.5 tps, lat 59.955 ms stddev 79.260 2023-07-14T18:21:07.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:07 smithi161 ceph-mon[128993]: pgmap v481: 129 pgs: 129 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 36 MiB/s wr, 2.42k op/s 2023-07-14T18:21:07.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:07 smithi171 ceph-mon[129241]: pgmap v481: 129 pgs: 129 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 36 MiB/s wr, 2.42k op/s 2023-07-14T18:21:07.661 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:07 smithi133 ceph-mon[126384]: pgmap v481: 129 pgs: 129 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 36 MiB/s wr, 2.42k op/s 2023-07-14T18:21:09.663 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:09 smithi133 ceph-mon[126384]: pgmap v482: 129 pgs: 129 active+clean; 8.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 41 MiB/s wr, 2.67k op/s 2023-07-14T18:21:09.855 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:09 smithi161 ceph-mon[128993]: pgmap v482: 129 pgs: 129 active+clean; 8.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 41 MiB/s wr, 2.67k op/s 2023-07-14T18:21:09.871 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:09 smithi171 ceph-mon[129241]: pgmap v482: 129 pgs: 129 active+clean; 8.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 41 MiB/s wr, 2.67k op/s 2023-07-14T18:21:10.879 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 105.0 s, 1499.6 tps, lat 66.464 ms stddev 113.761 2023-07-14T18:21:11.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:11 smithi161 ceph-mon[128993]: pgmap v483: 129 pgs: 129 active+clean; 8.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 40 MiB/s wr, 2.47k op/s 2023-07-14T18:21:11.621 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:11 smithi171 ceph-mon[129241]: pgmap v483: 129 pgs: 129 active+clean; 8.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 40 MiB/s wr, 2.47k op/s 2023-07-14T18:21:11.663 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:11 smithi133 ceph-mon[126384]: pgmap v483: 129 pgs: 129 active+clean; 8.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 40 MiB/s wr, 2.47k op/s 2023-07-14T18:21:12.644 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:21:13.143 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.091+0000 7f41f2bff700 1 -- 172.21.15.133:0/1746159568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41ec1006d0 msgr2=0x7f41ec102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:21:13.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.091+0000 7f41f2bff700 1 --2- 172.21.15.133:0/1746159568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41ec1006d0 0x7f41ec102ab0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009990 tx=0x7f41e8009ca0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.092+0000 7f41f2bff700 1 -- 172.21.15.133:0/1746159568 shutdown_connections 2023-07-14T18:21:13.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.092+0000 7f41f2bff700 1 --2- 172.21.15.133:0/1746159568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41ec1006d0 0x7f41ec102ab0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.092+0000 7f41f2bff700 1 --2- 172.21.15.133:0/1746159568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f41ec105910 0x7f41ec107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.092+0000 7f41f2bff700 1 --2- 172.21.15.133:0/1746159568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41ec102ff0 0x7f41ec1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.092+0000 7f41f2bff700 1 -- 172.21.15.133:0/1746159568 >> 172.21.15.133:0/1746159568 conn(0x7f41ec0fa130 msgr2=0x7f41ec0fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:21:13.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.092+0000 7f41f2bff700 1 -- 172.21.15.133:0/1746159568 shutdown_connections 2023-07-14T18:21:13.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.092+0000 7f41f2bff700 1 -- 172.21.15.133:0/1746159568 wait complete. 2023-07-14T18:21:13.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.094+0000 7f41f2bff700 1 Processor -- start 2023-07-14T18:21:13.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.096+0000 7f41f2bff700 1 -- start start 2023-07-14T18:21:13.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.097+0000 7f41f2bff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f41ec1006d0 0x7f41ec10eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:13.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.097+0000 7f41f2bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41ec102ff0 0x7f41ec10f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:13.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.097+0000 7f41f2bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41ec105910 0x7f41ec1140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:13.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.097+0000 7f41f2bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec1146a0 con 0x7f41ec105910 2023-07-14T18:21:13.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.097+0000 7f41f2bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec114810 con 0x7f41ec102ff0 2023-07-14T18:21:13.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.097+0000 7f41f2bff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec1149d0 con 0x7f41ec1006d0 2023-07-14T18:21:13.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.098+0000 7f41f119c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41ec105910 0x7f41ec1140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:13.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.098+0000 7f41f119c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41ec105910 0x7f41ec1140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49848/0 (socket says 172.21.15.133:49848) 2023-07-14T18:21:13.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.098+0000 7f41f119c700 1 -- 172.21.15.133:0/915959358 learned_addr learned my addr 172.21.15.133:0/915959358 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:21:13.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.098+0000 7f41f119c700 1 -- 172.21.15.133:0/915959358 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f41ec1006d0 msgr2=0x7f41ec10eb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T18:21:13.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.098+0000 7f41f119c700 1 --2- 172.21.15.133:0/915959358 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f41ec1006d0 0x7f41ec10eb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.098+0000 7f41f119c700 1 -- 172.21.15.133:0/915959358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41ec102ff0 msgr2=0x7f41ec10f070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T18:21:13.151 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.098+0000 7f41f119c700 1 --2- 172.21.15.133:0/915959358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41ec102ff0 0x7f41ec10f070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.151 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.098+0000 7f41f119c700 1 -- 172.21.15.133:0/915959358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e80095f0 con 0x7f41ec105910 2023-07-14T18:21:13.151 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.099+0000 7f41f119c700 1 --2- 172.21.15.133:0/915959358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41ec105910 0x7f41ec1140d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f41e400ef30 tx=0x7f41e400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:21:13.152 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.100+0000 7f41e1ffb700 1 -- 172.21.15.133:0/915959358 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e4018070 con 0x7f41ec105910 2023-07-14T18:21:13.152 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.101+0000 7f41e1ffb700 1 -- 172.21.15.133:0/915959358 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41e400f040 con 0x7f41ec105910 2023-07-14T18:21:13.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.101+0000 7f41e1ffb700 1 -- 172.21.15.133:0/915959358 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e4004280 con 0x7f41ec105910 2023-07-14T18:21:13.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.101+0000 7f41f2bff700 1 -- 172.21.15.133:0/915959358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41ec114bf0 con 0x7f41ec105910 2023-07-14T18:21:13.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.101+0000 7f41f2bff700 1 -- 172.21.15.133:0/915959358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41ec1acf60 con 0x7f41ec105910 2023-07-14T18:21:13.157 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.107+0000 7f4206548b80 1 Processor -- start 2023-07-14T18:21:13.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.111+0000 7f41e1ffb700 1 -- 172.21.15.133:0/915959358 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f41e40092e0 con 0x7f41ec105910 2023-07-14T18:21:13.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.113+0000 7f4206548b80 1 -- start start 2023-07-14T18:21:13.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.114+0000 7f4206548b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5af31cf60 0x55d5af31d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:13.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.114+0000 7f4206548b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5af31d8b0 0x55d5af31ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:13.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.114+0000 7f4206548b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d5af31e4b0 0x55d5af31e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:13.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.114+0000 7f4206548b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5af221cb0 con 0x55d5af31cf60 2023-07-14T18:21:13.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.114+0000 7f4206548b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5af274a40 con 0x55d5af31d8b0 2023-07-14T18:21:13.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.114+0000 7f4206548b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5af26ded0 con 0x55d5af31e4b0 2023-07-14T18:21:13.165 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.114+0000 7f41f099b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5af31d8b0 0x55d5af31ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:13.165 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.114+0000 7f41f099b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5af31d8b0 0x55d5af31ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:39654/0 (socket says 172.21.15.133:39654) 2023-07-14T18:21:13.165 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.114+0000 7f41f099b700 1 -- 172.21.15.133:0/2291304048 learned_addr learned my addr 172.21.15.133:0/2291304048 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:21:13.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.115+0000 7f41e3fff700 1 --2- 172.21.15.133:0/2291304048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d5af31e4b0 0x55d5af31e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:13.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.115+0000 7f41e1ffb700 1 --2- 172.21.15.133:0/915959358 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f41d40a12d0 0x7f41d40a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:13.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.115+0000 7f41e1ffb700 1 -- 172.21.15.133:0/915959358 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f41e40cbf60 con 0x7f41ec105910 2023-07-14T18:21:13.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.115+0000 7f41f099b700 1 -- 172.21.15.133:0/2291304048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d5af31e4b0 msgr2=0x55d5af31e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:21:13.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.115+0000 7f41f099b700 1 --2- 172.21.15.133:0/2291304048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d5af31e4b0 0x55d5af31e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.115+0000 7f41f099b700 1 -- 172.21.15.133:0/2291304048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5af31cf60 msgr2=0x55d5af31d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:21:13.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.115+0000 7f41f099b700 1 --2- 172.21.15.133:0/2291304048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5af31cf60 0x55d5af31d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.115+0000 7f41f099b700 1 -- 172.21.15.133:0/2291304048 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5af09bde0 con 0x55d5af31d8b0 2023-07-14T18:21:13.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.116+0000 7f41f119c700 1 --2- 172.21.15.133:0/915959358 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f41d40a12d0 0x7f41d40a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:13.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.117+0000 7f41f119c700 1 --2- 172.21.15.133:0/915959358 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f41d40a12d0 0x7f41d40a3780 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f41e40095b0 tx=0x7f41e4095410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:21:13.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.118+0000 7f41f099b700 1 --2- 172.21.15.133:0/2291304048 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5af31d8b0 0x55d5af31ded0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f41e8005910 tx=0x7f41e8005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbda06901e66919 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:21:13.168 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.118+0000 7f41d2ffd700 1 -- 172.21.15.133:0/2291304048 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e801d070 con 0x55d5af31d8b0 2023-07-14T18:21:13.168 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.118+0000 7f41d2ffd700 1 -- 172.21.15.133:0/2291304048 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41e800bd10 con 0x55d5af31d8b0 2023-07-14T18:21:13.168 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.118+0000 7f41d2ffd700 1 -- 172.21.15.133:0/2291304048 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e800fe40 con 0x55d5af31d8b0 2023-07-14T18:21:13.169 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.119+0000 7f4206548b80 1 -- 172.21.15.133:0/2291304048 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5af31d8b0 msgr2=0x55d5af31ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:21:13.169 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.119+0000 7f4206548b80 1 --2- 172.21.15.133:0/2291304048 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5af31d8b0 0x55d5af31ded0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f41e8005910 tx=0x7f41e8005c00 comp rx=0 tx=0).stop 2023-07-14T18:21:13.171 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.121+0000 7f4206548b80 1 -- 172.21.15.133:0/2291304048 shutdown_connections 2023-07-14T18:21:13.171 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.121+0000 7f4206548b80 1 --2- 172.21.15.133:0/2291304048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d5af31e4b0 0x55d5af31e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.171 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.121+0000 7f4206548b80 1 --2- 172.21.15.133:0/2291304048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5af31cf60 0x55d5af31d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.171 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.121+0000 7f4206548b80 1 --2- 172.21.15.133:0/2291304048 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5af31d8b0 0x55d5af31ded0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.171 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.121+0000 7f4206548b80 1 -- 172.21.15.133:0/2291304048 >> 172.21.15.133:0/2291304048 conn(0x55d5af312680 msgr2=0x55d5af3179d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:21:13.174 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.124+0000 7f4206548b80 1 -- 172.21.15.133:0/2291304048 shutdown_connections 2023-07-14T18:21:13.174 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.125+0000 7f4206548b80 1 -- 172.21.15.133:0/2291304048 wait complete. 2023-07-14T18:21:13.175 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.125+0000 7f4206548b80 1 -- start start 2023-07-14T18:21:13.175 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.126+0000 7f4206548b80 1 Processor -- start 2023-07-14T18:21:13.176 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.126+0000 7f4206548b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5af31cf60 0x55d5af3269a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:13.176 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.126+0000 7f4206548b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5af31d8b0 0x55d5af326ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:13.176 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.126+0000 7f4206548b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d5af31e4b0 0x55d5af32df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:13.176 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.126+0000 7f4206548b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5af20dac0 con 0x55d5af31cf60 2023-07-14T18:21:13.177 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.126+0000 7f4206548b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5af20d7c0 con 0x55d5af31d8b0 2023-07-14T18:21:13.177 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.126+0000 7f4206548b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5af0f1920 con 0x55d5af31e4b0 2023-07-14T18:21:13.177 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.127+0000 7f4206548b80 10 client.0 _ll_register_callbacks cb 0x55d5af20b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:21:13.177 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.128+0000 7f41f099b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5af31d8b0 0x55d5af326ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:13.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.128+0000 7f41e3fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d5af31e4b0 0x55d5af32df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:13.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.128+0000 7f41f099b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5af31d8b0 0x55d5af326ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:39660/0 (socket says 172.21.15.133:39660) 2023-07-14T18:21:13.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.128+0000 7f41f099b700 1 -- 172.21.15.133:0/362941729 learned_addr learned my addr 172.21.15.133:0/362941729 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:21:13.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.128+0000 7f41e3fff700 1 --2- 172.21.15.133:0/362941729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5af31cf60 0x55d5af3269a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:13.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.129+0000 7f41f099b700 1 -- 172.21.15.133:0/362941729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d5af31e4b0 msgr2=0x55d5af32df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:21:13.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.129+0000 7f41f099b700 1 --2- 172.21.15.133:0/362941729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d5af31e4b0 0x55d5af32df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.129+0000 7f41f099b700 1 -- 172.21.15.133:0/362941729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5af31cf60 msgr2=0x55d5af3269a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:21:13.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.129+0000 7f41f099b700 1 --2- 172.21.15.133:0/362941729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5af31cf60 0x55d5af3269a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.129+0000 7f41f099b700 1 -- 172.21.15.133:0/362941729 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41ec0fe9b0 con 0x55d5af31d8b0 2023-07-14T18:21:13.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.129+0000 7f41f099b700 1 --2- 172.21.15.133:0/362941729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5af31d8b0 0x55d5af326ee0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f41e800b680 tx=0x7f41e8010e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63a60e2ac743103 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:21:13.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.129+0000 7f41cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:21:13.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.130+0000 7f41cbfff700 1 -- 172.21.15.133:0/362941729 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e801d070 con 0x55d5af31d8b0 2023-07-14T18:21:13.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.130+0000 7f41cbfff700 1 -- 172.21.15.133:0/362941729 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f41e800ba60 con 0x55d5af31d8b0 2023-07-14T18:21:13.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.130+0000 7f4206548b80 1 -- 172.21.15.133:0/362941729 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5af1c74f0 con 0x55d5af31d8b0 2023-07-14T18:21:13.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.130+0000 7f41cbfff700 1 -- 172.21.15.133:0/362941729 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41e8022420 con 0x55d5af31d8b0 2023-07-14T18:21:13.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.130+0000 7f4206548b80 10 client.25736 fetch_fsmap learned FSMap version 34 2023-07-14T18:21:13.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.130+0000 7f4206548b80 1 -- 172.21.15.133:0/362941729 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d5af084c60 con 0x55d5af31d8b0 2023-07-14T18:21:13.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.131+0000 7f41cbfff700 1 -- 172.21.15.133:0/362941729 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d5af1c74f0 con 0x55d5af31d8b0 2023-07-14T18:21:13.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.131+0000 7f41cbfff700 1 -- 172.21.15.133:0/362941729 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f41e8032070 con 0x55d5af31d8b0 2023-07-14T18:21:13.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.131+0000 7f4206548b80 10 client.25736 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T18:21:13.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.131+0000 7f4206548b80 10 client.25736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:21:13.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.131+0000 7f4206548b80 20 client.25736 populate_metadata read hostname 'smithi133' 2023-07-14T18:21:13.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.131+0000 7f4206548b80 1 --2- 172.21.15.133:0/362941729 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55d5af3321e0 0x55d5af3345f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:13.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.131+0000 7f4206548b80 4 client.25736 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:21:13.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.131+0000 7f4206548b80 1 -- 172.21.15.133:0/362941729 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5af2ebda0 con 0x55d5af3321e0 2023-07-14T18:21:13.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.132+0000 7f41e3fff700 1 --2- 172.21.15.133:0/362941729 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55d5af3321e0 0x55d5af3345f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:13.187 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.132+0000 7f41e3fff700 1 --2- 172.21.15.133:0/362941729 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55d5af3321e0 0x55d5af3345f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f727f47863aa23fd server_cookie=5545be8fd6ae23b1 in_seq=0 out_seq=0 2023-07-14T18:21:13.187 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.137+0000 7f41cbfff700 10 client.25736 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:21:13.187 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.137+0000 7f41cbfff700 1 -- 172.21.15.133:0/362941729 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55d5af2ebda0 con 0x55d5af3321e0 2023-07-14T18:21:13.188 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.137+0000 7f41cbfff700 10 client.25736 handle_command_reply: tid=0 2023-07-14T18:21:13.188 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.138+0000 7f4206548b80 1 client.25736 shutdown 2023-07-14T18:21:13.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.139+0000 7f4206548b80 1 -- 172.21.15.133:0/362941729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5af31d8b0 msgr2=0x55d5af326ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:21:13.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.139+0000 7f4206548b80 1 --2- 172.21.15.133:0/362941729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5af31d8b0 0x55d5af326ee0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f41e800b680 tx=0x7f41e8010e70 comp rx=0 tx=0).stop 2023-07-14T18:21:13.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.139+0000 7f4206548b80 1 -- 172.21.15.133:0/362941729 shutdown_connections 2023-07-14T18:21:13.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.139+0000 7f4206548b80 1 --2- 172.21.15.133:0/362941729 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55d5af3321e0 0x55d5af3345f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.139+0000 7f4206548b80 1 --2- 172.21.15.133:0/362941729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d5af31e4b0 0x55d5af32df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.139+0000 7f4206548b80 1 --2- 172.21.15.133:0/362941729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5af31cf60 0x55d5af3269a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.139+0000 7f4206548b80 1 --2- 172.21.15.133:0/362941729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5af31d8b0 0x55d5af326ee0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.139+0000 7f4206548b80 1 -- 172.21.15.133:0/362941729 >> 172.21.15.133:0/362941729 conn(0x55d5af312680 msgr2=0x55d5af316650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:21:13.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.139+0000 7f41cbfff700 0 client.25736 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:21:13.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.140+0000 7f4206548b80 1 -- 172.21.15.133:0/362941729 shutdown_connections 2023-07-14T18:21:13.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.140+0000 7f4206548b80 1 -- 172.21.15.133:0/362941729 wait complete. 2023-07-14T18:21:13.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.140+0000 7f4206548b80 20 client.25736 trim_cache size 0 max 16384 2023-07-14T18:21:13.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.174+0000 7f4206548b80 1 Processor -- start 2023-07-14T18:21:13.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.174+0000 7f4206548b80 1 -- start start 2023-07-14T18:21:13.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.174+0000 7f4206548b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5af31cf60 0x55d5af32e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:13.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.174+0000 7f4206548b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d5af31d8b0 0x55d5af32de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:13.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.174+0000 7f4206548b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5af31e4b0 0x55d5af324650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:13.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.174+0000 7f4206548b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5af3122a0 con 0x55d5af31cf60 2023-07-14T18:21:13.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.174+0000 7f4206548b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5af31e330 con 0x55d5af31e4b0 2023-07-14T18:21:13.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.174+0000 7f4206548b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5af325e50 con 0x55d5af31d8b0 2023-07-14T18:21:13.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.175+0000 7f41e3fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5af31cf60 0x55d5af32e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:13.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.175+0000 7f41e3fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5af31cf60 0x55d5af32e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49884/0 (socket says 172.21.15.133:49884) 2023-07-14T18:21:13.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.175+0000 7f41e3fff700 1 -- 172.21.15.133:0/694629599 learned_addr learned my addr 172.21.15.133:0/694629599 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:21:13.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.175+0000 7f41e3fff700 1 --2- 172.21.15.133:0/694629599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5af31e4b0 0x55d5af324650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:13.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.176+0000 7f41f099b700 1 --2- 172.21.15.133:0/694629599 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d5af31d8b0 0x55d5af32de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:13.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.176+0000 7f41e3fff700 1 -- 172.21.15.133:0/694629599 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d5af31d8b0 msgr2=0x55d5af32de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:21:13.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.176+0000 7f41e3fff700 1 --2- 172.21.15.133:0/694629599 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d5af31d8b0 0x55d5af32de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.176+0000 7f41e3fff700 1 -- 172.21.15.133:0/694629599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5af31cf60 msgr2=0x55d5af32e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:21:13.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.176+0000 7f41e3fff700 1 --2- 172.21.15.133:0/694629599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5af31cf60 0x55d5af32e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.176+0000 7f41e3fff700 1 -- 172.21.15.133:0/694629599 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5af0c2250 con 0x55d5af31e4b0 2023-07-14T18:21:13.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.177+0000 7f41e3fff700 1 --2- 172.21.15.133:0/694629599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5af31e4b0 0x55d5af324650 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f41dc004530 tx=0x7f41dc00b650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54d688594dca93fe server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:21:13.234 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.185+0000 7f41d37fe700 1 -- 172.21.15.133:0/694629599 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc012040 con 0x55d5af31e4b0 2023-07-14T18:21:13.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.185+0000 7f41d37fe700 1 -- 172.21.15.133:0/694629599 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41dc004620 con 0x55d5af31e4b0 2023-07-14T18:21:13.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.185+0000 7f41d37fe700 1 -- 172.21.15.133:0/694629599 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc006670 con 0x55d5af31e4b0 2023-07-14T18:21:13.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.190+0000 7f4206548b80 1 -- 172.21.15.133:0/694629599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5af31e4b0 msgr2=0x55d5af324650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:21:13.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.190+0000 7f4206548b80 1 --2- 172.21.15.133:0/694629599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5af31e4b0 0x55d5af324650 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f41dc004530 tx=0x7f41dc00b650 comp rx=0 tx=0).stop 2023-07-14T18:21:13.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.196+0000 7f4206548b80 1 -- 172.21.15.133:0/694629599 shutdown_connections 2023-07-14T18:21:13.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.196+0000 7f4206548b80 1 --2- 172.21.15.133:0/694629599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5af31e4b0 0x55d5af324650 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.251 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.196+0000 7f4206548b80 1 --2- 172.21.15.133:0/694629599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5af31cf60 0x55d5af32e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.251 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.196+0000 7f4206548b80 1 --2- 172.21.15.133:0/694629599 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d5af31d8b0 0x55d5af32de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.251 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.196+0000 7f4206548b80 1 -- 172.21.15.133:0/694629599 >> 172.21.15.133:0/694629599 conn(0x55d5af312680 msgr2=0x55d5af3256a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:21:13.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.201+0000 7f4206548b80 1 -- 172.21.15.133:0/694629599 shutdown_connections 2023-07-14T18:21:13.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.209+0000 7f4206548b80 1 -- 172.21.15.133:0/694629599 wait complete. 2023-07-14T18:21:13.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.209+0000 7f4206548b80 1 -- start start 2023-07-14T18:21:13.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.209+0000 7f4206548b80 1 Processor -- start 2023-07-14T18:21:13.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.210+0000 7f4206548b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d5af31cf60 0x55d5af32dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:13.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.210+0000 7f4206548b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5af31d8b0 0x55d5af32a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:13.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.210+0000 7f4206548b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5af32ac10 0x55d5af385e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:13.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.210+0000 7f4206548b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5af32b0c0 con 0x55d5af31d8b0 2023-07-14T18:21:13.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.210+0000 7f4206548b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5af32b230 con 0x55d5af32ac10 2023-07-14T18:21:13.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.210+0000 7f4206548b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5af386370 con 0x55d5af31cf60 2023-07-14T18:21:13.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.210+0000 7f4206548b80 10 client.0 _ll_register_callbacks cb 0x7f41dc00b4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:21:13.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.214+0000 7f41e3fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d5af31cf60 0x55d5af32dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:13.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.214+0000 7f41e3fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d5af31cf60 0x55d5af32dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:44982/0 (socket says 172.21.15.133:44982) 2023-07-14T18:21:13.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.214+0000 7f41e3fff700 1 -- 172.21.15.133:0/3108248703 learned_addr learned my addr 172.21.15.133:0/3108248703 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:21:13.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.214+0000 7f41e3fff700 1 --2- 172.21.15.133:0/3108248703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5af32ac10 0x55d5af385e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:13.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.214+0000 7f41e3fff700 1 -- 172.21.15.133:0/3108248703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5af32ac10 msgr2=0x55d5af385e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:21:13.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.214+0000 7f41e3fff700 1 --2- 172.21.15.133:0/3108248703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5af32ac10 0x55d5af385e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.214+0000 7f41e3fff700 1 -- 172.21.15.133:0/3108248703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5af31d8b0 msgr2=0x55d5af32a6d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T18:21:13.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.214+0000 7f41e3fff700 1 --2- 172.21.15.133:0/3108248703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5af31d8b0 0x55d5af32a6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.214+0000 7f41e3fff700 1 -- 172.21.15.133:0/3108248703 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41dc023040 con 0x55d5af31cf60 2023-07-14T18:21:13.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.215+0000 7f41e3fff700 1 --2- 172.21.15.133:0/3108248703 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d5af31cf60 0x55d5af32dc30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f41dc004530 tx=0x7f41dc00b970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e817f44f58f4cc0a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:21:13.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.215+0000 7f41cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:21:13.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.215+0000 7f41cbfff700 1 -- 172.21.15.133:0/3108248703 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc04e030 con 0x55d5af31cf60 2023-07-14T18:21:13.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.215+0000 7f41cbfff700 1 -- 172.21.15.133:0/3108248703 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f41dc004860 con 0x55d5af31cf60 2023-07-14T18:21:13.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.215+0000 7f4206548b80 1 -- 172.21.15.133:0/3108248703 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41e800bd10 con 0x55d5af31cf60 2023-07-14T18:21:13.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.226+0000 7f41cbfff700 1 -- 172.21.15.133:0/3108248703 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41dc012070 con 0x55d5af31cf60 2023-07-14T18:21:13.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.233+0000 7f4206548b80 10 client.25759 fetch_fsmap learned FSMap version 34 2023-07-14T18:21:13.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.233+0000 7f4206548b80 1 -- 172.21.15.133:0/3108248703 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d5af09a2b0 con 0x55d5af31cf60 2023-07-14T18:21:13.294 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.245+0000 7f41cbfff700 1 -- 172.21.15.133:0/3108248703 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41e800bd10 con 0x55d5af31cf60 2023-07-14T18:21:13.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.245+0000 7f41cbfff700 1 -- 172.21.15.133:0/3108248703 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f41dc003b40 con 0x55d5af31cf60 2023-07-14T18:21:13.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.245+0000 7f4206548b80 10 client.25759 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T18:21:13.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.245+0000 7f4206548b80 10 client.25759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:21:13.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.245+0000 7f4206548b80 20 client.25759 populate_metadata read hostname 'smithi133' 2023-07-14T18:21:13.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.245+0000 7f4206548b80 1 --2- 172.21.15.133:0/3108248703 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55d5af387b00 0x55d5af389f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:13.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.245+0000 7f4206548b80 4 client.25759 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:21:13.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.245+0000 7f4206548b80 1 -- 172.21.15.133:0/3108248703 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d5af266eb0 con 0x55d5af387b00 2023-07-14T18:21:13.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.256+0000 7f41f099b700 1 --2- 172.21.15.133:0/3108248703 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55d5af387b00 0x55d5af389f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:13.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.273+0000 7f41f099b700 1 --2- 172.21.15.133:0/3108248703 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55d5af387b00 0x55d5af389f10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=278e5cf3c8b824c2 server_cookie=5ef54d89f9976fe4 in_seq=0 out_seq=0 2023-07-14T18:21:13.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.274+0000 7f41cbfff700 10 client.25759 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:21:13.333 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.283+0000 7f41cbfff700 1 -- 172.21.15.133:0/3108248703 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55d5af266eb0 con 0x55d5af387b00 2023-07-14T18:21:13.333 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.283+0000 7f41cbfff700 10 client.25759 handle_command_reply: tid=0 2023-07-14T18:21:13.333 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.283+0000 7f4206548b80 1 client.25759 shutdown 2023-07-14T18:21:13.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.284+0000 7f4206548b80 1 -- 172.21.15.133:0/3108248703 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d5af31cf60 msgr2=0x55d5af32dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:21:13.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.284+0000 7f4206548b80 1 --2- 172.21.15.133:0/3108248703 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d5af31cf60 0x55d5af32dc30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f41dc004530 tx=0x7f41dc00b970 comp rx=0 tx=0).stop 2023-07-14T18:21:13.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.286+0000 7f4206548b80 1 -- 172.21.15.133:0/3108248703 shutdown_connections 2023-07-14T18:21:13.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.286+0000 7f4206548b80 1 --2- 172.21.15.133:0/3108248703 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55d5af387b00 0x55d5af389f10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.286+0000 7f4206548b80 1 --2- 172.21.15.133:0/3108248703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5af32ac10 0x55d5af385e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.286+0000 7f4206548b80 1 --2- 172.21.15.133:0/3108248703 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d5af31cf60 0x55d5af32dc30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.286+0000 7f4206548b80 1 --2- 172.21.15.133:0/3108248703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5af31d8b0 0x55d5af32a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.286+0000 7f4206548b80 1 -- 172.21.15.133:0/3108248703 >> 172.21.15.133:0/3108248703 conn(0x55d5af312680 msgr2=0x55d5af32e810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:21:13.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.291+0000 7f41cbfff700 0 client.25759 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:21:13.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.292+0000 7f4206548b80 1 -- 172.21.15.133:0/3108248703 shutdown_connections 2023-07-14T18:21:13.345 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:21:13.345 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-14T18:21:13.345 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:21:13.345 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:21:13.346 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:21:13.346 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:21:13.346 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:21:13.346 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:21:13.346 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:21:13.346 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:21:13.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.295+0000 7f4206548b80 1 -- 172.21.15.133:0/3108248703 wait complete. 2023-07-14T18:21:13.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.295+0000 7f4206548b80 20 client.25759 trim_cache size 0 max 16384 2023-07-14T18:21:13.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.296+0000 7f41f2bff700 1 -- 172.21.15.133:0/915959358 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f41d40a12d0 msgr2=0x7f41d40a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:21:13.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.296+0000 7f41f2bff700 1 --2- 172.21.15.133:0/915959358 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f41d40a12d0 0x7f41d40a3780 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f41e40095b0 tx=0x7f41e4095410 comp rx=0 tx=0).stop 2023-07-14T18:21:13.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.296+0000 7f41f2bff700 1 -- 172.21.15.133:0/915959358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41ec105910 msgr2=0x7f41ec1140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:21:13.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.296+0000 7f41f2bff700 1 --2- 172.21.15.133:0/915959358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41ec105910 0x7f41ec1140d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f41e400ef30 tx=0x7f41e400c450 comp rx=0 tx=0).stop 2023-07-14T18:21:13.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.296+0000 7f41f2bff700 1 -- 172.21.15.133:0/915959358 shutdown_connections 2023-07-14T18:21:13.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.296+0000 7f41f2bff700 1 --2- 172.21.15.133:0/915959358 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f41d40a12d0 0x7f41d40a3780 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.296+0000 7f41f2bff700 1 --2- 172.21.15.133:0/915959358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41ec105910 0x7f41ec1140d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.296+0000 7f41f2bff700 1 --2- 172.21.15.133:0/915959358 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f41ec1006d0 0x7f41ec10eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.296+0000 7f41f2bff700 1 --2- 172.21.15.133:0/915959358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41ec102ff0 0x7f41ec10f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:13.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.296+0000 7f41f2bff700 1 -- 172.21.15.133:0/915959358 >> 172.21.15.133:0/915959358 conn(0x7f41ec0fa130 msgr2=0x7f41ec101330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:21:13.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.296+0000 7f41f2bff700 1 -- 172.21.15.133:0/915959358 shutdown_connections 2023-07-14T18:21:13.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:13.296+0000 7f41f2bff700 1 -- 172.21.15.133:0/915959358 wait complete. 2023-07-14T18:21:13.378 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:21:13.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:13 smithi161 ceph-mon[128993]: pgmap v484: 129 pgs: 129 active+clean; 8.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 MiB/s wr, 2.21k op/s 2023-07-14T18:21:13.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:13 smithi171 ceph-mon[129241]: pgmap v484: 129 pgs: 129 active+clean; 8.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 MiB/s wr, 2.21k op/s 2023-07-14T18:21:13.661 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:13 smithi133 ceph-mon[126384]: pgmap v484: 129 pgs: 129 active+clean; 8.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 MiB/s wr, 2.21k op/s 2023-07-14T18:21:15.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:15 smithi161 ceph-mon[128993]: pgmap v485: 129 pgs: 129 active+clean; 8.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 MiB/s wr, 1.62k op/s 2023-07-14T18:21:15.621 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:15 smithi171 ceph-mon[129241]: pgmap v485: 129 pgs: 129 active+clean; 8.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 MiB/s wr, 1.62k op/s 2023-07-14T18:21:15.662 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:15 smithi133 ceph-mon[126384]: pgmap v485: 129 pgs: 129 active+clean; 8.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 MiB/s wr, 1.62k op/s 2023-07-14T18:21:15.881 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 110.0 s, 1318.0 tps, lat 71.646 ms stddev 56.848 2023-07-14T18:21:17.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:17 smithi161 ceph-mon[128993]: pgmap v486: 129 pgs: 129 active+clean; 8.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 MiB/s wr, 972 op/s 2023-07-14T18:21:17.621 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:17 smithi171 ceph-mon[129241]: pgmap v486: 129 pgs: 129 active+clean; 8.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 MiB/s wr, 972 op/s 2023-07-14T18:21:17.663 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:17 smithi133 ceph-mon[126384]: pgmap v486: 129 pgs: 129 active+clean; 8.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 MiB/s wr, 972 op/s 2023-07-14T18:21:19.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:19 smithi161 ceph-mon[128993]: pgmap v487: 129 pgs: 129 active+clean; 8.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 MiB/s wr, 1.50k op/s 2023-07-14T18:21:19.627 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:19 smithi171 ceph-mon[129241]: pgmap v487: 129 pgs: 129 active+clean; 8.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 MiB/s wr, 1.50k op/s 2023-07-14T18:21:19.662 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:19 smithi133 ceph-mon[126384]: pgmap v487: 129 pgs: 129 active+clean; 8.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 MiB/s wr, 1.50k op/s 2023-07-14T18:21:20.881 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 115.0 s, 985.4 tps, lat 98.780 ms stddev 227.253 2023-07-14T18:21:21.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:21 smithi161 ceph-mon[128993]: pgmap v488: 129 pgs: 129 active+clean; 8.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 32 MiB/s wr, 1.68k op/s 2023-07-14T18:21:21.621 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:21 smithi171 ceph-mon[129241]: pgmap v488: 129 pgs: 129 active+clean; 8.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 32 MiB/s wr, 1.68k op/s 2023-07-14T18:21:21.666 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:21 smithi133 ceph-mon[126384]: pgmap v488: 129 pgs: 129 active+clean; 8.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 32 MiB/s wr, 1.68k op/s 2023-07-14T18:21:23.664 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:23 smithi133 ceph-mon[126384]: pgmap v489: 129 pgs: 129 active+clean; 8.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 36 MiB/s wr, 2.27k op/s 2023-07-14T18:21:23.856 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:23 smithi161 ceph-mon[128993]: pgmap v489: 129 pgs: 129 active+clean; 8.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 36 MiB/s wr, 2.27k op/s 2023-07-14T18:21:23.871 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:23 smithi171 ceph-mon[129241]: pgmap v489: 129 pgs: 129 active+clean; 8.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 36 MiB/s wr, 2.27k op/s 2023-07-14T18:21:25.856 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:25 smithi161 ceph-mon[128993]: pgmap v490: 129 pgs: 129 active+clean; 8.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 40 MiB/s wr, 2.85k op/s 2023-07-14T18:21:25.871 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:25 smithi171 ceph-mon[129241]: pgmap v490: 129 pgs: 129 active+clean; 8.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 40 MiB/s wr, 2.85k op/s 2023-07-14T18:21:25.879 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 120.0 s, 612.0 tps, lat 177.094 ms stddev 248.794 2023-07-14T18:21:25.913 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:25 smithi133 ceph-mon[126384]: pgmap v490: 129 pgs: 129 active+clean; 8.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 40 MiB/s wr, 2.85k op/s 2023-07-14T18:21:26.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:26 smithi161 ceph-mon[128993]: pgmap v491: 129 pgs: 129 active+clean; 8.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 35 MiB/s wr, 2.60k op/s 2023-07-14T18:21:26.881 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:26 smithi171 ceph-mon[129241]: pgmap v491: 129 pgs: 129 active+clean; 8.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 35 MiB/s wr, 2.60k op/s 2023-07-14T18:21:26.912 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:26 smithi133 ceph-mon[126384]: pgmap v491: 129 pgs: 129 active+clean; 8.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 35 MiB/s wr, 2.60k op/s 2023-07-14T18:21:29.105 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:28 smithi161 ceph-mon[128993]: pgmap v492: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 MiB/s wr, 3.17k op/s 2023-07-14T18:21:29.121 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:28 smithi171 ceph-mon[129241]: pgmap v492: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 MiB/s wr, 3.17k op/s 2023-07-14T18:21:29.164 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:28 smithi133 ceph-mon[126384]: pgmap v492: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 MiB/s wr, 3.17k op/s 2023-07-14T18:21:30.881 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 125.0 s, 1854.0 tps, lat 53.664 ms stddev 59.819 2023-07-14T18:21:31.606 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:31 smithi161 ceph-mon[128993]: pgmap v493: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 MiB/s wr, 2.86k op/s 2023-07-14T18:21:31.625 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:31 smithi171 ceph-mon[129241]: pgmap v493: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 MiB/s wr, 2.86k op/s 2023-07-14T18:21:31.665 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:31 smithi133 ceph-mon[126384]: pgmap v493: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 MiB/s wr, 2.86k op/s 2023-07-14T18:21:33.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:33 smithi161 ceph-mon[128993]: pgmap v494: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 39 MiB/s wr, 2.46k op/s 2023-07-14T18:21:33.623 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:33 smithi171 ceph-mon[129241]: pgmap v494: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 39 MiB/s wr, 2.46k op/s 2023-07-14T18:21:33.665 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:33 smithi133 ceph-mon[126384]: pgmap v494: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 39 MiB/s wr, 2.46k op/s 2023-07-14T18:21:35.662 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:35 smithi133 ceph-mon[126384]: pgmap v495: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 MiB/s wr, 1.85k op/s 2023-07-14T18:21:35.857 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:35 smithi161 ceph-mon[128993]: pgmap v495: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 MiB/s wr, 1.85k op/s 2023-07-14T18:21:35.871 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:35 smithi171 ceph-mon[129241]: pgmap v495: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 MiB/s wr, 1.85k op/s 2023-07-14T18:21:35.881 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 130.0 s, 1248.3 tps, lat 79.236 ms stddev 204.310 2023-07-14T18:21:37.856 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:37 smithi161 ceph-mon[128993]: pgmap v496: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 MiB/s wr, 1.09k op/s 2023-07-14T18:21:37.871 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:37 smithi171 ceph-mon[129241]: pgmap v496: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 MiB/s wr, 1.09k op/s 2023-07-14T18:21:37.912 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:37 smithi133 ceph-mon[126384]: pgmap v496: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 MiB/s wr, 1.09k op/s 2023-07-14T18:21:38.856 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:38 smithi161 ceph-mon[128993]: pgmap v497: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 MiB/s wr, 1.38k op/s 2023-07-14T18:21:38.871 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:38 smithi171 ceph-mon[129241]: pgmap v497: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 MiB/s wr, 1.38k op/s 2023-07-14T18:21:38.912 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:38 smithi133 ceph-mon[126384]: pgmap v497: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 MiB/s wr, 1.38k op/s 2023-07-14T18:21:40.880 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 135.0 s, 1226.0 tps, lat 82.765 ms stddev 92.981 2023-07-14T18:21:41.606 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:41 smithi161 ceph-mon[128993]: pgmap v498: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 28 MiB/s wr, 1.07k op/s 2023-07-14T18:21:41.621 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:41 smithi171 ceph-mon[129241]: pgmap v498: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 28 MiB/s wr, 1.07k op/s 2023-07-14T18:21:41.663 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:41 smithi133 ceph-mon[126384]: pgmap v498: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 28 MiB/s wr, 1.07k op/s 2023-07-14T18:21:43.379 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:21:43.564 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:43 smithi133 ceph-mon[126384]: pgmap v499: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 26 MiB/s wr, 909 op/s 2023-07-14T18:21:43.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:43 smithi161 ceph-mon[128993]: pgmap v499: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 26 MiB/s wr, 909 op/s 2023-07-14T18:21:43.623 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:43 smithi171 ceph-mon[129241]: pgmap v499: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 26 MiB/s wr, 909 op/s 2023-07-14T18:21:43.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.843+0000 7f93d73ff700 1 -- 172.21.15.133:0/1874273351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93d0071960 msgr2=0x7f93d010b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:21:43.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.843+0000 7f93d73ff700 1 --2- 172.21.15.133:0/1874273351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93d0071960 0x7f93d010b900 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f93c00098f0 tx=0x7f93c0009c00 comp rx=0 tx=0).stop 2023-07-14T18:21:43.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.867+0000 7f93d73ff700 1 -- 172.21.15.133:0/1874273351 shutdown_connections 2023-07-14T18:21:43.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.867+0000 7f93d73ff700 1 --2- 172.21.15.133:0/1874273351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d010be90 0x7f93d010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:43.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.867+0000 7f93d73ff700 1 --2- 172.21.15.133:0/1874273351 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93d0071010 0x7f93d0071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:43.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.867+0000 7f93d73ff700 1 --2- 172.21.15.133:0/1874273351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93d0071960 0x7f93d010b900 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:43.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.867+0000 7f93d73ff700 1 -- 172.21.15.133:0/1874273351 >> 172.21.15.133:0/1874273351 conn(0x7f93d006cbc0 msgr2=0x7f93d006eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:21:43.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.876+0000 7f93d73ff700 1 -- 172.21.15.133:0/1874273351 shutdown_connections 2023-07-14T18:21:43.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.879+0000 7f93d73ff700 1 -- 172.21.15.133:0/1874273351 wait complete. 2023-07-14T18:21:43.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.880+0000 7f93d73ff700 1 Processor -- start 2023-07-14T18:21:43.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.881+0000 7f93d73ff700 1 -- start start 2023-07-14T18:21:43.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.882+0000 7f93d73ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93d0071010 0x7f93d01c08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:43.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.882+0000 7f93d73ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d0071960 0x7f93d01c2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:43.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.882+0000 7f93d73ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93d010be90 0x7f93d01c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:43.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.882+0000 7f93d73ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d01c5bc0 con 0x7f93d0071010 2023-07-14T18:21:43.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.882+0000 7f93d73ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d01c5d30 con 0x7f93d0071960 2023-07-14T18:21:43.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.882+0000 7f93d73ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d01c5ef0 con 0x7f93d010be90 2023-07-14T18:21:43.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.882+0000 7f93d599c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93d010be90 0x7f93d01c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:43.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.883+0000 7f93d599c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93d010be90 0x7f93d01c53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:54802/0 (socket says 172.21.15.133:54802) 2023-07-14T18:21:43.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.883+0000 7f93d599c700 1 -- 172.21.15.133:0/2044965053 learned_addr learned my addr 172.21.15.133:0/2044965053 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:21:43.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.883+0000 7f93d499a700 1 --2- 172.21.15.133:0/2044965053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d0071960 0x7f93d01c2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:43.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.885+0000 7f93d599c700 1 -- 172.21.15.133:0/2044965053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d0071960 msgr2=0x7f93d01c2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:21:43.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.885+0000 7f93d599c700 1 --2- 172.21.15.133:0/2044965053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d0071960 0x7f93d01c2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:43.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.885+0000 7f93d599c700 1 -- 172.21.15.133:0/2044965053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93d0071010 msgr2=0x7f93d01c08d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T18:21:43.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.885+0000 7f93d599c700 1 --2- 172.21.15.133:0/2044965053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93d0071010 0x7f93d01c08d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:43.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.885+0000 7f93d599c700 1 -- 172.21.15.133:0/2044965053 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c0009520 con 0x7f93d010be90 2023-07-14T18:21:43.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.888+0000 7f93d599c700 1 --2- 172.21.15.133:0/2044965053 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93d010be90 0x7f93d01c53f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f93c800b970 tx=0x7f93c800bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:21:43.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.892+0000 7f93ce7fc700 1 -- 172.21.15.133:0/2044965053 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c800c9b0 con 0x7f93d010be90 2023-07-14T18:21:43.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.892+0000 7f93ce7fc700 1 -- 172.21.15.133:0/2044965053 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93c8014440 con 0x7f93d010be90 2023-07-14T18:21:43.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.892+0000 7f93ce7fc700 1 -- 172.21.15.133:0/2044965053 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c80075d0 con 0x7f93d010be90 2023-07-14T18:21:43.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.892+0000 7f93d73ff700 1 -- 172.21.15.133:0/2044965053 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d01c61d0 con 0x7f93d010be90 2023-07-14T18:21:43.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.892+0000 7f93d73ff700 1 -- 172.21.15.133:0/2044965053 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d01c66e0 con 0x7f93d010be90 2023-07-14T18:21:43.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.900+0000 7f93ce7fc700 1 -- 172.21.15.133:0/2044965053 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f93c800cb10 con 0x7f93d010be90 2023-07-14T18:21:43.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.900+0000 7f93ead28b80 1 Processor -- start 2023-07-14T18:21:43.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.901+0000 7f93ce7fc700 1 --2- 172.21.15.133:0/2044965053 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f93b80a1350 0x7f93b80a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:43.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.902+0000 7f93ce7fc700 1 -- 172.21.15.133:0/2044965053 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f93c8006e40 con 0x7f93d010be90 2023-07-14T18:21:43.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.903+0000 7f93ead28b80 1 -- start start 2023-07-14T18:21:43.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.903+0000 7f93ead28b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e37fa5f60 0x563e37fa6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:43.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.903+0000 7f93ead28b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563e37fa68b0 0x563e37fa6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:43.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.903+0000 7f93ead28b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fa74b0 0x563e37fa7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:43.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.903+0000 7f93ead28b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e37eaacb0 con 0x563e37fa5f60 2023-07-14T18:21:43.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.903+0000 7f93ead28b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e37efda40 con 0x563e37fa74b0 2023-07-14T18:21:43.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.903+0000 7f93ead28b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e37ef6ed0 con 0x563e37fa68b0 2023-07-14T18:21:43.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.905+0000 7f93d599c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fa74b0 0x563e37fa7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:43.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.905+0000 7f93d599c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fa74b0 0x563e37fa7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:55718/0 (socket says 172.21.15.133:55718) 2023-07-14T18:21:43.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.905+0000 7f93d599c700 1 -- 172.21.15.133:0/3916086887 learned_addr learned my addr 172.21.15.133:0/3916086887 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:21:43.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.905+0000 7f93d499a700 1 --2- 172.21.15.133:0/3916086887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563e37fa68b0 0x563e37fa6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:43.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.906+0000 7f93d499a700 1 -- 172.21.15.133:0/3916086887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fa74b0 msgr2=0x563e37fa7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:21:43.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.906+0000 7f93d499a700 1 --2- 172.21.15.133:0/3916086887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fa74b0 0x563e37fa7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:43.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.906+0000 7f93d499a700 1 -- 172.21.15.133:0/3916086887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e37fa5f60 msgr2=0x563e37fa6370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T18:21:43.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.906+0000 7f93d499a700 1 --2- 172.21.15.133:0/3916086887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e37fa5f60 0x563e37fa6370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:43.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.906+0000 7f93d499a700 1 -- 172.21.15.133:0/3916086887 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e37d24de0 con 0x563e37fa68b0 2023-07-14T18:21:43.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.920+0000 7f93d499a700 1 --2- 172.21.15.133:0/3916086887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563e37fa68b0 0x563e37fa6ed0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f93c0009230 tx=0x7f93c000f4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=316c90e45787858c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:21:43.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.920+0000 7f93d499a700 1 --2- 172.21.15.133:0/2044965053 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f93b80a1350 0x7f93b80a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:43.978 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.926+0000 7f93d499a700 1 --2- 172.21.15.133:0/2044965053 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f93b80a1350 0x7f93b80a3800 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f93c0023000 tx=0x7f93c0028000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:21:43.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.928+0000 7f93b77fe700 1 -- 172.21.15.133:0/3916086887 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c001e070 con 0x563e37fa68b0 2023-07-14T18:21:43.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.928+0000 7f93b77fe700 1 -- 172.21.15.133:0/3916086887 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93c000bb60 con 0x563e37fa68b0 2023-07-14T18:21:43.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.928+0000 7f93b77fe700 1 -- 172.21.15.133:0/3916086887 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c0003bc0 con 0x563e37fa68b0 2023-07-14T18:21:43.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.929+0000 7f93ead28b80 1 -- 172.21.15.133:0/3916086887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563e37fa68b0 msgr2=0x563e37fa6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:21:43.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.929+0000 7f93ead28b80 1 --2- 172.21.15.133:0/3916086887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563e37fa68b0 0x563e37fa6ed0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f93c0009230 tx=0x7f93c000f4a0 comp rx=0 tx=0).stop 2023-07-14T18:21:43.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.929+0000 7f93ead28b80 1 -- 172.21.15.133:0/3916086887 shutdown_connections 2023-07-14T18:21:43.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.929+0000 7f93ead28b80 1 --2- 172.21.15.133:0/3916086887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fa74b0 0x563e37fa7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:43.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.930+0000 7f93ead28b80 1 --2- 172.21.15.133:0/3916086887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e37fa5f60 0x563e37fa6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:43.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.930+0000 7f93ead28b80 1 --2- 172.21.15.133:0/3916086887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563e37fa68b0 0x563e37fa6ed0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:43.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.930+0000 7f93ead28b80 1 -- 172.21.15.133:0/3916086887 >> 172.21.15.133:0/3916086887 conn(0x563e37f9b680 msgr2=0x563e37fa09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:21:43.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.930+0000 7f93ead28b80 1 -- 172.21.15.133:0/3916086887 shutdown_connections 2023-07-14T18:21:43.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.930+0000 7f93ead28b80 1 -- 172.21.15.133:0/3916086887 wait complete. 2023-07-14T18:21:43.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.931+0000 7f93ead28b80 1 -- start start 2023-07-14T18:21:43.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.931+0000 7f93ead28b80 1 Processor -- start 2023-07-14T18:21:43.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.932+0000 7f93ead28b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fa5f60 0x563e37faf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:43.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.932+0000 7f93ead28b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563e37fa68b0 0x563e37faff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:43.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.932+0000 7f93ead28b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e37fa74b0 0x563e37fb6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:43.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.932+0000 7f93ead28b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e37e96ac0 con 0x563e37fa74b0 2023-07-14T18:21:43.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.932+0000 7f93ead28b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e37e967c0 con 0x563e37fa5f60 2023-07-14T18:21:43.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.932+0000 7f93ead28b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e37d7a920 con 0x563e37fa68b0 2023-07-14T18:21:43.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.932+0000 7f93ead28b80 10 client.0 _ll_register_callbacks cb 0x563e37e94760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:21:43.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.933+0000 7f93d499a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563e37fa68b0 0x563e37faff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:43.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.933+0000 7f93d599c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e37fa74b0 0x563e37fb6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:43.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.933+0000 7f93d599c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e37fa74b0 0x563e37fb6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40722/0 (socket says 172.21.15.133:40722) 2023-07-14T18:21:43.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.933+0000 7f93d599c700 1 -- 172.21.15.133:0/1944964191 learned_addr learned my addr 172.21.15.133:0/1944964191 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:21:43.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.934+0000 7f93d599c700 1 -- 172.21.15.133:0/1944964191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563e37fa68b0 msgr2=0x563e37faff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:21:43.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.934+0000 7f93d599c700 1 --2- 172.21.15.133:0/1944964191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563e37fa68b0 0x563e37faff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:43.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.934+0000 7f93d599c700 1 -- 172.21.15.133:0/1944964191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fa5f60 msgr2=0x563e37faf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:21:43.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.934+0000 7f93d599c700 1 --2- 172.21.15.133:0/1944964191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fa5f60 0x563e37faf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:43.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.934+0000 7f93d599c700 1 -- 172.21.15.133:0/1944964191 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d0117770 con 0x563e37fa74b0 2023-07-14T18:21:43.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.935+0000 7f93d599c700 1 --2- 172.21.15.133:0/1944964191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e37fa74b0 0x563e37fb6f40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f93c800be50 tx=0x7f93c80997b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed3c2ff05ff1b280 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:21:43.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.940+0000 7f93b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T18:21:43.994 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.940+0000 7f93b4ff9700 1 -- 172.21.15.133:0/1944964191 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c80a1230 con 0x563e37fa74b0 2023-07-14T18:21:43.994 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.940+0000 7f93b4ff9700 1 -- 172.21.15.133:0/1944964191 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f93c80a1390 con 0x563e37fa74b0 2023-07-14T18:21:43.994 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.943+0000 7f93ead28b80 1 -- 172.21.15.133:0/1944964191 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e37e504f0 con 0x563e37fa74b0 2023-07-14T18:21:43.994 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.944+0000 7f93b4ff9700 1 -- 172.21.15.133:0/1944964191 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93c8095060 con 0x563e37fa74b0 2023-07-14T18:21:43.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.945+0000 7f93ead28b80 10 client.15717 fetch_fsmap learned FSMap version 34 2023-07-14T18:21:43.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.945+0000 7f93ead28b80 1 -- 172.21.15.133:0/1944964191 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563e37d0dc60 con 0x563e37fa74b0 2023-07-14T18:21:43.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.946+0000 7f93b4ff9700 1 -- 172.21.15.133:0/1944964191 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563e37e504f0 con 0x563e37fa74b0 2023-07-14T18:21:43.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.946+0000 7f93b4ff9700 1 -- 172.21.15.133:0/1944964191 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f93c80b72d0 con 0x563e37fa74b0 2023-07-14T18:21:43.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.946+0000 7f93ead28b80 10 client.15717 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T18:21:43.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.946+0000 7f93ead28b80 10 client.15717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:21:43.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.947+0000 7f93ead28b80 20 client.15717 populate_metadata read hostname 'smithi133' 2023-07-14T18:21:43.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.947+0000 7f93ead28b80 1 --2- 172.21.15.133:0/1944964191 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x563e37fbb800 0x563e37fbdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:43.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.947+0000 7f93ead28b80 4 client.15717 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:21:43.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.947+0000 7f93ead28b80 1 -- 172.21.15.133:0/1944964191 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e37f74da0 con 0x563e37fbb800 2023-07-14T18:21:43.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.949+0000 7f93d519b700 1 --2- 172.21.15.133:0/1944964191 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x563e37fbb800 0x563e37fbdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:44.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.958+0000 7f93d519b700 1 --2- 172.21.15.133:0/1944964191 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x563e37fbb800 0x563e37fbdc10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73d210aa7941cfd5 server_cookie=44dd08c283615cf5 in_seq=0 out_seq=0 2023-07-14T18:21:44.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.958+0000 7f93b4ff9700 10 client.15717 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:21:44.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.971+0000 7f93b4ff9700 1 -- 172.21.15.133:0/1944964191 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x563e37f74da0 con 0x563e37fbb800 2023-07-14T18:21:44.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.971+0000 7f93b4ff9700 10 client.15717 handle_command_reply: tid=0 2023-07-14T18:21:44.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.971+0000 7f93ead28b80 1 client.15717 shutdown 2023-07-14T18:21:44.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.973+0000 7f93ead28b80 1 -- 172.21.15.133:0/1944964191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e37fa74b0 msgr2=0x563e37fb6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:21:44.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.973+0000 7f93ead28b80 1 --2- 172.21.15.133:0/1944964191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e37fa74b0 0x563e37fb6f40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f93c800be50 tx=0x7f93c80997b0 comp rx=0 tx=0).stop 2023-07-14T18:21:44.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.974+0000 7f93ead28b80 1 -- 172.21.15.133:0/1944964191 shutdown_connections 2023-07-14T18:21:44.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.974+0000 7f93ead28b80 1 --2- 172.21.15.133:0/1944964191 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x563e37fbb800 0x563e37fbdc10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:44.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.974+0000 7f93ead28b80 1 --2- 172.21.15.133:0/1944964191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fa5f60 0x563e37faf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:44.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.974+0000 7f93ead28b80 1 --2- 172.21.15.133:0/1944964191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e37fa74b0 0x563e37fb6f40 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:44.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.974+0000 7f93ead28b80 1 --2- 172.21.15.133:0/1944964191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563e37fa68b0 0x563e37faff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:44.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.974+0000 7f93ead28b80 1 -- 172.21.15.133:0/1944964191 >> 172.21.15.133:0/1944964191 conn(0x563e37f9b680 msgr2=0x563e37f9f650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:21:44.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.974+0000 7f93b4ff9700 0 client.15717 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:21:44.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.975+0000 7f93ead28b80 1 -- 172.21.15.133:0/1944964191 shutdown_connections 2023-07-14T18:21:44.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.975+0000 7f93ead28b80 1 -- 172.21.15.133:0/1944964191 wait complete. 2023-07-14T18:21:44.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:43.985+0000 7f93ead28b80 20 client.15717 trim_cache size 0 max 16384 2023-07-14T18:21:44.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.023+0000 7f93ead28b80 1 Processor -- start 2023-07-14T18:21:44.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.024+0000 7f93ead28b80 1 -- start start 2023-07-14T18:21:44.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.024+0000 7f93ead28b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563e37fa5f60 0x563e37fb7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:44.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.024+0000 7f93ead28b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e37fa68b0 0x563e37fb6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:44.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.024+0000 7f93ead28b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fa74b0 0x563e37fbd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:44.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.024+0000 7f93ead28b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e37fa7330 con 0x563e37fa68b0 2023-07-14T18:21:44.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.024+0000 7f93ead28b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e37fa1cc0 con 0x563e37fa74b0 2023-07-14T18:21:44.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.024+0000 7f93ead28b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e37fa2500 con 0x563e37fa5f60 2023-07-14T18:21:44.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.036+0000 7f93d599c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fa74b0 0x563e37fbd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:44.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.036+0000 7f93d599c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fa74b0 0x563e37fbd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:55736/0 (socket says 172.21.15.133:55736) 2023-07-14T18:21:44.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.036+0000 7f93d599c700 1 -- 172.21.15.133:0/1856021407 learned_addr learned my addr 172.21.15.133:0/1856021407 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:21:44.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.039+0000 7f93d519b700 1 --2- 172.21.15.133:0/1856021407 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563e37fa5f60 0x563e37fb7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:44.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.039+0000 7f93d499a700 1 --2- 172.21.15.133:0/1856021407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e37fa68b0 0x563e37fb6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:44.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.042+0000 7f93d499a700 1 -- 172.21.15.133:0/1856021407 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563e37fa5f60 msgr2=0x563e37fb7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:21:44.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.042+0000 7f93d499a700 1 --2- 172.21.15.133:0/1856021407 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563e37fa5f60 0x563e37fb7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:44.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.042+0000 7f93d499a700 1 -- 172.21.15.133:0/1856021407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fa74b0 msgr2=0x563e37fbd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:21:44.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.042+0000 7f93d499a700 1 --2- 172.21.15.133:0/1856021407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fa74b0 0x563e37fbd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:44.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.042+0000 7f93d499a700 1 -- 172.21.15.133:0/1856021407 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e37d4b250 con 0x563e37fa68b0 2023-07-14T18:21:44.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.050+0000 7f93d499a700 1 --2- 172.21.15.133:0/1856021407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e37fa68b0 0x563e37fb6f00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f93c000fad0 tx=0x7f93c0003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26f663ef58943f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:21:44.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.055+0000 7f93b7fff700 1 -- 172.21.15.133:0/1856021407 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c001e040 con 0x563e37fa68b0 2023-07-14T18:21:44.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.055+0000 7f93b7fff700 1 -- 172.21.15.133:0/1856021407 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93c0003bb0 con 0x563e37fa68b0 2023-07-14T18:21:44.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.055+0000 7f93b7fff700 1 -- 172.21.15.133:0/1856021407 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c0027680 con 0x563e37fa68b0 2023-07-14T18:21:44.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.062+0000 7f93ead28b80 1 -- 172.21.15.133:0/1856021407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e37fa68b0 msgr2=0x563e37fb6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:21:44.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.063+0000 7f93ead28b80 1 --2- 172.21.15.133:0/1856021407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e37fa68b0 0x563e37fb6f00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f93c000fad0 tx=0x7f93c0003680 comp rx=0 tx=0).stop 2023-07-14T18:21:44.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.064+0000 7f93ead28b80 1 -- 172.21.15.133:0/1856021407 shutdown_connections 2023-07-14T18:21:44.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.064+0000 7f93ead28b80 1 --2- 172.21.15.133:0/1856021407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fa74b0 0x563e37fbd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:44.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.064+0000 7f93ead28b80 1 --2- 172.21.15.133:0/1856021407 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563e37fa5f60 0x563e37fb7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:44.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.064+0000 7f93ead28b80 1 --2- 172.21.15.133:0/1856021407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e37fa68b0 0x563e37fb6f00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:44.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.064+0000 7f93ead28b80 1 -- 172.21.15.133:0/1856021407 >> 172.21.15.133:0/1856021407 conn(0x563e37f9b680 msgr2=0x563e37fa00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:21:44.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.068+0000 7f93ead28b80 1 -- 172.21.15.133:0/1856021407 shutdown_connections 2023-07-14T18:21:44.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.069+0000 7f93ead28b80 1 -- 172.21.15.133:0/1856021407 wait complete. 2023-07-14T18:21:44.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.070+0000 7f93ead28b80 1 -- start start 2023-07-14T18:21:44.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.070+0000 7f93ead28b80 1 Processor -- start 2023-07-14T18:21:44.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.074+0000 7f93ead28b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fa5f60 0x563e37fb6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:44.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.074+0000 7f93ead28b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e37fa68b0 0x563e37fb6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:44.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.074+0000 7f93ead28b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563e37fa74b0 0x563e37fb31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:44.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.074+0000 7f93ead28b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e37fb3730 con 0x563e37fa68b0 2023-07-14T18:21:44.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.074+0000 7f93ead28b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e37fb38a0 con 0x563e37fa5f60 2023-07-14T18:21:44.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.074+0000 7f93ead28b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e37fb3a60 con 0x563e37fa74b0 2023-07-14T18:21:44.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.074+0000 7f93ead28b80 10 client.0 _ll_register_callbacks cb 0x7f93c8099270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:21:44.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.079+0000 7f93d519b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fa5f60 0x563e37fb6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:44.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.080+0000 7f93d599c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563e37fa74b0 0x563e37fb31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:44.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.080+0000 7f93d519b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fa5f60 0x563e37fb6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:55750/0 (socket says 172.21.15.133:55750) 2023-07-14T18:21:44.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.080+0000 7f93d519b700 1 -- 172.21.15.133:0/1415757104 learned_addr learned my addr 172.21.15.133:0/1415757104 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:21:44.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.081+0000 7f93d499a700 1 --2- 172.21.15.133:0/1415757104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e37fa68b0 0x563e37fb6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:44.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.084+0000 7f93d499a700 1 -- 172.21.15.133:0/1415757104 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563e37fa74b0 msgr2=0x563e37fb31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:21:44.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.084+0000 7f93d499a700 1 --2- 172.21.15.133:0/1415757104 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563e37fa74b0 0x563e37fb31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:44.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.084+0000 7f93d499a700 1 -- 172.21.15.133:0/1415757104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fa5f60 msgr2=0x563e37fb6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:21:44.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.084+0000 7f93d499a700 1 --2- 172.21.15.133:0/1415757104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fa5f60 0x563e37fb6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:44.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.084+0000 7f93d499a700 1 -- 172.21.15.133:0/1415757104 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e37d24de0 con 0x563e37fa68b0 2023-07-14T18:21:44.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.085+0000 7f93d499a700 1 --2- 172.21.15.133:0/1415757104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e37fa68b0 0x563e37fb6c50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f93c0000c00 tx=0x7f93c000ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4435092edb2f787 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:21:44.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.086+0000 7f93b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T18:21:44.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.090+0000 7f93b4ff9700 1 -- 172.21.15.133:0/1415757104 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c001e030 con 0x563e37fa68b0 2023-07-14T18:21:44.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.091+0000 7f93b4ff9700 1 -- 172.21.15.133:0/1415757104 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f93c00039c0 con 0x563e37fa68b0 2023-07-14T18:21:44.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.091+0000 7f93ead28b80 1 -- 172.21.15.133:0/1415757104 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93c000bb60 con 0x563e37fa68b0 2023-07-14T18:21:44.151 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.102+0000 7f93b4ff9700 1 -- 172.21.15.133:0/1415757104 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93c003d070 con 0x563e37fa68b0 2023-07-14T18:21:44.155 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.105+0000 7f93ead28b80 10 client.15723 fetch_fsmap learned FSMap version 34 2023-07-14T18:21:44.155 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.105+0000 7f93ead28b80 1 -- 172.21.15.133:0/1415757104 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563e37d232b0 con 0x563e37fa68b0 2023-07-14T18:21:44.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.113+0000 7f93b4ff9700 1 -- 172.21.15.133:0/1415757104 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93c000bb60 con 0x563e37fa68b0 2023-07-14T18:21:44.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.113+0000 7f93b4ff9700 1 -- 172.21.15.133:0/1415757104 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f93c0036020 con 0x563e37fa68b0 2023-07-14T18:21:44.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.113+0000 7f93ead28b80 10 client.15723 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T18:21:44.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.113+0000 7f93ead28b80 10 client.15723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:21:44.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.113+0000 7f93ead28b80 20 client.15723 populate_metadata read hostname 'smithi133' 2023-07-14T18:21:44.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.113+0000 7f93ead28b80 1 --2- 172.21.15.133:0/1415757104 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x563e380103e0 0x563e380127f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:21:44.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.113+0000 7f93ead28b80 4 client.15723 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:21:44.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.113+0000 7f93ead28b80 1 -- 172.21.15.133:0/1415757104 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563e37eefeb0 con 0x563e380103e0 2023-07-14T18:21:44.176 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.119+0000 7f93d519b700 1 --2- 172.21.15.133:0/1415757104 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x563e380103e0 0x563e380127f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:21:44.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.139+0000 7f93d519b700 1 --2- 172.21.15.133:0/1415757104 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x563e380103e0 0x563e380127f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3731dda47913c84 server_cookie=49c08c21d95f3a14 in_seq=0 out_seq=0 2023-07-14T18:21:44.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.139+0000 7f93b4ff9700 10 client.15723 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:21:44.201 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.149+0000 7f93b4ff9700 1 -- 172.21.15.133:0/1415757104 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x563e37eefeb0 con 0x563e380103e0 2023-07-14T18:21:44.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.149+0000 7f93b4ff9700 10 client.15723 handle_command_reply: tid=0 2023-07-14T18:21:44.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.149+0000 7f93ead28b80 1 client.15723 shutdown 2023-07-14T18:21:44.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.153+0000 7f93ead28b80 1 -- 172.21.15.133:0/1415757104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e37fa68b0 msgr2=0x563e37fb6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:21:44.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.153+0000 7f93ead28b80 1 --2- 172.21.15.133:0/1415757104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e37fa68b0 0x563e37fb6c50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f93c0000c00 tx=0x7f93c000ba60 comp rx=0 tx=0).stop 2023-07-14T18:21:44.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.158+0000 7f93ead28b80 1 -- 172.21.15.133:0/1415757104 shutdown_connections 2023-07-14T18:21:44.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.158+0000 7f93ead28b80 1 --2- 172.21.15.133:0/1415757104 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x563e380103e0 0x563e380127f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:44.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.158+0000 7f93ead28b80 1 --2- 172.21.15.133:0/1415757104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fa5f60 0x563e37fb6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:44.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.158+0000 7f93ead28b80 1 --2- 172.21.15.133:0/1415757104 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563e37fa74b0 0x563e37fb31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:44.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.158+0000 7f93ead28b80 1 --2- 172.21.15.133:0/1415757104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e37fa68b0 0x563e37fb6c50 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:44.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.158+0000 7f93ead28b80 1 -- 172.21.15.133:0/1415757104 >> 172.21.15.133:0/1415757104 conn(0x563e37f9b680 msgr2=0x563e37fbd810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:21:44.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.159+0000 7f93b4ff9700 0 client.15723 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:21:44.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.170+0000 7f93ead28b80 1 -- 172.21.15.133:0/1415757104 shutdown_connections 2023-07-14T18:21:44.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.175+0000 7f93ead28b80 1 -- 172.21.15.133:0/1415757104 wait complete. 2023-07-14T18:21:44.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.178+0000 7f93ead28b80 20 client.15723 trim_cache size 0 max 16384 2023-07-14T18:21:44.233 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:21:44.233 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-14T18:21:44.234 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:21:44.234 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:21:44.234 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:21:44.234 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:21:44.234 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:21:44.234 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:21:44.234 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:21:44.234 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:21:44.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.190+0000 7f93d73ff700 1 -- 172.21.15.133:0/2044965053 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f93b80a1350 msgr2=0x7f93b80a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:21:44.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.190+0000 7f93d73ff700 1 --2- 172.21.15.133:0/2044965053 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f93b80a1350 0x7f93b80a3800 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f93c0023000 tx=0x7f93c0028000 comp rx=0 tx=0).stop 2023-07-14T18:21:44.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.190+0000 7f93d73ff700 1 -- 172.21.15.133:0/2044965053 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93d010be90 msgr2=0x7f93d01c53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:21:44.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.190+0000 7f93d73ff700 1 --2- 172.21.15.133:0/2044965053 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93d010be90 0x7f93d01c53f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f93c800b970 tx=0x7f93c800bd30 comp rx=0 tx=0).stop 2023-07-14T18:21:44.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.199+0000 7f93d73ff700 1 -- 172.21.15.133:0/2044965053 shutdown_connections 2023-07-14T18:21:44.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.199+0000 7f93d73ff700 1 --2- 172.21.15.133:0/2044965053 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f93b80a1350 0x7f93b80a3800 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:44.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.199+0000 7f93d73ff700 1 --2- 172.21.15.133:0/2044965053 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93d010be90 0x7f93d01c53f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:44.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.199+0000 7f93d73ff700 1 --2- 172.21.15.133:0/2044965053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93d0071010 0x7f93d01c08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:44.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.199+0000 7f93d73ff700 1 --2- 172.21.15.133:0/2044965053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d0071960 0x7f93d01c2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:21:44.251 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.199+0000 7f93d73ff700 1 -- 172.21.15.133:0/2044965053 >> 172.21.15.133:0/2044965053 conn(0x7f93d006cbc0 msgr2=0x7f93d006dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:21:44.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.202+0000 7f93d73ff700 1 -- 172.21.15.133:0/2044965053 shutdown_connections 2023-07-14T18:21:44.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:21:44.208+0000 7f93d73ff700 1 -- 172.21.15.133:0/2044965053 wait complete. 2023-07-14T18:21:44.315 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:21:45.606 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:45 smithi161 ceph-mon[128993]: pgmap v500: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 990 op/s 2023-07-14T18:21:45.621 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:45 smithi171 ceph-mon[129241]: pgmap v500: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 990 op/s 2023-07-14T18:21:45.662 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:45 smithi133 ceph-mon[126384]: pgmap v500: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 990 op/s 2023-07-14T18:21:45.882 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 140.0 s, 1059.2 tps, lat 67.604 ms stddev 177.624 2023-07-14T18:21:47.623 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:47 smithi171 ceph-mon[129241]: pgmap v501: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 22 MiB/s wr, 852 op/s 2023-07-14T18:21:47.623 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:47 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:21:47.666 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:47 smithi133 ceph-mon[126384]: pgmap v501: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 22 MiB/s wr, 852 op/s 2023-07-14T18:21:47.666 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:47 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:21:47.856 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:47 smithi161 ceph-mon[128993]: pgmap v501: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 22 MiB/s wr, 852 op/s 2023-07-14T18:21:47.857 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:47 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:21:49.670 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:49 smithi133 ceph-mon[126384]: pgmap v502: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 30 MiB/s wr, 1.69k op/s 2023-07-14T18:21:49.671 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:49 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:21:49.671 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:49 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:21:49.671 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:49 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:21:49.856 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:49 smithi161 ceph-mon[128993]: pgmap v502: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 30 MiB/s wr, 1.69k op/s 2023-07-14T18:21:49.856 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:49 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:21:49.856 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:49 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:21:49.857 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:49 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:21:49.871 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:49 smithi171 ceph-mon[129241]: pgmap v502: 129 pgs: 129 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 30 MiB/s wr, 1.69k op/s 2023-07-14T18:21:49.871 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:49 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:21:49.872 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:49 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:21:49.872 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:49 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:21:50.880 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 145.0 s, 393.0 tps, lat 327.345 ms stddev 561.391 2023-07-14T18:21:51.665 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:51 smithi133 ceph-mon[126384]: pgmap v503: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 35 MiB/s wr, 2.54k op/s 2023-07-14T18:21:51.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:51 smithi161 ceph-mon[128993]: pgmap v503: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 35 MiB/s wr, 2.54k op/s 2023-07-14T18:21:51.873 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:51 smithi171 ceph-mon[129241]: pgmap v503: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 35 MiB/s wr, 2.54k op/s 2023-07-14T18:21:53.662 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:53 smithi133 ceph-mon[126384]: pgmap v504: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 35 MiB/s wr, 2.70k op/s 2023-07-14T18:21:53.856 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:53 smithi161 ceph-mon[128993]: pgmap v504: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 35 MiB/s wr, 2.70k op/s 2023-07-14T18:21:53.871 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:53 smithi171 ceph-mon[129241]: pgmap v504: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 35 MiB/s wr, 2.70k op/s 2023-07-14T18:21:55.856 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:55 smithi161 ceph-mon[128993]: pgmap v505: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 MiB/s wr, 3.29k op/s 2023-07-14T18:21:55.871 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:55 smithi171 ceph-mon[129241]: pgmap v505: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 MiB/s wr, 3.29k op/s 2023-07-14T18:21:55.880 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 150.0 s, 1984.6 tps, lat 50.291 ms stddev 35.083 2023-07-14T18:21:55.913 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:55 smithi133 ceph-mon[126384]: pgmap v505: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 MiB/s wr, 3.29k op/s 2023-07-14T18:21:57.856 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:57 smithi161 ceph-mon[128993]: pgmap v506: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 MiB/s wr, 3.10k op/s 2023-07-14T18:21:57.871 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:57 smithi171 ceph-mon[129241]: pgmap v506: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 MiB/s wr, 3.10k op/s 2023-07-14T18:21:57.915 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:57 smithi133 ceph-mon[126384]: pgmap v506: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 MiB/s wr, 3.10k op/s 2023-07-14T18:21:59.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:21:59 smithi161 ceph-mon[128993]: pgmap v507: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 MiB/s wr, 3.19k op/s 2023-07-14T18:21:59.878 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:21:59 smithi171 ceph-mon[129241]: pgmap v507: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 MiB/s wr, 3.19k op/s 2023-07-14T18:21:59.912 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:21:59 smithi133 ceph-mon[126384]: pgmap v507: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 MiB/s wr, 3.19k op/s 2023-07-14T18:22:00.881 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 155.0 s, 1150.3 tps, lat 78.566 ms stddev 156.470 2023-07-14T18:22:01.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:01 smithi161 ceph-mon[128993]: pgmap v508: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 MiB/s wr, 2.50k op/s 2023-07-14T18:22:01.621 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:01 smithi171 ceph-mon[129241]: pgmap v508: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 MiB/s wr, 2.50k op/s 2023-07-14T18:22:01.666 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:01 smithi133 ceph-mon[126384]: pgmap v508: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 MiB/s wr, 2.50k op/s 2023-07-14T18:22:03.606 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:03 smithi161 ceph-mon[128993]: pgmap v509: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 MiB/s wr, 1.45k op/s 2023-07-14T18:22:03.623 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:03 smithi171 ceph-mon[129241]: pgmap v509: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 MiB/s wr, 1.45k op/s 2023-07-14T18:22:03.666 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:03 smithi133 ceph-mon[126384]: pgmap v509: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 MiB/s wr, 1.45k op/s 2023-07-14T18:22:05.666 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:05 smithi133 ceph-mon[126384]: pgmap v510: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 MiB/s wr, 1.37k op/s 2023-07-14T18:22:05.856 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:05 smithi161 ceph-mon[128993]: pgmap v510: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 MiB/s wr, 1.37k op/s 2023-07-14T18:22:05.871 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:05 smithi171 ceph-mon[129241]: pgmap v510: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 MiB/s wr, 1.37k op/s 2023-07-14T18:22:05.880 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 160.0 s, 1047.5 tps, lat 104.538 ms stddev 125.566 2023-07-14T18:22:07.856 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:07 smithi161 ceph-mon[128993]: pgmap v511: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 21 MiB/s wr, 665 op/s 2023-07-14T18:22:07.879 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:07 smithi171 ceph-mon[129241]: pgmap v511: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 21 MiB/s wr, 665 op/s 2023-07-14T18:22:07.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:07 smithi133 ceph-mon[126384]: pgmap v511: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 21 MiB/s wr, 665 op/s 2023-07-14T18:22:09.857 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:09 smithi161 ceph-mon[128993]: pgmap v512: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 779 op/s 2023-07-14T18:22:09.872 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:09 smithi171 ceph-mon[129241]: pgmap v512: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 779 op/s 2023-07-14T18:22:09.913 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:09 smithi133 ceph-mon[126384]: pgmap v512: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 779 op/s 2023-07-14T18:22:10.881 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 165.0 s, 1439.3 tps, lat 61.852 ms stddev 51.335 2023-07-14T18:22:11.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:11 smithi161 ceph-mon[128993]: pgmap v513: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 987 op/s 2023-07-14T18:22:11.622 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:11 smithi171 ceph-mon[129241]: pgmap v513: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 987 op/s 2023-07-14T18:22:12.802 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:11 smithi133 ceph-mon[126384]: pgmap v513: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 987 op/s 2023-07-14T18:22:13.621 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:13 smithi171 ceph-mon[129241]: pgmap v514: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 22 MiB/s wr, 1.09k op/s 2023-07-14T18:22:13.662 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:13 smithi133 ceph-mon[126384]: pgmap v514: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 22 MiB/s wr, 1.09k op/s 2023-07-14T18:22:13.856 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:13 smithi161 ceph-mon[128993]: pgmap v514: 129 pgs: 129 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 22 MiB/s wr, 1.09k op/s 2023-07-14T18:22:14.316 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:22:15.671 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:15 smithi133 ceph-mon[126384]: pgmap v515: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 38 MiB/s wr, 2.48k op/s 2023-07-14T18:22:15.856 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:15 smithi161 ceph-mon[128993]: pgmap v515: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 38 MiB/s wr, 2.48k op/s 2023-07-14T18:22:15.873 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:15 smithi171 ceph-mon[129241]: pgmap v515: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 38 MiB/s wr, 2.48k op/s 2023-07-14T18:22:15.882 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 170.0 s, 97.0 tps, lat 1019.869 ms stddev 746.368 2023-07-14T18:22:17.696 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:17 smithi133 ceph-mon[126384]: pgmap v516: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 32 MiB/s wr, 2.15k op/s 2023-07-14T18:22:17.856 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:17 smithi161 ceph-mon[128993]: pgmap v516: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 32 MiB/s wr, 2.15k op/s 2023-07-14T18:22:17.872 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:17 smithi171 ceph-mon[129241]: pgmap v516: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 32 MiB/s wr, 2.15k op/s 2023-07-14T18:22:18.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.320+0000 7fd7f79ff700 1 -- 172.21.15.133:0/2842595961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7f0101e40 msgr2=0x7fd7f0102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:22:18.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.320+0000 7fd7f79ff700 1 --2- 172.21.15.133:0/2842595961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7f0101e40 0x7fd7f0102250 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800b0a0 tx=0x7fd7e800b3b0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.321+0000 7fd7f79ff700 1 -- 172.21.15.133:0/2842595961 shutdown_connections 2023-07-14T18:22:18.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.321+0000 7fd7f79ff700 1 --2- 172.21.15.133:0/2842595961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7f0101e40 0x7fd7f0102250 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.372 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.321+0000 7fd7f79ff700 1 --2- 172.21.15.133:0/2842595961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7f0103940 0x7fd7f010a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.372 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.321+0000 7fd7f79ff700 1 --2- 172.21.15.133:0/2842595961 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7f0102f90 0x7fd7f0103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.372 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.321+0000 7fd7f79ff700 1 -- 172.21.15.133:0/2842595961 >> 172.21.15.133:0/2842595961 conn(0x7fd7f00fd480 msgr2=0x7fd7f00ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:22:18.372 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.321+0000 7fd7f79ff700 1 -- 172.21.15.133:0/2842595961 shutdown_connections 2023-07-14T18:22:18.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.321+0000 7fd7f79ff700 1 -- 172.21.15.133:0/2842595961 wait complete. 2023-07-14T18:22:18.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.322+0000 7fd7f79ff700 1 Processor -- start 2023-07-14T18:22:18.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.323+0000 7fd7f79ff700 1 -- start start 2023-07-14T18:22:18.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.323+0000 7fd7f79ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7f0101e40 0x7fd7f0197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:18.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.323+0000 7fd7f79ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7f0102f90 0x7fd7f01981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:18.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.323+0000 7fd7f79ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7f0103940 0x7fd7f019f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:18.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.323+0000 7fd7f79ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f019f7b0 con 0x7fd7f0101e40 2023-07-14T18:22:18.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.323+0000 7fd7f79ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f019f920 con 0x7fd7f0103940 2023-07-14T18:22:18.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.323+0000 7fd7f79ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f019fae0 con 0x7fd7f0102f90 2023-07-14T18:22:18.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.324+0000 7fd7f5f9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7f0103940 0x7fd7f019f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:18.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.324+0000 7fd7f5f9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7f0103940 0x7fd7f019f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:35534/0 (socket says 172.21.15.133:35534) 2023-07-14T18:22:18.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.324+0000 7fd7f5f9c700 1 -- 172.21.15.133:0/1010379171 learned_addr learned my addr 172.21.15.133:0/1010379171 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:22:18.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.324+0000 7fd7f4f9a700 1 --2- 172.21.15.133:0/1010379171 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7f0102f90 0x7fd7f01981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:18.377 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.326+0000 7fd7f4f9a700 1 -- 172.21.15.133:0/1010379171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7f0103940 msgr2=0x7fd7f019f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:22:18.377 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.326+0000 7fd7f4f9a700 1 --2- 172.21.15.133:0/1010379171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7f0103940 0x7fd7f019f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.377 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.326+0000 7fd7f4f9a700 1 -- 172.21.15.133:0/1010379171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7f0101e40 msgr2=0x7fd7f0197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:22:18.377 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.326+0000 7fd7f4f9a700 1 --2- 172.21.15.133:0/1010379171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7f0101e40 0x7fd7f0197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.377 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.326+0000 7fd7f4f9a700 1 -- 172.21.15.133:0/1010379171 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e8009d00 con 0x7fd7f0102f90 2023-07-14T18:22:18.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.327+0000 7fd7f4f9a700 1 --2- 172.21.15.133:0/1010379171 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7f0102f90 0x7fd7f01981b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00c970 tx=0x7fd7ec00cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:22:18.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.328+0000 7fd7e67fc700 1 -- 172.21.15.133:0/1010379171 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec007b00 con 0x7fd7f0102f90 2023-07-14T18:22:18.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.328+0000 7fd7e67fc700 1 -- 172.21.15.133:0/1010379171 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7ec00ce60 con 0x7fd7f0102f90 2023-07-14T18:22:18.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.328+0000 7fd7e67fc700 1 -- 172.21.15.133:0/1010379171 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec00e610 con 0x7fd7f0102f90 2023-07-14T18:22:18.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.328+0000 7fd7f79ff700 1 -- 172.21.15.133:0/1010379171 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f019fdc0 con 0x7fd7f0102f90 2023-07-14T18:22:18.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.329+0000 7fd7f79ff700 1 -- 172.21.15.133:0/1010379171 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f0074a80 con 0x7fd7f0102f90 2023-07-14T18:22:18.385 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.335+0000 7fd7e67fc700 1 -- 172.21.15.133:0/1010379171 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fd7ec007c60 con 0x7fd7f0102f90 2023-07-14T18:22:18.386 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.336+0000 7fd80b343b80 1 Processor -- start 2023-07-14T18:22:18.387 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.337+0000 7fd80b343b80 1 -- start start 2023-07-14T18:22:18.387 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.337+0000 7fd80b343b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55627b8b5f60 0x55627b8b6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:18.388 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.337+0000 7fd80b343b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55627b8b68b0 0x55627b8b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:18.388 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.337+0000 7fd80b343b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55627b8b74b0 0x55627b8b7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:18.388 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.337+0000 7fd80b343b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55627b7bacb0 con 0x55627b8b68b0 2023-07-14T18:22:18.388 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.337+0000 7fd80b343b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55627b80da40 con 0x55627b8b74b0 2023-07-14T18:22:18.388 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.337+0000 7fd80b343b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55627b806ed0 con 0x55627b8b5f60 2023-07-14T18:22:18.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.337+0000 7fd7e67fc700 1 --2- 172.21.15.133:0/1010379171 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd7dc0a1350 0x7fd7dc0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:18.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.338+0000 7fd7f4f9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55627b8b74b0 0x55627b8b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:18.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.338+0000 7fd7f4f9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55627b8b74b0 0x55627b8b7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:35544/0 (socket says 172.21.15.133:35544) 2023-07-14T18:22:18.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.338+0000 7fd7f4f9a700 1 -- 172.21.15.133:0/2561248987 learned_addr learned my addr 172.21.15.133:0/2561248987 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:22:18.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.338+0000 7fd7f5f9c700 1 --2- 172.21.15.133:0/2561248987 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55627b8b5f60 0x55627b8b6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:18.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.338+0000 7fd7f579b700 1 --2- 172.21.15.133:0/2561248987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55627b8b68b0 0x55627b8b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:18.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.338+0000 7fd7e67fc700 1 -- 172.21.15.133:0/1010379171 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd7ec0cca10 con 0x7fd7f0102f90 2023-07-14T18:22:18.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.339+0000 7fd7f579b700 1 -- 172.21.15.133:0/2561248987 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55627b8b5f60 msgr2=0x55627b8b6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:22:18.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.339+0000 7fd7f579b700 1 --2- 172.21.15.133:0/2561248987 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55627b8b5f60 0x55627b8b6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.339+0000 7fd7f579b700 1 -- 172.21.15.133:0/2561248987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55627b8b74b0 msgr2=0x55627b8b7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:22:18.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.339+0000 7fd7f579b700 1 --2- 172.21.15.133:0/2561248987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55627b8b74b0 0x55627b8b7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.339+0000 7fd7f579b700 1 -- 172.21.15.133:0/2561248987 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55627b634de0 con 0x55627b8b68b0 2023-07-14T18:22:18.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.339+0000 7fd7f5f9c700 1 --2- 172.21.15.133:0/1010379171 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd7dc0a1350 0x7fd7dc0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:18.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.340+0000 7fd7f579b700 1 --2- 172.21.15.133:0/2561248987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55627b8b68b0 0x55627b8b6ed0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800b070 tx=0x7fd7e8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e854ad8e7ef267 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:22:18.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.341+0000 7fd7db7fe700 1 -- 172.21.15.133:0/2561248987 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e800e040 con 0x55627b8b68b0 2023-07-14T18:22:18.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.341+0000 7fd7db7fe700 1 -- 172.21.15.133:0/2561248987 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7e80076e0 con 0x55627b8b68b0 2023-07-14T18:22:18.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.341+0000 7fd7db7fe700 1 -- 172.21.15.133:0/2561248987 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e801d6e0 con 0x55627b8b68b0 2023-07-14T18:22:18.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.342+0000 7fd80b343b80 1 -- 172.21.15.133:0/2561248987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55627b8b68b0 msgr2=0x55627b8b6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:22:18.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.342+0000 7fd80b343b80 1 --2- 172.21.15.133:0/2561248987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55627b8b68b0 0x55627b8b6ed0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800b070 tx=0x7fd7e8009c20 comp rx=0 tx=0).stop 2023-07-14T18:22:18.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.342+0000 7fd7f5f9c700 1 --2- 172.21.15.133:0/1010379171 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd7dc0a1350 0x7fd7dc0a3800 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd7e00124b0 tx=0x7fd7e000b400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:22:18.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.342+0000 7fd80b343b80 1 -- 172.21.15.133:0/2561248987 shutdown_connections 2023-07-14T18:22:18.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.342+0000 7fd80b343b80 1 --2- 172.21.15.133:0/2561248987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55627b8b74b0 0x55627b8b7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.342+0000 7fd80b343b80 1 --2- 172.21.15.133:0/2561248987 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55627b8b5f60 0x55627b8b6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.342+0000 7fd80b343b80 1 --2- 172.21.15.133:0/2561248987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55627b8b68b0 0x55627b8b6ed0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.342+0000 7fd80b343b80 1 -- 172.21.15.133:0/2561248987 >> 172.21.15.133:0/2561248987 conn(0x55627b8ab680 msgr2=0x55627b8b09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:22:18.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.343+0000 7fd80b343b80 1 -- 172.21.15.133:0/2561248987 shutdown_connections 2023-07-14T18:22:18.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.343+0000 7fd80b343b80 1 -- 172.21.15.133:0/2561248987 wait complete. 2023-07-14T18:22:18.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.344+0000 7fd80b343b80 1 -- start start 2023-07-14T18:22:18.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.344+0000 7fd80b343b80 1 Processor -- start 2023-07-14T18:22:18.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.345+0000 7fd80b343b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55627b8b5f60 0x55627b8d0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:18.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.345+0000 7fd80b343b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55627b8b68b0 0x55627b8d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:18.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.345+0000 7fd80b343b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55627b8b74b0 0x55627b8d7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:18.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.345+0000 7fd80b343b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55627b7a6ac0 con 0x55627b8b74b0 2023-07-14T18:22:18.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.345+0000 7fd80b343b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55627b7a67c0 con 0x55627b8b5f60 2023-07-14T18:22:18.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.345+0000 7fd80b343b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55627b68a920 con 0x55627b8b68b0 2023-07-14T18:22:18.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.345+0000 7fd80b343b80 10 client.0 _ll_register_callbacks cb 0x55627b7a4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:22:18.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.346+0000 7fd7f4f9a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55627b8b68b0 0x55627b8d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:18.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.346+0000 7fd7f4f9a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55627b8b68b0 0x55627b8d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:47918/0 (socket says 172.21.15.133:47918) 2023-07-14T18:22:18.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.346+0000 7fd7f4f9a700 1 -- 172.21.15.133:0/4149160659 learned_addr learned my addr 172.21.15.133:0/4149160659 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:22:18.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.346+0000 7fd7f5f9c700 1 --2- 172.21.15.133:0/4149160659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55627b8b74b0 0x55627b8d7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:18.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.346+0000 7fd7f579b700 1 --2- 172.21.15.133:0/4149160659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55627b8b5f60 0x55627b8d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:18.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.347+0000 7fd7f4f9a700 1 -- 172.21.15.133:0/4149160659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55627b8b5f60 msgr2=0x55627b8d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:22:18.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.347+0000 7fd7f4f9a700 1 --2- 172.21.15.133:0/4149160659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55627b8b5f60 0x55627b8d0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.347+0000 7fd7f4f9a700 1 -- 172.21.15.133:0/4149160659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55627b8b74b0 msgr2=0x55627b8d7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:22:18.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.347+0000 7fd7f4f9a700 1 --2- 172.21.15.133:0/4149160659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55627b8b74b0 0x55627b8d7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.347+0000 7fd7f4f9a700 1 -- 172.21.15.133:0/4149160659 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f010ab70 con 0x55627b8b68b0 2023-07-14T18:22:18.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.347+0000 7fd7f4f9a700 1 --2- 172.21.15.133:0/4149160659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55627b8b68b0 0x55627b8d0ea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0a19b0 tx=0x7fd7ec00fe60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7279bf1463fa7718 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:22:18.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.347+0000 7fd7d8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:22:18.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.348+0000 7fd7d8ff9700 1 -- 172.21.15.133:0/4149160659 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec0cfea0 con 0x55627b8b68b0 2023-07-14T18:22:18.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.348+0000 7fd7d8ff9700 1 -- 172.21.15.133:0/4149160659 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd7ec09dac0 con 0x55627b8b68b0 2023-07-14T18:22:18.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.348+0000 7fd80b343b80 1 -- 172.21.15.133:0/4149160659 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55627b7604f0 con 0x55627b8b68b0 2023-07-14T18:22:18.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.349+0000 7fd7d8ff9700 1 -- 172.21.15.133:0/4149160659 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7ec0a1ee0 con 0x55627b8b68b0 2023-07-14T18:22:18.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.349+0000 7fd80b343b80 10 client.25786 fetch_fsmap learned FSMap version 34 2023-07-14T18:22:18.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.349+0000 7fd80b343b80 1 -- 172.21.15.133:0/4149160659 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55627b61dc60 con 0x55627b8b68b0 2023-07-14T18:22:18.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.349+0000 7fd7d8ff9700 1 -- 172.21.15.133:0/4149160659 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55627b7604f0 con 0x55627b8b68b0 2023-07-14T18:22:18.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.349+0000 7fd7d8ff9700 1 -- 172.21.15.133:0/4149160659 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd7ec0989f0 con 0x55627b8b68b0 2023-07-14T18:22:18.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.349+0000 7fd80b343b80 10 client.25786 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T18:22:18.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.349+0000 7fd80b343b80 10 client.25786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:22:18.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.349+0000 7fd80b343b80 20 client.25786 populate_metadata read hostname 'smithi133' 2023-07-14T18:22:18.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.349+0000 7fd80b343b80 1 --2- 172.21.15.133:0/4149160659 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55627b8dc710 0x55627b8deb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:18.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.349+0000 7fd80b343b80 4 client.25786 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:22:18.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.349+0000 7fd80b343b80 1 -- 172.21.15.133:0/4149160659 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55627b884da0 con 0x55627b8dc710 2023-07-14T18:22:18.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.350+0000 7fd7f579b700 1 --2- 172.21.15.133:0/4149160659 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55627b8dc710 0x55627b8deb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:18.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.350+0000 7fd7f579b700 1 --2- 172.21.15.133:0/4149160659 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55627b8dc710 0x55627b8deb20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f6edf0b7231165f server_cookie=a05e70c02f018930 in_seq=0 out_seq=0 2023-07-14T18:22:18.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.350+0000 7fd7d8ff9700 10 client.25786 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:22:18.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.351+0000 7fd7d8ff9700 1 -- 172.21.15.133:0/4149160659 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55627b884da0 con 0x55627b8dc710 2023-07-14T18:22:18.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.351+0000 7fd7d8ff9700 10 client.25786 handle_command_reply: tid=0 2023-07-14T18:22:18.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.351+0000 7fd80b343b80 1 client.25786 shutdown 2023-07-14T18:22:18.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.351+0000 7fd80b343b80 1 -- 172.21.15.133:0/4149160659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55627b8b68b0 msgr2=0x55627b8d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:22:18.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.351+0000 7fd80b343b80 1 --2- 172.21.15.133:0/4149160659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55627b8b68b0 0x55627b8d0ea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0a19b0 tx=0x7fd7ec00fe60 comp rx=0 tx=0).stop 2023-07-14T18:22:18.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.351+0000 7fd80b343b80 1 -- 172.21.15.133:0/4149160659 shutdown_connections 2023-07-14T18:22:18.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.351+0000 7fd80b343b80 1 --2- 172.21.15.133:0/4149160659 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55627b8dc710 0x55627b8deb20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.351+0000 7fd80b343b80 1 --2- 172.21.15.133:0/4149160659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55627b8b5f60 0x55627b8d0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.351+0000 7fd80b343b80 1 --2- 172.21.15.133:0/4149160659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55627b8b74b0 0x55627b8d7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.351+0000 7fd80b343b80 1 --2- 172.21.15.133:0/4149160659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55627b8b68b0 0x55627b8d0ea0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.351+0000 7fd80b343b80 1 -- 172.21.15.133:0/4149160659 >> 172.21.15.133:0/4149160659 conn(0x55627b8ab680 msgr2=0x55627b8af650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:22:18.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.351+0000 7fd7d8ff9700 0 client.25786 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:22:18.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.351+0000 7fd80b343b80 1 -- 172.21.15.133:0/4149160659 shutdown_connections 2023-07-14T18:22:18.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.352+0000 7fd80b343b80 1 -- 172.21.15.133:0/4149160659 wait complete. 2023-07-14T18:22:18.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.352+0000 7fd80b343b80 20 client.25786 trim_cache size 0 max 16384 2023-07-14T18:22:18.431 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.381+0000 7fd80b343b80 1 Processor -- start 2023-07-14T18:22:18.431 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.381+0000 7fd80b343b80 1 -- start start 2023-07-14T18:22:18.432 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.381+0000 7fd80b343b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55627b8b5f60 0x55627b8bd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:18.432 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.381+0000 7fd80b343b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55627b8b68b0 0x55627b8d7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:18.433 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.381+0000 7fd80b343b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55627b8b74b0 0x55627b8bdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:18.433 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.381+0000 7fd80b343b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55627b8d1b50 con 0x55627b8b5f60 2023-07-14T18:22:18.433 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.381+0000 7fd80b343b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55627b8d04d0 con 0x55627b8b68b0 2023-07-14T18:22:18.433 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.381+0000 7fd80b343b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55627b8a89f0 con 0x55627b8b74b0 2023-07-14T18:22:18.433 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.382+0000 7fd7f5f9c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55627b8b74b0 0x55627b8bdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:18.433 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.382+0000 7fd7f5f9c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55627b8b74b0 0x55627b8bdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:47932/0 (socket says 172.21.15.133:47932) 2023-07-14T18:22:18.433 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.382+0000 7fd7f5f9c700 1 -- 172.21.15.133:0/2741013322 learned_addr learned my addr 172.21.15.133:0/2741013322 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:22:18.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.382+0000 7fd7f579b700 1 --2- 172.21.15.133:0/2741013322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55627b8b5f60 0x55627b8bd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:18.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.382+0000 7fd7f5f9c700 1 -- 172.21.15.133:0/2741013322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55627b8b68b0 msgr2=0x55627b8d7ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T18:22:18.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.382+0000 7fd7f5f9c700 1 --2- 172.21.15.133:0/2741013322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55627b8b68b0 0x55627b8d7ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.382+0000 7fd7f5f9c700 1 -- 172.21.15.133:0/2741013322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55627b8b5f60 msgr2=0x55627b8bd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:22:18.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.382+0000 7fd7f5f9c700 1 --2- 172.21.15.133:0/2741013322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55627b8b5f60 0x55627b8bd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.382+0000 7fd7f5f9c700 1 -- 172.21.15.133:0/2741013322 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55627b65b250 con 0x55627b8b74b0 2023-07-14T18:22:18.458 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.407+0000 7fd7f5f9c700 1 --2- 172.21.15.133:0/2741013322 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55627b8b74b0 0x55627b8bdd30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0010720 tx=0x7fd7e0010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e16bed0d6387bee server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:22:18.463 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.412+0000 7fd7dbfff700 1 -- 172.21.15.133:0/2741013322 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e0018070 con 0x55627b8b74b0 2023-07-14T18:22:18.463 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.412+0000 7fd7dbfff700 1 -- 172.21.15.133:0/2741013322 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7e0025070 con 0x55627b8b74b0 2023-07-14T18:22:18.463 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.412+0000 7fd7dbfff700 1 -- 172.21.15.133:0/2741013322 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e00217f0 con 0x55627b8b74b0 2023-07-14T18:22:18.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.413+0000 7fd80b343b80 1 -- 172.21.15.133:0/2741013322 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55627b8b74b0 msgr2=0x55627b8bdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:22:18.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.413+0000 7fd80b343b80 1 --2- 172.21.15.133:0/2741013322 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55627b8b74b0 0x55627b8bdd30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0010720 tx=0x7fd7e0010000 comp rx=0 tx=0).stop 2023-07-14T18:22:18.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.426+0000 7fd80b343b80 1 -- 172.21.15.133:0/2741013322 shutdown_connections 2023-07-14T18:22:18.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.426+0000 7fd80b343b80 1 --2- 172.21.15.133:0/2741013322 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55627b8b74b0 0x55627b8bdd30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.480 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.426+0000 7fd80b343b80 1 --2- 172.21.15.133:0/2741013322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55627b8b5f60 0x55627b8bd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.480 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.426+0000 7fd80b343b80 1 --2- 172.21.15.133:0/2741013322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55627b8b68b0 0x55627b8d7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.480 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.426+0000 7fd80b343b80 1 -- 172.21.15.133:0/2741013322 >> 172.21.15.133:0/2741013322 conn(0x55627b8ab680 msgr2=0x55627b8cf730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:22:18.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.434+0000 7fd80b343b80 1 -- 172.21.15.133:0/2741013322 shutdown_connections 2023-07-14T18:22:18.508 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.449+0000 7fd80b343b80 1 -- 172.21.15.133:0/2741013322 wait complete. 2023-07-14T18:22:18.508 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.449+0000 7fd80b343b80 1 -- start start 2023-07-14T18:22:18.508 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.451+0000 7fd80b343b80 1 Processor -- start 2023-07-14T18:22:18.509 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.453+0000 7fd80b343b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55627b8b5f60 0x55627b8d7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:18.509 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.453+0000 7fd80b343b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55627b8b68b0 0x55627b8d45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:18.509 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.453+0000 7fd80b343b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55627b8b74b0 0x55627b8d4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:18.509 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.453+0000 7fd80b343b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55627b8d5050 con 0x55627b8b5f60 2023-07-14T18:22:18.509 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.453+0000 7fd80b343b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55627b8d51c0 con 0x55627b8b74b0 2023-07-14T18:22:18.509 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.453+0000 7fd80b343b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55627b9413b0 con 0x55627b8b68b0 2023-07-14T18:22:18.509 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.453+0000 7fd80b343b80 10 client.0 _ll_register_callbacks cb 0x7fd7ec099950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:22:18.509 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.454+0000 7fd7f4f9a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55627b8b68b0 0x55627b8d45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:18.510 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.454+0000 7fd7f4f9a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55627b8b68b0 0x55627b8d45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:47940/0 (socket says 172.21.15.133:47940) 2023-07-14T18:22:18.510 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.454+0000 7fd7f4f9a700 1 -- 172.21.15.133:0/2097250851 learned_addr learned my addr 172.21.15.133:0/2097250851 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:22:18.510 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.455+0000 7fd7f4f9a700 1 -- 172.21.15.133:0/2097250851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55627b8b74b0 msgr2=0x55627b8d4b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T18:22:18.510 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.455+0000 7fd7f4f9a700 1 --2- 172.21.15.133:0/2097250851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55627b8b74b0 0x55627b8d4b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.510 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.455+0000 7fd7f4f9a700 1 -- 172.21.15.133:0/2097250851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55627b8b5f60 msgr2=0x55627b8d7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:22:18.510 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.455+0000 7fd7f4f9a700 1 --2- 172.21.15.133:0/2097250851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55627b8b5f60 0x55627b8d7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.510 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.455+0000 7fd7f4f9a700 1 -- 172.21.15.133:0/2097250851 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e0010460 con 0x55627b8b68b0 2023-07-14T18:22:18.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.455+0000 7fd7f4f9a700 1 --2- 172.21.15.133:0/2097250851 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55627b8b68b0 0x55627b8d45d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec095f90 tx=0x7fd7ec00c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7371a3210554c933 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:22:18.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.455+0000 7fd7d8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:22:18.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.457+0000 7fd7d8ff9700 1 -- 172.21.15.133:0/2097250851 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec0ccc20 con 0x55627b8b68b0 2023-07-14T18:22:18.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.457+0000 7fd7d8ff9700 1 -- 172.21.15.133:0/2097250851 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd7ec0d2140 con 0x55627b8b68b0 2023-07-14T18:22:18.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.457+0000 7fd80b343b80 1 -- 172.21.15.133:0/2097250851 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7e80076e0 con 0x55627b8b68b0 2023-07-14T18:22:18.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.458+0000 7fd7d8ff9700 1 -- 172.21.15.133:0/2097250851 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7ec09dac0 con 0x55627b8b68b0 2023-07-14T18:22:18.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.460+0000 7fd80b343b80 10 client.25792 fetch_fsmap learned FSMap version 34 2023-07-14T18:22:18.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.460+0000 7fd80b343b80 1 -- 172.21.15.133:0/2097250851 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55627b6332b0 con 0x55627b8b68b0 2023-07-14T18:22:18.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.460+0000 7fd7d8ff9700 1 -- 172.21.15.133:0/2097250851 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7e80076e0 con 0x55627b8b68b0 2023-07-14T18:22:18.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.462+0000 7fd7d8ff9700 1 -- 172.21.15.133:0/2097250851 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd7ec095880 con 0x55627b8b68b0 2023-07-14T18:22:18.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.462+0000 7fd80b343b80 10 client.25792 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T18:22:18.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.462+0000 7fd80b343b80 10 client.25792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:22:18.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.462+0000 7fd80b343b80 20 client.25792 populate_metadata read hostname 'smithi133' 2023-07-14T18:22:18.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.462+0000 7fd80b343b80 1 --2- 172.21.15.133:0/2097250851 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55627b92c840 0x55627b92ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:18.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.462+0000 7fd80b343b80 4 client.25792 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:22:18.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.462+0000 7fd80b343b80 1 -- 172.21.15.133:0/2097250851 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55627b7ffeb0 con 0x55627b92c840 2023-07-14T18:22:18.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.467+0000 7fd7f579b700 1 --2- 172.21.15.133:0/2097250851 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55627b92c840 0x55627b92ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:18.522 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.472+0000 7fd7f579b700 1 --2- 172.21.15.133:0/2097250851 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55627b92c840 0x55627b92ec20 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cc0aa96e6fcf3cb server_cookie=ed85d57aa4f12aaf in_seq=0 out_seq=0 2023-07-14T18:22:18.522 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.472+0000 7fd7d8ff9700 10 client.25792 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:22:18.522 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.472+0000 7fd7d8ff9700 1 -- 172.21.15.133:0/2097250851 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55627b7ffeb0 con 0x55627b92c840 2023-07-14T18:22:18.523 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.472+0000 7fd7d8ff9700 10 client.25792 handle_command_reply: tid=0 2023-07-14T18:22:18.523 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.472+0000 7fd80b343b80 1 client.25792 shutdown 2023-07-14T18:22:18.523 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.473+0000 7fd80b343b80 1 -- 172.21.15.133:0/2097250851 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55627b8b68b0 msgr2=0x55627b8d45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:22:18.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.473+0000 7fd80b343b80 1 --2- 172.21.15.133:0/2097250851 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55627b8b68b0 0x55627b8d45d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec095f90 tx=0x7fd7ec00c900 comp rx=0 tx=0).stop 2023-07-14T18:22:18.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.473+0000 7fd80b343b80 1 -- 172.21.15.133:0/2097250851 shutdown_connections 2023-07-14T18:22:18.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.473+0000 7fd80b343b80 1 --2- 172.21.15.133:0/2097250851 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55627b92c840 0x55627b92ec20 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.473+0000 7fd80b343b80 1 --2- 172.21.15.133:0/2097250851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55627b8b74b0 0x55627b8d4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.473+0000 7fd80b343b80 1 --2- 172.21.15.133:0/2097250851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55627b8b5f60 0x55627b8d7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.473+0000 7fd80b343b80 1 --2- 172.21.15.133:0/2097250851 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55627b8b68b0 0x55627b8d45d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.473+0000 7fd7d8ff9700 0 client.25792 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:22:18.525 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.473+0000 7fd80b343b80 1 -- 172.21.15.133:0/2097250851 >> 172.21.15.133:0/2097250851 conn(0x55627b8ab680 msgr2=0x55627b8ba340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:22:18.525 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.473+0000 7fd80b343b80 1 -- 172.21.15.133:0/2097250851 shutdown_connections 2023-07-14T18:22:18.525 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.473+0000 7fd80b343b80 1 -- 172.21.15.133:0/2097250851 wait complete. 2023-07-14T18:22:18.525 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.474+0000 7fd80b343b80 20 client.25792 trim_cache size 0 max 16384 2023-07-14T18:22:18.526 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:22:18.526 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-14T18:22:18.526 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:22:18.526 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:22:18.526 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:22:18.526 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:22:18.527 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:22:18.527 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:22:18.527 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:22:18.527 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:22:18.527 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.474+0000 7fd7f79ff700 1 -- 172.21.15.133:0/1010379171 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd7dc0a1350 msgr2=0x7fd7dc0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:22:18.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.474+0000 7fd7f79ff700 1 --2- 172.21.15.133:0/1010379171 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd7dc0a1350 0x7fd7dc0a3800 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd7e00124b0 tx=0x7fd7e000b400 comp rx=0 tx=0).stop 2023-07-14T18:22:18.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.474+0000 7fd7f79ff700 1 -- 172.21.15.133:0/1010379171 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7f0102f90 msgr2=0x7fd7f01981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:22:18.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.474+0000 7fd7f79ff700 1 --2- 172.21.15.133:0/1010379171 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7f0102f90 0x7fd7f01981b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00c970 tx=0x7fd7ec00cc80 comp rx=0 tx=0).stop 2023-07-14T18:22:18.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.475+0000 7fd7f79ff700 1 -- 172.21.15.133:0/1010379171 shutdown_connections 2023-07-14T18:22:18.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.475+0000 7fd7f79ff700 1 --2- 172.21.15.133:0/1010379171 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fd7dc0a1350 0x7fd7dc0a3800 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.475+0000 7fd7f79ff700 1 --2- 172.21.15.133:0/1010379171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7f0103940 0x7fd7f019f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.475+0000 7fd7f79ff700 1 --2- 172.21.15.133:0/1010379171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7f0101e40 0x7fd7f0197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.475+0000 7fd7f79ff700 1 --2- 172.21.15.133:0/1010379171 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7f0102f90 0x7fd7f01981b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:18.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.475+0000 7fd7f79ff700 1 -- 172.21.15.133:0/1010379171 >> 172.21.15.133:0/1010379171 conn(0x7fd7f00fd480 msgr2=0x7fd7f00ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:22:18.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.475+0000 7fd7f79ff700 1 -- 172.21.15.133:0/1010379171 shutdown_connections 2023-07-14T18:22:18.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:18.475+0000 7fd7f79ff700 1 -- 172.21.15.133:0/1010379171 wait complete. 2023-07-14T18:22:18.559 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:22:19.856 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:19 smithi161 ceph-mon[128993]: pgmap v517: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 36 MiB/s wr, 2.66k op/s 2023-07-14T18:22:19.871 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:19 smithi171 ceph-mon[129241]: pgmap v517: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 36 MiB/s wr, 2.66k op/s 2023-07-14T18:22:19.912 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:19 smithi133 ceph-mon[126384]: pgmap v517: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 36 MiB/s wr, 2.66k op/s 2023-07-14T18:22:20.884 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 175.0 s, 1535.3 tps, lat 72.964 ms stddev 114.121 2023-07-14T18:22:21.856 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:21 smithi161 ceph-mon[128993]: pgmap v518: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 42 MiB/s wr, 3.33k op/s 2023-07-14T18:22:21.872 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:21 smithi171 ceph-mon[129241]: pgmap v518: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 42 MiB/s wr, 3.33k op/s 2023-07-14T18:22:21.920 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:21 smithi133 ceph-mon[126384]: pgmap v518: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 42 MiB/s wr, 3.33k op/s 2023-07-14T18:22:23.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:23 smithi161 ceph-mon[128993]: pgmap v519: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 36 MiB/s wr, 3.09k op/s 2023-07-14T18:22:23.879 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:23 smithi171 ceph-mon[129241]: pgmap v519: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 36 MiB/s wr, 3.09k op/s 2023-07-14T18:22:23.916 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:23 smithi133 ceph-mon[126384]: pgmap v519: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 36 MiB/s wr, 3.09k op/s 2023-07-14T18:22:25.856 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:25 smithi161 ceph-mon[128993]: pgmap v520: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 36 MiB/s wr, 2.91k op/s 2023-07-14T18:22:25.872 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:25 smithi171 ceph-mon[129241]: pgmap v520: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 36 MiB/s wr, 2.91k op/s 2023-07-14T18:22:25.881 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 180.0 s, 846.7 tps, lat 113.957 ms stddev 332.401 2023-07-14T18:22:25.915 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:25 smithi133 ceph-mon[126384]: pgmap v520: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 36 MiB/s wr, 2.91k op/s 2023-07-14T18:22:26.856 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:26 smithi161 ceph-mon[128993]: pgmap v521: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 27 MiB/s wr, 1.94k op/s 2023-07-14T18:22:26.872 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:26 smithi171 ceph-mon[129241]: pgmap v521: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 27 MiB/s wr, 1.94k op/s 2023-07-14T18:22:26.912 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:26 smithi133 ceph-mon[126384]: pgmap v521: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 27 MiB/s wr, 1.94k op/s 2023-07-14T18:22:29.106 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:28 smithi161 ceph-mon[128993]: pgmap v522: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 32 MiB/s wr, 2.32k op/s 2023-07-14T18:22:29.122 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:28 smithi171 ceph-mon[129241]: pgmap v522: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 32 MiB/s wr, 2.32k op/s 2023-07-14T18:22:29.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:28 smithi133 ceph-mon[126384]: pgmap v522: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 32 MiB/s wr, 2.32k op/s 2023-07-14T18:22:30.881 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 185.0 s, 1190.3 tps, lat 86.514 ms stddev 107.159 2023-07-14T18:22:31.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:31 smithi161 ceph-mon[128993]: pgmap v523: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 33 MiB/s wr, 2.33k op/s 2023-07-14T18:22:31.622 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:31 smithi171 ceph-mon[129241]: pgmap v523: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 33 MiB/s wr, 2.33k op/s 2023-07-14T18:22:31.668 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:31 smithi133 ceph-mon[126384]: pgmap v523: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 33 MiB/s wr, 2.33k op/s 2023-07-14T18:22:33.622 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:33 smithi171 ceph-mon[129241]: pgmap v524: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 29 MiB/s wr, 1.76k op/s 2023-07-14T18:22:33.670 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:33 smithi133 ceph-mon[126384]: pgmap v524: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 29 MiB/s wr, 1.76k op/s 2023-07-14T18:22:33.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:33 smithi161 ceph-mon[128993]: pgmap v524: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 29 MiB/s wr, 1.76k op/s 2023-07-14T18:22:35.663 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:35 smithi133 ceph-mon[126384]: pgmap v525: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 28 MiB/s wr, 1.82k op/s 2023-07-14T18:22:35.857 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:35 smithi161 ceph-mon[128993]: pgmap v525: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 28 MiB/s wr, 1.82k op/s 2023-07-14T18:22:35.874 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:35 smithi171 ceph-mon[129241]: pgmap v525: 129 pgs: 129 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 28 MiB/s wr, 1.82k op/s 2023-07-14T18:22:35.885 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 190.0 s, 454.2 tps, lat 221.343 ms stddev 544.441 2023-07-14T18:22:37.857 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:37 smithi161 ceph-mon[128993]: pgmap v526: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 31 MiB/s wr, 2.32k op/s 2023-07-14T18:22:37.872 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:37 smithi171 ceph-mon[129241]: pgmap v526: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 31 MiB/s wr, 2.32k op/s 2023-07-14T18:22:37.912 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:37 smithi133 ceph-mon[126384]: pgmap v526: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 31 MiB/s wr, 2.32k op/s 2023-07-14T18:22:39.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:39 smithi161 ceph-mon[128993]: pgmap v527: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 27 MiB/s wr, 2.19k op/s 2023-07-14T18:22:39.873 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:39 smithi171 ceph-mon[129241]: pgmap v527: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 27 MiB/s wr, 2.19k op/s 2023-07-14T18:22:39.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:39 smithi133 ceph-mon[126384]: pgmap v527: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 27 MiB/s wr, 2.19k op/s 2023-07-14T18:22:40.881 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 195.0 s, 883.8 tps, lat 104.924 ms stddev 189.645 2023-07-14T18:22:41.857 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:41 smithi161 ceph-mon[128993]: pgmap v528: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 25 MiB/s wr, 2.03k op/s 2023-07-14T18:22:41.872 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:41 smithi171 ceph-mon[129241]: pgmap v528: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 25 MiB/s wr, 2.03k op/s 2023-07-14T18:22:41.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:41 smithi133 ceph-mon[126384]: pgmap v528: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 25 MiB/s wr, 2.03k op/s 2023-07-14T18:22:43.857 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:43 smithi161 ceph-mon[128993]: pgmap v529: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 25 MiB/s wr, 2.02k op/s 2023-07-14T18:22:43.872 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:43 smithi171 ceph-mon[129241]: pgmap v529: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 25 MiB/s wr, 2.02k op/s 2023-07-14T18:22:43.915 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:43 smithi133 ceph-mon[126384]: pgmap v529: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 25 MiB/s wr, 2.02k op/s 2023-07-14T18:22:45.857 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:45 smithi161 ceph-mon[128993]: pgmap v530: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 23 MiB/s wr, 1.97k op/s 2023-07-14T18:22:45.872 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:45 smithi171 ceph-mon[129241]: pgmap v530: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 23 MiB/s wr, 1.97k op/s 2023-07-14T18:22:45.881 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 200.0 s, 1088.6 tps, lat 98.860 ms stddev 229.852 2023-07-14T18:22:45.913 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:45 smithi133 ceph-mon[126384]: pgmap v530: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 23 MiB/s wr, 1.97k op/s 2023-07-14T18:22:47.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:47 smithi161 ceph-mon[128993]: pgmap v531: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 25 MiB/s wr, 2.11k op/s 2023-07-14T18:22:47.872 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:47 smithi171 ceph-mon[129241]: pgmap v531: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 25 MiB/s wr, 2.11k op/s 2023-07-14T18:22:47.916 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:47 smithi133 ceph-mon[126384]: pgmap v531: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 25 MiB/s wr, 2.11k op/s 2023-07-14T18:22:48.559 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:22:48.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:48 smithi133 ceph-mon[126384]: pgmap v532: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 24 MiB/s wr, 1.88k op/s 2023-07-14T18:22:49.107 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:48 smithi161 ceph-mon[128993]: pgmap v532: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 24 MiB/s wr, 1.88k op/s 2023-07-14T18:22:49.122 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:48 smithi171 ceph-mon[129241]: pgmap v532: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 24 MiB/s wr, 1.88k op/s 2023-07-14T18:22:50.082 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:49 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:22:50.107 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:49 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:22:50.122 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:49 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:22:50.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.525+0000 7fa25ddff700 1 -- 172.21.15.133:0/1342918376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa25810be90 msgr2=0x7fa25810e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:22:50.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.525+0000 7fa25ddff700 1 --2- 172.21.15.133:0/1342918376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa25810be90 0x7fa25810e320 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa24c0098f0 tx=0x7fa24c009c00 comp rx=0 tx=0).stop 2023-07-14T18:22:50.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.525+0000 7fa25ddff700 1 -- 172.21.15.133:0/1342918376 shutdown_connections 2023-07-14T18:22:50.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.525+0000 7fa25ddff700 1 --2- 172.21.15.133:0/1342918376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa25810be90 0x7fa25810e320 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.525+0000 7fa25ddff700 1 --2- 172.21.15.133:0/1342918376 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa258071010 0x7fa258071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.525+0000 7fa25ddff700 1 --2- 172.21.15.133:0/1342918376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa258071960 0x7fa25810b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.525+0000 7fa25ddff700 1 -- 172.21.15.133:0/1342918376 >> 172.21.15.133:0/1342918376 conn(0x7fa25806cbc0 msgr2=0x7fa25806eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:22:50.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.525+0000 7fa25ddff700 1 -- 172.21.15.133:0/1342918376 shutdown_connections 2023-07-14T18:22:50.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.525+0000 7fa25ddff700 1 -- 172.21.15.133:0/1342918376 wait complete. 2023-07-14T18:22:50.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.527+0000 7fa25ddff700 1 Processor -- start 2023-07-14T18:22:50.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.528+0000 7fa25ddff700 1 -- start start 2023-07-14T18:22:50.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.528+0000 7fa25ddff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa258071010 0x7fa2581a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:50.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.528+0000 7fa25ddff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa258071960 0x7fa2581a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:50.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.528+0000 7fa25ddff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa25810be90 0x7fa2581abe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:50.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.528+0000 7fa25ddff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2581ac420 con 0x7fa258071960 2023-07-14T18:22:50.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.528+0000 7fa25ddff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2581ac560 con 0x7fa258071010 2023-07-14T18:22:50.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.528+0000 7fa25ddff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2581ac6f0 con 0x7fa25810be90 2023-07-14T18:22:50.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.529+0000 7fa257fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa25810be90 0x7fa2581abe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:50.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.529+0000 7fa257fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa25810be90 0x7fa2581abe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:46212/0 (socket says 172.21.15.133:46212) 2023-07-14T18:22:50.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.529+0000 7fa257fff700 1 -- 172.21.15.133:0/1338005913 learned_addr learned my addr 172.21.15.133:0/1338005913 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:22:50.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.530+0000 7fa2577fe700 1 --2- 172.21.15.133:0/1338005913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa258071010 0x7fa2581a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:50.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.531+0000 7fa257fff700 1 -- 172.21.15.133:0/1338005913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa258071010 msgr2=0x7fa2581a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:22:50.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.531+0000 7fa257fff700 1 --2- 172.21.15.133:0/1338005913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa258071010 0x7fa2581a48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.531+0000 7fa257fff700 1 -- 172.21.15.133:0/1338005913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa258071960 msgr2=0x7fa2581a4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:22:50.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.531+0000 7fa257fff700 1 --2- 172.21.15.133:0/1338005913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa258071960 0x7fa2581a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.531+0000 7fa257fff700 1 -- 172.21.15.133:0/1338005913 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa24c009520 con 0x7fa25810be90 2023-07-14T18:22:50.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.531+0000 7fa257fff700 1 --2- 172.21.15.133:0/1338005913 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa25810be90 0x7fa2581abe50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009230 tx=0x7fa24c00b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:22:50.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.531+0000 7fa254ff9700 1 -- 172.21.15.133:0/1338005913 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c01c070 con 0x7fa25810be90 2023-07-14T18:22:50.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.531+0000 7fa254ff9700 1 -- 172.21.15.133:0/1338005913 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa24c005500 con 0x7fa25810be90 2023-07-14T18:22:50.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.531+0000 7fa254ff9700 1 -- 172.21.15.133:0/1338005913 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c017400 con 0x7fa25810be90 2023-07-14T18:22:50.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.532+0000 7fa25ddff700 1 -- 172.21.15.133:0/1338005913 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2581ac970 con 0x7fa25810be90 2023-07-14T18:22:50.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.532+0000 7fa25ddff700 1 -- 172.21.15.133:0/1338005913 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2581acd40 con 0x7fa25810be90 2023-07-14T18:22:50.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.540+0000 7fa254ff9700 1 -- 172.21.15.133:0/1338005913 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fa24c005040 con 0x7fa25810be90 2023-07-14T18:22:50.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.543+0000 7fa254ff9700 1 --2- 172.21.15.133:0/1338005913 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa2400a1250 0x7fa2400a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:50.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.543+0000 7fa254ff9700 1 -- 172.21.15.133:0/1338005913 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa24c0ce0f0 con 0x7fa25810be90 2023-07-14T18:22:50.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.548+0000 7fa2577fe700 1 --2- 172.21.15.133:0/1338005913 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa2400a1250 0x7fa2400a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:50.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.550+0000 7fa2577fe700 1 --2- 172.21.15.133:0/1338005913 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa2400a1250 0x7fa2400a3700 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa248005fd0 tx=0x7fa248009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:22:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.590+0000 7fa27180bb80 1 Processor -- start 2023-07-14T18:22:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.590+0000 7fa27180bb80 1 -- start start 2023-07-14T18:22:50.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.591+0000 7fa27180bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6e906df60 0x55c6e906e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:50.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.591+0000 7fa27180bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6e906e8b0 0x55c6e906eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:50.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.591+0000 7fa27180bb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c6e906f4b0 0x55c6e906f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:50.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.591+0000 7fa27180bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e8f72cb0 con 0x55c6e906e8b0 2023-07-14T18:22:50.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.591+0000 7fa27180bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e8fc5a40 con 0x55c6e906df60 2023-07-14T18:22:50.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.591+0000 7fa27180bb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e8fbeed0 con 0x55c6e906f4b0 2023-07-14T18:22:50.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.591+0000 7fa2577fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6e906df60 0x55c6e906e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:50.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.591+0000 7fa2577fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6e906df60 0x55c6e906e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:48584/0 (socket says 172.21.15.133:48584) 2023-07-14T18:22:50.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.591+0000 7fa2577fe700 1 -- 172.21.15.133:0/1205726934 learned_addr learned my addr 172.21.15.133:0/1205726934 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:22:50.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.591+0000 7fa256ffd700 1 --2- 172.21.15.133:0/1205726934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6e906e8b0 0x55c6e906eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:50.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.591+0000 7fa257fff700 1 --2- 172.21.15.133:0/1205726934 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c6e906f4b0 0x55c6e906f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:50.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.592+0000 7fa257fff700 1 -- 172.21.15.133:0/1205726934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6e906df60 msgr2=0x55c6e906e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:22:50.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.592+0000 7fa257fff700 1 --2- 172.21.15.133:0/1205726934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6e906df60 0x55c6e906e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.592+0000 7fa257fff700 1 -- 172.21.15.133:0/1205726934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6e906e8b0 msgr2=0x55c6e906eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:22:50.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.592+0000 7fa257fff700 1 --2- 172.21.15.133:0/1205726934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6e906e8b0 0x55c6e906eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.592+0000 7fa257fff700 1 -- 172.21.15.133:0/1205726934 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6e8decde0 con 0x55c6e906f4b0 2023-07-14T18:22:50.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.592+0000 7fa257fff700 1 --2- 172.21.15.133:0/1205726934 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c6e906f4b0 0x55c6e906f920 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009fd0 tx=0x7fa24c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=561ebeff20a111a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:22:50.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.593+0000 7fa23dffb700 1 -- 172.21.15.133:0/1205726934 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c09ab60 con 0x55c6e906f4b0 2023-07-14T18:22:50.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.593+0000 7fa23dffb700 1 -- 172.21.15.133:0/1205726934 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa24c015300 con 0x55c6e906f4b0 2023-07-14T18:22:50.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.593+0000 7fa23dffb700 1 -- 172.21.15.133:0/1205726934 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c097910 con 0x55c6e906f4b0 2023-07-14T18:22:50.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.594+0000 7fa27180bb80 1 -- 172.21.15.133:0/1205726934 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c6e906f4b0 msgr2=0x55c6e906f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:22:50.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.594+0000 7fa27180bb80 1 --2- 172.21.15.133:0/1205726934 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c6e906f4b0 0x55c6e906f920 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009fd0 tx=0x7fa24c009260 comp rx=0 tx=0).stop 2023-07-14T18:22:50.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.595+0000 7fa27180bb80 1 -- 172.21.15.133:0/1205726934 shutdown_connections 2023-07-14T18:22:50.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.595+0000 7fa27180bb80 1 --2- 172.21.15.133:0/1205726934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6e906df60 0x55c6e906e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.595+0000 7fa27180bb80 1 --2- 172.21.15.133:0/1205726934 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c6e906f4b0 0x55c6e906f920 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.595+0000 7fa27180bb80 1 --2- 172.21.15.133:0/1205726934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6e906e8b0 0x55c6e906eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.595+0000 7fa27180bb80 1 -- 172.21.15.133:0/1205726934 >> 172.21.15.133:0/1205726934 conn(0x55c6e9063680 msgr2=0x55c6e90689d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:22:50.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.595+0000 7fa27180bb80 1 -- 172.21.15.133:0/1205726934 shutdown_connections 2023-07-14T18:22:50.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.595+0000 7fa27180bb80 1 -- 172.21.15.133:0/1205726934 wait complete. 2023-07-14T18:22:50.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.596+0000 7fa27180bb80 1 -- start start 2023-07-14T18:22:50.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.596+0000 7fa27180bb80 1 Processor -- start 2023-07-14T18:22:50.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.597+0000 7fa27180bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6e906df60 0x55c6e9077580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:50.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.597+0000 7fa27180bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6e906e8b0 0x55c6e9077ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:50.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.597+0000 7fa27180bb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c6e906f4b0 0x55c6e907eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:50.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.597+0000 7fa27180bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e8f5eac0 con 0x55c6e906e8b0 2023-07-14T18:22:50.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.597+0000 7fa27180bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e8f5e7c0 con 0x55c6e906df60 2023-07-14T18:22:50.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.599+0000 7fa27180bb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e8e42920 con 0x55c6e906f4b0 2023-07-14T18:22:50.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.598+0000 7fa257fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c6e906f4b0 0x55c6e907eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:50.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.599+0000 7fa27180bb80 10 client.0 _ll_register_callbacks cb 0x55c6e8f5c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:22:50.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.599+0000 7fa257fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c6e906f4b0 0x55c6e907eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:46240/0 (socket says 172.21.15.133:46240) 2023-07-14T18:22:50.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.599+0000 7fa257fff700 1 -- 172.21.15.133:0/4006828803 learned_addr learned my addr 172.21.15.133:0/4006828803 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:22:50.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.598+0000 7fa2577fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6e906df60 0x55c6e9077580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:50.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.599+0000 7fa256ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6e906e8b0 0x55c6e9077ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:50.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.600+0000 7fa2577fe700 1 -- 172.21.15.133:0/4006828803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c6e906f4b0 msgr2=0x55c6e907eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:22:50.653 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.600+0000 7fa2577fe700 1 --2- 172.21.15.133:0/4006828803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c6e906f4b0 0x55c6e907eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.653 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.600+0000 7fa2577fe700 1 -- 172.21.15.133:0/4006828803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6e906e8b0 msgr2=0x55c6e9077ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:22:50.653 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.600+0000 7fa2577fe700 1 --2- 172.21.15.133:0/4006828803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6e906e8b0 0x55c6e9077ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.654 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.600+0000 7fa2577fe700 1 -- 172.21.15.133:0/4006828803 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2581acd40 con 0x55c6e906df60 2023-07-14T18:22:50.654 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.600+0000 7fa2577fe700 1 --2- 172.21.15.133:0/4006828803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6e906df60 0x55c6e9077580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa248011770 tx=0x7fa2480047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7c576eb0af15a36 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:22:50.654 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.600+0000 7fa236ffd700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:22:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.601+0000 7fa236ffd700 1 -- 172.21.15.133:0/4006828803 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa248020840 con 0x55c6e906df60 2023-07-14T18:22:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.601+0000 7fa236ffd700 1 -- 172.21.15.133:0/4006828803 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa248018070 con 0x55c6e906df60 2023-07-14T18:22:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.601+0000 7fa27180bb80 1 -- 172.21.15.133:0/4006828803 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6e8f184f0 con 0x55c6e906df60 2023-07-14T18:22:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.601+0000 7fa236ffd700 1 -- 172.21.15.133:0/4006828803 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2480209a0 con 0x55c6e906df60 2023-07-14T18:22:50.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.601+0000 7fa27180bb80 10 client.25775 fetch_fsmap learned FSMap version 34 2023-07-14T18:22:50.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.601+0000 7fa27180bb80 1 -- 172.21.15.133:0/4006828803 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c6e8dd5c60 con 0x55c6e906df60 2023-07-14T18:22:50.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.602+0000 7fa236ffd700 1 -- 172.21.15.133:0/4006828803 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c6e8f184f0 con 0x55c6e906df60 2023-07-14T18:22:50.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.602+0000 7fa236ffd700 1 -- 172.21.15.133:0/4006828803 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa24801f430 con 0x55c6e906df60 2023-07-14T18:22:50.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.602+0000 7fa27180bb80 10 client.25775 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T18:22:50.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.602+0000 7fa27180bb80 10 client.25775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:22:50.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.602+0000 7fa27180bb80 20 client.25775 populate_metadata read hostname 'smithi133' 2023-07-14T18:22:50.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.602+0000 7fa27180bb80 1 --2- 172.21.15.133:0/4006828803 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55c6e9094370 0x55c6e9096780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:50.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.602+0000 7fa27180bb80 4 client.25775 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:22:50.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.602+0000 7fa27180bb80 1 -- 172.21.15.133:0/4006828803 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6e903cda0 con 0x55c6e9094370 2023-07-14T18:22:50.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.602+0000 7fa256ffd700 1 --2- 172.21.15.133:0/4006828803 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55c6e9094370 0x55c6e9096780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:50.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.603+0000 7fa236ffd700 10 client.25775 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:22:50.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.603+0000 7fa256ffd700 1 --2- 172.21.15.133:0/4006828803 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55c6e9094370 0x55c6e9096780 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8f8425681b94205 server_cookie=13dd2fb66d5a8448 in_seq=0 out_seq=0 2023-07-14T18:22:50.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.604+0000 7fa236ffd700 1 -- 172.21.15.133:0/4006828803 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55c6e903cda0 con 0x55c6e9094370 2023-07-14T18:22:50.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.604+0000 7fa236ffd700 10 client.25775 handle_command_reply: tid=0 2023-07-14T18:22:50.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.605+0000 7fa27180bb80 1 client.25775 shutdown 2023-07-14T18:22:50.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.605+0000 7fa27180bb80 1 -- 172.21.15.133:0/4006828803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6e906df60 msgr2=0x55c6e9077580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:22:50.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.605+0000 7fa27180bb80 1 --2- 172.21.15.133:0/4006828803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6e906df60 0x55c6e9077580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa248011770 tx=0x7fa2480047c0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.605+0000 7fa27180bb80 1 -- 172.21.15.133:0/4006828803 shutdown_connections 2023-07-14T18:22:50.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.605+0000 7fa27180bb80 1 --2- 172.21.15.133:0/4006828803 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55c6e9094370 0x55c6e9096780 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.605+0000 7fa27180bb80 1 --2- 172.21.15.133:0/4006828803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6e906df60 0x55c6e9077580 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa248011770 tx=0x7fa2480047c0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.605+0000 7fa27180bb80 1 --2- 172.21.15.133:0/4006828803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c6e906f4b0 0x55c6e907eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.605+0000 7fa236ffd700 0 client.25775 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:22:50.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.605+0000 7fa27180bb80 1 --2- 172.21.15.133:0/4006828803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6e906e8b0 0x55c6e9077ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.605+0000 7fa27180bb80 1 -- 172.21.15.133:0/4006828803 >> 172.21.15.133:0/4006828803 conn(0x55c6e9063680 msgr2=0x55c6e9067650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:22:50.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.607+0000 7fa27180bb80 1 -- 172.21.15.133:0/4006828803 shutdown_connections 2023-07-14T18:22:50.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.607+0000 7fa27180bb80 1 -- 172.21.15.133:0/4006828803 wait complete. 2023-07-14T18:22:50.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.607+0000 7fa27180bb80 20 client.25775 trim_cache size 0 max 16384 2023-07-14T18:22:50.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.637+0000 7fa27180bb80 1 Processor -- start 2023-07-14T18:22:50.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.638+0000 7fa27180bb80 1 -- start start 2023-07-14T18:22:50.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.638+0000 7fa27180bb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c6e906e8b0 0x55c6e9077330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:50.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.638+0000 7fa27180bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6e906f4b0 0x55c6e907eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:50.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.638+0000 7fa27180bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6e906a200 0x55c6e9077870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:50.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.638+0000 7fa27180bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e903cda0 con 0x55c6e906f4b0 2023-07-14T18:22:50.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.638+0000 7fa27180bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e9091800 con 0x55c6e906a200 2023-07-14T18:22:50.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.638+0000 7fa27180bb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e9069510 con 0x55c6e906e8b0 2023-07-14T18:22:50.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.638+0000 7fa2577fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c6e906e8b0 0x55c6e9077330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:50.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.638+0000 7fa2577fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c6e906e8b0 0x55c6e9077330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:46256/0 (socket says 172.21.15.133:46256) 2023-07-14T18:22:50.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.638+0000 7fa2577fe700 1 -- 172.21.15.133:0/723861954 learned_addr learned my addr 172.21.15.133:0/723861954 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:22:50.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.639+0000 7fa257fff700 1 --2- 172.21.15.133:0/723861954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6e906a200 0x55c6e9077870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:50.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.639+0000 7fa257fff700 1 -- 172.21.15.133:0/723861954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c6e906e8b0 msgr2=0x55c6e9077330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:22:50.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.639+0000 7fa257fff700 1 --2- 172.21.15.133:0/723861954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c6e906e8b0 0x55c6e9077330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.639+0000 7fa257fff700 1 -- 172.21.15.133:0/723861954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6e906f4b0 msgr2=0x55c6e907eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:22:50.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.639+0000 7fa257fff700 1 --2- 172.21.15.133:0/723861954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6e906f4b0 0x55c6e907eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.639+0000 7fa257fff700 1 -- 172.21.15.133:0/723861954 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6e8e13250 con 0x55c6e906a200 2023-07-14T18:22:50.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.639+0000 7fa257fff700 1 --2- 172.21.15.133:0/723861954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6e906a200 0x55c6e9077870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa24c097700 tx=0x7fa24c00bca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=172b9b8a0332ed4f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:22:50.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.639+0000 7fa23e7fc700 1 -- 172.21.15.133:0/723861954 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c09ab60 con 0x55c6e906a200 2023-07-14T18:22:50.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.640+0000 7fa23e7fc700 1 -- 172.21.15.133:0/723861954 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa24c0a35c0 con 0x55c6e906a200 2023-07-14T18:22:50.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.640+0000 7fa23e7fc700 1 -- 172.21.15.133:0/723861954 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c097910 con 0x55c6e906a200 2023-07-14T18:22:50.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.640+0000 7fa27180bb80 1 -- 172.21.15.133:0/723861954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6e906a200 msgr2=0x55c6e9077870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:22:50.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.640+0000 7fa27180bb80 1 --2- 172.21.15.133:0/723861954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6e906a200 0x55c6e9077870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa24c097700 tx=0x7fa24c00bca0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.694 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.641+0000 7fa27180bb80 1 -- 172.21.15.133:0/723861954 shutdown_connections 2023-07-14T18:22:50.694 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.641+0000 7fa27180bb80 1 --2- 172.21.15.133:0/723861954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6e906a200 0x55c6e9077870 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.694 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.641+0000 7fa27180bb80 1 --2- 172.21.15.133:0/723861954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c6e906e8b0 0x55c6e9077330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.694 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.641+0000 7fa27180bb80 1 --2- 172.21.15.133:0/723861954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6e906f4b0 0x55c6e907eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.641+0000 7fa27180bb80 1 -- 172.21.15.133:0/723861954 >> 172.21.15.133:0/723861954 conn(0x55c6e9063680 msgr2=0x55c6e9072290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:22:50.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.641+0000 7fa27180bb80 1 -- 172.21.15.133:0/723861954 shutdown_connections 2023-07-14T18:22:50.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.641+0000 7fa27180bb80 1 -- 172.21.15.133:0/723861954 wait complete. 2023-07-14T18:22:50.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.642+0000 7fa27180bb80 1 -- start start 2023-07-14T18:22:50.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.642+0000 7fa27180bb80 1 Processor -- start 2023-07-14T18:22:50.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.643+0000 7fa27180bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6e906a200 0x55c6e907b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:50.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.643+0000 7fa27180bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6e906e8b0 0x55c6e907bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:50.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.643+0000 7fa27180bb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c6e906f4b0 0x55c6e90e8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:50.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.643+0000 7fa27180bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e90e8540 con 0x55c6e906e8b0 2023-07-14T18:22:50.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.643+0000 7fa27180bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e90e8680 con 0x55c6e906a200 2023-07-14T18:22:50.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.643+0000 7fa27180bb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e90e87c0 con 0x55c6e906f4b0 2023-07-14T18:22:50.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.643+0000 7fa27180bb80 10 client.0 _ll_register_callbacks cb 0x7fa2480127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:22:50.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.643+0000 7fa2577fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6e906a200 0x55c6e907b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:50.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.643+0000 7fa2577fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6e906a200 0x55c6e907b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:48598/0 (socket says 172.21.15.133:48598) 2023-07-14T18:22:50.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.643+0000 7fa2577fe700 1 -- 172.21.15.133:0/3662425085 learned_addr learned my addr 172.21.15.133:0/3662425085 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:22:50.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.644+0000 7fa257fff700 1 --2- 172.21.15.133:0/3662425085 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c6e906f4b0 0x55c6e90e8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:50.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.644+0000 7fa256ffd700 1 --2- 172.21.15.133:0/3662425085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6e906e8b0 0x55c6e907bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:50.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.644+0000 7fa2577fe700 1 -- 172.21.15.133:0/3662425085 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c6e906f4b0 msgr2=0x55c6e90e8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:22:50.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.644+0000 7fa2577fe700 1 --2- 172.21.15.133:0/3662425085 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c6e906f4b0 0x55c6e90e8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.644+0000 7fa2577fe700 1 -- 172.21.15.133:0/3662425085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6e906e8b0 msgr2=0x55c6e907bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:22:50.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.644+0000 7fa2577fe700 1 --2- 172.21.15.133:0/3662425085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6e906e8b0 0x55c6e907bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.644+0000 7fa2577fe700 1 -- 172.21.15.133:0/3662425085 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6e8decde0 con 0x55c6e906a200 2023-07-14T18:22:50.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.644+0000 7fa257fff700 1 --2- 172.21.15.133:0/3662425085 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c6e906f4b0 0x55c6e90e8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:22:50.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.644+0000 7fa256ffd700 1 --2- 172.21.15.133:0/3662425085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6e906e8b0 0x55c6e907bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:22:50.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.645+0000 7fa2577fe700 1 --2- 172.21.15.133:0/3662425085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6e906a200 0x55c6e907b9f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa248005b40 tx=0x7fa248011ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3bdbb5b4bbb64a2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:22:50.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.645+0000 7fa236ffd700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:22:50.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.645+0000 7fa236ffd700 1 -- 172.21.15.133:0/3662425085 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24801fec0 con 0x55c6e906a200 2023-07-14T18:22:50.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.645+0000 7fa236ffd700 1 -- 172.21.15.133:0/3662425085 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa248018050 con 0x55c6e906a200 2023-07-14T18:22:50.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.645+0000 7fa27180bb80 1 -- 172.21.15.133:0/3662425085 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa24c015300 con 0x55c6e906a200 2023-07-14T18:22:50.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.645+0000 7fa236ffd700 1 -- 172.21.15.133:0/3662425085 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa248020640 con 0x55c6e906a200 2023-07-14T18:22:50.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.645+0000 7fa27180bb80 10 client.25781 fetch_fsmap learned FSMap version 34 2023-07-14T18:22:50.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.645+0000 7fa27180bb80 1 -- 172.21.15.133:0/3662425085 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c6e8deb2b0 con 0x55c6e906a200 2023-07-14T18:22:50.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.645+0000 7fa236ffd700 1 -- 172.21.15.133:0/3662425085 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa24c015300 con 0x55c6e906a200 2023-07-14T18:22:50.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.646+0000 7fa236ffd700 1 -- 172.21.15.133:0/3662425085 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa248029880 con 0x55c6e906a200 2023-07-14T18:22:50.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.646+0000 7fa27180bb80 10 client.25781 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T18:22:50.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.646+0000 7fa27180bb80 10 client.25781 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:22:50.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.646+0000 7fa27180bb80 20 client.25781 populate_metadata read hostname 'smithi133' 2023-07-14T18:22:50.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.646+0000 7fa27180bb80 1 --2- 172.21.15.133:0/3662425085 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55c6e90ea160 0x55c6e90ec570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:22:50.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.646+0000 7fa27180bb80 4 client.25781 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:22:50.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.646+0000 7fa27180bb80 1 -- 172.21.15.133:0/3662425085 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c6e8fb7eb0 con 0x55c6e90ea160 2023-07-14T18:22:50.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.646+0000 7fa256ffd700 1 --2- 172.21.15.133:0/3662425085 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55c6e90ea160 0x55c6e90ec570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:22:50.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.647+0000 7fa256ffd700 1 --2- 172.21.15.133:0/3662425085 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55c6e90ea160 0x55c6e90ec570 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdc20decc73b6f68 server_cookie=bd5fcaefc28c48d1 in_seq=0 out_seq=0 2023-07-14T18:22:50.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.647+0000 7fa236ffd700 10 client.25781 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:22:50.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.647+0000 7fa236ffd700 1 -- 172.21.15.133:0/3662425085 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55c6e8fb7eb0 con 0x55c6e90ea160 2023-07-14T18:22:50.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.647+0000 7fa236ffd700 10 client.25781 handle_command_reply: tid=0 2023-07-14T18:22:50.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.647+0000 7fa27180bb80 1 client.25781 shutdown 2023-07-14T18:22:50.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.648+0000 7fa27180bb80 1 -- 172.21.15.133:0/3662425085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6e906a200 msgr2=0x55c6e907b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:22:50.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.648+0000 7fa27180bb80 1 --2- 172.21.15.133:0/3662425085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6e906a200 0x55c6e907b9f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa248005b40 tx=0x7fa248011ae0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.648+0000 7fa27180bb80 1 -- 172.21.15.133:0/3662425085 shutdown_connections 2023-07-14T18:22:50.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.648+0000 7fa27180bb80 1 --2- 172.21.15.133:0/3662425085 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55c6e90ea160 0x55c6e90ec570 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.648+0000 7fa27180bb80 1 --2- 172.21.15.133:0/3662425085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6e906a200 0x55c6e907b9f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.648+0000 7fa27180bb80 1 --2- 172.21.15.133:0/3662425085 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c6e906f4b0 0x55c6e90e8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.648+0000 7fa27180bb80 1 --2- 172.21.15.133:0/3662425085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6e906e8b0 0x55c6e907bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.648+0000 7fa27180bb80 1 -- 172.21.15.133:0/3662425085 >> 172.21.15.133:0/3662425085 conn(0x55c6e9063680 msgr2=0x55c6e9077330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:22:50.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.648+0000 7fa236ffd700 0 client.25781 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:22:50.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.648+0000 7fa27180bb80 1 -- 172.21.15.133:0/3662425085 shutdown_connections 2023-07-14T18:22:50.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.648+0000 7fa27180bb80 1 -- 172.21.15.133:0/3662425085 wait complete. 2023-07-14T18:22:50.706 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:22:50.706 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-14T18:22:50.706 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:22:50.706 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:22:50.706 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:22:50.707 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:22:50.707 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:22:50.707 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:22:50.707 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:22:50.707 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:22:50.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.651+0000 7fa27180bb80 20 client.25781 trim_cache size 0 max 16384 2023-07-14T18:22:50.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.659+0000 7fa25ddff700 1 -- 172.21.15.133:0/1338005913 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa2400a1250 msgr2=0x7fa2400a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:22:50.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.659+0000 7fa25ddff700 1 --2- 172.21.15.133:0/1338005913 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa2400a1250 0x7fa2400a3700 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa248005fd0 tx=0x7fa248009310 comp rx=0 tx=0).stop 2023-07-14T18:22:50.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.659+0000 7fa25ddff700 1 -- 172.21.15.133:0/1338005913 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa25810be90 msgr2=0x7fa2581abe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:22:50.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.659+0000 7fa25ddff700 1 --2- 172.21.15.133:0/1338005913 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa25810be90 0x7fa2581abe50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009230 tx=0x7fa24c00b840 comp rx=0 tx=0).stop 2023-07-14T18:22:50.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.667+0000 7fa25ddff700 1 -- 172.21.15.133:0/1338005913 shutdown_connections 2023-07-14T18:22:50.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.667+0000 7fa25ddff700 1 --2- 172.21.15.133:0/1338005913 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fa2400a1250 0x7fa2400a3700 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.667+0000 7fa25ddff700 1 --2- 172.21.15.133:0/1338005913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa258071010 0x7fa2581a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.667+0000 7fa25ddff700 1 --2- 172.21.15.133:0/1338005913 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa25810be90 0x7fa2581abe50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.667+0000 7fa25ddff700 1 --2- 172.21.15.133:0/1338005913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa258071960 0x7fa2581a4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:22:50.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.667+0000 7fa25ddff700 1 -- 172.21.15.133:0/1338005913 >> 172.21.15.133:0/1338005913 conn(0x7fa25806cbc0 msgr2=0x7fa25806de10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:22:50.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.694+0000 7fa25ddff700 1 -- 172.21.15.133:0/1338005913 shutdown_connections 2023-07-14T18:22:50.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:22:50.695+0000 7fa25ddff700 1 -- 172.21.15.133:0/1338005913 wait complete. 2023-07-14T18:22:50.802 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:22:50.881 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 205.0 s, 669.0 tps, lat 149.244 ms stddev 197.793 2023-07-14T18:22:51.666 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:51 smithi133 ceph-mon[126384]: pgmap v533: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 23 MiB/s wr, 1.67k op/s 2023-07-14T18:22:51.857 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:51 smithi161 ceph-mon[128993]: pgmap v533: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 23 MiB/s wr, 1.67k op/s 2023-07-14T18:22:51.872 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:51 smithi171 ceph-mon[129241]: pgmap v533: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 23 MiB/s wr, 1.67k op/s 2023-07-14T18:22:52.669 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:52 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:22:52.670 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:52 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:22:52.670 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:52 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:22:52.670 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:52 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:22:52.670 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:52 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:22:52.670 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:52 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:22:52.671 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:52 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:22:52.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:52 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:22:52.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:52 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:22:52.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:52 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:22:52.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:52 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:22:52.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:52 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:22:52.863 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:52 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:22:52.863 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:52 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:22:52.876 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:52 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:22:52.876 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:52 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:22:52.877 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:52 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:22:52.877 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:52 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:22:52.877 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:52 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:22:52.877 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:52 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:22:52.877 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:52 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:22:53.672 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:53 smithi133 ceph-mon[126384]: Adjusting osd_memory_target on smithi133 to 220.9M 2023-07-14T18:22:53.673 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:53 smithi133 ceph-mon[126384]: Unable to set osd_memory_target on smithi133 to 231661158: error parsing value: Value '231661158' is below minimum 939524096 2023-07-14T18:22:53.673 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:53 smithi133 ceph-mon[126384]: pgmap v534: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 25 MiB/s wr, 1.88k op/s 2023-07-14T18:22:53.857 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:53 smithi161 ceph-mon[128993]: Adjusting osd_memory_target on smithi133 to 220.9M 2023-07-14T18:22:53.857 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:53 smithi161 ceph-mon[128993]: Unable to set osd_memory_target on smithi133 to 231661158: error parsing value: Value '231661158' is below minimum 939524096 2023-07-14T18:22:53.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:53 smithi161 ceph-mon[128993]: pgmap v534: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 25 MiB/s wr, 1.88k op/s 2023-07-14T18:22:53.872 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:53 smithi171 ceph-mon[129241]: Adjusting osd_memory_target on smithi133 to 220.9M 2023-07-14T18:22:53.873 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:53 smithi171 ceph-mon[129241]: Unable to set osd_memory_target on smithi133 to 231661158: error parsing value: Value '231661158' is below minimum 939524096 2023-07-14T18:22:53.873 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:53 smithi171 ceph-mon[129241]: pgmap v534: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 25 MiB/s wr, 1.88k op/s 2023-07-14T18:22:55.857 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:55 smithi161 ceph-mon[128993]: pgmap v535: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 21 MiB/s wr, 1.50k op/s 2023-07-14T18:22:55.872 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:55 smithi171 ceph-mon[129241]: pgmap v535: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 21 MiB/s wr, 1.50k op/s 2023-07-14T18:22:55.881 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 210.0 s, 1569.8 tps, lat 62.978 ms stddev 45.596 2023-07-14T18:22:55.913 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:55 smithi133 ceph-mon[126384]: pgmap v535: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 21 MiB/s wr, 1.50k op/s 2023-07-14T18:22:57.857 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:57 smithi161 ceph-mon[128993]: pgmap v536: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 23 MiB/s wr, 1.47k op/s 2023-07-14T18:22:57.872 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:57 smithi171 ceph-mon[129241]: pgmap v536: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 23 MiB/s wr, 1.47k op/s 2023-07-14T18:22:57.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:57 smithi133 ceph-mon[126384]: pgmap v536: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 23 MiB/s wr, 1.47k op/s 2023-07-14T18:22:59.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:22:59 smithi161 ceph-mon[128993]: pgmap v537: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 24 MiB/s wr, 1.58k op/s 2023-07-14T18:22:59.872 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:22:59 smithi171 ceph-mon[129241]: pgmap v537: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 24 MiB/s wr, 1.58k op/s 2023-07-14T18:22:59.913 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:22:59 smithi133 ceph-mon[126384]: pgmap v537: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 24 MiB/s wr, 1.58k op/s 2023-07-14T18:23:00.884 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 215.0 s, 880.3 tps, lat 112.967 ms stddev 374.441 2023-07-14T18:23:01.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:01 smithi161 ceph-mon[128993]: pgmap v538: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 23 MiB/s wr, 1.53k op/s 2023-07-14T18:23:01.873 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:01 smithi171 ceph-mon[129241]: pgmap v538: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 23 MiB/s wr, 1.53k op/s 2023-07-14T18:23:01.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:01 smithi133 ceph-mon[126384]: pgmap v538: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 23 MiB/s wr, 1.53k op/s 2023-07-14T18:23:03.857 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:03 smithi161 ceph-mon[128993]: pgmap v539: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 27 MiB/s wr, 1.99k op/s 2023-07-14T18:23:03.872 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:03 smithi171 ceph-mon[129241]: pgmap v539: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 27 MiB/s wr, 1.99k op/s 2023-07-14T18:23:03.913 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:03 smithi133 ceph-mon[126384]: pgmap v539: 129 pgs: 129 active+clean; 8.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 27 MiB/s wr, 1.99k op/s 2023-07-14T18:23:05.108 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:04 smithi161 ceph-mon[128993]: pgmap v540: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 28 MiB/s wr, 2.02k op/s 2023-07-14T18:23:05.126 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:04 smithi171 ceph-mon[129241]: pgmap v540: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 28 MiB/s wr, 2.02k op/s 2023-07-14T18:23:05.164 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:04 smithi133 ceph-mon[126384]: pgmap v540: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 28 MiB/s wr, 2.02k op/s 2023-07-14T18:23:05.884 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 220.0 s, 471.9 tps, lat 215.556 ms stddev 472.193 2023-07-14T18:23:07.863 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:07 smithi161 ceph-mon[128993]: pgmap v541: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 29 MiB/s wr, 2.26k op/s 2023-07-14T18:23:07.877 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:07 smithi171 ceph-mon[129241]: pgmap v541: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 29 MiB/s wr, 2.26k op/s 2023-07-14T18:23:07.923 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:07 smithi133 ceph-mon[126384]: pgmap v541: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 29 MiB/s wr, 2.26k op/s 2023-07-14T18:23:09.864 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:09 smithi161 ceph-mon[128993]: pgmap v542: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 27 MiB/s wr, 2.42k op/s 2023-07-14T18:23:09.872 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:09 smithi171 ceph-mon[129241]: pgmap v542: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 27 MiB/s wr, 2.42k op/s 2023-07-14T18:23:09.916 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:09 smithi133 ceph-mon[126384]: pgmap v542: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 27 MiB/s wr, 2.42k op/s 2023-07-14T18:23:10.885 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 225.0 s, 548.4 tps, lat 180.097 ms stddev 262.275 2023-07-14T18:23:11.857 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:11 smithi161 ceph-mon[128993]: pgmap v543: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 25 MiB/s wr, 2.18k op/s 2023-07-14T18:23:11.874 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:11 smithi171 ceph-mon[129241]: pgmap v543: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 25 MiB/s wr, 2.18k op/s 2023-07-14T18:23:11.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:11 smithi133 ceph-mon[126384]: pgmap v543: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 25 MiB/s wr, 2.18k op/s 2023-07-14T18:23:13.857 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:13 smithi161 ceph-mon[128993]: pgmap v544: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 23 MiB/s wr, 2.13k op/s 2023-07-14T18:23:13.872 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:13 smithi171 ceph-mon[129241]: pgmap v544: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 23 MiB/s wr, 2.13k op/s 2023-07-14T18:23:13.918 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:13 smithi133 ceph-mon[126384]: pgmap v544: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 23 MiB/s wr, 2.13k op/s 2023-07-14T18:23:15.857 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:15 smithi161 ceph-mon[128993]: pgmap v545: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 20 MiB/s wr, 1.88k op/s 2023-07-14T18:23:15.873 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:15 smithi171 ceph-mon[129241]: pgmap v545: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 20 MiB/s wr, 1.88k op/s 2023-07-14T18:23:15.885 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 230.0 s, 955.9 tps, lat 105.605 ms stddev 122.007 2023-07-14T18:23:15.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:15 smithi133 ceph-mon[126384]: pgmap v545: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 20 MiB/s wr, 1.88k op/s 2023-07-14T18:23:17.872 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:17 smithi171 ceph-mon[129241]: pgmap v546: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 17 MiB/s wr, 1.60k op/s 2023-07-14T18:23:17.915 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:17 smithi133 ceph-mon[126384]: pgmap v546: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 17 MiB/s wr, 1.60k op/s 2023-07-14T18:23:18.107 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:17 smithi161 ceph-mon[128993]: pgmap v546: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 17 MiB/s wr, 1.60k op/s 2023-07-14T18:23:19.873 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:19 smithi171 ceph-mon[129241]: pgmap v547: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 17 MiB/s wr, 1.47k op/s 2023-07-14T18:23:19.916 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:19 smithi133 ceph-mon[126384]: pgmap v547: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 17 MiB/s wr, 1.47k op/s 2023-07-14T18:23:20.107 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:19 smithi161 ceph-mon[128993]: pgmap v547: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 17 MiB/s wr, 1.47k op/s 2023-07-14T18:23:20.804 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:23:20.881 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 235.0 s, 916.6 tps, lat 108.884 ms stddev 295.694 2023-07-14T18:23:21.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.533+0000 7f17333ff700 1 -- 172.21.15.133:0/2893260946 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f172c071010 msgr2=0x7f172c071420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:23:21.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.533+0000 7f17333ff700 1 --2- 172.21.15.133:0/2893260946 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f172c071010 0x7f172c071420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f17280098f0 tx=0x7f1728009c00 comp rx=0 tx=0).stop 2023-07-14T18:23:21.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.533+0000 7f17333ff700 1 -- 172.21.15.133:0/2893260946 shutdown_connections 2023-07-14T18:23:21.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.533+0000 7f17333ff700 1 --2- 172.21.15.133:0/2893260946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f172c10be90 0x7f172c10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.533+0000 7f17333ff700 1 --2- 172.21.15.133:0/2893260946 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f172c071010 0x7f172c071420 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f17280098f0 tx=0x7f1728009c00 comp rx=0 tx=0).stop 2023-07-14T18:23:21.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.533+0000 7f17333ff700 1 --2- 172.21.15.133:0/2893260946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f172c071960 0x7f172c10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.533+0000 7f17333ff700 1 -- 172.21.15.133:0/2893260946 >> 172.21.15.133:0/2893260946 conn(0x7f172c06cbc0 msgr2=0x7f172c06eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:23:21.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.550+0000 7f17333ff700 1 -- 172.21.15.133:0/2893260946 shutdown_connections 2023-07-14T18:23:21.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.550+0000 7f17333ff700 1 -- 172.21.15.133:0/2893260946 wait complete. 2023-07-14T18:23:21.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.551+0000 7f17333ff700 1 Processor -- start 2023-07-14T18:23:21.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.551+0000 7f17333ff700 1 -- start start 2023-07-14T18:23:21.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.552+0000 7f17333ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f172c071960 0x7f172c19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:21.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.552+0000 7f17333ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f172c10be90 0x7f172c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:21.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.552+0000 7f17333ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f172c076890 0x7f172c078d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:21.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.552+0000 7f17333ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c079280 con 0x7f172c076890 2023-07-14T18:23:21.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.552+0000 7f17333ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c0793f0 con 0x7f172c10be90 2023-07-14T18:23:21.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.552+0000 7f17333ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c0795b0 con 0x7f172c071960 2023-07-14T18:23:21.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.555+0000 7f173199c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f172c076890 0x7f172c078d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:21.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.555+0000 7f173199c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f172c076890 0x7f172c078d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46900/0 (socket says 172.21.15.133:46900) 2023-07-14T18:23:21.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.555+0000 7f173199c700 1 -- 172.21.15.133:0/3841125659 learned_addr learned my addr 172.21.15.133:0/3841125659 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:23:21.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.556+0000 7f173199c700 1 -- 172.21.15.133:0/3841125659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f172c071960 msgr2=0x7f172c19c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:23:21.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.556+0000 7f173199c700 1 --2- 172.21.15.133:0/3841125659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f172c071960 0x7f172c19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.556+0000 7f173199c700 1 -- 172.21.15.133:0/3841125659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f172c10be90 msgr2=0x7f172c19ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:23:21.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.556+0000 7f173199c700 1 --2- 172.21.15.133:0/3841125659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f172c10be90 0x7f172c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.556+0000 7f173199c700 1 -- 172.21.15.133:0/3841125659 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1728009520 con 0x7f172c076890 2023-07-14T18:23:21.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.557+0000 7f173199c700 1 --2- 172.21.15.133:0/3841125659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f172c076890 0x7f172c078d40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f172400b970 tx=0x7f172400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:23:21.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.558+0000 7f17227fc700 1 -- 172.21.15.133:0/3841125659 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172400c9b0 con 0x7f172c076890 2023-07-14T18:23:21.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.558+0000 7f17227fc700 1 -- 172.21.15.133:0/3841125659 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1724014440 con 0x7f172c076890 2023-07-14T18:23:21.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.558+0000 7f17227fc700 1 -- 172.21.15.133:0/3841125659 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17240075d0 con 0x7f172c076890 2023-07-14T18:23:21.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.558+0000 7f17333ff700 1 -- 172.21.15.133:0/3841125659 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f172c079890 con 0x7f172c076890 2023-07-14T18:23:21.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.558+0000 7f17333ff700 1 -- 172.21.15.133:0/3841125659 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f172c079dd0 con 0x7f172c076890 2023-07-14T18:23:21.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.568+0000 7f17227fc700 1 -- 172.21.15.133:0/3841125659 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f172400cb10 con 0x7f172c076890 2023-07-14T18:23:21.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.585+0000 7f1746c79b80 1 Processor -- start 2023-07-14T18:23:21.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.585+0000 7f1746c79b80 1 -- start start 2023-07-14T18:23:21.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.586+0000 7f1746c79b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a6b5d4f60 0x562a6b5d5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:21.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.586+0000 7f1746c79b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a6b5d58b0 0x562a6b5d5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:21.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.586+0000 7f1746c79b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b5d64b0 0x562a6b5d6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:21.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.586+0000 7f1746c79b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6b4d9cb0 con 0x562a6b5d58b0 2023-07-14T18:23:21.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.586+0000 7f1746c79b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6b52ca40 con 0x562a6b5d64b0 2023-07-14T18:23:21.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.586+0000 7f1746c79b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6b525ed0 con 0x562a6b5d4f60 2023-07-14T18:23:21.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.592+0000 7f173099a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a6b5d4f60 0x562a6b5d5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:21.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.592+0000 7f173099a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a6b5d4f60 0x562a6b5d5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:42100/0 (socket says 172.21.15.133:42100) 2023-07-14T18:23:21.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.592+0000 7f173099a700 1 -- 172.21.15.133:0/4024970965 learned_addr learned my addr 172.21.15.133:0/4024970965 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:23:21.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.592+0000 7f173099a700 1 --2- 172.21.15.133:0/4024970965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b5d64b0 0x562a6b5d6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:21.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.603+0000 7f17227fc700 1 --2- 172.21.15.133:0/3841125659 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f17180a1350 0x7f17180a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:21.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.603+0000 7f17227fc700 1 -- 172.21.15.133:0/3841125659 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1724006e40 con 0x7f172c076890 2023-07-14T18:23:21.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.605+0000 7f173119b700 1 --2- 172.21.15.133:0/4024970965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a6b5d58b0 0x562a6b5d5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:21.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.605+0000 7f173099a700 1 -- 172.21.15.133:0/4024970965 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a6b5d4f60 msgr2=0x562a6b5d5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:23:21.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.605+0000 7f173099a700 1 --2- 172.21.15.133:0/4024970965 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a6b5d4f60 0x562a6b5d5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.605+0000 7f173099a700 1 -- 172.21.15.133:0/4024970965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a6b5d58b0 msgr2=0x562a6b5d5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:23:21.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.605+0000 7f173099a700 1 --2- 172.21.15.133:0/4024970965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a6b5d58b0 0x562a6b5d5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.605+0000 7f173099a700 1 -- 172.21.15.133:0/4024970965 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a6b353de0 con 0x562a6b5d64b0 2023-07-14T18:23:21.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.605+0000 7f173119b700 1 --2- 172.21.15.133:0/4024970965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a6b5d58b0 0x562a6b5d5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T18:23:21.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.614+0000 7f173099a700 1 --2- 172.21.15.133:0/4024970965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b5d64b0 0x562a6b5d6920 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f171c0248a0 tx=0x7f171c0058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4531c4bf78122b6e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:23:21.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.615+0000 7f173199c700 1 --2- 172.21.15.133:0/3841125659 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f17180a1350 0x7f17180a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:21.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:21 smithi133 ceph-mon[126384]: pgmap v548: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 19 MiB/s wr, 1.50k op/s 2023-07-14T18:23:21.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.624+0000 7f17177fe700 1 -- 172.21.15.133:0/4024970965 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c0240d0 con 0x562a6b5d64b0 2023-07-14T18:23:21.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.624+0000 7f17177fe700 1 -- 172.21.15.133:0/4024970965 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f171c02a030 con 0x562a6b5d64b0 2023-07-14T18:23:21.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.624+0000 7f17177fe700 1 -- 172.21.15.133:0/4024970965 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c0053a0 con 0x562a6b5d64b0 2023-07-14T18:23:21.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.625+0000 7f1746c79b80 1 -- 172.21.15.133:0/4024970965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b5d64b0 msgr2=0x562a6b5d6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:23:21.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.626+0000 7f1746c79b80 1 --2- 172.21.15.133:0/4024970965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b5d64b0 0x562a6b5d6920 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f171c0248a0 tx=0x7f171c0058e0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.627+0000 7f1746c79b80 1 -- 172.21.15.133:0/4024970965 shutdown_connections 2023-07-14T18:23:21.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.627+0000 7f1746c79b80 1 --2- 172.21.15.133:0/4024970965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b5d64b0 0x562a6b5d6920 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f171c0248a0 tx=0x7f171c0058e0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.627+0000 7f1746c79b80 1 --2- 172.21.15.133:0/4024970965 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a6b5d4f60 0x562a6b5d5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.627+0000 7f1746c79b80 1 --2- 172.21.15.133:0/4024970965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a6b5d58b0 0x562a6b5d5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.627+0000 7f1746c79b80 1 -- 172.21.15.133:0/4024970965 >> 172.21.15.133:0/4024970965 conn(0x562a6b5ca680 msgr2=0x562a6b5cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:23:21.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.640+0000 7f1746c79b80 1 -- 172.21.15.133:0/4024970965 shutdown_connections 2023-07-14T18:23:21.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.654+0000 7f1746c79b80 1 -- 172.21.15.133:0/4024970965 wait complete. 2023-07-14T18:23:21.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.654+0000 7f1746c79b80 1 -- start start 2023-07-14T18:23:21.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.655+0000 7f1746c79b80 1 Processor -- start 2023-07-14T18:23:21.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.657+0000 7f173199c700 1 --2- 172.21.15.133:0/3841125659 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f17180a1350 0x7f17180a3800 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1724095db0 tx=0x7f1724004ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:23:21.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.659+0000 7f1746c79b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a6b5d4f60 0x562a6b5deb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:21.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.659+0000 7f1746c79b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b5d58b0 0x562a6b5df0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:21.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.659+0000 7f1746c79b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a6b5e40f0 0x562a6b5e64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:21.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.659+0000 7f1746c79b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6b4c5ac0 con 0x562a6b5d4f60 2023-07-14T18:23:21.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.659+0000 7f1746c79b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6b4c57c0 con 0x562a6b5d58b0 2023-07-14T18:23:21.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.659+0000 7f1746c79b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6b3a9920 con 0x562a6b5e40f0 2023-07-14T18:23:21.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.659+0000 7f1746c79b80 10 client.0 _ll_register_callbacks cb 0x562a6b4c3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:23:21.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.661+0000 7f173099a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a6b5e40f0 0x562a6b5e64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:21.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.661+0000 7f173099a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a6b5e40f0 0x562a6b5e64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:42110/0 (socket says 172.21.15.133:42110) 2023-07-14T18:23:21.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.661+0000 7f173099a700 1 -- 172.21.15.133:0/3746915841 learned_addr learned my addr 172.21.15.133:0/3746915841 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:23:21.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.664+0000 7f173119b700 1 --2- 172.21.15.133:0/3746915841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b5d58b0 0x562a6b5df0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:21.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.664+0000 7f173099a700 1 --2- 172.21.15.133:0/3746915841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a6b5d4f60 0x562a6b5deb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:21.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.665+0000 7f173099a700 1 -- 172.21.15.133:0/3746915841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b5d58b0 msgr2=0x562a6b5df0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:23:21.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.665+0000 7f173099a700 1 --2- 172.21.15.133:0/3746915841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b5d58b0 0x562a6b5df0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.665+0000 7f173099a700 1 -- 172.21.15.133:0/3746915841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a6b5d4f60 msgr2=0x562a6b5deb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:23:21.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.665+0000 7f173099a700 1 --2- 172.21.15.133:0/3746915841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a6b5d4f60 0x562a6b5deb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.665+0000 7f173099a700 1 -- 172.21.15.133:0/3746915841 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171c024550 con 0x562a6b5e40f0 2023-07-14T18:23:21.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.665+0000 7f173099a700 1 --2- 172.21.15.133:0/3746915841 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a6b5e40f0 0x562a6b5e64d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f171c01aba0 tx=0x7f171c01ab30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c61e15ca29dc0619 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:23:21.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.667+0000 7f1714ff9700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:23:21.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.667+0000 7f1714ff9700 1 -- 172.21.15.133:0/3746915841 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c036070 con 0x562a6b5e40f0 2023-07-14T18:23:21.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.667+0000 7f1714ff9700 1 -- 172.21.15.133:0/3746915841 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f171c004830 con 0x562a6b5e40f0 2023-07-14T18:23:21.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.668+0000 7f1746c79b80 1 -- 172.21.15.133:0/3746915841 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a6b47f4f0 con 0x562a6b5e40f0 2023-07-14T18:23:21.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.671+0000 7f1714ff9700 1 -- 172.21.15.133:0/3746915841 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f171c005710 con 0x562a6b5e40f0 2023-07-14T18:23:21.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.693+0000 7f1746c79b80 10 client.25819 fetch_fsmap learned FSMap version 34 2023-07-14T18:23:21.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.693+0000 7f1746c79b80 1 -- 172.21.15.133:0/3746915841 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x562a6b33cc60 con 0x562a6b5e40f0 2023-07-14T18:23:21.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.700+0000 7f1714ff9700 1 -- 172.21.15.133:0/3746915841 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562a6b47f4f0 con 0x562a6b5e40f0 2023-07-14T18:23:21.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.700+0000 7f1714ff9700 1 -- 172.21.15.133:0/3746915841 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f171c039030 con 0x562a6b5e40f0 2023-07-14T18:23:21.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.700+0000 7f1746c79b80 10 client.25819 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T18:23:21.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.700+0000 7f1746c79b80 10 client.25819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:23:21.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.700+0000 7f1746c79b80 20 client.25819 populate_metadata read hostname 'smithi133' 2023-07-14T18:23:21.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.700+0000 7f1746c79b80 1 --2- 172.21.15.133:0/3746915841 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x562a6b5eaa20 0x562a6b5ece30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:21.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.700+0000 7f1746c79b80 4 client.25819 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:23:21.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.700+0000 7f1746c79b80 1 -- 172.21.15.133:0/3746915841 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a6b5a3da0 con 0x562a6b5eaa20 2023-07-14T18:23:21.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.706+0000 7f173119b700 1 --2- 172.21.15.133:0/3746915841 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x562a6b5eaa20 0x562a6b5ece30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:21.814 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.762+0000 7f173119b700 1 --2- 172.21.15.133:0/3746915841 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x562a6b5eaa20 0x562a6b5ece30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad884c7f688e1a8c server_cookie=f251edddf86b26de in_seq=0 out_seq=0 2023-07-14T18:23:21.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.764+0000 7f1714ff9700 10 client.25819 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:23:21.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.770+0000 7f1714ff9700 1 -- 172.21.15.133:0/3746915841 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x562a6b5a3da0 con 0x562a6b5eaa20 2023-07-14T18:23:21.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.770+0000 7f1714ff9700 10 client.25819 handle_command_reply: tid=0 2023-07-14T18:23:21.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.770+0000 7f1746c79b80 1 client.25819 shutdown 2023-07-14T18:23:21.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.781+0000 7f1746c79b80 1 -- 172.21.15.133:0/3746915841 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a6b5e40f0 msgr2=0x562a6b5e64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:23:21.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.781+0000 7f1746c79b80 1 --2- 172.21.15.133:0/3746915841 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a6b5e40f0 0x562a6b5e64d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f171c01aba0 tx=0x7f171c01ab30 comp rx=0 tx=0).stop 2023-07-14T18:23:21.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.786+0000 7f1746c79b80 1 -- 172.21.15.133:0/3746915841 shutdown_connections 2023-07-14T18:23:21.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.786+0000 7f1746c79b80 1 --2- 172.21.15.133:0/3746915841 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x562a6b5eaa20 0x562a6b5ece30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.786+0000 7f1746c79b80 1 --2- 172.21.15.133:0/3746915841 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a6b5e40f0 0x562a6b5e64d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.786+0000 7f1746c79b80 1 --2- 172.21.15.133:0/3746915841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a6b5d4f60 0x562a6b5deb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.786+0000 7f1746c79b80 1 --2- 172.21.15.133:0/3746915841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b5d58b0 0x562a6b5df0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.786+0000 7f1746c79b80 1 -- 172.21.15.133:0/3746915841 >> 172.21.15.133:0/3746915841 conn(0x562a6b5ca680 msgr2=0x562a6b5ce650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:23:21.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.787+0000 7f1714ff9700 0 client.25819 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:23:21.852 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.801+0000 7f1746c79b80 1 -- 172.21.15.133:0/3746915841 shutdown_connections 2023-07-14T18:23:21.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:21 smithi161 ceph-mon[128993]: pgmap v548: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 19 MiB/s wr, 1.50k op/s 2023-07-14T18:23:21.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.803+0000 7f1746c79b80 1 -- 172.21.15.133:0/3746915841 wait complete. 2023-07-14T18:23:21.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.804+0000 7f1746c79b80 20 client.25819 trim_cache size 0 max 16384 2023-07-14T18:23:21.875 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:21 smithi171 ceph-mon[129241]: pgmap v548: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 19 MiB/s wr, 1.50k op/s 2023-07-14T18:23:21.886 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.835+0000 7f1746c79b80 1 Processor -- start 2023-07-14T18:23:21.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.835+0000 7f1746c79b80 1 -- start start 2023-07-14T18:23:21.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.835+0000 7f1746c79b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a6b5d4f60 0x562a6b5fa450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:21.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.835+0000 7f1746c79b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a6b5d58b0 0x562a6b5d89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:21.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.835+0000 7f1746c79b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b5e40f0 0x562a6b5deb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:21.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.835+0000 7f1746c79b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6b5a3da0 con 0x562a6b5d4f60 2023-07-14T18:23:21.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.835+0000 7f1746c79b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6b47f4f0 con 0x562a6b5e40f0 2023-07-14T18:23:21.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.835+0000 7f1746c79b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6b5dfd70 con 0x562a6b5d58b0 2023-07-14T18:23:21.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.836+0000 7f173119b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a6b5d58b0 0x562a6b5d89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:21.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.836+0000 7f173119b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a6b5d58b0 0x562a6b5d89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:42126/0 (socket says 172.21.15.133:42126) 2023-07-14T18:23:21.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.836+0000 7f173119b700 1 -- 172.21.15.133:0/4267568917 learned_addr learned my addr 172.21.15.133:0/4267568917 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:23:21.889 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.838+0000 7f173099a700 1 --2- 172.21.15.133:0/4267568917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b5e40f0 0x562a6b5deb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:21.889 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.838+0000 7f173099a700 1 --2- 172.21.15.133:0/4267568917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a6b5d4f60 0x562a6b5fa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:21.890 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.839+0000 7f173099a700 1 -- 172.21.15.133:0/4267568917 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a6b5d58b0 msgr2=0x562a6b5d89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:23:21.890 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.839+0000 7f173099a700 1 --2- 172.21.15.133:0/4267568917 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a6b5d58b0 0x562a6b5d89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.890 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.839+0000 7f173099a700 1 -- 172.21.15.133:0/4267568917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b5e40f0 msgr2=0x562a6b5deb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:23:21.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.839+0000 7f173099a700 1 --2- 172.21.15.133:0/4267568917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b5e40f0 0x562a6b5deb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.839+0000 7f173099a700 1 -- 172.21.15.133:0/4267568917 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a6b37a250 con 0x562a6b5d4f60 2023-07-14T18:23:21.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.840+0000 7f173099a700 1 --2- 172.21.15.133:0/4267568917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a6b5d4f60 0x562a6b5fa450 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f171c01f650 tx=0x7f171c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84e807ff27675d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:23:21.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.840+0000 7f1717fff700 1 -- 172.21.15.133:0/4267568917 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c036030 con 0x562a6b5d4f60 2023-07-14T18:23:21.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.840+0000 7f1717fff700 1 -- 172.21.15.133:0/4267568917 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f171c005710 con 0x562a6b5d4f60 2023-07-14T18:23:21.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.840+0000 7f1717fff700 1 -- 172.21.15.133:0/4267568917 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c01f3c0 con 0x562a6b5d4f60 2023-07-14T18:23:21.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.841+0000 7f1746c79b80 1 -- 172.21.15.133:0/4267568917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a6b5d4f60 msgr2=0x562a6b5fa450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:23:21.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.841+0000 7f1746c79b80 1 --2- 172.21.15.133:0/4267568917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a6b5d4f60 0x562a6b5fa450 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f171c01f650 tx=0x7f171c000bc0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.845+0000 7f1746c79b80 1 -- 172.21.15.133:0/4267568917 shutdown_connections 2023-07-14T18:23:21.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.845+0000 7f1746c79b80 1 --2- 172.21.15.133:0/4267568917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b5e40f0 0x562a6b5deb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.845+0000 7f1746c79b80 1 --2- 172.21.15.133:0/4267568917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a6b5d4f60 0x562a6b5fa450 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.845+0000 7f1746c79b80 1 --2- 172.21.15.133:0/4267568917 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a6b5d58b0 0x562a6b5d89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.845+0000 7f1746c79b80 1 -- 172.21.15.133:0/4267568917 >> 172.21.15.133:0/4267568917 conn(0x562a6b5ca680 msgr2=0x562a6b5cbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:23:21.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.848+0000 7f1746c79b80 1 -- 172.21.15.133:0/4267568917 shutdown_connections 2023-07-14T18:23:21.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.848+0000 7f1746c79b80 1 -- 172.21.15.133:0/4267568917 wait complete. 2023-07-14T18:23:21.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.849+0000 7f1746c79b80 1 -- start start 2023-07-14T18:23:21.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.849+0000 7f1746c79b80 1 Processor -- start 2023-07-14T18:23:21.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.850+0000 7f1746c79b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a6b5d4f60 0x562a6b5fa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:21.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.850+0000 7f1746c79b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b5d58b0 0x562a6b5f8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:21.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.850+0000 7f1746c79b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a6b5e40f0 0x562a6b5f8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:21.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.850+0000 7f1746c79b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6b5f90e0 con 0x562a6b5d4f60 2023-07-14T18:23:21.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.850+0000 7f1746c79b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6b5f9250 con 0x562a6b5d58b0 2023-07-14T18:23:21.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.850+0000 7f1746c79b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6b5f9410 con 0x562a6b5e40f0 2023-07-14T18:23:21.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.850+0000 7f1746c79b80 10 client.0 _ll_register_callbacks cb 0x7f171c01aa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:23:21.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.851+0000 7f173119b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b5d58b0 0x562a6b5f8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:21.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.851+0000 7f173119b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b5d58b0 0x562a6b5f8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:54892/0 (socket says 172.21.15.133:54892) 2023-07-14T18:23:21.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.851+0000 7f173119b700 1 -- 172.21.15.133:0/2689676127 learned_addr learned my addr 172.21.15.133:0/2689676127 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:23:21.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.851+0000 7f173099a700 1 --2- 172.21.15.133:0/2689676127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a6b5d4f60 0x562a6b5fa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:21.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.852+0000 7f173099a700 1 -- 172.21.15.133:0/2689676127 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a6b5e40f0 msgr2=0x562a6b5f8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:23:21.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.852+0000 7f173099a700 1 --2- 172.21.15.133:0/2689676127 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a6b5e40f0 0x562a6b5f8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.852+0000 7f173099a700 1 -- 172.21.15.133:0/2689676127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b5d58b0 msgr2=0x562a6b5f8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:23:21.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.852+0000 7f173099a700 1 --2- 172.21.15.133:0/2689676127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b5d58b0 0x562a6b5f8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.852+0000 7f173099a700 1 -- 172.21.15.133:0/2689676127 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a6b33cc60 con 0x562a6b5d4f60 2023-07-14T18:23:21.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.853+0000 7f173099a700 1 --2- 172.21.15.133:0/2689676127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a6b5d4f60 0x562a6b5fa050 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f171c024e90 tx=0x7f171c01a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5bbe68a2f4d51bc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:23:21.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.853+0000 7f1714ff9700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T18:23:21.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.854+0000 7f1714ff9700 1 -- 172.21.15.133:0/2689676127 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c036070 con 0x562a6b5d4f60 2023-07-14T18:23:21.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.854+0000 7f1714ff9700 1 -- 172.21.15.133:0/2689676127 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f171c037070 con 0x562a6b5d4f60 2023-07-14T18:23:21.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.854+0000 7f1746c79b80 1 -- 172.21.15.133:0/2689676127 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f171c02a030 con 0x562a6b5d4f60 2023-07-14T18:23:21.906 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.854+0000 7f1714ff9700 1 -- 172.21.15.133:0/2689676127 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f171c024300 con 0x562a6b5d4f60 2023-07-14T18:23:21.906 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.854+0000 7f1746c79b80 10 client.15759 fetch_fsmap learned FSMap version 34 2023-07-14T18:23:21.906 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.854+0000 7f1746c79b80 1 -- 172.21.15.133:0/2689676127 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x562a6b3522b0 con 0x562a6b5d4f60 2023-07-14T18:23:21.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.855+0000 7f1714ff9700 1 -- 172.21.15.133:0/2689676127 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f171c02a030 con 0x562a6b5d4f60 2023-07-14T18:23:21.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.855+0000 7f1714ff9700 1 -- 172.21.15.133:0/2689676127 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f171c039070 con 0x562a6b5d4f60 2023-07-14T18:23:21.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.855+0000 7f1746c79b80 10 client.15759 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T18:23:21.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.855+0000 7f1746c79b80 10 client.15759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:23:21.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.855+0000 7f1746c79b80 20 client.15759 populate_metadata read hostname 'smithi133' 2023-07-14T18:23:21.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.855+0000 7f1746c79b80 1 --2- 172.21.15.133:0/2689676127 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x562a6b63fe50 0x562a6b642260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:21.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.855+0000 7f1746c79b80 4 client.15759 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:23:21.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.856+0000 7f1746c79b80 1 -- 172.21.15.133:0/2689676127 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562a6b51eeb0 con 0x562a6b63fe50 2023-07-14T18:23:21.915 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.864+0000 7f173119b700 1 --2- 172.21.15.133:0/2689676127 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x562a6b63fe50 0x562a6b642260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:21.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.897+0000 7f173119b700 1 --2- 172.21.15.133:0/2689676127 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x562a6b63fe50 0x562a6b642260 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c5c20c2b7675b3d server_cookie=88012a99e9257591 in_seq=0 out_seq=0 2023-07-14T18:23:21.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.900+0000 7f1714ff9700 10 client.15759 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:23:21.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.902+0000 7f1714ff9700 1 -- 172.21.15.133:0/2689676127 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x562a6b51eeb0 con 0x562a6b63fe50 2023-07-14T18:23:21.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.902+0000 7f1714ff9700 10 client.15759 handle_command_reply: tid=0 2023-07-14T18:23:21.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.902+0000 7f1746c79b80 1 client.15759 shutdown 2023-07-14T18:23:21.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.903+0000 7f1746c79b80 1 -- 172.21.15.133:0/2689676127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a6b5d4f60 msgr2=0x562a6b5fa050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:23:21.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.903+0000 7f1746c79b80 1 --2- 172.21.15.133:0/2689676127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a6b5d4f60 0x562a6b5fa050 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f171c024e90 tx=0x7f171c01a540 comp rx=0 tx=0).stop 2023-07-14T18:23:21.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.903+0000 7f1746c79b80 1 -- 172.21.15.133:0/2689676127 shutdown_connections 2023-07-14T18:23:21.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.903+0000 7f1746c79b80 1 --2- 172.21.15.133:0/2689676127 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x562a6b63fe50 0x562a6b642260 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.903+0000 7f1746c79b80 1 --2- 172.21.15.133:0/2689676127 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562a6b5e40f0 0x562a6b5f8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.903+0000 7f1746c79b80 1 --2- 172.21.15.133:0/2689676127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a6b5d4f60 0x562a6b5fa050 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.903+0000 7f1746c79b80 1 --2- 172.21.15.133:0/2689676127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a6b5d58b0 0x562a6b5f8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.903+0000 7f1746c79b80 1 -- 172.21.15.133:0/2689676127 >> 172.21.15.133:0/2689676127 conn(0x562a6b5ca680 msgr2=0x562a6b5d89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:23:21.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.903+0000 7f1714ff9700 0 client.15759 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:23:21.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.912+0000 7f1746c79b80 1 -- 172.21.15.133:0/2689676127 shutdown_connections 2023-07-14T18:23:21.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.914+0000 7f1746c79b80 1 -- 172.21.15.133:0/2689676127 wait complete. 2023-07-14T18:23:21.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.921+0000 7f1746c79b80 20 client.15759 trim_cache size 0 max 16384 2023-07-14T18:23:21.973 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:23:21.973 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-14T18:23:21.973 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:23:21.973 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:23:21.974 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:23:21.974 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:23:21.974 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:23:21.974 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:23:21.974 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:23:21.974 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:23:21.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.934+0000 7f17333ff700 1 -- 172.21.15.133:0/3841125659 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f17180a1350 msgr2=0x7f17180a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:23:21.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.934+0000 7f17333ff700 1 --2- 172.21.15.133:0/3841125659 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f17180a1350 0x7f17180a3800 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1724095db0 tx=0x7f1724004ad0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.934+0000 7f17333ff700 1 -- 172.21.15.133:0/3841125659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f172c076890 msgr2=0x7f172c078d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:23:21.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.934+0000 7f17333ff700 1 --2- 172.21.15.133:0/3841125659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f172c076890 0x7f172c078d40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f172400b970 tx=0x7f172400bd30 comp rx=0 tx=0).stop 2023-07-14T18:23:21.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.940+0000 7f17333ff700 1 -- 172.21.15.133:0/3841125659 shutdown_connections 2023-07-14T18:23:21.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.940+0000 7f17333ff700 1 --2- 172.21.15.133:0/3841125659 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f17180a1350 0x7f17180a3800 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.940+0000 7f17333ff700 1 --2- 172.21.15.133:0/3841125659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f172c076890 0x7f172c078d40 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.940+0000 7f17333ff700 1 --2- 172.21.15.133:0/3841125659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f172c071960 0x7f172c19c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.940+0000 7f17333ff700 1 --2- 172.21.15.133:0/3841125659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f172c10be90 0x7f172c19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:21.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.940+0000 7f17333ff700 1 -- 172.21.15.133:0/3841125659 >> 172.21.15.133:0/3841125659 conn(0x7f172c06cbc0 msgr2=0x7f172c06df80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:23:22.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.955+0000 7f17333ff700 1 -- 172.21.15.133:0/3841125659 shutdown_connections 2023-07-14T18:23:22.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:21.957+0000 7f17333ff700 1 -- 172.21.15.133:0/3841125659 wait complete. 2023-07-14T18:23:22.060 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:23:23.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:23 smithi161 ceph-mon[128993]: pgmap v549: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 20 MiB/s wr, 1.67k op/s 2023-07-14T18:23:23.873 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:23 smithi171 ceph-mon[129241]: pgmap v549: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 20 MiB/s wr, 1.67k op/s 2023-07-14T18:23:23.915 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:23 smithi133 ceph-mon[126384]: pgmap v549: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 20 MiB/s wr, 1.67k op/s 2023-07-14T18:23:25.857 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:25 smithi161 ceph-mon[128993]: pgmap v550: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 24 MiB/s wr, 1.92k op/s 2023-07-14T18:23:25.873 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:25 smithi171 ceph-mon[129241]: pgmap v550: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 24 MiB/s wr, 1.92k op/s 2023-07-14T18:23:25.882 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 240.0 s, 466.4 tps, lat 215.070 ms stddev 421.889 2023-07-14T18:23:25.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:25 smithi133 ceph-mon[126384]: pgmap v550: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 24 MiB/s wr, 1.92k op/s 2023-07-14T18:23:27.857 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:27 smithi161 ceph-mon[128993]: pgmap v551: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 22 MiB/s wr, 1.77k op/s 2023-07-14T18:23:27.873 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:27 smithi171 ceph-mon[129241]: pgmap v551: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 22 MiB/s wr, 1.77k op/s 2023-07-14T18:23:27.913 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:27 smithi133 ceph-mon[126384]: pgmap v551: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 22 MiB/s wr, 1.77k op/s 2023-07-14T18:23:29.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:29 smithi161 ceph-mon[128993]: pgmap v552: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 22 MiB/s wr, 1.75k op/s 2023-07-14T18:23:29.879 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:29 smithi171 ceph-mon[129241]: pgmap v552: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 22 MiB/s wr, 1.75k op/s 2023-07-14T18:23:29.915 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:29 smithi133 ceph-mon[126384]: pgmap v552: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 22 MiB/s wr, 1.75k op/s 2023-07-14T18:23:30.884 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 245.0 s, 1485.0 tps, lat 66.138 ms stddev 49.362 2023-07-14T18:23:31.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:31 smithi161 ceph-mon[128993]: pgmap v553: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 22 MiB/s wr, 1.71k op/s 2023-07-14T18:23:31.875 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:31 smithi171 ceph-mon[129241]: pgmap v553: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 22 MiB/s wr, 1.71k op/s 2023-07-14T18:23:31.915 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:31 smithi133 ceph-mon[126384]: pgmap v553: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 22 MiB/s wr, 1.71k op/s 2023-07-14T18:23:33.857 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:33 smithi161 ceph-mon[128993]: pgmap v554: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 19 MiB/s wr, 1.55k op/s 2023-07-14T18:23:33.875 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:33 smithi171 ceph-mon[129241]: pgmap v554: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 19 MiB/s wr, 1.55k op/s 2023-07-14T18:23:33.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:33 smithi133 ceph-mon[126384]: pgmap v554: 129 pgs: 129 active+clean; 8.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 19 MiB/s wr, 1.55k op/s 2023-07-14T18:23:35.873 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:35 smithi171 ceph-mon[129241]: pgmap v555: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 25 MiB/s wr, 1.99k op/s 2023-07-14T18:23:35.881 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 250.0 s, 676.6 tps, lat 149.640 ms stddev 374.664 2023-07-14T18:23:35.913 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:35 smithi133 ceph-mon[126384]: pgmap v555: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 25 MiB/s wr, 1.99k op/s 2023-07-14T18:23:36.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:35 smithi161 ceph-mon[128993]: pgmap v555: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 25 MiB/s wr, 1.99k op/s 2023-07-14T18:23:38.107 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:37 smithi161 ceph-mon[128993]: pgmap v556: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 23 MiB/s wr, 1.82k op/s 2023-07-14T18:23:38.123 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:37 smithi171 ceph-mon[129241]: pgmap v556: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 23 MiB/s wr, 1.82k op/s 2023-07-14T18:23:38.166 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:37 smithi133 ceph-mon[126384]: pgmap v556: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 23 MiB/s wr, 1.82k op/s 2023-07-14T18:23:40.108 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:39 smithi161 ceph-mon[128993]: pgmap v557: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 24 MiB/s wr, 1.91k op/s 2023-07-14T18:23:40.128 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:39 smithi171 ceph-mon[129241]: pgmap v557: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 24 MiB/s wr, 1.91k op/s 2023-07-14T18:23:40.167 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:39 smithi133 ceph-mon[126384]: pgmap v557: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 24 MiB/s wr, 1.91k op/s 2023-07-14T18:23:40.885 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 255.0 s, 877.8 tps, lat 110.773 ms stddev 270.065 2023-07-14T18:23:41.665 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:41 smithi133 ceph-mon[126384]: pgmap v558: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 26 MiB/s wr, 2.00k op/s 2023-07-14T18:23:41.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:41 smithi161 ceph-mon[128993]: pgmap v558: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 26 MiB/s wr, 2.00k op/s 2023-07-14T18:23:41.874 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:41 smithi171 ceph-mon[129241]: pgmap v558: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 26 MiB/s wr, 2.00k op/s 2023-07-14T18:23:43.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:43 smithi161 ceph-mon[128993]: pgmap v559: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 25 MiB/s wr, 2.05k op/s 2023-07-14T18:23:43.879 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:43 smithi171 ceph-mon[129241]: pgmap v559: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 25 MiB/s wr, 2.05k op/s 2023-07-14T18:23:43.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:43 smithi133 ceph-mon[126384]: pgmap v559: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 25 MiB/s wr, 2.05k op/s 2023-07-14T18:23:45.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:45 smithi161 ceph-mon[128993]: pgmap v560: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 27 MiB/s wr, 2.28k op/s 2023-07-14T18:23:45.875 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:45 smithi171 ceph-mon[129241]: pgmap v560: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 27 MiB/s wr, 2.28k op/s 2023-07-14T18:23:45.882 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 260.0 s, 354.2 tps, lat 289.323 ms stddev 418.086 2023-07-14T18:23:45.917 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:45 smithi133 ceph-mon[126384]: pgmap v560: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 27 MiB/s wr, 2.28k op/s 2023-07-14T18:23:47.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:47 smithi161 ceph-mon[128993]: pgmap v561: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 23 MiB/s wr, 1.84k op/s 2023-07-14T18:23:47.873 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:47 smithi171 ceph-mon[129241]: pgmap v561: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 23 MiB/s wr, 1.84k op/s 2023-07-14T18:23:47.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:47 smithi133 ceph-mon[126384]: pgmap v561: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 23 MiB/s wr, 1.84k op/s 2023-07-14T18:23:49.863 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:49 smithi161 ceph-mon[128993]: pgmap v562: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 20 MiB/s wr, 1.73k op/s 2023-07-14T18:23:49.874 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:49 smithi171 ceph-mon[129241]: pgmap v562: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 20 MiB/s wr, 1.73k op/s 2023-07-14T18:23:49.916 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:49 smithi133 ceph-mon[126384]: pgmap v562: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 20 MiB/s wr, 1.73k op/s 2023-07-14T18:23:50.882 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 265.0 s, 575.1 tps, lat 173.798 ms stddev 209.851 2023-07-14T18:23:51.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:51 smithi133 ceph-mon[126384]: pgmap v563: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 19 MiB/s wr, 1.56k op/s 2023-07-14T18:23:52.061 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:23:52.108 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:51 smithi161 ceph-mon[128993]: pgmap v563: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 19 MiB/s wr, 1.56k op/s 2023-07-14T18:23:52.123 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:51 smithi171 ceph-mon[129241]: pgmap v563: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 19 MiB/s wr, 1.56k op/s 2023-07-14T18:23:52.684 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:52 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:23:52.770 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.714+0000 7f8e189ff700 1 -- 172.21.15.133:0/1644963615 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e1410be90 msgr2=0x7f8e1410e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:23:52.770 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.714+0000 7f8e189ff700 1 --2- 172.21.15.133:0/1644963615 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e1410be90 0x7f8e1410e320 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c01c0d0 tx=0x7f8e0c01c3e0 comp rx=0 tx=0).stop 2023-07-14T18:23:52.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.714+0000 7f8e189ff700 1 -- 172.21.15.133:0/1644963615 shutdown_connections 2023-07-14T18:23:52.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.714+0000 7f8e189ff700 1 --2- 172.21.15.133:0/1644963615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e14071010 0x7f8e14071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:52.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.714+0000 7f8e189ff700 1 --2- 172.21.15.133:0/1644963615 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e1410be90 0x7f8e1410e320 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:52.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.714+0000 7f8e189ff700 1 --2- 172.21.15.133:0/1644963615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e14071960 0x7f8e1410b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:52.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.714+0000 7f8e189ff700 1 -- 172.21.15.133:0/1644963615 >> 172.21.15.133:0/1644963615 conn(0x7f8e1406cbc0 msgr2=0x7f8e1406eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:23:52.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.732+0000 7f8e189ff700 1 -- 172.21.15.133:0/1644963615 shutdown_connections 2023-07-14T18:23:52.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.742+0000 7f8e189ff700 1 -- 172.21.15.133:0/1644963615 wait complete. 2023-07-14T18:23:52.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.743+0000 7f8e189ff700 1 Processor -- start 2023-07-14T18:23:52.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.749+0000 7f8e189ff700 1 -- start start 2023-07-14T18:23:52.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.749+0000 7f8e189ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e14071010 0x7f8e1419e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:52.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.749+0000 7f8e189ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e14071960 0x7f8e141a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:52.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.749+0000 7f8e189ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e1410be90 0x7f8e141a3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:52.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.749+0000 7f8e189ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e141a3b10 con 0x7f8e14071010 2023-07-14T18:23:52.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.749+0000 7f8e189ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e141a3c80 con 0x7f8e1410be90 2023-07-14T18:23:52.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.749+0000 7f8e189ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e141a3e40 con 0x7f8e14071960 2023-07-14T18:23:52.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.751+0000 7f8e12d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e1410be90 0x7f8e141a3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:52.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.751+0000 7f8e12d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e1410be90 0x7f8e141a3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:56692/0 (socket says 172.21.15.133:56692) 2023-07-14T18:23:52.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.751+0000 7f8e12d9d700 1 -- 172.21.15.133:0/3865408191 learned_addr learned my addr 172.21.15.133:0/3865408191 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:23:52.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.753+0000 7f8e11d9b700 1 --2- 172.21.15.133:0/3865408191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e14071960 0x7f8e141a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:52.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.759+0000 7f8e12d9d700 1 -- 172.21.15.133:0/3865408191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e14071960 msgr2=0x7f8e141a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:23:52.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.759+0000 7f8e12d9d700 1 --2- 172.21.15.133:0/3865408191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e14071960 0x7f8e141a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:52.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.759+0000 7f8e12d9d700 1 -- 172.21.15.133:0/3865408191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e14071010 msgr2=0x7f8e1419e8c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T18:23:52.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.759+0000 7f8e12d9d700 1 --2- 172.21.15.133:0/3865408191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e14071010 0x7f8e1419e8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:52.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.759+0000 7f8e12d9d700 1 -- 172.21.15.133:0/3865408191 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0c01ad10 con 0x7f8e1410be90 2023-07-14T18:23:52.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.761+0000 7f8e12d9d700 1 --2- 172.21.15.133:0/3865408191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e1410be90 0x7f8e141a3360 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c01abe0 tx=0x7f8e0c004c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:23:52.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.762+0000 7f8e037fe700 1 -- 172.21.15.133:0/3865408191 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c01f040 con 0x7f8e1410be90 2023-07-14T18:23:52.814 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.762+0000 7f8e037fe700 1 -- 172.21.15.133:0/3865408191 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e0c019ed0 con 0x7f8e1410be90 2023-07-14T18:23:52.814 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.762+0000 7f8e037fe700 1 -- 172.21.15.133:0/3865408191 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c02ea30 con 0x7f8e1410be90 2023-07-14T18:23:52.820 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.762+0000 7f8e189ff700 1 -- 172.21.15.133:0/3865408191 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e141a40c0 con 0x7f8e1410be90 2023-07-14T18:23:52.820 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.762+0000 7f8e189ff700 1 -- 172.21.15.133:0/3865408191 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e141a4490 con 0x7f8e1410be90 2023-07-14T18:23:52.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.771+0000 7f8e2c373b80 1 Processor -- start 2023-07-14T18:23:52.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.781+0000 7f8e2c373b80 1 -- start start 2023-07-14T18:23:52.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.781+0000 7f8e2c373b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89ae81f60 0x55d89ae82370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:52.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.781+0000 7f8e2c373b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d89ae828b0 0x55d89ae82ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:52.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.781+0000 7f8e2c373b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d89ae834b0 0x55d89ae83920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:52.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.782+0000 7f8e2c373b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89ad86cb0 con 0x55d89ae828b0 2023-07-14T18:23:52.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.782+0000 7f8e2c373b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89add9a40 con 0x55d89ae81f60 2023-07-14T18:23:52.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.782+0000 7f8e2c373b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89add2ed0 con 0x55d89ae834b0 2023-07-14T18:23:52.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.782+0000 7f8e037fe700 1 -- 172.21.15.133:0/3865408191 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f8e0c037410 con 0x7f8e1410be90 2023-07-14T18:23:52.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.783+0000 7f8e11d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89ae81f60 0x55d89ae82370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:52.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.783+0000 7f8e11d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89ae81f60 0x55d89ae82370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:56704/0 (socket says 172.21.15.133:56704) 2023-07-14T18:23:52.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.783+0000 7f8e11d9b700 1 -- 172.21.15.133:0/2187703526 learned_addr learned my addr 172.21.15.133:0/2187703526 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:23:52.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.783+0000 7f8e11d9b700 1 --2- 172.21.15.133:0/2187703526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d89ae834b0 0x55d89ae83920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:52.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.783+0000 7f8e11d9b700 1 -- 172.21.15.133:0/2187703526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89ae81f60 msgr2=0x55d89ae82370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:23:52.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.783+0000 7f8e11d9b700 1 --2- 172.21.15.133:0/2187703526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89ae81f60 0x55d89ae82370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:52.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.783+0000 7f8e11d9b700 1 -- 172.21.15.133:0/2187703526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d89ae828b0 msgr2=0x55d89ae82ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T18:23:52.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.784+0000 7f8e11d9b700 1 --2- 172.21.15.133:0/2187703526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d89ae828b0 0x55d89ae82ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:52.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.784+0000 7f8e11d9b700 1 -- 172.21.15.133:0/2187703526 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d89ac00de0 con 0x55d89ae834b0 2023-07-14T18:23:52.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.785+0000 7f8e037fe700 1 --2- 172.21.15.133:0/3865408191 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8dfc0a1350 0x7f8dfc0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:52.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.785+0000 7f8e037fe700 1 -- 172.21.15.133:0/3865408191 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8e0c0dece0 con 0x7f8e1410be90 2023-07-14T18:23:52.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.787+0000 7f8e11d9b700 1 --2- 172.21.15.133:0/2187703526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d89ae834b0 0x55d89ae83920 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8e08017780 tx=0x7f8e08009190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a16fab80cf266aa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:23:52.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.792+0000 7f8e00ff9700 1 -- 172.21.15.133:0/2187703526 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e08017e60 con 0x55d89ae834b0 2023-07-14T18:23:52.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.792+0000 7f8e1259c700 1 --2- 172.21.15.133:0/3865408191 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8dfc0a1350 0x7f8dfc0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:52.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.792+0000 7f8e00ff9700 1 -- 172.21.15.133:0/2187703526 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e08016aa0 con 0x55d89ae834b0 2023-07-14T18:23:52.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.792+0000 7f8e00ff9700 1 -- 172.21.15.133:0/2187703526 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e08018870 con 0x55d89ae834b0 2023-07-14T18:23:52.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.793+0000 7f8e2c373b80 1 -- 172.21.15.133:0/2187703526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d89ae834b0 msgr2=0x55d89ae83920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:23:52.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.793+0000 7f8e2c373b80 1 --2- 172.21.15.133:0/2187703526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d89ae834b0 0x55d89ae83920 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8e08017780 tx=0x7f8e08009190 comp rx=0 tx=0).stop 2023-07-14T18:23:52.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.794+0000 7f8e2c373b80 1 -- 172.21.15.133:0/2187703526 shutdown_connections 2023-07-14T18:23:52.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.794+0000 7f8e2c373b80 1 --2- 172.21.15.133:0/2187703526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89ae81f60 0x55d89ae82370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:52.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.794+0000 7f8e2c373b80 1 --2- 172.21.15.133:0/2187703526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d89ae834b0 0x55d89ae83920 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:52.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.794+0000 7f8e2c373b80 1 --2- 172.21.15.133:0/2187703526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d89ae828b0 0x55d89ae82ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:52.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.794+0000 7f8e2c373b80 1 -- 172.21.15.133:0/2187703526 >> 172.21.15.133:0/2187703526 conn(0x55d89ae77680 msgr2=0x55d89ae7c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:23:52.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.796+0000 7f8e2c373b80 1 -- 172.21.15.133:0/2187703526 shutdown_connections 2023-07-14T18:23:52.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.800+0000 7f8e2c373b80 1 -- 172.21.15.133:0/2187703526 wait complete. 2023-07-14T18:23:52.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.801+0000 7f8e2c373b80 1 -- start start 2023-07-14T18:23:52.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.801+0000 7f8e2c373b80 1 Processor -- start 2023-07-14T18:23:52.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.802+0000 7f8e2c373b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89ae81f60 0x55d89ae8ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:52.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.802+0000 7f8e2c373b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d89ae828b0 0x55d89ae8bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:52.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.802+0000 7f8e2c373b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d89ae834b0 0x55d89ae92fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:52.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.802+0000 7f8e2c373b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89ad72ac0 con 0x55d89ae834b0 2023-07-14T18:23:52.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.802+0000 7f8e2c373b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89ad727c0 con 0x55d89ae81f60 2023-07-14T18:23:52.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.802+0000 7f8e2c373b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89ac56920 con 0x55d89ae828b0 2023-07-14T18:23:52.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.802+0000 7f8e2c373b80 10 client.0 _ll_register_callbacks cb 0x55d89ad70760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:23:52.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.803+0000 7f8e11d9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d89ae828b0 0x55d89ae8bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:52.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.803+0000 7f8e11d9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d89ae828b0 0x55d89ae8bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:44614/0 (socket says 172.21.15.133:44614) 2023-07-14T18:23:52.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.803+0000 7f8e11d9b700 1 -- 172.21.15.133:0/111595578 learned_addr learned my addr 172.21.15.133:0/111595578 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:23:52.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.803+0000 7f8e11d9b700 1 -- 172.21.15.133:0/111595578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89ae81f60 msgr2=0x55d89ae8ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T18:23:52.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.803+0000 7f8e11d9b700 1 --2- 172.21.15.133:0/111595578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89ae81f60 0x55d89ae8ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:52.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.803+0000 7f8e11d9b700 1 -- 172.21.15.133:0/111595578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d89ae834b0 msgr2=0x55d89ae92fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:23:52.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.803+0000 7f8e11d9b700 1 --2- 172.21.15.133:0/111595578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d89ae834b0 0x55d89ae92fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:52.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.803+0000 7f8e11d9b700 1 -- 172.21.15.133:0/111595578 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e08017460 con 0x55d89ae828b0 2023-07-14T18:23:52.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.804+0000 7f8e11d9b700 1 --2- 172.21.15.133:0/111595578 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d89ae828b0 0x55d89ae8bf80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8e08017020 tx=0x7f8e08016e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7453baa1f8b69865 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:23:52.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.804+0000 7f8df9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:23:52.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.804+0000 7f8df9ffb700 1 -- 172.21.15.133:0/111595578 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e080187f0 con 0x55d89ae828b0 2023-07-14T18:23:52.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.804+0000 7f8df9ffb700 1 -- 172.21.15.133:0/111595578 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8e080168b0 con 0x55d89ae828b0 2023-07-14T18:23:52.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.805+0000 7f8e2c373b80 1 -- 172.21.15.133:0/111595578 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d89ad2c4f0 con 0x55d89ae828b0 2023-07-14T18:23:52.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.805+0000 7f8e1259c700 1 --2- 172.21.15.133:0/3865408191 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8dfc0a1350 0x7f8dfc0a3800 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55d89ae82cc0 tx=0x7f8e0400b3f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:23:52.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.805+0000 7f8df9ffb700 1 -- 172.21.15.133:0/111595578 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e08018950 con 0x55d89ae828b0 2023-07-14T18:23:52.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.805+0000 7f8e2c373b80 10 client.25834 fetch_fsmap learned FSMap version 34 2023-07-14T18:23:52.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.805+0000 7f8e2c373b80 1 -- 172.21.15.133:0/111595578 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d89abe9c60 con 0x55d89ae828b0 2023-07-14T18:23:52.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.806+0000 7f8df9ffb700 1 -- 172.21.15.133:0/111595578 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d89ad2c4f0 con 0x55d89ae828b0 2023-07-14T18:23:52.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.806+0000 7f8df9ffb700 1 -- 172.21.15.133:0/111595578 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8e08021c20 con 0x55d89ae828b0 2023-07-14T18:23:52.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.806+0000 7f8e2c373b80 10 client.25834 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T18:23:52.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.806+0000 7f8e2c373b80 10 client.25834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:23:52.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.806+0000 7f8e2c373b80 20 client.25834 populate_metadata read hostname 'smithi133' 2023-07-14T18:23:52.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.806+0000 7f8e2c373b80 1 --2- 172.21.15.133:0/111595578 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55d89ae97660 0x55d89ae99a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:52.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.806+0000 7f8e2c373b80 4 client.25834 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:23:52.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.806+0000 7f8e2c373b80 1 -- 172.21.15.133:0/111595578 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d89ae50da0 con 0x55d89ae97660 2023-07-14T18:23:52.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.807+0000 7f8e1259c700 1 --2- 172.21.15.133:0/111595578 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55d89ae97660 0x55d89ae99a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:52.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.808+0000 7f8df9ffb700 10 client.25834 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:23:52.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.809+0000 7f8e1259c700 1 --2- 172.21.15.133:0/111595578 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55d89ae97660 0x55d89ae99a70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7225d0cd58c9413c server_cookie=7ef97ac7ddf3a04d in_seq=0 out_seq=0 2023-07-14T18:23:52.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.814+0000 7f8df9ffb700 1 -- 172.21.15.133:0/111595578 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55d89ae50da0 con 0x55d89ae97660 2023-07-14T18:23:52.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.814+0000 7f8df9ffb700 10 client.25834 handle_command_reply: tid=0 2023-07-14T18:23:52.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.814+0000 7f8e2c373b80 1 client.25834 shutdown 2023-07-14T18:23:52.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.819+0000 7f8e2c373b80 1 -- 172.21.15.133:0/111595578 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d89ae828b0 msgr2=0x55d89ae8bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:23:52.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.819+0000 7f8e2c373b80 1 --2- 172.21.15.133:0/111595578 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d89ae828b0 0x55d89ae8bf80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8e08017020 tx=0x7f8e08016e40 comp rx=0 tx=0).stop 2023-07-14T18:23:52.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.819+0000 7f8e2c373b80 1 -- 172.21.15.133:0/111595578 shutdown_connections 2023-07-14T18:23:52.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.819+0000 7f8e2c373b80 1 --2- 172.21.15.133:0/111595578 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55d89ae97660 0x55d89ae99a70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:52.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.819+0000 7f8e2c373b80 1 --2- 172.21.15.133:0/111595578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89ae81f60 0x55d89ae8ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:52.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.819+0000 7f8e2c373b80 1 --2- 172.21.15.133:0/111595578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d89ae834b0 0x55d89ae92fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:52.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.819+0000 7f8e2c373b80 1 --2- 172.21.15.133:0/111595578 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d89ae828b0 0x55d89ae8bf80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:52.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.819+0000 7f8e2c373b80 1 -- 172.21.15.133:0/111595578 >> 172.21.15.133:0/111595578 conn(0x55d89ae77680 msgr2=0x55d89ae7b650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:23:52.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.819+0000 7f8df9ffb700 0 client.25834 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:23:52.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.819+0000 7f8e2c373b80 1 -- 172.21.15.133:0/111595578 shutdown_connections 2023-07-14T18:23:52.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.820+0000 7f8e2c373b80 1 -- 172.21.15.133:0/111595578 wait complete. 2023-07-14T18:23:52.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.825+0000 7f8e2c373b80 20 client.25834 trim_cache size 0 max 16384 2023-07-14T18:23:52.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.857+0000 7f8e2c373b80 1 Processor -- start 2023-07-14T18:23:52.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.858+0000 7f8e2c373b80 1 -- start start 2023-07-14T18:23:52.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.858+0000 7f8e2c373b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d89ae81f60 0x55d89ae99670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:52.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.858+0000 7f8e2c373b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d89ae834b0 0x55d89ae92f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:52.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.858+0000 7f8e2c373b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89ae99bb0 0x55d89ae8a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:52.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.858+0000 7f8e2c373b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89ae772a0 con 0x55d89ae81f60 2023-07-14T18:23:52.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.858+0000 7f8e2c373b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89ae7dcc0 con 0x55d89ae99bb0 2023-07-14T18:23:52.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.858+0000 7f8e2c373b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89ae7d8b0 con 0x55d89ae834b0 2023-07-14T18:23:52.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.859+0000 7f8e12d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89ae99bb0 0x55d89ae8a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:52.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.859+0000 7f8e12d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89ae99bb0 0x55d89ae8a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:56716/0 (socket says 172.21.15.133:56716) 2023-07-14T18:23:52.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.859+0000 7f8e12d9d700 1 -- 172.21.15.133:0/570601470 learned_addr learned my addr 172.21.15.133:0/570601470 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:23:52.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.859+0000 7f8e11d9b700 1 --2- 172.21.15.133:0/570601470 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d89ae834b0 0x55d89ae92f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:52.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.860+0000 7f8e12d9d700 1 -- 172.21.15.133:0/570601470 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d89ae834b0 msgr2=0x55d89ae92f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:23:52.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.860+0000 7f8e12d9d700 1 --2- 172.21.15.133:0/570601470 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d89ae834b0 0x55d89ae92f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:52.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.860+0000 7f8e12d9d700 1 -- 172.21.15.133:0/570601470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d89ae81f60 msgr2=0x55d89ae99670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:23:52.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.860+0000 7f8e12d9d700 1 --2- 172.21.15.133:0/570601470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d89ae81f60 0x55d89ae99670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:52.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.860+0000 7f8e12d9d700 1 -- 172.21.15.133:0/570601470 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d89ac27250 con 0x55d89ae99bb0 2023-07-14T18:23:52.914 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.860+0000 7f8e12d9d700 1 --2- 172.21.15.133:0/570601470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89ae99bb0 0x55d89ae8a700 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c018bb0 tx=0x7f8e0c0abd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bce6ed3b4628d4a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:23:52.914 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.861+0000 7f8e017fa700 1 -- 172.21.15.133:0/570601470 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c0abe70 con 0x55d89ae99bb0 2023-07-14T18:23:52.914 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.861+0000 7f8e017fa700 1 -- 172.21.15.133:0/570601470 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e0c0afea0 con 0x55d89ae99bb0 2023-07-14T18:23:52.914 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.861+0000 7f8e017fa700 1 -- 172.21.15.133:0/570601470 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c0ab5c0 con 0x55d89ae99bb0 2023-07-14T18:23:52.915 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.862+0000 7f8e2c373b80 1 -- 172.21.15.133:0/570601470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89ae99bb0 msgr2=0x55d89ae8a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:23:52.915 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.862+0000 7f8e2c373b80 1 --2- 172.21.15.133:0/570601470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89ae99bb0 0x55d89ae8a700 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c018bb0 tx=0x7f8e0c0abd10 comp rx=0 tx=0).stop 2023-07-14T18:23:52.915 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.863+0000 7f8e2c373b80 1 -- 172.21.15.133:0/570601470 shutdown_connections 2023-07-14T18:23:52.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.863+0000 7f8e2c373b80 1 --2- 172.21.15.133:0/570601470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89ae99bb0 0x55d89ae8a700 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:52.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.863+0000 7f8e2c373b80 1 --2- 172.21.15.133:0/570601470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d89ae81f60 0x55d89ae99670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:52.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.863+0000 7f8e2c373b80 1 --2- 172.21.15.133:0/570601470 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d89ae834b0 0x55d89ae92f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:52.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.863+0000 7f8e2c373b80 1 -- 172.21.15.133:0/570601470 >> 172.21.15.133:0/570601470 conn(0x55d89ae77680 msgr2=0x55d89ae898a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:23:52.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.863+0000 7f8e2c373b80 1 -- 172.21.15.133:0/570601470 shutdown_connections 2023-07-14T18:23:52.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.863+0000 7f8e2c373b80 1 -- 172.21.15.133:0/570601470 wait complete. 2023-07-14T18:23:52.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.863+0000 7f8e2c373b80 1 -- start start 2023-07-14T18:23:52.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.866+0000 7f8e2c373b80 1 Processor -- start 2023-07-14T18:23:52.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.866+0000 7f8e2c373b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d89ae81f60 0x55d89ae92980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:52.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.866+0000 7f8e2c373b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89ae834b0 0x55d89ae92ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:52.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.866+0000 7f8e2c373b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d89ae99bb0 0x55d89ae8f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:52.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.866+0000 7f8e2c373b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89ae8fb90 con 0x55d89ae81f60 2023-07-14T18:23:52.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.866+0000 7f8e2c373b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89ae8fd00 con 0x55d89ae834b0 2023-07-14T18:23:52.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.866+0000 7f8e2c373b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89ae8fec0 con 0x55d89ae99bb0 2023-07-14T18:23:52.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.866+0000 7f8e2c373b80 10 client.0 _ll_register_callbacks cb 0x7f8e080098a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:23:52.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.880+0000 7f8e12d9d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d89ae99bb0 0x55d89ae8f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:52.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.880+0000 7f8e12d9d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d89ae99bb0 0x55d89ae8f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:44630/0 (socket says 172.21.15.133:44630) 2023-07-14T18:23:52.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.880+0000 7f8e12d9d700 1 -- 172.21.15.133:0/1683330164 learned_addr learned my addr 172.21.15.133:0/1683330164 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:23:52.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.886+0000 7f8e11d9b700 1 --2- 172.21.15.133:0/1683330164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89ae834b0 0x55d89ae92ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:52.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.887+0000 7f8e1259c700 1 --2- 172.21.15.133:0/1683330164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d89ae81f60 0x55d89ae92980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:52.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.897+0000 7f8e12d9d700 1 -- 172.21.15.133:0/1683330164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89ae834b0 msgr2=0x55d89ae92ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:23:52.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.897+0000 7f8e12d9d700 1 --2- 172.21.15.133:0/1683330164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89ae834b0 0x55d89ae92ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:52.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.897+0000 7f8e12d9d700 1 -- 172.21.15.133:0/1683330164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d89ae81f60 msgr2=0x55d89ae92980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:23:52.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.897+0000 7f8e12d9d700 1 --2- 172.21.15.133:0/1683330164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d89ae81f60 0x55d89ae92980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:52.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.897+0000 7f8e12d9d700 1 -- 172.21.15.133:0/1683330164 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e141a4490 con 0x55d89ae99bb0 2023-07-14T18:23:52.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.908+0000 7f8e12d9d700 1 --2- 172.21.15.133:0/1683330164 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d89ae99bb0 0x55d89ae8f480 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0b3d00 tx=0x7f8e0c018560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e54b0e2eaff7a246 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:23:52.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.909+0000 7f8df9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:23:52.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.920+0000 7f8df9ffb700 1 -- 172.21.15.133:0/1683330164 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c0b3fa0 con 0x55d89ae99bb0 2023-07-14T18:23:52.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.920+0000 7f8df9ffb700 1 -- 172.21.15.133:0/1683330164 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8e0c0abe70 con 0x55d89ae99bb0 2023-07-14T18:23:52.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.920+0000 7f8e2c373b80 1 -- 172.21.15.133:0/1683330164 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e08016aa0 con 0x55d89ae99bb0 2023-07-14T18:23:52.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.933+0000 7f8df9ffb700 1 -- 172.21.15.133:0/1683330164 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e0c0dc2b0 con 0x55d89ae99bb0 2023-07-14T18:23:53.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.944+0000 7f8e2c373b80 10 client.25840 fetch_fsmap learned FSMap version 34 2023-07-14T18:23:53.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.944+0000 7f8e2c373b80 1 -- 172.21.15.133:0/1683330164 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d89abff2b0 con 0x55d89ae99bb0 2023-07-14T18:23:53.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.950+0000 7f8df9ffb700 1 -- 172.21.15.133:0/1683330164 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e08016aa0 con 0x55d89ae99bb0 2023-07-14T18:23:53.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.950+0000 7f8df9ffb700 1 -- 172.21.15.133:0/1683330164 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8e0c02ad70 con 0x55d89ae99bb0 2023-07-14T18:23:53.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.956+0000 7f8e2c373b80 10 client.25840 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T18:23:53.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.956+0000 7f8e2c373b80 10 client.25840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:23:53.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.956+0000 7f8e2c373b80 20 client.25840 populate_metadata read hostname 'smithi133' 2023-07-14T18:23:53.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.956+0000 7f8e2c373b80 1 --2- 172.21.15.133:0/1683330164 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55d89aeec3d0 0x55d89aeee7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:23:53.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.956+0000 7f8e2c373b80 4 client.25840 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:23:53.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.956+0000 7f8e2c373b80 1 -- 172.21.15.133:0/1683330164 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d89adcbeb0 con 0x55d89aeec3d0 2023-07-14T18:23:53.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.961+0000 7f8e1259c700 1 --2- 172.21.15.133:0/1683330164 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55d89aeec3d0 0x55d89aeee7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:23:53.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.964+0000 7f8e1259c700 1 --2- 172.21.15.133:0/1683330164 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55d89aeec3d0 0x55d89aeee7e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8549e91d59a98da5 server_cookie=d6abf3dc8b81bfed in_seq=0 out_seq=0 2023-07-14T18:23:53.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.965+0000 7f8df9ffb700 10 client.25840 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:23:53.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.974+0000 7f8df9ffb700 1 -- 172.21.15.133:0/1683330164 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55d89adcbeb0 con 0x55d89aeec3d0 2023-07-14T18:23:53.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.974+0000 7f8df9ffb700 10 client.25840 handle_command_reply: tid=0 2023-07-14T18:23:53.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.974+0000 7f8e2c373b80 1 client.25840 shutdown 2023-07-14T18:23:53.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.976+0000 7f8e2c373b80 1 -- 172.21.15.133:0/1683330164 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d89ae99bb0 msgr2=0x55d89ae8f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:23:53.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.976+0000 7f8e2c373b80 1 --2- 172.21.15.133:0/1683330164 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d89ae99bb0 0x55d89ae8f480 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0b3d00 tx=0x7f8e0c018560 comp rx=0 tx=0).stop 2023-07-14T18:23:53.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.977+0000 7f8e2c373b80 1 -- 172.21.15.133:0/1683330164 shutdown_connections 2023-07-14T18:23:53.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.977+0000 7f8e2c373b80 1 --2- 172.21.15.133:0/1683330164 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55d89aeec3d0 0x55d89aeee7e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:53.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.977+0000 7f8e2c373b80 1 --2- 172.21.15.133:0/1683330164 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d89ae99bb0 0x55d89ae8f480 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:53.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.977+0000 7f8e2c373b80 1 --2- 172.21.15.133:0/1683330164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d89ae81f60 0x55d89ae92980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:53.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.977+0000 7f8e2c373b80 1 --2- 172.21.15.133:0/1683330164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89ae834b0 0x55d89ae92ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:53.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.977+0000 7f8e2c373b80 1 -- 172.21.15.133:0/1683330164 >> 172.21.15.133:0/1683330164 conn(0x55d89ae77680 msgr2=0x55d89ae898a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:23:53.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.977+0000 7f8df9ffb700 0 client.25840 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:23:53.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.993+0000 7f8e2c373b80 1 -- 172.21.15.133:0/1683330164 shutdown_connections 2023-07-14T18:23:53.050 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:23:53.051 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-14T18:23:53.051 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:23:53.051 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:23:53.051 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:23:53.051 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:23:53.051 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:23:53.051 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:23:53.051 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:23:53.052 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:23:53.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.997+0000 7f8e2c373b80 1 -- 172.21.15.133:0/1683330164 wait complete. 2023-07-14T18:23:53.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:52.998+0000 7f8e2c373b80 20 client.25840 trim_cache size 0 max 16384 2023-07-14T18:23:53.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:53.005+0000 7f8e189ff700 1 -- 172.21.15.133:0/3865408191 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8dfc0a1350 msgr2=0x7f8dfc0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:23:53.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:53.005+0000 7f8e189ff700 1 --2- 172.21.15.133:0/3865408191 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8dfc0a1350 0x7f8dfc0a3800 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55d89ae82cc0 tx=0x7f8e0400b3f0 comp rx=0 tx=0).stop 2023-07-14T18:23:53.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:53.005+0000 7f8e189ff700 1 -- 172.21.15.133:0/3865408191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e1410be90 msgr2=0x7f8e141a3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:23:53.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:53.005+0000 7f8e189ff700 1 --2- 172.21.15.133:0/3865408191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e1410be90 0x7f8e141a3360 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c01abe0 tx=0x7f8e0c004c40 comp rx=0 tx=0).stop 2023-07-14T18:23:53.068 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:53.013+0000 7f8e189ff700 1 -- 172.21.15.133:0/3865408191 shutdown_connections 2023-07-14T18:23:53.068 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:53.013+0000 7f8e189ff700 1 --2- 172.21.15.133:0/3865408191 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f8dfc0a1350 0x7f8dfc0a3800 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:53.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:53.013+0000 7f8e189ff700 1 --2- 172.21.15.133:0/3865408191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e1410be90 0x7f8e141a3360 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:53.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:53.013+0000 7f8e189ff700 1 --2- 172.21.15.133:0/3865408191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e14071010 0x7f8e1419e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:53.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:53.013+0000 7f8e189ff700 1 --2- 172.21.15.133:0/3865408191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e14071960 0x7f8e141a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:23:53.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:53.013+0000 7f8e189ff700 1 -- 172.21.15.133:0/3865408191 >> 172.21.15.133:0/3865408191 conn(0x7f8e1406cbc0 msgr2=0x7f8e1406dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:23:53.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:53.017+0000 7f8e189ff700 1 -- 172.21.15.133:0/3865408191 shutdown_connections 2023-07-14T18:23:53.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:23:53.017+0000 7f8e189ff700 1 -- 172.21.15.133:0/3865408191 wait complete. 2023-07-14T18:23:53.108 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:52 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:23:53.112 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:23:53.123 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:52 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:23:54.108 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:53 smithi161 ceph-mon[128993]: pgmap v564: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 17 MiB/s wr, 1.49k op/s 2023-07-14T18:23:54.108 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:53 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:23:54.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:53 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:23:54.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:53 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:23:54.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:53 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:23:54.126 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:53 smithi171 ceph-mon[129241]: pgmap v564: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 17 MiB/s wr, 1.49k op/s 2023-07-14T18:23:54.127 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:53 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:23:54.127 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:53 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:23:54.127 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:53 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:23:54.127 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:53 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:23:54.164 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:53 smithi133 ceph-mon[126384]: pgmap v564: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 17 MiB/s wr, 1.49k op/s 2023-07-14T18:23:54.165 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:53 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:23:54.165 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:53 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:23:54.165 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:53 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:23:54.165 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:53 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:23:55.108 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:54 smithi161 ceph-mon[128993]: Adjusting osd_memory_target on smithi161 to 196.2M 2023-07-14T18:23:55.108 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:54 smithi161 ceph-mon[128993]: Unable to set osd_memory_target on smithi161 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-14T18:23:55.123 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:54 smithi171 ceph-mon[129241]: Adjusting osd_memory_target on smithi161 to 196.2M 2023-07-14T18:23:55.123 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:54 smithi171 ceph-mon[129241]: Unable to set osd_memory_target on smithi161 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-14T18:23:55.165 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:54 smithi133 ceph-mon[126384]: Adjusting osd_memory_target on smithi161 to 196.2M 2023-07-14T18:23:55.165 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:54 smithi133 ceph-mon[126384]: Unable to set osd_memory_target on smithi161 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-14T18:23:55.885 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 270.0 s, 1001.2 tps, lat 98.903 ms stddev 144.375 2023-07-14T18:23:56.108 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:55 smithi161 ceph-mon[128993]: pgmap v565: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 17 MiB/s wr, 1.48k op/s 2023-07-14T18:23:56.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:55 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:23:56.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:55 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:23:56.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:55 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:23:56.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:55 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:23:56.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:55 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:23:56.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:55 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:23:56.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:55 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:23:56.123 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:55 smithi171 ceph-mon[129241]: pgmap v565: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 17 MiB/s wr, 1.48k op/s 2023-07-14T18:23:56.123 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:55 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:23:56.123 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:55 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:23:56.124 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:55 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:23:56.124 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:55 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:23:56.124 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:55 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:23:56.124 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:55 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:23:56.124 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:55 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:23:56.164 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:55 smithi133 ceph-mon[126384]: pgmap v565: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 17 MiB/s wr, 1.48k op/s 2023-07-14T18:23:56.164 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:55 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:23:56.165 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:55 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:23:56.165 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:55 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:23:56.165 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:55 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:23:56.166 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:55 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:23:56.166 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:55 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:23:56.166 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:55 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:23:57.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:56 smithi161 ceph-mon[128993]: Adjusting osd_memory_target on smithi171 to 196.2M 2023-07-14T18:23:57.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:56 smithi161 ceph-mon[128993]: Unable to set osd_memory_target on smithi171 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-14T18:23:57.125 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:56 smithi171 ceph-mon[129241]: Adjusting osd_memory_target on smithi171 to 196.2M 2023-07-14T18:23:57.125 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:56 smithi171 ceph-mon[129241]: Unable to set osd_memory_target on smithi171 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-14T18:23:57.164 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:56 smithi133 ceph-mon[126384]: Adjusting osd_memory_target on smithi171 to 196.2M 2023-07-14T18:23:57.165 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:56 smithi133 ceph-mon[126384]: Unable to set osd_memory_target on smithi171 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-14T18:23:58.112 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:57 smithi161 ceph-mon[128993]: pgmap v566: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 16 MiB/s wr, 1.27k op/s 2023-07-14T18:23:58.124 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:57 smithi171 ceph-mon[129241]: pgmap v566: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 16 MiB/s wr, 1.27k op/s 2023-07-14T18:23:58.165 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:57 smithi133 ceph-mon[126384]: pgmap v566: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 16 MiB/s wr, 1.27k op/s 2023-07-14T18:23:59.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:23:58 smithi161 ceph-mon[128993]: pgmap v567: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 16 MiB/s wr, 1.13k op/s 2023-07-14T18:23:59.123 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:23:58 smithi171 ceph-mon[129241]: pgmap v567: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 16 MiB/s wr, 1.13k op/s 2023-07-14T18:23:59.167 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:23:58 smithi133 ceph-mon[126384]: pgmap v567: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 16 MiB/s wr, 1.13k op/s 2023-07-14T18:24:00.886 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 275.0 s, 1137.5 tps, lat 78.192 ms stddev 62.382 2023-07-14T18:24:01.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:01 smithi161 ceph-mon[128993]: pgmap v568: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 17 MiB/s wr, 1.06k op/s 2023-07-14T18:24:01.873 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:01 smithi171 ceph-mon[129241]: pgmap v568: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 17 MiB/s wr, 1.06k op/s 2023-07-14T18:24:01.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:01 smithi133 ceph-mon[126384]: pgmap v568: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 17 MiB/s wr, 1.06k op/s 2023-07-14T18:24:03.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:03 smithi161 ceph-mon[128993]: pgmap v569: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 19 MiB/s wr, 1.23k op/s 2023-07-14T18:24:03.873 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:03 smithi171 ceph-mon[129241]: pgmap v569: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 19 MiB/s wr, 1.23k op/s 2023-07-14T18:24:03.915 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:03 smithi133 ceph-mon[126384]: pgmap v569: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 19 MiB/s wr, 1.23k op/s 2023-07-14T18:24:05.866 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:05 smithi161 ceph-mon[128993]: pgmap v570: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 24 MiB/s wr, 1.73k op/s 2023-07-14T18:24:05.873 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:05 smithi171 ceph-mon[129241]: pgmap v570: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 24 MiB/s wr, 1.73k op/s 2023-07-14T18:24:05.885 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 280.0 s, 629.5 tps, lat 177.842 ms stddev 407.593 2023-07-14T18:24:05.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:05 smithi133 ceph-mon[126384]: pgmap v570: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 24 MiB/s wr, 1.73k op/s 2023-07-14T18:24:07.875 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:07 smithi171 ceph-mon[129241]: pgmap v571: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 23 MiB/s wr, 1.70k op/s 2023-07-14T18:24:07.915 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:07 smithi133 ceph-mon[126384]: pgmap v571: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 23 MiB/s wr, 1.70k op/s 2023-07-14T18:24:08.108 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:07 smithi161 ceph-mon[128993]: pgmap v571: 129 pgs: 129 active+clean; 8.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 23 MiB/s wr, 1.70k op/s 2023-07-14T18:24:09.915 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:09 smithi133 ceph-mon[126384]: pgmap v572: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 26 MiB/s wr, 1.88k op/s 2023-07-14T18:24:10.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:09 smithi161 ceph-mon[128993]: pgmap v572: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 26 MiB/s wr, 1.88k op/s 2023-07-14T18:24:10.124 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:09 smithi171 ceph-mon[129241]: pgmap v572: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 26 MiB/s wr, 1.88k op/s 2023-07-14T18:24:10.882 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 285.0 s, 1471.8 tps, lat 66.747 ms stddev 47.612 2023-07-14T18:24:12.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:11 smithi161 ceph-mon[128993]: pgmap v573: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 25 MiB/s wr, 1.84k op/s 2023-07-14T18:24:12.124 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:11 smithi171 ceph-mon[129241]: pgmap v573: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 25 MiB/s wr, 1.84k op/s 2023-07-14T18:24:12.165 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:11 smithi133 ceph-mon[126384]: pgmap v573: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 25 MiB/s wr, 1.84k op/s 2023-07-14T18:24:14.111 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:13 smithi161 ceph-mon[128993]: pgmap v574: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 22 MiB/s wr, 1.71k op/s 2023-07-14T18:24:14.124 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:13 smithi171 ceph-mon[129241]: pgmap v574: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 22 MiB/s wr, 1.71k op/s 2023-07-14T18:24:14.164 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:13 smithi133 ceph-mon[126384]: pgmap v574: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 22 MiB/s wr, 1.71k op/s 2023-07-14T18:24:15.883 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 290.0 s, 344.4 tps, lat 274.819 ms stddev 668.771 2023-07-14T18:24:16.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:15 smithi161 ceph-mon[128993]: pgmap v575: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 29 MiB/s wr, 2.48k op/s 2023-07-14T18:24:16.126 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:15 smithi171 ceph-mon[129241]: pgmap v575: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 29 MiB/s wr, 2.48k op/s 2023-07-14T18:24:16.166 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:15 smithi133 ceph-mon[126384]: pgmap v575: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 29 MiB/s wr, 2.48k op/s 2023-07-14T18:24:17.167 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:16 smithi133 ceph-mon[126384]: pgmap v576: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 25 MiB/s wr, 2.02k op/s 2023-07-14T18:24:17.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:16 smithi161 ceph-mon[128993]: pgmap v576: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 25 MiB/s wr, 2.02k op/s 2023-07-14T18:24:17.374 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:16 smithi171 ceph-mon[129241]: pgmap v576: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 25 MiB/s wr, 2.02k op/s 2023-07-14T18:24:19.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:18 smithi161 ceph-mon[128993]: pgmap v577: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 26 MiB/s wr, 2.00k op/s 2023-07-14T18:24:19.375 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:18 smithi171 ceph-mon[129241]: pgmap v577: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 26 MiB/s wr, 2.00k op/s 2023-07-14T18:24:19.415 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:18 smithi133 ceph-mon[126384]: pgmap v577: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 26 MiB/s wr, 2.00k op/s 2023-07-14T18:24:20.883 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 295.0 s, 1077.4 tps, lat 95.447 ms stddev 139.625 2023-07-14T18:24:21.866 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:21 smithi161 ceph-mon[128993]: pgmap v578: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 23 MiB/s wr, 1.92k op/s 2023-07-14T18:24:21.874 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:21 smithi171 ceph-mon[129241]: pgmap v578: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 23 MiB/s wr, 1.92k op/s 2023-07-14T18:24:21.915 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:21 smithi133 ceph-mon[126384]: pgmap v578: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 23 MiB/s wr, 1.92k op/s 2023-07-14T18:24:23.113 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:24:23.560 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:23 smithi133 ceph-mon[126384]: pgmap v579: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 21 MiB/s wr, 1.83k op/s 2023-07-14T18:24:23.721 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.667+0000 7f12757ff700 1 -- 172.21.15.133:0/1185937621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f127010be90 msgr2=0x7f127010e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:24:23.721 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.667+0000 7f12757ff700 1 --2- 172.21.15.133:0/1185937621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f127010be90 0x7f127010e320 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f12600098f0 tx=0x7f1260009c00 comp rx=0 tx=0).stop 2023-07-14T18:24:23.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.689+0000 7f12757ff700 1 -- 172.21.15.133:0/1185937621 shutdown_connections 2023-07-14T18:24:23.741 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.689+0000 7f12757ff700 1 --2- 172.21.15.133:0/1185937621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f127010be90 0x7f127010e320 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:23.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.689+0000 7f12757ff700 1 --2- 172.21.15.133:0/1185937621 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1270071010 0x7f1270071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:23.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.689+0000 7f12757ff700 1 --2- 172.21.15.133:0/1185937621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1270071960 0x7f127010b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:23.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.689+0000 7f12757ff700 1 -- 172.21.15.133:0/1185937621 >> 172.21.15.133:0/1185937621 conn(0x7f127006cbc0 msgr2=0x7f127006eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:24:23.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.702+0000 7f12757ff700 1 -- 172.21.15.133:0/1185937621 shutdown_connections 2023-07-14T18:24:23.764 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.711+0000 7f12757ff700 1 -- 172.21.15.133:0/1185937621 wait complete. 2023-07-14T18:24:23.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.712+0000 7f12757ff700 1 Processor -- start 2023-07-14T18:24:23.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.715+0000 7f12757ff700 1 -- start start 2023-07-14T18:24:23.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.715+0000 7f12757ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1270071010 0x7f1270114d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:23.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.715+0000 7f12757ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1270071960 0x7f1270115260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:23.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.715+0000 7f12757ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f127010be90 0x7f12701162a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:23.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.715+0000 7f12757ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1270116870 con 0x7f1270071010 2023-07-14T18:24:23.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.715+0000 7f12757ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12701169e0 con 0x7f127010be90 2023-07-14T18:24:23.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.715+0000 7f12757ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1270116ba0 con 0x7f1270071960 2023-07-14T18:24:23.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.721+0000 7f126f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f127010be90 0x7f12701162a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:23.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.721+0000 7f126f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f127010be90 0x7f12701162a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:54150/0 (socket says 172.21.15.133:54150) 2023-07-14T18:24:23.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.721+0000 7f126f7fe700 1 -- 172.21.15.133:0/898653844 learned_addr learned my addr 172.21.15.133:0/898653844 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:24:23.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.721+0000 7f126e7fc700 1 --2- 172.21.15.133:0/898653844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1270071960 0x7f1270115260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:23.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.725+0000 7f126f7fe700 1 -- 172.21.15.133:0/898653844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1270071960 msgr2=0x7f1270115260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:24:23.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.725+0000 7f126f7fe700 1 --2- 172.21.15.133:0/898653844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1270071960 0x7f1270115260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:23.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.725+0000 7f126f7fe700 1 -- 172.21.15.133:0/898653844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1270071010 msgr2=0x7f1270114d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:24:23.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.725+0000 7f126f7fe700 1 --2- 172.21.15.133:0/898653844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1270071010 0x7f1270114d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:23.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.725+0000 7f126f7fe700 1 -- 172.21.15.133:0/898653844 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1260009520 con 0x7f127010be90 2023-07-14T18:24:23.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.740+0000 7f126f7fe700 1 --2- 172.21.15.133:0/898653844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f127010be90 0x7f12701162a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f126000fa40 tx=0x7f126000b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:24:23.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.751+0000 7f1267fff700 1 -- 172.21.15.133:0/898653844 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126001c070 con 0x7f127010be90 2023-07-14T18:24:23.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.751+0000 7f1267fff700 1 -- 172.21.15.133:0/898653844 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f126000f840 con 0x7f127010be90 2023-07-14T18:24:23.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.751+0000 7f1267fff700 1 -- 172.21.15.133:0/898653844 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1260003b20 con 0x7f127010be90 2023-07-14T18:24:23.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.751+0000 7f12757ff700 1 -- 172.21.15.133:0/898653844 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1270116dc0 con 0x7f127010be90 2023-07-14T18:24:23.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.751+0000 7f12757ff700 1 -- 172.21.15.133:0/898653844 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1270117220 con 0x7f127010be90 2023-07-14T18:24:23.814 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.757+0000 7f12890f4b80 1 Processor -- start 2023-07-14T18:24:23.814 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.758+0000 7f12890f4b80 1 -- start start 2023-07-14T18:24:23.814 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.758+0000 7f12890f4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562741cc3f60 0x562741cc4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:23.815 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.758+0000 7f12890f4b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562741cc48b0 0x562741cc4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:23.815 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.758+0000 7f12890f4b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562741cc54b0 0x562741cc5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:23.815 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.758+0000 7f12890f4b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562741bc8cb0 con 0x562741cc48b0 2023-07-14T18:24:23.815 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.758+0000 7f12890f4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562741c1ba40 con 0x562741cc3f60 2023-07-14T18:24:23.816 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.758+0000 7f12890f4b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x562741c14ed0 con 0x562741cc54b0 2023-07-14T18:24:23.817 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.764+0000 7f126effd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562741cc48b0 0x562741cc4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:23.818 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.764+0000 7f126effd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562741cc48b0 0x562741cc4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50396/0 (socket says 172.21.15.133:50396) 2023-07-14T18:24:23.818 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.764+0000 7f126effd700 1 -- 172.21.15.133:0/387926018 learned_addr learned my addr 172.21.15.133:0/387926018 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:24:23.818 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.765+0000 7f126e7fc700 1 --2- 172.21.15.133:0/387926018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562741cc54b0 0x562741cc5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:23.819 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.765+0000 7f126e7fc700 1 --2- 172.21.15.133:0/387926018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562741cc3f60 0x562741cc4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:23.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.777+0000 7f126e7fc700 1 -- 172.21.15.133:0/387926018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562741cc3f60 msgr2=0x562741cc4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:24:23.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.777+0000 7f126e7fc700 1 --2- 172.21.15.133:0/387926018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562741cc3f60 0x562741cc4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:23.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.777+0000 7f126e7fc700 1 -- 172.21.15.133:0/387926018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562741cc48b0 msgr2=0x562741cc4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:24:23.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.777+0000 7f126e7fc700 1 --2- 172.21.15.133:0/387926018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562741cc48b0 0x562741cc4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:23.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.777+0000 7f126e7fc700 1 -- 172.21.15.133:0/387926018 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562741a42de0 con 0x562741cc54b0 2023-07-14T18:24:23.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.780+0000 7f1267fff700 1 -- 172.21.15.133:0/898653844 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f1260020030 con 0x7f127010be90 2023-07-14T18:24:23.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.782+0000 7f1267fff700 1 --2- 172.21.15.133:0/898653844 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f125c0a1350 0x7f125c0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:23.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.782+0000 7f1267fff700 1 -- 172.21.15.133:0/898653844 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f12600ce2a0 con 0x7f127010be90 2023-07-14T18:24:23.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.787+0000 7f126e7fc700 1 --2- 172.21.15.133:0/387926018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562741cc54b0 0x562741cc5920 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1270072670 tx=0x7f12680099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6896251b11ec6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:24:23.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.795+0000 7f126effd700 1 --2- 172.21.15.133:0/898653844 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f125c0a1350 0x7f125c0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:23.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:23 smithi161 ceph-mon[128993]: pgmap v579: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 21 MiB/s wr, 1.83k op/s 2023-07-14T18:24:23.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.808+0000 7f12657fa700 1 -- 172.21.15.133:0/387926018 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126800e040 con 0x562741cc54b0 2023-07-14T18:24:23.865 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.808+0000 7f12657fa700 1 -- 172.21.15.133:0/387926018 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1268004db0 con 0x562741cc54b0 2023-07-14T18:24:23.865 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.808+0000 7f12657fa700 1 -- 172.21.15.133:0/387926018 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1268003cf0 con 0x562741cc54b0 2023-07-14T18:24:23.865 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.812+0000 7f12890f4b80 1 -- 172.21.15.133:0/387926018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562741cc54b0 msgr2=0x562741cc5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:24:23.865 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.812+0000 7f12890f4b80 1 --2- 172.21.15.133:0/387926018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562741cc54b0 0x562741cc5920 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1270072670 tx=0x7f12680099e0 comp rx=0 tx=0).stop 2023-07-14T18:24:23.865 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.813+0000 7f12890f4b80 1 -- 172.21.15.133:0/387926018 shutdown_connections 2023-07-14T18:24:23.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.813+0000 7f12890f4b80 1 --2- 172.21.15.133:0/387926018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562741cc3f60 0x562741cc4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:23.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.813+0000 7f12890f4b80 1 --2- 172.21.15.133:0/387926018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562741cc54b0 0x562741cc5920 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:23.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.813+0000 7f12890f4b80 1 --2- 172.21.15.133:0/387926018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562741cc48b0 0x562741cc4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:23.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.813+0000 7f12890f4b80 1 -- 172.21.15.133:0/387926018 >> 172.21.15.133:0/387926018 conn(0x562741cb9680 msgr2=0x562741cbe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:24:23.874 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:23 smithi171 ceph-mon[129241]: pgmap v579: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 21 MiB/s wr, 1.83k op/s 2023-07-14T18:24:23.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.824+0000 7f12890f4b80 1 -- 172.21.15.133:0/387926018 shutdown_connections 2023-07-14T18:24:23.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.830+0000 7f126effd700 1 --2- 172.21.15.133:0/898653844 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f125c0a1350 0x7f125c0a3800 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x562741cc4cc0 tx=0x7f1258009660 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:24:23.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.841+0000 7f12890f4b80 1 -- 172.21.15.133:0/387926018 wait complete. 2023-07-14T18:24:23.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.841+0000 7f12890f4b80 1 -- start start 2023-07-14T18:24:23.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.842+0000 7f12890f4b80 1 Processor -- start 2023-07-14T18:24:23.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.842+0000 7f12890f4b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562741cc3f60 0x562741cef9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:23.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.842+0000 7f12890f4b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562741cc48b0 0x562741cefef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:23.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.842+0000 7f12890f4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562741cc54b0 0x562741cf6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:23.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.842+0000 7f12890f4b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562741bb4ac0 con 0x562741cc3f60 2023-07-14T18:24:23.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.842+0000 7f12890f4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562741bb47c0 con 0x562741cc54b0 2023-07-14T18:24:23.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.842+0000 7f12890f4b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x562741a98920 con 0x562741cc48b0 2023-07-14T18:24:23.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.842+0000 7f12890f4b80 10 client.0 _ll_register_callbacks cb 0x562741bb2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:24:23.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.843+0000 7f126effd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562741cc3f60 0x562741cef9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:23.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.843+0000 7f126effd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562741cc3f60 0x562741cef9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50402/0 (socket says 172.21.15.133:50402) 2023-07-14T18:24:23.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.843+0000 7f126effd700 1 -- 172.21.15.133:0/4286070433 learned_addr learned my addr 172.21.15.133:0/4286070433 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:24:23.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.843+0000 7f126e7fc700 1 --2- 172.21.15.133:0/4286070433 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562741cc48b0 0x562741cefef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:23.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.844+0000 7f126effd700 1 -- 172.21.15.133:0/4286070433 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562741cc48b0 msgr2=0x562741cefef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:24:23.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.844+0000 7f126effd700 1 --2- 172.21.15.133:0/4286070433 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562741cc48b0 0x562741cefef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:23.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.845+0000 7f126effd700 1 -- 172.21.15.133:0/4286070433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562741cc54b0 msgr2=0x562741cf6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:24:23.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.845+0000 7f126effd700 1 --2- 172.21.15.133:0/4286070433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562741cc54b0 0x562741cf6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:23.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.845+0000 7f126effd700 1 -- 172.21.15.133:0/4286070433 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1268015b20 con 0x562741cc3f60 2023-07-14T18:24:23.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.845+0000 7f126effd700 1 --2- 172.21.15.133:0/4286070433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562741cc3f60 0x562741cef9b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f12580127c0 tx=0x7f1258004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5223b02c0178cb92 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:24:23.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.845+0000 7f12567fc700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T18:24:23.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.850+0000 7f12567fc700 1 -- 172.21.15.133:0/4286070433 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1258004a30 con 0x562741cc3f60 2023-07-14T18:24:23.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.850+0000 7f12567fc700 1 -- 172.21.15.133:0/4286070433 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1258018070 con 0x562741cc3f60 2023-07-14T18:24:23.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.856+0000 7f12890f4b80 1 -- 172.21.15.133:0/4286070433 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562741b6e4f0 con 0x562741cc3f60 2023-07-14T18:24:23.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.872+0000 7f12567fc700 1 -- 172.21.15.133:0/4286070433 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1258004b90 con 0x562741cc3f60 2023-07-14T18:24:23.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.875+0000 7f12890f4b80 10 client.15771 fetch_fsmap learned FSMap version 34 2023-07-14T18:24:23.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.875+0000 7f12890f4b80 1 -- 172.21.15.133:0/4286070433 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x562741a2bc60 con 0x562741cc3f60 2023-07-14T18:24:23.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.896+0000 7f12567fc700 1 -- 172.21.15.133:0/4286070433 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562741b6e4f0 con 0x562741cc3f60 2023-07-14T18:24:23.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.896+0000 7f12567fc700 1 -- 172.21.15.133:0/4286070433 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1258025070 con 0x562741cc3f60 2023-07-14T18:24:23.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.902+0000 7f12890f4b80 10 client.15771 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T18:24:23.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.902+0000 7f12890f4b80 10 client.15771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:24:23.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.902+0000 7f12890f4b80 20 client.15771 populate_metadata read hostname 'smithi133' 2023-07-14T18:24:23.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.902+0000 7f12890f4b80 1 --2- 172.21.15.133:0/4286070433 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x562741cfb870 0x562741cfdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:23.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.902+0000 7f12890f4b80 4 client.15771 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:24:23.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.902+0000 7f12890f4b80 1 -- 172.21.15.133:0/4286070433 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562741c92da0 con 0x562741cfb870 2023-07-14T18:24:23.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.916+0000 7f126e7fc700 1 --2- 172.21.15.133:0/4286070433 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x562741cfb870 0x562741cfdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:24.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.951+0000 7f126e7fc700 1 --2- 172.21.15.133:0/4286070433 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x562741cfb870 0x562741cfdc80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c740dc8dc9a9869 server_cookie=8804bec53f004b02 in_seq=0 out_seq=0 2023-07-14T18:24:24.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.952+0000 7f12567fc700 10 client.15771 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:24:24.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.963+0000 7f12567fc700 1 -- 172.21.15.133:0/4286070433 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x562741c92da0 con 0x562741cfb870 2023-07-14T18:24:24.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.963+0000 7f12567fc700 10 client.15771 handle_command_reply: tid=0 2023-07-14T18:24:24.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.964+0000 7f12890f4b80 1 client.15771 shutdown 2023-07-14T18:24:24.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.970+0000 7f12890f4b80 1 -- 172.21.15.133:0/4286070433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562741cc3f60 msgr2=0x562741cef9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:24:24.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.970+0000 7f12890f4b80 1 --2- 172.21.15.133:0/4286070433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562741cc3f60 0x562741cef9b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f12580127c0 tx=0x7f1258004750 comp rx=0 tx=0).stop 2023-07-14T18:24:24.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.971+0000 7f12890f4b80 1 -- 172.21.15.133:0/4286070433 shutdown_connections 2023-07-14T18:24:24.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.971+0000 7f12890f4b80 1 --2- 172.21.15.133:0/4286070433 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x562741cfb870 0x562741cfdc80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:24.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.971+0000 7f12890f4b80 1 --2- 172.21.15.133:0/4286070433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562741cc54b0 0x562741cf6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:24.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.971+0000 7f12890f4b80 1 --2- 172.21.15.133:0/4286070433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562741cc3f60 0x562741cef9b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:24.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.971+0000 7f12890f4b80 1 --2- 172.21.15.133:0/4286070433 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562741cc48b0 0x562741cefef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:24.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.971+0000 7f12890f4b80 1 -- 172.21.15.133:0/4286070433 >> 172.21.15.133:0/4286070433 conn(0x562741cb9680 msgr2=0x562741cbd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:24:24.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.975+0000 7f12567fc700 0 client.15771 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:24:24.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.979+0000 7f12890f4b80 1 -- 172.21.15.133:0/4286070433 shutdown_connections 2023-07-14T18:24:24.039 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.984+0000 7f12890f4b80 1 -- 172.21.15.133:0/4286070433 wait complete. 2023-07-14T18:24:24.039 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:23.985+0000 7f12890f4b80 20 client.15771 trim_cache size 0 max 16384 2023-07-14T18:24:24.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.016+0000 7f12890f4b80 1 Processor -- start 2023-07-14T18:24:24.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.017+0000 7f12890f4b80 1 -- start start 2023-07-14T18:24:24.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.017+0000 7f12890f4b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562741cc3f60 0x562741cbe0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:24.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.017+0000 7f12890f4b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562741cc48b0 0x562741cf6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:24.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.017+0000 7f12890f4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562741cc54b0 0x562741cfd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:24.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.017+0000 7f12890f4b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562741cc5da0 con 0x562741cc48b0 2023-07-14T18:24:24.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.017+0000 7f12890f4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562741ceee60 con 0x562741cc54b0 2023-07-14T18:24:24.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.017+0000 7f12890f4b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x562741cbfcc0 con 0x562741cc3f60 2023-07-14T18:24:24.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.018+0000 7f126f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562741cc54b0 0x562741cfd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:24.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.018+0000 7f126f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562741cc54b0 0x562741cfd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:54160/0 (socket says 172.21.15.133:54160) 2023-07-14T18:24:24.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.018+0000 7f126f7fe700 1 -- 172.21.15.133:0/881126955 learned_addr learned my addr 172.21.15.133:0/881126955 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:24:24.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.018+0000 7f126effd700 1 --2- 172.21.15.133:0/881126955 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562741cc3f60 0x562741cbe0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:24.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.019+0000 7f126effd700 1 -- 172.21.15.133:0/881126955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562741cc54b0 msgr2=0x562741cfd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:24:24.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.019+0000 7f126effd700 1 --2- 172.21.15.133:0/881126955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562741cc54b0 0x562741cfd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:24.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.019+0000 7f126effd700 1 -- 172.21.15.133:0/881126955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562741cc48b0 msgr2=0x562741cf6ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T18:24:24.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.019+0000 7f126effd700 1 --2- 172.21.15.133:0/881126955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562741cc48b0 0x562741cf6ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:24.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.019+0000 7f126effd700 1 -- 172.21.15.133:0/881126955 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562741a69250 con 0x562741cc3f60 2023-07-14T18:24:24.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.028+0000 7f126effd700 1 --2- 172.21.15.133:0/881126955 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562741cc3f60 0x562741cbe0f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f125801f9e0 tx=0x7f1258009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4af90489194ebe1a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:24:24.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.029+0000 7f1265ffb700 1 -- 172.21.15.133:0/881126955 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125801fc50 con 0x562741cc3f60 2023-07-14T18:24:24.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.029+0000 7f1265ffb700 1 -- 172.21.15.133:0/881126955 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f125801fe90 con 0x562741cc3f60 2023-07-14T18:24:24.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.029+0000 7f1265ffb700 1 -- 172.21.15.133:0/881126955 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1258005420 con 0x562741cc3f60 2023-07-14T18:24:24.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.030+0000 7f12890f4b80 1 -- 172.21.15.133:0/881126955 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562741cc3f60 msgr2=0x562741cbe0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:24:24.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.030+0000 7f12890f4b80 1 --2- 172.21.15.133:0/881126955 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562741cc3f60 0x562741cbe0f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f125801f9e0 tx=0x7f1258009260 comp rx=0 tx=0).stop 2023-07-14T18:24:24.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.031+0000 7f12890f4b80 1 -- 172.21.15.133:0/881126955 shutdown_connections 2023-07-14T18:24:24.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.031+0000 7f12890f4b80 1 --2- 172.21.15.133:0/881126955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562741cc54b0 0x562741cfd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:24.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.031+0000 7f12890f4b80 1 --2- 172.21.15.133:0/881126955 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562741cc3f60 0x562741cbe0f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:24.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.031+0000 7f12890f4b80 1 --2- 172.21.15.133:0/881126955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562741cc48b0 0x562741cf6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:24.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.031+0000 7f12890f4b80 1 -- 172.21.15.133:0/881126955 >> 172.21.15.133:0/881126955 conn(0x562741cb9680 msgr2=0x562741cee6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:24:24.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.043+0000 7f12890f4b80 1 -- 172.21.15.133:0/881126955 shutdown_connections 2023-07-14T18:24:24.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.043+0000 7f12890f4b80 1 -- 172.21.15.133:0/881126955 wait complete. 2023-07-14T18:24:24.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.043+0000 7f12890f4b80 1 -- start start 2023-07-14T18:24:24.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.044+0000 7f12890f4b80 1 Processor -- start 2023-07-14T18:24:24.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.044+0000 7f12890f4b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562741cc3f60 0x562741cf3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:24.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.044+0000 7f12890f4b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562741cc48b0 0x562741cf35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:24.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.044+0000 7f12890f4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562741cc54b0 0x562741cf65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:24.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.044+0000 7f12890f4b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562741cf6b00 con 0x562741cc48b0 2023-07-14T18:24:24.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.044+0000 7f12890f4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562741cf6c70 con 0x562741cc54b0 2023-07-14T18:24:24.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.044+0000 7f12890f4b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x562741cf6e30 con 0x562741cc3f60 2023-07-14T18:24:24.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.044+0000 7f12890f4b80 10 client.0 _ll_register_callbacks cb 0x562741bb2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:24:24.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.048+0000 7f126f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562741cc54b0 0x562741cf65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:24.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.048+0000 7f126effd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562741cc3f60 0x562741cf3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:24.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.049+0000 7f126f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562741cc54b0 0x562741cf65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:54162/0 (socket says 172.21.15.133:54162) 2023-07-14T18:24:24.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.049+0000 7f126f7fe700 1 -- 172.21.15.133:0/1990836838 learned_addr learned my addr 172.21.15.133:0/1990836838 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:24:24.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.057+0000 7f126e7fc700 1 --2- 172.21.15.133:0/1990836838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562741cc48b0 0x562741cf35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:24.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.060+0000 7f126f7fe700 1 -- 172.21.15.133:0/1990836838 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562741cc3f60 msgr2=0x562741cf3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:24:24.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.060+0000 7f126f7fe700 1 --2- 172.21.15.133:0/1990836838 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562741cc3f60 0x562741cf3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:24.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.060+0000 7f126f7fe700 1 -- 172.21.15.133:0/1990836838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562741cc48b0 msgr2=0x562741cf35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:24:24.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.060+0000 7f126f7fe700 1 --2- 172.21.15.133:0/1990836838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562741cc48b0 0x562741cf35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:24.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.060+0000 7f126f7fe700 1 -- 172.21.15.133:0/1990836838 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562741a2bc60 con 0x562741cc54b0 2023-07-14T18:24:24.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.061+0000 7f126f7fe700 1 --2- 172.21.15.133:0/1990836838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562741cc54b0 0x562741cf65c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f12600098c0 tx=0x7f126009f3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f495bd135313137 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:24:24.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.061+0000 7f12567fc700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:24:24.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.073+0000 7f12567fc700 1 -- 172.21.15.133:0/1990836838 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12600cbb20 con 0x562741cc54b0 2023-07-14T18:24:24.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.073+0000 7f12567fc700 1 -- 172.21.15.133:0/1990836838 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f126009f030 con 0x562741cc54b0 2023-07-14T18:24:24.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.074+0000 7f12890f4b80 1 -- 172.21.15.133:0/1990836838 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1268004db0 con 0x562741cc54b0 2023-07-14T18:24:24.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.085+0000 7f12567fc700 1 -- 172.21.15.133:0/1990836838 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f126009f1f0 con 0x562741cc54b0 2023-07-14T18:24:24.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.097+0000 7f12890f4b80 10 client.25826 fetch_fsmap learned FSMap version 34 2023-07-14T18:24:24.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.097+0000 7f12890f4b80 1 -- 172.21.15.133:0/1990836838 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x562741a412b0 con 0x562741cc54b0 2023-07-14T18:24:24.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.105+0000 7f12567fc700 1 -- 172.21.15.133:0/1990836838 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1268004db0 con 0x562741cc54b0 2023-07-14T18:24:24.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.105+0000 7f12567fc700 1 -- 172.21.15.133:0/1990836838 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f126009b610 con 0x562741cc54b0 2023-07-14T18:24:24.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.105+0000 7f12890f4b80 10 client.25826 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T18:24:24.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.105+0000 7f12890f4b80 10 client.25826 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:24:24.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.105+0000 7f12890f4b80 20 client.25826 populate_metadata read hostname 'smithi133' 2023-07-14T18:24:24.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.105+0000 7f12890f4b80 1 --2- 172.21.15.133:0/1990836838 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x562741d50530 0x562741d52940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:24.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.105+0000 7f12890f4b80 4 client.25826 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:24:24.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.105+0000 7f12890f4b80 1 -- 172.21.15.133:0/1990836838 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562741c0deb0 con 0x562741d50530 2023-07-14T18:24:24.165 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.107+0000 7f126effd700 1 --2- 172.21.15.133:0/1990836838 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x562741d50530 0x562741d52940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:24.205 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.149+0000 7f126effd700 1 --2- 172.21.15.133:0/1990836838 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x562741d50530 0x562741d52940 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3425967c4edfee47 server_cookie=45a20f7b8ec5fa39 in_seq=0 out_seq=0 2023-07-14T18:24:24.205 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.149+0000 7f12567fc700 10 client.25826 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:24:24.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.160+0000 7f12567fc700 1 -- 172.21.15.133:0/1990836838 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x562741c0deb0 con 0x562741d50530 2023-07-14T18:24:24.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.160+0000 7f12567fc700 10 client.25826 handle_command_reply: tid=0 2023-07-14T18:24:24.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.161+0000 7f12890f4b80 1 client.25826 shutdown 2023-07-14T18:24:24.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.191+0000 7f12890f4b80 1 -- 172.21.15.133:0/1990836838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562741cc54b0 msgr2=0x562741cf65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:24:24.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.191+0000 7f12890f4b80 1 --2- 172.21.15.133:0/1990836838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562741cc54b0 0x562741cf65c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f12600098c0 tx=0x7f126009f3e0 comp rx=0 tx=0).stop 2023-07-14T18:24:24.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.192+0000 7f12890f4b80 1 -- 172.21.15.133:0/1990836838 shutdown_connections 2023-07-14T18:24:24.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.192+0000 7f12890f4b80 1 --2- 172.21.15.133:0/1990836838 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x562741d50530 0x562741d52940 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:24.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.192+0000 7f12890f4b80 1 --2- 172.21.15.133:0/1990836838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562741cc54b0 0x562741cf65c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:24.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.192+0000 7f12890f4b80 1 --2- 172.21.15.133:0/1990836838 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562741cc3f60 0x562741cf3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:24.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.192+0000 7f12890f4b80 1 --2- 172.21.15.133:0/1990836838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562741cc48b0 0x562741cf35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:24.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.192+0000 7f12890f4b80 1 -- 172.21.15.133:0/1990836838 >> 172.21.15.133:0/1990836838 conn(0x562741cb9680 msgr2=0x562741cbe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:24:24.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.193+0000 7f12567fc700 0 client.25826 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:24:24.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.210+0000 7f12890f4b80 1 -- 172.21.15.133:0/1990836838 shutdown_connections 2023-07-14T18:24:24.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.214+0000 7f12890f4b80 1 -- 172.21.15.133:0/1990836838 wait complete. 2023-07-14T18:24:24.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.219+0000 7f12890f4b80 20 client.25826 trim_cache size 0 max 16384 2023-07-14T18:24:24.272 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:24:24.272 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-14T18:24:24.272 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:24:24.272 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:24:24.272 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:24:24.272 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:24:24.273 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:24:24.273 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:24:24.273 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:24:24.273 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:24:24.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.223+0000 7f12757ff700 1 -- 172.21.15.133:0/898653844 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f125c0a1350 msgr2=0x7f125c0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:24:24.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.223+0000 7f12757ff700 1 --2- 172.21.15.133:0/898653844 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f125c0a1350 0x7f125c0a3800 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x562741cc4cc0 tx=0x7f1258009660 comp rx=0 tx=0).stop 2023-07-14T18:24:24.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.223+0000 7f12757ff700 1 -- 172.21.15.133:0/898653844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f127010be90 msgr2=0x7f12701162a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:24:24.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.223+0000 7f12757ff700 1 --2- 172.21.15.133:0/898653844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f127010be90 0x7f12701162a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f126000fa40 tx=0x7f126000b840 comp rx=0 tx=0).stop 2023-07-14T18:24:24.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.226+0000 7f12757ff700 1 -- 172.21.15.133:0/898653844 shutdown_connections 2023-07-14T18:24:24.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.226+0000 7f12757ff700 1 --2- 172.21.15.133:0/898653844 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f125c0a1350 0x7f125c0a3800 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:24.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.226+0000 7f12757ff700 1 --2- 172.21.15.133:0/898653844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f127010be90 0x7f12701162a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:24.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.226+0000 7f12757ff700 1 --2- 172.21.15.133:0/898653844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1270071010 0x7f1270114d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:24.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.226+0000 7f12757ff700 1 --2- 172.21.15.133:0/898653844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1270071960 0x7f1270115260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:24.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.226+0000 7f12757ff700 1 -- 172.21.15.133:0/898653844 >> 172.21.15.133:0/898653844 conn(0x7f127006cbc0 msgr2=0x7f127006e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:24:24.301 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.247+0000 7f12757ff700 1 -- 172.21.15.133:0/898653844 shutdown_connections 2023-07-14T18:24:24.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:24.258+0000 7f12757ff700 1 -- 172.21.15.133:0/898653844 wait complete. 2023-07-14T18:24:24.374 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:24:25.884 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 300.0 s, 241.8 tps, lat 422.134 ms stddev 643.061 2023-07-14T18:24:25.916 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:25 smithi133 ceph-mon[126384]: pgmap v580: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 28 MiB/s wr, 2.33k op/s 2023-07-14T18:24:26.112 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:25 smithi161 ceph-mon[128993]: pgmap v580: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 28 MiB/s wr, 2.33k op/s 2023-07-14T18:24:26.123 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:25 smithi171 ceph-mon[129241]: pgmap v580: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 28 MiB/s wr, 2.33k op/s 2023-07-14T18:24:28.108 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:27 smithi161 ceph-mon[128993]: pgmap v581: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 18 MiB/s wr, 1.39k op/s 2023-07-14T18:24:28.130 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:27 smithi171 ceph-mon[129241]: pgmap v581: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 18 MiB/s wr, 1.39k op/s 2023-07-14T18:24:28.169 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:27 smithi133 ceph-mon[126384]: pgmap v581: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 18 MiB/s wr, 1.39k op/s 2023-07-14T18:24:30.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:29 smithi161 ceph-mon[128993]: pgmap v582: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 17 MiB/s wr, 1.33k op/s 2023-07-14T18:24:30.125 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:29 smithi171 ceph-mon[129241]: pgmap v582: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 17 MiB/s wr, 1.33k op/s 2023-07-14T18:24:30.165 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:29 smithi133 ceph-mon[126384]: pgmap v582: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 17 MiB/s wr, 1.33k op/s 2023-07-14T18:24:30.885 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 305.0 s, 539.2 tps, lat 186.971 ms stddev 250.804 2023-07-14T18:24:32.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:31 smithi161 ceph-mon[128993]: pgmap v583: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 17 MiB/s wr, 1.42k op/s 2023-07-14T18:24:32.127 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:31 smithi171 ceph-mon[129241]: pgmap v583: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 17 MiB/s wr, 1.42k op/s 2023-07-14T18:24:32.166 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:31 smithi133 ceph-mon[126384]: pgmap v583: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 17 MiB/s wr, 1.42k op/s 2023-07-14T18:24:34.124 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:33 smithi171 ceph-mon[129241]: pgmap v584: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 14 MiB/s wr, 1.20k op/s 2023-07-14T18:24:34.164 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:33 smithi133 ceph-mon[126384]: pgmap v584: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 14 MiB/s wr, 1.20k op/s 2023-07-14T18:24:34.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:33 smithi161 ceph-mon[128993]: pgmap v584: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 14 MiB/s wr, 1.20k op/s 2023-07-14T18:24:35.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:34 smithi161 ceph-mon[128993]: pgmap v585: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 19 MiB/s wr, 1.49k op/s 2023-07-14T18:24:35.374 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:34 smithi171 ceph-mon[129241]: pgmap v585: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 19 MiB/s wr, 1.49k op/s 2023-07-14T18:24:35.423 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:34 smithi133 ceph-mon[126384]: pgmap v585: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 19 MiB/s wr, 1.49k op/s 2023-07-14T18:24:35.885 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 310.0 s, 931.8 tps, lat 108.046 ms stddev 187.800 2023-07-14T18:24:37.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:37 smithi161 ceph-mon[128993]: pgmap v586: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 13 MiB/s wr, 1.04k op/s 2023-07-14T18:24:37.874 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:37 smithi171 ceph-mon[129241]: pgmap v586: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 13 MiB/s wr, 1.04k op/s 2023-07-14T18:24:37.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:37 smithi133 ceph-mon[126384]: pgmap v586: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 13 MiB/s wr, 1.04k op/s 2023-07-14T18:24:39.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:39 smithi161 ceph-mon[128993]: pgmap v587: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 15 MiB/s wr, 990 op/s 2023-07-14T18:24:39.873 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:39 smithi171 ceph-mon[129241]: pgmap v587: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 15 MiB/s wr, 990 op/s 2023-07-14T18:24:39.915 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:39 smithi133 ceph-mon[126384]: pgmap v587: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 15 MiB/s wr, 990 op/s 2023-07-14T18:24:40.883 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 315.0 s, 1168.7 tps, lat 82.018 ms stddev 81.920 2023-07-14T18:24:41.923 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:41 smithi133 ceph-mon[126384]: pgmap v588: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 16 MiB/s wr, 980 op/s 2023-07-14T18:24:42.111 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:41 smithi161 ceph-mon[128993]: pgmap v588: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 16 MiB/s wr, 980 op/s 2023-07-14T18:24:42.124 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:41 smithi171 ceph-mon[129241]: pgmap v588: 129 pgs: 129 active+clean; 8.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 16 MiB/s wr, 980 op/s 2023-07-14T18:24:43.918 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:43 smithi133 ceph-mon[126384]: pgmap v589: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 15 MiB/s wr, 783 op/s 2023-07-14T18:24:44.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:43 smithi161 ceph-mon[128993]: pgmap v589: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 15 MiB/s wr, 783 op/s 2023-07-14T18:24:44.124 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:43 smithi171 ceph-mon[129241]: pgmap v589: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 15 MiB/s wr, 783 op/s 2023-07-14T18:24:45.883 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 320.0 s, 511.6 tps, lat 206.225 ms stddev 442.454 2023-07-14T18:24:46.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:45 smithi161 ceph-mon[128993]: pgmap v590: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 24 MiB/s wr, 1.57k op/s 2023-07-14T18:24:46.124 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:45 smithi171 ceph-mon[129241]: pgmap v590: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 24 MiB/s wr, 1.57k op/s 2023-07-14T18:24:46.171 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:45 smithi133 ceph-mon[126384]: pgmap v590: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 24 MiB/s wr, 1.57k op/s 2023-07-14T18:24:48.108 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:47 smithi161 ceph-mon[128993]: pgmap v591: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 21 MiB/s wr, 1.45k op/s 2023-07-14T18:24:48.124 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:47 smithi171 ceph-mon[129241]: pgmap v591: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 21 MiB/s wr, 1.45k op/s 2023-07-14T18:24:48.170 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:47 smithi133 ceph-mon[126384]: pgmap v591: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 21 MiB/s wr, 1.45k op/s 2023-07-14T18:24:50.114 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:49 smithi161 ceph-mon[128993]: pgmap v592: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 23 MiB/s wr, 1.48k op/s 2023-07-14T18:24:50.124 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:49 smithi171 ceph-mon[129241]: pgmap v592: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 23 MiB/s wr, 1.48k op/s 2023-07-14T18:24:50.169 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:49 smithi133 ceph-mon[126384]: pgmap v592: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 23 MiB/s wr, 1.48k op/s 2023-07-14T18:24:50.883 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 325.0 s, 1300.6 tps, lat 70.028 ms stddev 50.393 2023-07-14T18:24:52.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:51 smithi161 ceph-mon[128993]: pgmap v593: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 23 MiB/s wr, 1.54k op/s 2023-07-14T18:24:52.124 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:51 smithi171 ceph-mon[129241]: pgmap v593: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 23 MiB/s wr, 1.54k op/s 2023-07-14T18:24:52.166 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:51 smithi133 ceph-mon[126384]: pgmap v593: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 23 MiB/s wr, 1.54k op/s 2023-07-14T18:24:54.361 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:53 smithi161 ceph-mon[128993]: pgmap v594: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 21 MiB/s wr, 1.44k op/s 2023-07-14T18:24:54.375 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:24:54.377 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:53 smithi171 ceph-mon[129241]: pgmap v594: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 21 MiB/s wr, 1.44k op/s 2023-07-14T18:24:54.420 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:53 smithi133 ceph-mon[126384]: pgmap v594: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 21 MiB/s wr, 1.44k op/s 2023-07-14T18:24:54.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.822+0000 7f67a4fff700 1 -- 172.21.15.133:0/3294474742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67a00fefa0 msgr2=0x7f67a00ff410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:24:54.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.822+0000 7f67a4fff700 1 --2- 172.21.15.133:0/3294474742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67a00fefa0 0x7f67a00ff410 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6794007510 tx=0x7f6794007820 comp rx=0 tx=0).stop 2023-07-14T18:24:54.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.822+0000 7f67a4fff700 1 -- 172.21.15.133:0/3294474742 shutdown_connections 2023-07-14T18:24:54.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.822+0000 7f67a4fff700 1 --2- 172.21.15.133:0/3294474742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67a00fde50 0x7f67a00fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:54.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.822+0000 7f67a4fff700 1 --2- 172.21.15.133:0/3294474742 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67a00ff950 0x7f67a0106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:54.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.822+0000 7f67a4fff700 1 --2- 172.21.15.133:0/3294474742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67a00fefa0 0x7f67a00ff410 secure :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6794007510 tx=0x7f6794007820 comp rx=0 tx=0).stop 2023-07-14T18:24:54.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.822+0000 7f67a4fff700 1 -- 172.21.15.133:0/3294474742 >> 172.21.15.133:0/3294474742 conn(0x7f67a00f9470 msgr2=0x7f67a00fb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:24:54.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.845+0000 7f67a4fff700 1 -- 172.21.15.133:0/3294474742 shutdown_connections 2023-07-14T18:24:54.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.856+0000 7f67a4fff700 1 -- 172.21.15.133:0/3294474742 wait complete. 2023-07-14T18:24:54.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.857+0000 7f67a4fff700 1 Processor -- start 2023-07-14T18:24:54.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.868+0000 7f67a4fff700 1 -- start start 2023-07-14T18:24:54.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.868+0000 7f67a4fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67a00fde50 0x7f67a006af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:54.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.868+0000 7f67a4fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67a00ff950 0x7f67a006b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:54.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.868+0000 7f67a4fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67a00704f0 0x7f67a0070960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:54.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.868+0000 7f67a4fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a0070ea0 con 0x7f67a00ff950 2023-07-14T18:24:54.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.868+0000 7f67a4fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a0070fe0 con 0x7f67a00fde50 2023-07-14T18:24:54.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.868+0000 7f67a4fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a00711a0 con 0x7f67a00704f0 2023-07-14T18:24:54.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.873+0000 7f679dd9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67a00ff950 0x7f67a006b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:54.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.873+0000 7f679dd9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67a00ff950 0x7f67a006b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:53894/0 (socket says 172.21.15.133:53894) 2023-07-14T18:24:54.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.873+0000 7f679dd9b700 1 -- 172.21.15.133:0/3773925589 learned_addr learned my addr 172.21.15.133:0/3773925589 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:24:54.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.874+0000 7f679e59c700 1 --2- 172.21.15.133:0/3773925589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67a00fde50 0x7f67a006af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:54.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.875+0000 7f679dd9b700 1 -- 172.21.15.133:0/3773925589 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67a00704f0 msgr2=0x7f67a0070960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T18:24:54.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.875+0000 7f679dd9b700 1 --2- 172.21.15.133:0/3773925589 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67a00704f0 0x7f67a0070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:54.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.875+0000 7f679dd9b700 1 -- 172.21.15.133:0/3773925589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67a00fde50 msgr2=0x7f67a006af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:24:54.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.875+0000 7f679dd9b700 1 --2- 172.21.15.133:0/3773925589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67a00fde50 0x7f67a006af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:54.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.875+0000 7f679dd9b700 1 -- 172.21.15.133:0/3773925589 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67940071c0 con 0x7f67a00ff950 2023-07-14T18:24:54.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.877+0000 7f679dd9b700 1 --2- 172.21.15.133:0/3773925589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67a00ff950 0x7f67a006b490 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f679400a040 tx=0x7f679400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:24:54.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.881+0000 7f678f7fe700 1 -- 172.21.15.133:0/3773925589 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679400f040 con 0x7f67a00ff950 2023-07-14T18:24:54.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.882+0000 7f678f7fe700 1 -- 172.21.15.133:0/3773925589 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f679400a690 con 0x7f67a00ff950 2023-07-14T18:24:54.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.882+0000 7f678f7fe700 1 -- 172.21.15.133:0/3773925589 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67940085f0 con 0x7f67a00ff950 2023-07-14T18:24:54.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.882+0000 7f67a4fff700 1 -- 172.21.15.133:0/3773925589 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a0071420 con 0x7f67a00ff950 2023-07-14T18:24:54.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.882+0000 7f67a4fff700 1 -- 172.21.15.133:0/3773925589 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a0071900 con 0x7f67a00ff950 2023-07-14T18:24:54.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.889+0000 7f67b8849b80 1 Processor -- start 2023-07-14T18:24:54.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.892+0000 7f67b8849b80 1 -- start start 2023-07-14T18:24:54.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.892+0000 7f67b8849b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d9e65ef60 0x563d9e65f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:54.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.892+0000 7f67b8849b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d9e65f8b0 0x563d9e65fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:54.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.892+0000 7f67b8849b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d9e6604b0 0x563d9e660920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:54.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.892+0000 7f67b8849b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9e563cb0 con 0x563d9e65ef60 2023-07-14T18:24:54.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.892+0000 7f67b8849b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9e5b6a40 con 0x563d9e6604b0 2023-07-14T18:24:54.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.892+0000 7f67b8849b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9e5afed0 con 0x563d9e65f8b0 2023-07-14T18:24:54.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.894+0000 7f679e59c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d9e65f8b0 0x563d9e65fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:54.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.894+0000 7f679e59c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d9e65f8b0 0x563d9e65fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:45660/0 (socket says 172.21.15.133:45660) 2023-07-14T18:24:54.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.894+0000 7f679e59c700 1 -- 172.21.15.133:0/1804805988 learned_addr learned my addr 172.21.15.133:0/1804805988 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:24:54.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.894+0000 7f679dd9b700 1 --2- 172.21.15.133:0/1804805988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d9e6604b0 0x563d9e660920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:54.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.895+0000 7f679e59c700 1 -- 172.21.15.133:0/1804805988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d9e6604b0 msgr2=0x563d9e660920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:24:54.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.895+0000 7f679e59c700 1 --2- 172.21.15.133:0/1804805988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d9e6604b0 0x563d9e660920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:54.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.895+0000 7f679e59c700 1 -- 172.21.15.133:0/1804805988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d9e65ef60 msgr2=0x563d9e65f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:24:54.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.895+0000 7f679e59c700 1 --2- 172.21.15.133:0/1804805988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d9e65ef60 0x563d9e65f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:54.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.895+0000 7f679e59c700 1 -- 172.21.15.133:0/1804805988 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d9e3ddde0 con 0x563d9e65f8b0 2023-07-14T18:24:54.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.895+0000 7f679e59c700 1 --2- 172.21.15.133:0/1804805988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d9e65f8b0 0x563d9e65fed0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6790009960 tx=0x7f6790009c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14a048207b3dd1b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:24:54.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.896+0000 7f678cff9700 1 -- 172.21.15.133:0/1804805988 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67900308d0 con 0x563d9e65f8b0 2023-07-14T18:24:54.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.896+0000 7f678cff9700 1 -- 172.21.15.133:0/1804805988 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6790035480 con 0x563d9e65f8b0 2023-07-14T18:24:54.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.896+0000 7f678cff9700 1 -- 172.21.15.133:0/1804805988 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6790034620 con 0x563d9e65f8b0 2023-07-14T18:24:54.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.897+0000 7f67b8849b80 1 -- 172.21.15.133:0/1804805988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d9e65f8b0 msgr2=0x563d9e65fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:24:54.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.897+0000 7f67b8849b80 1 --2- 172.21.15.133:0/1804805988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d9e65f8b0 0x563d9e65fed0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6790009960 tx=0x7f6790009c70 comp rx=0 tx=0).stop 2023-07-14T18:24:54.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.904+0000 7f67b8849b80 1 -- 172.21.15.133:0/1804805988 shutdown_connections 2023-07-14T18:24:54.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.904+0000 7f67b8849b80 1 --2- 172.21.15.133:0/1804805988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d9e6604b0 0x563d9e660920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:54.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.904+0000 7f67b8849b80 1 --2- 172.21.15.133:0/1804805988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d9e65ef60 0x563d9e65f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:54.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.904+0000 7f67b8849b80 1 --2- 172.21.15.133:0/1804805988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d9e65f8b0 0x563d9e65fed0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:54.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.904+0000 7f67b8849b80 1 -- 172.21.15.133:0/1804805988 >> 172.21.15.133:0/1804805988 conn(0x563d9e654680 msgr2=0x563d9e6599d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:24:54.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.909+0000 7f678f7fe700 1 -- 172.21.15.133:0/3773925589 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f679400a800 con 0x7f67a00ff950 2023-07-14T18:24:54.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.910+0000 7f67b8849b80 1 -- 172.21.15.133:0/1804805988 shutdown_connections 2023-07-14T18:24:54.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.911+0000 7f678f7fe700 1 --2- 172.21.15.133:0/3773925589 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f67880a1260 0x7f67880a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:54.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.912+0000 7f678f7fe700 1 -- 172.21.15.133:0/3773925589 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f67940ccc30 con 0x7f67a00ff950 2023-07-14T18:24:54.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.914+0000 7f67b8849b80 1 -- 172.21.15.133:0/1804805988 wait complete. 2023-07-14T18:24:54.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.915+0000 7f67b8849b80 1 -- start start 2023-07-14T18:24:54.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.915+0000 7f67b8849b80 1 Processor -- start 2023-07-14T18:24:54.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.916+0000 7f679e59c700 1 --2- 172.21.15.133:0/3773925589 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f67880a1260 0x7f67880a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:54.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.916+0000 7f67b8849b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d9e65ef60 0x563d9e668980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:54.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.916+0000 7f67b8849b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d9e65f8b0 0x563d9e668ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:54.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.916+0000 7f67b8849b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d9e6604b0 0x563d9e66feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:54.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.916+0000 7f67b8849b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9e54fac0 con 0x563d9e6604b0 2023-07-14T18:24:54.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.916+0000 7f67b8849b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9e54f7c0 con 0x563d9e65f8b0 2023-07-14T18:24:54.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.916+0000 7f67b8849b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9e433920 con 0x563d9e65ef60 2023-07-14T18:24:54.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.916+0000 7f67b8849b80 10 client.0 _ll_register_callbacks cb 0x563d9e54d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:24:54.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.917+0000 7f679e59c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d9e65ef60 0x563d9e668980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:54.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.917+0000 7f679e59c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d9e65ef60 0x563d9e668980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:45664/0 (socket says 172.21.15.133:45664) 2023-07-14T18:24:54.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.917+0000 7f679e59c700 1 -- 172.21.15.133:0/1087082115 learned_addr learned my addr 172.21.15.133:0/1087082115 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:24:54.974 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.921+0000 7f679ed9d700 1 --2- 172.21.15.133:0/1087082115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d9e6604b0 0x563d9e66feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:54.974 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.921+0000 7f679dd9b700 1 --2- 172.21.15.133:0/1087082115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d9e65f8b0 0x563d9e668ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:54.975 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.921+0000 7f679e59c700 1 -- 172.21.15.133:0/1087082115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d9e65f8b0 msgr2=0x563d9e668ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:24:54.975 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.921+0000 7f679e59c700 1 --2- 172.21.15.133:0/1087082115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d9e65f8b0 0x563d9e668ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:54.975 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.921+0000 7f679e59c700 1 -- 172.21.15.133:0/1087082115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d9e6604b0 msgr2=0x563d9e66feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:24:54.976 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.921+0000 7f679e59c700 1 --2- 172.21.15.133:0/1087082115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d9e6604b0 0x563d9e66feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:54.976 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.921+0000 7f679e59c700 1 -- 172.21.15.133:0/1087082115 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f679002f040 con 0x563d9e65ef60 2023-07-14T18:24:54.977 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.922+0000 7f679e59c700 1 --2- 172.21.15.133:0/3773925589 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f67880a1260 0x7f67880a3710 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f67900304c0 tx=0x7f6790004750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:24:54.977 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.922+0000 7f679e59c700 1 --2- 172.21.15.133:0/1087082115 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d9e65ef60 0x563d9e668980 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6790006fa0 tx=0x7f6790009190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebee9e4ccdfeabc1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:24:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.930+0000 7f6785ffb700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:24:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.930+0000 7f6785ffb700 1 -- 172.21.15.133:0/1087082115 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679003e920 con 0x563d9e65ef60 2023-07-14T18:24:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.930+0000 7f6785ffb700 1 -- 172.21.15.133:0/1087082115 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f679003ea80 con 0x563d9e65ef60 2023-07-14T18:24:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.930+0000 7f67b8849b80 1 -- 172.21.15.133:0/1087082115 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d9e5094f0 con 0x563d9e65ef60 2023-07-14T18:24:55.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.961+0000 7f6785ffb700 1 -- 172.21.15.133:0/1087082115 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6790005a50 con 0x563d9e65ef60 2023-07-14T18:24:55.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.965+0000 7f67b8849b80 10 client.25867 fetch_fsmap learned FSMap version 34 2023-07-14T18:24:55.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.965+0000 7f67b8849b80 1 -- 172.21.15.133:0/1087082115 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563d9e3c6c60 con 0x563d9e65ef60 2023-07-14T18:24:55.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.972+0000 7f6785ffb700 1 -- 172.21.15.133:0/1087082115 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563d9e5094f0 con 0x563d9e65ef60 2023-07-14T18:24:55.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.972+0000 7f6785ffb700 1 -- 172.21.15.133:0/1087082115 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f679004a410 con 0x563d9e65ef60 2023-07-14T18:24:55.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.972+0000 7f67b8849b80 10 client.25867 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T18:24:55.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.972+0000 7f67b8849b80 10 client.25867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:24:55.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.972+0000 7f67b8849b80 20 client.25867 populate_metadata read hostname 'smithi133' 2023-07-14T18:24:55.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.972+0000 7f67b8849b80 1 --2- 172.21.15.133:0/1087082115 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x563d9e6745d0 0x563d9e6769e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:55.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.972+0000 7f67b8849b80 4 client.25867 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:24:55.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.972+0000 7f67b8849b80 1 -- 172.21.15.133:0/1087082115 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d9e62dda0 con 0x563d9e6745d0 2023-07-14T18:24:55.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:54.976+0000 7f679dd9b700 1 --2- 172.21.15.133:0/1087082115 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x563d9e6745d0 0x563d9e6769e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:55.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.011+0000 7f679dd9b700 1 --2- 172.21.15.133:0/1087082115 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x563d9e6745d0 0x563d9e6769e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=388034c2f00fc69f server_cookie=401b6ab78f8a07e6 in_seq=0 out_seq=0 2023-07-14T18:24:55.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.014+0000 7f6785ffb700 10 client.25867 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:24:55.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.014+0000 7f6785ffb700 1 -- 172.21.15.133:0/1087082115 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x563d9e62dda0 con 0x563d9e6745d0 2023-07-14T18:24:55.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.014+0000 7f6785ffb700 10 client.25867 handle_command_reply: tid=0 2023-07-14T18:24:55.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.014+0000 7f67b8849b80 1 client.25867 shutdown 2023-07-14T18:24:55.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.026+0000 7f67b8849b80 1 -- 172.21.15.133:0/1087082115 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d9e65ef60 msgr2=0x563d9e668980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:24:55.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.026+0000 7f67b8849b80 1 --2- 172.21.15.133:0/1087082115 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d9e65ef60 0x563d9e668980 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6790006fa0 tx=0x7f6790009190 comp rx=0 tx=0).stop 2023-07-14T18:24:55.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.026+0000 7f67b8849b80 1 -- 172.21.15.133:0/1087082115 shutdown_connections 2023-07-14T18:24:55.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.026+0000 7f67b8849b80 1 --2- 172.21.15.133:0/1087082115 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x563d9e6745d0 0x563d9e6769e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:55.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.026+0000 7f67b8849b80 1 --2- 172.21.15.133:0/1087082115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d9e6604b0 0x563d9e66feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:55.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.026+0000 7f67b8849b80 1 --2- 172.21.15.133:0/1087082115 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d9e65ef60 0x563d9e668980 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:55.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.026+0000 7f67b8849b80 1 --2- 172.21.15.133:0/1087082115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d9e65f8b0 0x563d9e668ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:55.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.026+0000 7f67b8849b80 1 -- 172.21.15.133:0/1087082115 >> 172.21.15.133:0/1087082115 conn(0x563d9e654680 msgr2=0x563d9e658650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:24:55.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.030+0000 7f6785ffb700 0 client.25867 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:24:55.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.030+0000 7f67b8849b80 1 -- 172.21.15.133:0/1087082115 shutdown_connections 2023-07-14T18:24:55.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.030+0000 7f67b8849b80 1 -- 172.21.15.133:0/1087082115 wait complete. 2023-07-14T18:24:55.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.032+0000 7f67b8849b80 20 client.25867 trim_cache size 0 max 16384 2023-07-14T18:24:55.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.065+0000 7f67b8849b80 1 Processor -- start 2023-07-14T18:24:55.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.065+0000 7f67b8849b80 1 -- start start 2023-07-14T18:24:55.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.065+0000 7f67b8849b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d9e65ef60 0x563d9e666820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:55.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.065+0000 7f67b8849b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d9e65f8b0 0x563d9e666d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:55.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.065+0000 7f67b8849b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d9e6604b0 0x563d9e66fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:55.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.065+0000 7f67b8849b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9e670e80 con 0x563d9e6604b0 2023-07-14T18:24:55.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.065+0000 7f67b8849b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9e654250 con 0x563d9e65ef60 2023-07-14T18:24:55.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.066+0000 7f67b8849b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9e6665d0 con 0x563d9e65f8b0 2023-07-14T18:24:55.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.066+0000 7f679e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d9e65ef60 0x563d9e666820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:55.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.066+0000 7f679e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d9e65ef60 0x563d9e666820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:59478/0 (socket says 172.21.15.133:59478) 2023-07-14T18:24:55.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.066+0000 7f679e59c700 1 -- 172.21.15.133:0/3055769014 learned_addr learned my addr 172.21.15.133:0/3055769014 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:24:55.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.070+0000 7f679e59c700 1 -- 172.21.15.133:0/3055769014 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d9e65f8b0 msgr2=0x563d9e666d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T18:24:55.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.070+0000 7f679e59c700 1 --2- 172.21.15.133:0/3055769014 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d9e65f8b0 0x563d9e666d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:55.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.070+0000 7f679e59c700 1 -- 172.21.15.133:0/3055769014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d9e6604b0 msgr2=0x563d9e66fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:24:55.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.070+0000 7f679e59c700 1 --2- 172.21.15.133:0/3055769014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d9e6604b0 0x563d9e66fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:55.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.070+0000 7f679e59c700 1 -- 172.21.15.133:0/3055769014 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d9e404250 con 0x563d9e65ef60 2023-07-14T18:24:55.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.071+0000 7f679e59c700 1 --2- 172.21.15.133:0/3055769014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d9e65ef60 0x563d9e666820 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f679004a890 tx=0x7f6790009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=184d0f8375372117 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:24:55.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.071+0000 7f678d7fa700 1 -- 172.21.15.133:0/3055769014 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679004a410 con 0x563d9e65ef60 2023-07-14T18:24:55.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.071+0000 7f678d7fa700 1 -- 172.21.15.133:0/3055769014 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f679004a570 con 0x563d9e65ef60 2023-07-14T18:24:55.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.071+0000 7f678d7fa700 1 -- 172.21.15.133:0/3055769014 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679005d980 con 0x563d9e65ef60 2023-07-14T18:24:55.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.072+0000 7f67b8849b80 1 -- 172.21.15.133:0/3055769014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d9e65ef60 msgr2=0x563d9e666820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:24:55.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.072+0000 7f67b8849b80 1 --2- 172.21.15.133:0/3055769014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d9e65ef60 0x563d9e666820 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f679004a890 tx=0x7f6790009f30 comp rx=0 tx=0).stop 2023-07-14T18:24:55.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.080+0000 7f67b8849b80 1 -- 172.21.15.133:0/3055769014 shutdown_connections 2023-07-14T18:24:55.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.080+0000 7f67b8849b80 1 --2- 172.21.15.133:0/3055769014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d9e65ef60 0x563d9e666820 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:55.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.080+0000 7f67b8849b80 1 --2- 172.21.15.133:0/3055769014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d9e6604b0 0x563d9e66fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:55.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.080+0000 7f67b8849b80 1 --2- 172.21.15.133:0/3055769014 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d9e65f8b0 0x563d9e666d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:55.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.080+0000 7f67b8849b80 1 -- 172.21.15.133:0/3055769014 >> 172.21.15.133:0/3055769014 conn(0x563d9e654680 msgr2=0x563d9e6765e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:24:55.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.081+0000 7f67b8849b80 1 -- 172.21.15.133:0/3055769014 shutdown_connections 2023-07-14T18:24:55.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.081+0000 7f67b8849b80 1 -- 172.21.15.133:0/3055769014 wait complete. 2023-07-14T18:24:55.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.082+0000 7f67b8849b80 1 -- start start 2023-07-14T18:24:55.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.082+0000 7f67b8849b80 1 Processor -- start 2023-07-14T18:24:55.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.083+0000 7f67b8849b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d9e65ef60 0x563d9e66f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:55.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.083+0000 7f67b8849b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d9e65f8b0 0x563d9e66fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:55.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.083+0000 7f67b8849b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d9e6604b0 0x563d9e66c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:55.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.083+0000 7f67b8849b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9e66c6a0 con 0x563d9e65ef60 2023-07-14T18:24:55.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.083+0000 7f67b8849b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9e66c810 con 0x563d9e6604b0 2023-07-14T18:24:55.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.083+0000 7f67b8849b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9e66c9d0 con 0x563d9e65f8b0 2023-07-14T18:24:55.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.083+0000 7f67b8849b80 10 client.0 _ll_register_callbacks cb 0x7f67900359c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:24:55.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.083+0000 7f679ed9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d9e6604b0 0x563d9e66c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:55.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.083+0000 7f679ed9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d9e6604b0 0x563d9e66c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:59492/0 (socket says 172.21.15.133:59492) 2023-07-14T18:24:55.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.083+0000 7f679ed9d700 1 -- 172.21.15.133:0/2522082901 learned_addr learned my addr 172.21.15.133:0/2522082901 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:24:55.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.083+0000 7f679dd9b700 1 --2- 172.21.15.133:0/2522082901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d9e65f8b0 0x563d9e66fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:55.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.084+0000 7f679ed9d700 1 -- 172.21.15.133:0/2522082901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d9e65f8b0 msgr2=0x563d9e66fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:24:55.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.084+0000 7f679ed9d700 1 --2- 172.21.15.133:0/2522082901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d9e65f8b0 0x563d9e66fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:55.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.084+0000 7f679ed9d700 1 -- 172.21.15.133:0/2522082901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d9e65ef60 msgr2=0x563d9e66f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:24:55.143 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.084+0000 7f679ed9d700 1 --2- 172.21.15.133:0/2522082901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d9e65ef60 0x563d9e66f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:55.143 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.084+0000 7f679ed9d700 1 -- 172.21.15.133:0/2522082901 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d9e3c6c60 con 0x563d9e6604b0 2023-07-14T18:24:55.143 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.084+0000 7f679ed9d700 1 --2- 172.21.15.133:0/2522082901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d9e6604b0 0x563d9e66c120 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f67980062a0 tx=0x7f6798007de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1179727032aab7f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:24:55.143 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.084+0000 7f6785ffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:24:55.143 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.085+0000 7f6785ffb700 1 -- 172.21.15.133:0/2522082901 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679800e050 con 0x563d9e6604b0 2023-07-14T18:24:55.143 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.085+0000 7f6785ffb700 1 -- 172.21.15.133:0/2522082901 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6798010a10 con 0x563d9e6604b0 2023-07-14T18:24:55.143 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.085+0000 7f67b8849b80 1 -- 172.21.15.133:0/2522082901 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6790035480 con 0x563d9e6604b0 2023-07-14T18:24:55.143 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.086+0000 7f6785ffb700 1 -- 172.21.15.133:0/2522082901 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6798010bd0 con 0x563d9e6604b0 2023-07-14T18:24:55.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.086+0000 7f67b8849b80 10 client.25841 fetch_fsmap learned FSMap version 34 2023-07-14T18:24:55.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.086+0000 7f67b8849b80 1 -- 172.21.15.133:0/2522082901 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563d9e3dc2b0 con 0x563d9e6604b0 2023-07-14T18:24:55.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.092+0000 7f6785ffb700 1 -- 172.21.15.133:0/2522082901 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6790035480 con 0x563d9e6604b0 2023-07-14T18:24:55.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.093+0000 7f6785ffb700 1 -- 172.21.15.133:0/2522082901 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f67980163f0 con 0x563d9e6604b0 2023-07-14T18:24:55.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.093+0000 7f67b8849b80 10 client.25841 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T18:24:55.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.093+0000 7f67b8849b80 10 client.25841 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:24:55.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.093+0000 7f67b8849b80 20 client.25841 populate_metadata read hostname 'smithi133' 2023-07-14T18:24:55.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.093+0000 7f67b8849b80 1 --2- 172.21.15.133:0/2522082901 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x563d9e6c97c0 0x563d9e6cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:24:55.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.093+0000 7f67b8849b80 4 client.25841 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:24:55.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.093+0000 7f67b8849b80 1 -- 172.21.15.133:0/2522082901 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563d9e5a8eb0 con 0x563d9e6c97c0 2023-07-14T18:24:55.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.093+0000 7f679e59c700 1 --2- 172.21.15.133:0/2522082901 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x563d9e6c97c0 0x563d9e6cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:24:55.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.094+0000 7f679e59c700 1 --2- 172.21.15.133:0/2522082901 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x563d9e6c97c0 0x563d9e6cbbd0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4977818163d205e4 server_cookie=cdfd27069628ff29 in_seq=0 out_seq=0 2023-07-14T18:24:55.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.094+0000 7f6785ffb700 10 client.25841 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:24:55.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.094+0000 7f6785ffb700 1 -- 172.21.15.133:0/2522082901 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x563d9e5a8eb0 con 0x563d9e6c97c0 2023-07-14T18:24:55.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.094+0000 7f6785ffb700 10 client.25841 handle_command_reply: tid=0 2023-07-14T18:24:55.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.094+0000 7f67b8849b80 1 client.25841 shutdown 2023-07-14T18:24:55.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.095+0000 7f67b8849b80 1 -- 172.21.15.133:0/2522082901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d9e6604b0 msgr2=0x563d9e66c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:24:55.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.095+0000 7f67b8849b80 1 --2- 172.21.15.133:0/2522082901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d9e6604b0 0x563d9e66c120 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f67980062a0 tx=0x7f6798007de0 comp rx=0 tx=0).stop 2023-07-14T18:24:55.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.095+0000 7f67b8849b80 1 -- 172.21.15.133:0/2522082901 shutdown_connections 2023-07-14T18:24:55.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.095+0000 7f67b8849b80 1 --2- 172.21.15.133:0/2522082901 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x563d9e6c97c0 0x563d9e6cbbd0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:55.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.095+0000 7f67b8849b80 1 --2- 172.21.15.133:0/2522082901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d9e6604b0 0x563d9e66c120 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:55.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.095+0000 7f67b8849b80 1 --2- 172.21.15.133:0/2522082901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d9e65ef60 0x563d9e66f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:55.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.095+0000 7f67b8849b80 1 --2- 172.21.15.133:0/2522082901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563d9e65f8b0 0x563d9e66fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:55.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.095+0000 7f67b8849b80 1 -- 172.21.15.133:0/2522082901 >> 172.21.15.133:0/2522082901 conn(0x563d9e654680 msgr2=0x563d9e666820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:24:55.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.095+0000 7f6785ffb700 0 client.25841 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:24:55.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.095+0000 7f67b8849b80 1 -- 172.21.15.133:0/2522082901 shutdown_connections 2023-07-14T18:24:55.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.095+0000 7f67b8849b80 1 -- 172.21.15.133:0/2522082901 wait complete. 2023-07-14T18:24:55.150 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:24:55.150 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-14T18:24:55.151 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:24:55.151 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:24:55.151 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:24:55.151 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:24:55.151 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:24:55.151 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:24:55.151 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:24:55.152 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:24:55.152 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.096+0000 7f67b8849b80 20 client.25841 trim_cache size 0 max 16384 2023-07-14T18:24:55.152 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.097+0000 7f67a4fff700 1 -- 172.21.15.133:0/3773925589 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f67880a1260 msgr2=0x7f67880a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:24:55.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.097+0000 7f67a4fff700 1 --2- 172.21.15.133:0/3773925589 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f67880a1260 0x7f67880a3710 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f67900304c0 tx=0x7f6790004750 comp rx=0 tx=0).stop 2023-07-14T18:24:55.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.097+0000 7f67a4fff700 1 -- 172.21.15.133:0/3773925589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67a00ff950 msgr2=0x7f67a006b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:24:55.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.097+0000 7f67a4fff700 1 --2- 172.21.15.133:0/3773925589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67a00ff950 0x7f67a006b490 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f679400a040 tx=0x7f679400c7f0 comp rx=0 tx=0).stop 2023-07-14T18:24:55.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.097+0000 7f67a4fff700 1 -- 172.21.15.133:0/3773925589 shutdown_connections 2023-07-14T18:24:55.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.097+0000 7f67a4fff700 1 --2- 172.21.15.133:0/3773925589 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f67880a1260 0x7f67880a3710 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:55.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.097+0000 7f67a4fff700 1 --2- 172.21.15.133:0/3773925589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67a00fde50 0x7f67a006af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:55.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.097+0000 7f67a4fff700 1 --2- 172.21.15.133:0/3773925589 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67a00704f0 0x7f67a0070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:55.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.097+0000 7f67a4fff700 1 --2- 172.21.15.133:0/3773925589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67a00ff950 0x7f67a006b490 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:24:55.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.097+0000 7f67a4fff700 1 -- 172.21.15.133:0/3773925589 >> 172.21.15.133:0/3773925589 conn(0x7f67a00f9470 msgr2=0x7f67a00fb770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:24:55.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.098+0000 7f67a4fff700 1 -- 172.21.15.133:0/3773925589 shutdown_connections 2023-07-14T18:24:55.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:24:55.098+0000 7f67a4fff700 1 -- 172.21.15.133:0/3773925589 wait complete. 2023-07-14T18:24:55.180 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:24:55.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:54 smithi161 ceph-mon[128993]: pgmap v595: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 29 MiB/s wr, 2.26k op/s 2023-07-14T18:24:55.374 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:54 smithi171 ceph-mon[129241]: pgmap v595: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 29 MiB/s wr, 2.26k op/s 2023-07-14T18:24:55.420 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:54 smithi133 ceph-mon[126384]: pgmap v595: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 29 MiB/s wr, 2.26k op/s 2023-07-14T18:24:55.888 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 330.0 s, 750.1 tps, lat 145.321 ms stddev 433.361 2023-07-14T18:24:56.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:56 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:24:56.874 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:56 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:24:56.915 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:56 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:24:57.920 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:57 smithi133 ceph-mon[126384]: pgmap v596: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 22 MiB/s wr, 1.60k op/s 2023-07-14T18:24:57.920 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:57 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:24:57.920 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:57 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:24:57.920 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:57 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:24:58.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:57 smithi161 ceph-mon[128993]: pgmap v596: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 22 MiB/s wr, 1.60k op/s 2023-07-14T18:24:58.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:57 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:24:58.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:57 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:24:58.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:57 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:24:58.124 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:57 smithi171 ceph-mon[129241]: pgmap v596: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 22 MiB/s wr, 1.60k op/s 2023-07-14T18:24:58.124 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:57 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:24:58.124 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:57 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:24:58.125 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:57 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:24:59.915 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:24:59 smithi133 ceph-mon[126384]: pgmap v597: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 21 MiB/s wr, 1.45k op/s 2023-07-14T18:25:00.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:24:59 smithi161 ceph-mon[128993]: pgmap v597: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 21 MiB/s wr, 1.45k op/s 2023-07-14T18:25:00.125 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:24:59 smithi171 ceph-mon[129241]: pgmap v597: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 21 MiB/s wr, 1.45k op/s 2023-07-14T18:25:00.884 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 335.0 s, 1220.2 tps, lat 76.749 ms stddev 63.435 2023-07-14T18:25:02.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:01 smithi161 ceph-mon[128993]: pgmap v598: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 24 MiB/s wr, 1.49k op/s 2023-07-14T18:25:02.124 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:01 smithi171 ceph-mon[129241]: pgmap v598: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 24 MiB/s wr, 1.49k op/s 2023-07-14T18:25:02.166 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:01 smithi133 ceph-mon[126384]: pgmap v598: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 24 MiB/s wr, 1.49k op/s 2023-07-14T18:25:04.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:03 smithi161 ceph-mon[128993]: pgmap v599: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 22 MiB/s wr, 1.41k op/s 2023-07-14T18:25:04.124 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:03 smithi171 ceph-mon[129241]: pgmap v599: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 22 MiB/s wr, 1.41k op/s 2023-07-14T18:25:04.171 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:03 smithi133 ceph-mon[126384]: pgmap v599: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 22 MiB/s wr, 1.41k op/s 2023-07-14T18:25:05.886 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 340.0 s, 24.6 tps, lat 3782.847 ms stddev 1123.481 2023-07-14T18:25:06.108 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:05 smithi161 ceph-mon[128993]: pgmap v600: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 29 MiB/s wr, 2.06k op/s 2023-07-14T18:25:06.127 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:05 smithi171 ceph-mon[129241]: pgmap v600: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 29 MiB/s wr, 2.06k op/s 2023-07-14T18:25:06.173 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:05 smithi133 ceph-mon[126384]: pgmap v600: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 29 MiB/s wr, 2.06k op/s 2023-07-14T18:25:08.123 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:07 smithi171 ceph-mon[129241]: pgmap v601: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 21 MiB/s wr, 1.35k op/s 2023-07-14T18:25:08.171 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:07 smithi133 ceph-mon[126384]: pgmap v601: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 21 MiB/s wr, 1.35k op/s 2023-07-14T18:25:08.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:07 smithi161 ceph-mon[128993]: pgmap v601: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 21 MiB/s wr, 1.35k op/s 2023-07-14T18:25:10.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:09 smithi161 ceph-mon[128993]: pgmap v602: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 20 MiB/s wr, 1.30k op/s 2023-07-14T18:25:10.373 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:09 smithi171 ceph-mon[129241]: pgmap v602: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 20 MiB/s wr, 1.30k op/s 2023-07-14T18:25:10.415 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:09 smithi133 ceph-mon[126384]: pgmap v602: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 20 MiB/s wr, 1.30k op/s 2023-07-14T18:25:10.891 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 345.0 s, 544.7 tps, lat 199.273 ms stddev 312.564 2023-07-14T18:25:12.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:11 smithi161 ceph-mon[128993]: pgmap v603: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 22 MiB/s wr, 1.59k op/s 2023-07-14T18:25:12.379 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:11 smithi171 ceph-mon[129241]: pgmap v603: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 22 MiB/s wr, 1.59k op/s 2023-07-14T18:25:12.414 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:11 smithi133 ceph-mon[126384]: pgmap v603: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 22 MiB/s wr, 1.59k op/s 2023-07-14T18:25:13.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:12 smithi161 ceph-mon[128993]: pgmap v604: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 18 MiB/s wr, 1.54k op/s 2023-07-14T18:25:13.373 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:12 smithi171 ceph-mon[129241]: pgmap v604: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 18 MiB/s wr, 1.54k op/s 2023-07-14T18:25:13.415 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:12 smithi133 ceph-mon[126384]: pgmap v604: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 18 MiB/s wr, 1.54k op/s 2023-07-14T18:25:15.361 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:15 smithi161 ceph-mon[128993]: pgmap v605: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 20 MiB/s wr, 1.73k op/s 2023-07-14T18:25:15.374 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:15 smithi171 ceph-mon[129241]: pgmap v605: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 20 MiB/s wr, 1.73k op/s 2023-07-14T18:25:15.414 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:15 smithi133 ceph-mon[126384]: pgmap v605: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 20 MiB/s wr, 1.73k op/s 2023-07-14T18:25:15.883 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 350.0 s, 940.1 tps, lat 109.082 ms stddev 126.591 2023-07-14T18:25:17.913 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:17 smithi133 ceph-mon[126384]: pgmap v606: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-07-14T18:25:18.107 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:17 smithi161 ceph-mon[128993]: pgmap v606: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-07-14T18:25:18.122 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:17 smithi171 ceph-mon[129241]: pgmap v606: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-07-14T18:25:20.108 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:19 smithi161 ceph-mon[128993]: pgmap v607: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 14 MiB/s wr, 982 op/s 2023-07-14T18:25:20.124 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:19 smithi171 ceph-mon[129241]: pgmap v607: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 14 MiB/s wr, 982 op/s 2023-07-14T18:25:20.166 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:19 smithi133 ceph-mon[126384]: pgmap v607: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 14 MiB/s wr, 982 op/s 2023-07-14T18:25:20.882 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 355.0 s, 1132.2 tps, lat 80.967 ms stddev 62.355 2023-07-14T18:25:22.114 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:21 smithi161 ceph-mon[128993]: pgmap v608: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 17 MiB/s wr, 960 op/s 2023-07-14T18:25:22.122 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:21 smithi171 ceph-mon[129241]: pgmap v608: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 17 MiB/s wr, 960 op/s 2023-07-14T18:25:22.168 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:21 smithi133 ceph-mon[126384]: pgmap v608: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 17 MiB/s wr, 960 op/s 2023-07-14T18:25:24.114 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:23 smithi161 ceph-mon[128993]: pgmap v609: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 16 MiB/s wr, 725 op/s 2023-07-14T18:25:24.124 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:23 smithi171 ceph-mon[129241]: pgmap v609: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 16 MiB/s wr, 725 op/s 2023-07-14T18:25:24.169 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:23 smithi133 ceph-mon[126384]: pgmap v609: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 16 MiB/s wr, 725 op/s 2023-07-14T18:25:25.181 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:25:25.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.552+0000 7f94fe7ff700 1 -- 172.21.15.133:0/256818335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94f8077340 msgr2=0x7f94f80797c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:25:25.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.552+0000 7f94fe7ff700 1 --2- 172.21.15.133:0/256818335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94f8077340 0x7f94f80797c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f94f000d330 tx=0x7f94f000d640 comp rx=0 tx=0).stop 2023-07-14T18:25:25.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.553+0000 7f94fe7ff700 1 -- 172.21.15.133:0/256818335 shutdown_connections 2023-07-14T18:25:25.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.553+0000 7f94fe7ff700 1 --2- 172.21.15.133:0/256818335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94f8071330 0x7f94f8071740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.553+0000 7f94fe7ff700 1 --2- 172.21.15.133:0/256818335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94f8077340 0x7f94f80797c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.553+0000 7f94fe7ff700 1 --2- 172.21.15.133:0/256818335 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94f8074980 0x7f94f8076e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.553+0000 7f94fe7ff700 1 -- 172.21.15.133:0/256818335 >> 172.21.15.133:0/256818335 conn(0x7f94f806cbc0 msgr2=0x7f94f806eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:25:25.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.563+0000 7f94fe7ff700 1 -- 172.21.15.133:0/256818335 shutdown_connections 2023-07-14T18:25:25.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.568+0000 7f94fe7ff700 1 -- 172.21.15.133:0/256818335 wait complete. 2023-07-14T18:25:25.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.569+0000 7f94fe7ff700 1 Processor -- start 2023-07-14T18:25:25.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.571+0000 7f94fe7ff700 1 -- start start 2023-07-14T18:25:25.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.572+0000 7f94fe7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94f8071330 0x7f94f8082580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:25.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.572+0000 7f94fe7ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94f8074980 0x7f94f8082ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:25.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.572+0000 7f94fe7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94f8077340 0x7f94f81bce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:25.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.572+0000 7f94fe7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f81bd3e0 con 0x7f94f8071330 2023-07-14T18:25:25.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.572+0000 7f94fe7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f8129680 con 0x7f94f8077340 2023-07-14T18:25:25.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.572+0000 7f94fe7ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f8129840 con 0x7f94f8074980 2023-07-14T18:25:25.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.573+0000 7f94f77fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94f8074980 0x7f94f8082ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:25:25.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.573+0000 7f94f77fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94f8074980 0x7f94f8082ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:59768/0 (socket says 172.21.15.133:59768) 2023-07-14T18:25:25.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.573+0000 7f94f77fe700 1 -- 172.21.15.133:0/969821730 learned_addr learned my addr 172.21.15.133:0/969821730 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:25:25.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.574+0000 7f94f77fe700 1 -- 172.21.15.133:0/969821730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94f8077340 msgr2=0x7f94f81bce10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-14T18:25:25.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.574+0000 7f94f77fe700 1 --2- 172.21.15.133:0/969821730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94f8077340 0x7f94f81bce10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.574+0000 7f94f77fe700 1 -- 172.21.15.133:0/969821730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94f8071330 msgr2=0x7f94f8082580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T18:25:25.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.574+0000 7f94f77fe700 1 --2- 172.21.15.133:0/969821730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94f8071330 0x7f94f8082580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.574+0000 7f94f77fe700 1 -- 172.21.15.133:0/969821730 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f0009d00 con 0x7f94f8074980 2023-07-14T18:25:25.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.575+0000 7f94f77fe700 1 --2- 172.21.15.133:0/969821730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94f8074980 0x7f94f8082ac0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f94e800dad0 tx=0x7f94e800de90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:25:25.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.576+0000 7f94f57fa700 1 -- 172.21.15.133:0/969821730 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e800caf0 con 0x7f94f8074980 2023-07-14T18:25:25.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.576+0000 7f94f57fa700 1 -- 172.21.15.133:0/969821730 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94e800cc50 con 0x7f94f8074980 2023-07-14T18:25:25.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.576+0000 7f94f57fa700 1 -- 172.21.15.133:0/969821730 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e8014610 con 0x7f94f8074980 2023-07-14T18:25:25.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.576+0000 7f94fe7ff700 1 -- 172.21.15.133:0/969821730 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f8129b20 con 0x7f94f8074980 2023-07-14T18:25:25.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.576+0000 7f94fe7ff700 1 -- 172.21.15.133:0/969821730 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f8129fe0 con 0x7f94f8074980 2023-07-14T18:25:25.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.585+0000 7f94f57fa700 1 -- 172.21.15.133:0/969821730 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f94e8014770 con 0x7f94f8074980 2023-07-14T18:25:25.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.588+0000 7f94f57fa700 1 --2- 172.21.15.133:0/969821730 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f94e00a1350 0x7f94e00a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:25.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.588+0000 7f94f57fa700 1 -- 172.21.15.133:0/969821730 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f94e80cd570 con 0x7f94f8074980 2023-07-14T18:25:25.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.590+0000 7f951212db80 1 Processor -- start 2023-07-14T18:25:25.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.590+0000 7f951212db80 1 -- start start 2023-07-14T18:25:25.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.591+0000 7f951212db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635f484bf60 0x5635f484c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:25.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.591+0000 7f951212db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635f484c8b0 0x5635f484ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:25.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.591+0000 7f951212db80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635f484d4b0 0x5635f484d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:25.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.591+0000 7f951212db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f4750cb0 con 0x5635f484c8b0 2023-07-14T18:25:25.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.591+0000 7f951212db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f47a3a40 con 0x5635f484bf60 2023-07-14T18:25:25.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.591+0000 7f951212db80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f479ced0 con 0x5635f484d4b0 2023-07-14T18:25:25.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.591+0000 7f94fcd9c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635f484d4b0 0x5635f484d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:25:25.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.591+0000 7f94fcd9c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635f484d4b0 0x5635f484d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:59780/0 (socket says 172.21.15.133:59780) 2023-07-14T18:25:25.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.591+0000 7f94fcd9c700 1 -- 172.21.15.133:0/2817627008 learned_addr learned my addr 172.21.15.133:0/2817627008 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:25:25.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.592+0000 7f94f77fe700 1 --2- 172.21.15.133:0/2817627008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635f484c8b0 0x5635f484ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:25:25.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.592+0000 7f94fcd9c700 1 -- 172.21.15.133:0/2817627008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635f484bf60 msgr2=0x5635f484c370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T18:25:25.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.592+0000 7f94fcd9c700 1 --2- 172.21.15.133:0/2817627008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635f484bf60 0x5635f484c370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.592+0000 7f94fcd9c700 1 -- 172.21.15.133:0/2817627008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635f484c8b0 msgr2=0x5635f484ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:25:25.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.592+0000 7f94fcd9c700 1 --2- 172.21.15.133:0/2817627008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635f484c8b0 0x5635f484ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.592+0000 7f94fcd9c700 1 -- 172.21.15.133:0/2817627008 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635f45cade0 con 0x5635f484d4b0 2023-07-14T18:25:25.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.592+0000 7f94fcd9c700 1 --2- 172.21.15.133:0/2817627008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635f484d4b0 0x5635f484d920 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f94f0007e10 tx=0x7f94f000df40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71cda32d1ff330b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:25:25.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.593+0000 7f94de7fc700 1 -- 172.21.15.133:0/2817627008 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f0009150 con 0x5635f484d4b0 2023-07-14T18:25:25.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.593+0000 7f94de7fc700 1 -- 172.21.15.133:0/2817627008 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94f00092b0 con 0x5635f484d4b0 2023-07-14T18:25:25.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.593+0000 7f94de7fc700 1 -- 172.21.15.133:0/2817627008 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f001a450 con 0x5635f484d4b0 2023-07-14T18:25:25.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.594+0000 7f951212db80 1 -- 172.21.15.133:0/2817627008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635f484d4b0 msgr2=0x5635f484d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:25:25.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.594+0000 7f951212db80 1 --2- 172.21.15.133:0/2817627008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635f484d4b0 0x5635f484d920 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f94f0007e10 tx=0x7f94f000df40 comp rx=0 tx=0).stop 2023-07-14T18:25:25.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.594+0000 7f951212db80 1 -- 172.21.15.133:0/2817627008 shutdown_connections 2023-07-14T18:25:25.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.594+0000 7f951212db80 1 --2- 172.21.15.133:0/2817627008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635f484bf60 0x5635f484c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.594+0000 7f951212db80 1 --2- 172.21.15.133:0/2817627008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635f484d4b0 0x5635f484d920 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.594+0000 7f951212db80 1 --2- 172.21.15.133:0/2817627008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635f484c8b0 0x5635f484ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.594+0000 7f951212db80 1 -- 172.21.15.133:0/2817627008 >> 172.21.15.133:0/2817627008 conn(0x5635f4841680 msgr2=0x5635f48469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:25:25.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.596+0000 7f94f7fff700 1 --2- 172.21.15.133:0/969821730 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f94e00a1350 0x7f94e00a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:25:25.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.599+0000 7f94f7fff700 1 --2- 172.21.15.133:0/969821730 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f94e00a1350 0x7f94e00a3800 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f94ec005cd0 tx=0x7f94ec00c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:25:25.654 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.604+0000 7f951212db80 1 -- 172.21.15.133:0/2817627008 shutdown_connections 2023-07-14T18:25:25.655 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.605+0000 7f951212db80 1 -- 172.21.15.133:0/2817627008 wait complete. 2023-07-14T18:25:25.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.606+0000 7f951212db80 1 -- start start 2023-07-14T18:25:25.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.606+0000 7f951212db80 1 Processor -- start 2023-07-14T18:25:25.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.607+0000 7f951212db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635f484bf60 0x5635f4855900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:25.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.607+0000 7f951212db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635f484c8b0 0x5635f4855e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:25.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.607+0000 7f951212db80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635f484d4b0 0x5635f485ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:25.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.607+0000 7f951212db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f473cac0 con 0x5635f484c8b0 2023-07-14T18:25:25.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.607+0000 7f951212db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f473c7c0 con 0x5635f484bf60 2023-07-14T18:25:25.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.607+0000 7f951212db80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f4620920 con 0x5635f484d4b0 2023-07-14T18:25:25.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.607+0000 7f951212db80 10 client.0 _ll_register_callbacks cb 0x5635f473a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:25:25.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.607+0000 7f94f7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635f484bf60 0x5635f4855900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:25:25.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.607+0000 7f94f7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635f484bf60 0x5635f4855900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:51542/0 (socket says 172.21.15.133:51542) 2023-07-14T18:25:25.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.607+0000 7f94f7fff700 1 -- 172.21.15.133:0/817558552 learned_addr learned my addr 172.21.15.133:0/817558552 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:25:25.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.607+0000 7f94f77fe700 1 --2- 172.21.15.133:0/817558552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635f484c8b0 0x5635f4855e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:25:25.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.610+0000 7f94fcd9c700 1 --2- 172.21.15.133:0/817558552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635f484d4b0 0x5635f485ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:25:25.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.611+0000 7f94f77fe700 1 -- 172.21.15.133:0/817558552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635f484d4b0 msgr2=0x5635f485ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:25:25.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.611+0000 7f94f77fe700 1 --2- 172.21.15.133:0/817558552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635f484d4b0 0x5635f485ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.611+0000 7f94f77fe700 1 -- 172.21.15.133:0/817558552 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635f484bf60 msgr2=0x5635f4855900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:25:25.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.611+0000 7f94f77fe700 1 --2- 172.21.15.133:0/817558552 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635f484bf60 0x5635f4855900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.611+0000 7f94f77fe700 1 -- 172.21.15.133:0/817558552 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f807a1f0 con 0x5635f484c8b0 2023-07-14T18:25:25.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.614+0000 7f94f77fe700 1 --2- 172.21.15.133:0/817558552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635f484c8b0 0x5635f4855e40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f94e800dfd0 tx=0x7f94e809ab30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255d6a551a6ae532 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:25:25.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.614+0000 7f94d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T18:25:25.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.614+0000 7f94d77fe700 1 -- 172.21.15.133:0/817558552 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e809a5f0 con 0x5635f484c8b0 2023-07-14T18:25:25.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.614+0000 7f94d77fe700 1 -- 172.21.15.133:0/817558552 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f94e809e620 con 0x5635f484c8b0 2023-07-14T18:25:25.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.615+0000 7f951212db80 1 -- 172.21.15.133:0/817558552 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635f46f64f0 con 0x5635f484c8b0 2023-07-14T18:25:25.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.616+0000 7f94d77fe700 1 -- 172.21.15.133:0/817558552 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94e80a2a40 con 0x5635f484c8b0 2023-07-14T18:25:25.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.616+0000 7f951212db80 10 client.15792 fetch_fsmap learned FSMap version 34 2023-07-14T18:25:25.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.616+0000 7f951212db80 1 -- 172.21.15.133:0/817558552 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5635f45b3c60 con 0x5635f484c8b0 2023-07-14T18:25:25.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.617+0000 7f94d77fe700 1 -- 172.21.15.133:0/817558552 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5635f46f64f0 con 0x5635f484c8b0 2023-07-14T18:25:25.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.617+0000 7f94d77fe700 1 -- 172.21.15.133:0/817558552 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f94e8099550 con 0x5635f484c8b0 2023-07-14T18:25:25.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.617+0000 7f951212db80 10 client.15792 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T18:25:25.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.617+0000 7f951212db80 10 client.15792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:25:25.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.617+0000 7f951212db80 20 client.15792 populate_metadata read hostname 'smithi133' 2023-07-14T18:25:25.671 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.617+0000 7f951212db80 1 --2- 172.21.15.133:0/817558552 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5635f4861790 0x5635f4863ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:25.671 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.617+0000 7f951212db80 4 client.15792 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:25:25.671 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.617+0000 7f951212db80 1 -- 172.21.15.133:0/817558552 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635f481ada0 con 0x5635f4861790 2023-07-14T18:25:25.671 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.619+0000 7f94f7fff700 1 --2- 172.21.15.133:0/817558552 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5635f4861790 0x5635f4863ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:25:25.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.620+0000 7f94f7fff700 1 --2- 172.21.15.133:0/817558552 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5635f4861790 0x5635f4863ba0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ffae8b8239e114a server_cookie=8b2ba2480eaa1b1c in_seq=0 out_seq=0 2023-07-14T18:25:25.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.621+0000 7f94d77fe700 10 client.15792 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:25:25.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.621+0000 7f94d77fe700 1 -- 172.21.15.133:0/817558552 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5635f481ada0 con 0x5635f4861790 2023-07-14T18:25:25.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.621+0000 7f94d77fe700 10 client.15792 handle_command_reply: tid=0 2023-07-14T18:25:25.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.621+0000 7f951212db80 1 client.15792 shutdown 2023-07-14T18:25:25.673 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.622+0000 7f951212db80 1 -- 172.21.15.133:0/817558552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635f484c8b0 msgr2=0x5635f4855e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:25:25.673 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.622+0000 7f951212db80 1 --2- 172.21.15.133:0/817558552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635f484c8b0 0x5635f4855e40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f94e800dfd0 tx=0x7f94e809ab30 comp rx=0 tx=0).stop 2023-07-14T18:25:25.673 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.622+0000 7f951212db80 1 -- 172.21.15.133:0/817558552 shutdown_connections 2023-07-14T18:25:25.673 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.622+0000 7f951212db80 1 --2- 172.21.15.133:0/817558552 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5635f4861790 0x5635f4863ba0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.622+0000 7f951212db80 1 --2- 172.21.15.133:0/817558552 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635f484bf60 0x5635f4855900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.622+0000 7f951212db80 1 --2- 172.21.15.133:0/817558552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635f484d4b0 0x5635f485ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.622+0000 7f951212db80 1 --2- 172.21.15.133:0/817558552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635f484c8b0 0x5635f4855e40 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.622+0000 7f951212db80 1 -- 172.21.15.133:0/817558552 >> 172.21.15.133:0/817558552 conn(0x5635f4841680 msgr2=0x5635f4845650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:25:25.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.622+0000 7f94d77fe700 0 client.15792 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:25:25.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.622+0000 7f951212db80 1 -- 172.21.15.133:0/817558552 shutdown_connections 2023-07-14T18:25:25.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.623+0000 7f951212db80 1 -- 172.21.15.133:0/817558552 wait complete. 2023-07-14T18:25:25.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.624+0000 7f951212db80 20 client.15792 trim_cache size 0 max 16384 2023-07-14T18:25:25.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.656+0000 7f951212db80 1 Processor -- start 2023-07-14T18:25:25.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.656+0000 7f951212db80 1 -- start start 2023-07-14T18:25:25.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.657+0000 7f951212db80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635f484bf60 0x5635f48537a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:25.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.657+0000 7f951212db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635f484c8b0 0x5635f485cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:25.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.657+0000 7f951212db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635f484d4b0 0x5635f4855900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:25.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.657+0000 7f951212db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f4856800 con 0x5635f484d4b0 2023-07-14T18:25:25.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.657+0000 7f951212db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f4855470 con 0x5635f484c8b0 2023-07-14T18:25:25.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.657+0000 7f951212db80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f4848200 con 0x5635f484bf60 2023-07-14T18:25:25.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.658+0000 7f94f7fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635f484bf60 0x5635f48537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:25:25.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.658+0000 7f94f7fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635f484bf60 0x5635f48537a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:59792/0 (socket says 172.21.15.133:59792) 2023-07-14T18:25:25.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.658+0000 7f94f7fff700 1 -- 172.21.15.133:0/2736743804 learned_addr learned my addr 172.21.15.133:0/2736743804 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:25:25.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.658+0000 7f94f7fff700 1 -- 172.21.15.133:0/2736743804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635f484c8b0 msgr2=0x5635f485cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:25:25.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.658+0000 7f94f7fff700 1 --2- 172.21.15.133:0/2736743804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635f484c8b0 0x5635f485cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.658+0000 7f94f7fff700 1 -- 172.21.15.133:0/2736743804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635f484d4b0 msgr2=0x5635f4855900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:25:25.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.658+0000 7f94f7fff700 1 --2- 172.21.15.133:0/2736743804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635f484d4b0 0x5635f4855900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.658+0000 7f94f7fff700 1 -- 172.21.15.133:0/2736743804 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635f45f1250 con 0x5635f484bf60 2023-07-14T18:25:25.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.659+0000 7f94f7fff700 1 --2- 172.21.15.133:0/2736743804 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635f484bf60 0x5635f48537a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00a730 tx=0x7f94ec010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d89f3d7a1aa7194 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:25:25.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.659+0000 7f94deffd700 1 -- 172.21.15.133:0/2736743804 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ec01c040 con 0x5635f484bf60 2023-07-14T18:25:25.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.659+0000 7f94deffd700 1 -- 172.21.15.133:0/2736743804 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94ec01d040 con 0x5635f484bf60 2023-07-14T18:25:25.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.659+0000 7f94deffd700 1 -- 172.21.15.133:0/2736743804 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ec00f040 con 0x5635f484bf60 2023-07-14T18:25:25.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.660+0000 7f951212db80 1 -- 172.21.15.133:0/2736743804 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635f484bf60 msgr2=0x5635f48537a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:25:25.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.660+0000 7f951212db80 1 --2- 172.21.15.133:0/2736743804 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635f484bf60 0x5635f48537a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00a730 tx=0x7f94ec010000 comp rx=0 tx=0).stop 2023-07-14T18:25:25.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.661+0000 7f951212db80 1 -- 172.21.15.133:0/2736743804 shutdown_connections 2023-07-14T18:25:25.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.661+0000 7f951212db80 1 --2- 172.21.15.133:0/2736743804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635f484d4b0 0x5635f4855900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.661+0000 7f951212db80 1 --2- 172.21.15.133:0/2736743804 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635f484bf60 0x5635f48537a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.661+0000 7f951212db80 1 --2- 172.21.15.133:0/2736743804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635f484c8b0 0x5635f485cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.661+0000 7f951212db80 1 -- 172.21.15.133:0/2736743804 >> 172.21.15.133:0/2736743804 conn(0x5635f4841680 msgr2=0x5635f48637a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:25:25.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.662+0000 7f951212db80 1 -- 172.21.15.133:0/2736743804 shutdown_connections 2023-07-14T18:25:25.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.662+0000 7f951212db80 1 -- 172.21.15.133:0/2736743804 wait complete. 2023-07-14T18:25:25.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.663+0000 7f951212db80 1 -- start start 2023-07-14T18:25:25.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.664+0000 7f951212db80 1 Processor -- start 2023-07-14T18:25:25.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.665+0000 7f951212db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635f484bf60 0x5635f485ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:25.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.665+0000 7f951212db80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635f484d4b0 0x5635f4859750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:25.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.665+0000 7f951212db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635f4859c90 0x5635f48b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:25.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.665+0000 7f951212db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f485a250 con 0x5635f484bf60 2023-07-14T18:25:25.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.665+0000 7f951212db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f485a3c0 con 0x5635f4859c90 2023-07-14T18:25:25.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.665+0000 7f951212db80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f48b59a0 con 0x5635f484d4b0 2023-07-14T18:25:25.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.665+0000 7f951212db80 10 client.0 _ll_register_callbacks cb 0x7f94e8099f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:25:25.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.665+0000 7f94fcd9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635f4859c90 0x5635f48b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:25:25.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.665+0000 7f94fcd9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635f4859c90 0x5635f48b53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:51572/0 (socket says 172.21.15.133:51572) 2023-07-14T18:25:25.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.665+0000 7f94fcd9c700 1 -- 172.21.15.133:0/3167586326 learned_addr learned my addr 172.21.15.133:0/3167586326 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:25:25.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.666+0000 7f94f7fff700 1 --2- 172.21.15.133:0/3167586326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635f484bf60 0x5635f485ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:25:25.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.666+0000 7f94f77fe700 1 --2- 172.21.15.133:0/3167586326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635f484d4b0 0x5635f4859750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:25:25.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.666+0000 7f94fcd9c700 1 -- 172.21.15.133:0/3167586326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635f484d4b0 msgr2=0x5635f4859750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:25:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.666+0000 7f94fcd9c700 1 --2- 172.21.15.133:0/3167586326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635f484d4b0 0x5635f4859750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.666+0000 7f94fcd9c700 1 -- 172.21.15.133:0/3167586326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635f484bf60 msgr2=0x5635f485ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:25:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.666+0000 7f94fcd9c700 1 --2- 172.21.15.133:0/3167586326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635f484bf60 0x5635f485ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.666+0000 7f94fcd9c700 1 -- 172.21.15.133:0/3167586326 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ec007840 con 0x5635f4859c90 2023-07-14T18:25:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.666+0000 7f94f7fff700 1 --2- 172.21.15.133:0/3167586326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635f484bf60 0x5635f485ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T18:25:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.667+0000 7f94fcd9c700 1 --2- 172.21.15.133:0/3167586326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635f4859c90 0x5635f48b53e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f94f0007e10 tx=0x7f94f0009150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edda5e527f674176 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:25:25.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.667+0000 7f94d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:25:25.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.667+0000 7f94d77fe700 1 -- 172.21.15.133:0/3167586326 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f00077f0 con 0x5635f4859c90 2023-07-14T18:25:25.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.667+0000 7f94d77fe700 1 -- 172.21.15.133:0/3167586326 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f94f0007950 con 0x5635f4859c90 2023-07-14T18:25:25.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.667+0000 7f951212db80 1 -- 172.21.15.133:0/3167586326 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94f00092b0 con 0x5635f4859c90 2023-07-14T18:25:25.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.668+0000 7f94d77fe700 1 -- 172.21.15.133:0/3167586326 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94f0007b10 con 0x5635f4859c90 2023-07-14T18:25:25.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.668+0000 7f951212db80 10 client.25850 fetch_fsmap learned FSMap version 34 2023-07-14T18:25:25.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.668+0000 7f951212db80 1 -- 172.21.15.133:0/3167586326 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5635f45c92b0 con 0x5635f4859c90 2023-07-14T18:25:25.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.668+0000 7f94d77fe700 1 -- 172.21.15.133:0/3167586326 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94f00092b0 con 0x5635f4859c90 2023-07-14T18:25:25.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.669+0000 7f94d77fe700 1 -- 172.21.15.133:0/3167586326 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f94f0021070 con 0x5635f4859c90 2023-07-14T18:25:25.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.669+0000 7f951212db80 10 client.25850 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T18:25:25.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.669+0000 7f951212db80 10 client.25850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:25:25.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.669+0000 7f951212db80 20 client.25850 populate_metadata read hostname 'smithi133' 2023-07-14T18:25:25.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.669+0000 7f951212db80 1 --2- 172.21.15.133:0/3167586326 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5635f48b6f70 0x5635f48b9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:25.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.669+0000 7f951212db80 4 client.25850 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:25:25.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.669+0000 7f951212db80 1 -- 172.21.15.133:0/3167586326 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5635f4795eb0 con 0x5635f48b6f70 2023-07-14T18:25:25.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.670+0000 7f94f7fff700 1 --2- 172.21.15.133:0/3167586326 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5635f48b6f70 0x5635f48b9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:25:25.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.670+0000 7f94d77fe700 10 client.25850 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:25:25.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.670+0000 7f94f7fff700 1 --2- 172.21.15.133:0/3167586326 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5635f48b6f70 0x5635f48b9380 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e15c50cde20c3d33 server_cookie=86283a055d53a999 in_seq=0 out_seq=0 2023-07-14T18:25:25.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.671+0000 7f94d77fe700 1 -- 172.21.15.133:0/3167586326 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5635f4795eb0 con 0x5635f48b6f70 2023-07-14T18:25:25.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.671+0000 7f94d77fe700 10 client.25850 handle_command_reply: tid=0 2023-07-14T18:25:25.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.671+0000 7f951212db80 1 client.25850 shutdown 2023-07-14T18:25:25.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.672+0000 7f951212db80 1 -- 172.21.15.133:0/3167586326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635f4859c90 msgr2=0x5635f48b53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:25:25.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.672+0000 7f951212db80 1 --2- 172.21.15.133:0/3167586326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635f4859c90 0x5635f48b53e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f94f0007e10 tx=0x7f94f0009150 comp rx=0 tx=0).stop 2023-07-14T18:25:25.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.672+0000 7f951212db80 1 -- 172.21.15.133:0/3167586326 shutdown_connections 2023-07-14T18:25:25.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.672+0000 7f951212db80 1 --2- 172.21.15.133:0/3167586326 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5635f48b6f70 0x5635f48b9380 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.672+0000 7f951212db80 1 --2- 172.21.15.133:0/3167586326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635f4859c90 0x5635f48b53e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.672+0000 7f951212db80 1 --2- 172.21.15.133:0/3167586326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5635f484bf60 0x5635f485ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.672+0000 7f951212db80 1 --2- 172.21.15.133:0/3167586326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635f484d4b0 0x5635f4859750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.672+0000 7f951212db80 1 -- 172.21.15.133:0/3167586326 >> 172.21.15.133:0/3167586326 conn(0x5635f4841680 msgr2=0x5635f48537a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:25:25.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.672+0000 7f94d77fe700 0 client.25850 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:25:25.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.673+0000 7f951212db80 1 -- 172.21.15.133:0/3167586326 shutdown_connections 2023-07-14T18:25:25.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.673+0000 7f951212db80 1 -- 172.21.15.133:0/3167586326 wait complete. 2023-07-14T18:25:25.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.673+0000 7f951212db80 20 client.25850 trim_cache size 0 max 16384 2023-07-14T18:25:25.730 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:25:25.730 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-14T18:25:25.730 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:25:25.731 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:25:25.731 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:25:25.731 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:25:25.731 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:25:25.731 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:25:25.732 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:25:25.732 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:25:25.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.673+0000 7f94fe7ff700 1 -- 172.21.15.133:0/969821730 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f94e00a1350 msgr2=0x7f94e00a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:25:25.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.673+0000 7f94fe7ff700 1 --2- 172.21.15.133:0/969821730 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f94e00a1350 0x7f94e00a3800 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f94ec005cd0 tx=0x7f94ec00c040 comp rx=0 tx=0).stop 2023-07-14T18:25:25.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.673+0000 7f94fe7ff700 1 -- 172.21.15.133:0/969821730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94f8074980 msgr2=0x7f94f8082ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:25:25.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.673+0000 7f94fe7ff700 1 --2- 172.21.15.133:0/969821730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94f8074980 0x7f94f8082ac0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f94e800dad0 tx=0x7f94e800de90 comp rx=0 tx=0).stop 2023-07-14T18:25:25.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.674+0000 7f94fe7ff700 1 -- 172.21.15.133:0/969821730 shutdown_connections 2023-07-14T18:25:25.735 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.674+0000 7f94fe7ff700 1 --2- 172.21.15.133:0/969821730 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f94e00a1350 0x7f94e00a3800 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.735 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.674+0000 7f94fe7ff700 1 --2- 172.21.15.133:0/969821730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94f8077340 0x7f94f81bce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.735 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.674+0000 7f94fe7ff700 1 --2- 172.21.15.133:0/969821730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94f8071330 0x7f94f8082580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.674+0000 7f94fe7ff700 1 --2- 172.21.15.133:0/969821730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94f8074980 0x7f94f8082ac0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:25.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.674+0000 7f94fe7ff700 1 -- 172.21.15.133:0/969821730 >> 172.21.15.133:0/969821730 conn(0x7f94f806cbc0 msgr2=0x7f94f80761f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:25:25.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.674+0000 7f94fe7ff700 1 -- 172.21.15.133:0/969821730 shutdown_connections 2023-07-14T18:25:25.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:25.674+0000 7f94fe7ff700 1 -- 172.21.15.133:0/969821730 wait complete. 2023-07-14T18:25:25.808 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:25:25.881 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 360.0 s, 251.2 tps, lat 436.480 ms stddev 1162.778 2023-07-14T18:25:26.167 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:25 smithi133 ceph-mon[126384]: pgmap v610: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 23 MiB/s wr, 1.48k op/s 2023-07-14T18:25:26.356 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:25 smithi161 ceph-mon[128993]: pgmap v610: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 23 MiB/s wr, 1.48k op/s 2023-07-14T18:25:26.375 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:25 smithi171 ceph-mon[129241]: pgmap v610: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 23 MiB/s wr, 1.48k op/s 2023-07-14T18:25:27.164 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:26 smithi133 ceph-mon[126384]: pgmap v611: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 20 MiB/s wr, 1.30k op/s 2023-07-14T18:25:27.356 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:26 smithi161 ceph-mon[128993]: pgmap v611: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 20 MiB/s wr, 1.30k op/s 2023-07-14T18:25:27.371 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:26 smithi171 ceph-mon[129241]: pgmap v611: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 20 MiB/s wr, 1.30k op/s 2023-07-14T18:25:28.356 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:28 smithi161 ceph-mon[128993]: pgmap v612: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 21 MiB/s wr, 1.43k op/s 2023-07-14T18:25:28.373 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:28 smithi171 ceph-mon[129241]: pgmap v612: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 21 MiB/s wr, 1.43k op/s 2023-07-14T18:25:28.413 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:28 smithi133 ceph-mon[126384]: pgmap v612: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 21 MiB/s wr, 1.43k op/s 2023-07-14T18:25:30.356 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:30 smithi161 ceph-mon[128993]: pgmap v613: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 22 MiB/s wr, 1.46k op/s 2023-07-14T18:25:30.371 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:30 smithi171 ceph-mon[129241]: pgmap v613: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 22 MiB/s wr, 1.46k op/s 2023-07-14T18:25:30.412 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:30 smithi133 ceph-mon[126384]: pgmap v613: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 22 MiB/s wr, 1.46k op/s 2023-07-14T18:25:30.880 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 365.0 s, 1252.8 tps, lat 80.840 ms stddev 62.625 2023-07-14T18:25:32.413 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:32 smithi133 ceph-mon[126384]: pgmap v614: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 18 MiB/s wr, 1.37k op/s 2023-07-14T18:25:32.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:32 smithi161 ceph-mon[128993]: pgmap v614: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 18 MiB/s wr, 1.37k op/s 2023-07-14T18:25:32.621 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:32 smithi171 ceph-mon[129241]: pgmap v614: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 18 MiB/s wr, 1.37k op/s 2023-07-14T18:25:34.356 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:34 smithi161 ceph-mon[128993]: pgmap v615: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 26 MiB/s wr, 2.06k op/s 2023-07-14T18:25:34.371 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:34 smithi171 ceph-mon[129241]: pgmap v615: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 26 MiB/s wr, 2.06k op/s 2023-07-14T18:25:34.412 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:34 smithi133 ceph-mon[126384]: pgmap v615: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 26 MiB/s wr, 2.06k op/s 2023-07-14T18:25:35.881 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 370.0 s, 512.2 tps, lat 179.260 ms stddev 341.842 2023-07-14T18:25:37.105 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:36 smithi161 ceph-mon[128993]: pgmap v616: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 17 MiB/s wr, 1.21k op/s 2023-07-14T18:25:37.123 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:36 smithi171 ceph-mon[129241]: pgmap v616: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 17 MiB/s wr, 1.21k op/s 2023-07-14T18:25:37.162 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:36 smithi133 ceph-mon[126384]: pgmap v616: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 17 MiB/s wr, 1.21k op/s 2023-07-14T18:25:38.355 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:38 smithi161 ceph-mon[128993]: pgmap v617: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 18 MiB/s wr, 1.28k op/s 2023-07-14T18:25:38.371 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:38 smithi171 ceph-mon[129241]: pgmap v617: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 18 MiB/s wr, 1.28k op/s 2023-07-14T18:25:38.416 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:38 smithi133 ceph-mon[126384]: pgmap v617: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 18 MiB/s wr, 1.28k op/s 2023-07-14T18:25:40.376 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:40 smithi171 ceph-mon[129241]: pgmap v618: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 20 MiB/s wr, 1.38k op/s 2023-07-14T18:25:40.412 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:40 smithi133 ceph-mon[126384]: pgmap v618: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 20 MiB/s wr, 1.38k op/s 2023-07-14T18:25:40.606 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:40 smithi161 ceph-mon[128993]: pgmap v618: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 20 MiB/s wr, 1.38k op/s 2023-07-14T18:25:40.880 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 375.0 s, 885.8 tps, lat 113.801 ms stddev 229.454 2023-07-14T18:25:42.413 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:42 smithi133 ceph-mon[126384]: pgmap v619: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 17 MiB/s wr, 1.27k op/s 2023-07-14T18:25:42.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:42 smithi161 ceph-mon[128993]: pgmap v619: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 17 MiB/s wr, 1.27k op/s 2023-07-14T18:25:42.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:42 smithi171 ceph-mon[129241]: pgmap v619: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 17 MiB/s wr, 1.27k op/s 2023-07-14T18:25:44.414 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:44 smithi133 ceph-mon[126384]: pgmap v620: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 24 MiB/s wr, 1.83k op/s 2023-07-14T18:25:44.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:44 smithi161 ceph-mon[128993]: pgmap v620: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 24 MiB/s wr, 1.83k op/s 2023-07-14T18:25:44.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:44 smithi171 ceph-mon[129241]: pgmap v620: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 24 MiB/s wr, 1.83k op/s 2023-07-14T18:25:45.879 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 380.0 s, 671.4 tps, lat 146.359 ms stddev 298.689 2023-07-14T18:25:47.105 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:46 smithi161 ceph-mon[128993]: pgmap v621: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 15 MiB/s wr, 1.09k op/s 2023-07-14T18:25:47.124 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:46 smithi171 ceph-mon[129241]: pgmap v621: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 15 MiB/s wr, 1.09k op/s 2023-07-14T18:25:47.161 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:46 smithi133 ceph-mon[126384]: pgmap v621: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 15 MiB/s wr, 1.09k op/s 2023-07-14T18:25:48.410 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:48 smithi133 ceph-mon[126384]: pgmap v622: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 17 MiB/s wr, 1.26k op/s 2023-07-14T18:25:48.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:48 smithi161 ceph-mon[128993]: pgmap v622: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 17 MiB/s wr, 1.26k op/s 2023-07-14T18:25:48.620 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:48 smithi171 ceph-mon[129241]: pgmap v622: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 17 MiB/s wr, 1.26k op/s 2023-07-14T18:25:50.412 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:50 smithi133 ceph-mon[126384]: pgmap v623: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 20 MiB/s wr, 1.44k op/s 2023-07-14T18:25:50.606 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:50 smithi161 ceph-mon[128993]: pgmap v623: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 20 MiB/s wr, 1.44k op/s 2023-07-14T18:25:50.623 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:50 smithi171 ceph-mon[129241]: pgmap v623: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 20 MiB/s wr, 1.44k op/s 2023-07-14T18:25:50.878 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 385.0 s, 283.6 tps, lat 357.905 ms stddev 653.773 2023-07-14T18:25:52.417 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:52 smithi133 ceph-mon[126384]: pgmap v624: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 17 MiB/s wr, 1.22k op/s 2023-07-14T18:25:52.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:52 smithi161 ceph-mon[128993]: pgmap v624: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 17 MiB/s wr, 1.22k op/s 2023-07-14T18:25:52.623 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:52 smithi171 ceph-mon[129241]: pgmap v624: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 17 MiB/s wr, 1.22k op/s 2023-07-14T18:25:54.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:54 smithi161 ceph-mon[128993]: pgmap v625: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 19 MiB/s wr, 1.44k op/s 2023-07-14T18:25:54.621 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:54 smithi171 ceph-mon[129241]: pgmap v625: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 19 MiB/s wr, 1.44k op/s 2023-07-14T18:25:54.660 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:54 smithi133 ceph-mon[126384]: pgmap v625: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 19 MiB/s wr, 1.44k op/s 2023-07-14T18:25:55.809 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:25:55.879 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 390.0 s, 322.7 tps, lat 316.215 ms stddev 469.231 2023-07-14T18:25:57.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:56.966+0000 7fea0bdff700 1 -- 172.21.15.133:0/3623224050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea04074620 msgr2=0x7fea04072a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:25:57.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:56.966+0000 7fea0bdff700 1 --2- 172.21.15.133:0/3623224050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea04074620 0x7fea04072a70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fea00009990 tx=0x7fea00009ca0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:56.968+0000 7fea0bdff700 1 -- 172.21.15.133:0/3623224050 shutdown_connections 2023-07-14T18:25:57.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:56.968+0000 7fea0bdff700 1 --2- 172.21.15.133:0/3623224050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea04074620 0x7fea04072a70 secure :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fea00009990 tx=0x7fea00009ca0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:56.968+0000 7fea0bdff700 1 --2- 172.21.15.133:0/3623224050 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea04107ac0 0x7fea04109f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:56.968+0000 7fea0bdff700 1 --2- 172.21.15.133:0/3623224050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea04073040 0x7fea040734b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:56.968+0000 7fea0bdff700 1 -- 172.21.15.133:0/3623224050 >> 172.21.15.133:0/3623224050 conn(0x7fea040fb250 msgr2=0x7fea040fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:25:57.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:56.985+0000 7fea0bdff700 1 -- 172.21.15.133:0/3623224050 shutdown_connections 2023-07-14T18:25:57.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:56.996+0000 7fea0bdff700 1 -- 172.21.15.133:0/3623224050 wait complete. 2023-07-14T18:25:57.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:56.997+0000 7fea0bdff700 1 Processor -- start 2023-07-14T18:25:57.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:56.997+0000 7fea0bdff700 1 -- start start 2023-07-14T18:25:57.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:56.997+0000 7fea0bdff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea04073040 0x7fea0406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:57.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:56.998+0000 7fea0bdff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea04107ac0 0x7fea0406f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:57.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:56.998+0000 7fea0bdff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea041a5e60 0x7fea041a8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:57.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:56.998+0000 7fea0bdff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0406fcd0 con 0x7fea041a5e60 2023-07-14T18:25:57.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:56.998+0000 7fea0bdff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea041a8780 con 0x7fea04073040 2023-07-14T18:25:57.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:56.998+0000 7fea0bdff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea041a88c0 con 0x7fea04107ac0 2023-07-14T18:25:57.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:56.999+0000 7fea0a39c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea041a5e60 0x7fea041a8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:25:57.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:56.999+0000 7fea0a39c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea041a5e60 0x7fea041a8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35412/0 (socket says 172.21.15.133:35412) 2023-07-14T18:25:57.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:56.999+0000 7fea0a39c700 1 -- 172.21.15.133:0/538211887 learned_addr learned my addr 172.21.15.133:0/538211887 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:25:57.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:56.999+0000 7fea0939a700 1 --2- 172.21.15.133:0/538211887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea04107ac0 0x7fea0406f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:25:57.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.000+0000 7fea0a39c700 1 -- 172.21.15.133:0/538211887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea04107ac0 msgr2=0x7fea0406f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:25:57.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.000+0000 7fea0a39c700 1 --2- 172.21.15.133:0/538211887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea04107ac0 0x7fea0406f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.000+0000 7fea0a39c700 1 -- 172.21.15.133:0/538211887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea04073040 msgr2=0x7fea0406f0e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T18:25:57.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.000+0000 7fea0a39c700 1 --2- 172.21.15.133:0/538211887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea04073040 0x7fea0406f0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.000+0000 7fea0a39c700 1 -- 172.21.15.133:0/538211887 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea000095f0 con 0x7fea041a5e60 2023-07-14T18:25:57.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.000+0000 7fea0a39c700 1 --2- 172.21.15.133:0/538211887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea041a5e60 0x7fea041a8240 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc00b7b0 tx=0x7fe9fc00bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:25:57.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.001+0000 7fe9faffd700 1 -- 172.21.15.133:0/538211887 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc032870 con 0x7fea041a5e60 2023-07-14T18:25:57.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.001+0000 7fe9faffd700 1 -- 172.21.15.133:0/538211887 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9fc032e90 con 0x7fea041a5e60 2023-07-14T18:25:57.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.001+0000 7fe9faffd700 1 -- 172.21.15.133:0/538211887 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc007370 con 0x7fea041a5e60 2023-07-14T18:25:57.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.001+0000 7fea0bdff700 1 -- 172.21.15.133:0/538211887 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea0410e190 con 0x7fea041a5e60 2023-07-14T18:25:57.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.001+0000 7fea0bdff700 1 -- 172.21.15.133:0/538211887 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea0410e6d0 con 0x7fea041a5e60 2023-07-14T18:25:57.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.013+0000 7fea1f6d7b80 1 Processor -- start 2023-07-14T18:25:57.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.014+0000 7fe9faffd700 1 -- 172.21.15.133:0/538211887 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe9fc031650 con 0x7fea041a5e60 2023-07-14T18:25:57.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.014+0000 7fea1f6d7b80 1 -- start start 2023-07-14T18:25:57.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.015+0000 7fea1f6d7b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652a13fd770 0x5652a13fdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:57.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.015+0000 7fea1f6d7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652a13fe0c0 0x5652a13fe6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:57.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.015+0000 7fea1f6d7b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5652a13fecc0 0x5652a13ff130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:57.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.015+0000 7fea1f6d7b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652a12f9cb0 con 0x5652a13fd770 2023-07-14T18:25:57.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.015+0000 7fea1f6d7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652a134ca40 con 0x5652a13fe0c0 2023-07-14T18:25:57.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.015+0000 7fea1f6d7b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652a1345ed0 con 0x5652a13fecc0 2023-07-14T18:25:57.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.016+0000 7fe9faffd700 1 --2- 172.21.15.133:0/538211887 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe9f00a12d0 0x7fe9f00a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:57.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.016+0000 7fe9faffd700 1 -- 172.21.15.133:0/538211887 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe9fc0ee9c0 con 0x7fea041a5e60 2023-07-14T18:25:57.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.023+0000 7fea0a39c700 1 --2- 172.21.15.133:0/538211887 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe9f00a12d0 0x7fe9f00a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:25:57.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.023+0000 7fea0939a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5652a13fecc0 0x5652a13ff130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:25:57.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.023+0000 7fea0939a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5652a13fecc0 0x5652a13ff130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:57658/0 (socket says 172.21.15.133:57658) 2023-07-14T18:25:57.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.023+0000 7fea0939a700 1 -- 172.21.15.133:0/1510835282 learned_addr learned my addr 172.21.15.133:0/1510835282 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:25:57.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.024+0000 7fea0939a700 1 --2- 172.21.15.133:0/1510835282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652a13fd770 0x5652a13fdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:25:57.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.027+0000 7fea0939a700 1 -- 172.21.15.133:0/1510835282 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5652a13fecc0 msgr2=0x5652a13ff130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:25:57.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.027+0000 7fea0939a700 1 --2- 172.21.15.133:0/1510835282 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5652a13fecc0 0x5652a13ff130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.027+0000 7fea0939a700 1 -- 172.21.15.133:0/1510835282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652a13fe0c0 msgr2=0x5652a13fe6e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T18:25:57.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.027+0000 7fea0939a700 1 --2- 172.21.15.133:0/1510835282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652a13fe0c0 0x5652a13fe6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.027+0000 7fea0939a700 1 -- 172.21.15.133:0/1510835282 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652a1173de0 con 0x5652a13fd770 2023-07-14T18:25:57.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.028+0000 7fea0a39c700 1 --2- 172.21.15.133:0/538211887 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe9f00a12d0 0x7fe9f00a3780 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc0bb760 tx=0x7fe9fc009470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:25:57.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.028+0000 7fea0939a700 1 --2- 172.21.15.133:0/1510835282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652a13fd770 0x5652a13fdb80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400bf80 tx=0x7fe9f400b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db851f4b13821ffd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:25:57.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.028+0000 7fe9effff700 1 -- 172.21.15.133:0/1510835282 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f401e640 con 0x5652a13fd770 2023-07-14T18:25:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.029+0000 7fe9effff700 1 -- 172.21.15.133:0/1510835282 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9f400f8b0 con 0x5652a13fd770 2023-07-14T18:25:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.029+0000 7fe9effff700 1 -- 172.21.15.133:0/1510835282 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f400e730 con 0x5652a13fd770 2023-07-14T18:25:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.030+0000 7fea1f6d7b80 1 -- 172.21.15.133:0/1510835282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652a13fd770 msgr2=0x5652a13fdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:25:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.030+0000 7fea1f6d7b80 1 --2- 172.21.15.133:0/1510835282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652a13fd770 0x5652a13fdb80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400bf80 tx=0x7fe9f400b410 comp rx=0 tx=0).stop 2023-07-14T18:25:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.031+0000 7fea1f6d7b80 1 -- 172.21.15.133:0/1510835282 shutdown_connections 2023-07-14T18:25:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.031+0000 7fea1f6d7b80 1 --2- 172.21.15.133:0/1510835282 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5652a13fecc0 0x5652a13ff130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.031+0000 7fea1f6d7b80 1 --2- 172.21.15.133:0/1510835282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652a13fd770 0x5652a13fdb80 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.031+0000 7fea1f6d7b80 1 --2- 172.21.15.133:0/1510835282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652a13fe0c0 0x5652a13fe6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.031+0000 7fea1f6d7b80 1 -- 172.21.15.133:0/1510835282 >> 172.21.15.133:0/1510835282 conn(0x5652a13ea680 msgr2=0x5652a13ef9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:25:57.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.046+0000 7fea1f6d7b80 1 -- 172.21.15.133:0/1510835282 shutdown_connections 2023-07-14T18:25:57.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.057+0000 7fea1f6d7b80 1 -- 172.21.15.133:0/1510835282 wait complete. 2023-07-14T18:25:57.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.057+0000 7fea1f6d7b80 1 -- start start 2023-07-14T18:25:57.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.058+0000 7fea1f6d7b80 1 Processor -- start 2023-07-14T18:25:57.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:56 smithi161 ceph-mon[128993]: pgmap v626: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 12 MiB/s wr, 872 op/s 2023-07-14T18:25:57.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.058+0000 7fea1f6d7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652a13fd770 0x5652a1407270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:57.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.058+0000 7fea1f6d7b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652a13fe0c0 0x5652a14077b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:57.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.058+0000 7fea1f6d7b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5652a13fecc0 0x5652a140e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:57.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.058+0000 7fea1f6d7b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652a12e5ac0 con 0x5652a13fe0c0 2023-07-14T18:25:57.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.058+0000 7fea1f6d7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652a12e57c0 con 0x5652a13fd770 2023-07-14T18:25:57.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.059+0000 7fea1f6d7b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652a11c9920 con 0x5652a13fecc0 2023-07-14T18:25:57.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.059+0000 7fea1f6d7b80 10 client.0 _ll_register_callbacks cb 0x5652a12e3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:25:57.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.059+0000 7fea0939a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652a13fd770 0x5652a1407270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:25:57.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.059+0000 7fea0939a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652a13fd770 0x5652a1407270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:51142/0 (socket says 172.21.15.133:51142) 2023-07-14T18:25:57.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.059+0000 7fea0939a700 1 -- 172.21.15.133:0/3281404935 learned_addr learned my addr 172.21.15.133:0/3281404935 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:25:57.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.060+0000 7fea0939a700 1 --2- 172.21.15.133:0/3281404935 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5652a13fecc0 0x5652a140e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:25:57.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.060+0000 7fea0939a700 1 -- 172.21.15.133:0/3281404935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652a13fd770 msgr2=0x5652a1407270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:25:57.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.060+0000 7fea0939a700 1 --2- 172.21.15.133:0/3281404935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652a13fd770 0x5652a1407270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.060+0000 7fea0939a700 1 -- 172.21.15.133:0/3281404935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652a13fe0c0 msgr2=0x5652a14077b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T18:25:57.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.060+0000 7fea0939a700 1 --2- 172.21.15.133:0/3281404935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652a13fe0c0 0x5652a14077b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.060+0000 7fea0939a700 1 -- 172.21.15.133:0/3281404935 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f40123f0 con 0x5652a13fecc0 2023-07-14T18:25:57.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.061+0000 7fea0939a700 1 --2- 172.21.15.133:0/3281404935 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5652a13fecc0 0x5652a140e7f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4012fd0 tx=0x7fe9f400bcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f3d1a592c278732 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:25:57.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.061+0000 7fe9ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:25:57.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.061+0000 7fe9ed7fa700 1 -- 172.21.15.133:0/3281404935 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f400e8e0 con 0x5652a13fecc0 2023-07-14T18:25:57.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.061+0000 7fe9ed7fa700 1 -- 172.21.15.133:0/3281404935 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe9f400ea40 con 0x5652a13fecc0 2023-07-14T18:25:57.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.062+0000 7fea1f6d7b80 1 -- 172.21.15.133:0/3281404935 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5652a129f4f0 con 0x5652a13fecc0 2023-07-14T18:25:57.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.062+0000 7fe9ed7fa700 1 -- 172.21.15.133:0/3281404935 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9f400f700 con 0x5652a13fecc0 2023-07-14T18:25:57.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.062+0000 7fea1f6d7b80 10 client.25897 fetch_fsmap learned FSMap version 35 2023-07-14T18:25:57.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.062+0000 7fea1f6d7b80 1 -- 172.21.15.133:0/3281404935 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5652a115cc60 con 0x5652a13fecc0 2023-07-14T18:25:57.120 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:56 smithi171 ceph-mon[129241]: pgmap v626: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 12 MiB/s wr, 872 op/s 2023-07-14T18:25:57.122 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.075+0000 7fe9ed7fa700 1 -- 172.21.15.133:0/3281404935 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5652a129f4f0 con 0x5652a13fecc0 2023-07-14T18:25:57.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.075+0000 7fe9ed7fa700 1 -- 172.21.15.133:0/3281404935 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 35) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fe9f401ebe0 con 0x5652a13fecc0 2023-07-14T18:25:57.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.075+0000 7fea1f6d7b80 10 client.25897 fetch_fsmap finished waiting for FSMap version 35 2023-07-14T18:25:57.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.075+0000 7fea1f6d7b80 10 client.25897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:25:57.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.075+0000 7fea1f6d7b80 20 client.25897 populate_metadata read hostname 'smithi133' 2023-07-14T18:25:57.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.075+0000 7fea1f6d7b80 1 --2- 172.21.15.133:0/3281404935 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5652a13f5d00 0x5652a13f8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:57.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.075+0000 7fea1f6d7b80 4 client.25897 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:25:57.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.075+0000 7fea1f6d7b80 1 -- 172.21.15.133:0/3281404935 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652a13c3da0 con 0x5652a13f5d00 2023-07-14T18:25:57.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.086+0000 7fea09b9b700 1 --2- 172.21.15.133:0/3281404935 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5652a13f5d00 0x5652a13f8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:25:57.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.087+0000 7fea09b9b700 1 --2- 172.21.15.133:0/3281404935 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5652a13f5d00 0x5652a13f8110 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a53fc6ec673cfae9 server_cookie=108915676d947908 in_seq=0 out_seq=0 2023-07-14T18:25:57.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.087+0000 7fe9ed7fa700 10 client.25897 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:25:57.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.094+0000 7fe9ed7fa700 1 -- 172.21.15.133:0/3281404935 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5652a13c3da0 con 0x5652a13f5d00 2023-07-14T18:25:57.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.094+0000 7fe9ed7fa700 10 client.25897 handle_command_reply: tid=0 2023-07-14T18:25:57.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.094+0000 7fea1f6d7b80 1 client.25897 shutdown 2023-07-14T18:25:57.143 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.096+0000 7fea1f6d7b80 1 -- 172.21.15.133:0/3281404935 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5652a13fecc0 msgr2=0x5652a140e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:25:57.143 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.096+0000 7fea1f6d7b80 1 --2- 172.21.15.133:0/3281404935 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5652a13fecc0 0x5652a140e7f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4012fd0 tx=0x7fe9f400bcd0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.096+0000 7fea1f6d7b80 1 -- 172.21.15.133:0/3281404935 shutdown_connections 2023-07-14T18:25:57.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.096+0000 7fea1f6d7b80 1 --2- 172.21.15.133:0/3281404935 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5652a13f5d00 0x5652a13f8110 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.096+0000 7fea1f6d7b80 1 --2- 172.21.15.133:0/3281404935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652a13fd770 0x5652a1407270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.096+0000 7fea1f6d7b80 1 --2- 172.21.15.133:0/3281404935 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5652a13fecc0 0x5652a140e7f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.096+0000 7fea1f6d7b80 1 --2- 172.21.15.133:0/3281404935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652a13fe0c0 0x5652a14077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.096+0000 7fea1f6d7b80 1 -- 172.21.15.133:0/3281404935 >> 172.21.15.133:0/3281404935 conn(0x5652a13ea680 msgr2=0x5652a13ee650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:25:57.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.104+0000 7fe9ed7fa700 0 client.25897 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:25:57.155 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.107+0000 7fea1f6d7b80 1 -- 172.21.15.133:0/3281404935 shutdown_connections 2023-07-14T18:25:57.155 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.107+0000 7fea1f6d7b80 1 -- 172.21.15.133:0/3281404935 wait complete. 2023-07-14T18:25:57.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.111+0000 7fea1f6d7b80 20 client.25897 trim_cache size 0 max 16384 2023-07-14T18:25:57.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:56 smithi133 ceph-mon[126384]: pgmap v626: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 12 MiB/s wr, 872 op/s 2023-07-14T18:25:57.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.143+0000 7fea1f6d7b80 1 Processor -- start 2023-07-14T18:25:57.204 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.156+0000 7fea1f6d7b80 1 -- start start 2023-07-14T18:25:57.204 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.156+0000 7fea1f6d7b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5652a13fd770 0x5652a13f2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:57.205 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.156+0000 7fea1f6d7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652a13fe0c0 0x5652a1405f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:57.205 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.156+0000 7fea1f6d7b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652a13fecc0 0x5652a13fca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:57.205 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.156+0000 7fea1f6d7b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652a13f08e0 con 0x5652a13fecc0 2023-07-14T18:25:57.205 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.156+0000 7fea1f6d7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652a14082d0 con 0x5652a13fe0c0 2023-07-14T18:25:57.206 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.156+0000 7fea1f6d7b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652a13f1550 con 0x5652a13fd770 2023-07-14T18:25:57.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.159+0000 7fea0939a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5652a13fd770 0x5652a13f2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:25:57.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.170+0000 7fea09b9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652a13fe0c0 0x5652a1405f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:25:57.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.170+0000 7fea09b9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652a13fe0c0 0x5652a1405f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:51158/0 (socket says 172.21.15.133:51158) 2023-07-14T18:25:57.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.170+0000 7fea09b9b700 1 -- 172.21.15.133:0/3402843654 learned_addr learned my addr 172.21.15.133:0/3402843654 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:25:57.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.172+0000 7fea09b9b700 1 -- 172.21.15.133:0/3402843654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5652a13fd770 msgr2=0x5652a13f2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:25:57.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.172+0000 7fea09b9b700 1 --2- 172.21.15.133:0/3402843654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5652a13fd770 0x5652a13f2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.172+0000 7fea09b9b700 1 -- 172.21.15.133:0/3402843654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652a13fecc0 msgr2=0x5652a13fca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:25:57.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.172+0000 7fea09b9b700 1 --2- 172.21.15.133:0/3402843654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652a13fecc0 0x5652a13fca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.172+0000 7fea09b9b700 1 -- 172.21.15.133:0/3402843654 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652a119a250 con 0x5652a13fe0c0 2023-07-14T18:25:57.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.173+0000 7fea09b9b700 1 --2- 172.21.15.133:0/3402843654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652a13fe0c0 0x5652a1405f10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fea00000c00 tx=0x7fea000036a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f70a986d5513e7b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:25:57.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.179+0000 7fe9f8bff700 1 -- 172.21.15.133:0/3402843654 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0001d070 con 0x5652a13fe0c0 2023-07-14T18:25:57.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.179+0000 7fe9f8bff700 1 -- 172.21.15.133:0/3402843654 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea0001e040 con 0x5652a13fe0c0 2023-07-14T18:25:57.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.179+0000 7fe9f8bff700 1 -- 172.21.15.133:0/3402843654 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea00021670 con 0x5652a13fe0c0 2023-07-14T18:25:57.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.180+0000 7fea1f6d7b80 1 -- 172.21.15.133:0/3402843654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652a13fe0c0 msgr2=0x5652a1405f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:25:57.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.180+0000 7fea1f6d7b80 1 --2- 172.21.15.133:0/3402843654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652a13fe0c0 0x5652a1405f10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fea00000c00 tx=0x7fea000036a0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.181+0000 7fea1f6d7b80 1 -- 172.21.15.133:0/3402843654 shutdown_connections 2023-07-14T18:25:57.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.181+0000 7fea1f6d7b80 1 --2- 172.21.15.133:0/3402843654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652a13fecc0 0x5652a13fca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.181+0000 7fea1f6d7b80 1 --2- 172.21.15.133:0/3402843654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5652a13fd770 0x5652a13f2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.181+0000 7fea1f6d7b80 1 --2- 172.21.15.133:0/3402843654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652a13fe0c0 0x5652a1405f10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.181+0000 7fea1f6d7b80 1 -- 172.21.15.133:0/3402843654 >> 172.21.15.133:0/3402843654 conn(0x5652a13ea680 msgr2=0x5652a1405100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:25:57.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.196+0000 7fea1f6d7b80 1 -- 172.21.15.133:0/3402843654 shutdown_connections 2023-07-14T18:25:57.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.196+0000 7fea1f6d7b80 1 -- 172.21.15.133:0/3402843654 wait complete. 2023-07-14T18:25:57.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.197+0000 7fea1f6d7b80 1 -- start start 2023-07-14T18:25:57.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.197+0000 7fea1f6d7b80 1 Processor -- start 2023-07-14T18:25:57.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.198+0000 7fea1f6d7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652a13fd770 0x5652a1467190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:57.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.198+0000 7fea1f6d7b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652a13fe0c0 0x5652a14676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:57.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.198+0000 7fea1f6d7b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5652a13fecc0 0x5652a146e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:57.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.198+0000 7fea1f6d7b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652a146ec50 con 0x5652a13fe0c0 2023-07-14T18:25:57.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.198+0000 7fea1f6d7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652a146ed90 con 0x5652a13fd770 2023-07-14T18:25:57.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.198+0000 7fea1f6d7b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652a146ef20 con 0x5652a13fecc0 2023-07-14T18:25:57.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.198+0000 7fea1f6d7b80 10 client.0 _ll_register_callbacks cb 0x7fe9f400fc40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:25:57.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.198+0000 7fea0939a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652a13fd770 0x5652a1467190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:25:57.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.198+0000 7fea0939a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652a13fd770 0x5652a1467190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:51174/0 (socket says 172.21.15.133:51174) 2023-07-14T18:25:57.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.198+0000 7fea0939a700 1 -- 172.21.15.133:0/1106279617 learned_addr learned my addr 172.21.15.133:0/1106279617 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:25:57.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.199+0000 7fea0939a700 1 --2- 172.21.15.133:0/1106279617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5652a13fecc0 0x5652a146e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:25:57.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.199+0000 7fea0939a700 1 -- 172.21.15.133:0/1106279617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5652a13fecc0 msgr2=0x5652a146e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:25:57.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.199+0000 7fea0939a700 1 --2- 172.21.15.133:0/1106279617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5652a13fecc0 0x5652a146e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.199+0000 7fea0939a700 1 -- 172.21.15.133:0/1106279617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652a13fe0c0 msgr2=0x5652a14676d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:25:57.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.199+0000 7fea0939a700 1 --2- 172.21.15.133:0/1106279617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652a13fe0c0 0x5652a14676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.199+0000 7fea0939a700 1 -- 172.21.15.133:0/1106279617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea0410ab60 con 0x5652a13fd770 2023-07-14T18:25:57.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.199+0000 7fea0939a700 1 --2- 172.21.15.133:0/1106279617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652a13fd770 0x5652a1467190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4012e00 tx=0x7fe9f4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6eef924ca91f29a6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:25:57.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.200+0000 7fe9ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:25:57.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.200+0000 7fe9ed7fa700 1 -- 172.21.15.133:0/1106279617 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f40052c0 con 0x5652a13fd770 2023-07-14T18:25:57.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.200+0000 7fe9ed7fa700 1 -- 172.21.15.133:0/1106279617 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe9f4000e40 con 0x5652a13fd770 2023-07-14T18:25:57.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.200+0000 7fea1f6d7b80 1 -- 172.21.15.133:0/1106279617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9f400f8b0 con 0x5652a13fd770 2023-07-14T18:25:57.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.201+0000 7fe9ed7fa700 1 -- 172.21.15.133:0/1106279617 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9f400faa0 con 0x5652a13fd770 2023-07-14T18:25:57.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.207+0000 7fea1f6d7b80 10 client.25862 fetch_fsmap learned FSMap version 35 2023-07-14T18:25:57.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.207+0000 7fea1f6d7b80 1 -- 172.21.15.133:0/1106279617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5652a11722b0 con 0x5652a13fd770 2023-07-14T18:25:57.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.208+0000 7fe9ed7fa700 1 -- 172.21.15.133:0/1106279617 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9f400f8b0 con 0x5652a13fd770 2023-07-14T18:25:57.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.208+0000 7fe9ed7fa700 1 -- 172.21.15.133:0/1106279617 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 35) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fe9f4005420 con 0x5652a13fd770 2023-07-14T18:25:57.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.208+0000 7fea1f6d7b80 10 client.25862 fetch_fsmap finished waiting for FSMap version 35 2023-07-14T18:25:57.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.208+0000 7fea1f6d7b80 10 client.25862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:25:57.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.208+0000 7fea1f6d7b80 20 client.25862 populate_metadata read hostname 'smithi133' 2023-07-14T18:25:57.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.208+0000 7fea1f6d7b80 1 --2- 172.21.15.133:0/1106279617 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5652a1470470 0x5652a1472880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:25:57.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.208+0000 7fea1f6d7b80 4 client.25862 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:25:57.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.208+0000 7fea1f6d7b80 1 -- 172.21.15.133:0/1106279617 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5652a133eeb0 con 0x5652a1470470 2023-07-14T18:25:57.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.208+0000 7fea09b9b700 1 --2- 172.21.15.133:0/1106279617 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5652a1470470 0x5652a1472880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:25:57.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.210+0000 7fea09b9b700 1 --2- 172.21.15.133:0/1106279617 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5652a1470470 0x5652a1472880 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43fc349c35d4556d server_cookie=386b8f8559898a43 in_seq=0 out_seq=0 2023-07-14T18:25:57.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.210+0000 7fe9ed7fa700 10 client.25862 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:25:57.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.210+0000 7fe9ed7fa700 1 -- 172.21.15.133:0/1106279617 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5652a133eeb0 con 0x5652a1470470 2023-07-14T18:25:57.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.210+0000 7fe9ed7fa700 10 client.25862 handle_command_reply: tid=0 2023-07-14T18:25:57.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.210+0000 7fea1f6d7b80 1 client.25862 shutdown 2023-07-14T18:25:57.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.211+0000 7fea1f6d7b80 1 -- 172.21.15.133:0/1106279617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652a13fd770 msgr2=0x5652a1467190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:25:57.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.211+0000 7fea1f6d7b80 1 --2- 172.21.15.133:0/1106279617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652a13fd770 0x5652a1467190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4012e00 tx=0x7fe9f4000bc0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.211+0000 7fea1f6d7b80 1 -- 172.21.15.133:0/1106279617 shutdown_connections 2023-07-14T18:25:57.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.211+0000 7fea1f6d7b80 1 --2- 172.21.15.133:0/1106279617 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5652a1470470 0x5652a1472880 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.211+0000 7fea1f6d7b80 1 --2- 172.21.15.133:0/1106279617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652a13fd770 0x5652a1467190 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.211+0000 7fea1f6d7b80 1 --2- 172.21.15.133:0/1106279617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5652a13fecc0 0x5652a146e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.211+0000 7fea1f6d7b80 1 --2- 172.21.15.133:0/1106279617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652a13fe0c0 0x5652a14676d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.211+0000 7fea1f6d7b80 1 -- 172.21.15.133:0/1106279617 >> 172.21.15.133:0/1106279617 conn(0x5652a13ea680 msgr2=0x5652a13f2020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:25:57.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.211+0000 7fe9ed7fa700 0 client.25862 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:25:57.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.212+0000 7fea1f6d7b80 1 -- 172.21.15.133:0/1106279617 shutdown_connections 2023-07-14T18:25:57.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.212+0000 7fea1f6d7b80 1 -- 172.21.15.133:0/1106279617 wait complete. 2023-07-14T18:25:57.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.217+0000 7fea1f6d7b80 20 client.25862 trim_cache size 0 max 16384 2023-07-14T18:25:57.272 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:25:57.272 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T18:25:57.273 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:25:57.273 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:25:57.273 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:25:57.273 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:25:57.273 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:25:57.273 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:25:57.274 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:25:57.274 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:25:57.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.230+0000 7fea0bdff700 1 -- 172.21.15.133:0/538211887 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe9f00a12d0 msgr2=0x7fe9f00a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:25:57.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.230+0000 7fea0bdff700 1 --2- 172.21.15.133:0/538211887 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe9f00a12d0 0x7fe9f00a3780 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc0bb760 tx=0x7fe9fc009470 comp rx=0 tx=0).stop 2023-07-14T18:25:57.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.230+0000 7fea0bdff700 1 -- 172.21.15.133:0/538211887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea041a5e60 msgr2=0x7fea041a8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:25:57.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.230+0000 7fea0bdff700 1 --2- 172.21.15.133:0/538211887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea041a5e60 0x7fea041a8240 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc00b7b0 tx=0x7fe9fc00bac0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.238+0000 7fea0bdff700 1 -- 172.21.15.133:0/538211887 shutdown_connections 2023-07-14T18:25:57.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.238+0000 7fea0bdff700 1 --2- 172.21.15.133:0/538211887 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe9f00a12d0 0x7fe9f00a3780 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.238+0000 7fea0bdff700 1 --2- 172.21.15.133:0/538211887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea04073040 0x7fea0406f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.238+0000 7fea0bdff700 1 --2- 172.21.15.133:0/538211887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea041a5e60 0x7fea041a8240 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.238+0000 7fea0bdff700 1 --2- 172.21.15.133:0/538211887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea04107ac0 0x7fea0406f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:25:57.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.238+0000 7fea0bdff700 1 -- 172.21.15.133:0/538211887 >> 172.21.15.133:0/538211887 conn(0x7fea040fb250 msgr2=0x7fea040fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:25:57.302 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.249+0000 7fea0bdff700 1 -- 172.21.15.133:0/538211887 shutdown_connections 2023-07-14T18:25:57.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:25:57.254+0000 7fea0bdff700 1 -- 172.21.15.133:0/538211887 wait complete. 2023-07-14T18:25:57.366 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:25:58.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:57 smithi161 ceph-mon[128993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:25:58.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:57 smithi161 ceph-mon[128993]: scrub summary: active paths [/] 2023-07-14T18:25:58.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:57 smithi161 ceph-mon[128993]: scrub summary: active paths [/] 2023-07-14T18:25:58.104 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:57 smithi161 ceph-mon[128993]: scrub summary: idle+waiting paths [/] 2023-07-14T18:25:58.119 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:57 smithi171 ceph-mon[129241]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:25:58.119 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:57 smithi171 ceph-mon[129241]: scrub summary: active paths [/] 2023-07-14T18:25:58.120 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:57 smithi171 ceph-mon[129241]: scrub summary: active paths [/] 2023-07-14T18:25:58.120 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:57 smithi171 ceph-mon[129241]: scrub summary: idle+waiting paths [/] 2023-07-14T18:25:58.165 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:57 smithi133 ceph-mon[126384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:25:58.165 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:57 smithi133 ceph-mon[126384]: scrub summary: active paths [/] 2023-07-14T18:25:58.165 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:57 smithi133 ceph-mon[126384]: scrub summary: active paths [/] 2023-07-14T18:25:58.165 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:57 smithi133 ceph-mon[126384]: scrub summary: idle+waiting paths [/] 2023-07-14T18:25:59.106 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:58 smithi161 ceph-mon[128993]: scrub summary: idle+waiting paths [/] 2023-07-14T18:25:59.106 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:58 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:25:59.106 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:58 smithi161 ceph-mon[128993]: pgmap v627: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 14 MiB/s wr, 1.01k op/s 2023-07-14T18:25:59.119 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:58 smithi171 ceph-mon[129241]: scrub summary: idle+waiting paths [/] 2023-07-14T18:25:59.119 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:58 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:25:59.119 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:58 smithi171 ceph-mon[129241]: pgmap v627: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 14 MiB/s wr, 1.01k op/s 2023-07-14T18:25:59.163 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:58 smithi133 ceph-mon[126384]: scrub summary: idle+waiting paths [/] 2023-07-14T18:25:59.163 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:58 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:25:59.163 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:58 smithi133 ceph-mon[126384]: pgmap v627: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 14 MiB/s wr, 1.01k op/s 2023-07-14T18:26:00.159 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:59 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:26:00.160 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:59 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:26:00.160 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:25:59 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:26:00.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:59 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:26:00.354 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:59 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:26:00.354 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:25:59 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:26:00.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:59 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:26:00.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:59 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:26:00.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:25:59 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:26:00.882 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 395.0 s, 505.9 tps, lat 207.574 ms stddev 216.273 2023-07-14T18:26:01.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:00 smithi161 ceph-mon[128993]: pgmap v628: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 15 MiB/s wr, 967 op/s 2023-07-14T18:26:01.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:00 smithi171 ceph-mon[129241]: pgmap v628: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 15 MiB/s wr, 967 op/s 2023-07-14T18:26:01.416 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:00 smithi133 ceph-mon[126384]: pgmap v628: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 15 MiB/s wr, 967 op/s 2023-07-14T18:26:03.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:02 smithi161 ceph-mon[128993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:26:03.353 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:02 smithi161 ceph-mon[128993]: pgmap v629: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 717 op/s 2023-07-14T18:26:03.368 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:02 smithi171 ceph-mon[129241]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:26:03.369 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:02 smithi171 ceph-mon[129241]: pgmap v629: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 717 op/s 2023-07-14T18:26:03.425 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:02 smithi133 ceph-mon[126384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:26:03.425 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:02 smithi133 ceph-mon[126384]: pgmap v629: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 717 op/s 2023-07-14T18:26:04.603 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:04 smithi161 ceph-mon[128993]: pgmap v630: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 13 MiB/s wr, 881 op/s 2023-07-14T18:26:04.618 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:04 smithi171 ceph-mon[129241]: pgmap v630: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 13 MiB/s wr, 881 op/s 2023-07-14T18:26:04.659 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:04 smithi133 ceph-mon[126384]: pgmap v630: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 13 MiB/s wr, 881 op/s 2023-07-14T18:26:05.880 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 400.0 s, 958.3 tps, lat 105.310 ms stddev 198.592 2023-07-14T18:26:07.106 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:06 smithi161 ceph-mon[128993]: pgmap v631: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 11 MiB/s wr, 660 op/s 2023-07-14T18:26:07.120 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:06 smithi171 ceph-mon[129241]: pgmap v631: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 11 MiB/s wr, 660 op/s 2023-07-14T18:26:07.161 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:06 smithi133 ceph-mon[126384]: pgmap v631: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 11 MiB/s wr, 660 op/s 2023-07-14T18:26:08.606 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:08 smithi161 ceph-mon[128993]: pgmap v632: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 16 MiB/s wr, 941 op/s 2023-07-14T18:26:08.618 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:08 smithi171 ceph-mon[129241]: pgmap v632: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 16 MiB/s wr, 941 op/s 2023-07-14T18:26:08.663 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:08 smithi133 ceph-mon[126384]: pgmap v632: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 16 MiB/s wr, 941 op/s 2023-07-14T18:26:10.602 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:10 smithi161 ceph-mon[128993]: pgmap v633: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 17 MiB/s wr, 966 op/s 2023-07-14T18:26:10.623 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:10 smithi171 ceph-mon[129241]: pgmap v633: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 17 MiB/s wr, 966 op/s 2023-07-14T18:26:10.659 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:10 smithi133 ceph-mon[126384]: pgmap v633: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 17 MiB/s wr, 966 op/s 2023-07-14T18:26:10.883 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 405.0 s, 626.1 tps, lat 157.557 ms stddev 361.211 2023-07-14T18:26:12.602 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:12 smithi161 ceph-mon[128993]: pgmap v634: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 16 MiB/s wr, 977 op/s 2023-07-14T18:26:12.617 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:12 smithi171 ceph-mon[129241]: pgmap v634: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 16 MiB/s wr, 977 op/s 2023-07-14T18:26:12.659 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:12 smithi133 ceph-mon[126384]: pgmap v634: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 16 MiB/s wr, 977 op/s 2023-07-14T18:26:14.606 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:14 smithi161 ceph-mon[128993]: pgmap v635: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 19 MiB/s wr, 1.15k op/s 2023-07-14T18:26:14.619 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:14 smithi171 ceph-mon[129241]: pgmap v635: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 19 MiB/s wr, 1.15k op/s 2023-07-14T18:26:14.661 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:14 smithi133 ceph-mon[126384]: pgmap v635: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 19 MiB/s wr, 1.15k op/s 2023-07-14T18:26:15.880 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 410.0 s, 590.2 tps, lat 169.730 ms stddev 177.577 2023-07-14T18:26:17.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:16 smithi161 ceph-mon[128993]: pgmap v636: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 17 MiB/s wr, 1.02k op/s 2023-07-14T18:26:17.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:16 smithi171 ceph-mon[129241]: pgmap v636: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 17 MiB/s wr, 1.02k op/s 2023-07-14T18:26:17.164 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:16 smithi133 ceph-mon[126384]: pgmap v636: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 17 MiB/s wr, 1.02k op/s 2023-07-14T18:26:19.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:18 smithi161 ceph-mon[128993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:26:19.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:18 smithi161 ceph-mon[128993]: pgmap v637: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 20 MiB/s wr, 1.33k op/s 2023-07-14T18:26:19.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:18 smithi171 ceph-mon[129241]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:26:19.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:18 smithi171 ceph-mon[129241]: pgmap v637: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 20 MiB/s wr, 1.33k op/s 2023-07-14T18:26:19.165 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:18 smithi133 ceph-mon[126384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:26:19.166 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:18 smithi133 ceph-mon[126384]: pgmap v637: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 20 MiB/s wr, 1.33k op/s 2023-07-14T18:26:20.602 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:20 smithi161 ceph-mon[128993]: pgmap v638: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 17 MiB/s wr, 1.13k op/s 2023-07-14T18:26:20.617 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:20 smithi171 ceph-mon[129241]: pgmap v638: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 17 MiB/s wr, 1.13k op/s 2023-07-14T18:26:20.658 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:20 smithi133 ceph-mon[126384]: pgmap v638: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 17 MiB/s wr, 1.13k op/s 2023-07-14T18:26:20.877 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 415.0 s, 1294.1 tps, lat 77.983 ms stddev 61.360 2023-07-14T18:26:22.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:22 smithi161 ceph-mon[128993]: pgmap v639: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 16 MiB/s wr, 1.01k op/s 2023-07-14T18:26:22.618 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:22 smithi171 ceph-mon[129241]: pgmap v639: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 16 MiB/s wr, 1.01k op/s 2023-07-14T18:26:22.659 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:22 smithi133 ceph-mon[126384]: pgmap v639: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 16 MiB/s wr, 1.01k op/s 2023-07-14T18:26:24.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:24 smithi161 ceph-mon[128993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:26:24.602 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:24 smithi161 ceph-mon[128993]: pgmap v640: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 20 MiB/s wr, 1.23k op/s 2023-07-14T18:26:24.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:24 smithi171 ceph-mon[129241]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:26:24.617 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:24 smithi171 ceph-mon[129241]: pgmap v640: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 20 MiB/s wr, 1.23k op/s 2023-07-14T18:26:24.657 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:24 smithi133 ceph-mon[126384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:26:24.658 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:24 smithi133 ceph-mon[126384]: pgmap v640: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 20 MiB/s wr, 1.23k op/s 2023-07-14T18:26:25.875 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 420.0 s, 569.4 tps, lat 175.890 ms stddev 392.358 2023-07-14T18:26:27.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:26 smithi161 ceph-mon[128993]: pgmap v641: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 18 MiB/s wr, 1.21k op/s 2023-07-14T18:26:27.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:26 smithi171 ceph-mon[129241]: pgmap v641: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 18 MiB/s wr, 1.21k op/s 2023-07-14T18:26:27.158 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:26 smithi133 ceph-mon[126384]: pgmap v641: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 18 MiB/s wr, 1.21k op/s 2023-07-14T18:26:27.368 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:26:27.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.865+0000 7f0231bff700 1 -- 172.21.15.133:0/2949888002 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f022c071010 msgr2=0x7f022c071420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:26:27.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.865+0000 7f0231bff700 1 --2- 172.21.15.133:0/2949888002 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f022c071010 0x7f022c071420 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f02140098f0 tx=0x7f0214009c00 comp rx=0 tx=0).stop 2023-07-14T18:26:27.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.873+0000 7f0231bff700 1 -- 172.21.15.133:0/2949888002 shutdown_connections 2023-07-14T18:26:27.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.873+0000 7f0231bff700 1 --2- 172.21.15.133:0/2949888002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f022c10be90 0x7f022c10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:27.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.873+0000 7f0231bff700 1 --2- 172.21.15.133:0/2949888002 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f022c071010 0x7f022c071420 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:27.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.873+0000 7f0231bff700 1 --2- 172.21.15.133:0/2949888002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f022c071960 0x7f022c10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:27.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.873+0000 7f0231bff700 1 -- 172.21.15.133:0/2949888002 >> 172.21.15.133:0/2949888002 conn(0x7f022c06cbc0 msgr2=0x7f022c06eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:26:27.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.896+0000 7f0231bff700 1 -- 172.21.15.133:0/2949888002 shutdown_connections 2023-07-14T18:26:27.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.897+0000 7f0231bff700 1 -- 172.21.15.133:0/2949888002 wait complete. 2023-07-14T18:26:27.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.898+0000 7f0231bff700 1 Processor -- start 2023-07-14T18:26:27.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.898+0000 7f0231bff700 1 -- start start 2023-07-14T18:26:27.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.899+0000 7f0231bff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f022c071010 0x7f022c114d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:27.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.899+0000 7f0231bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f022c071960 0x7f022c1152b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:27.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.899+0000 7f0231bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f022c10be90 0x7f022c1162f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:27.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.899+0000 7f0231bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022c1168c0 con 0x7f022c071960 2023-07-14T18:26:27.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.899+0000 7f0231bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022c116a30 con 0x7f022c10be90 2023-07-14T18:26:27.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.899+0000 7f0231bff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022c116bf0 con 0x7f022c071010 2023-07-14T18:26:27.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.900+0000 7f022b7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f022c071010 0x7f022c114d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:26:27.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.900+0000 7f022b7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f022c071010 0x7f022c114d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:38296/0 (socket says 172.21.15.133:38296) 2023-07-14T18:26:27.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.900+0000 7f022b7fe700 1 -- 172.21.15.133:0/2412072868 learned_addr learned my addr 172.21.15.133:0/2412072868 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:26:27.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.907+0000 7f022bfff700 1 --2- 172.21.15.133:0/2412072868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f022c10be90 0x7f022c1162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:26:27.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.910+0000 7f022affd700 1 --2- 172.21.15.133:0/2412072868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f022c071960 0x7f022c1152b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:26:27.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.913+0000 7f022b7fe700 1 -- 172.21.15.133:0/2412072868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f022c10be90 msgr2=0x7f022c1162f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:26:27.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.913+0000 7f022b7fe700 1 --2- 172.21.15.133:0/2412072868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f022c10be90 0x7f022c1162f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:27.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.913+0000 7f022b7fe700 1 -- 172.21.15.133:0/2412072868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f022c071960 msgr2=0x7f022c1152b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:26:27.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.913+0000 7f022b7fe700 1 --2- 172.21.15.133:0/2412072868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f022c071960 0x7f022c1152b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:27.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.913+0000 7f022b7fe700 1 -- 172.21.15.133:0/2412072868 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0214009520 con 0x7f022c071010 2023-07-14T18:26:27.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.922+0000 7f022b7fe700 1 --2- 172.21.15.133:0/2412072868 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f022c071010 0x7f022c114d70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f021400bfc0 tx=0x7f021400b560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:26:27.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.935+0000 7f0228ff9700 1 -- 172.21.15.133:0/2412072868 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021401c070 con 0x7f022c071010 2023-07-14T18:26:27.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.935+0000 7f0228ff9700 1 -- 172.21.15.133:0/2412072868 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0214005040 con 0x7f022c071010 2023-07-14T18:26:27.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.935+0000 7f0228ff9700 1 -- 172.21.15.133:0/2412072868 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214004a70 con 0x7f022c071010 2023-07-14T18:26:27.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.935+0000 7f0231bff700 1 -- 172.21.15.133:0/2412072868 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f022c116e10 con 0x7f022c071010 2023-07-14T18:26:27.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.935+0000 7f0231bff700 1 -- 172.21.15.133:0/2412072868 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f022c117270 con 0x7f022c071010 2023-07-14T18:26:27.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.951+0000 7f0245620b80 1 Processor -- start 2023-07-14T18:26:27.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.953+0000 7f0228ff9700 1 -- 172.21.15.133:0/2412072868 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f0214020070 con 0x7f022c071010 2023-07-14T18:26:28.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.956+0000 7f0228ff9700 1 --2- 172.21.15.133:0/2412072868 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f02180a1350 0x7f02180a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:28.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.956+0000 7f0228ff9700 1 -- 172.21.15.133:0/2412072868 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f02140cd810 con 0x7f022c071010 2023-07-14T18:26:28.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.957+0000 7f0245620b80 1 -- start start 2023-07-14T18:26:28.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.957+0000 7f022bfff700 1 --2- 172.21.15.133:0/2412072868 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f02180a1350 0x7f02180a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:26:28.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.957+0000 7f0245620b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639abb7ff60 0x5639abb80370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:28.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.957+0000 7f0245620b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639abb808b0 0x5639abb80ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:28.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.957+0000 7f0245620b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639abb814b0 0x5639abb81920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:28.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.957+0000 7f0245620b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639aba84cb0 con 0x5639abb814b0 2023-07-14T18:26:28.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.957+0000 7f0245620b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639abad7a40 con 0x5639abb808b0 2023-07-14T18:26:28.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.957+0000 7f0245620b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639abad0ed0 con 0x5639abb7ff60 2023-07-14T18:26:28.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.964+0000 7f022affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639abb808b0 0x5639abb80ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:26:28.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.964+0000 7f022affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639abb808b0 0x5639abb80ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:33294/0 (socket says 172.21.15.133:33294) 2023-07-14T18:26:28.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.964+0000 7f022affd700 1 -- 172.21.15.133:0/1646491089 learned_addr learned my addr 172.21.15.133:0/1646491089 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:26:28.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.964+0000 7f022b7fe700 1 --2- 172.21.15.133:0/1646491089 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639abb7ff60 0x5639abb80370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:26:28.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.981+0000 7f022affd700 1 -- 172.21.15.133:0/1646491089 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639abb7ff60 msgr2=0x5639abb80370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:26:28.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.981+0000 7f022affd700 1 --2- 172.21.15.133:0/1646491089 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639abb7ff60 0x5639abb80370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.981+0000 7f022affd700 1 -- 172.21.15.133:0/1646491089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639abb814b0 msgr2=0x5639abb81920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:26:28.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.981+0000 7f022affd700 1 --2- 172.21.15.133:0/1646491089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639abb814b0 0x5639abb81920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.981+0000 7f022affd700 1 -- 172.21.15.133:0/1646491089 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639ab8fede0 con 0x5639abb808b0 2023-07-14T18:26:28.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.982+0000 7f022affd700 1 --2- 172.21.15.133:0/1646491089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639abb808b0 0x5639abb80ed0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f021c0093b0 tx=0x7f021c0099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f39d6dcf7ab6b881 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:26:28.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.992+0000 7f0221ffb700 1 -- 172.21.15.133:0/1646491089 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c00e070 con 0x5639abb808b0 2023-07-14T18:26:28.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.992+0000 7f0221ffb700 1 -- 172.21.15.133:0/1646491089 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f021c00f040 con 0x5639abb808b0 2023-07-14T18:26:28.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.992+0000 7f0221ffb700 1 -- 172.21.15.133:0/1646491089 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c014800 con 0x5639abb808b0 2023-07-14T18:26:28.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.996+0000 7f0245620b80 1 -- 172.21.15.133:0/1646491089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639abb808b0 msgr2=0x5639abb80ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:26:28.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:27.996+0000 7f0245620b80 1 --2- 172.21.15.133:0/1646491089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639abb808b0 0x5639abb80ed0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f021c0093b0 tx=0x7f021c0099e0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.001+0000 7f022bfff700 1 --2- 172.21.15.133:0/2412072868 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f02180a1350 0x7f02180a3800 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0224009d00 tx=0x7f0224011720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:26:28.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.016+0000 7f0245620b80 1 -- 172.21.15.133:0/1646491089 shutdown_connections 2023-07-14T18:26:28.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.016+0000 7f0245620b80 1 --2- 172.21.15.133:0/1646491089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639abb814b0 0x5639abb81920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.016+0000 7f0245620b80 1 --2- 172.21.15.133:0/1646491089 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639abb7ff60 0x5639abb80370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.016+0000 7f0245620b80 1 --2- 172.21.15.133:0/1646491089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639abb808b0 0x5639abb80ed0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.016+0000 7f0245620b80 1 -- 172.21.15.133:0/1646491089 >> 172.21.15.133:0/1646491089 conn(0x5639abb75680 msgr2=0x5639abb7a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:26:28.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.030+0000 7f0245620b80 1 -- 172.21.15.133:0/1646491089 shutdown_connections 2023-07-14T18:26:28.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.035+0000 7f0245620b80 1 -- 172.21.15.133:0/1646491089 wait complete. 2023-07-14T18:26:28.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.035+0000 7f0245620b80 1 -- start start 2023-07-14T18:26:28.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.036+0000 7f0245620b80 1 Processor -- start 2023-07-14T18:26:28.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.036+0000 7f0245620b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639abb7ff60 0x5639abb89a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:28.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.037+0000 7f0245620b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639abb808b0 0x5639abb89f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:28.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.037+0000 7f0245620b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639abb814b0 0x5639abb90f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:28.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.037+0000 7f0245620b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639aba70ac0 con 0x5639abb808b0 2023-07-14T18:26:28.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.037+0000 7f0245620b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639aba707c0 con 0x5639abb7ff60 2023-07-14T18:26:28.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.037+0000 7f0245620b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639ab954920 con 0x5639abb814b0 2023-07-14T18:26:28.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.037+0000 7f0245620b80 10 client.0 _ll_register_callbacks cb 0x5639aba6e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:26:28.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.037+0000 7f022bfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639abb814b0 0x5639abb90f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:26:28.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.038+0000 7f022bfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639abb814b0 0x5639abb90f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:38310/0 (socket says 172.21.15.133:38310) 2023-07-14T18:26:28.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.038+0000 7f022bfff700 1 -- 172.21.15.133:0/609639440 learned_addr learned my addr 172.21.15.133:0/609639440 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:26:28.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.038+0000 7f022affd700 1 --2- 172.21.15.133:0/609639440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639abb808b0 0x5639abb89f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:26:28.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.038+0000 7f022bfff700 1 -- 172.21.15.133:0/609639440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639abb7ff60 msgr2=0x5639abb89a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T18:26:28.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.038+0000 7f022bfff700 1 --2- 172.21.15.133:0/609639440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639abb7ff60 0x5639abb89a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.038+0000 7f022bfff700 1 -- 172.21.15.133:0/609639440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639abb808b0 msgr2=0x5639abb89f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:26:28.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.038+0000 7f022bfff700 1 --2- 172.21.15.133:0/609639440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639abb808b0 0x5639abb89f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.038+0000 7f022bfff700 1 -- 172.21.15.133:0/609639440 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f021c009090 con 0x5639abb814b0 2023-07-14T18:26:28.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.039+0000 7f022bfff700 1 --2- 172.21.15.133:0/609639440 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639abb814b0 0x5639abb90f90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f022400e6e0 tx=0x7f02240047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=454b687b1e44723b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:26:28.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.040+0000 7f020affd700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:26:28.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.040+0000 7f020affd700 1 -- 172.21.15.133:0/609639440 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0224026630 con 0x5639abb814b0 2023-07-14T18:26:28.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.040+0000 7f020affd700 1 -- 172.21.15.133:0/609639440 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f022401c410 con 0x5639abb814b0 2023-07-14T18:26:28.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.043+0000 7f0245620b80 1 -- 172.21.15.133:0/609639440 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639aba2a4f0 con 0x5639abb814b0 2023-07-14T18:26:28.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.052+0000 7f020affd700 1 -- 172.21.15.133:0/609639440 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0224026790 con 0x5639abb814b0 2023-07-14T18:26:28.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.059+0000 7f0245620b80 10 client.25915 fetch_fsmap learned FSMap version 38 2023-07-14T18:26:28.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.059+0000 7f0245620b80 1 -- 172.21.15.133:0/609639440 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5639ab8e7c60 con 0x5639abb814b0 2023-07-14T18:26:28.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.067+0000 7f020affd700 1 -- 172.21.15.133:0/609639440 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5639aba2a4f0 con 0x5639abb814b0 2023-07-14T18:26:28.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.068+0000 7f020affd700 1 -- 172.21.15.133:0/609639440 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0224030470 con 0x5639abb814b0 2023-07-14T18:26:28.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.070+0000 7f0245620b80 10 client.25915 fetch_fsmap finished waiting for FSMap version 38 2023-07-14T18:26:28.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.070+0000 7f0245620b80 10 client.25915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:26:28.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.070+0000 7f0245620b80 20 client.25915 populate_metadata read hostname 'smithi133' 2023-07-14T18:26:28.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.070+0000 7f0245620b80 1 --2- 172.21.15.133:0/609639440 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5639abb957f0 0x5639abb97c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:28.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.070+0000 7f0245620b80 4 client.25915 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:26:28.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.070+0000 7f0245620b80 1 -- 172.21.15.133:0/609639440 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639abb4eda0 con 0x5639abb957f0 2023-07-14T18:26:28.122 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.076+0000 7f022b7fe700 1 --2- 172.21.15.133:0/609639440 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5639abb957f0 0x5639abb97c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:26:28.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.100+0000 7f022b7fe700 1 --2- 172.21.15.133:0/609639440 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5639abb957f0 0x5639abb97c00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b641ed906ba26569 server_cookie=8b62289a852926ad in_seq=0 out_seq=0 2023-07-14T18:26:28.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.105+0000 7f020affd700 10 client.25915 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:26:28.156 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.111+0000 7f020affd700 1 -- 172.21.15.133:0/609639440 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5639abb4eda0 con 0x5639abb957f0 2023-07-14T18:26:28.156 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.111+0000 7f020affd700 10 client.25915 handle_command_reply: tid=0 2023-07-14T18:26:28.156 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.111+0000 7f0245620b80 1 client.25915 shutdown 2023-07-14T18:26:28.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.119+0000 7f0245620b80 1 -- 172.21.15.133:0/609639440 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639abb814b0 msgr2=0x5639abb90f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:26:28.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.119+0000 7f0245620b80 1 --2- 172.21.15.133:0/609639440 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639abb814b0 0x5639abb90f90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f022400e6e0 tx=0x7f02240047c0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.186 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.141+0000 7f0245620b80 1 -- 172.21.15.133:0/609639440 shutdown_connections 2023-07-14T18:26:28.186 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.141+0000 7f0245620b80 1 --2- 172.21.15.133:0/609639440 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5639abb957f0 0x5639abb97c00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.186 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.141+0000 7f0245620b80 1 --2- 172.21.15.133:0/609639440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639abb7ff60 0x5639abb89a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.186 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.141+0000 7f0245620b80 1 --2- 172.21.15.133:0/609639440 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639abb814b0 0x5639abb90f90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.186 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.141+0000 7f0245620b80 1 --2- 172.21.15.133:0/609639440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639abb808b0 0x5639abb89f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.187 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.141+0000 7f0245620b80 1 -- 172.21.15.133:0/609639440 >> 172.21.15.133:0/609639440 conn(0x5639abb75680 msgr2=0x5639abb79650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:26:28.187 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.141+0000 7f020affd700 0 client.25915 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:26:28.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.146+0000 7f0245620b80 1 -- 172.21.15.133:0/609639440 shutdown_connections 2023-07-14T18:26:28.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.150+0000 7f0245620b80 1 -- 172.21.15.133:0/609639440 wait complete. 2023-07-14T18:26:28.205 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.160+0000 7f0245620b80 20 client.25915 trim_cache size 0 max 16384 2023-07-14T18:26:28.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.198+0000 7f0245620b80 1 Processor -- start 2023-07-14T18:26:28.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.198+0000 7f0245620b80 1 -- start start 2023-07-14T18:26:28.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.199+0000 7f0245620b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639abb7ff60 0x5639abb87870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:28.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.199+0000 7f0245620b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639abb808b0 0x5639abb90eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:28.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.199+0000 7f0245620b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639abb814b0 0x5639abb89a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:28.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.199+0000 7f0245620b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639abb752a0 con 0x5639abb7ff60 2023-07-14T18:26:28.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.199+0000 7f0245620b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639abb7bcc0 con 0x5639abb808b0 2023-07-14T18:26:28.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.199+0000 7f0245620b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639abb92c80 con 0x5639abb814b0 2023-07-14T18:26:28.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.200+0000 7f022bfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639abb814b0 0x5639abb89a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:26:28.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.211+0000 7f022bfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639abb814b0 0x5639abb89a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:38322/0 (socket says 172.21.15.133:38322) 2023-07-14T18:26:28.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.211+0000 7f022bfff700 1 -- 172.21.15.133:0/3441268904 learned_addr learned my addr 172.21.15.133:0/3441268904 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:26:28.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.211+0000 7f022b7fe700 1 --2- 172.21.15.133:0/3441268904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639abb7ff60 0x5639abb87870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:26:28.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.212+0000 7f022bfff700 1 -- 172.21.15.133:0/3441268904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639abb808b0 msgr2=0x5639abb90eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:26:28.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.212+0000 7f022bfff700 1 --2- 172.21.15.133:0/3441268904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639abb808b0 0x5639abb90eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.212+0000 7f022bfff700 1 -- 172.21.15.133:0/3441268904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639abb7ff60 msgr2=0x5639abb87870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:26:28.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.212+0000 7f022bfff700 1 --2- 172.21.15.133:0/3441268904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639abb7ff60 0x5639abb87870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.212+0000 7f022bfff700 1 -- 172.21.15.133:0/3441268904 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639ab925250 con 0x5639abb814b0 2023-07-14T18:26:28.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.222+0000 7f022bfff700 1 --2- 172.21.15.133:0/3441268904 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639abb814b0 0x5639abb89a10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f022400e6b0 tx=0x7f0224004710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7fe48766e98f363 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:26:28.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.231+0000 7f02227fc700 1 -- 172.21.15.133:0/3441268904 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0224030af0 con 0x5639abb814b0 2023-07-14T18:26:28.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.231+0000 7f02227fc700 1 -- 172.21.15.133:0/3441268904 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0224021070 con 0x5639abb814b0 2023-07-14T18:26:28.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.231+0000 7f02227fc700 1 -- 172.21.15.133:0/3441268904 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0224026430 con 0x5639abb814b0 2023-07-14T18:26:28.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.232+0000 7f0245620b80 1 -- 172.21.15.133:0/3441268904 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639abb814b0 msgr2=0x5639abb89a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:26:28.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.232+0000 7f0245620b80 1 --2- 172.21.15.133:0/3441268904 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639abb814b0 0x5639abb89a10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f022400e6b0 tx=0x7f0224004710 comp rx=0 tx=0).stop 2023-07-14T18:26:28.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.252+0000 7f0245620b80 1 -- 172.21.15.133:0/3441268904 shutdown_connections 2023-07-14T18:26:28.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.252+0000 7f0245620b80 1 --2- 172.21.15.133:0/3441268904 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639abb814b0 0x5639abb89a10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.252+0000 7f0245620b80 1 --2- 172.21.15.133:0/3441268904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639abb7ff60 0x5639abb87870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.252+0000 7f0245620b80 1 --2- 172.21.15.133:0/3441268904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639abb808b0 0x5639abb90eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.252+0000 7f0245620b80 1 -- 172.21.15.133:0/3441268904 >> 172.21.15.133:0/3441268904 conn(0x5639abb75680 msgr2=0x5639abb91880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:26:28.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.266+0000 7f0245620b80 1 -- 172.21.15.133:0/3441268904 shutdown_connections 2023-07-14T18:26:28.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.270+0000 7f0245620b80 1 -- 172.21.15.133:0/3441268904 wait complete. 2023-07-14T18:26:28.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.271+0000 7f0245620b80 1 -- start start 2023-07-14T18:26:28.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.271+0000 7f0245620b80 1 Processor -- start 2023-07-14T18:26:28.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.272+0000 7f0245620b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639abb7ff60 0x5639abb909a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:28.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.272+0000 7f0245620b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639abb808b0 0x5639abb8d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:28.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.272+0000 7f0245620b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639abb814b0 0x5639abb8da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:28.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.272+0000 7f0245620b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639abb8e1d0 con 0x5639abb7ff60 2023-07-14T18:26:28.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.272+0000 7f0245620b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639abb91050 con 0x5639abb808b0 2023-07-14T18:26:28.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.272+0000 7f0245620b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639abb91210 con 0x5639abb814b0 2023-07-14T18:26:28.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.272+0000 7f0245620b80 10 client.0 _ll_register_callbacks cb 0x7f0224011ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:26:28.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.274+0000 7f022bfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639abb814b0 0x5639abb8da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:26:28.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.276+0000 7f022affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639abb808b0 0x5639abb8d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:26:28.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.276+0000 7f022affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639abb808b0 0x5639abb8d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:33326/0 (socket says 172.21.15.133:33326) 2023-07-14T18:26:28.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.276+0000 7f022affd700 1 -- 172.21.15.133:0/4251759814 learned_addr learned my addr 172.21.15.133:0/4251759814 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:26:28.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.277+0000 7f022affd700 1 -- 172.21.15.133:0/4251759814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639abb814b0 msgr2=0x5639abb8da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:26:28.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.277+0000 7f022affd700 1 --2- 172.21.15.133:0/4251759814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639abb814b0 0x5639abb8da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.277+0000 7f022affd700 1 -- 172.21.15.133:0/4251759814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639abb7ff60 msgr2=0x5639abb909a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:26:28.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.277+0000 7f022affd700 1 --2- 172.21.15.133:0/4251759814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639abb7ff60 0x5639abb909a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.277+0000 7f022affd700 1 -- 172.21.15.133:0/4251759814 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639ab8e7c60 con 0x5639abb808b0 2023-07-14T18:26:28.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.278+0000 7f022affd700 1 --2- 172.21.15.133:0/4251759814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639abb808b0 0x5639abb8d550 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x5639abb80cc0 tx=0x7f021c016730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6daac2f4154dcfc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:26:28.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.285+0000 7f020affd700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:26:28.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.285+0000 7f020affd700 1 -- 172.21.15.133:0/4251759814 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c00e040 con 0x5639abb808b0 2023-07-14T18:26:28.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.285+0000 7f020affd700 1 -- 172.21.15.133:0/4251759814 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f021c016ae0 con 0x5639abb808b0 2023-07-14T18:26:28.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.285+0000 7f0245620b80 1 -- 172.21.15.133:0/4251759814 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f021c00f040 con 0x5639abb808b0 2023-07-14T18:26:28.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.286+0000 7f020affd700 1 -- 172.21.15.133:0/4251759814 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f021c016ca0 con 0x5639abb808b0 2023-07-14T18:26:28.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.286+0000 7f0245620b80 10 client.25874 fetch_fsmap learned FSMap version 38 2023-07-14T18:26:28.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.286+0000 7f0245620b80 1 -- 172.21.15.133:0/4251759814 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5639ab8fd2b0 con 0x5639abb808b0 2023-07-14T18:26:28.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.293+0000 7f020affd700 1 -- 172.21.15.133:0/4251759814 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f021c00f040 con 0x5639abb808b0 2023-07-14T18:26:28.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.293+0000 7f020affd700 1 -- 172.21.15.133:0/4251759814 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f021c01d500 con 0x5639abb808b0 2023-07-14T18:26:28.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.293+0000 7f0245620b80 10 client.25874 fetch_fsmap finished waiting for FSMap version 38 2023-07-14T18:26:28.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.293+0000 7f0245620b80 10 client.25874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:26:28.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.293+0000 7f0245620b80 20 client.25874 populate_metadata read hostname 'smithi133' 2023-07-14T18:26:28.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.293+0000 7f0245620b80 1 --2- 172.21.15.133:0/4251759814 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5639abbeab20 0x5639abbecf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:28.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.293+0000 7f0245620b80 4 client.25874 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:26:28.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.293+0000 7f0245620b80 1 -- 172.21.15.133:0/4251759814 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5639abac9eb0 con 0x5639abbeab20 2023-07-14T18:26:28.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.305+0000 7f022b7fe700 1 --2- 172.21.15.133:0/4251759814 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5639abbeab20 0x5639abbecf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:26:28.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.325+0000 7f020affd700 10 client.25874 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:26:28.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.325+0000 7f022b7fe700 1 --2- 172.21.15.133:0/4251759814 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5639abbeab20 0x5639abbecf30 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fad61da19312089d server_cookie=8bc236ef484288fc in_seq=0 out_seq=0 2023-07-14T18:26:28.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.341+0000 7f020affd700 1 -- 172.21.15.133:0/4251759814 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5639abac9eb0 con 0x5639abbeab20 2023-07-14T18:26:28.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.341+0000 7f020affd700 10 client.25874 handle_command_reply: tid=0 2023-07-14T18:26:28.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.341+0000 7f0245620b80 1 client.25874 shutdown 2023-07-14T18:26:28.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.353+0000 7f0245620b80 1 -- 172.21.15.133:0/4251759814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639abb808b0 msgr2=0x5639abb8d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:26:28.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.353+0000 7f0245620b80 1 --2- 172.21.15.133:0/4251759814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639abb808b0 0x5639abb8d550 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x5639abb80cc0 tx=0x7f021c016730 comp rx=0 tx=0).stop 2023-07-14T18:26:28.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.354+0000 7f0245620b80 1 -- 172.21.15.133:0/4251759814 shutdown_connections 2023-07-14T18:26:28.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.354+0000 7f0245620b80 1 --2- 172.21.15.133:0/4251759814 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5639abbeab20 0x5639abbecf30 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.354+0000 7f0245620b80 1 --2- 172.21.15.133:0/4251759814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639abb814b0 0x5639abb8da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.354+0000 7f0245620b80 1 --2- 172.21.15.133:0/4251759814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639abb7ff60 0x5639abb909a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.354+0000 7f0245620b80 1 --2- 172.21.15.133:0/4251759814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639abb808b0 0x5639abb8d550 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.354+0000 7f0245620b80 1 -- 172.21.15.133:0/4251759814 >> 172.21.15.133:0/4251759814 conn(0x5639abb75680 msgr2=0x5639abb87870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:26:28.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.354+0000 7f020affd700 0 client.25874 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:26:28.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.355+0000 7f0245620b80 1 -- 172.21.15.133:0/4251759814 shutdown_connections 2023-07-14T18:26:28.410 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:26:28.411 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T18:26:28.411 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:26:28.411 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:26:28.411 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:26:28.411 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:26:28.411 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:26:28.411 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:26:28.412 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:26:28.412 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:26:28.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.358+0000 7f0245620b80 1 -- 172.21.15.133:0/4251759814 wait complete. 2023-07-14T18:26:28.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.365+0000 7f0245620b80 20 client.25874 trim_cache size 0 max 16384 2023-07-14T18:26:28.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.374+0000 7f0231bff700 1 -- 172.21.15.133:0/2412072868 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f02180a1350 msgr2=0x7f02180a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:26:28.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.374+0000 7f0231bff700 1 --2- 172.21.15.133:0/2412072868 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f02180a1350 0x7f02180a3800 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0224009d00 tx=0x7f0224011720 comp rx=0 tx=0).stop 2023-07-14T18:26:28.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.374+0000 7f0231bff700 1 -- 172.21.15.133:0/2412072868 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f022c071010 msgr2=0x7f022c114d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:26:28.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.374+0000 7f0231bff700 1 --2- 172.21.15.133:0/2412072868 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f022c071010 0x7f022c114d70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f021400bfc0 tx=0x7f021400b560 comp rx=0 tx=0).stop 2023-07-14T18:26:28.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.393+0000 7f0231bff700 1 -- 172.21.15.133:0/2412072868 shutdown_connections 2023-07-14T18:26:28.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.394+0000 7f0231bff700 1 --2- 172.21.15.133:0/2412072868 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f02180a1350 0x7f02180a3800 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.394+0000 7f0231bff700 1 --2- 172.21.15.133:0/2412072868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f022c10be90 0x7f022c1162f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.394+0000 7f0231bff700 1 --2- 172.21.15.133:0/2412072868 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f022c071010 0x7f022c114d70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.394+0000 7f0231bff700 1 --2- 172.21.15.133:0/2412072868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f022c071960 0x7f022c1152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:28.443 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.394+0000 7f0231bff700 1 -- 172.21.15.133:0/2412072868 >> 172.21.15.133:0/2412072868 conn(0x7f022c06cbc0 msgr2=0x7f022c06deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:26:28.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.414+0000 7f0231bff700 1 -- 172.21.15.133:0/2412072868 shutdown_connections 2023-07-14T18:26:28.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:28.425+0000 7f0231bff700 1 -- 172.21.15.133:0/2412072868 wait complete. 2023-07-14T18:26:28.543 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:26:28.544 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:28 smithi133 ceph-mon[126384]: pgmap v642: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 22 MiB/s wr, 1.58k op/s 2023-07-14T18:26:28.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:28 smithi161 ceph-mon[128993]: pgmap v642: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 22 MiB/s wr, 1.58k op/s 2023-07-14T18:26:28.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:28 smithi171 ceph-mon[129241]: pgmap v642: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 22 MiB/s wr, 1.58k op/s 2023-07-14T18:26:30.876 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 425.0 s, 771.9 tps, lat 127.639 ms stddev 245.666 2023-07-14T18:26:31.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:31 smithi161 ceph-mon[128993]: pgmap v643: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 21 MiB/s wr, 1.47k op/s 2023-07-14T18:26:31.617 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:31 smithi171 ceph-mon[129241]: pgmap v643: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 21 MiB/s wr, 1.47k op/s 2023-07-14T18:26:31.672 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:31 smithi133 ceph-mon[126384]: pgmap v643: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 21 MiB/s wr, 1.47k op/s 2023-07-14T18:26:32.602 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:32 smithi161 ceph-mon[128993]: pgmap v644: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 21 MiB/s wr, 1.53k op/s 2023-07-14T18:26:32.621 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:32 smithi171 ceph-mon[129241]: pgmap v644: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 21 MiB/s wr, 1.53k op/s 2023-07-14T18:26:32.664 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:32 smithi133 ceph-mon[126384]: pgmap v644: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 21 MiB/s wr, 1.53k op/s 2023-07-14T18:26:34.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:34 smithi161 ceph-mon[128993]: pgmap v645: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 22 MiB/s wr, 1.71k op/s 2023-07-14T18:26:34.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:34 smithi171 ceph-mon[129241]: pgmap v645: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 22 MiB/s wr, 1.71k op/s 2023-07-14T18:26:34.662 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:34 smithi133 ceph-mon[126384]: pgmap v645: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 22 MiB/s wr, 1.71k op/s 2023-07-14T18:26:35.874 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 430.0 s, 423.9 tps, lat 233.416 ms stddev 630.003 2023-07-14T18:26:37.107 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:36 smithi161 ceph-mon[128993]: pgmap v646: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 19 MiB/s wr, 1.52k op/s 2023-07-14T18:26:37.120 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:36 smithi171 ceph-mon[129241]: pgmap v646: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 19 MiB/s wr, 1.52k op/s 2023-07-14T18:26:37.163 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:36 smithi133 ceph-mon[126384]: pgmap v646: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 19 MiB/s wr, 1.52k op/s 2023-07-14T18:26:38.600 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:38 smithi161 ceph-mon[128993]: pgmap v647: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 20 MiB/s wr, 1.57k op/s 2023-07-14T18:26:38.618 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:38 smithi171 ceph-mon[129241]: pgmap v647: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 20 MiB/s wr, 1.57k op/s 2023-07-14T18:26:38.657 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:38 smithi133 ceph-mon[126384]: pgmap v647: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 20 MiB/s wr, 1.57k op/s 2023-07-14T18:26:40.600 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:40 smithi161 ceph-mon[128993]: pgmap v648: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 16 MiB/s wr, 1.21k op/s 2023-07-14T18:26:40.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:40 smithi171 ceph-mon[129241]: pgmap v648: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 16 MiB/s wr, 1.21k op/s 2023-07-14T18:26:40.658 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:40 smithi133 ceph-mon[126384]: pgmap v648: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 16 MiB/s wr, 1.21k op/s 2023-07-14T18:26:40.875 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 435.0 s, 432.8 tps, lat 227.121 ms stddev 328.305 2023-07-14T18:26:42.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:42 smithi161 ceph-mon[128993]: pgmap v649: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 15 MiB/s wr, 1.18k op/s 2023-07-14T18:26:42.617 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:42 smithi171 ceph-mon[129241]: pgmap v649: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 15 MiB/s wr, 1.18k op/s 2023-07-14T18:26:42.662 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:42 smithi133 ceph-mon[126384]: pgmap v649: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 15 MiB/s wr, 1.18k op/s 2023-07-14T18:26:45.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:45 smithi161 ceph-mon[128993]: pgmap v650: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 14 MiB/s wr, 1.16k op/s 2023-07-14T18:26:45.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:45 smithi171 ceph-mon[129241]: pgmap v650: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 14 MiB/s wr, 1.16k op/s 2023-07-14T18:26:45.664 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:45 smithi133 ceph-mon[126384]: pgmap v650: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 14 MiB/s wr, 1.16k op/s 2023-07-14T18:26:45.875 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 440.0 s, 369.1 tps, lat 269.305 ms stddev 277.913 2023-07-14T18:26:47.102 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:46 smithi161 ceph-mon[128993]: pgmap v651: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 13 MiB/s wr, 986 op/s 2023-07-14T18:26:47.115 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:46 smithi171 ceph-mon[129241]: pgmap v651: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 13 MiB/s wr, 986 op/s 2023-07-14T18:26:47.159 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:46 smithi133 ceph-mon[126384]: pgmap v651: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 13 MiB/s wr, 986 op/s 2023-07-14T18:26:49.600 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:49 smithi161 ceph-mon[128993]: pgmap v652: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 12 MiB/s wr, 954 op/s 2023-07-14T18:26:49.619 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:49 smithi171 ceph-mon[129241]: pgmap v652: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 12 MiB/s wr, 954 op/s 2023-07-14T18:26:49.657 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:49 smithi133 ceph-mon[126384]: pgmap v652: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 12 MiB/s wr, 954 op/s 2023-07-14T18:26:50.878 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 445.0 s, 848.7 tps, lat 122.484 ms stddev 119.758 2023-07-14T18:26:51.600 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:51 smithi161 ceph-mon[128993]: pgmap v653: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 11 MiB/s wr, 804 op/s 2023-07-14T18:26:51.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:51 smithi171 ceph-mon[129241]: pgmap v653: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 11 MiB/s wr, 804 op/s 2023-07-14T18:26:51.656 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:51 smithi133 ceph-mon[126384]: pgmap v653: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 11 MiB/s wr, 804 op/s 2023-07-14T18:26:52.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:52 smithi161 ceph-mon[128993]: pgmap v654: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 11 MiB/s wr, 834 op/s 2023-07-14T18:26:52.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:52 smithi171 ceph-mon[129241]: pgmap v654: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 11 MiB/s wr, 834 op/s 2023-07-14T18:26:52.661 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:52 smithi133 ceph-mon[126384]: pgmap v654: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 11 MiB/s wr, 834 op/s 2023-07-14T18:26:54.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:54 smithi161 ceph-mon[128993]: pgmap v655: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 13 MiB/s wr, 840 op/s 2023-07-14T18:26:54.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:54 smithi171 ceph-mon[129241]: pgmap v655: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 13 MiB/s wr, 840 op/s 2023-07-14T18:26:54.656 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:54 smithi133 ceph-mon[126384]: pgmap v655: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 13 MiB/s wr, 840 op/s 2023-07-14T18:26:55.873 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 450.0 s, 590.4 tps, lat 165.718 ms stddev 345.904 2023-07-14T18:26:57.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:56 smithi171 ceph-mon[129241]: pgmap v656: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 14 MiB/s wr, 872 op/s 2023-07-14T18:26:57.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:56 smithi133 ceph-mon[126384]: pgmap v656: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 14 MiB/s wr, 872 op/s 2023-07-14T18:26:57.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:56 smithi161 ceph-mon[128993]: pgmap v656: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 14 MiB/s wr, 872 op/s 2023-07-14T18:26:58.545 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:26:58.656 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:26:58 smithi133 ceph-mon[126384]: pgmap v657: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 14 MiB/s wr, 951 op/s 2023-07-14T18:26:58.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:26:58 smithi161 ceph-mon[128993]: pgmap v657: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 14 MiB/s wr, 951 op/s 2023-07-14T18:26:58.864 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:26:58 smithi171 ceph-mon[129241]: pgmap v657: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 14 MiB/s wr, 951 op/s 2023-07-14T18:26:59.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.087+0000 7f60539ff700 1 -- 172.21.15.133:0/938074648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f604c071960 msgr2=0x7f604c10b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:26:59.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.087+0000 7f60539ff700 1 --2- 172.21.15.133:0/938074648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f604c071960 0x7f604c10b900 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6044009f90 tx=0x7f604400b330 comp rx=0 tx=0).stop 2023-07-14T18:26:59.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.096+0000 7f60539ff700 1 -- 172.21.15.133:0/938074648 shutdown_connections 2023-07-14T18:26:59.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.096+0000 7f60539ff700 1 --2- 172.21.15.133:0/938074648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f604c071010 0x7f604c071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.096+0000 7f60539ff700 1 --2- 172.21.15.133:0/938074648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f604c10be90 0x7f604c10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.096+0000 7f60539ff700 1 --2- 172.21.15.133:0/938074648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f604c071960 0x7f604c10b900 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.096+0000 7f60539ff700 1 -- 172.21.15.133:0/938074648 >> 172.21.15.133:0/938074648 conn(0x7f604c06cbc0 msgr2=0x7f604c06eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:26:59.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.113+0000 7f60539ff700 1 -- 172.21.15.133:0/938074648 shutdown_connections 2023-07-14T18:26:59.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.114+0000 7f60539ff700 1 -- 172.21.15.133:0/938074648 wait complete. 2023-07-14T18:26:59.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.115+0000 7f60539ff700 1 Processor -- start 2023-07-14T18:26:59.176 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.122+0000 7f60539ff700 1 -- start start 2023-07-14T18:26:59.177 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.123+0000 7f60539ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f604c071010 0x7f604c19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:59.177 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.123+0000 7f60539ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f604c071960 0x7f604c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:59.177 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.123+0000 7f60539ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f604c10be90 0x7f604c1a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:59.177 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.123+0000 7f60539ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c1a3c90 con 0x7f604c071010 2023-07-14T18:26:59.177 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.123+0000 7f60539ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c1a3e00 con 0x7f604c10be90 2023-07-14T18:26:59.177 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.123+0000 7f60539ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c1a3fc0 con 0x7f604c071960 2023-07-14T18:26:59.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.124+0000 7f6051f9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f604c10be90 0x7f604c1a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:26:59.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.124+0000 7f6051f9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f604c10be90 0x7f604c1a3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:44800/0 (socket says 172.21.15.133:44800) 2023-07-14T18:26:59.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.124+0000 7f6051f9c700 1 -- 172.21.15.133:0/2386714292 learned_addr learned my addr 172.21.15.133:0/2386714292 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:26:59.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.124+0000 7f605179b700 1 --2- 172.21.15.133:0/2386714292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f604c071010 0x7f604c19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:26:59.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.124+0000 7f6051f9c700 1 -- 172.21.15.133:0/2386714292 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f604c071960 msgr2=0x7f604c19ce30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T18:26:59.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.124+0000 7f6051f9c700 1 --2- 172.21.15.133:0/2386714292 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f604c071960 0x7f604c19ce30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.124+0000 7f6051f9c700 1 -- 172.21.15.133:0/2386714292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f604c071010 msgr2=0x7f604c19c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:26:59.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.124+0000 7f6051f9c700 1 --2- 172.21.15.133:0/2386714292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f604c071010 0x7f604c19c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.125+0000 7f6051f9c700 1 -- 172.21.15.133:0/2386714292 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6044009c40 con 0x7f604c10be90 2023-07-14T18:26:59.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.125+0000 7f6051f9c700 1 --2- 172.21.15.133:0/2386714292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f604c10be90 0x7f604c1a3750 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6048009f60 tx=0x7f6048009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:26:59.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.130+0000 7f60427fc700 1 -- 172.21.15.133:0/2386714292 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6048033640 con 0x7f604c10be90 2023-07-14T18:26:59.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.130+0000 7f60427fc700 1 -- 172.21.15.133:0/2386714292 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6048033c60 con 0x7f604c10be90 2023-07-14T18:26:59.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.130+0000 7f60427fc700 1 -- 172.21.15.133:0/2386714292 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6048032c80 con 0x7f604c10be90 2023-07-14T18:26:59.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.138+0000 7f60539ff700 1 -- 172.21.15.133:0/2386714292 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f604c1a42a0 con 0x7f604c10be90 2023-07-14T18:26:59.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.138+0000 7f60539ff700 1 -- 172.21.15.133:0/2386714292 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f604c1a4730 con 0x7f604c10be90 2023-07-14T18:26:59.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.144+0000 7f60672beb80 1 Processor -- start 2023-07-14T18:26:59.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.144+0000 7f60672beb80 1 -- start start 2023-07-14T18:26:59.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.145+0000 7f60672beb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623357b9f60 0x5623357ba370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:59.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.145+0000 7f60672beb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623357ba8b0 0x5623357baed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:59.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.145+0000 7f60672beb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623357bb4b0 0x5623357bb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:59.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.145+0000 7f60672beb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623356becb0 con 0x5623357b9f60 2023-07-14T18:26:59.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.145+0000 7f60672beb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562335711a40 con 0x5623357bb4b0 2023-07-14T18:26:59.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.145+0000 7f60672beb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x56233570aed0 con 0x5623357ba8b0 2023-07-14T18:26:59.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.146+0000 7f605179b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623357ba8b0 0x5623357baed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:26:59.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.146+0000 7f605179b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623357ba8b0 0x5623357baed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:51554/0 (socket says 172.21.15.133:51554) 2023-07-14T18:26:59.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.146+0000 7f605179b700 1 -- 172.21.15.133:0/1084407490 learned_addr learned my addr 172.21.15.133:0/1084407490 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:26:59.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.146+0000 7f605179b700 1 -- 172.21.15.133:0/1084407490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623357bb4b0 msgr2=0x5623357bb920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T18:26:59.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.146+0000 7f605179b700 1 --2- 172.21.15.133:0/1084407490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623357bb4b0 0x5623357bb920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.146+0000 7f605179b700 1 -- 172.21.15.133:0/1084407490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623357b9f60 msgr2=0x5623357ba370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T18:26:59.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.146+0000 7f605179b700 1 --2- 172.21.15.133:0/1084407490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623357b9f60 0x5623357ba370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.146+0000 7f605179b700 1 -- 172.21.15.133:0/1084407490 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562335538de0 con 0x5623357ba8b0 2023-07-14T18:26:59.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.154+0000 7f605179b700 1 --2- 172.21.15.133:0/1084407490 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623357ba8b0 0x5623357baed0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f603c01c770 tx=0x7f603c01ca80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=779a235c72a8512a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:26:59.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.155+0000 7f60377fe700 1 -- 172.21.15.133:0/1084407490 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c003a90 con 0x5623357ba8b0 2023-07-14T18:26:59.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.155+0000 7f60377fe700 1 -- 172.21.15.133:0/1084407490 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f603c003bf0 con 0x5623357ba8b0 2023-07-14T18:26:59.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.155+0000 7f60377fe700 1 -- 172.21.15.133:0/1084407490 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c00c620 con 0x5623357ba8b0 2023-07-14T18:26:59.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.156+0000 7f60672beb80 1 -- 172.21.15.133:0/1084407490 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623357ba8b0 msgr2=0x5623357baed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:26:59.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.156+0000 7f60672beb80 1 --2- 172.21.15.133:0/1084407490 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623357ba8b0 0x5623357baed0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f603c01c770 tx=0x7f603c01ca80 comp rx=0 tx=0).stop 2023-07-14T18:26:59.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.158+0000 7f60427fc700 1 -- 172.21.15.133:0/2386714292 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f60480337a0 con 0x7f604c10be90 2023-07-14T18:26:59.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.158+0000 7f60672beb80 1 -- 172.21.15.133:0/1084407490 shutdown_connections 2023-07-14T18:26:59.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.158+0000 7f60672beb80 1 --2- 172.21.15.133:0/1084407490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623357bb4b0 0x5623357bb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.158+0000 7f60672beb80 1 --2- 172.21.15.133:0/1084407490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623357b9f60 0x5623357ba370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.158+0000 7f60672beb80 1 --2- 172.21.15.133:0/1084407490 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623357ba8b0 0x5623357baed0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.158+0000 7f60672beb80 1 -- 172.21.15.133:0/1084407490 >> 172.21.15.133:0/1084407490 conn(0x5623357af680 msgr2=0x5623357b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:26:59.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.158+0000 7f60672beb80 1 -- 172.21.15.133:0/1084407490 shutdown_connections 2023-07-14T18:26:59.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.158+0000 7f60672beb80 1 -- 172.21.15.133:0/1084407490 wait complete. 2023-07-14T18:26:59.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.159+0000 7f60672beb80 1 -- start start 2023-07-14T18:26:59.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.160+0000 7f60672beb80 1 Processor -- start 2023-07-14T18:26:59.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.160+0000 7f60672beb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623357b9f60 0x5623357e59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:59.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.160+0000 7f60672beb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623357ba8b0 0x5623357e5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:59.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.160+0000 7f60672beb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623357bb4b0 0x5623357ecf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:59.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.160+0000 7f60672beb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623356aaac0 con 0x5623357ba8b0 2023-07-14T18:26:59.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.160+0000 7f60672beb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623356aa7c0 con 0x5623357b9f60 2023-07-14T18:26:59.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.160+0000 7f60672beb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x56233558e920 con 0x5623357bb4b0 2023-07-14T18:26:59.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.160+0000 7f60672beb80 10 client.0 _ll_register_callbacks cb 0x5623356a8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:26:59.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.161+0000 7f6050f9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623357b9f60 0x5623357e59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:26:59.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.161+0000 7f6050f9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623357b9f60 0x5623357e59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:44806/0 (socket says 172.21.15.133:44806) 2023-07-14T18:26:59.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.161+0000 7f6050f9a700 1 -- 172.21.15.133:0/202144067 learned_addr learned my addr 172.21.15.133:0/202144067 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:26:59.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.161+0000 7f6050f9a700 1 --2- 172.21.15.133:0/202144067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623357bb4b0 0x5623357ecf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:26:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.161+0000 7f6050f9a700 1 -- 172.21.15.133:0/202144067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623357bb4b0 msgr2=0x5623357ecf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:26:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.161+0000 7f6050f9a700 1 --2- 172.21.15.133:0/202144067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623357bb4b0 0x5623357ecf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.161+0000 7f6050f9a700 1 -- 172.21.15.133:0/202144067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623357ba8b0 msgr2=0x5623357e5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:26:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.161+0000 7f6050f9a700 1 --2- 172.21.15.133:0/202144067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623357ba8b0 0x5623357e5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.161+0000 7f6050f9a700 1 -- 172.21.15.133:0/202144067 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603c004310 con 0x5623357b9f60 2023-07-14T18:26:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.161+0000 7f60427fc700 1 --2- 172.21.15.133:0/2386714292 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f60380a1260 0x7f60380a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.162+0000 7f6050f9a700 1 --2- 172.21.15.133:0/202144067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623357b9f60 0x5623357e59d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6044009f60 tx=0x7f6044005ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c67549e762ce649 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:26:59.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.162+0000 7f6034ff9700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:26:59.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.162+0000 7f60427fc700 1 -- 172.21.15.133:0/2386714292 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f60480ee410 con 0x7f604c10be90 2023-07-14T18:26:59.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.167+0000 7f6034ff9700 1 -- 172.21.15.133:0/202144067 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604400e040 con 0x5623357b9f60 2023-07-14T18:26:59.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.167+0000 7f6034ff9700 1 -- 172.21.15.133:0/202144067 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6044004c90 con 0x5623357b9f60 2023-07-14T18:26:59.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.167+0000 7f60672beb80 1 -- 172.21.15.133:0/202144067 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5623356644f0 con 0x5623357b9f60 2023-07-14T18:26:59.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.168+0000 7f6034ff9700 1 -- 172.21.15.133:0/202144067 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6044004e50 con 0x5623357b9f60 2023-07-14T18:26:59.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.168+0000 7f60672beb80 10 client.25883 fetch_fsmap learned FSMap version 38 2023-07-14T18:26:59.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.168+0000 7f60672beb80 1 -- 172.21.15.133:0/202144067 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x562335521c60 con 0x5623357b9f60 2023-07-14T18:26:59.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.168+0000 7f6034ff9700 1 -- 172.21.15.133:0/202144067 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5623356644f0 con 0x5623357b9f60 2023-07-14T18:26:59.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.168+0000 7f6034ff9700 1 -- 172.21.15.133:0/202144067 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6044019040 con 0x5623357b9f60 2023-07-14T18:26:59.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.169+0000 7f60672beb80 10 client.25883 fetch_fsmap finished waiting for FSMap version 38 2023-07-14T18:26:59.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.169+0000 7f60672beb80 10 client.25883 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:26:59.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.169+0000 7f60672beb80 20 client.25883 populate_metadata read hostname 'smithi133' 2023-07-14T18:26:59.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.169+0000 7f60672beb80 1 --2- 172.21.15.133:0/202144067 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5623357f1750 0x5623357f3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:59.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.169+0000 7f60672beb80 4 client.25883 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:26:59.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.169+0000 7f60672beb80 1 -- 172.21.15.133:0/202144067 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562335788da0 con 0x5623357f1750 2023-07-14T18:26:59.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.171+0000 7f605179b700 1 --2- 172.21.15.133:0/2386714292 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f60380a1260 0x7f60380a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:26:59.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.171+0000 7f6050f9a700 1 --2- 172.21.15.133:0/202144067 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5623357f1750 0x5623357f3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:26:59.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.172+0000 7f605179b700 1 --2- 172.21.15.133:0/2386714292 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f60380a1260 0x7f60380a3710 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f603c01ce80 tx=0x7f603c01c490 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:26:59.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.172+0000 7f6050f9a700 1 --2- 172.21.15.133:0/202144067 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5623357f1750 0x5623357f3b60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8e93466093cc678 server_cookie=46c7368a667ff2bc in_seq=0 out_seq=0 2023-07-14T18:26:59.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.173+0000 7f6034ff9700 10 client.25883 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:26:59.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.183+0000 7f6034ff9700 1 -- 172.21.15.133:0/202144067 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x562335788da0 con 0x5623357f1750 2023-07-14T18:26:59.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.183+0000 7f6034ff9700 10 client.25883 handle_command_reply: tid=0 2023-07-14T18:26:59.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.183+0000 7f60672beb80 1 client.25883 shutdown 2023-07-14T18:26:59.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.188+0000 7f60672beb80 1 -- 172.21.15.133:0/202144067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623357b9f60 msgr2=0x5623357e59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:26:59.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.188+0000 7f60672beb80 1 --2- 172.21.15.133:0/202144067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623357b9f60 0x5623357e59d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6044009f60 tx=0x7f6044005ae0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.188+0000 7f60672beb80 1 -- 172.21.15.133:0/202144067 shutdown_connections 2023-07-14T18:26:59.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.188+0000 7f60672beb80 1 --2- 172.21.15.133:0/202144067 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5623357f1750 0x5623357f3b60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.188+0000 7f60672beb80 1 --2- 172.21.15.133:0/202144067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623357b9f60 0x5623357e59d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.188+0000 7f60672beb80 1 --2- 172.21.15.133:0/202144067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623357bb4b0 0x5623357ecf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.188+0000 7f60672beb80 1 --2- 172.21.15.133:0/202144067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623357ba8b0 0x5623357e5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.188+0000 7f60672beb80 1 -- 172.21.15.133:0/202144067 >> 172.21.15.133:0/202144067 conn(0x5623357af680 msgr2=0x5623357b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:26:59.234 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.188+0000 7f6034ff9700 0 client.25883 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:26:59.234 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.188+0000 7f60672beb80 1 -- 172.21.15.133:0/202144067 shutdown_connections 2023-07-14T18:26:59.234 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.189+0000 7f60672beb80 1 -- 172.21.15.133:0/202144067 wait complete. 2023-07-14T18:26:59.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.189+0000 7f60672beb80 20 client.25883 trim_cache size 0 max 16384 2023-07-14T18:26:59.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.220+0000 7f60672beb80 1 Processor -- start 2023-07-14T18:26:59.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.220+0000 7f60672beb80 1 -- start start 2023-07-14T18:26:59.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.221+0000 7f60672beb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623357b9f60 0x5623357e5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:59.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.221+0000 7f60672beb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623357ba8b0 0x5623357ecf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:59.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.221+0000 7f60672beb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623357bb4b0 0x5623357c1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:59.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.221+0000 7f60672beb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623357b5d00 con 0x5623357ba8b0 2023-07-14T18:26:59.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.221+0000 7f60672beb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623357e4eb0 con 0x5623357bb4b0 2023-07-14T18:26:59.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.221+0000 7f60672beb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623357e6bf0 con 0x5623357b9f60 2023-07-14T18:26:59.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.221+0000 7f6050f9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623357ba8b0 0x5623357ecf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:26:59.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.222+0000 7f6050f9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623357ba8b0 0x5623357ecf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33180/0 (socket says 172.21.15.133:33180) 2023-07-14T18:26:59.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.222+0000 7f6050f9a700 1 -- 172.21.15.133:0/1210516862 learned_addr learned my addr 172.21.15.133:0/1210516862 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:26:59.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.222+0000 7f6051f9c700 1 --2- 172.21.15.133:0/1210516862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623357bb4b0 0x5623357c1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:26:59.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.222+0000 7f6050f9a700 1 -- 172.21.15.133:0/1210516862 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623357b9f60 msgr2=0x5623357e5080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T18:26:59.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.222+0000 7f6050f9a700 1 --2- 172.21.15.133:0/1210516862 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623357b9f60 0x5623357e5080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.222+0000 7f6050f9a700 1 -- 172.21.15.133:0/1210516862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623357bb4b0 msgr2=0x5623357c1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:26:59.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.222+0000 7f6050f9a700 1 --2- 172.21.15.133:0/1210516862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623357bb4b0 0x5623357c1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.222+0000 7f6050f9a700 1 -- 172.21.15.133:0/1210516862 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56233555f250 con 0x5623357ba8b0 2023-07-14T18:26:59.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.223+0000 7f6050f9a700 1 --2- 172.21.15.133:0/1210516862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623357ba8b0 0x5623357ecf10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6044004590 tx=0x7f604401b3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad4b2bd38f310dd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:26:59.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.228+0000 7f6037fff700 1 -- 172.21.15.133:0/1210516862 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604400e070 con 0x5623357ba8b0 2023-07-14T18:26:59.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.228+0000 7f6037fff700 1 -- 172.21.15.133:0/1210516862 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f604401d460 con 0x5623357ba8b0 2023-07-14T18:26:59.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.228+0000 7f6037fff700 1 -- 172.21.15.133:0/1210516862 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604401d660 con 0x5623357ba8b0 2023-07-14T18:26:59.272 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.230+0000 7f60672beb80 1 -- 172.21.15.133:0/1210516862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623357ba8b0 msgr2=0x5623357ecf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:26:59.272 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.230+0000 7f60672beb80 1 --2- 172.21.15.133:0/1210516862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623357ba8b0 0x5623357ecf10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6044004590 tx=0x7f604401b3d0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.231+0000 7f60672beb80 1 -- 172.21.15.133:0/1210516862 shutdown_connections 2023-07-14T18:26:59.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.231+0000 7f60672beb80 1 --2- 172.21.15.133:0/1210516862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623357bb4b0 0x5623357c1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.231+0000 7f60672beb80 1 --2- 172.21.15.133:0/1210516862 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623357b9f60 0x5623357e5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.231+0000 7f60672beb80 1 --2- 172.21.15.133:0/1210516862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623357ba8b0 0x5623357ecf10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.232+0000 7f60672beb80 1 -- 172.21.15.133:0/1210516862 >> 172.21.15.133:0/1210516862 conn(0x5623357af680 msgr2=0x5623357e46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:26:59.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.245+0000 7f60672beb80 1 -- 172.21.15.133:0/1210516862 shutdown_connections 2023-07-14T18:26:59.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.246+0000 7f60672beb80 1 -- 172.21.15.133:0/1210516862 wait complete. 2023-07-14T18:26:59.296 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.247+0000 7f60672beb80 1 -- start start 2023-07-14T18:26:59.296 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.247+0000 7f60672beb80 1 Processor -- start 2023-07-14T18:26:59.296 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.248+0000 7f60672beb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623357b9f60 0x5623357e9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:59.296 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.248+0000 7f60672beb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623357ba8b0 0x5623357e96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:59.296 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.248+0000 7f60672beb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623357bb4b0 0x5623357ec690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:59.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.248+0000 7f60672beb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623357ecbd0 con 0x5623357b9f60 2023-07-14T18:26:59.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.248+0000 7f60672beb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623357ecd40 con 0x5623357bb4b0 2023-07-14T18:26:59.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.248+0000 7f60672beb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623357ecf00 con 0x5623357ba8b0 2023-07-14T18:26:59.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.248+0000 7f60672beb80 10 client.0 _ll_register_callbacks cb 0x7f604400bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:26:59.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.249+0000 7f6051f9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623357bb4b0 0x5623357ec690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:26:59.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.249+0000 7f6051f9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623357bb4b0 0x5623357ec690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:44836/0 (socket says 172.21.15.133:44836) 2023-07-14T18:26:59.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.249+0000 7f6051f9c700 1 -- 172.21.15.133:0/2196489088 learned_addr learned my addr 172.21.15.133:0/2196489088 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:26:59.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.249+0000 7f6050f9a700 1 --2- 172.21.15.133:0/2196489088 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623357ba8b0 0x5623357e96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:26:59.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.249+0000 7f6050f9a700 1 -- 172.21.15.133:0/2196489088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623357bb4b0 msgr2=0x5623357ec690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:26:59.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.249+0000 7f6050f9a700 1 --2- 172.21.15.133:0/2196489088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623357bb4b0 0x5623357ec690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.249+0000 7f6050f9a700 1 -- 172.21.15.133:0/2196489088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623357b9f60 msgr2=0x5623357e9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:26:59.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.249+0000 7f6050f9a700 1 --2- 172.21.15.133:0/2196489088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623357b9f60 0x5623357e9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.249+0000 7f6050f9a700 1 -- 172.21.15.133:0/2196489088 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562335521c60 con 0x5623357ba8b0 2023-07-14T18:26:59.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.250+0000 7f6050f9a700 1 --2- 172.21.15.133:0/2196489088 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623357ba8b0 0x5623357e96a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x5623357b5a10 tx=0x7f6044009460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=583642b443d306c2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:26:59.299 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.250+0000 7f6034ff9700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:26:59.300 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.257+0000 7f6034ff9700 1 -- 172.21.15.133:0/2196489088 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604400e070 con 0x5623357ba8b0 2023-07-14T18:26:59.300 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.257+0000 7f6034ff9700 1 -- 172.21.15.133:0/2196489088 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f604401cd80 con 0x5623357ba8b0 2023-07-14T18:26:59.300 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.257+0000 7f60672beb80 1 -- 172.21.15.133:0/2196489088 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562335703eb0 con 0x5623357ba8b0 2023-07-14T18:26:59.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.264+0000 7f6034ff9700 1 -- 172.21.15.133:0/2196489088 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6044009ac0 con 0x5623357ba8b0 2023-07-14T18:26:59.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.264+0000 7f60672beb80 10 client.25930 fetch_fsmap learned FSMap version 38 2023-07-14T18:26:59.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.264+0000 7f60672beb80 1 -- 172.21.15.133:0/2196489088 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5623355372b0 con 0x5623357ba8b0 2023-07-14T18:26:59.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.267+0000 7f6034ff9700 1 -- 172.21.15.133:0/2196489088 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562335703eb0 con 0x5623357ba8b0 2023-07-14T18:26:59.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.267+0000 7f6034ff9700 1 -- 172.21.15.133:0/2196489088 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6044019070 con 0x5623357ba8b0 2023-07-14T18:26:59.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.267+0000 7f60672beb80 10 client.25930 fetch_fsmap finished waiting for FSMap version 38 2023-07-14T18:26:59.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.267+0000 7f60672beb80 10 client.25930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:26:59.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.267+0000 7f60672beb80 20 client.25930 populate_metadata read hostname 'smithi133' 2023-07-14T18:26:59.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.267+0000 7f60672beb80 1 --2- 172.21.15.133:0/2196489088 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x562335846350 0x562335848730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:26:59.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.268+0000 7f60672beb80 4 client.25930 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:26:59.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.268+0000 7f60672beb80 1 -- 172.21.15.133:0/2196489088 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5623356c6050 con 0x562335846350 2023-07-14T18:26:59.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.268+0000 7f605179b700 1 --2- 172.21.15.133:0/2196489088 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x562335846350 0x562335848730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:26:59.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.269+0000 7f605179b700 1 --2- 172.21.15.133:0/2196489088 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x562335846350 0x562335848730 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74b409ebdd20622c server_cookie=fb6e1c5e865c62db in_seq=0 out_seq=0 2023-07-14T18:26:59.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.270+0000 7f6034ff9700 10 client.25930 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:26:59.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.270+0000 7f6034ff9700 1 -- 172.21.15.133:0/2196489088 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5623356c6050 con 0x562335846350 2023-07-14T18:26:59.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.270+0000 7f6034ff9700 10 client.25930 handle_command_reply: tid=0 2023-07-14T18:26:59.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.270+0000 7f60672beb80 1 client.25930 shutdown 2023-07-14T18:26:59.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.276+0000 7f60672beb80 1 -- 172.21.15.133:0/2196489088 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623357ba8b0 msgr2=0x5623357e96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:26:59.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.276+0000 7f60672beb80 1 --2- 172.21.15.133:0/2196489088 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623357ba8b0 0x5623357e96a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x5623357b5a10 tx=0x7f6044009460 comp rx=0 tx=0).stop 2023-07-14T18:26:59.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.277+0000 7f60672beb80 1 -- 172.21.15.133:0/2196489088 shutdown_connections 2023-07-14T18:26:59.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.277+0000 7f60672beb80 1 --2- 172.21.15.133:0/2196489088 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x562335846350 0x562335848730 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.277+0000 7f60672beb80 1 --2- 172.21.15.133:0/2196489088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623357bb4b0 0x5623357ec690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.277+0000 7f60672beb80 1 --2- 172.21.15.133:0/2196489088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623357b9f60 0x5623357e9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.277+0000 7f60672beb80 1 --2- 172.21.15.133:0/2196489088 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623357ba8b0 0x5623357e96a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.277+0000 7f60672beb80 1 -- 172.21.15.133:0/2196489088 >> 172.21.15.133:0/2196489088 conn(0x5623357af680 msgr2=0x5623357e46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:26:59.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.277+0000 7f6034ff9700 0 client.25930 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:26:59.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.277+0000 7f60672beb80 1 -- 172.21.15.133:0/2196489088 shutdown_connections 2023-07-14T18:26:59.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.282+0000 7f60672beb80 1 -- 172.21.15.133:0/2196489088 wait complete. 2023-07-14T18:26:59.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.282+0000 7f60672beb80 20 client.25930 trim_cache size 0 max 16384 2023-07-14T18:26:59.325 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:26:59.326 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T18:26:59.326 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:26:59.326 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:26:59.326 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:26:59.326 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:26:59.326 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:26:59.327 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:26:59.327 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:26:59.327 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:26:59.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.286+0000 7f60539ff700 1 -- 172.21.15.133:0/2386714292 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f60380a1260 msgr2=0x7f60380a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:26:59.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.286+0000 7f60539ff700 1 --2- 172.21.15.133:0/2386714292 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f60380a1260 0x7f60380a3710 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f603c01ce80 tx=0x7f603c01c490 comp rx=0 tx=0).stop 2023-07-14T18:26:59.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.286+0000 7f60539ff700 1 -- 172.21.15.133:0/2386714292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f604c10be90 msgr2=0x7f604c1a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:26:59.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.286+0000 7f60539ff700 1 --2- 172.21.15.133:0/2386714292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f604c10be90 0x7f604c1a3750 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6048009f60 tx=0x7f6048009f90 comp rx=0 tx=0).stop 2023-07-14T18:26:59.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.288+0000 7f60539ff700 1 -- 172.21.15.133:0/2386714292 shutdown_connections 2023-07-14T18:26:59.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.288+0000 7f60539ff700 1 --2- 172.21.15.133:0/2386714292 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f60380a1260 0x7f60380a3710 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.288+0000 7f60539ff700 1 --2- 172.21.15.133:0/2386714292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f604c10be90 0x7f604c1a3750 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.288+0000 7f60539ff700 1 --2- 172.21.15.133:0/2386714292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f604c071010 0x7f604c19c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.288+0000 7f60539ff700 1 --2- 172.21.15.133:0/2386714292 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f604c071960 0x7f604c19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:26:59.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.288+0000 7f60539ff700 1 -- 172.21.15.133:0/2386714292 >> 172.21.15.133:0/2386714292 conn(0x7f604c06cbc0 msgr2=0x7f604c06dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:26:59.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.295+0000 7f60539ff700 1 -- 172.21.15.133:0/2386714292 shutdown_connections 2023-07-14T18:26:59.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:26:59.297+0000 7f60539ff700 1 -- 172.21.15.133:0/2386714292 wait complete. 2023-07-14T18:26:59.407 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:27:00.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:00 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:27:00.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:00 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:27:00.620 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:00 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:27:00.879 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 455.0 s, 564.1 tps, lat 175.070 ms stddev 402.723 2023-07-14T18:27:01.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:01 smithi161 ceph-mon[128993]: pgmap v658: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 16 MiB/s wr, 1.06k op/s 2023-07-14T18:27:01.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:01 smithi171 ceph-mon[129241]: pgmap v658: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 16 MiB/s wr, 1.06k op/s 2023-07-14T18:27:01.661 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:01 smithi133 ceph-mon[126384]: pgmap v658: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 16 MiB/s wr, 1.06k op/s 2023-07-14T18:27:03.311 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:03 smithi133 ceph-mon[126384]: pgmap v659: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 17 MiB/s wr, 1.22k op/s 2023-07-14T18:27:03.334 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:03 smithi161 ceph-mon[128993]: pgmap v659: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 17 MiB/s wr, 1.22k op/s 2023-07-14T18:27:03.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:03 smithi171 ceph-mon[129241]: pgmap v659: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 17 MiB/s wr, 1.22k op/s 2023-07-14T18:27:05.098 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:04 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:27:05.098 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:04 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:27:05.099 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:04 smithi161 ceph-mon[128993]: pgmap v660: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 17 MiB/s wr, 1.21k op/s 2023-07-14T18:27:05.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:04 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:27:05.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:04 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:27:05.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:04 smithi171 ceph-mon[129241]: pgmap v660: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 17 MiB/s wr, 1.21k op/s 2023-07-14T18:27:05.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:04 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:27:05.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:04 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:27:05.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:04 smithi133 ceph-mon[126384]: pgmap v660: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 17 MiB/s wr, 1.21k op/s 2023-07-14T18:27:05.879 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 460.0 s, 539.6 tps, lat 190.459 ms stddev 312.805 2023-07-14T18:27:06.853 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:06 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:27:06.853 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:06 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:27:06.853 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:06 smithi161 ceph-mon[128993]: pgmap v661: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-07-14T18:27:06.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:06 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:27:06.864 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:06 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:27:06.864 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:06 smithi171 ceph-mon[129241]: pgmap v661: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-07-14T18:27:06.912 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:06 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:27:06.913 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:06 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:27:06.913 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:06 smithi133 ceph-mon[126384]: pgmap v661: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-07-14T18:27:09.371 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:09 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:27:09.371 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:09 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:27:09.371 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:09 smithi171 ceph-mon[129241]: pgmap v662: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-07-14T18:27:09.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:09 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:27:09.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:09 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:27:09.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:09 smithi133 ceph-mon[126384]: pgmap v662: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-07-14T18:27:09.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:09 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:27:09.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:09 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:27:09.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:09 smithi161 ceph-mon[128993]: pgmap v662: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-07-14T18:27:10.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:10 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:27:10.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:10 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:27:10.407 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:10 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:27:10.601 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:10 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:27:10.602 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:10 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:27:10.602 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:10 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:27:10.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:10 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:27:10.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:10 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:27:10.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:10 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:27:10.872 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 465.0 s, 797.2 tps, lat 127.375 ms stddev 121.325 2023-07-14T18:27:11.607 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:11 smithi161 ceph-mon[128993]: pgmap v663: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 14 MiB/s wr, 1.01k op/s 2023-07-14T18:27:11.618 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:11 smithi171 ceph-mon[129241]: pgmap v663: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 14 MiB/s wr, 1.01k op/s 2023-07-14T18:27:11.654 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:11 smithi133 ceph-mon[126384]: pgmap v663: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 14 MiB/s wr, 1.01k op/s 2023-07-14T18:27:13.599 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:13 smithi161 ceph-mon[128993]: pgmap v664: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 13 MiB/s wr, 911 op/s 2023-07-14T18:27:13.618 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:13 smithi171 ceph-mon[129241]: pgmap v664: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 13 MiB/s wr, 911 op/s 2023-07-14T18:27:13.663 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:13 smithi133 ceph-mon[126384]: pgmap v664: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 13 MiB/s wr, 911 op/s 2023-07-14T18:27:15.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:15 smithi161 ceph-mon[128993]: pgmap v665: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 678 op/s 2023-07-14T18:27:15.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:15 smithi171 ceph-mon[129241]: pgmap v665: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 678 op/s 2023-07-14T18:27:15.657 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:15 smithi133 ceph-mon[126384]: pgmap v665: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 678 op/s 2023-07-14T18:27:15.874 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 470.0 s, 626.0 tps, lat 126.666 ms stddev 234.411 2023-07-14T18:27:17.115 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:16 smithi171 ceph-mon[129241]: pgmap v666: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 14 MiB/s wr, 822 op/s 2023-07-14T18:27:17.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:16 smithi133 ceph-mon[126384]: pgmap v666: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 14 MiB/s wr, 822 op/s 2023-07-14T18:27:17.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:16 smithi161 ceph-mon[128993]: pgmap v666: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 14 MiB/s wr, 822 op/s 2023-07-14T18:27:19.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:19 smithi161 ceph-mon[128993]: pgmap v667: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 18 MiB/s wr, 1.20k op/s 2023-07-14T18:27:19.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:19 smithi171 ceph-mon[129241]: pgmap v667: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 18 MiB/s wr, 1.20k op/s 2023-07-14T18:27:19.654 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:19 smithi133 ceph-mon[126384]: pgmap v667: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 18 MiB/s wr, 1.20k op/s 2023-07-14T18:27:20.873 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 475.0 s, 610.9 tps, lat 192.879 ms stddev 430.211 2023-07-14T18:27:21.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:21 smithi171 ceph-mon[129241]: pgmap v668: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 17 MiB/s wr, 1.24k op/s 2023-07-14T18:27:21.653 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:21 smithi133 ceph-mon[126384]: pgmap v668: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 17 MiB/s wr, 1.24k op/s 2023-07-14T18:27:21.851 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:21 smithi161 ceph-mon[128993]: pgmap v668: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 17 MiB/s wr, 1.24k op/s 2023-07-14T18:27:22.655 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:22 smithi133 ceph-mon[126384]: pgmap v669: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 20 MiB/s wr, 1.48k op/s 2023-07-14T18:27:22.848 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:22 smithi161 ceph-mon[128993]: pgmap v669: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 20 MiB/s wr, 1.48k op/s 2023-07-14T18:27:22.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:22 smithi171 ceph-mon[129241]: pgmap v669: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 20 MiB/s wr, 1.48k op/s 2023-07-14T18:27:25.653 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:25 smithi133 ceph-mon[126384]: pgmap v670: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 17 MiB/s wr, 1.33k op/s 2023-07-14T18:27:25.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:25 smithi161 ceph-mon[128993]: pgmap v670: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 17 MiB/s wr, 1.33k op/s 2023-07-14T18:27:25.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:25 smithi171 ceph-mon[129241]: pgmap v670: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 17 MiB/s wr, 1.33k op/s 2023-07-14T18:27:25.873 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 480.0 s, 775.8 tps, lat 83.506 ms stddev 79.749 2023-07-14T18:27:27.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:26 smithi161 ceph-mon[128993]: pgmap v671: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 19 MiB/s wr, 1.51k op/s 2023-07-14T18:27:27.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:26 smithi171 ceph-mon[129241]: pgmap v671: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 19 MiB/s wr, 1.51k op/s 2023-07-14T18:27:27.412 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:26 smithi133 ceph-mon[126384]: pgmap v671: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 19 MiB/s wr, 1.51k op/s 2023-07-14T18:27:29.408 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:27:29.682 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:29 smithi133 ceph-mon[126384]: pgmap v672: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 19 MiB/s wr, 1.59k op/s 2023-07-14T18:27:29.850 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:29 smithi161 ceph-mon[128993]: pgmap v672: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 19 MiB/s wr, 1.59k op/s 2023-07-14T18:27:29.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:29 smithi171 ceph-mon[129241]: pgmap v672: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 19 MiB/s wr, 1.59k op/s 2023-07-14T18:27:30.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:29.956+0000 7f51c03ff700 1 -- 172.21.15.133:0/1257312561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51b8071960 msgr2=0x7f51b810b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:27:30.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:29.956+0000 7f51c03ff700 1 --2- 172.21.15.133:0/1257312561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51b8071960 0x7f51b810b900 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f51a801a9a0 tx=0x7f51a801acb0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:29.959+0000 7f51c03ff700 1 -- 172.21.15.133:0/1257312561 shutdown_connections 2023-07-14T18:27:30.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:29.959+0000 7f51c03ff700 1 --2- 172.21.15.133:0/1257312561 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51b810be90 0x7f51b810e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:29.959+0000 7f51c03ff700 1 --2- 172.21.15.133:0/1257312561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51b8071010 0x7f51b8071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:29.959+0000 7f51c03ff700 1 --2- 172.21.15.133:0/1257312561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51b8071960 0x7f51b810b900 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:29.959+0000 7f51c03ff700 1 -- 172.21.15.133:0/1257312561 >> 172.21.15.133:0/1257312561 conn(0x7f51b806cbc0 msgr2=0x7f51b806eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:27:30.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:29.980+0000 7f51c03ff700 1 -- 172.21.15.133:0/1257312561 shutdown_connections 2023-07-14T18:27:30.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:29.982+0000 7f51c03ff700 1 -- 172.21.15.133:0/1257312561 wait complete. 2023-07-14T18:27:30.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:29.984+0000 7f51c03ff700 1 Processor -- start 2023-07-14T18:27:30.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:29.984+0000 7f51c03ff700 1 -- start start 2023-07-14T18:27:30.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:29.984+0000 7f51c03ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51b8071010 0x7f51b819e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:27:30.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:29.984+0000 7f51c03ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51b8071960 0x7f51b81a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:27:30.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:29.984+0000 7f51c03ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51b810be90 0x7f51b81a3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:27:30.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:29.984+0000 7f51c03ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b81a3b50 con 0x7f51b8071960 2023-07-14T18:27:30.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:29.984+0000 7f51c03ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b81a3cc0 con 0x7f51b810be90 2023-07-14T18:27:30.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:29.984+0000 7f51c03ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b81a3e80 con 0x7f51b8071010 2023-07-14T18:27:30.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:29.987+0000 7f51be99c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51b810be90 0x7f51b81a3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:27:30.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:29.987+0000 7f51be99c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51b810be90 0x7f51b81a3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:59594/0 (socket says 172.21.15.133:59594) 2023-07-14T18:27:30.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:29.987+0000 7f51be99c700 1 -- 172.21.15.133:0/2920399785 learned_addr learned my addr 172.21.15.133:0/2920399785 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:27:30.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:29.987+0000 7f51be19b700 1 --2- 172.21.15.133:0/2920399785 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51b8071010 0x7f51b819e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:27:30.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:29.992+0000 7f51be99c700 1 -- 172.21.15.133:0/2920399785 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51b8071010 msgr2=0x7f51b819e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:27:30.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:29.992+0000 7f51be99c700 1 --2- 172.21.15.133:0/2920399785 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51b8071010 0x7f51b819e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:29.992+0000 7f51be99c700 1 -- 172.21.15.133:0/2920399785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51b8071960 msgr2=0x7f51b81a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:27:30.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:29.992+0000 7f51be99c700 1 --2- 172.21.15.133:0/2920399785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51b8071960 0x7f51b81a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:29.992+0000 7f51be99c700 1 -- 172.21.15.133:0/2920399785 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a801a600 con 0x7f51b810be90 2023-07-14T18:27:30.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.004+0000 7f51be99c700 1 --2- 172.21.15.133:0/2920399785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51b810be90 0x7f51b81a3380 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f51b0007e60 tx=0x7f51b000c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:27:30.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.016+0000 7f51af7fe700 1 -- 172.21.15.133:0/2920399785 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b000a0d0 con 0x7f51b810be90 2023-07-14T18:27:30.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.016+0000 7f51af7fe700 1 -- 172.21.15.133:0/2920399785 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51b0031070 con 0x7f51b810be90 2023-07-14T18:27:30.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.016+0000 7f51af7fe700 1 -- 172.21.15.133:0/2920399785 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b0005460 con 0x7f51b810be90 2023-07-14T18:27:30.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.016+0000 7f51c03ff700 1 -- 172.21.15.133:0/2920399785 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b81a4160 con 0x7f51b810be90 2023-07-14T18:27:30.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.016+0000 7f51c03ff700 1 -- 172.21.15.133:0/2920399785 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b81a46a0 con 0x7f51b810be90 2023-07-14T18:27:30.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.022+0000 7f51d3cd5b80 1 Processor -- start 2023-07-14T18:27:30.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.032+0000 7f51d3cd5b80 1 -- start start 2023-07-14T18:27:30.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.033+0000 7f51af7fe700 1 -- 172.21.15.133:0/2920399785 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f51b0004ad0 con 0x7f51b810be90 2023-07-14T18:27:30.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.033+0000 7f51d3cd5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4a654f60 0x55ee4a655370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:27:30.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.033+0000 7f51d3cd5b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee4a6558b0 0x55ee4a655ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:27:30.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.033+0000 7f51d3cd5b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee4a6564b0 0x55ee4a656920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:27:30.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.033+0000 7f51d3cd5b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4a559cb0 con 0x55ee4a6558b0 2023-07-14T18:27:30.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.033+0000 7f51d3cd5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4a5aca40 con 0x55ee4a654f60 2023-07-14T18:27:30.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.033+0000 7f51d3cd5b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4a5a5ed0 con 0x55ee4a6564b0 2023-07-14T18:27:30.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.035+0000 7f51af7fe700 1 --2- 172.21.15.133:0/2920399785 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f51a40a1350 0x7f51a40a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:27:30.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.035+0000 7f51af7fe700 1 -- 172.21.15.133:0/2920399785 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f51b00edd50 con 0x7f51b810be90 2023-07-14T18:27:30.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.036+0000 7f51be19b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee4a6558b0 0x55ee4a655ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:27:30.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.036+0000 7f51be19b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee4a6558b0 0x55ee4a655ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:53736/0 (socket says 172.21.15.133:53736) 2023-07-14T18:27:30.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.036+0000 7f51be19b700 1 -- 172.21.15.133:0/2256105355 learned_addr learned my addr 172.21.15.133:0/2256105355 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:27:30.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.037+0000 7f51be99c700 1 --2- 172.21.15.133:0/2920399785 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f51a40a1350 0x7f51a40a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:27:30.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.046+0000 7f51bd99a700 1 --2- 172.21.15.133:0/2256105355 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee4a6564b0 0x55ee4a656920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:27:30.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.046+0000 7f51bd99a700 1 --2- 172.21.15.133:0/2256105355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4a654f60 0x55ee4a655370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:27:30.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.051+0000 7f51be99c700 1 --2- 172.21.15.133:0/2920399785 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f51a40a1350 0x7f51a40a3800 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f51b00c2cf0 tx=0x7f51b00b69f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:27:30.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.051+0000 7f51bd99a700 1 -- 172.21.15.133:0/2256105355 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee4a6564b0 msgr2=0x55ee4a656920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:27:30.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.051+0000 7f51bd99a700 1 --2- 172.21.15.133:0/2256105355 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee4a6564b0 0x55ee4a656920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.051+0000 7f51bd99a700 1 -- 172.21.15.133:0/2256105355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee4a6558b0 msgr2=0x55ee4a655ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:27:30.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.051+0000 7f51bd99a700 1 --2- 172.21.15.133:0/2256105355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee4a6558b0 0x55ee4a655ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.051+0000 7f51bd99a700 1 -- 172.21.15.133:0/2256105355 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee4a3d3de0 con 0x55ee4a654f60 2023-07-14T18:27:30.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.058+0000 7f51bd99a700 1 --2- 172.21.15.133:0/2256105355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4a654f60 0x55ee4a655370 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f51a8003820 tx=0x7f51a8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1977adeb9ac12c2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:27:30.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.064+0000 7f51be19b700 1 --2- 172.21.15.133:0/2256105355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee4a6558b0 0x55ee4a655ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:27:30.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.065+0000 7f51acbfe700 1 -- 172.21.15.133:0/2256105355 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a801c9f0 con 0x55ee4a654f60 2023-07-14T18:27:30.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.065+0000 7f51acbfe700 1 -- 172.21.15.133:0/2256105355 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51a801ce00 con 0x55ee4a654f60 2023-07-14T18:27:30.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.065+0000 7f51acbfe700 1 -- 172.21.15.133:0/2256105355 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a800f6c0 con 0x55ee4a654f60 2023-07-14T18:27:30.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.066+0000 7f51d3cd5b80 1 -- 172.21.15.133:0/2256105355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4a654f60 msgr2=0x55ee4a655370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:27:30.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.066+0000 7f51d3cd5b80 1 --2- 172.21.15.133:0/2256105355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4a654f60 0x55ee4a655370 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f51a8003820 tx=0x7f51a8000bc0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.066+0000 7f51d3cd5b80 1 -- 172.21.15.133:0/2256105355 shutdown_connections 2023-07-14T18:27:30.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.066+0000 7f51d3cd5b80 1 --2- 172.21.15.133:0/2256105355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4a654f60 0x55ee4a655370 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.066+0000 7f51d3cd5b80 1 --2- 172.21.15.133:0/2256105355 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee4a6564b0 0x55ee4a656920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.066+0000 7f51d3cd5b80 1 --2- 172.21.15.133:0/2256105355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee4a6558b0 0x55ee4a655ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.066+0000 7f51d3cd5b80 1 -- 172.21.15.133:0/2256105355 >> 172.21.15.133:0/2256105355 conn(0x55ee4a64a680 msgr2=0x55ee4a64f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:27:30.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.067+0000 7f51d3cd5b80 1 -- 172.21.15.133:0/2256105355 shutdown_connections 2023-07-14T18:27:30.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.067+0000 7f51d3cd5b80 1 -- 172.21.15.133:0/2256105355 wait complete. 2023-07-14T18:27:30.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.068+0000 7f51d3cd5b80 1 -- start start 2023-07-14T18:27:30.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.068+0000 7f51d3cd5b80 1 Processor -- start 2023-07-14T18:27:30.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.069+0000 7f51d3cd5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4a654f60 0x55ee4a65e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:27:30.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.069+0000 7f51d3cd5b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee4a6558b0 0x55ee4a65ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:27:30.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.069+0000 7f51d3cd5b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee4a6564b0 0x55ee4a665f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:27:30.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.069+0000 7f51d3cd5b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4a545ac0 con 0x55ee4a6558b0 2023-07-14T18:27:30.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.069+0000 7f51d3cd5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4a5457c0 con 0x55ee4a654f60 2023-07-14T18:27:30.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.069+0000 7f51d3cd5b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4a429920 con 0x55ee4a6564b0 2023-07-14T18:27:30.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.069+0000 7f51d3cd5b80 10 client.0 _ll_register_callbacks cb 0x55ee4a543760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:27:30.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.070+0000 7f51be19b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee4a6558b0 0x55ee4a65ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:27:30.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.070+0000 7f51be19b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee4a6558b0 0x55ee4a65ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:53742/0 (socket says 172.21.15.133:53742) 2023-07-14T18:27:30.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.070+0000 7f51be19b700 1 -- 172.21.15.133:0/2966700482 learned_addr learned my addr 172.21.15.133:0/2966700482 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:27:30.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.070+0000 7f51bd99a700 1 --2- 172.21.15.133:0/2966700482 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee4a6564b0 0x55ee4a665f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:27:30.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.070+0000 7f51bd99a700 1 --2- 172.21.15.133:0/2966700482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4a654f60 0x55ee4a65e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:27:30.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.070+0000 7f51bd99a700 1 -- 172.21.15.133:0/2966700482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4a654f60 msgr2=0x55ee4a65e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:27:30.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.070+0000 7f51bd99a700 1 --2- 172.21.15.133:0/2966700482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4a654f60 0x55ee4a65e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.070+0000 7f51bd99a700 1 -- 172.21.15.133:0/2966700482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee4a6558b0 msgr2=0x55ee4a65ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:27:30.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.070+0000 7f51bd99a700 1 --2- 172.21.15.133:0/2966700482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee4a6558b0 0x55ee4a65ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.070+0000 7f51bd99a700 1 -- 172.21.15.133:0/2966700482 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b40096f0 con 0x55ee4a6564b0 2023-07-14T18:27:30.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.071+0000 7f51bd99a700 1 --2- 172.21.15.133:0/2966700482 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee4a6564b0 0x55ee4a665f50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f51a8006010 tx=0x7f51a801cf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43f90693945aad75 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:27:30.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.071+0000 7f51a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:27:30.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.071+0000 7f51a1ffb700 1 -- 172.21.15.133:0/2966700482 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a800f8e0 con 0x55ee4a6564b0 2023-07-14T18:27:30.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.071+0000 7f51a1ffb700 1 -- 172.21.15.133:0/2966700482 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f51a800fa40 con 0x55ee4a6564b0 2023-07-14T18:27:30.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.072+0000 7f51d3cd5b80 1 -- 172.21.15.133:0/2966700482 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee4a4ff4f0 con 0x55ee4a6564b0 2023-07-14T18:27:30.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.078+0000 7f51a1ffb700 1 -- 172.21.15.133:0/2966700482 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51a800fc00 con 0x55ee4a6564b0 2023-07-14T18:27:30.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.078+0000 7f51d3cd5b80 10 client.25939 fetch_fsmap learned FSMap version 38 2023-07-14T18:27:30.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.078+0000 7f51d3cd5b80 1 -- 172.21.15.133:0/2966700482 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f51b8117770 con 0x55ee4a6564b0 2023-07-14T18:27:30.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.089+0000 7f51a1ffb700 1 -- 172.21.15.133:0/2966700482 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ee4a4ff4f0 con 0x55ee4a6564b0 2023-07-14T18:27:30.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.089+0000 7f51a1ffb700 1 -- 172.21.15.133:0/2966700482 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f51a800b030 con 0x55ee4a6564b0 2023-07-14T18:27:30.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.089+0000 7f51d3cd5b80 10 client.25939 fetch_fsmap finished waiting for FSMap version 38 2023-07-14T18:27:30.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.089+0000 7f51d3cd5b80 10 client.25939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:27:30.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.089+0000 7f51d3cd5b80 20 client.25939 populate_metadata read hostname 'smithi133' 2023-07-14T18:27:30.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.089+0000 7f51d3cd5b80 1 --2- 172.21.15.133:0/2966700482 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55ee4a66a260 0x55ee4a66c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:27:30.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.089+0000 7f51d3cd5b80 4 client.25939 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:27:30.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.089+0000 7f51d3cd5b80 1 -- 172.21.15.133:0/2966700482 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee4a623da0 con 0x55ee4a66a260 2023-07-14T18:27:30.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.090+0000 7f51be19b700 1 --2- 172.21.15.133:0/2966700482 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55ee4a66a260 0x55ee4a66c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:27:30.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.106+0000 7f51be19b700 1 --2- 172.21.15.133:0/2966700482 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55ee4a66a260 0x55ee4a66c670 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8061501ce84e288 server_cookie=24e0a1021b5339b4 in_seq=0 out_seq=0 2023-07-14T18:27:30.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.108+0000 7f51a1ffb700 10 client.25939 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:27:30.165 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.119+0000 7f51a1ffb700 1 -- 172.21.15.133:0/2966700482 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55ee4a623da0 con 0x55ee4a66a260 2023-07-14T18:27:30.168 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.119+0000 7f51a1ffb700 10 client.25939 handle_command_reply: tid=0 2023-07-14T18:27:30.168 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.119+0000 7f51d3cd5b80 1 client.25939 shutdown 2023-07-14T18:27:30.176 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.136+0000 7f51d3cd5b80 1 -- 172.21.15.133:0/2966700482 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee4a6564b0 msgr2=0x55ee4a665f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:27:30.177 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.136+0000 7f51d3cd5b80 1 --2- 172.21.15.133:0/2966700482 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee4a6564b0 0x55ee4a665f50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f51a8006010 tx=0x7f51a801cf70 comp rx=0 tx=0).stop 2023-07-14T18:27:30.177 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.136+0000 7f51d3cd5b80 1 -- 172.21.15.133:0/2966700482 shutdown_connections 2023-07-14T18:27:30.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.136+0000 7f51d3cd5b80 1 --2- 172.21.15.133:0/2966700482 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55ee4a66a260 0x55ee4a66c670 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.136+0000 7f51d3cd5b80 1 --2- 172.21.15.133:0/2966700482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4a654f60 0x55ee4a65e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.136+0000 7f51d3cd5b80 1 --2- 172.21.15.133:0/2966700482 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee4a6564b0 0x55ee4a665f50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.136+0000 7f51d3cd5b80 1 --2- 172.21.15.133:0/2966700482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee4a6558b0 0x55ee4a65ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.137+0000 7f51d3cd5b80 1 -- 172.21.15.133:0/2966700482 >> 172.21.15.133:0/2966700482 conn(0x55ee4a64a680 msgr2=0x55ee4a64e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:27:30.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.137+0000 7f51a1ffb700 0 client.25939 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:27:30.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.137+0000 7f51d3cd5b80 1 -- 172.21.15.133:0/2966700482 shutdown_connections 2023-07-14T18:27:30.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.137+0000 7f51d3cd5b80 1 -- 172.21.15.133:0/2966700482 wait complete. 2023-07-14T18:27:30.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.138+0000 7f51d3cd5b80 20 client.25939 trim_cache size 0 max 16384 2023-07-14T18:27:30.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.180+0000 7f51d3cd5b80 1 Processor -- start 2023-07-14T18:27:30.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.180+0000 7f51d3cd5b80 1 -- start start 2023-07-14T18:27:30.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.180+0000 7f51d3cd5b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee4a654f60 0x55ee4a65d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:27:30.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.180+0000 7f51d3cd5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4a6558b0 0x55ee4a665e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:27:30.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.180+0000 7f51d3cd5b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee4a6564b0 0x55ee4a65e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:27:30.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.180+0000 7f51d3cd5b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4a4ff4f0 con 0x55ee4a654f60 2023-07-14T18:27:30.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.180+0000 7f51d3cd5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4a65fbc0 con 0x55ee4a6558b0 2023-07-14T18:27:30.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.180+0000 7f51d3cd5b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4a6508b0 con 0x55ee4a6564b0 2023-07-14T18:27:30.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.182+0000 7f51be19b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4a6558b0 0x55ee4a665e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:27:30.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.182+0000 7f51be19b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4a6558b0 0x55ee4a665e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:59622/0 (socket says 172.21.15.133:59622) 2023-07-14T18:27:30.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.182+0000 7f51be19b700 1 -- 172.21.15.133:0/576670482 learned_addr learned my addr 172.21.15.133:0/576670482 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:27:30.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.183+0000 7f51bd99a700 1 --2- 172.21.15.133:0/576670482 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee4a6564b0 0x55ee4a65e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:27:30.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.183+0000 7f51be19b700 1 -- 172.21.15.133:0/576670482 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee4a6564b0 msgr2=0x55ee4a65e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:27:30.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.183+0000 7f51be19b700 1 --2- 172.21.15.133:0/576670482 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee4a6564b0 0x55ee4a65e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.183+0000 7f51be19b700 1 -- 172.21.15.133:0/576670482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee4a654f60 msgr2=0x55ee4a65d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:27:30.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.183+0000 7f51be19b700 1 --2- 172.21.15.133:0/576670482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee4a654f60 0x55ee4a65d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.183+0000 7f51be19b700 1 -- 172.21.15.133:0/576670482 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee4a3bcc60 con 0x55ee4a6558b0 2023-07-14T18:27:30.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.190+0000 7f51be19b700 1 --2- 172.21.15.133:0/576670482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4a6558b0 0x55ee4a665e70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f51b4009980 tx=0x7f51b4008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=116b81c70c7cedc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:27:30.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.192+0000 7f51ad3ff700 1 -- 172.21.15.133:0/576670482 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b401d310 con 0x55ee4a6558b0 2023-07-14T18:27:30.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.192+0000 7f51ad3ff700 1 -- 172.21.15.133:0/576670482 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51b401c040 con 0x55ee4a6558b0 2023-07-14T18:27:30.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.192+0000 7f51ad3ff700 1 -- 172.21.15.133:0/576670482 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b4015620 con 0x55ee4a6558b0 2023-07-14T18:27:30.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.193+0000 7f51d3cd5b80 1 -- 172.21.15.133:0/576670482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4a6558b0 msgr2=0x55ee4a665e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:27:30.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.193+0000 7f51d3cd5b80 1 --2- 172.21.15.133:0/576670482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4a6558b0 0x55ee4a665e70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f51b4009980 tx=0x7f51b4008040 comp rx=0 tx=0).stop 2023-07-14T18:27:30.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.194+0000 7f51d3cd5b80 1 -- 172.21.15.133:0/576670482 shutdown_connections 2023-07-14T18:27:30.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.194+0000 7f51d3cd5b80 1 --2- 172.21.15.133:0/576670482 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee4a6564b0 0x55ee4a65e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.194+0000 7f51d3cd5b80 1 --2- 172.21.15.133:0/576670482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee4a654f60 0x55ee4a65d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.194+0000 7f51d3cd5b80 1 --2- 172.21.15.133:0/576670482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4a6558b0 0x55ee4a665e70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.194+0000 7f51d3cd5b80 1 -- 172.21.15.133:0/576670482 >> 172.21.15.133:0/576670482 conn(0x55ee4a64a680 msgr2=0x55ee4a65c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:27:30.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.213+0000 7f51d3cd5b80 1 -- 172.21.15.133:0/576670482 shutdown_connections 2023-07-14T18:27:30.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.217+0000 7f51d3cd5b80 1 -- 172.21.15.133:0/576670482 wait complete. 2023-07-14T18:27:30.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.217+0000 7f51d3cd5b80 1 -- start start 2023-07-14T18:27:30.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.217+0000 7f51d3cd5b80 1 Processor -- start 2023-07-14T18:27:30.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.218+0000 7f51d3cd5b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee4a654f60 0x55ee4a6644a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:27:30.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.218+0000 7f51d3cd5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4a6558b0 0x55ee4a6649e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:27:30.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.218+0000 7f51d3cd5b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee4a6564b0 0x55ee4a664f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:27:30.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.218+0000 7f51d3cd5b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4a666120 con 0x55ee4a654f60 2023-07-14T18:27:30.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.218+0000 7f51d3cd5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4a6bdeb0 con 0x55ee4a6558b0 2023-07-14T18:27:30.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.218+0000 7f51d3cd5b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4a6bdff0 con 0x55ee4a6564b0 2023-07-14T18:27:30.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.218+0000 7f51d3cd5b80 10 client.0 _ll_register_callbacks cb 0x7f51a8017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:27:30.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.219+0000 7f51bd99a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee4a6564b0 0x55ee4a664f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:27:30.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.219+0000 7f51bd99a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee4a6564b0 0x55ee4a664f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:55656/0 (socket says 172.21.15.133:55656) 2023-07-14T18:27:30.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.219+0000 7f51bd99a700 1 -- 172.21.15.133:0/182877871 learned_addr learned my addr 172.21.15.133:0/182877871 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:27:30.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.219+0000 7f51be19b700 1 --2- 172.21.15.133:0/182877871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4a6558b0 0x55ee4a6649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:27:30.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.230+0000 7f51bd99a700 1 -- 172.21.15.133:0/182877871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4a6558b0 msgr2=0x55ee4a6649e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:27:30.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.230+0000 7f51bd99a700 1 --2- 172.21.15.133:0/182877871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4a6558b0 0x55ee4a6649e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.230+0000 7f51bd99a700 1 -- 172.21.15.133:0/182877871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee4a654f60 msgr2=0x55ee4a6644a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:27:30.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.230+0000 7f51bd99a700 1 --2- 172.21.15.133:0/182877871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee4a654f60 0x55ee4a6644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.230+0000 7f51bd99a700 1 -- 172.21.15.133:0/182877871 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b4009b00 con 0x55ee4a6564b0 2023-07-14T18:27:30.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.230+0000 7f51bd99a700 1 --2- 172.21.15.133:0/182877871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee4a6564b0 0x55ee4a664f20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f51a8006010 tx=0x7f51a8012660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba8b9111d796f1be server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:27:30.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.230+0000 7f51a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:27:30.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.232+0000 7f51a1ffb700 1 -- 172.21.15.133:0/182877871 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a80127d0 con 0x55ee4a6564b0 2023-07-14T18:27:30.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.232+0000 7f51a1ffb700 1 -- 172.21.15.133:0/182877871 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f51a8000e40 con 0x55ee4a6564b0 2023-07-14T18:27:30.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.232+0000 7f51d3cd5b80 1 -- 172.21.15.133:0/182877871 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51a801ce00 con 0x55ee4a6564b0 2023-07-14T18:27:30.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.233+0000 7f51a1ffb700 1 -- 172.21.15.133:0/182877871 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51a8012450 con 0x55ee4a6564b0 2023-07-14T18:27:30.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.240+0000 7f51d3cd5b80 10 client.25942 fetch_fsmap learned FSMap version 38 2023-07-14T18:27:30.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.240+0000 7f51d3cd5b80 1 -- 172.21.15.133:0/182877871 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55ee4a3fa250 con 0x55ee4a6564b0 2023-07-14T18:27:30.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.266+0000 7f51a1ffb700 1 -- 172.21.15.133:0/182877871 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51a801ce00 con 0x55ee4a6564b0 2023-07-14T18:27:30.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.266+0000 7f51a1ffb700 1 -- 172.21.15.133:0/182877871 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f51a800b070 con 0x55ee4a6564b0 2023-07-14T18:27:30.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.278+0000 7f51d3cd5b80 10 client.25942 fetch_fsmap finished waiting for FSMap version 38 2023-07-14T18:27:30.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.278+0000 7f51d3cd5b80 10 client.25942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:27:30.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.278+0000 7f51d3cd5b80 20 client.25942 populate_metadata read hostname 'smithi133' 2023-07-14T18:27:30.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.278+0000 7f51d3cd5b80 1 --2- 172.21.15.133:0/182877871 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55ee4a6bf570 0x55ee4a6c1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:27:30.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.278+0000 7f51d3cd5b80 4 client.25942 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:27:30.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.278+0000 7f51d3cd5b80 1 -- 172.21.15.133:0/182877871 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ee4a59eeb0 con 0x55ee4a6bf570 2023-07-14T18:27:30.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.279+0000 7f51be19b700 1 --2- 172.21.15.133:0/182877871 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55ee4a6bf570 0x55ee4a6c1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:27:30.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.291+0000 7f51be19b700 1 --2- 172.21.15.133:0/182877871 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55ee4a6bf570 0x55ee4a6c1980 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a17022720c0cfdbd server_cookie=c398e339c67b743e in_seq=0 out_seq=0 2023-07-14T18:27:30.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.297+0000 7f51a1ffb700 10 client.25942 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:27:30.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.297+0000 7f51a1ffb700 1 -- 172.21.15.133:0/182877871 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55ee4a59eeb0 con 0x55ee4a6bf570 2023-07-14T18:27:30.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.297+0000 7f51a1ffb700 10 client.25942 handle_command_reply: tid=0 2023-07-14T18:27:30.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.297+0000 7f51d3cd5b80 1 client.25942 shutdown 2023-07-14T18:27:30.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.299+0000 7f51d3cd5b80 1 -- 172.21.15.133:0/182877871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee4a6564b0 msgr2=0x55ee4a664f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:27:30.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.299+0000 7f51d3cd5b80 1 --2- 172.21.15.133:0/182877871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee4a6564b0 0x55ee4a664f20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f51a8006010 tx=0x7f51a8012660 comp rx=0 tx=0).stop 2023-07-14T18:27:30.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.300+0000 7f51d3cd5b80 1 -- 172.21.15.133:0/182877871 shutdown_connections 2023-07-14T18:27:30.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.300+0000 7f51d3cd5b80 1 --2- 172.21.15.133:0/182877871 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55ee4a6bf570 0x55ee4a6c1980 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.300+0000 7f51d3cd5b80 1 --2- 172.21.15.133:0/182877871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee4a6564b0 0x55ee4a664f20 secure :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f51a8006010 tx=0x7f51a8012660 comp rx=0 tx=0).stop 2023-07-14T18:27:30.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.300+0000 7f51d3cd5b80 1 --2- 172.21.15.133:0/182877871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee4a654f60 0x55ee4a6644a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.300+0000 7f51d3cd5b80 1 --2- 172.21.15.133:0/182877871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4a6558b0 0x55ee4a6649e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.300+0000 7f51d3cd5b80 1 -- 172.21.15.133:0/182877871 >> 172.21.15.133:0/182877871 conn(0x55ee4a64a680 msgr2=0x55ee4a65d670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:27:30.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.305+0000 7f51a1ffb700 0 client.25942 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:27:30.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.305+0000 7f51d3cd5b80 1 -- 172.21.15.133:0/182877871 shutdown_connections 2023-07-14T18:27:30.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.306+0000 7f51d3cd5b80 1 -- 172.21.15.133:0/182877871 wait complete. 2023-07-14T18:27:30.355 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.315+0000 7f51d3cd5b80 20 client.25942 trim_cache size 0 max 16384 2023-07-14T18:27:30.356 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:27:30.357 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T18:27:30.357 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:27:30.357 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:27:30.358 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:27:30.358 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:27:30.358 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:27:30.359 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:27:30.359 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:27:30.359 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:27:30.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.319+0000 7f51c03ff700 1 -- 172.21.15.133:0/2920399785 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f51a40a1350 msgr2=0x7f51a40a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:27:30.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.319+0000 7f51c03ff700 1 --2- 172.21.15.133:0/2920399785 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f51a40a1350 0x7f51a40a3800 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f51b00c2cf0 tx=0x7f51b00b69f0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.319+0000 7f51c03ff700 1 -- 172.21.15.133:0/2920399785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51b810be90 msgr2=0x7f51b81a3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:27:30.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.319+0000 7f51c03ff700 1 --2- 172.21.15.133:0/2920399785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51b810be90 0x7f51b81a3380 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f51b0007e60 tx=0x7f51b000c330 comp rx=0 tx=0).stop 2023-07-14T18:27:30.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.320+0000 7f51c03ff700 1 -- 172.21.15.133:0/2920399785 shutdown_connections 2023-07-14T18:27:30.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.320+0000 7f51c03ff700 1 --2- 172.21.15.133:0/2920399785 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f51a40a1350 0x7f51a40a3800 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.320+0000 7f51c03ff700 1 --2- 172.21.15.133:0/2920399785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51b810be90 0x7f51b81a3380 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.320+0000 7f51c03ff700 1 --2- 172.21.15.133:0/2920399785 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51b8071010 0x7f51b819e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.320+0000 7f51c03ff700 1 --2- 172.21.15.133:0/2920399785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51b8071960 0x7f51b81a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:27:30.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.320+0000 7f51c03ff700 1 -- 172.21.15.133:0/2920399785 >> 172.21.15.133:0/2920399785 conn(0x7f51b806cbc0 msgr2=0x7f51b806de60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:27:30.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.327+0000 7f51c03ff700 1 -- 172.21.15.133:0/2920399785 shutdown_connections 2023-07-14T18:27:30.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:27:30.327+0000 7f51c03ff700 1 -- 172.21.15.133:0/2920399785 wait complete. 2023-07-14T18:27:30.407 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:27:30.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:30 smithi161 ceph-mon[128993]: pgmap v673: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 17 MiB/s wr, 1.29k op/s 2023-07-14T18:27:30.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:30 smithi171 ceph-mon[129241]: pgmap v673: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 17 MiB/s wr, 1.29k op/s 2023-07-14T18:27:30.870 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 485.0 s, 553.3 tps, lat 232.321 ms stddev 503.481 2023-07-14T18:27:30.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:30 smithi133 ceph-mon[126384]: pgmap v673: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 17 MiB/s wr, 1.29k op/s 2023-07-14T18:27:33.598 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:33 smithi161 ceph-mon[128993]: pgmap v674: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 19 MiB/s wr, 1.42k op/s 2023-07-14T18:27:33.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:33 smithi171 ceph-mon[129241]: pgmap v674: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 19 MiB/s wr, 1.42k op/s 2023-07-14T18:27:33.659 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:33 smithi133 ceph-mon[126384]: pgmap v674: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 19 MiB/s wr, 1.42k op/s 2023-07-14T18:27:35.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:35 smithi161 ceph-mon[128993]: pgmap v675: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-07-14T18:27:35.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:35 smithi171 ceph-mon[129241]: pgmap v675: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-07-14T18:27:35.872 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 490.0 s, 408.8 tps, lat 262.119 ms stddev 226.516 2023-07-14T18:27:35.905 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:35 smithi133 ceph-mon[126384]: pgmap v675: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-07-14T18:27:37.354 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:36 smithi161 ceph-mon[128993]: pgmap v676: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 16 MiB/s wr, 1.19k op/s 2023-07-14T18:27:37.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:36 smithi171 ceph-mon[129241]: pgmap v676: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 16 MiB/s wr, 1.19k op/s 2023-07-14T18:27:37.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:36 smithi133 ceph-mon[126384]: pgmap v676: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 16 MiB/s wr, 1.19k op/s 2023-07-14T18:27:39.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:39 smithi161 ceph-mon[128993]: pgmap v677: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 15 MiB/s wr, 1.17k op/s 2023-07-14T18:27:39.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:39 smithi171 ceph-mon[129241]: pgmap v677: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 15 MiB/s wr, 1.17k op/s 2023-07-14T18:27:39.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:39 smithi133 ceph-mon[126384]: pgmap v677: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 15 MiB/s wr, 1.17k op/s 2023-07-14T18:27:40.874 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 495.0 s, 389.9 tps, lat 200.080 ms stddev 132.724 2023-07-14T18:27:41.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:41 smithi133 ceph-mon[126384]: pgmap v678: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 11 MiB/s wr, 898 op/s 2023-07-14T18:27:41.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:41 smithi161 ceph-mon[128993]: pgmap v678: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 11 MiB/s wr, 898 op/s 2023-07-14T18:27:41.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:41 smithi171 ceph-mon[129241]: pgmap v678: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 11 MiB/s wr, 898 op/s 2023-07-14T18:27:43.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:43 smithi161 ceph-mon[128993]: pgmap v679: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 13 MiB/s wr, 1.03k op/s 2023-07-14T18:27:43.869 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:43 smithi171 ceph-mon[129241]: pgmap v679: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 13 MiB/s wr, 1.03k op/s 2023-07-14T18:27:43.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:43 smithi133 ceph-mon[126384]: pgmap v679: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 13 MiB/s wr, 1.03k op/s 2023-07-14T18:27:44.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:44 smithi161 ceph-mon[128993]: pgmap v680: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.8 MiB/s wr, 704 op/s 2023-07-14T18:27:44.864 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:44 smithi171 ceph-mon[129241]: pgmap v680: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.8 MiB/s wr, 704 op/s 2023-07-14T18:27:44.905 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:44 smithi133 ceph-mon[126384]: pgmap v680: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.8 MiB/s wr, 704 op/s 2023-07-14T18:27:45.870 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 500.0 s, 502.3 tps, lat 246.125 ms stddev 356.347 2023-07-14T18:27:47.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:46 smithi133 ceph-mon[126384]: pgmap v681: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 822 op/s 2023-07-14T18:27:47.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:46 smithi161 ceph-mon[128993]: pgmap v681: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 822 op/s 2023-07-14T18:27:47.366 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:46 smithi171 ceph-mon[129241]: pgmap v681: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 822 op/s 2023-07-14T18:27:48.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:47 smithi161 ceph-mon[128993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:27:48.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:47 smithi171 ceph-mon[129241]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:27:48.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:47 smithi133 ceph-mon[126384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:27:49.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:48 smithi161 ceph-mon[128993]: pgmap v682: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 12 MiB/s wr, 862 op/s 2023-07-14T18:27:49.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:48 smithi171 ceph-mon[129241]: pgmap v682: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 12 MiB/s wr, 862 op/s 2023-07-14T18:27:49.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:48 smithi133 ceph-mon[126384]: pgmap v682: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 12 MiB/s wr, 862 op/s 2023-07-14T18:27:50.870 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 505.0 s, 564.2 tps, lat 109.886 ms stddev 99.356 2023-07-14T18:27:51.654 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:51 smithi133 ceph-mon[126384]: pgmap v683: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 12 MiB/s wr, 755 op/s 2023-07-14T18:27:51.850 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:51 smithi161 ceph-mon[128993]: pgmap v683: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 12 MiB/s wr, 755 op/s 2023-07-14T18:27:51.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:51 smithi171 ceph-mon[129241]: pgmap v683: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 12 MiB/s wr, 755 op/s 2023-07-14T18:27:53.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:52 smithi161 ceph-mon[128993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:27:53.348 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:52 smithi161 ceph-mon[128993]: pgmap v684: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 15 MiB/s wr, 1.03k op/s 2023-07-14T18:27:53.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:52 smithi171 ceph-mon[129241]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:27:53.362 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:52 smithi171 ceph-mon[129241]: pgmap v684: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 15 MiB/s wr, 1.03k op/s 2023-07-14T18:27:53.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:52 smithi133 ceph-mon[126384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:27:53.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:52 smithi133 ceph-mon[126384]: pgmap v684: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 15 MiB/s wr, 1.03k op/s 2023-07-14T18:27:55.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:55 smithi133 ceph-mon[126384]: pgmap v685: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 12 MiB/s wr, 788 op/s 2023-07-14T18:27:55.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:55 smithi161 ceph-mon[128993]: pgmap v685: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 12 MiB/s wr, 788 op/s 2023-07-14T18:27:55.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:55 smithi171 ceph-mon[129241]: pgmap v685: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 12 MiB/s wr, 788 op/s 2023-07-14T18:27:55.869 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 510.0 s, 631.9 tps, lat 218.544 ms stddev 435.538 2023-07-14T18:27:57.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:56 smithi161 ceph-mon[128993]: pgmap v686: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 13 MiB/s wr, 893 op/s 2023-07-14T18:27:57.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:56 smithi171 ceph-mon[129241]: pgmap v686: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 13 MiB/s wr, 893 op/s 2023-07-14T18:27:57.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:56 smithi133 ceph-mon[126384]: pgmap v686: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 13 MiB/s wr, 893 op/s 2023-07-14T18:27:59.653 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:27:59 smithi133 ceph-mon[126384]: pgmap v687: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 16 MiB/s wr, 997 op/s 2023-07-14T18:27:59.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:27:59 smithi161 ceph-mon[128993]: pgmap v687: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 16 MiB/s wr, 997 op/s 2023-07-14T18:27:59.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:27:59 smithi171 ceph-mon[129241]: pgmap v687: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 16 MiB/s wr, 997 op/s 2023-07-14T18:28:00.408 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:28:00.869 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 515.0 s, 665.5 tps, lat 129.902 ms stddev 259.311 2023-07-14T18:28:01.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.971+0000 7f1dc4bff700 1 -- 172.21.15.133:0/3297975897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1dc0071960 msgr2=0x7f1dc010b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:28:01.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.971+0000 7f1dc4bff700 1 --2- 172.21.15.133:0/3297975897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1dc0071960 0x7f1dc010b900 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1da8009990 tx=0x7f1da8009ca0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.972+0000 7f1dc4bff700 1 -- 172.21.15.133:0/3297975897 shutdown_connections 2023-07-14T18:28:01.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.972+0000 7f1dc4bff700 1 --2- 172.21.15.133:0/3297975897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dc010be90 0x7f1dc010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.972+0000 7f1dc4bff700 1 --2- 172.21.15.133:0/3297975897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1dc0071010 0x7f1dc0071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.972+0000 7f1dc4bff700 1 --2- 172.21.15.133:0/3297975897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1dc0071960 0x7f1dc010b900 secure :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1da8009990 tx=0x7f1da8009ca0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.972+0000 7f1dc4bff700 1 -- 172.21.15.133:0/3297975897 >> 172.21.15.133:0/3297975897 conn(0x7f1dc006cbc0 msgr2=0x7f1dc006eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:28:01.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.977+0000 7f1dc4bff700 1 -- 172.21.15.133:0/3297975897 shutdown_connections 2023-07-14T18:28:01.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.985+0000 7f1dc4bff700 1 -- 172.21.15.133:0/3297975897 wait complete. 2023-07-14T18:28:01.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.987+0000 7f1dc4bff700 1 Processor -- start 2023-07-14T18:28:01.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.987+0000 7f1dc4bff700 1 -- start start 2023-07-14T18:28:01.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.988+0000 7f1dc4bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1dc0071010 0x7f1dc01a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:01.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.988+0000 7f1dc4bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dc010be90 0x7f1dc01a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:01.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.988+0000 7f1dc4bff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1dc0076890 0x7f1dc0078d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:01.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.988+0000 7f1dc4bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc0079280 con 0x7f1dc0071010 2023-07-14T18:28:01.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.988+0000 7f1dc4bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc00793f0 con 0x7f1dc010be90 2023-07-14T18:28:01.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.988+0000 7f1dc4bff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc00795b0 con 0x7f1dc0076890 2023-07-14T18:28:01.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.990+0000 7f1dbdd9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dc010be90 0x7f1dc01a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:01.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.990+0000 7f1dbdd9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dc010be90 0x7f1dc01a4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:59442/0 (socket says 172.21.15.133:59442) 2023-07-14T18:28:01.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.990+0000 7f1dbdd9b700 1 -- 172.21.15.133:0/4246268274 learned_addr learned my addr 172.21.15.133:0/4246268274 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:28:01.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.990+0000 7f1dbed9d700 1 --2- 172.21.15.133:0/4246268274 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1dc0076890 0x7f1dc0078d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:01.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.990+0000 7f1dbdd9b700 1 -- 172.21.15.133:0/4246268274 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1dc0076890 msgr2=0x7f1dc0078d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:28:01.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.990+0000 7f1dbdd9b700 1 --2- 172.21.15.133:0/4246268274 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1dc0076890 0x7f1dc0078d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.990+0000 7f1dbdd9b700 1 -- 172.21.15.133:0/4246268274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1dc0071010 msgr2=0x7f1dc01a4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:28:01.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.990+0000 7f1dbdd9b700 1 --2- 172.21.15.133:0/4246268274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1dc0071010 0x7f1dc01a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.991+0000 7f1dbdd9b700 1 -- 172.21.15.133:0/4246268274 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da80095f0 con 0x7f1dc010be90 2023-07-14T18:28:01.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.991+0000 7f1dbdd9b700 1 --2- 172.21.15.133:0/4246268274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dc010be90 0x7f1dc01a4f90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1da8006010 tx=0x7f1da8003cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:28:01.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.992+0000 7f1db77fe700 1 -- 172.21.15.133:0/4246268274 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da801d070 con 0x7f1dc010be90 2023-07-14T18:28:01.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.992+0000 7f1db77fe700 1 -- 172.21.15.133:0/4246268274 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1da800f460 con 0x7f1dc010be90 2023-07-14T18:28:01.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.992+0000 7f1db77fe700 1 -- 172.21.15.133:0/4246268274 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da8017650 con 0x7f1dc010be90 2023-07-14T18:28:01.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.992+0000 7f1dc4bff700 1 -- 172.21.15.133:0/4246268274 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc00797d0 con 0x7f1dc010be90 2023-07-14T18:28:01.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:00.992+0000 7f1dc4bff700 1 -- 172.21.15.133:0/4246268274 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc0079c80 con 0x7f1dc010be90 2023-07-14T18:28:01.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.000+0000 7f1dd85afb80 1 Processor -- start 2023-07-14T18:28:01.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.005+0000 7f1db77fe700 1 -- 172.21.15.133:0/4246268274 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f1da800f8d0 con 0x7f1dc010be90 2023-07-14T18:28:01.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.007+0000 7f1db77fe700 1 --2- 172.21.15.133:0/4246268274 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f1dac0a1350 0x7f1dac0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:01.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.007+0000 7f1db77fe700 1 -- 172.21.15.133:0/4246268274 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1da80cd8d0 con 0x7f1dc010be90 2023-07-14T18:28:01.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.008+0000 7f1dbed9d700 1 --2- 172.21.15.133:0/4246268274 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f1dac0a1350 0x7f1dac0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:01.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.020+0000 7f1dd85afb80 1 -- start start 2023-07-14T18:28:01.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.020+0000 7f1dd85afb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4de47af60 0x55b4de47b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:01.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.020+0000 7f1dd85afb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4de47b8b0 0x55b4de47bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:01.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.020+0000 7f1dd85afb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4de47c4b0 0x55b4de47c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:01.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.020+0000 7f1dd85afb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4de37fcb0 con 0x55b4de47b8b0 2023-07-14T18:28:01.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.020+0000 7f1dd85afb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4de3d2a40 con 0x55b4de47af60 2023-07-14T18:28:01.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.020+0000 7f1dd85afb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4de3cbed0 con 0x55b4de47c4b0 2023-07-14T18:28:01.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.021+0000 7f1dbe59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4de47af60 0x55b4de47b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:01.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.021+0000 7f1dbe59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4de47af60 0x55b4de47b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:59456/0 (socket says 172.21.15.133:59456) 2023-07-14T18:28:01.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.021+0000 7f1dbe59c700 1 -- 172.21.15.133:0/2180670383 learned_addr learned my addr 172.21.15.133:0/2180670383 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:28:01.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.022+0000 7f1dbed9d700 1 --2- 172.21.15.133:0/2180670383 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4de47c4b0 0x55b4de47c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:01.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.022+0000 7f1dbe59c700 1 -- 172.21.15.133:0/2180670383 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4de47c4b0 msgr2=0x55b4de47c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:28:01.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.022+0000 7f1dbed9d700 1 --2- 172.21.15.133:0/4246268274 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f1dac0a1350 0x7f1dac0a3800 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1db0009c90 tx=0x7f1db0009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:28:01.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.022+0000 7f1dbe59c700 1 --2- 172.21.15.133:0/2180670383 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4de47c4b0 0x55b4de47c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.022+0000 7f1dbe59c700 1 -- 172.21.15.133:0/2180670383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4de47b8b0 msgr2=0x55b4de47bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:28:01.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.022+0000 7f1dbe59c700 1 --2- 172.21.15.133:0/2180670383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4de47b8b0 0x55b4de47bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.022+0000 7f1dbe59c700 1 -- 172.21.15.133:0/2180670383 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4de1f9de0 con 0x55b4de47af60 2023-07-14T18:28:01.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.022+0000 7f1dbed9d700 1 --2- 172.21.15.133:0/2180670383 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4de47c4b0 0x55b4de47c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T18:28:01.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.022+0000 7f1dbe59c700 1 --2- 172.21.15.133:0/2180670383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4de47af60 0x55b4de47b370 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1db8009840 tx=0x7f1db8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e633ef031e555e22 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:28:01.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.023+0000 7f1db4ff9700 1 -- 172.21.15.133:0/2180670383 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db8011830 con 0x55b4de47af60 2023-07-14T18:28:01.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.023+0000 7f1db4ff9700 1 -- 172.21.15.133:0/2180670383 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1db8011e50 con 0x55b4de47af60 2023-07-14T18:28:01.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.023+0000 7f1db4ff9700 1 -- 172.21.15.133:0/2180670383 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db8010380 con 0x55b4de47af60 2023-07-14T18:28:01.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.024+0000 7f1dd85afb80 1 -- 172.21.15.133:0/2180670383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4de47af60 msgr2=0x55b4de47b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:28:01.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.024+0000 7f1dd85afb80 1 --2- 172.21.15.133:0/2180670383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4de47af60 0x55b4de47b370 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1db8009840 tx=0x7f1db8009b50 comp rx=0 tx=0).stop 2023-07-14T18:28:01.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.028+0000 7f1dd85afb80 1 -- 172.21.15.133:0/2180670383 shutdown_connections 2023-07-14T18:28:01.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.028+0000 7f1dd85afb80 1 --2- 172.21.15.133:0/2180670383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4de47af60 0x55b4de47b370 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.028+0000 7f1dd85afb80 1 --2- 172.21.15.133:0/2180670383 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4de47c4b0 0x55b4de47c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.028+0000 7f1dd85afb80 1 --2- 172.21.15.133:0/2180670383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4de47b8b0 0x55b4de47bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.028+0000 7f1dd85afb80 1 -- 172.21.15.133:0/2180670383 >> 172.21.15.133:0/2180670383 conn(0x55b4de470680 msgr2=0x55b4de4759d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:28:01.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.028+0000 7f1dd85afb80 1 -- 172.21.15.133:0/2180670383 shutdown_connections 2023-07-14T18:28:01.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.028+0000 7f1dd85afb80 1 -- 172.21.15.133:0/2180670383 wait complete. 2023-07-14T18:28:01.068 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.029+0000 7f1dd85afb80 1 -- start start 2023-07-14T18:28:01.068 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.030+0000 7f1dd85afb80 1 Processor -- start 2023-07-14T18:28:01.068 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.030+0000 7f1dd85afb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4de47af60 0x55b4de4959f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:01.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.030+0000 7f1dd85afb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4de47b8b0 0x55b4de495f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:01.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.030+0000 7f1dd85afb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4de47c4b0 0x55b4de49cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:01.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.030+0000 7f1dd85afb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4de36bac0 con 0x55b4de47b8b0 2023-07-14T18:28:01.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.030+0000 7f1dd85afb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4de36b7c0 con 0x55b4de47c4b0 2023-07-14T18:28:01.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.030+0000 7f1dd85afb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4de24f920 con 0x55b4de47af60 2023-07-14T18:28:01.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.030+0000 7f1dd85afb80 10 client.0 _ll_register_callbacks cb 0x55b4de369760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:28:01.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.031+0000 7f1dbed9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4de47c4b0 0x55b4de49cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:01.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.031+0000 7f1dbed9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4de47c4b0 0x55b4de49cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:59466/0 (socket says 172.21.15.133:59466) 2023-07-14T18:28:01.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.031+0000 7f1dbed9d700 1 -- 172.21.15.133:0/426717159 learned_addr learned my addr 172.21.15.133:0/426717159 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:28:01.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.031+0000 7f1dbe59c700 1 --2- 172.21.15.133:0/426717159 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4de47af60 0x55b4de4959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:01.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.032+0000 7f1dbed9d700 1 -- 172.21.15.133:0/426717159 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4de47af60 msgr2=0x55b4de4959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:28:01.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.032+0000 7f1dbed9d700 1 --2- 172.21.15.133:0/426717159 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4de47af60 0x55b4de4959f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.032+0000 7f1dbed9d700 1 -- 172.21.15.133:0/426717159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4de47b8b0 msgr2=0x55b4de495f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:28:01.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.032+0000 7f1dbed9d700 1 --2- 172.21.15.133:0/426717159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4de47b8b0 0x55b4de495f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.032+0000 7f1dbed9d700 1 -- 172.21.15.133:0/426717159 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db8009470 con 0x55b4de47c4b0 2023-07-14T18:28:01.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.034+0000 7f1dbe59c700 1 --2- 172.21.15.133:0/426717159 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4de47af60 0x55b4de4959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T18:28:01.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.040+0000 7f1dbed9d700 1 --2- 172.21.15.133:0/426717159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4de47c4b0 0x55b4de49cf70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1db00169d0 tx=0x7f1db0004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=564c2a3aeba1c52c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:28:01.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.040+0000 7f1da5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:28:01.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.040+0000 7f1da5ffb700 1 -- 172.21.15.133:0/426717159 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db0014c70 con 0x55b4de47c4b0 2023-07-14T18:28:01.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.041+0000 7f1da5ffb700 1 -- 172.21.15.133:0/426717159 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1db0016e40 con 0x55b4de47c4b0 2023-07-14T18:28:01.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.047+0000 7f1dd85afb80 1 -- 172.21.15.133:0/426717159 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b4de3254f0 con 0x55b4de47c4b0 2023-07-14T18:28:01.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.049+0000 7f1da5ffb700 1 -- 172.21.15.133:0/426717159 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1db0014dd0 con 0x55b4de47c4b0 2023-07-14T18:28:01.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.050+0000 7f1dd85afb80 10 client.25919 fetch_fsmap learned FSMap version 40 2023-07-14T18:28:01.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.050+0000 7f1dd85afb80 1 -- 172.21.15.133:0/426717159 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55b4de1e2c60 con 0x55b4de47c4b0 2023-07-14T18:28:01.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.055+0000 7f1da5ffb700 1 -- 172.21.15.133:0/426717159 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b4de3254f0 con 0x55b4de47c4b0 2023-07-14T18:28:01.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.055+0000 7f1da5ffb700 1 -- 172.21.15.133:0/426717159 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 40) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f1db0021620 con 0x55b4de47c4b0 2023-07-14T18:28:01.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.056+0000 7f1dd85afb80 10 client.25919 fetch_fsmap finished waiting for FSMap version 40 2023-07-14T18:28:01.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.056+0000 7f1dd85afb80 10 client.25919 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:28:01.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.056+0000 7f1dd85afb80 20 client.25919 populate_metadata read hostname 'smithi133' 2023-07-14T18:28:01.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.056+0000 7f1dd85afb80 1 --2- 172.21.15.133:0/426717159 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55b4de4a17f0 0x55b4de4a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:01.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.056+0000 7f1dd85afb80 4 client.25919 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:28:01.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.056+0000 7f1dd85afb80 1 -- 172.21.15.133:0/426717159 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b4de449da0 con 0x55b4de4a17f0 2023-07-14T18:28:01.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.061+0000 7f1dbe59c700 1 --2- 172.21.15.133:0/426717159 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55b4de4a17f0 0x55b4de4a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:01.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.064+0000 7f1dbe59c700 1 --2- 172.21.15.133:0/426717159 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55b4de4a17f0 0x55b4de4a3c00 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3a41c9188120fba server_cookie=df40c4e22d7c4677 in_seq=0 out_seq=0 2023-07-14T18:28:01.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.065+0000 7f1da5ffb700 10 client.25919 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:28:01.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.065+0000 7f1da5ffb700 1 -- 172.21.15.133:0/426717159 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55b4de449da0 con 0x55b4de4a17f0 2023-07-14T18:28:01.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.066+0000 7f1da5ffb700 10 client.25919 handle_command_reply: tid=0 2023-07-14T18:28:01.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.066+0000 7f1dd85afb80 1 client.25919 shutdown 2023-07-14T18:28:01.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.067+0000 7f1dd85afb80 1 -- 172.21.15.133:0/426717159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4de47c4b0 msgr2=0x55b4de49cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:28:01.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.067+0000 7f1dd85afb80 1 --2- 172.21.15.133:0/426717159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4de47c4b0 0x55b4de49cf70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1db00169d0 tx=0x7f1db0004b40 comp rx=0 tx=0).stop 2023-07-14T18:28:01.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.068+0000 7f1dd85afb80 1 -- 172.21.15.133:0/426717159 shutdown_connections 2023-07-14T18:28:01.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.068+0000 7f1dd85afb80 1 --2- 172.21.15.133:0/426717159 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55b4de4a17f0 0x55b4de4a3c00 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.068+0000 7f1dd85afb80 1 --2- 172.21.15.133:0/426717159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4de47c4b0 0x55b4de49cf70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.068+0000 7f1dd85afb80 1 --2- 172.21.15.133:0/426717159 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4de47af60 0x55b4de4959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.068+0000 7f1dd85afb80 1 --2- 172.21.15.133:0/426717159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4de47b8b0 0x55b4de495f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.068+0000 7f1dd85afb80 1 -- 172.21.15.133:0/426717159 >> 172.21.15.133:0/426717159 conn(0x55b4de470680 msgr2=0x55b4de474650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:28:01.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.068+0000 7f1da5ffb700 0 client.25919 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:28:01.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.068+0000 7f1dd85afb80 1 -- 172.21.15.133:0/426717159 shutdown_connections 2023-07-14T18:28:01.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.069+0000 7f1dd85afb80 1 -- 172.21.15.133:0/426717159 wait complete. 2023-07-14T18:28:01.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.069+0000 7f1dd85afb80 20 client.25919 trim_cache size 0 max 16384 2023-07-14T18:28:01.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.103+0000 7f1dd85afb80 1 Processor -- start 2023-07-14T18:28:01.143 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.103+0000 7f1dd85afb80 1 -- start start 2023-07-14T18:28:01.143 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.103+0000 7f1dd85afb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4de47af60 0x55b4de494710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:01.143 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.103+0000 7f1dd85afb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4de47b8b0 0x55b4de49ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:01.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.103+0000 7f1dd85afb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4de47c4b0 0x55b4de4a3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:01.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.103+0000 7f1dd85afb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4de477530 con 0x55b4de47af60 2023-07-14T18:28:01.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.103+0000 7f1dd85afb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4de495560 con 0x55b4de47c4b0 2023-07-14T18:28:01.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.104+0000 7f1dd85afb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4de47bd20 con 0x55b4de47b8b0 2023-07-14T18:28:01.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.104+0000 7f1dbdd9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4de47b8b0 0x55b4de49ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:01.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.104+0000 7f1dbdd9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4de47b8b0 0x55b4de49ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:55432/0 (socket says 172.21.15.133:55432) 2023-07-14T18:28:01.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.104+0000 7f1dbdd9b700 1 -- 172.21.15.133:0/3509670352 learned_addr learned my addr 172.21.15.133:0/3509670352 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:28:01.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.105+0000 7f1dbdd9b700 1 -- 172.21.15.133:0/3509670352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4de47c4b0 msgr2=0x55b4de4a3800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T18:28:01.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.105+0000 7f1dbdd9b700 1 --2- 172.21.15.133:0/3509670352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4de47c4b0 0x55b4de4a3800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.105+0000 7f1dbdd9b700 1 -- 172.21.15.133:0/3509670352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4de47af60 msgr2=0x55b4de494710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:28:01.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.105+0000 7f1dbdd9b700 1 --2- 172.21.15.133:0/3509670352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4de47af60 0x55b4de494710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.105+0000 7f1dbdd9b700 1 -- 172.21.15.133:0/3509670352 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4de220250 con 0x55b4de47b8b0 2023-07-14T18:28:01.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.105+0000 7f1dbdd9b700 1 --2- 172.21.15.133:0/3509670352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4de47b8b0 0x55b4de49ce90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1da8096ee0 tx=0x7f1da8096e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9ed1fb775c9d3a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:28:01.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.106+0000 7f1db57fa700 1 -- 172.21.15.133:0/3509670352 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da8029340 con 0x55b4de47b8b0 2023-07-14T18:28:01.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.106+0000 7f1db57fa700 1 -- 172.21.15.133:0/3509670352 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1da809a9e0 con 0x55b4de47b8b0 2023-07-14T18:28:01.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.106+0000 7f1db57fa700 1 -- 172.21.15.133:0/3509670352 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da809ac40 con 0x55b4de47b8b0 2023-07-14T18:28:01.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.107+0000 7f1dd85afb80 1 -- 172.21.15.133:0/3509670352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4de47b8b0 msgr2=0x55b4de49ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:28:01.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.107+0000 7f1dd85afb80 1 --2- 172.21.15.133:0/3509670352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4de47b8b0 0x55b4de49ce90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1da8096ee0 tx=0x7f1da8096e40 comp rx=0 tx=0).stop 2023-07-14T18:28:01.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.108+0000 7f1dd85afb80 1 -- 172.21.15.133:0/3509670352 shutdown_connections 2023-07-14T18:28:01.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.108+0000 7f1dd85afb80 1 --2- 172.21.15.133:0/3509670352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4de47c4b0 0x55b4de4a3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.108+0000 7f1dd85afb80 1 --2- 172.21.15.133:0/3509670352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4de47af60 0x55b4de494710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.151 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.108+0000 7f1dd85afb80 1 --2- 172.21.15.133:0/3509670352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4de47b8b0 0x55b4de49ce90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.151 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.108+0000 7f1dd85afb80 1 -- 172.21.15.133:0/3509670352 >> 172.21.15.133:0/3509670352 conn(0x55b4de470680 msgr2=0x55b4de49d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:28:01.151 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.108+0000 7f1dd85afb80 1 -- 172.21.15.133:0/3509670352 shutdown_connections 2023-07-14T18:28:01.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.109+0000 7f1dd85afb80 1 -- 172.21.15.133:0/3509670352 wait complete. 2023-07-14T18:28:01.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.109+0000 7f1dd85afb80 1 -- start start 2023-07-14T18:28:01.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.110+0000 7f1dd85afb80 1 Processor -- start 2023-07-14T18:28:01.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.110+0000 7f1dd85afb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4de47af60 0x55b4de498d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:01.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.110+0000 7f1dd85afb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4de47b8b0 0x55b4de4992c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:01.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.110+0000 7f1dd85afb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4de47c4b0 0x55b4de49c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:01.155 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.110+0000 7f1dd85afb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4de49c820 con 0x55b4de47af60 2023-07-14T18:28:01.155 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.110+0000 7f1dd85afb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4de49c990 con 0x55b4de47b8b0 2023-07-14T18:28:01.155 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.110+0000 7f1dd85afb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4de49cb50 con 0x55b4de47c4b0 2023-07-14T18:28:01.155 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.110+0000 7f1dd85afb80 10 client.0 _ll_register_callbacks cb 0x7f1db0009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:28:01.156 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.111+0000 7f1dbed9d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4de47c4b0 0x55b4de49c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:01.156 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.111+0000 7f1dbed9d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4de47c4b0 0x55b4de49c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:55434/0 (socket says 172.21.15.133:55434) 2023-07-14T18:28:01.156 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.111+0000 7f1dbed9d700 1 -- 172.21.15.133:0/1786874811 learned_addr learned my addr 172.21.15.133:0/1786874811 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:28:01.157 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.111+0000 7f1dbe59c700 1 --2- 172.21.15.133:0/1786874811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4de47af60 0x55b4de498d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:01.157 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.111+0000 7f1dbdd9b700 1 --2- 172.21.15.133:0/1786874811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4de47b8b0 0x55b4de4992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:01.157 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.112+0000 7f1dbed9d700 1 -- 172.21.15.133:0/1786874811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4de47b8b0 msgr2=0x55b4de4992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:28:01.157 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.112+0000 7f1dbed9d700 1 --2- 172.21.15.133:0/1786874811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4de47b8b0 0x55b4de4992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.112+0000 7f1dbed9d700 1 -- 172.21.15.133:0/1786874811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4de47af60 msgr2=0x55b4de498d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:28:01.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.112+0000 7f1dbed9d700 1 --2- 172.21.15.133:0/1786874811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4de47af60 0x55b4de498d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.112+0000 7f1dbed9d700 1 -- 172.21.15.133:0/1786874811 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc0079c80 con 0x55b4de47c4b0 2023-07-14T18:28:01.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.112+0000 7f1dbed9d700 1 --2- 172.21.15.133:0/1786874811 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4de47c4b0 0x55b4de49c2e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1db0009120 tx=0x7f1db0021ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e34a4e50b9a3003 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:28:01.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.112+0000 7f1da5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:28:01.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.113+0000 7f1da5ffb700 1 -- 172.21.15.133:0/1786874811 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db0021610 con 0x55b4de47c4b0 2023-07-14T18:28:01.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.113+0000 7f1da5ffb700 1 -- 172.21.15.133:0/1786874811 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1db0021770 con 0x55b4de47c4b0 2023-07-14T18:28:01.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.113+0000 7f1dd85afb80 1 -- 172.21.15.133:0/1786874811 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1db8011e50 con 0x55b4de47c4b0 2023-07-14T18:28:01.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.113+0000 7f1da5ffb700 1 -- 172.21.15.133:0/1786874811 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1db0014a70 con 0x55b4de47c4b0 2023-07-14T18:28:01.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.113+0000 7f1dd85afb80 10 client.25954 fetch_fsmap learned FSMap version 40 2023-07-14T18:28:01.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.113+0000 7f1dd85afb80 1 -- 172.21.15.133:0/1786874811 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55b4de1f82b0 con 0x55b4de47c4b0 2023-07-14T18:28:01.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.113+0000 7f1da5ffb700 1 -- 172.21.15.133:0/1786874811 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1db8011e50 con 0x55b4de47c4b0 2023-07-14T18:28:01.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.114+0000 7f1da5ffb700 1 -- 172.21.15.133:0/1786874811 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 40) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f1db002a3b0 con 0x55b4de47c4b0 2023-07-14T18:28:01.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.114+0000 7f1dd85afb80 10 client.25954 fetch_fsmap finished waiting for FSMap version 40 2023-07-14T18:28:01.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.114+0000 7f1dd85afb80 10 client.25954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:28:01.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.114+0000 7f1dd85afb80 20 client.25954 populate_metadata read hostname 'smithi133' 2023-07-14T18:28:01.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.114+0000 7f1dd85afb80 1 --2- 172.21.15.133:0/1786874811 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55b4de4f6680 0x55b4de4f8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:01.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.114+0000 7f1dd85afb80 4 client.25954 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:28:01.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.114+0000 7f1dd85afb80 1 -- 172.21.15.133:0/1786874811 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b4de3c4eb0 con 0x55b4de4f6680 2023-07-14T18:28:01.165 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.115+0000 7f1dbe59c700 1 --2- 172.21.15.133:0/1786874811 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55b4de4f6680 0x55b4de4f8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:01.165 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.122+0000 7f1dbe59c700 1 --2- 172.21.15.133:0/1786874811 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55b4de4f6680 0x55b4de4f8a90 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24ebdeeadf411874 server_cookie=3c5fe1fc045c0812 in_seq=0 out_seq=0 2023-07-14T18:28:01.165 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.122+0000 7f1da5ffb700 10 client.25954 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:28:01.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.122+0000 7f1da5ffb700 1 -- 172.21.15.133:0/1786874811 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55b4de3c4eb0 con 0x55b4de4f6680 2023-07-14T18:28:01.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.122+0000 7f1da5ffb700 10 client.25954 handle_command_reply: tid=0 2023-07-14T18:28:01.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.122+0000 7f1dd85afb80 1 client.25954 shutdown 2023-07-14T18:28:01.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.123+0000 7f1dd85afb80 1 -- 172.21.15.133:0/1786874811 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4de47c4b0 msgr2=0x55b4de49c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:28:01.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.123+0000 7f1dd85afb80 1 --2- 172.21.15.133:0/1786874811 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4de47c4b0 0x55b4de49c2e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1db0009120 tx=0x7f1db0021ca0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.123+0000 7f1dd85afb80 1 -- 172.21.15.133:0/1786874811 shutdown_connections 2023-07-14T18:28:01.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.123+0000 7f1dd85afb80 1 --2- 172.21.15.133:0/1786874811 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55b4de4f6680 0x55b4de4f8a90 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.123+0000 7f1dd85afb80 1 --2- 172.21.15.133:0/1786874811 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4de47c4b0 0x55b4de49c2e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.123+0000 7f1dd85afb80 1 --2- 172.21.15.133:0/1786874811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4de47af60 0x55b4de498d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.123+0000 7f1dd85afb80 1 --2- 172.21.15.133:0/1786874811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4de47b8b0 0x55b4de4992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.123+0000 7f1da5ffb700 0 client.25954 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:28:01.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.123+0000 7f1dd85afb80 1 -- 172.21.15.133:0/1786874811 >> 172.21.15.133:0/1786874811 conn(0x55b4de470680 msgr2=0x55b4de4a3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:28:01.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.123+0000 7f1dd85afb80 1 -- 172.21.15.133:0/1786874811 shutdown_connections 2023-07-14T18:28:01.168 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.123+0000 7f1dd85afb80 1 -- 172.21.15.133:0/1786874811 wait complete. 2023-07-14T18:28:01.168 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.124+0000 7f1dd85afb80 20 client.25954 trim_cache size 0 max 16384 2023-07-14T18:28:01.169 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:28:01.169 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T18:28:01.170 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:28:01.170 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:28:01.170 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:28:01.171 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:28:01.171 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:28:01.171 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:28:01.171 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:28:01.172 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:28:01.173 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.127+0000 7f1dc4bff700 1 -- 172.21.15.133:0/4246268274 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f1dac0a1350 msgr2=0x7f1dac0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:28:01.173 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.127+0000 7f1dc4bff700 1 --2- 172.21.15.133:0/4246268274 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f1dac0a1350 0x7f1dac0a3800 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1db0009c90 tx=0x7f1db0009040 comp rx=0 tx=0).stop 2023-07-14T18:28:01.173 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.127+0000 7f1dc4bff700 1 -- 172.21.15.133:0/4246268274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dc010be90 msgr2=0x7f1dc01a4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:28:01.174 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.127+0000 7f1dc4bff700 1 --2- 172.21.15.133:0/4246268274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dc010be90 0x7f1dc01a4f90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1da8006010 tx=0x7f1da8003cc0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.174 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.130+0000 7f1dc4bff700 1 -- 172.21.15.133:0/4246268274 shutdown_connections 2023-07-14T18:28:01.174 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.130+0000 7f1dc4bff700 1 --2- 172.21.15.133:0/4246268274 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f1dac0a1350 0x7f1dac0a3800 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.174 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.130+0000 7f1dc4bff700 1 --2- 172.21.15.133:0/4246268274 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1dc0076890 0x7f1dc0078d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.175 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.130+0000 7f1dc4bff700 1 --2- 172.21.15.133:0/4246268274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1dc0071010 0x7f1dc01a4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.175 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.130+0000 7f1dc4bff700 1 --2- 172.21.15.133:0/4246268274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dc010be90 0x7f1dc01a4f90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:01.175 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.130+0000 7f1dc4bff700 1 -- 172.21.15.133:0/4246268274 >> 172.21.15.133:0/4246268274 conn(0x7f1dc006cbc0 msgr2=0x7f1dc006dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:28:01.176 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.132+0000 7f1dc4bff700 1 -- 172.21.15.133:0/4246268274 shutdown_connections 2023-07-14T18:28:01.176 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:01.132+0000 7f1dc4bff700 1 -- 172.21.15.133:0/4246268274 wait complete. 2023-07-14T18:28:01.211 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:28:01.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:01 smithi133 ceph-mon[126384]: pgmap v688: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 14 MiB/s wr, 939 op/s 2023-07-14T18:28:01.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:01 smithi161 ceph-mon[128993]: pgmap v688: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 14 MiB/s wr, 939 op/s 2023-07-14T18:28:01.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:01 smithi171 ceph-mon[129241]: pgmap v688: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 14 MiB/s wr, 939 op/s 2023-07-14T18:28:03.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:02 smithi161 ceph-mon[128993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:28:03.345 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:02 smithi161 ceph-mon[128993]: pgmap v689: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 16 MiB/s wr, 1.17k op/s 2023-07-14T18:28:03.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:02 smithi171 ceph-mon[129241]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:28:03.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:02 smithi171 ceph-mon[129241]: pgmap v689: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 16 MiB/s wr, 1.17k op/s 2023-07-14T18:28:03.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:02 smithi133 ceph-mon[126384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:28:03.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:02 smithi133 ceph-mon[126384]: pgmap v689: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 16 MiB/s wr, 1.17k op/s 2023-07-14T18:28:05.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:05 smithi161 ceph-mon[128993]: pgmap v690: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 13 MiB/s wr, 918 op/s 2023-07-14T18:28:05.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:05 smithi171 ceph-mon[129241]: pgmap v690: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 13 MiB/s wr, 918 op/s 2023-07-14T18:28:05.868 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 520.0 s, 797.3 tps, lat 142.876 ms stddev 220.418 2023-07-14T18:28:05.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:05 smithi133 ceph-mon[126384]: pgmap v690: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 13 MiB/s wr, 918 op/s 2023-07-14T18:28:07.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:06 smithi161 ceph-mon[128993]: pgmap v691: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 15 MiB/s wr, 1.03k op/s 2023-07-14T18:28:07.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:06 smithi171 ceph-mon[129241]: pgmap v691: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 15 MiB/s wr, 1.03k op/s 2023-07-14T18:28:07.402 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:06 smithi133 ceph-mon[126384]: pgmap v691: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 15 MiB/s wr, 1.03k op/s 2023-07-14T18:28:09.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:09 smithi133 ceph-mon[126384]: pgmap v692: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 17 MiB/s wr, 1.07k op/s 2023-07-14T18:28:09.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:09 smithi161 ceph-mon[128993]: pgmap v692: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 17 MiB/s wr, 1.07k op/s 2023-07-14T18:28:09.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:09 smithi171 ceph-mon[129241]: pgmap v692: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 17 MiB/s wr, 1.07k op/s 2023-07-14T18:28:10.867 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 525.0 s, 1264.4 tps, lat 79.299 ms stddev 55.793 2023-07-14T18:28:11.848 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:11 smithi161 ceph-mon[128993]: pgmap v693: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 15 MiB/s wr, 912 op/s 2023-07-14T18:28:11.848 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:11 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:28:11.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:11 smithi171 ceph-mon[129241]: pgmap v693: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 15 MiB/s wr, 912 op/s 2023-07-14T18:28:11.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:11 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:28:11.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:11 smithi133 ceph-mon[126384]: pgmap v693: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 15 MiB/s wr, 912 op/s 2023-07-14T18:28:11.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:11 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:28:13.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:13 smithi161 ceph-mon[128993]: pgmap v694: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 16 MiB/s wr, 925 op/s 2023-07-14T18:28:13.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:13 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:28:13.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:13 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:28:13.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:13 smithi171 ceph-mon[129241]: pgmap v694: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 16 MiB/s wr, 925 op/s 2023-07-14T18:28:13.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:13 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:28:13.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:13 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:28:13.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:13 smithi133 ceph-mon[126384]: pgmap v694: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 16 MiB/s wr, 925 op/s 2023-07-14T18:28:13.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:13 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:28:13.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:13 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:28:14.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:14 smithi161 ceph-mon[128993]: pgmap v695: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 16 MiB/s wr, 933 op/s 2023-07-14T18:28:14.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:14 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:28:14.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:14 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:28:14.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:14 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:28:14.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:14 smithi171 ceph-mon[129241]: pgmap v695: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 16 MiB/s wr, 933 op/s 2023-07-14T18:28:14.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:14 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:28:14.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:14 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:28:14.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:14 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:28:14.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:14 smithi133 ceph-mon[126384]: pgmap v695: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 16 MiB/s wr, 933 op/s 2023-07-14T18:28:14.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:14 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:28:14.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:14 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:28:14.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:14 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:28:15.870 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 530.0 s, 241.1 tps, lat 410.906 ms stddev 892.697 2023-07-14T18:28:17.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:17 smithi161 ceph-mon[128993]: pgmap v696: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 19 MiB/s wr, 1.38k op/s 2023-07-14T18:28:17.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:17 smithi171 ceph-mon[129241]: pgmap v696: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 19 MiB/s wr, 1.38k op/s 2023-07-14T18:28:17.417 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:17 smithi133 ceph-mon[126384]: pgmap v696: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 19 MiB/s wr, 1.38k op/s 2023-07-14T18:28:19.399 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:19 smithi133 ceph-mon[126384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:28:19.399 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:19 smithi133 ceph-mon[126384]: pgmap v697: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 18 MiB/s wr, 1.35k op/s 2023-07-14T18:28:19.593 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:19 smithi161 ceph-mon[128993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:28:19.594 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:19 smithi161 ceph-mon[128993]: pgmap v697: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 18 MiB/s wr, 1.35k op/s 2023-07-14T18:28:19.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:19 smithi171 ceph-mon[129241]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:28:19.609 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:19 smithi171 ceph-mon[129241]: pgmap v697: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 18 MiB/s wr, 1.35k op/s 2023-07-14T18:28:20.872 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 535.0 s, 688.0 tps, lat 145.368 ms stddev 281.026 2023-07-14T18:28:21.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:21 smithi161 ceph-mon[128993]: pgmap v698: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 19 MiB/s wr, 1.50k op/s 2023-07-14T18:28:21.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:21 smithi171 ceph-mon[129241]: pgmap v698: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 19 MiB/s wr, 1.50k op/s 2023-07-14T18:28:21.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:21 smithi133 ceph-mon[126384]: pgmap v698: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 19 MiB/s wr, 1.50k op/s 2023-07-14T18:28:23.343 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:23 smithi161 ceph-mon[128993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:28:23.343 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:23 smithi161 ceph-mon[128993]: pgmap v699: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 17 MiB/s wr, 1.53k op/s 2023-07-14T18:28:23.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:23 smithi171 ceph-mon[129241]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:28:23.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:23 smithi171 ceph-mon[129241]: pgmap v699: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 17 MiB/s wr, 1.53k op/s 2023-07-14T18:28:23.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:23 smithi133 ceph-mon[126384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:28:23.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:23 smithi133 ceph-mon[126384]: pgmap v699: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 17 MiB/s wr, 1.53k op/s 2023-07-14T18:28:25.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:25 smithi161 ceph-mon[128993]: pgmap v700: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 19 MiB/s wr, 1.63k op/s 2023-07-14T18:28:25.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:25 smithi171 ceph-mon[129241]: pgmap v700: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 19 MiB/s wr, 1.63k op/s 2023-07-14T18:28:25.868 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 540.0 s, 744.3 tps, lat 133.370 ms stddev 316.022 2023-07-14T18:28:25.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:25 smithi133 ceph-mon[126384]: pgmap v700: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 19 MiB/s wr, 1.63k op/s 2023-07-14T18:28:27.343 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:26 smithi161 ceph-mon[128993]: pgmap v701: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 19 MiB/s wr, 1.55k op/s 2023-07-14T18:28:27.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:26 smithi171 ceph-mon[129241]: pgmap v701: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 19 MiB/s wr, 1.55k op/s 2023-07-14T18:28:27.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:26 smithi133 ceph-mon[126384]: pgmap v701: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 19 MiB/s wr, 1.55k op/s 2023-07-14T18:28:29.848 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:29 smithi161 ceph-mon[128993]: pgmap v702: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 14 MiB/s wr, 1.08k op/s 2023-07-14T18:28:29.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:29 smithi171 ceph-mon[129241]: pgmap v702: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 14 MiB/s wr, 1.08k op/s 2023-07-14T18:28:29.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:29 smithi133 ceph-mon[126384]: pgmap v702: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 14 MiB/s wr, 1.08k op/s 2023-07-14T18:28:30.873 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 545.0 s, 278.0 tps, lat 359.797 ms stddev 489.155 2023-07-14T18:28:31.213 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:28:31.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.714+0000 7f63fc3ff700 1 -- 172.21.15.133:0/3287878019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f4071010 msgr2=0x7f63f4071420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:28:31.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.714+0000 7f63fc3ff700 1 --2- 172.21.15.133:0/3287878019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f4071010 0x7f63f4071420 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009990 tx=0x7f63f0009ca0 comp rx=0 tx=0).stop 2023-07-14T18:28:31.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.715+0000 7f63fc3ff700 1 -- 172.21.15.133:0/3287878019 shutdown_connections 2023-07-14T18:28:31.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.715+0000 7f63fc3ff700 1 --2- 172.21.15.133:0/3287878019 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63f410be90 0x7f63f410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:31.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.715+0000 7f63fc3ff700 1 --2- 172.21.15.133:0/3287878019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f4071010 0x7f63f4071420 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:31.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.715+0000 7f63fc3ff700 1 --2- 172.21.15.133:0/3287878019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63f4071960 0x7f63f410b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:31.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.715+0000 7f63fc3ff700 1 -- 172.21.15.133:0/3287878019 >> 172.21.15.133:0/3287878019 conn(0x7f63f406cbc0 msgr2=0x7f63f406eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:28:31.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.715+0000 7f63fc3ff700 1 -- 172.21.15.133:0/3287878019 shutdown_connections 2023-07-14T18:28:31.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.726+0000 7f63fc3ff700 1 -- 172.21.15.133:0/3287878019 wait complete. 2023-07-14T18:28:31.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.727+0000 7f63fc3ff700 1 Processor -- start 2023-07-14T18:28:31.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.737+0000 7f63fc3ff700 1 -- start start 2023-07-14T18:28:31.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.737+0000 7f63fc3ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63f4071010 0x7f63f4113bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:31.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.737+0000 7f63fc3ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63f4071960 0x7f63f4114130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:31.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.737+0000 7f63fc3ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f410be90 0x7f63f41b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:31.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.737+0000 7f63fc3ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f4117210 con 0x7f63f410be90 2023-07-14T18:28:31.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.737+0000 7f63fc3ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f4117380 con 0x7f63f4071010 2023-07-14T18:28:31.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.737+0000 7f63fc3ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f41b5040 con 0x7f63f4071960 2023-07-14T18:28:31.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.738+0000 7f63f999a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63f4071960 0x7f63f4114130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:31.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.738+0000 7f63f999a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63f4071960 0x7f63f4114130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:51928/0 (socket says 172.21.15.133:51928) 2023-07-14T18:28:31.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.738+0000 7f63f999a700 1 -- 172.21.15.133:0/2949036079 learned_addr learned my addr 172.21.15.133:0/2949036079 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:28:31.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.750+0000 7f63fa99c700 1 --2- 172.21.15.133:0/2949036079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f410be90 0x7f63f41b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:31.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.750+0000 7f63fa19b700 1 --2- 172.21.15.133:0/2949036079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63f4071010 0x7f63f4113bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:31.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.759+0000 7f63f999a700 1 -- 172.21.15.133:0/2949036079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63f4071010 msgr2=0x7f63f4113bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:28:31.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.759+0000 7f63f999a700 1 --2- 172.21.15.133:0/2949036079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63f4071010 0x7f63f4113bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:31.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.759+0000 7f63f999a700 1 -- 172.21.15.133:0/2949036079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f410be90 msgr2=0x7f63f41b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:28:31.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.759+0000 7f63f999a700 1 --2- 172.21.15.133:0/2949036079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f410be90 0x7f63f41b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:31.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.759+0000 7f63f999a700 1 -- 172.21.15.133:0/2949036079 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f00095f0 con 0x7f63f4071960 2023-07-14T18:28:31.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.772+0000 7f63f999a700 1 --2- 172.21.15.133:0/2949036079 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63f4071960 0x7f63f4114130 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f63e400b500 tx=0x7f63e400b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:28:31.824 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.787+0000 7f63eb7fe700 1 -- 172.21.15.133:0/2949036079 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e4011830 con 0x7f63f4071960 2023-07-14T18:28:31.824 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.787+0000 7f63eb7fe700 1 -- 172.21.15.133:0/2949036079 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63e4011e50 con 0x7f63f4071960 2023-07-14T18:28:31.824 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.787+0000 7f63eb7fe700 1 -- 172.21.15.133:0/2949036079 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e400f2e0 con 0x7f63f4071960 2023-07-14T18:28:31.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.788+0000 7f63fc3ff700 1 -- 172.21.15.133:0/2949036079 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f41b5240 con 0x7f63f4071960 2023-07-14T18:28:31.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.788+0000 7f63fc3ff700 1 -- 172.21.15.133:0/2949036079 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f41b5780 con 0x7f63f4071960 2023-07-14T18:28:31.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.795+0000 7f640fcc4b80 1 Processor -- start 2023-07-14T18:28:31.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:31 smithi161 ceph-mon[128993]: pgmap v703: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 18 MiB/s wr, 1.35k op/s 2023-07-14T18:28:31.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.803+0000 7f640fcc4b80 1 -- start start 2023-07-14T18:28:31.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.804+0000 7f640fcc4b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56470bb8cf60 0x56470bb8d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:31.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.804+0000 7f640fcc4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56470bb8d8b0 0x56470bb8ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:31.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.804+0000 7f640fcc4b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56470bb8e4b0 0x56470bb8e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:31.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.804+0000 7f640fcc4b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56470ba91cb0 con 0x56470bb8cf60 2023-07-14T18:28:31.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.804+0000 7f640fcc4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56470bae4a40 con 0x56470bb8d8b0 2023-07-14T18:28:31.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.804+0000 7f640fcc4b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x56470badded0 con 0x56470bb8e4b0 2023-07-14T18:28:31.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.811+0000 7f63fa19b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56470bb8d8b0 0x56470bb8ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:31.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.813+0000 7f63f999a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56470bb8e4b0 0x56470bb8e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:31.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.813+0000 7f63f999a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56470bb8e4b0 0x56470bb8e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:51944/0 (socket says 172.21.15.133:51944) 2023-07-14T18:28:31.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.813+0000 7f63f999a700 1 -- 172.21.15.133:0/3277598537 learned_addr learned my addr 172.21.15.133:0/3277598537 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:28:31.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.815+0000 7f63eb7fe700 1 -- 172.21.15.133:0/2949036079 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f63e4011990 con 0x7f63f4071960 2023-07-14T18:28:31.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.817+0000 7f63eb7fe700 1 --2- 172.21.15.133:0/2949036079 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f63e00a1350 0x7f63e00a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:31.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.817+0000 7f63eb7fe700 1 -- 172.21.15.133:0/2949036079 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f63e40cc230 con 0x7f63f4071960 2023-07-14T18:28:31.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.818+0000 7f63fa99c700 1 --2- 172.21.15.133:0/3277598537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56470bb8cf60 0x56470bb8d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:31.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:31 smithi171 ceph-mon[129241]: pgmap v703: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 18 MiB/s wr, 1.35k op/s 2023-07-14T18:28:31.865 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.829+0000 7f63fa99c700 1 --2- 172.21.15.133:0/2949036079 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f63e00a1350 0x7f63e00a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:31.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.837+0000 7f63f999a700 1 -- 172.21.15.133:0/3277598537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56470bb8d8b0 msgr2=0x56470bb8ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:28:31.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.837+0000 7f63f999a700 1 --2- 172.21.15.133:0/3277598537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56470bb8d8b0 0x56470bb8ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:31.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.837+0000 7f63f999a700 1 -- 172.21.15.133:0/3277598537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56470bb8cf60 msgr2=0x56470bb8d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:28:31.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.837+0000 7f63f999a700 1 --2- 172.21.15.133:0/3277598537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56470bb8cf60 0x56470bb8d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:31.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.837+0000 7f63f999a700 1 -- 172.21.15.133:0/3277598537 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56470b90bde0 con 0x56470bb8e4b0 2023-07-14T18:28:31.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.842+0000 7f63fa99c700 1 --2- 172.21.15.133:0/2949036079 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f63e00a1350 0x7f63e00a3800 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f63ec009fd0 tx=0x7f63ec004b10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:28:31.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.862+0000 7f63f999a700 1 --2- 172.21.15.133:0/3277598537 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56470bb8e4b0 0x56470bb8e920 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f63e400b4d0 tx=0x7f63e4095710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf25b5a10cdaf672 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:28:31.908 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:31 smithi133 ceph-mon[126384]: pgmap v703: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 18 MiB/s wr, 1.35k op/s 2023-07-14T18:28:31.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.865+0000 7f63e8bfe700 1 -- 172.21.15.133:0/3277598537 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e4098c10 con 0x56470bb8e4b0 2023-07-14T18:28:31.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.865+0000 7f63e8bfe700 1 -- 172.21.15.133:0/3277598537 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63e40992b0 con 0x56470bb8e4b0 2023-07-14T18:28:31.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.865+0000 7f63e8bfe700 1 -- 172.21.15.133:0/3277598537 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e4099770 con 0x56470bb8e4b0 2023-07-14T18:28:31.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.867+0000 7f640fcc4b80 1 -- 172.21.15.133:0/3277598537 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56470bb8e4b0 msgr2=0x56470bb8e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:28:31.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.867+0000 7f640fcc4b80 1 --2- 172.21.15.133:0/3277598537 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56470bb8e4b0 0x56470bb8e920 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f63e400b4d0 tx=0x7f63e4095710 comp rx=0 tx=0).stop 2023-07-14T18:28:31.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.867+0000 7f640fcc4b80 1 -- 172.21.15.133:0/3277598537 shutdown_connections 2023-07-14T18:28:31.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.867+0000 7f640fcc4b80 1 --2- 172.21.15.133:0/3277598537 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56470bb8e4b0 0x56470bb8e920 secure :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f63e400b4d0 tx=0x7f63e4095710 comp rx=0 tx=0).stop 2023-07-14T18:28:31.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.867+0000 7f640fcc4b80 1 --2- 172.21.15.133:0/3277598537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56470bb8cf60 0x56470bb8d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:31.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.867+0000 7f640fcc4b80 1 --2- 172.21.15.133:0/3277598537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56470bb8d8b0 0x56470bb8ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:31.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.867+0000 7f640fcc4b80 1 -- 172.21.15.133:0/3277598537 >> 172.21.15.133:0/3277598537 conn(0x56470bb82680 msgr2=0x56470bb879d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:28:31.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.882+0000 7f640fcc4b80 1 -- 172.21.15.133:0/3277598537 shutdown_connections 2023-07-14T18:28:31.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.885+0000 7f640fcc4b80 1 -- 172.21.15.133:0/3277598537 wait complete. 2023-07-14T18:28:31.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.886+0000 7f640fcc4b80 1 -- start start 2023-07-14T18:28:31.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.887+0000 7f640fcc4b80 1 Processor -- start 2023-07-14T18:28:31.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.888+0000 7f640fcc4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56470bb8cf60 0x56470bb96c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:31.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.888+0000 7f640fcc4b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56470bb8d8b0 0x56470bb971b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:31.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.888+0000 7f640fcc4b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56470bb9c1e0 0x56470bb9e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:31.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.888+0000 7f640fcc4b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56470ba7dac0 con 0x56470bb8d8b0 2023-07-14T18:28:31.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.888+0000 7f640fcc4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56470ba7d7c0 con 0x56470bb8cf60 2023-07-14T18:28:31.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.888+0000 7f640fcc4b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x56470b961920 con 0x56470bb9c1e0 2023-07-14T18:28:31.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.888+0000 7f640fcc4b80 10 client.0 _ll_register_callbacks cb 0x56470ba7b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:28:31.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.894+0000 7f63fa19b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56470bb8cf60 0x56470bb96c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:31.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.894+0000 7f63fa19b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56470bb8cf60 0x56470bb96c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:46870/0 (socket says 172.21.15.133:46870) 2023-07-14T18:28:31.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.894+0000 7f63fa19b700 1 -- 172.21.15.133:0/567356573 learned_addr learned my addr 172.21.15.133:0/567356573 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:28:31.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.894+0000 7f63fa99c700 1 --2- 172.21.15.133:0/567356573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56470bb9c1e0 0x56470bb9e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:31.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.894+0000 7f63fa19b700 1 -- 172.21.15.133:0/567356573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56470bb9c1e0 msgr2=0x56470bb9e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:28:31.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.894+0000 7f63fa19b700 1 --2- 172.21.15.133:0/567356573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56470bb9c1e0 0x56470bb9e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:31.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.894+0000 7f63fa19b700 1 -- 172.21.15.133:0/567356573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56470bb8d8b0 msgr2=0x56470bb971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:28:31.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.894+0000 7f63fa19b700 1 --2- 172.21.15.133:0/567356573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56470bb8d8b0 0x56470bb971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:31.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.894+0000 7f63fa19b700 1 -- 172.21.15.133:0/567356573 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f41b5780 con 0x56470bb8cf60 2023-07-14T18:28:31.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.895+0000 7f63fa19b700 1 --2- 172.21.15.133:0/567356573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56470bb8cf60 0x56470bb96c70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f63f00098e0 tx=0x7f63f0003a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40c792a4d9bbda18 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:28:31.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.895+0000 7f63ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:28:31.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.915+0000 7f63ddffb700 1 -- 172.21.15.133:0/567356573 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f00052c0 con 0x56470bb8cf60 2023-07-14T18:28:31.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.915+0000 7f63ddffb700 1 -- 172.21.15.133:0/567356573 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f63f0005420 con 0x56470bb8cf60 2023-07-14T18:28:31.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.917+0000 7f640fcc4b80 1 -- 172.21.15.133:0/567356573 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56470ba374f0 con 0x56470bb8cf60 2023-07-14T18:28:31.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.929+0000 7f63ddffb700 1 -- 172.21.15.133:0/567356573 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63f0005620 con 0x56470bb8cf60 2023-07-14T18:28:31.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.932+0000 7f640fcc4b80 10 client.25931 fetch_fsmap learned FSMap version 43 2023-07-14T18:28:31.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.932+0000 7f640fcc4b80 1 -- 172.21.15.133:0/567356573 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x56470b8f4c60 con 0x56470bb8cf60 2023-07-14T18:28:31.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.951+0000 7f63ddffb700 1 -- 172.21.15.133:0/567356573 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56470ba374f0 con 0x56470bb8cf60 2023-07-14T18:28:31.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.951+0000 7f63ddffb700 1 -- 172.21.15.133:0/567356573 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 43) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f63f002a070 con 0x56470bb8cf60 2023-07-14T18:28:31.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.958+0000 7f640fcc4b80 10 client.25931 fetch_fsmap finished waiting for FSMap version 43 2023-07-14T18:28:31.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.958+0000 7f640fcc4b80 10 client.25931 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:28:31.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.958+0000 7f640fcc4b80 20 client.25931 populate_metadata read hostname 'smithi133' 2023-07-14T18:28:31.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.958+0000 7f640fcc4b80 1 --2- 172.21.15.133:0/567356573 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x56470bba2e60 0x56470bba5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:31.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.958+0000 7f640fcc4b80 4 client.25931 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:28:31.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.958+0000 7f640fcc4b80 1 -- 172.21.15.133:0/567356573 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56470bb5bda0 con 0x56470bba2e60 2023-07-14T18:28:31.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.959+0000 7f63f999a700 1 --2- 172.21.15.133:0/567356573 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x56470bba2e60 0x56470bba5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:32.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.991+0000 7f63f999a700 1 --2- 172.21.15.133:0/567356573 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x56470bba2e60 0x56470bba5270 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16bf3d8e6fc80fad server_cookie=7296178bed713040 in_seq=0 out_seq=0 2023-07-14T18:28:32.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:31.992+0000 7f63ddffb700 10 client.25931 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:28:32.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.013+0000 7f63ddffb700 1 -- 172.21.15.133:0/567356573 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x56470bb5bda0 con 0x56470bba2e60 2023-07-14T18:28:32.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.013+0000 7f63ddffb700 10 client.25931 handle_command_reply: tid=0 2023-07-14T18:28:32.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.018+0000 7f640fcc4b80 1 client.25931 shutdown 2023-07-14T18:28:32.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.032+0000 7f640fcc4b80 1 -- 172.21.15.133:0/567356573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56470bb8cf60 msgr2=0x56470bb96c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:28:32.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.032+0000 7f640fcc4b80 1 --2- 172.21.15.133:0/567356573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56470bb8cf60 0x56470bb96c70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f63f00098e0 tx=0x7f63f0003a10 comp rx=0 tx=0).stop 2023-07-14T18:28:32.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.032+0000 7f640fcc4b80 1 -- 172.21.15.133:0/567356573 shutdown_connections 2023-07-14T18:28:32.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.032+0000 7f640fcc4b80 1 --2- 172.21.15.133:0/567356573 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x56470bba2e60 0x56470bba5270 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:32.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.032+0000 7f640fcc4b80 1 --2- 172.21.15.133:0/567356573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56470bb8cf60 0x56470bb96c70 secure :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f63f00098e0 tx=0x7f63f0003a10 comp rx=0 tx=0).stop 2023-07-14T18:28:32.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.032+0000 7f640fcc4b80 1 --2- 172.21.15.133:0/567356573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56470bb9c1e0 0x56470bb9e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:32.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.032+0000 7f640fcc4b80 1 --2- 172.21.15.133:0/567356573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56470bb8d8b0 0x56470bb971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:32.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.032+0000 7f640fcc4b80 1 -- 172.21.15.133:0/567356573 >> 172.21.15.133:0/567356573 conn(0x56470bb82680 msgr2=0x56470bb86650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:28:32.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.033+0000 7f63ddffb700 0 client.25931 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:28:32.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.060+0000 7f640fcc4b80 1 -- 172.21.15.133:0/567356573 shutdown_connections 2023-07-14T18:28:32.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.060+0000 7f640fcc4b80 1 -- 172.21.15.133:0/567356573 wait complete. 2023-07-14T18:28:32.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.071+0000 7f640fcc4b80 20 client.25931 trim_cache size 0 max 16384 2023-07-14T18:28:32.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.108+0000 7f640fcc4b80 1 Processor -- start 2023-07-14T18:28:32.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.123+0000 7f640fcc4b80 1 -- start start 2023-07-14T18:28:32.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.123+0000 7f640fcc4b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56470bb8d8b0 0x56470bb915c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:32.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.123+0000 7f640fcc4b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56470bb9c1e0 0x56470bb91b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:32.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.123+0000 7f640fcc4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56470bb8dd90 0x56470bbb28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:32.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.123+0000 7f640fcc4b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56470bb96150 con 0x56470bb9c1e0 2023-07-14T18:28:32.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.123+0000 7f640fcc4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56470bb5bda0 con 0x56470bb8dd90 2023-07-14T18:28:32.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.123+0000 7f640fcc4b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x56470ba374f0 con 0x56470bb8d8b0 2023-07-14T18:28:32.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.134+0000 7f63fa99c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56470bb8dd90 0x56470bbb28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:32.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.134+0000 7f63fa99c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56470bb8dd90 0x56470bbb28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:46876/0 (socket says 172.21.15.133:46876) 2023-07-14T18:28:32.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.134+0000 7f63fa99c700 1 -- 172.21.15.133:0/2927252749 learned_addr learned my addr 172.21.15.133:0/2927252749 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:28:32.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.142+0000 7f63fa19b700 1 --2- 172.21.15.133:0/2927252749 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56470bb8d8b0 0x56470bb915c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:32.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.151+0000 7f63fa19b700 1 -- 172.21.15.133:0/2927252749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56470bb8dd90 msgr2=0x56470bbb28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:28:32.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.151+0000 7f63fa19b700 1 --2- 172.21.15.133:0/2927252749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56470bb8dd90 0x56470bbb28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:32.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.151+0000 7f63fa19b700 1 -- 172.21.15.133:0/2927252749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56470bb9c1e0 msgr2=0x56470bb91b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:28:32.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.151+0000 7f63fa19b700 1 --2- 172.21.15.133:0/2927252749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56470bb9c1e0 0x56470bb91b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:32.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.151+0000 7f63fa19b700 1 -- 172.21.15.133:0/2927252749 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56470b932250 con 0x56470bb8d8b0 2023-07-14T18:28:32.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.154+0000 7f63fa19b700 1 --2- 172.21.15.133:0/2927252749 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56470bb8d8b0 0x56470bb915c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f63f001c000 tx=0x7f63f0001e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9415dc517e11c780 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:28:32.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.171+0000 7f63e93ff700 1 -- 172.21.15.133:0/2927252749 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f000ece0 con 0x56470bb8d8b0 2023-07-14T18:28:32.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.171+0000 7f63e93ff700 1 -- 172.21.15.133:0/2927252749 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63f001e040 con 0x56470bb8d8b0 2023-07-14T18:28:32.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.171+0000 7f63e93ff700 1 -- 172.21.15.133:0/2927252749 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f0022b10 con 0x56470bb8d8b0 2023-07-14T18:28:32.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.182+0000 7f640fcc4b80 1 -- 172.21.15.133:0/2927252749 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56470bb8d8b0 msgr2=0x56470bb915c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:28:32.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.182+0000 7f640fcc4b80 1 --2- 172.21.15.133:0/2927252749 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56470bb8d8b0 0x56470bb915c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f63f001c000 tx=0x7f63f0001e50 comp rx=0 tx=0).stop 2023-07-14T18:28:32.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.198+0000 7f640fcc4b80 1 -- 172.21.15.133:0/2927252749 shutdown_connections 2023-07-14T18:28:32.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.198+0000 7f640fcc4b80 1 --2- 172.21.15.133:0/2927252749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56470bb8dd90 0x56470bbb28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:32.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.198+0000 7f640fcc4b80 1 --2- 172.21.15.133:0/2927252749 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56470bb8d8b0 0x56470bb915c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:32.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.198+0000 7f640fcc4b80 1 --2- 172.21.15.133:0/2927252749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56470bb9c1e0 0x56470bb91b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:32.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.198+0000 7f640fcc4b80 1 -- 172.21.15.133:0/2927252749 >> 172.21.15.133:0/2927252749 conn(0x56470bb82680 msgr2=0x56470bb95700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:28:32.251 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.212+0000 7f640fcc4b80 1 -- 172.21.15.133:0/2927252749 shutdown_connections 2023-07-14T18:28:32.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.216+0000 7f640fcc4b80 1 -- 172.21.15.133:0/2927252749 wait complete. 2023-07-14T18:28:32.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.217+0000 7f640fcc4b80 1 -- start start 2023-07-14T18:28:32.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.217+0000 7f640fcc4b80 1 Processor -- start 2023-07-14T18:28:32.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.218+0000 7f640fcc4b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56470bb8d8b0 0x56470bbb0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:32.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.218+0000 7f640fcc4b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56470bb8dd90 0x56470bbb08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:32.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.218+0000 7f640fcc4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56470bb9c1e0 0x56470bbb1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:32.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.218+0000 7f640fcc4b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56470bbb1db0 con 0x56470bb8dd90 2023-07-14T18:28:32.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.218+0000 7f640fcc4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56470bbb1f20 con 0x56470bb9c1e0 2023-07-14T18:28:32.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.218+0000 7f640fcc4b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x56470bbb20e0 con 0x56470bb8d8b0 2023-07-14T18:28:32.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.218+0000 7f640fcc4b80 10 client.0 _ll_register_callbacks cb 0x7f63f000bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:28:32.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.227+0000 7f63fa99c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56470bb9c1e0 0x56470bbb1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:32.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.227+0000 7f63fa99c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56470bb9c1e0 0x56470bbb1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:46882/0 (socket says 172.21.15.133:46882) 2023-07-14T18:28:32.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.227+0000 7f63fa99c700 1 -- 172.21.15.133:0/1929946021 learned_addr learned my addr 172.21.15.133:0/1929946021 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:28:32.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.227+0000 7f63fa19b700 1 --2- 172.21.15.133:0/1929946021 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56470bb8d8b0 0x56470bbb0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:32.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.239+0000 7f63fa19b700 1 -- 172.21.15.133:0/1929946021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56470bb9c1e0 msgr2=0x56470bbb1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:28:32.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.239+0000 7f63fa19b700 1 --2- 172.21.15.133:0/1929946021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56470bb9c1e0 0x56470bbb1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:32.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.239+0000 7f63fa19b700 1 -- 172.21.15.133:0/1929946021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56470bb8dd90 msgr2=0x56470bbb08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:28:32.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.239+0000 7f63fa19b700 1 --2- 172.21.15.133:0/1929946021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56470bb8dd90 0x56470bbb08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:32.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.239+0000 7f63fa19b700 1 -- 172.21.15.133:0/1929946021 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56470b8f4c60 con 0x56470bb8d8b0 2023-07-14T18:28:32.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.242+0000 7f63fa19b700 1 --2- 172.21.15.133:0/1929946021 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56470bb8d8b0 0x56470bbb0380 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f63f0003790 tx=0x7f63f000e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a51f3ab8b6161965 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:28:32.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.250+0000 7f63ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:28:32.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.250+0000 7f63ddffb700 1 -- 172.21.15.133:0/1929946021 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f0022cc0 con 0x56470bb8d8b0 2023-07-14T18:28:32.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.250+0000 7f63ddffb700 1 -- 172.21.15.133:0/1929946021 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f63f0022670 con 0x56470bb8d8b0 2023-07-14T18:28:32.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.250+0000 7f640fcc4b80 1 -- 172.21.15.133:0/1929946021 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63e40992b0 con 0x56470bb8d8b0 2023-07-14T18:28:32.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.270+0000 7f63ddffb700 1 -- 172.21.15.133:0/1929946021 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63f0005bc0 con 0x56470bb8d8b0 2023-07-14T18:28:32.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.270+0000 7f640fcc4b80 10 client.25966 fetch_fsmap learned FSMap version 43 2023-07-14T18:28:32.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.270+0000 7f640fcc4b80 1 -- 172.21.15.133:0/1929946021 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x56470b90a2b0 con 0x56470bb8d8b0 2023-07-14T18:28:32.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.286+0000 7f63ddffb700 1 -- 172.21.15.133:0/1929946021 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63e40992b0 con 0x56470bb8d8b0 2023-07-14T18:28:32.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.286+0000 7f63ddffb700 1 -- 172.21.15.133:0/1929946021 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 43) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f63f000b260 con 0x56470bb8d8b0 2023-07-14T18:28:32.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.287+0000 7f640fcc4b80 10 client.25966 fetch_fsmap finished waiting for FSMap version 43 2023-07-14T18:28:32.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.287+0000 7f640fcc4b80 10 client.25966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:28:32.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.287+0000 7f640fcc4b80 20 client.25966 populate_metadata read hostname 'smithi133' 2023-07-14T18:28:32.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.287+0000 7f640fcc4b80 1 --2- 172.21.15.133:0/1929946021 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x56470bbf7960 0x56470bbf9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:28:32.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.287+0000 7f640fcc4b80 4 client.25966 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:28:32.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.287+0000 7f640fcc4b80 1 -- 172.21.15.133:0/1929946021 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56470bad6eb0 con 0x56470bbf7960 2023-07-14T18:28:32.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.289+0000 7f63f999a700 1 --2- 172.21.15.133:0/1929946021 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x56470bbf7960 0x56470bbf9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:28:32.377 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.331+0000 7f63f999a700 1 --2- 172.21.15.133:0/1929946021 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x56470bbf7960 0x56470bbf9d70 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7e0547c7217f1e2 server_cookie=168034c71c116f41 in_seq=0 out_seq=0 2023-07-14T18:28:32.377 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.334+0000 7f63ddffb700 10 client.25966 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:28:32.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.340+0000 7f63ddffb700 1 -- 172.21.15.133:0/1929946021 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x56470bad6eb0 con 0x56470bbf7960 2023-07-14T18:28:32.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.340+0000 7f63ddffb700 10 client.25966 handle_command_reply: tid=0 2023-07-14T18:28:32.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.340+0000 7f640fcc4b80 1 client.25966 shutdown 2023-07-14T18:28:32.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.365+0000 7f640fcc4b80 1 -- 172.21.15.133:0/1929946021 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56470bb8d8b0 msgr2=0x56470bbb0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:28:32.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.365+0000 7f640fcc4b80 1 --2- 172.21.15.133:0/1929946021 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56470bb8d8b0 0x56470bbb0380 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f63f0003790 tx=0x7f63f000e6c0 comp rx=0 tx=0).stop 2023-07-14T18:28:32.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.369+0000 7f640fcc4b80 1 -- 172.21.15.133:0/1929946021 shutdown_connections 2023-07-14T18:28:32.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.369+0000 7f640fcc4b80 1 --2- 172.21.15.133:0/1929946021 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x56470bbf7960 0x56470bbf9d70 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:32.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.369+0000 7f640fcc4b80 1 --2- 172.21.15.133:0/1929946021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56470bb9c1e0 0x56470bbb1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:32.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.369+0000 7f640fcc4b80 1 --2- 172.21.15.133:0/1929946021 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56470bb8d8b0 0x56470bbb0380 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:32.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.369+0000 7f640fcc4b80 1 --2- 172.21.15.133:0/1929946021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56470bb8dd90 0x56470bbb08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:32.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.369+0000 7f640fcc4b80 1 -- 172.21.15.133:0/1929946021 >> 172.21.15.133:0/1929946021 conn(0x56470bb82680 msgr2=0x56470bb95700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:28:32.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.370+0000 7f63ddffb700 0 client.25966 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:28:32.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.381+0000 7f640fcc4b80 1 -- 172.21.15.133:0/1929946021 shutdown_connections 2023-07-14T18:28:32.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.383+0000 7f640fcc4b80 1 -- 172.21.15.133:0/1929946021 wait complete. 2023-07-14T18:28:32.426 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.389+0000 7f640fcc4b80 20 client.25966 trim_cache size 0 max 16384 2023-07-14T18:28:32.431 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:28:32.431 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T18:28:32.431 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:28:32.431 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:28:32.432 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:28:32.432 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:28:32.432 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:28:32.432 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:28:32.432 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:28:32.432 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:28:32.452 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.415+0000 7f63fc3ff700 1 -- 172.21.15.133:0/2949036079 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f63e00a1350 msgr2=0x7f63e00a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:28:32.453 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.415+0000 7f63fc3ff700 1 --2- 172.21.15.133:0/2949036079 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f63e00a1350 0x7f63e00a3800 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f63ec009fd0 tx=0x7f63ec004b10 comp rx=0 tx=0).stop 2023-07-14T18:28:32.453 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.415+0000 7f63fc3ff700 1 -- 172.21.15.133:0/2949036079 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63f4071960 msgr2=0x7f63f4114130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:28:32.453 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.415+0000 7f63fc3ff700 1 --2- 172.21.15.133:0/2949036079 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63f4071960 0x7f63f4114130 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f63e400b500 tx=0x7f63e400b810 comp rx=0 tx=0).stop 2023-07-14T18:28:32.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.418+0000 7f63fc3ff700 1 -- 172.21.15.133:0/2949036079 shutdown_connections 2023-07-14T18:28:32.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.418+0000 7f63fc3ff700 1 --2- 172.21.15.133:0/2949036079 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f63e00a1350 0x7f63e00a3800 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:32.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.418+0000 7f63fc3ff700 1 --2- 172.21.15.133:0/2949036079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63f4071010 0x7f63f4113bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:32.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.418+0000 7f63fc3ff700 1 --2- 172.21.15.133:0/2949036079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f410be90 0x7f63f41b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:32.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.418+0000 7f63fc3ff700 1 --2- 172.21.15.133:0/2949036079 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63f4071960 0x7f63f4114130 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:28:32.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.418+0000 7f63fc3ff700 1 -- 172.21.15.133:0/2949036079 >> 172.21.15.133:0/2949036079 conn(0x7f63f406cbc0 msgr2=0x7f63f406df00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:28:32.477 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.437+0000 7f63fc3ff700 1 -- 172.21.15.133:0/2949036079 shutdown_connections 2023-07-14T18:28:32.483 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:28:32.444+0000 7f63fc3ff700 1 -- 172.21.15.133:0/2949036079 wait complete. 2023-07-14T18:28:32.569 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:28:33.842 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:33 smithi161 ceph-mon[128993]: pgmap v704: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 15 MiB/s wr, 1.10k op/s 2023-07-14T18:28:33.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:33 smithi171 ceph-mon[129241]: pgmap v704: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 15 MiB/s wr, 1.10k op/s 2023-07-14T18:28:33.906 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:33 smithi133 ceph-mon[126384]: pgmap v704: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 15 MiB/s wr, 1.10k op/s 2023-07-14T18:28:34.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:34 smithi133 ceph-mon[126384]: pgmap v705: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 15 MiB/s wr, 1.15k op/s 2023-07-14T18:28:35.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:34 smithi161 ceph-mon[128993]: pgmap v705: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 15 MiB/s wr, 1.15k op/s 2023-07-14T18:28:35.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:34 smithi171 ceph-mon[129241]: pgmap v705: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 15 MiB/s wr, 1.15k op/s 2023-07-14T18:28:35.866 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 550.0 s, 377.5 tps, lat 237.276 ms stddev 270.595 2023-07-14T18:28:37.342 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:36 smithi161 ceph-mon[128993]: pgmap v706: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 14 MiB/s wr, 1.05k op/s 2023-07-14T18:28:37.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:36 smithi171 ceph-mon[129241]: pgmap v706: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 14 MiB/s wr, 1.05k op/s 2023-07-14T18:28:37.399 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:36 smithi133 ceph-mon[126384]: pgmap v706: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 14 MiB/s wr, 1.05k op/s 2023-07-14T18:28:39.842 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:39 smithi161 ceph-mon[128993]: pgmap v707: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 11 MiB/s wr, 920 op/s 2023-07-14T18:28:39.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:39 smithi171 ceph-mon[129241]: pgmap v707: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 11 MiB/s wr, 920 op/s 2023-07-14T18:28:39.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:39 smithi133 ceph-mon[126384]: pgmap v707: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 11 MiB/s wr, 920 op/s 2023-07-14T18:28:40.868 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 555.0 s, 471.4 tps, lat 232.811 ms stddev 296.276 2023-07-14T18:28:41.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:41 smithi161 ceph-mon[128993]: pgmap v708: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 13 MiB/s wr, 1.01k op/s 2023-07-14T18:28:41.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:41 smithi171 ceph-mon[129241]: pgmap v708: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 13 MiB/s wr, 1.01k op/s 2023-07-14T18:28:41.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:41 smithi133 ceph-mon[126384]: pgmap v708: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 13 MiB/s wr, 1.01k op/s 2023-07-14T18:28:43.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:43 smithi161 ceph-mon[128993]: pgmap v709: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 10 MiB/s wr, 778 op/s 2023-07-14T18:28:43.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:43 smithi171 ceph-mon[129241]: pgmap v709: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 10 MiB/s wr, 778 op/s 2023-07-14T18:28:43.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:43 smithi133 ceph-mon[126384]: pgmap v709: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 10 MiB/s wr, 778 op/s 2023-07-14T18:28:45.091 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:44 smithi161 ceph-mon[128993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:28:45.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:44 smithi161 ceph-mon[128993]: pgmap v710: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 12 MiB/s wr, 786 op/s 2023-07-14T18:28:45.106 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:44 smithi171 ceph-mon[129241]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:28:45.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:44 smithi171 ceph-mon[129241]: pgmap v710: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 12 MiB/s wr, 786 op/s 2023-07-14T18:28:45.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:44 smithi133 ceph-mon[126384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:28:45.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:44 smithi133 ceph-mon[126384]: pgmap v710: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 12 MiB/s wr, 786 op/s 2023-07-14T18:28:45.865 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 560.0 s, 679.3 tps, lat 139.545 ms stddev 295.179 2023-07-14T18:28:47.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:47 smithi161 ceph-mon[128993]: pgmap v711: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 12 MiB/s wr, 809 op/s 2023-07-14T18:28:47.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:47 smithi171 ceph-mon[129241]: pgmap v711: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 12 MiB/s wr, 809 op/s 2023-07-14T18:28:47.403 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:47 smithi133 ceph-mon[126384]: pgmap v711: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 12 MiB/s wr, 809 op/s 2023-07-14T18:28:49.841 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:49 smithi161 ceph-mon[128993]: pgmap v712: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 12 MiB/s wr, 837 op/s 2023-07-14T18:28:49.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:49 smithi171 ceph-mon[129241]: pgmap v712: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 12 MiB/s wr, 837 op/s 2023-07-14T18:28:49.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:49 smithi133 ceph-mon[126384]: pgmap v712: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 12 MiB/s wr, 837 op/s 2023-07-14T18:28:50.865 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 565.0 s, 665.2 tps, lat 158.000 ms stddev 317.423 2023-07-14T18:28:51.848 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:51 smithi161 ceph-mon[128993]: pgmap v713: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 14 MiB/s wr, 969 op/s 2023-07-14T18:28:51.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:51 smithi171 ceph-mon[129241]: pgmap v713: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 14 MiB/s wr, 969 op/s 2023-07-14T18:28:51.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:51 smithi133 ceph-mon[126384]: pgmap v713: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 14 MiB/s wr, 969 op/s 2023-07-14T18:28:53.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:53 smithi133 ceph-mon[126384]: pgmap v714: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 15 MiB/s wr, 1.04k op/s 2023-07-14T18:28:54.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:53 smithi161 ceph-mon[128993]: pgmap v714: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 15 MiB/s wr, 1.04k op/s 2023-07-14T18:28:54.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:53 smithi171 ceph-mon[129241]: pgmap v714: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 15 MiB/s wr, 1.04k op/s 2023-07-14T18:28:55.867 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 570.0 s, 586.6 tps, lat 128.441 ms stddev 342.500 2023-07-14T18:28:56.091 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:55 smithi161 ceph-mon[128993]: pgmap v715: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 14 MiB/s wr, 984 op/s 2023-07-14T18:28:56.106 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:55 smithi171 ceph-mon[129241]: pgmap v715: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 14 MiB/s wr, 984 op/s 2023-07-14T18:28:56.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:55 smithi133 ceph-mon[126384]: pgmap v715: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 14 MiB/s wr, 984 op/s 2023-07-14T18:28:57.343 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:57 smithi161 ceph-mon[128993]: pgmap v716: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 16 MiB/s wr, 1.21k op/s 2023-07-14T18:28:57.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:57 smithi171 ceph-mon[129241]: pgmap v716: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 16 MiB/s wr, 1.21k op/s 2023-07-14T18:28:57.398 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:57 smithi133 ceph-mon[126384]: pgmap v716: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 16 MiB/s wr, 1.21k op/s 2023-07-14T18:28:59.841 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:28:59 smithi161 ceph-mon[128993]: pgmap v717: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 15 MiB/s wr, 1.20k op/s 2023-07-14T18:28:59.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:28:59 smithi171 ceph-mon[129241]: pgmap v717: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 15 MiB/s wr, 1.20k op/s 2023-07-14T18:28:59.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:28:59 smithi133 ceph-mon[126384]: pgmap v717: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 15 MiB/s wr, 1.20k op/s 2023-07-14T18:29:00.864 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 575.0 s, 977.5 tps, lat 129.881 ms stddev 274.961 2023-07-14T18:29:01.831 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:01 smithi161 ceph-mon[128993]: pgmap v718: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 16 MiB/s wr, 1.16k op/s 2023-07-14T18:29:01.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:01 smithi171 ceph-mon[129241]: pgmap v718: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 16 MiB/s wr, 1.16k op/s 2023-07-14T18:29:01.896 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:01 smithi133 ceph-mon[126384]: pgmap v718: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 16 MiB/s wr, 1.16k op/s 2023-07-14T18:29:02.570 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:29:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.014+0000 7f34991ff700 1 -- 172.21.15.133:0/3676445675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3494100690 msgr2=0x7f3494106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:29:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.014+0000 7f34991ff700 1 --2- 172.21.15.133:0/3676445675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3494100690 0x7f3494106f50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f348401a9a0 tx=0x7f348401acb0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.015+0000 7f34991ff700 1 -- 172.21.15.133:0/3676445675 shutdown_connections 2023-07-14T18:29:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.015+0000 7f34991ff700 1 --2- 172.21.15.133:0/3676445675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34940feb10 0x7f34940fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.015+0000 7f34991ff700 1 --2- 172.21.15.133:0/3676445675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3494100690 0x7f3494106f50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.015+0000 7f34991ff700 1 --2- 172.21.15.133:0/3676445675 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34940ffd00 0x7f3494100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.015+0000 7f34991ff700 1 -- 172.21.15.133:0/3676445675 >> 172.21.15.133:0/3676445675 conn(0x7f34940fa130 msgr2=0x7f34940fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:29:03.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.022+0000 7f34991ff700 1 -- 172.21.15.133:0/3676445675 shutdown_connections 2023-07-14T18:29:03.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.022+0000 7f34991ff700 1 -- 172.21.15.133:0/3676445675 wait complete. 2023-07-14T18:29:03.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.024+0000 7f34991ff700 1 Processor -- start 2023-07-14T18:29:03.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.024+0000 7f34991ff700 1 -- start start 2023-07-14T18:29:03.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.025+0000 7f34991ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34940feb10 0x7f3494071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:03.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.025+0000 7f34991ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34940ffd00 0x7f34940716b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:03.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.025+0000 7f34991ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3494100690 0x7f349419b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:03.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.025+0000 7f34991ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3494071e10 con 0x7f34940ffd00 2023-07-14T18:29:03.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.025+0000 7f34991ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349419b590 con 0x7f34940feb10 2023-07-14T18:29:03.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.025+0000 7f34991ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349419b750 con 0x7f3494100690 2023-07-14T18:29:03.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.025+0000 7f3492d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34940feb10 0x7f3494071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:03.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.025+0000 7f349359e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3494100690 0x7f349419b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:03.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.025+0000 7f3492d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34940feb10 0x7f3494071170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:58736/0 (socket says 172.21.15.133:58736) 2023-07-14T18:29:03.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.025+0000 7f3492d9d700 1 -- 172.21.15.133:0/1602170642 learned_addr learned my addr 172.21.15.133:0/1602170642 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:29:03.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.025+0000 7f349259c700 1 --2- 172.21.15.133:0/1602170642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34940ffd00 0x7f34940716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:03.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.026+0000 7f3492d9d700 1 -- 172.21.15.133:0/1602170642 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3494100690 msgr2=0x7f349419b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:29:03.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.026+0000 7f3492d9d700 1 --2- 172.21.15.133:0/1602170642 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3494100690 0x7f349419b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.026+0000 7f3492d9d700 1 -- 172.21.15.133:0/1602170642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34940ffd00 msgr2=0x7f34940716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:29:03.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.026+0000 7f3492d9d700 1 --2- 172.21.15.133:0/1602170642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34940ffd00 0x7f34940716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.026+0000 7f3492d9d700 1 -- 172.21.15.133:0/1602170642 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348401a600 con 0x7f34940feb10 2023-07-14T18:29:03.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.026+0000 7f3492d9d700 1 --2- 172.21.15.133:0/1602170642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34940feb10 0x7f3494071170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f348800b550 tx=0x7f348800b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:29:03.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.026+0000 7f347bfff700 1 -- 172.21.15.133:0/1602170642 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3488011830 con 0x7f34940feb10 2023-07-14T18:29:03.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.026+0000 7f347bfff700 1 -- 172.21.15.133:0/1602170642 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3488011e50 con 0x7f34940feb10 2023-07-14T18:29:03.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.026+0000 7f347bfff700 1 -- 172.21.15.133:0/1602170642 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3488010d40 con 0x7f34940feb10 2023-07-14T18:29:03.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.027+0000 7f34991ff700 1 -- 172.21.15.133:0/1602170642 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f349419ba30 con 0x7f34940feb10 2023-07-14T18:29:03.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.027+0000 7f34991ff700 1 -- 172.21.15.133:0/1602170642 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f349419bef0 con 0x7f34940feb10 2023-07-14T18:29:03.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.032+0000 7f34acaadb80 1 Processor -- start 2023-07-14T18:29:03.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.032+0000 7f347bfff700 1 -- 172.21.15.133:0/1602170642 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f3488011990 con 0x7f34940feb10 2023-07-14T18:29:03.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.037+0000 7f34acaadb80 1 -- start start 2023-07-14T18:29:03.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.038+0000 7f34acaadb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f1ab7df60 0x555f1ab7e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:03.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.039+0000 7f34acaadb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1ab7e8b0 0x555f1ab7eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:03.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.039+0000 7f34acaadb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f1ab7f4b0 0x555f1ab7f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:03.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.039+0000 7f34acaadb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1aa82cb0 con 0x555f1ab7df60 2023-07-14T18:29:03.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.039+0000 7f34acaadb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1aad5a40 con 0x555f1ab7e8b0 2023-07-14T18:29:03.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.039+0000 7f34acaadb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1aaceed0 con 0x555f1ab7f4b0 2023-07-14T18:29:03.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.039+0000 7f349259c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f1ab7f4b0 0x555f1ab7f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:03.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.039+0000 7f349259c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f1ab7f4b0 0x555f1ab7f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:45872/0 (socket says 172.21.15.133:45872) 2023-07-14T18:29:03.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.039+0000 7f349259c700 1 -- 172.21.15.133:0/1187500975 learned_addr learned my addr 172.21.15.133:0/1187500975 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:29:03.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.039+0000 7f3492d9d700 1 --2- 172.21.15.133:0/1187500975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1ab7e8b0 0x555f1ab7eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:03.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.040+0000 7f349259c700 1 -- 172.21.15.133:0/1187500975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1ab7e8b0 msgr2=0x555f1ab7eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:29:03.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.040+0000 7f349259c700 1 --2- 172.21.15.133:0/1187500975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1ab7e8b0 0x555f1ab7eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.040+0000 7f349259c700 1 -- 172.21.15.133:0/1187500975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f1ab7df60 msgr2=0x555f1ab7e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:29:03.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.040+0000 7f349259c700 1 --2- 172.21.15.133:0/1187500975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f1ab7df60 0x555f1ab7e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.040+0000 7f349259c700 1 -- 172.21.15.133:0/1187500975 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f1a8fcde0 con 0x555f1ab7f4b0 2023-07-14T18:29:03.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.040+0000 7f349259c700 1 --2- 172.21.15.133:0/1187500975 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f1ab7f4b0 0x555f1ab7f920 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f348c00b580 tx=0x7f348c0096b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80d86dfe7edb56c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:29:03.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.041+0000 7f34797fa700 1 -- 172.21.15.133:0/1187500975 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c010040 con 0x555f1ab7f4b0 2023-07-14T18:29:03.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.041+0000 7f34797fa700 1 -- 172.21.15.133:0/1187500975 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f348c00acb0 con 0x555f1ab7f4b0 2023-07-14T18:29:03.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.041+0000 7f34797fa700 1 -- 172.21.15.133:0/1187500975 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c014bc0 con 0x555f1ab7f4b0 2023-07-14T18:29:03.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.041+0000 7f347bfff700 1 --2- 172.21.15.133:0/1602170642 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f347c0a1250 0x7f347c0a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:03.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.041+0000 7f347bfff700 1 -- 172.21.15.133:0/1602170642 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f34880cc590 con 0x7f34940feb10 2023-07-14T18:29:03.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.042+0000 7f34acaadb80 1 -- 172.21.15.133:0/1187500975 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f1ab7f4b0 msgr2=0x555f1ab7f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:29:03.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.042+0000 7f34acaadb80 1 --2- 172.21.15.133:0/1187500975 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f1ab7f4b0 0x555f1ab7f920 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f348c00b580 tx=0x7f348c0096b0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.042+0000 7f349359e700 1 --2- 172.21.15.133:0/1602170642 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f347c0a1250 0x7f347c0a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:03.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.042+0000 7f34acaadb80 1 -- 172.21.15.133:0/1187500975 shutdown_connections 2023-07-14T18:29:03.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.042+0000 7f34acaadb80 1 --2- 172.21.15.133:0/1187500975 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f1ab7f4b0 0x555f1ab7f920 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.042+0000 7f34acaadb80 1 --2- 172.21.15.133:0/1187500975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f1ab7df60 0x555f1ab7e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.042+0000 7f34acaadb80 1 --2- 172.21.15.133:0/1187500975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1ab7e8b0 0x555f1ab7eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.042+0000 7f34acaadb80 1 -- 172.21.15.133:0/1187500975 >> 172.21.15.133:0/1187500975 conn(0x555f1ab73680 msgr2=0x555f1ab789d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:29:03.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.043+0000 7f34acaadb80 1 -- 172.21.15.133:0/1187500975 shutdown_connections 2023-07-14T18:29:03.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.043+0000 7f34acaadb80 1 -- 172.21.15.133:0/1187500975 wait complete. 2023-07-14T18:29:03.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.043+0000 7f34acaadb80 1 -- start start 2023-07-14T18:29:03.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.044+0000 7f34acaadb80 1 Processor -- start 2023-07-14T18:29:03.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.044+0000 7f349359e700 1 --2- 172.21.15.133:0/1602170642 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f347c0a1250 0x7f347c0a3700 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f348401a360 tx=0x7f348402a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:29:03.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.045+0000 7f34acaadb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1ab7df60 0x555f1ab87960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:03.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.045+0000 7f34acaadb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f1ab7e8b0 0x555f1ab87ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:03.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.045+0000 7f34acaadb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f1ab7f4b0 0x555f1ab8eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:03.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.045+0000 7f34acaadb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1aa6eac0 con 0x555f1ab7f4b0 2023-07-14T18:29:03.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.045+0000 7f34acaadb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1aa6e7c0 con 0x555f1ab7df60 2023-07-14T18:29:03.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.045+0000 7f34acaadb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1a952920 con 0x555f1ab7e8b0 2023-07-14T18:29:03.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.045+0000 7f34acaadb80 10 client.0 _ll_register_callbacks cb 0x555f1aa6c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:29:03.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.045+0000 7f349359e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f1ab7f4b0 0x555f1ab8eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:03.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.045+0000 7f349359e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f1ab7f4b0 0x555f1ab8eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46892/0 (socket says 172.21.15.133:46892) 2023-07-14T18:29:03.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.045+0000 7f349359e700 1 -- 172.21.15.133:0/2814450102 learned_addr learned my addr 172.21.15.133:0/2814450102 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:29:03.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.046+0000 7f349259c700 1 --2- 172.21.15.133:0/2814450102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f1ab7e8b0 0x555f1ab87ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:03.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.046+0000 7f349359e700 1 -- 172.21.15.133:0/2814450102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f1ab7e8b0 msgr2=0x555f1ab87ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:29:03.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.046+0000 7f349359e700 1 --2- 172.21.15.133:0/2814450102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f1ab7e8b0 0x555f1ab87ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.046+0000 7f349359e700 1 -- 172.21.15.133:0/2814450102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1ab7df60 msgr2=0x555f1ab87960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:29:03.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.046+0000 7f349359e700 1 --2- 172.21.15.133:0/2814450102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1ab7df60 0x555f1ab87960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.046+0000 7f349359e700 1 -- 172.21.15.133:0/2814450102 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f348c009d60 con 0x555f1ab7f4b0 2023-07-14T18:29:03.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.047+0000 7f349359e700 1 --2- 172.21.15.133:0/2814450102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f1ab7f4b0 0x555f1ab8eee0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3484026000 tx=0x7f348401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba08995c2d202f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:29:03.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.047+0000 7f3476ff5700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T18:29:03.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.048+0000 7f3476ff5700 1 -- 172.21.15.133:0/2814450102 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34840283f0 con 0x555f1ab7f4b0 2023-07-14T18:29:03.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.048+0000 7f3476ff5700 1 -- 172.21.15.133:0/2814450102 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3484028550 con 0x555f1ab7f4b0 2023-07-14T18:29:03.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.048+0000 7f34acaadb80 1 -- 172.21.15.133:0/2814450102 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f1aa284f0 con 0x555f1ab7f4b0 2023-07-14T18:29:03.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.049+0000 7f3476ff5700 1 -- 172.21.15.133:0/2814450102 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3484028710 con 0x555f1ab7f4b0 2023-07-14T18:29:03.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.051+0000 7f34acaadb80 10 client.15846 fetch_fsmap learned FSMap version 44 2023-07-14T18:29:03.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.051+0000 7f34acaadb80 1 -- 172.21.15.133:0/2814450102 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x555f1a8e5c60 con 0x555f1ab7f4b0 2023-07-14T18:29:03.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.054+0000 7f3476ff5700 1 -- 172.21.15.133:0/2814450102 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555f1aa284f0 con 0x555f1ab7f4b0 2023-07-14T18:29:03.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.054+0000 7f3476ff5700 1 -- 172.21.15.133:0/2814450102 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 44) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3484034890 con 0x555f1ab7f4b0 2023-07-14T18:29:03.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.054+0000 7f34acaadb80 10 client.15846 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T18:29:03.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.054+0000 7f34acaadb80 10 client.15846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:29:03.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.055+0000 7f34acaadb80 20 client.15846 populate_metadata read hostname 'smithi133' 2023-07-14T18:29:03.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.055+0000 7f34acaadb80 1 --2- 172.21.15.133:0/2814450102 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x555f1ab93740 0x555f1ab95b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:03.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.055+0000 7f34acaadb80 4 client.15846 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:29:03.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.055+0000 7f34acaadb80 1 -- 172.21.15.133:0/2814450102 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f1ab4cda0 con 0x555f1ab93740 2023-07-14T18:29:03.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.055+0000 7f3492d9d700 1 --2- 172.21.15.133:0/2814450102 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x555f1ab93740 0x555f1ab95b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:03.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.056+0000 7f3492d9d700 1 --2- 172.21.15.133:0/2814450102 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x555f1ab93740 0x555f1ab95b70 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4169db6813e0edc2 server_cookie=51bfb85c3cc3fab1 in_seq=0 out_seq=0 2023-07-14T18:29:03.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.056+0000 7f3476ff5700 10 client.15846 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:29:03.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.057+0000 7f3476ff5700 1 -- 172.21.15.133:0/2814450102 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x555f1ab4cda0 con 0x555f1ab93740 2023-07-14T18:29:03.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.057+0000 7f3476ff5700 10 client.15846 handle_command_reply: tid=0 2023-07-14T18:29:03.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.058+0000 7f34acaadb80 1 client.15846 shutdown 2023-07-14T18:29:03.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.060+0000 7f34acaadb80 1 -- 172.21.15.133:0/2814450102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f1ab7f4b0 msgr2=0x555f1ab8eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:29:03.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.060+0000 7f34acaadb80 1 --2- 172.21.15.133:0/2814450102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f1ab7f4b0 0x555f1ab8eee0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3484026000 tx=0x7f348401acb0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.060+0000 7f34acaadb80 1 -- 172.21.15.133:0/2814450102 shutdown_connections 2023-07-14T18:29:03.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.060+0000 7f34acaadb80 1 --2- 172.21.15.133:0/2814450102 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x555f1ab93740 0x555f1ab95b70 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.060+0000 7f34acaadb80 1 --2- 172.21.15.133:0/2814450102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1ab7df60 0x555f1ab87960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.060+0000 7f34acaadb80 1 --2- 172.21.15.133:0/2814450102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f1ab7f4b0 0x555f1ab8eee0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.060+0000 7f34acaadb80 1 --2- 172.21.15.133:0/2814450102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f1ab7e8b0 0x555f1ab87ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.060+0000 7f34acaadb80 1 -- 172.21.15.133:0/2814450102 >> 172.21.15.133:0/2814450102 conn(0x555f1ab73680 msgr2=0x555f1ab77650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:29:03.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.060+0000 7f3476ff5700 0 client.15846 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:29:03.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.060+0000 7f34acaadb80 1 -- 172.21.15.133:0/2814450102 shutdown_connections 2023-07-14T18:29:03.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.060+0000 7f34acaadb80 1 -- 172.21.15.133:0/2814450102 wait complete. 2023-07-14T18:29:03.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.061+0000 7f34acaadb80 20 client.15846 trim_cache size 0 max 16384 2023-07-14T18:29:03.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.094+0000 7f34acaadb80 1 Processor -- start 2023-07-14T18:29:03.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.094+0000 7f34acaadb80 1 -- start start 2023-07-14T18:29:03.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.094+0000 7f34acaadb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1ab7df60 0x555f1ab85840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:03.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.094+0000 7f34acaadb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f1ab7e8b0 0x555f1ab8ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:03.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.094+0000 7f34acaadb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f1ab7f4b0 0x555f1ab82390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:03.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.094+0000 7f34acaadb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1ab87530 con 0x555f1ab7f4b0 2023-07-14T18:29:03.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.094+0000 7f34acaadb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1ab798b0 con 0x555f1ab7df60 2023-07-14T18:29:03.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.094+0000 7f34acaadb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1ab90b70 con 0x555f1ab7e8b0 2023-07-14T18:29:03.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.095+0000 7f3492d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1ab7df60 0x555f1ab85840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:03.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.095+0000 7f349259c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f1ab7e8b0 0x555f1ab8ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:03.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.095+0000 7f3492d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1ab7df60 0x555f1ab85840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:58750/0 (socket says 172.21.15.133:58750) 2023-07-14T18:29:03.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.095+0000 7f349259c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f1ab7e8b0 0x555f1ab8ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:45878/0 (socket says 172.21.15.133:45878) 2023-07-14T18:29:03.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.095+0000 7f3492d9d700 1 -- 172.21.15.133:0/1743532505 learned_addr learned my addr 172.21.15.133:0/1743532505 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:29:03.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.095+0000 7f349359e700 1 --2- 172.21.15.133:0/1743532505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f1ab7f4b0 0x555f1ab82390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:03.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.095+0000 7f349259c700 1 -- 172.21.15.133:0/1743532505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1ab7df60 msgr2=0x555f1ab85840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:29:03.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.095+0000 7f349259c700 1 --2- 172.21.15.133:0/1743532505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1ab7df60 0x555f1ab85840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.095+0000 7f349259c700 1 -- 172.21.15.133:0/1743532505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f1ab7f4b0 msgr2=0x555f1ab82390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:29:03.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.095+0000 7f349259c700 1 --2- 172.21.15.133:0/1743532505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f1ab7f4b0 0x555f1ab82390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.095+0000 7f349259c700 1 -- 172.21.15.133:0/1743532505 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f1a923250 con 0x555f1ab7e8b0 2023-07-14T18:29:03.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.095+0000 7f3492d9d700 1 --2- 172.21.15.133:0/1743532505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1ab7df60 0x555f1ab85840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:29:03.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.096+0000 7f349259c700 1 --2- 172.21.15.133:0/1743532505 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f1ab7e8b0 0x555f1ab8ee00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f348c00b550 tx=0x7f348c009cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63e3381aa56575dd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:29:03.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.096+0000 7f3479ffb700 1 -- 172.21.15.133:0/1743532505 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c010040 con 0x555f1ab7e8b0 2023-07-14T18:29:03.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.096+0000 7f3479ffb700 1 -- 172.21.15.133:0/1743532505 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f348c00a5f0 con 0x555f1ab7e8b0 2023-07-14T18:29:03.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.096+0000 7f3479ffb700 1 -- 172.21.15.133:0/1743532505 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c004560 con 0x555f1ab7e8b0 2023-07-14T18:29:03.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.097+0000 7f34acaadb80 1 -- 172.21.15.133:0/1743532505 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f1ab7e8b0 msgr2=0x555f1ab8ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:29:03.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.097+0000 7f34acaadb80 1 --2- 172.21.15.133:0/1743532505 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f1ab7e8b0 0x555f1ab8ee00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f348c00b550 tx=0x7f348c009cb0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.097+0000 7f34acaadb80 1 -- 172.21.15.133:0/1743532505 shutdown_connections 2023-07-14T18:29:03.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.097+0000 7f34acaadb80 1 --2- 172.21.15.133:0/1743532505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1ab7df60 0x555f1ab85840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.097+0000 7f34acaadb80 1 --2- 172.21.15.133:0/1743532505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f1ab7f4b0 0x555f1ab82390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.097+0000 7f34acaadb80 1 --2- 172.21.15.133:0/1743532505 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f1ab7e8b0 0x555f1ab8ee00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.097+0000 7f34acaadb80 1 -- 172.21.15.133:0/1743532505 >> 172.21.15.133:0/1743532505 conn(0x555f1ab73680 msgr2=0x555f1ab95750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:29:03.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.097+0000 7f34acaadb80 1 -- 172.21.15.133:0/1743532505 shutdown_connections 2023-07-14T18:29:03.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.097+0000 7f34acaadb80 1 -- 172.21.15.133:0/1743532505 wait complete. 2023-07-14T18:29:03.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.097+0000 7f34acaadb80 1 -- start start 2023-07-14T18:29:03.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.098+0000 7f34acaadb80 1 Processor -- start 2023-07-14T18:29:03.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.098+0000 7f34acaadb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f1ab7df60 0x555f1ab8e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:03.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.098+0000 7f34acaadb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1ab7e8b0 0x555f1ab8ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:03.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.098+0000 7f34acaadb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f1ab7f4b0 0x555f1ab8b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:03.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.099+0000 7f34acaadb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1ab8bd20 con 0x555f1ab7f4b0 2023-07-14T18:29:03.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.099+0000 7f34acaadb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1ab8be90 con 0x555f1ab7e8b0 2023-07-14T18:29:03.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.099+0000 7f34acaadb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1ab8c050 con 0x555f1ab7df60 2023-07-14T18:29:03.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.099+0000 7f34acaadb80 10 client.0 _ll_register_callbacks cb 0x7f3484004420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:29:03.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.099+0000 7f349259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1ab7e8b0 0x555f1ab8ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:03.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.099+0000 7f349259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1ab7e8b0 0x555f1ab8ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:58754/0 (socket says 172.21.15.133:58754) 2023-07-14T18:29:03.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.099+0000 7f349259c700 1 -- 172.21.15.133:0/4050969867 learned_addr learned my addr 172.21.15.133:0/4050969867 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:29:03.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.099+0000 7f3492d9d700 1 --2- 172.21.15.133:0/4050969867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f1ab7df60 0x555f1ab8e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:03.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.099+0000 7f349259c700 1 -- 172.21.15.133:0/4050969867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f1ab7df60 msgr2=0x555f1ab8e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:29:03.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.099+0000 7f349259c700 1 --2- 172.21.15.133:0/4050969867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f1ab7df60 0x555f1ab8e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.099+0000 7f349259c700 1 -- 172.21.15.133:0/4050969867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f1ab7f4b0 msgr2=0x555f1ab8b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:29:03.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.099+0000 7f349259c700 1 --2- 172.21.15.133:0/4050969867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f1ab7f4b0 0x555f1ab8b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.099+0000 7f349259c700 1 -- 172.21.15.133:0/4050969867 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f349419bef0 con 0x555f1ab7e8b0 2023-07-14T18:29:03.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.100+0000 7f349259c700 1 --2- 172.21.15.133:0/4050969867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1ab7e8b0 0x555f1ab8ee60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f348c009980 tx=0x7f348c003cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59d9964d50d07daa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:29:03.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.100+0000 7f3476ff5700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:29:03.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.101+0000 7f3476ff5700 1 -- 172.21.15.133:0/4050969867 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c010040 con 0x555f1ab7e8b0 2023-07-14T18:29:03.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.101+0000 7f3476ff5700 1 -- 172.21.15.133:0/4050969867 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f348c0042e0 con 0x555f1ab7e8b0 2023-07-14T18:29:03.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.102+0000 7f34acaadb80 1 -- 172.21.15.133:0/4050969867 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f348c00acb0 con 0x555f1ab7e8b0 2023-07-14T18:29:03.139 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.105+0000 7f3476ff5700 1 -- 172.21.15.133:0/4050969867 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f348c0044c0 con 0x555f1ab7e8b0 2023-07-14T18:29:03.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.105+0000 7f34acaadb80 10 client.25949 fetch_fsmap learned FSMap version 44 2023-07-14T18:29:03.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.105+0000 7f34acaadb80 1 -- 172.21.15.133:0/4050969867 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x555f1a8fcde0 con 0x555f1ab7e8b0 2023-07-14T18:29:03.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.106+0000 7f3476ff5700 1 -- 172.21.15.133:0/4050969867 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f348c00acb0 con 0x555f1ab7e8b0 2023-07-14T18:29:03.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.106+0000 7f3476ff5700 1 -- 172.21.15.133:0/4050969867 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 44) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f348c028ab0 con 0x555f1ab7e8b0 2023-07-14T18:29:03.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.106+0000 7f34acaadb80 10 client.25949 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T18:29:03.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.106+0000 7f34acaadb80 10 client.25949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:29:03.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.106+0000 7f34acaadb80 20 client.25949 populate_metadata read hostname 'smithi133' 2023-07-14T18:29:03.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.106+0000 7f34acaadb80 1 --2- 172.21.15.133:0/4050969867 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x555f1abe8820 0x555f1abeac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:03.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.106+0000 7f34acaadb80 4 client.25949 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:29:03.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.106+0000 7f34acaadb80 1 -- 172.21.15.133:0/4050969867 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555f1aac7eb0 con 0x555f1abe8820 2023-07-14T18:29:03.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.108+0000 7f3492d9d700 1 --2- 172.21.15.133:0/4050969867 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x555f1abe8820 0x555f1abeac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:03.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.110+0000 7f3492d9d700 1 --2- 172.21.15.133:0/4050969867 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x555f1abe8820 0x555f1abeac30 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc0a49405e78a057 server_cookie=5fcb4804369e87a9 in_seq=0 out_seq=0 2023-07-14T18:29:03.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.110+0000 7f3476ff5700 10 client.25949 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:29:03.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.110+0000 7f3476ff5700 1 -- 172.21.15.133:0/4050969867 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x555f1aac7eb0 con 0x555f1abe8820 2023-07-14T18:29:03.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.110+0000 7f3476ff5700 10 client.25949 handle_command_reply: tid=0 2023-07-14T18:29:03.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.110+0000 7f34acaadb80 1 client.25949 shutdown 2023-07-14T18:29:03.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.110+0000 7f34acaadb80 1 -- 172.21.15.133:0/4050969867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1ab7e8b0 msgr2=0x555f1ab8ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:29:03.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.110+0000 7f34acaadb80 1 --2- 172.21.15.133:0/4050969867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1ab7e8b0 0x555f1ab8ee60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f348c009980 tx=0x7f348c003cf0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.111+0000 7f34acaadb80 1 -- 172.21.15.133:0/4050969867 shutdown_connections 2023-07-14T18:29:03.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.111+0000 7f34acaadb80 1 --2- 172.21.15.133:0/4050969867 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x555f1abe8820 0x555f1abeac30 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.111+0000 7f34acaadb80 1 --2- 172.21.15.133:0/4050969867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f1ab7f4b0 0x555f1ab8b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.111+0000 7f34acaadb80 1 --2- 172.21.15.133:0/4050969867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555f1ab7df60 0x555f1ab8e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.111+0000 7f34acaadb80 1 --2- 172.21.15.133:0/4050969867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1ab7e8b0 0x555f1ab8ee60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.111+0000 7f34acaadb80 1 -- 172.21.15.133:0/4050969867 >> 172.21.15.133:0/4050969867 conn(0x555f1ab73680 msgr2=0x555f1ab85840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:29:03.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.111+0000 7f3476ff5700 0 client.25949 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:29:03.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.111+0000 7f34acaadb80 1 -- 172.21.15.133:0/4050969867 shutdown_connections 2023-07-14T18:29:03.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.111+0000 7f34acaadb80 1 -- 172.21.15.133:0/4050969867 wait complete. 2023-07-14T18:29:03.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.111+0000 7f34acaadb80 20 client.25949 trim_cache size 0 max 16384 2023-07-14T18:29:03.147 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:29:03.147 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T18:29:03.148 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:29:03.148 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:29:03.148 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:29:03.148 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:29:03.148 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:29:03.148 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:29:03.148 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:29:03.149 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:29:03.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.112+0000 7f34991ff700 1 -- 172.21.15.133:0/1602170642 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f347c0a1250 msgr2=0x7f347c0a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:29:03.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.112+0000 7f34991ff700 1 --2- 172.21.15.133:0/1602170642 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f347c0a1250 0x7f347c0a3700 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f348401a360 tx=0x7f348402a040 comp rx=0 tx=0).stop 2023-07-14T18:29:03.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.112+0000 7f34991ff700 1 -- 172.21.15.133:0/1602170642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34940feb10 msgr2=0x7f3494071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:29:03.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.112+0000 7f34991ff700 1 --2- 172.21.15.133:0/1602170642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34940feb10 0x7f3494071170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f348800b550 tx=0x7f348800b860 comp rx=0 tx=0).stop 2023-07-14T18:29:03.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.112+0000 7f34991ff700 1 -- 172.21.15.133:0/1602170642 shutdown_connections 2023-07-14T18:29:03.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.112+0000 7f34991ff700 1 --2- 172.21.15.133:0/1602170642 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f347c0a1250 0x7f347c0a3700 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.112+0000 7f34991ff700 1 --2- 172.21.15.133:0/1602170642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34940feb10 0x7f3494071170 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.112+0000 7f34991ff700 1 --2- 172.21.15.133:0/1602170642 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3494100690 0x7f349419b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.112+0000 7f34991ff700 1 --2- 172.21.15.133:0/1602170642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34940ffd00 0x7f34940716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:03.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.112+0000 7f34991ff700 1 -- 172.21.15.133:0/1602170642 >> 172.21.15.133:0/1602170642 conn(0x7f34940fa130 msgr2=0x7f34941031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:29:03.151 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.112+0000 7f34991ff700 1 -- 172.21.15.133:0/1602170642 shutdown_connections 2023-07-14T18:29:03.151 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:03.112+0000 7f34991ff700 1 -- 172.21.15.133:0/1602170642 wait complete. 2023-07-14T18:29:03.172 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:29:03.551 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:03 smithi133 ceph-mon[126384]: pgmap v719: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 16 MiB/s wr, 1.03k op/s 2023-07-14T18:29:03.846 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:03 smithi161 ceph-mon[128993]: pgmap v719: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 16 MiB/s wr, 1.03k op/s 2023-07-14T18:29:03.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:03 smithi171 ceph-mon[129241]: pgmap v719: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 16 MiB/s wr, 1.03k op/s 2023-07-14T18:29:05.867 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 580.0 s, 726.7 tps, lat 90.029 ms stddev 166.327 2023-07-14T18:29:05.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:05 smithi133 ceph-mon[126384]: pgmap v720: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 14 MiB/s wr, 820 op/s 2023-07-14T18:29:06.090 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:05 smithi161 ceph-mon[128993]: pgmap v720: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 14 MiB/s wr, 820 op/s 2023-07-14T18:29:06.105 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:05 smithi171 ceph-mon[129241]: pgmap v720: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 14 MiB/s wr, 820 op/s 2023-07-14T18:29:07.340 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:07 smithi161 ceph-mon[128993]: pgmap v721: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 18 MiB/s wr, 1.30k op/s 2023-07-14T18:29:07.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:07 smithi171 ceph-mon[129241]: pgmap v721: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 18 MiB/s wr, 1.30k op/s 2023-07-14T18:29:07.414 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:07 smithi133 ceph-mon[126384]: pgmap v721: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 18 MiB/s wr, 1.30k op/s 2023-07-14T18:29:09.840 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:09 smithi161 ceph-mon[128993]: pgmap v722: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 18 MiB/s wr, 1.28k op/s 2023-07-14T18:29:09.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:09 smithi171 ceph-mon[129241]: pgmap v722: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 18 MiB/s wr, 1.28k op/s 2023-07-14T18:29:09.896 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:09 smithi133 ceph-mon[126384]: pgmap v722: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 18 MiB/s wr, 1.28k op/s 2023-07-14T18:29:10.867 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 585.0 s, 498.8 tps, lat 265.507 ms stddev 572.609 2023-07-14T18:29:11.840 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:11 smithi161 ceph-mon[128993]: pgmap v723: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 17 MiB/s wr, 1.19k op/s 2023-07-14T18:29:11.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:11 smithi171 ceph-mon[129241]: pgmap v723: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 17 MiB/s wr, 1.19k op/s 2023-07-14T18:29:11.895 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:11 smithi133 ceph-mon[126384]: pgmap v723: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 17 MiB/s wr, 1.19k op/s 2023-07-14T18:29:14.091 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:13 smithi161 ceph-mon[128993]: pgmap v724: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 19 MiB/s wr, 1.43k op/s 2023-07-14T18:29:14.106 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:13 smithi171 ceph-mon[129241]: pgmap v724: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 19 MiB/s wr, 1.43k op/s 2023-07-14T18:29:14.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:13 smithi133 ceph-mon[126384]: pgmap v724: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 19 MiB/s wr, 1.43k op/s 2023-07-14T18:29:15.863 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 590.0 s, 708.0 tps, lat 137.930 ms stddev 367.755 2023-07-14T18:29:16.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:15 smithi161 ceph-mon[128993]: pgmap v725: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 16 MiB/s wr, 1.33k op/s 2023-07-14T18:29:16.090 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:15 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:29:16.090 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:15 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:29:16.090 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:15 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:29:16.090 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:15 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:29:16.090 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:15 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:29:16.105 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:15 smithi171 ceph-mon[129241]: pgmap v725: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 16 MiB/s wr, 1.33k op/s 2023-07-14T18:29:16.105 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:15 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:29:16.105 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:15 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:29:16.105 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:15 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:29:16.105 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:15 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:29:16.105 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:15 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:29:16.146 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:15 smithi133 ceph-mon[126384]: pgmap v725: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 16 MiB/s wr, 1.33k op/s 2023-07-14T18:29:16.146 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:15 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:29:16.146 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:15 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:29:16.146 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:15 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:29:16.146 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:15 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:29:16.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:15 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:29:17.339 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:17 smithi161 ceph-mon[128993]: pgmap v726: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 19 MiB/s wr, 1.65k op/s 2023-07-14T18:29:17.340 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:17 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:29:17.340 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:17 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:29:17.340 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:17 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:29:17.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:17 smithi171 ceph-mon[129241]: pgmap v726: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 19 MiB/s wr, 1.65k op/s 2023-07-14T18:29:17.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:17 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:29:17.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:17 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:29:17.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:17 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:29:17.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:17 smithi133 ceph-mon[126384]: pgmap v726: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 19 MiB/s wr, 1.65k op/s 2023-07-14T18:29:17.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:17 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:29:17.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:17 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:29:17.401 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:17 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:29:19.842 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:19 smithi161 ceph-mon[128993]: pgmap v727: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 17 MiB/s wr, 1.30k op/s 2023-07-14T18:29:19.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:19 smithi171 ceph-mon[129241]: pgmap v727: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 17 MiB/s wr, 1.30k op/s 2023-07-14T18:29:19.906 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:19 smithi133 ceph-mon[126384]: pgmap v727: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 17 MiB/s wr, 1.30k op/s 2023-07-14T18:29:20.865 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 595.0 s, 775.5 tps, lat 130.034 ms stddev 315.822 2023-07-14T18:29:21.840 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:21 smithi161 ceph-mon[128993]: pgmap v728: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 15 MiB/s wr, 1.12k op/s 2023-07-14T18:29:21.854 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:21 smithi171 ceph-mon[129241]: pgmap v728: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 15 MiB/s wr, 1.12k op/s 2023-07-14T18:29:21.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:21 smithi133 ceph-mon[126384]: pgmap v728: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 15 MiB/s wr, 1.12k op/s 2023-07-14T18:29:24.090 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:23 smithi161 ceph-mon[128993]: pgmap v729: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 18 MiB/s wr, 1.41k op/s 2023-07-14T18:29:24.106 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:23 smithi171 ceph-mon[129241]: pgmap v729: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 18 MiB/s wr, 1.41k op/s 2023-07-14T18:29:24.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:23 smithi133 ceph-mon[126384]: pgmap v729: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 18 MiB/s wr, 1.41k op/s 2023-07-14T18:29:25.865 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 600.0 s, 298.3 tps, lat 340.849 ms stddev 514.512 2023-07-14T18:29:26.090 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:25 smithi161 ceph-mon[128993]: pgmap v730: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 15 MiB/s wr, 1.12k op/s 2023-07-14T18:29:26.104 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:25 smithi171 ceph-mon[129241]: pgmap v730: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 15 MiB/s wr, 1.12k op/s 2023-07-14T18:29:26.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:25 smithi133 ceph-mon[126384]: pgmap v730: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 15 MiB/s wr, 1.12k op/s 2023-07-14T18:29:27.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:27 smithi161 ceph-mon[128993]: pgmap v731: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 17 MiB/s wr, 1.29k op/s 2023-07-14T18:29:27.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:27 smithi171 ceph-mon[129241]: pgmap v731: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 17 MiB/s wr, 1.29k op/s 2023-07-14T18:29:27.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:27 smithi133 ceph-mon[126384]: pgmap v731: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 17 MiB/s wr, 1.29k op/s 2023-07-14T18:29:29.839 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:29 smithi161 ceph-mon[128993]: pgmap v732: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-07-14T18:29:29.854 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:29 smithi171 ceph-mon[129241]: pgmap v732: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-07-14T18:29:29.906 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:29 smithi133 ceph-mon[126384]: pgmap v732: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-07-14T18:29:30.863 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 605.0 s, 443.5 tps, lat 228.265 ms stddev 256.958 2023-07-14T18:29:31.841 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:31 smithi161 ceph-mon[128993]: pgmap v733: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 12 MiB/s wr, 938 op/s 2023-07-14T18:29:31.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:31 smithi171 ceph-mon[129241]: pgmap v733: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 12 MiB/s wr, 938 op/s 2023-07-14T18:29:31.896 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:31 smithi133 ceph-mon[126384]: pgmap v733: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 12 MiB/s wr, 938 op/s 2023-07-14T18:29:33.173 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:29:33.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.721+0000 7f5b365ff700 1 -- 172.21.15.133:0/1976061177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b30071010 msgr2=0x7f5b30071420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:29:33.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.721+0000 7f5b365ff700 1 --2- 172.21.15.133:0/1976061177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b30071010 0x7f5b30071420 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5b24009990 tx=0x7f5b24009ca0 comp rx=0 tx=0).stop 2023-07-14T18:29:33.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.732+0000 7f5b365ff700 1 -- 172.21.15.133:0/1976061177 shutdown_connections 2023-07-14T18:29:33.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.732+0000 7f5b365ff700 1 --2- 172.21.15.133:0/1976061177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b3010be90 0x7f5b3010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:33.766 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.732+0000 7f5b365ff700 1 --2- 172.21.15.133:0/1976061177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b30071010 0x7f5b30071420 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:33.766 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.732+0000 7f5b365ff700 1 --2- 172.21.15.133:0/1976061177 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b30071960 0x7f5b3010b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:33.766 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.732+0000 7f5b365ff700 1 -- 172.21.15.133:0/1976061177 >> 172.21.15.133:0/1976061177 conn(0x7f5b3006cbc0 msgr2=0x7f5b3006eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:29:33.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.761+0000 7f5b365ff700 1 -- 172.21.15.133:0/1976061177 shutdown_connections 2023-07-14T18:29:33.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.769+0000 7f5b365ff700 1 -- 172.21.15.133:0/1976061177 wait complete. 2023-07-14T18:29:33.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.771+0000 7f5b365ff700 1 Processor -- start 2023-07-14T18:29:33.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.774+0000 7f5b365ff700 1 -- start start 2023-07-14T18:29:33.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.775+0000 7f5b365ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b30071010 0x7f5b301a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:33.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.775+0000 7f5b365ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b30071960 0x7f5b301a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:33.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.775+0000 7f5b365ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b3010be90 0x7f5b301abd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:33.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.775+0000 7f5b365ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b301ac320 con 0x7f5b3010be90 2023-07-14T18:29:33.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.775+0000 7f5b365ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3010ed70 con 0x7f5b30071010 2023-07-14T18:29:33.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.775+0000 7f5b365ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3010ef00 con 0x7f5b30071960 2023-07-14T18:29:33.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.776+0000 7f5b2f7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b30071960 0x7f5b301a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:33.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.776+0000 7f5b2f7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b30071960 0x7f5b301a4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:60254/0 (socket says 172.21.15.133:60254) 2023-07-14T18:29:33.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.776+0000 7f5b2f7fe700 1 -- 172.21.15.133:0/75042381 learned_addr learned my addr 172.21.15.133:0/75042381 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:29:33.815 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.781+0000 7f5b2f7fe700 1 -- 172.21.15.133:0/75042381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b30071010 msgr2=0x7f5b301a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:29:33.815 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.781+0000 7f5b2f7fe700 1 --2- 172.21.15.133:0/75042381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b30071010 0x7f5b301a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:33.815 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.782+0000 7f5b2f7fe700 1 -- 172.21.15.133:0/75042381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b3010be90 msgr2=0x7f5b301abd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:29:33.815 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.782+0000 7f5b2f7fe700 1 --2- 172.21.15.133:0/75042381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b3010be90 0x7f5b301abd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:33.815 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.782+0000 7f5b2f7fe700 1 -- 172.21.15.133:0/75042381 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b240095f0 con 0x7f5b30071960 2023-07-14T18:29:33.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.800+0000 7f5b2f7fe700 1 --2- 172.21.15.133:0/75042381 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b30071960 0x7f5b301a4ce0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5b28007f10 tx=0x7f5b2800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:29:33.839 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:33 smithi161 ceph-mon[128993]: pgmap v734: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 14 MiB/s wr, 1.04k op/s 2023-07-14T18:29:33.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.802+0000 7f5b2d7fa700 1 -- 172.21.15.133:0/75042381 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b28017070 con 0x7f5b30071960 2023-07-14T18:29:33.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.802+0000 7f5b2d7fa700 1 -- 172.21.15.133:0/75042381 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b2800f040 con 0x7f5b30071960 2023-07-14T18:29:33.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.802+0000 7f5b2d7fa700 1 -- 172.21.15.133:0/75042381 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b280046f0 con 0x7f5b30071960 2023-07-14T18:29:33.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.802+0000 7f5b365ff700 1 -- 172.21.15.133:0/75042381 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b3010f1e0 con 0x7f5b30071960 2023-07-14T18:29:33.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.802+0000 7f5b365ff700 1 -- 172.21.15.133:0/75042381 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b3010f6a0 con 0x7f5b30071960 2023-07-14T18:29:33.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.809+0000 7f5b49e3fb80 1 Processor -- start 2023-07-14T18:29:33.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.810+0000 7f5b49e3fb80 1 -- start start 2023-07-14T18:29:33.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.811+0000 7f5b49e3fb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588e5882f60 0x5588e5883370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:33.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.811+0000 7f5b49e3fb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588e58838b0 0x5588e5883ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:33.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.811+0000 7f5b49e3fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588e58844b0 0x5588e5884920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:33.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.811+0000 7f5b49e3fb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e5787cb0 con 0x5588e58838b0 2023-07-14T18:29:33.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.811+0000 7f5b49e3fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e57daa40 con 0x5588e58844b0 2023-07-14T18:29:33.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.811+0000 7f5b49e3fb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e57d3ed0 con 0x5588e5882f60 2023-07-14T18:29:33.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.812+0000 7f5b2f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588e58844b0 0x5588e5884920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:33.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.813+0000 7f5b2ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588e58838b0 0x5588e5883ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:33.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.814+0000 7f5b2ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588e58838b0 0x5588e5883ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33956/0 (socket says 172.21.15.133:33956) 2023-07-14T18:29:33.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.814+0000 7f5b2ffff700 1 -- 172.21.15.133:0/920116308 learned_addr learned my addr 172.21.15.133:0/920116308 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:29:33.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.814+0000 7f5b2ffff700 1 -- 172.21.15.133:0/920116308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588e5882f60 msgr2=0x5588e5883370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:29:33.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.814+0000 7f5b2ffff700 1 --2- 172.21.15.133:0/920116308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588e5882f60 0x5588e5883370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:33.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.814+0000 7f5b2ffff700 1 -- 172.21.15.133:0/920116308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588e58844b0 msgr2=0x5588e5884920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:29:33.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.814+0000 7f5b2ffff700 1 --2- 172.21.15.133:0/920116308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588e58844b0 0x5588e5884920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:33.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.814+0000 7f5b2ffff700 1 -- 172.21.15.133:0/920116308 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588e5601de0 con 0x5588e58838b0 2023-07-14T18:29:33.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.815+0000 7f5b2ffff700 1 --2- 172.21.15.133:0/920116308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588e58838b0 0x5588e5883ed0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5b24009530 tx=0x7f5b24009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1726a49ef7694c5c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:29:33.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.816+0000 7f5b167fc700 1 -- 172.21.15.133:0/920116308 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2401d070 con 0x5588e58838b0 2023-07-14T18:29:33.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.816+0000 7f5b167fc700 1 -- 172.21.15.133:0/920116308 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b2400be00 con 0x5588e58838b0 2023-07-14T18:29:33.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.816+0000 7f5b167fc700 1 -- 172.21.15.133:0/920116308 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b240054a0 con 0x5588e58838b0 2023-07-14T18:29:33.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.817+0000 7f5b49e3fb80 1 -- 172.21.15.133:0/920116308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588e58838b0 msgr2=0x5588e5883ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:29:33.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.817+0000 7f5b49e3fb80 1 --2- 172.21.15.133:0/920116308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588e58838b0 0x5588e5883ed0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5b24009530 tx=0x7f5b24009560 comp rx=0 tx=0).stop 2023-07-14T18:29:33.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.817+0000 7f5b49e3fb80 1 -- 172.21.15.133:0/920116308 shutdown_connections 2023-07-14T18:29:33.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.817+0000 7f5b49e3fb80 1 --2- 172.21.15.133:0/920116308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588e58844b0 0x5588e5884920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:33.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.817+0000 7f5b49e3fb80 1 --2- 172.21.15.133:0/920116308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588e5882f60 0x5588e5883370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:33.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.817+0000 7f5b49e3fb80 1 --2- 172.21.15.133:0/920116308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588e58838b0 0x5588e5883ed0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:33.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.817+0000 7f5b49e3fb80 1 -- 172.21.15.133:0/920116308 >> 172.21.15.133:0/920116308 conn(0x5588e5878680 msgr2=0x5588e587d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:29:33.854 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:33 smithi171 ceph-mon[129241]: pgmap v734: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 14 MiB/s wr, 1.04k op/s 2023-07-14T18:29:33.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.824+0000 7f5b49e3fb80 1 -- 172.21.15.133:0/920116308 shutdown_connections 2023-07-14T18:29:33.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.829+0000 7f5b2d7fa700 1 -- 172.21.15.133:0/75042381 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f5b28003a00 con 0x7f5b30071960 2023-07-14T18:29:33.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.832+0000 7f5b2d7fa700 1 --2- 172.21.15.133:0/75042381 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5b180a1260 0x7f5b180a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:33.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.832+0000 7f5b2d7fa700 1 -- 172.21.15.133:0/75042381 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5b280cc600 con 0x7f5b30071960 2023-07-14T18:29:33.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.833+0000 7f5b49e3fb80 1 -- 172.21.15.133:0/920116308 wait complete. 2023-07-14T18:29:33.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.834+0000 7f5b49e3fb80 1 -- start start 2023-07-14T18:29:33.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.835+0000 7f5b49e3fb80 1 Processor -- start 2023-07-14T18:29:33.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.835+0000 7f5b49e3fb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588e5882f60 0x5588e588c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:33.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.835+0000 7f5b49e3fb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588e58838b0 0x5588e588cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:33.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.835+0000 7f5b49e3fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588e58844b0 0x5588e5893f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:33.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.835+0000 7f5b49e3fb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e5773ac0 con 0x5588e58838b0 2023-07-14T18:29:33.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.835+0000 7f5b49e3fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e57737c0 con 0x5588e58844b0 2023-07-14T18:29:33.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.836+0000 7f5b49e3fb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e5657920 con 0x5588e5882f60 2023-07-14T18:29:33.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.836+0000 7f5b49e3fb80 10 client.0 _ll_register_callbacks cb 0x5588e5771760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:29:33.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.837+0000 7f5b34b9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588e58844b0 0x5588e5893f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:33.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.837+0000 7f5b34b9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588e58844b0 0x5588e5893f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:52166/0 (socket says 172.21.15.133:52166) 2023-07-14T18:29:33.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.837+0000 7f5b34b9c700 1 -- 172.21.15.133:0/3977212278 learned_addr learned my addr 172.21.15.133:0/3977212278 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:29:33.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.838+0000 7f5b2ffff700 1 --2- 172.21.15.133:0/75042381 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5b180a1260 0x7f5b180a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:33.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.838+0000 7f5b2f7fe700 1 --2- 172.21.15.133:0/3977212278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588e58838b0 0x5588e588cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:33.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.851+0000 7f5b2ffff700 1 --2- 172.21.15.133:0/3977212278 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588e5882f60 0x5588e588c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:33.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.853+0000 7f5b34b9c700 1 -- 172.21.15.133:0/3977212278 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588e5882f60 msgr2=0x5588e588c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:29:33.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.853+0000 7f5b34b9c700 1 --2- 172.21.15.133:0/3977212278 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588e5882f60 0x5588e588c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:33.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.853+0000 7f5b34b9c700 1 -- 172.21.15.133:0/3977212278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588e58838b0 msgr2=0x5588e588cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:29:33.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.853+0000 7f5b34b9c700 1 --2- 172.21.15.133:0/3977212278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588e58838b0 0x5588e588cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:33.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.853+0000 7f5b34b9c700 1 -- 172.21.15.133:0/3977212278 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b30117770 con 0x5588e58844b0 2023-07-14T18:29:33.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:33 smithi133 ceph-mon[126384]: pgmap v734: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 14 MiB/s wr, 1.04k op/s 2023-07-14T18:29:33.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.859+0000 7f5b34b9c700 1 --2- 172.21.15.133:0/3977212278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588e58844b0 0x5588e5893f40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000bd20 tx=0x7f5b2000d5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63a8e2553aa49a0a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:29:33.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.859+0000 7f5b0f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:29:33.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.866+0000 7f5b2ffff700 1 --2- 172.21.15.133:0/75042381 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5b180a1260 0x7f5b180a3710 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5b240098e0 tx=0x7f5b24009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:29:33.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.870+0000 7f5b0f7fe700 1 -- 172.21.15.133:0/3977212278 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2000ddd0 con 0x5588e58844b0 2023-07-14T18:29:33.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.870+0000 7f5b0f7fe700 1 -- 172.21.15.133:0/3977212278 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5b20010460 con 0x5588e58844b0 2023-07-14T18:29:33.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.871+0000 7f5b49e3fb80 1 -- 172.21.15.133:0/3977212278 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5588e572d4f0 con 0x5588e58844b0 2023-07-14T18:29:33.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.882+0000 7f5b0f7fe700 1 -- 172.21.15.133:0/3977212278 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b2000b560 con 0x5588e58844b0 2023-07-14T18:29:33.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.883+0000 7f5b49e3fb80 10 client.25958 fetch_fsmap learned FSMap version 44 2023-07-14T18:29:33.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.883+0000 7f5b49e3fb80 1 -- 172.21.15.133:0/3977212278 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5588e55eac60 con 0x5588e58844b0 2023-07-14T18:29:33.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.890+0000 7f5b0f7fe700 1 -- 172.21.15.133:0/3977212278 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5588e572d4f0 con 0x5588e58844b0 2023-07-14T18:29:33.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.890+0000 7f5b0f7fe700 1 -- 172.21.15.133:0/3977212278 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 44) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5b2001b820 con 0x5588e58844b0 2023-07-14T18:29:33.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.894+0000 7f5b49e3fb80 10 client.25958 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T18:29:33.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.894+0000 7f5b49e3fb80 10 client.25958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:29:33.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.894+0000 7f5b49e3fb80 20 client.25958 populate_metadata read hostname 'smithi133' 2023-07-14T18:29:33.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.894+0000 7f5b49e3fb80 1 --2- 172.21.15.133:0/3977212278 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5588e5898760 0x5588e589ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:33.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.894+0000 7f5b49e3fb80 4 client.25958 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:29:33.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.894+0000 7f5b49e3fb80 1 -- 172.21.15.133:0/3977212278 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588e5851da0 con 0x5588e5898760 2023-07-14T18:29:33.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.901+0000 7f5b2ffff700 1 --2- 172.21.15.133:0/3977212278 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5588e5898760 0x5588e589ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:33.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.913+0000 7f5b2ffff700 1 --2- 172.21.15.133:0/3977212278 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5588e5898760 0x5588e589ab70 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=512f20fe1582dd4b server_cookie=fcbbd77cad951683 in_seq=0 out_seq=0 2023-07-14T18:29:33.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.913+0000 7f5b0f7fe700 10 client.25958 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:29:33.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.927+0000 7f5b0f7fe700 1 -- 172.21.15.133:0/3977212278 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5588e5851da0 con 0x5588e5898760 2023-07-14T18:29:33.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.927+0000 7f5b0f7fe700 10 client.25958 handle_command_reply: tid=0 2023-07-14T18:29:33.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.928+0000 7f5b49e3fb80 1 client.25958 shutdown 2023-07-14T18:29:33.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.928+0000 7f5b49e3fb80 1 -- 172.21.15.133:0/3977212278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588e58844b0 msgr2=0x5588e5893f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:29:33.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.928+0000 7f5b49e3fb80 1 --2- 172.21.15.133:0/3977212278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588e58844b0 0x5588e5893f40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000bd20 tx=0x7f5b2000d5c0 comp rx=0 tx=0).stop 2023-07-14T18:29:33.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.935+0000 7f5b49e3fb80 1 -- 172.21.15.133:0/3977212278 shutdown_connections 2023-07-14T18:29:33.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.935+0000 7f5b49e3fb80 1 --2- 172.21.15.133:0/3977212278 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5588e5898760 0x5588e589ab70 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:33.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.935+0000 7f5b49e3fb80 1 --2- 172.21.15.133:0/3977212278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588e58844b0 0x5588e5893f40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:33.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.935+0000 7f5b49e3fb80 1 --2- 172.21.15.133:0/3977212278 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588e5882f60 0x5588e588c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:33.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.935+0000 7f5b49e3fb80 1 --2- 172.21.15.133:0/3977212278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588e58838b0 0x5588e588cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:33.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.935+0000 7f5b49e3fb80 1 -- 172.21.15.133:0/3977212278 >> 172.21.15.133:0/3977212278 conn(0x5588e5878680 msgr2=0x5588e587c650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:29:33.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.937+0000 7f5b0f7fe700 0 client.25958 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:29:33.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.942+0000 7f5b49e3fb80 1 -- 172.21.15.133:0/3977212278 shutdown_connections 2023-07-14T18:29:33.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.950+0000 7f5b49e3fb80 1 -- 172.21.15.133:0/3977212278 wait complete. 2023-07-14T18:29:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.951+0000 7f5b49e3fb80 20 client.25958 trim_cache size 0 max 16384 2023-07-14T18:29:34.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.985+0000 7f5b49e3fb80 1 Processor -- start 2023-07-14T18:29:34.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.986+0000 7f5b49e3fb80 1 -- start start 2023-07-14T18:29:34.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.986+0000 7f5b49e3fb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588e5882f60 0x5588e588b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:34.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.986+0000 7f5b49e3fb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588e58838b0 0x5588e5893e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:34.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.986+0000 7f5b49e3fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588e58844b0 0x5588e5887470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:34.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.986+0000 7f5b49e3fb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e588dbe0 con 0x5588e58838b0 2023-07-14T18:29:34.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.986+0000 7f5b49e3fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e587e8e0 con 0x5588e58844b0 2023-07-14T18:29:34.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.986+0000 7f5b49e3fb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e5895bb0 con 0x5588e5882f60 2023-07-14T18:29:34.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.987+0000 7f5b2ffff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588e5882f60 0x5588e588b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:34.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.989+0000 7f5b2f7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588e58838b0 0x5588e5893e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:34.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.989+0000 7f5b2f7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588e58838b0 0x5588e5893e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33978/0 (socket says 172.21.15.133:33978) 2023-07-14T18:29:34.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.989+0000 7f5b2f7fe700 1 -- 172.21.15.133:0/3627031588 learned_addr learned my addr 172.21.15.133:0/3627031588 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:29:34.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.990+0000 7f5b34b9c700 1 --2- 172.21.15.133:0/3627031588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588e58844b0 0x5588e5887470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:34.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.990+0000 7f5b2ffff700 1 -- 172.21.15.133:0/3627031588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588e58844b0 msgr2=0x5588e5887470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:29:34.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.990+0000 7f5b2ffff700 1 --2- 172.21.15.133:0/3627031588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588e58844b0 0x5588e5887470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:34.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.990+0000 7f5b2ffff700 1 -- 172.21.15.133:0/3627031588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588e58838b0 msgr2=0x5588e5893e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:29:34.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.990+0000 7f5b2ffff700 1 --2- 172.21.15.133:0/3627031588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588e58838b0 0x5588e5893e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:34.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.990+0000 7f5b2ffff700 1 -- 172.21.15.133:0/3627031588 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588e5628250 con 0x5588e5882f60 2023-07-14T18:29:34.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:33.991+0000 7f5b2ffff700 1 --2- 172.21.15.133:0/3627031588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588e5882f60 0x5588e588b6a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5b24009990 tx=0x7f5b2400b540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7108acd09cf261f5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:29:34.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.009+0000 7f5b16ffd700 1 -- 172.21.15.133:0/3627031588 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b24034040 con 0x5588e5882f60 2023-07-14T18:29:34.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.009+0000 7f5b16ffd700 1 -- 172.21.15.133:0/3627031588 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b24003960 con 0x5588e5882f60 2023-07-14T18:29:34.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.009+0000 7f5b16ffd700 1 -- 172.21.15.133:0/3627031588 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2401d040 con 0x5588e5882f60 2023-07-14T18:29:34.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.010+0000 7f5b49e3fb80 1 -- 172.21.15.133:0/3627031588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588e5882f60 msgr2=0x5588e588b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:29:34.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.010+0000 7f5b49e3fb80 1 --2- 172.21.15.133:0/3627031588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588e5882f60 0x5588e588b6a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5b24009990 tx=0x7f5b2400b540 comp rx=0 tx=0).stop 2023-07-14T18:29:34.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.024+0000 7f5b49e3fb80 1 -- 172.21.15.133:0/3627031588 shutdown_connections 2023-07-14T18:29:34.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.024+0000 7f5b49e3fb80 1 --2- 172.21.15.133:0/3627031588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588e58844b0 0x5588e5887470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:34.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.024+0000 7f5b49e3fb80 1 --2- 172.21.15.133:0/3627031588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588e5882f60 0x5588e588b6a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:34.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.024+0000 7f5b49e3fb80 1 --2- 172.21.15.133:0/3627031588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588e58838b0 0x5588e5893e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:34.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.024+0000 7f5b49e3fb80 1 -- 172.21.15.133:0/3627031588 >> 172.21.15.133:0/3627031588 conn(0x5588e5878680 msgr2=0x5588e589a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:29:34.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.037+0000 7f5b49e3fb80 1 -- 172.21.15.133:0/3627031588 shutdown_connections 2023-07-14T18:29:34.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.041+0000 7f5b49e3fb80 1 -- 172.21.15.133:0/3627031588 wait complete. 2023-07-14T18:29:34.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.042+0000 7f5b49e3fb80 1 -- start start 2023-07-14T18:29:34.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.042+0000 7f5b49e3fb80 1 Processor -- start 2023-07-14T18:29:34.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.051+0000 7f5b49e3fb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588e5882f60 0x5588e5891e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:34.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.051+0000 7f5b49e3fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588e58838b0 0x5588e58923d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:34.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.051+0000 7f5b49e3fb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588e58844b0 0x5588e58933e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:34.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.051+0000 7f5b49e3fb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e5893920 con 0x5588e5882f60 2023-07-14T18:29:34.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.051+0000 7f5b49e3fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e5893a90 con 0x5588e58838b0 2023-07-14T18:29:34.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.051+0000 7f5b49e3fb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e5893c50 con 0x5588e58844b0 2023-07-14T18:29:34.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.051+0000 7f5b49e3fb80 10 client.0 _ll_register_callbacks cb 0x7f5b2000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:29:34.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.054+0000 7f5b2ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588e5882f60 0x5588e5891e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:34.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.054+0000 7f5b2ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588e5882f60 0x5588e5891e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33990/0 (socket says 172.21.15.133:33990) 2023-07-14T18:29:34.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.054+0000 7f5b2ffff700 1 -- 172.21.15.133:0/3473469844 learned_addr learned my addr 172.21.15.133:0/3473469844 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:29:34.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.058+0000 7f5b2ffff700 1 -- 172.21.15.133:0/3473469844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588e58844b0 msgr2=0x5588e58933e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:29:34.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.058+0000 7f5b2ffff700 1 --2- 172.21.15.133:0/3473469844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588e58844b0 0x5588e58933e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:34.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.058+0000 7f5b2ffff700 1 -- 172.21.15.133:0/3473469844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588e58838b0 msgr2=0x5588e58923d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T18:29:34.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.058+0000 7f5b2ffff700 1 --2- 172.21.15.133:0/3473469844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588e58838b0 0x5588e58923d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:34.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.058+0000 7f5b2ffff700 1 -- 172.21.15.133:0/3473469844 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b24026070 con 0x5588e5882f60 2023-07-14T18:29:34.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.059+0000 7f5b2ffff700 1 --2- 172.21.15.133:0/3473469844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588e5882f60 0x5588e5891e90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5b24003f10 tx=0x7f5b24009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c29facedf7b2dd9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:29:34.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.060+0000 7f5b0f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T18:29:34.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.060+0000 7f5b0f7fe700 1 -- 172.21.15.133:0/3473469844 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b24034070 con 0x5588e5882f60 2023-07-14T18:29:34.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.060+0000 7f5b0f7fe700 1 -- 172.21.15.133:0/3473469844 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5b24039840 con 0x5588e5882f60 2023-07-14T18:29:34.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.066+0000 7f5b49e3fb80 1 -- 172.21.15.133:0/3473469844 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5588e57cceb0 con 0x5588e5882f60 2023-07-14T18:29:34.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.074+0000 7f5b0f7fe700 1 -- 172.21.15.133:0/3473469844 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b2401d040 con 0x5588e5882f60 2023-07-14T18:29:34.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.080+0000 7f5b49e3fb80 10 client.15864 fetch_fsmap learned FSMap version 44 2023-07-14T18:29:34.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.080+0000 7f5b49e3fb80 1 -- 172.21.15.133:0/3473469844 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5588e56002b0 con 0x5588e5882f60 2023-07-14T18:29:34.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.081+0000 7f5b0f7fe700 1 -- 172.21.15.133:0/3473469844 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5588e57cceb0 con 0x5588e5882f60 2023-07-14T18:29:34.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.081+0000 7f5b0f7fe700 1 -- 172.21.15.133:0/3473469844 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 44) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5b2404a070 con 0x5588e5882f60 2023-07-14T18:29:34.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.081+0000 7f5b49e3fb80 10 client.15864 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T18:29:34.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.081+0000 7f5b49e3fb80 10 client.15864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:29:34.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.081+0000 7f5b49e3fb80 20 client.15864 populate_metadata read hostname 'smithi133' 2023-07-14T18:29:34.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.081+0000 7f5b49e3fb80 1 --2- 172.21.15.133:0/3473469844 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5588e58ed0a0 0x5588e58ef4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:29:34.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.082+0000 7f5b49e3fb80 4 client.15864 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:29:34.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.082+0000 7f5b49e3fb80 1 -- 172.21.15.133:0/3473469844 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5588e578f050 con 0x5588e58ed0a0 2023-07-14T18:29:34.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.098+0000 7f5b2f7fe700 1 --2- 172.21.15.133:0/3473469844 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5588e58ed0a0 0x5588e58ef4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:29:34.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.114+0000 7f5b2f7fe700 1 --2- 172.21.15.133:0/3473469844 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5588e58ed0a0 0x5588e58ef4b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37a091d1bf095910 server_cookie=ed4667119d1b8514 in_seq=0 out_seq=0 2023-07-14T18:29:34.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.114+0000 7f5b0f7fe700 10 client.15864 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:29:34.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.114+0000 7f5b0f7fe700 1 -- 172.21.15.133:0/3473469844 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5588e578f050 con 0x5588e58ed0a0 2023-07-14T18:29:34.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.114+0000 7f5b0f7fe700 10 client.15864 handle_command_reply: tid=0 2023-07-14T18:29:34.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.115+0000 7f5b49e3fb80 1 client.15864 shutdown 2023-07-14T18:29:34.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.117+0000 7f5b49e3fb80 1 -- 172.21.15.133:0/3473469844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588e5882f60 msgr2=0x5588e5891e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:29:34.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.117+0000 7f5b49e3fb80 1 --2- 172.21.15.133:0/3473469844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588e5882f60 0x5588e5891e90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5b24003f10 tx=0x7f5b24009560 comp rx=0 tx=0).stop 2023-07-14T18:29:34.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.126+0000 7f5b49e3fb80 1 -- 172.21.15.133:0/3473469844 shutdown_connections 2023-07-14T18:29:34.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.126+0000 7f5b49e3fb80 1 --2- 172.21.15.133:0/3473469844 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5588e58ed0a0 0x5588e58ef4b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:34.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.126+0000 7f5b49e3fb80 1 --2- 172.21.15.133:0/3473469844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5588e58844b0 0x5588e58933e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:34.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.126+0000 7f5b49e3fb80 1 --2- 172.21.15.133:0/3473469844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588e5882f60 0x5588e5891e90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:34.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.126+0000 7f5b49e3fb80 1 --2- 172.21.15.133:0/3473469844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588e58838b0 0x5588e58923d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:34.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.126+0000 7f5b49e3fb80 1 -- 172.21.15.133:0/3473469844 >> 172.21.15.133:0/3473469844 conn(0x5588e5878680 msgr2=0x5588e588b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:29:34.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.126+0000 7f5b0f7fe700 0 client.15864 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:29:34.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.133+0000 7f5b49e3fb80 1 -- 172.21.15.133:0/3473469844 shutdown_connections 2023-07-14T18:29:34.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.145+0000 7f5b49e3fb80 1 -- 172.21.15.133:0/3473469844 wait complete. 2023-07-14T18:29:34.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.148+0000 7f5b49e3fb80 20 client.15864 trim_cache size 0 max 16384 2023-07-14T18:29:34.192 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:29:34.192 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T18:29:34.192 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:29:34.192 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:29:34.193 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:29:34.193 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:29:34.193 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:29:34.193 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:29:34.193 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:29:34.193 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:29:34.203 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.166+0000 7f5b365ff700 1 -- 172.21.15.133:0/75042381 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5b180a1260 msgr2=0x7f5b180a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:29:34.203 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.166+0000 7f5b365ff700 1 --2- 172.21.15.133:0/75042381 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5b180a1260 0x7f5b180a3710 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5b240098e0 tx=0x7f5b24009310 comp rx=0 tx=0).stop 2023-07-14T18:29:34.203 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.166+0000 7f5b365ff700 1 -- 172.21.15.133:0/75042381 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b30071960 msgr2=0x7f5b301a4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:29:34.203 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.166+0000 7f5b365ff700 1 --2- 172.21.15.133:0/75042381 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b30071960 0x7f5b301a4ce0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5b28007f10 tx=0x7f5b2800d3b0 comp rx=0 tx=0).stop 2023-07-14T18:29:34.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.177+0000 7f5b365ff700 1 -- 172.21.15.133:0/75042381 shutdown_connections 2023-07-14T18:29:34.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.177+0000 7f5b365ff700 1 --2- 172.21.15.133:0/75042381 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5b180a1260 0x7f5b180a3710 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:34.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.177+0000 7f5b365ff700 1 --2- 172.21.15.133:0/75042381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b30071010 0x7f5b301a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:34.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.177+0000 7f5b365ff700 1 --2- 172.21.15.133:0/75042381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b3010be90 0x7f5b301abd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:34.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.177+0000 7f5b365ff700 1 --2- 172.21.15.133:0/75042381 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b30071960 0x7f5b301a4ce0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:29:34.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.177+0000 7f5b365ff700 1 -- 172.21.15.133:0/75042381 >> 172.21.15.133:0/75042381 conn(0x7f5b3006cbc0 msgr2=0x7f5b3006de40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:29:34.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.189+0000 7f5b365ff700 1 -- 172.21.15.133:0/75042381 shutdown_connections 2023-07-14T18:29:34.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:29:34.198+0000 7f5b365ff700 1 -- 172.21.15.133:0/75042381 wait complete. 2023-07-14T18:29:34.289 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:29:35.865 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 610.0 s, 529.9 tps, lat 184.983 ms stddev 213.473 2023-07-14T18:29:35.903 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:35 smithi133 ceph-mon[126384]: pgmap v735: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.4 MiB/s wr, 704 op/s 2023-07-14T18:29:36.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:35 smithi161 ceph-mon[128993]: pgmap v735: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.4 MiB/s wr, 704 op/s 2023-07-14T18:29:36.103 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:35 smithi171 ceph-mon[129241]: pgmap v735: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.4 MiB/s wr, 704 op/s 2023-07-14T18:29:37.338 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:37 smithi161 ceph-mon[128993]: pgmap v736: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 854 op/s 2023-07-14T18:29:37.353 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:37 smithi171 ceph-mon[129241]: pgmap v736: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 854 op/s 2023-07-14T18:29:37.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:37 smithi133 ceph-mon[126384]: pgmap v736: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 854 op/s 2023-07-14T18:29:39.838 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:39 smithi161 ceph-mon[128993]: pgmap v737: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 11 MiB/s wr, 754 op/s 2023-07-14T18:29:39.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:39 smithi171 ceph-mon[129241]: pgmap v737: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 11 MiB/s wr, 754 op/s 2023-07-14T18:29:39.895 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:39 smithi133 ceph-mon[126384]: pgmap v737: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 11 MiB/s wr, 754 op/s 2023-07-14T18:29:40.862 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 615.0 s, 705.8 tps, lat 145.545 ms stddev 318.607 2023-07-14T18:29:41.838 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:41 smithi161 ceph-mon[128993]: pgmap v738: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 12 MiB/s wr, 742 op/s 2023-07-14T18:29:41.853 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:41 smithi171 ceph-mon[129241]: pgmap v738: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 12 MiB/s wr, 742 op/s 2023-07-14T18:29:41.894 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:41 smithi133 ceph-mon[126384]: pgmap v738: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 12 MiB/s wr, 742 op/s 2023-07-14T18:29:44.090 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:43 smithi161 ceph-mon[128993]: pgmap v739: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 15 MiB/s wr, 992 op/s 2023-07-14T18:29:44.106 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:43 smithi171 ceph-mon[129241]: pgmap v739: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 15 MiB/s wr, 992 op/s 2023-07-14T18:29:44.146 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:43 smithi133 ceph-mon[126384]: pgmap v739: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 15 MiB/s wr, 992 op/s 2023-07-14T18:29:45.863 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 620.0 s, 587.3 tps, lat 141.128 ms stddev 387.953 2023-07-14T18:29:46.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:45 smithi161 ceph-mon[128993]: pgmap v740: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 12 MiB/s wr, 819 op/s 2023-07-14T18:29:46.103 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:45 smithi171 ceph-mon[129241]: pgmap v740: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 12 MiB/s wr, 819 op/s 2023-07-14T18:29:46.145 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:45 smithi133 ceph-mon[126384]: pgmap v740: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 12 MiB/s wr, 819 op/s 2023-07-14T18:29:47.394 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:47 smithi133 ceph-mon[126384]: pgmap v741: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 15 MiB/s wr, 1.10k op/s 2023-07-14T18:29:47.592 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:47 smithi161 ceph-mon[128993]: pgmap v741: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 15 MiB/s wr, 1.10k op/s 2023-07-14T18:29:47.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:47 smithi171 ceph-mon[129241]: pgmap v741: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 15 MiB/s wr, 1.10k op/s 2023-07-14T18:29:49.841 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:49 smithi161 ceph-mon[128993]: pgmap v742: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 15 MiB/s wr, 1.11k op/s 2023-07-14T18:29:49.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:49 smithi171 ceph-mon[129241]: pgmap v742: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 15 MiB/s wr, 1.11k op/s 2023-07-14T18:29:49.894 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:49 smithi133 ceph-mon[126384]: pgmap v742: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 15 MiB/s wr, 1.11k op/s 2023-07-14T18:29:50.862 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 625.0 s, 616.1 tps, lat 190.227 ms stddev 403.552 2023-07-14T18:29:51.899 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:51 smithi133 ceph-mon[126384]: pgmap v743: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 14 MiB/s wr, 1.07k op/s 2023-07-14T18:29:52.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:51 smithi161 ceph-mon[128993]: pgmap v743: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 14 MiB/s wr, 1.07k op/s 2023-07-14T18:29:52.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:51 smithi171 ceph-mon[129241]: pgmap v743: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 14 MiB/s wr, 1.07k op/s 2023-07-14T18:29:54.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:53 smithi161 ceph-mon[128993]: pgmap v744: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 16 MiB/s wr, 1.22k op/s 2023-07-14T18:29:54.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:53 smithi171 ceph-mon[129241]: pgmap v744: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 16 MiB/s wr, 1.22k op/s 2023-07-14T18:29:54.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:53 smithi133 ceph-mon[126384]: pgmap v744: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 16 MiB/s wr, 1.22k op/s 2023-07-14T18:29:55.865 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 630.0 s, 727.8 tps, lat 132.393 ms stddev 191.532 2023-07-14T18:29:56.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:55 smithi161 ceph-mon[128993]: pgmap v745: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 12 MiB/s wr, 958 op/s 2023-07-14T18:29:56.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:55 smithi171 ceph-mon[129241]: pgmap v745: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 12 MiB/s wr, 958 op/s 2023-07-14T18:29:56.144 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:55 smithi133 ceph-mon[126384]: pgmap v745: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 12 MiB/s wr, 958 op/s 2023-07-14T18:29:57.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:57 smithi161 ceph-mon[128993]: pgmap v746: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 16 MiB/s wr, 1.14k op/s 2023-07-14T18:29:57.352 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:57 smithi171 ceph-mon[129241]: pgmap v746: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 16 MiB/s wr, 1.14k op/s 2023-07-14T18:29:57.395 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:57 smithi133 ceph-mon[126384]: pgmap v746: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 16 MiB/s wr, 1.14k op/s 2023-07-14T18:29:59.837 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:29:59 smithi161 ceph-mon[128993]: pgmap v747: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 15 MiB/s wr, 982 op/s 2023-07-14T18:29:59.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:29:59 smithi171 ceph-mon[129241]: pgmap v747: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 15 MiB/s wr, 982 op/s 2023-07-14T18:29:59.893 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:29:59 smithi133 ceph-mon[126384]: pgmap v747: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 15 MiB/s wr, 982 op/s 2023-07-14T18:30:00.839 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:00 smithi161 ceph-mon[128993]: overall HEALTH_OK 2023-07-14T18:30:00.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:00 smithi171 ceph-mon[129241]: overall HEALTH_OK 2023-07-14T18:30:00.861 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 635.0 s, 1229.5 tps, lat 84.136 ms stddev 66.835 2023-07-14T18:30:00.893 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:00 smithi133 ceph-mon[126384]: overall HEALTH_OK 2023-07-14T18:30:01.837 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:01 smithi161 ceph-mon[128993]: pgmap v748: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 14 MiB/s wr, 823 op/s 2023-07-14T18:30:01.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:01 smithi171 ceph-mon[129241]: pgmap v748: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 14 MiB/s wr, 823 op/s 2023-07-14T18:30:01.893 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:01 smithi133 ceph-mon[126384]: pgmap v748: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 14 MiB/s wr, 823 op/s 2023-07-14T18:30:03.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:03 smithi161 ceph-mon[128993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:30:03.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:03 smithi161 ceph-mon[128993]: pgmap v749: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 945 op/s 2023-07-14T18:30:03.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:03 smithi171 ceph-mon[129241]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:30:03.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:03 smithi171 ceph-mon[129241]: pgmap v749: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 945 op/s 2023-07-14T18:30:03.647 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:03 smithi133 ceph-mon[126384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:30:03.647 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:03 smithi133 ceph-mon[126384]: pgmap v749: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 945 op/s 2023-07-14T18:30:04.290 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:30:04.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.818+0000 7f6dd8bff700 1 -- 172.21.15.133:0/2715036356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6dd4100de0 msgr2=0x7f6dd4103260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:30:04.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.818+0000 7f6dd8bff700 1 --2- 172.21.15.133:0/2715036356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6dd4100de0 0x7f6dd4103260 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc01c0d0 tx=0x7f6dcc01c3e0 comp rx=0 tx=0).stop 2023-07-14T18:30:04.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.827+0000 7f6dd8bff700 1 -- 172.21.15.133:0/2715036356 shutdown_connections 2023-07-14T18:30:04.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.827+0000 7f6dd8bff700 1 --2- 172.21.15.133:0/2715036356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dd41037a0 0x7f6dd4103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:04.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.827+0000 7f6dd8bff700 1 --2- 172.21.15.133:0/2715036356 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6dd40fe480 0x7f6dd41008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:04.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.827+0000 7f6dd8bff700 1 --2- 172.21.15.133:0/2715036356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6dd4100de0 0x7f6dd4103260 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:04.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.827+0000 7f6dd8bff700 1 -- 172.21.15.133:0/2715036356 >> 172.21.15.133:0/2715036356 conn(0x7f6dd40fa130 msgr2=0x7f6dd40fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:30:04.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.830+0000 7f6dd8bff700 1 -- 172.21.15.133:0/2715036356 shutdown_connections 2023-07-14T18:30:04.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.830+0000 7f6dd8bff700 1 -- 172.21.15.133:0/2715036356 wait complete. 2023-07-14T18:30:04.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.831+0000 7f6dd8bff700 1 Processor -- start 2023-07-14T18:30:04.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.832+0000 7f6dd8bff700 1 -- start start 2023-07-14T18:30:04.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.832+0000 7f6dd8bff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6dd40fe480 0x7f6dd406c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:04.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.832+0000 7f6dd8bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6dd4100de0 0x7f6dd406e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:04.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.832+0000 7f6dd8bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dd41037a0 0x7f6dd4070c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:04.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.832+0000 7f6dd8bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd40713f0 con 0x7f6dd4100de0 2023-07-14T18:30:04.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.832+0000 7f6dd8bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd4071560 con 0x7f6dd41037a0 2023-07-14T18:30:04.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.832+0000 7f6dd8bff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd4071720 con 0x7f6dd40fe480 2023-07-14T18:30:04.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.833+0000 7f6dd259c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6dd40fe480 0x7f6dd406c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:30:04.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.834+0000 7f6dd1d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6dd4100de0 0x7f6dd406e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:30:04.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.834+0000 7f6dd1d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6dd4100de0 0x7f6dd406e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55430/0 (socket says 172.21.15.133:55430) 2023-07-14T18:30:04.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.834+0000 7f6dd1d9b700 1 -- 172.21.15.133:0/2223500911 learned_addr learned my addr 172.21.15.133:0/2223500911 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:30:04.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.834+0000 7f6dd1d9b700 1 -- 172.21.15.133:0/2223500911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6dd40fe480 msgr2=0x7f6dd406c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:30:04.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.834+0000 7f6dd1d9b700 1 --2- 172.21.15.133:0/2223500911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6dd40fe480 0x7f6dd406c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:04.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.834+0000 7f6dd1d9b700 1 -- 172.21.15.133:0/2223500911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dd41037a0 msgr2=0x7f6dd4070c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:30:04.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.834+0000 7f6dd1d9b700 1 --2- 172.21.15.133:0/2223500911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dd41037a0 0x7f6dd4070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:04.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.834+0000 7f6dd1d9b700 1 -- 172.21.15.133:0/2223500911 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dcc01ad10 con 0x7f6dd4100de0 2023-07-14T18:30:04.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.835+0000 7f6dd1d9b700 1 --2- 172.21.15.133:0/2223500911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6dd4100de0 0x7f6dd406e5c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc01e010 tx=0x7f6dcc01a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:30:04.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.836+0000 7f6dc37fe700 1 -- 172.21.15.133:0/2223500911 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc01f050 con 0x7f6dd4100de0 2023-07-14T18:30:04.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.836+0000 7f6dc37fe700 1 -- 172.21.15.133:0/2223500911 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dcc005570 con 0x7f6dd4100de0 2023-07-14T18:30:04.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.836+0000 7f6dc37fe700 1 -- 172.21.15.133:0/2223500911 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc02ca80 con 0x7f6dd4100de0 2023-07-14T18:30:04.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.836+0000 7f6dd8bff700 1 -- 172.21.15.133:0/2223500911 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd40719a0 con 0x7f6dd4100de0 2023-07-14T18:30:04.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.836+0000 7f6dd8bff700 1 -- 172.21.15.133:0/2223500911 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd4071e60 con 0x7f6dd4100de0 2023-07-14T18:30:04.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.842+0000 7f6dc37fe700 1 -- 172.21.15.133:0/2223500911 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6dcc02cbe0 con 0x7f6dd4100de0 2023-07-14T18:30:04.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.845+0000 7f6dc37fe700 1 --2- 172.21.15.133:0/2223500911 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6dbc0a1350 0x7f6dbc0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:04.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.845+0000 7f6dc37fe700 1 -- 172.21.15.133:0/2223500911 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6dcc0df160 con 0x7f6dd4100de0 2023-07-14T18:30:04.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.847+0000 7f6dd259c700 1 --2- 172.21.15.133:0/2223500911 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6dbc0a1350 0x7f6dbc0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:30:04.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.848+0000 7f6dd259c700 1 --2- 172.21.15.133:0/2223500911 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6dbc0a1350 0x7f6dbc0a3800 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4005fd0 tx=0x7f6dc400a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:30:04.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.861+0000 7f6dec47eb80 1 Processor -- start 2023-07-14T18:30:04.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.862+0000 7f6dec47eb80 1 -- start start 2023-07-14T18:30:04.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.862+0000 7f6dec47eb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9e0774f60 0x55a9e0775370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:04.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.862+0000 7f6dec47eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9e07758b0 0x55a9e0775ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:04.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.862+0000 7f6dec47eb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9e07764b0 0x55a9e0776920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:04.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.862+0000 7f6dec47eb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9e0679cb0 con 0x55a9e07764b0 2023-07-14T18:30:04.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.862+0000 7f6dec47eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9e06cca40 con 0x55a9e07758b0 2023-07-14T18:30:04.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.862+0000 7f6dec47eb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9e06c5ed0 con 0x55a9e0774f60 2023-07-14T18:30:04.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.863+0000 7f6dd259c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9e0774f60 0x55a9e0775370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:30:04.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.869+0000 7f6dd2d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9e07764b0 0x55a9e0776920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:30:04.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.869+0000 7f6dd2d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9e07764b0 0x55a9e0776920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55444/0 (socket says 172.21.15.133:55444) 2023-07-14T18:30:04.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.869+0000 7f6dd2d9d700 1 -- 172.21.15.133:0/3833825066 learned_addr learned my addr 172.21.15.133:0/3833825066 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:30:04.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.869+0000 7f6dd1d9b700 1 --2- 172.21.15.133:0/3833825066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9e07758b0 0x55a9e0775ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:30:04.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.870+0000 7f6dd259c700 1 -- 172.21.15.133:0/3833825066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9e07758b0 msgr2=0x55a9e0775ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:30:04.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.870+0000 7f6dd259c700 1 --2- 172.21.15.133:0/3833825066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9e07758b0 0x55a9e0775ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:04.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.870+0000 7f6dd259c700 1 -- 172.21.15.133:0/3833825066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9e07764b0 msgr2=0x55a9e0776920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:30:04.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.870+0000 7f6dd259c700 1 --2- 172.21.15.133:0/3833825066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9e07764b0 0x55a9e0776920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:04.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.870+0000 7f6dd259c700 1 -- 172.21.15.133:0/3833825066 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9e04f3de0 con 0x55a9e0774f60 2023-07-14T18:30:04.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.886+0000 7f6dd259c700 1 --2- 172.21.15.133:0/3833825066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9e0774f60 0x55a9e0775370 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4005950 tx=0x7f6dc4011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0e8352fd65a6710 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:30:04.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.898+0000 7f6dc0ff9700 1 -- 172.21.15.133:0/3833825066 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4010cd0 con 0x55a9e0774f60 2023-07-14T18:30:04.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.898+0000 7f6dc0ff9700 1 -- 172.21.15.133:0/3833825066 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dc400ee70 con 0x55a9e0774f60 2023-07-14T18:30:04.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.898+0000 7f6dc0ff9700 1 -- 172.21.15.133:0/3833825066 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc400f6b0 con 0x55a9e0774f60 2023-07-14T18:30:04.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.899+0000 7f6dec47eb80 1 -- 172.21.15.133:0/3833825066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9e0774f60 msgr2=0x55a9e0775370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:30:04.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.899+0000 7f6dec47eb80 1 --2- 172.21.15.133:0/3833825066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9e0774f60 0x55a9e0775370 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4005950 tx=0x7f6dc4011000 comp rx=0 tx=0).stop 2023-07-14T18:30:04.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.910+0000 7f6dec47eb80 1 -- 172.21.15.133:0/3833825066 shutdown_connections 2023-07-14T18:30:04.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.910+0000 7f6dec47eb80 1 --2- 172.21.15.133:0/3833825066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9e07764b0 0x55a9e0776920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:04.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.910+0000 7f6dec47eb80 1 --2- 172.21.15.133:0/3833825066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9e0774f60 0x55a9e0775370 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:04.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.910+0000 7f6dec47eb80 1 --2- 172.21.15.133:0/3833825066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9e07758b0 0x55a9e0775ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:04.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.910+0000 7f6dec47eb80 1 -- 172.21.15.133:0/3833825066 >> 172.21.15.133:0/3833825066 conn(0x55a9e076a680 msgr2=0x55a9e076f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:30:04.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.918+0000 7f6dec47eb80 1 -- 172.21.15.133:0/3833825066 shutdown_connections 2023-07-14T18:30:04.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.925+0000 7f6dec47eb80 1 -- 172.21.15.133:0/3833825066 wait complete. 2023-07-14T18:30:04.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.926+0000 7f6dec47eb80 1 -- start start 2023-07-14T18:30:04.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.926+0000 7f6dec47eb80 1 Processor -- start 2023-07-14T18:30:04.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.932+0000 7f6dec47eb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9e0774f60 0x55a9e077e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:04.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.932+0000 7f6dec47eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9e07758b0 0x55a9e077ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:04.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.932+0000 7f6dec47eb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9e07764b0 0x55a9e0785f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:04.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.932+0000 7f6dec47eb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9e0665ac0 con 0x55a9e07764b0 2023-07-14T18:30:04.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.932+0000 7f6dec47eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9e06657c0 con 0x55a9e07758b0 2023-07-14T18:30:04.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.932+0000 7f6dec47eb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9e0549920 con 0x55a9e0774f60 2023-07-14T18:30:04.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.933+0000 7f6dec47eb80 10 client.0 _ll_register_callbacks cb 0x55a9e0663760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:30:04.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.933+0000 7f6dd259c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9e0774f60 0x55a9e077e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:30:04.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.933+0000 7f6dd259c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9e0774f60 0x55a9e077e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:58714/0 (socket says 172.21.15.133:58714) 2023-07-14T18:30:04.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.933+0000 7f6dd259c700 1 -- 172.21.15.133:0/310610700 learned_addr learned my addr 172.21.15.133:0/310610700 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:30:04.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.934+0000 7f6dd1d9b700 1 --2- 172.21.15.133:0/310610700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9e07758b0 0x55a9e077ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:30:04.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.938+0000 7f6dd2d9d700 1 --2- 172.21.15.133:0/310610700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9e07764b0 0x55a9e0785f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:30:04.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.951+0000 7f6dd1d9b700 1 -- 172.21.15.133:0/310610700 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9e0774f60 msgr2=0x55a9e077e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:30:04.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.951+0000 7f6dd1d9b700 1 --2- 172.21.15.133:0/310610700 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9e0774f60 0x55a9e077e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:04.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.951+0000 7f6dd1d9b700 1 -- 172.21.15.133:0/310610700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9e07764b0 msgr2=0x55a9e0785f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:30:04.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.951+0000 7f6dd1d9b700 1 --2- 172.21.15.133:0/310610700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9e07764b0 0x55a9e0785f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:04.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.951+0000 7f6dd1d9b700 1 -- 172.21.15.133:0/310610700 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc400e870 con 0x55a9e07758b0 2023-07-14T18:30:05.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.972+0000 7f6dd1d9b700 1 --2- 172.21.15.133:0/310610700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9e07758b0 0x55a9e077ef10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0dc8d0 tx=0x7f6dcc0aa8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7f7961a3f636aff server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:30:05.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.972+0000 7f6db9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:30:05.011 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.980+0000 7f6db9ffb700 1 -- 172.21.15.133:0/310610700 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc0b0320 con 0x55a9e07758b0 2023-07-14T18:30:05.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.980+0000 7f6db9ffb700 1 -- 172.21.15.133:0/310610700 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6dcc0b46b0 con 0x55a9e07758b0 2023-07-14T18:30:05.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:04.981+0000 7f6dec47eb80 1 -- 172.21.15.133:0/310610700 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9e061f4f0 con 0x55a9e07758b0 2023-07-14T18:30:05.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.015+0000 7f6db9ffb700 1 -- 172.21.15.133:0/310610700 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6dcc0ac550 con 0x55a9e07758b0 2023-07-14T18:30:05.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.016+0000 7f6dec47eb80 10 client.25967 fetch_fsmap learned FSMap version 45 2023-07-14T18:30:05.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.016+0000 7f6dec47eb80 1 -- 172.21.15.133:0/310610700 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55a9e04dcc60 con 0x55a9e07758b0 2023-07-14T18:30:05.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.027+0000 7f6db9ffb700 1 -- 172.21.15.133:0/310610700 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a9e061f4f0 con 0x55a9e07758b0 2023-07-14T18:30:05.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.027+0000 7f6db9ffb700 1 -- 172.21.15.133:0/310610700 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 45) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f6dcc0b4220 con 0x55a9e07758b0 2023-07-14T18:30:05.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.027+0000 7f6dec47eb80 10 client.25967 fetch_fsmap finished waiting for FSMap version 45 2023-07-14T18:30:05.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.027+0000 7f6dec47eb80 10 client.25967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:30:05.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.027+0000 7f6dec47eb80 20 client.25967 populate_metadata read hostname 'smithi133' 2023-07-14T18:30:05.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.027+0000 7f6dec47eb80 1 --2- 172.21.15.133:0/310610700 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55a9e078a4a0 0x55a9e078c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:05.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.027+0000 7f6dec47eb80 4 client.25967 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:30:05.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.027+0000 7f6dec47eb80 1 -- 172.21.15.133:0/310610700 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9e0743da0 con 0x55a9e078a4a0 2023-07-14T18:30:05.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.034+0000 7f6dd259c700 1 --2- 172.21.15.133:0/310610700 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55a9e078a4a0 0x55a9e078c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:30:05.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.077+0000 7f6dd259c700 1 --2- 172.21.15.133:0/310610700 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55a9e078a4a0 0x55a9e078c8b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f5b1c500fd57db7 server_cookie=72ab1ede364485af in_seq=0 out_seq=0 2023-07-14T18:30:05.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.078+0000 7f6db9ffb700 10 client.25967 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:30:05.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.089+0000 7f6db9ffb700 1 -- 172.21.15.133:0/310610700 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55a9e0743da0 con 0x55a9e078a4a0 2023-07-14T18:30:05.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.089+0000 7f6db9ffb700 10 client.25967 handle_command_reply: tid=0 2023-07-14T18:30:05.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.089+0000 7f6dec47eb80 1 client.25967 shutdown 2023-07-14T18:30:05.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.118+0000 7f6dec47eb80 1 -- 172.21.15.133:0/310610700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9e07758b0 msgr2=0x55a9e077ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:30:05.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.118+0000 7f6dec47eb80 1 --2- 172.21.15.133:0/310610700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9e07758b0 0x55a9e077ef10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0dc8d0 tx=0x7f6dcc0aa8a0 comp rx=0 tx=0).stop 2023-07-14T18:30:05.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.118+0000 7f6dec47eb80 1 -- 172.21.15.133:0/310610700 shutdown_connections 2023-07-14T18:30:05.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.118+0000 7f6dec47eb80 1 --2- 172.21.15.133:0/310610700 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55a9e078a4a0 0x55a9e078c8b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:05.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.118+0000 7f6dec47eb80 1 --2- 172.21.15.133:0/310610700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9e07764b0 0x55a9e0785f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:05.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.118+0000 7f6dec47eb80 1 --2- 172.21.15.133:0/310610700 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9e0774f60 0x55a9e077e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:05.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.118+0000 7f6dec47eb80 1 --2- 172.21.15.133:0/310610700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9e07758b0 0x55a9e077ef10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:05.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.118+0000 7f6dec47eb80 1 -- 172.21.15.133:0/310610700 >> 172.21.15.133:0/310610700 conn(0x55a9e076a680 msgr2=0x55a9e076e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:30:05.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.118+0000 7f6db9ffb700 0 client.25967 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:30:05.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.118+0000 7f6dec47eb80 1 -- 172.21.15.133:0/310610700 shutdown_connections 2023-07-14T18:30:05.151 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.119+0000 7f6dec47eb80 1 -- 172.21.15.133:0/310610700 wait complete. 2023-07-14T18:30:05.157 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.126+0000 7f6dec47eb80 20 client.25967 trim_cache size 0 max 16384 2023-07-14T18:30:05.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.159+0000 7f6dec47eb80 1 Processor -- start 2023-07-14T18:30:05.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.161+0000 7f6dec47eb80 1 -- start start 2023-07-14T18:30:05.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.161+0000 7f6dec47eb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9e0774f60 0x55a9e077d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:05.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.161+0000 7f6dec47eb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9e07758b0 0x55a9e0785f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:05.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.161+0000 7f6dec47eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9e07764b0 0x55a9e078c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:05.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.161+0000 7f6dec47eb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9e076a2a0 con 0x55a9e07758b0 2023-07-14T18:30:05.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.161+0000 7f6dec47eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9e0776330 con 0x55a9e07764b0 2023-07-14T18:30:05.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.161+0000 7f6dec47eb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9e076f800 con 0x55a9e0774f60 2023-07-14T18:30:05.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.163+0000 7f6dd1d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9e07758b0 0x55a9e0785f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:30:05.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.163+0000 7f6dd1d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9e07758b0 0x55a9e0785f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55460/0 (socket says 172.21.15.133:55460) 2023-07-14T18:30:05.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.163+0000 7f6dd1d9b700 1 -- 172.21.15.133:0/1014149794 learned_addr learned my addr 172.21.15.133:0/1014149794 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:30:05.199 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.168+0000 7f6dd1d9b700 1 -- 172.21.15.133:0/1014149794 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9e0774f60 msgr2=0x55a9e077d6d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T18:30:05.199 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.168+0000 7f6dd1d9b700 1 --2- 172.21.15.133:0/1014149794 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9e0774f60 0x55a9e077d6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:05.199 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.168+0000 7f6dd1d9b700 1 -- 172.21.15.133:0/1014149794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9e07764b0 msgr2=0x55a9e078c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:30:05.199 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.168+0000 7f6dd1d9b700 1 --2- 172.21.15.133:0/1014149794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9e07764b0 0x55a9e078c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:05.199 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.168+0000 7f6dd1d9b700 1 -- 172.21.15.133:0/1014149794 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9e051a250 con 0x55a9e07758b0 2023-07-14T18:30:05.200 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.168+0000 7f6dd1d9b700 1 --2- 172.21.15.133:0/1014149794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9e07758b0 0x55a9e0785f10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0abff0 tx=0x7f6dcc0a8760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bdfaf7ee587b0c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:30:05.200 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.169+0000 7f6dc17fa700 1 -- 172.21.15.133:0/1014149794 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc02cf10 con 0x55a9e07758b0 2023-07-14T18:30:05.200 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.169+0000 7f6dc17fa700 1 -- 172.21.15.133:0/1014149794 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dcc0dc7a0 con 0x55a9e07758b0 2023-07-14T18:30:05.200 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.169+0000 7f6dc17fa700 1 -- 172.21.15.133:0/1014149794 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc0ab7e0 con 0x55a9e07758b0 2023-07-14T18:30:05.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.178+0000 7f6dec47eb80 1 -- 172.21.15.133:0/1014149794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9e07758b0 msgr2=0x55a9e0785f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:30:05.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.178+0000 7f6dec47eb80 1 --2- 172.21.15.133:0/1014149794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9e07758b0 0x55a9e0785f10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0abff0 tx=0x7f6dcc0a8760 comp rx=0 tx=0).stop 2023-07-14T18:30:05.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.182+0000 7f6dec47eb80 1 -- 172.21.15.133:0/1014149794 shutdown_connections 2023-07-14T18:30:05.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.182+0000 7f6dec47eb80 1 --2- 172.21.15.133:0/1014149794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9e07764b0 0x55a9e078c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:05.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.182+0000 7f6dec47eb80 1 --2- 172.21.15.133:0/1014149794 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9e0774f60 0x55a9e077d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:05.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.182+0000 7f6dec47eb80 1 --2- 172.21.15.133:0/1014149794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9e07758b0 0x55a9e0785f10 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:05.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.182+0000 7f6dec47eb80 1 -- 172.21.15.133:0/1014149794 >> 172.21.15.133:0/1014149794 conn(0x55a9e076a680 msgr2=0x55a9e076f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:30:05.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.188+0000 7f6dec47eb80 1 -- 172.21.15.133:0/1014149794 shutdown_connections 2023-07-14T18:30:05.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.188+0000 7f6dec47eb80 1 -- 172.21.15.133:0/1014149794 wait complete. 2023-07-14T18:30:05.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.189+0000 7f6dec47eb80 1 -- start start 2023-07-14T18:30:05.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.190+0000 7f6dec47eb80 1 Processor -- start 2023-07-14T18:30:05.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.190+0000 7f6dec47eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9e0774f60 0x55a9e0781dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:05.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.191+0000 7f6dec47eb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9e07758b0 0x55a9e0782310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:05.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.191+0000 7f6dec47eb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9e07764b0 0x55a9e07852c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:05.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.191+0000 7f6dec47eb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9e0785800 con 0x55a9e07758b0 2023-07-14T18:30:05.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.191+0000 7f6dec47eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9e0785970 con 0x55a9e0774f60 2023-07-14T18:30:05.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.191+0000 7f6dec47eb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9e0785b30 con 0x55a9e07764b0 2023-07-14T18:30:05.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.191+0000 7f6dec47eb80 10 client.0 _ll_register_callbacks cb 0x7f6dcc0abc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:30:05.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.191+0000 7f6dd259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9e0774f60 0x55a9e0781dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:30:05.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.192+0000 7f6dd2d9d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9e07764b0 0x55a9e07852c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:30:05.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.192+0000 7f6dd2d9d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9e07764b0 0x55a9e07852c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:58738/0 (socket says 172.21.15.133:58738) 2023-07-14T18:30:05.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.192+0000 7f6dd2d9d700 1 -- 172.21.15.133:0/153265832 learned_addr learned my addr 172.21.15.133:0/153265832 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:30:05.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.197+0000 7f6dd1d9b700 1 --2- 172.21.15.133:0/153265832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9e07758b0 0x55a9e0782310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:30:05.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.197+0000 7f6dd2d9d700 1 -- 172.21.15.133:0/153265832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9e0774f60 msgr2=0x55a9e0781dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:30:05.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.197+0000 7f6dd2d9d700 1 --2- 172.21.15.133:0/153265832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9e0774f60 0x55a9e0781dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:05.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.197+0000 7f6dd2d9d700 1 -- 172.21.15.133:0/153265832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9e07758b0 msgr2=0x55a9e0782310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:30:05.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.197+0000 7f6dd2d9d700 1 --2- 172.21.15.133:0/153265832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9e07758b0 0x55a9e0782310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:05.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.197+0000 7f6dd2d9d700 1 -- 172.21.15.133:0/153265832 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dcc0da050 con 0x55a9e07764b0 2023-07-14T18:30:05.251 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.221+0000 7f6dd2d9d700 1 --2- 172.21.15.133:0/153265832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9e07764b0 0x55a9e07852c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8011540 tx=0x7f6dc8011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5232e097cba40eda server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:30:05.252 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.222+0000 7f6db9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:30:05.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.224+0000 7f6db9ffb700 1 -- 172.21.15.133:0/153265832 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8009e70 con 0x55a9e07764b0 2023-07-14T18:30:05.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.224+0000 7f6db9ffb700 1 -- 172.21.15.133:0/153265832 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6dc8014650 con 0x55a9e07764b0 2023-07-14T18:30:05.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.224+0000 7f6dec47eb80 1 -- 172.21.15.133:0/153265832 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9e06beeb0 con 0x55a9e07764b0 2023-07-14T18:30:05.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.230+0000 7f6db9ffb700 1 -- 172.21.15.133:0/153265832 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6dc8014810 con 0x55a9e07764b0 2023-07-14T18:30:05.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.236+0000 7f6dec47eb80 10 client.26005 fetch_fsmap learned FSMap version 45 2023-07-14T18:30:05.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.236+0000 7f6dec47eb80 1 -- 172.21.15.133:0/153265832 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55a9e04f22b0 con 0x55a9e07764b0 2023-07-14T18:30:05.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.244+0000 7f6db9ffb700 1 -- 172.21.15.133:0/153265832 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a9e06beeb0 con 0x55a9e07764b0 2023-07-14T18:30:05.279 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.244+0000 7f6db9ffb700 1 -- 172.21.15.133:0/153265832 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 45) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f6dc801b020 con 0x55a9e07764b0 2023-07-14T18:30:05.279 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.248+0000 7f6dec47eb80 10 client.26005 fetch_fsmap finished waiting for FSMap version 45 2023-07-14T18:30:05.279 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.248+0000 7f6dec47eb80 10 client.26005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:30:05.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.248+0000 7f6dec47eb80 20 client.26005 populate_metadata read hostname 'smithi133' 2023-07-14T18:30:05.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.248+0000 7f6dec47eb80 1 --2- 172.21.15.133:0/153265832 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55a9e07df190 0x55a9e07e15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:05.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.248+0000 7f6dec47eb80 4 client.26005 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:30:05.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.248+0000 7f6dec47eb80 1 -- 172.21.15.133:0/153265832 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a9e0681050 con 0x55a9e07df190 2023-07-14T18:30:05.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.248+0000 7f6dd259c700 1 --2- 172.21.15.133:0/153265832 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55a9e07df190 0x55a9e07e15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:30:05.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.278+0000 7f6dd259c700 1 --2- 172.21.15.133:0/153265832 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55a9e07df190 0x55a9e07e15a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7745a7db9f099a89 server_cookie=f5483b595d43c21b in_seq=0 out_seq=0 2023-07-14T18:30:05.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.278+0000 7f6db9ffb700 10 client.26005 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:30:05.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.292+0000 7f6db9ffb700 1 -- 172.21.15.133:0/153265832 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55a9e0681050 con 0x55a9e07df190 2023-07-14T18:30:05.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.292+0000 7f6db9ffb700 10 client.26005 handle_command_reply: tid=0 2023-07-14T18:30:05.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.293+0000 7f6dec47eb80 1 client.26005 shutdown 2023-07-14T18:30:05.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.305+0000 7f6dec47eb80 1 -- 172.21.15.133:0/153265832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9e07764b0 msgr2=0x55a9e07852c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:30:05.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.305+0000 7f6dec47eb80 1 --2- 172.21.15.133:0/153265832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9e07764b0 0x55a9e07852c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8011540 tx=0x7f6dc8011900 comp rx=0 tx=0).stop 2023-07-14T18:30:05.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.307+0000 7f6dec47eb80 1 -- 172.21.15.133:0/153265832 shutdown_connections 2023-07-14T18:30:05.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.307+0000 7f6dec47eb80 1 --2- 172.21.15.133:0/153265832 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55a9e07df190 0x55a9e07e15a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:05.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.307+0000 7f6dec47eb80 1 --2- 172.21.15.133:0/153265832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9e0774f60 0x55a9e0781dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:05.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.307+0000 7f6dec47eb80 1 --2- 172.21.15.133:0/153265832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9e07764b0 0x55a9e07852c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:05.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.307+0000 7f6dec47eb80 1 --2- 172.21.15.133:0/153265832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9e07758b0 0x55a9e0782310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:05.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.307+0000 7f6dec47eb80 1 -- 172.21.15.133:0/153265832 >> 172.21.15.133:0/153265832 conn(0x55a9e076a680 msgr2=0x55a9e078c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:30:05.345 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:30:05.345 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T18:30:05.345 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:30:05.345 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:30:05.345 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:30:05.345 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:30:05.346 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:30:05.346 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:30:05.346 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:30:05.346 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:30:05.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.313+0000 7f6db9ffb700 0 client.26005 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:30:05.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.313+0000 7f6dec47eb80 1 -- 172.21.15.133:0/153265832 shutdown_connections 2023-07-14T18:30:05.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.313+0000 7f6dec47eb80 1 -- 172.21.15.133:0/153265832 wait complete. 2023-07-14T18:30:05.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.313+0000 7f6dec47eb80 20 client.26005 trim_cache size 0 max 16384 2023-07-14T18:30:05.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.321+0000 7f6dd8bff700 1 -- 172.21.15.133:0/2223500911 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6dbc0a1350 msgr2=0x7f6dbc0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:30:05.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.321+0000 7f6dd8bff700 1 --2- 172.21.15.133:0/2223500911 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6dbc0a1350 0x7f6dbc0a3800 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4005fd0 tx=0x7f6dc400a560 comp rx=0 tx=0).stop 2023-07-14T18:30:05.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.321+0000 7f6dd8bff700 1 -- 172.21.15.133:0/2223500911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6dd4100de0 msgr2=0x7f6dd406e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:30:05.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.321+0000 7f6dd8bff700 1 --2- 172.21.15.133:0/2223500911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6dd4100de0 0x7f6dd406e5c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc01e010 tx=0x7f6dcc01a710 comp rx=0 tx=0).stop 2023-07-14T18:30:05.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.331+0000 7f6dd8bff700 1 -- 172.21.15.133:0/2223500911 shutdown_connections 2023-07-14T18:30:05.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.331+0000 7f6dd8bff700 1 --2- 172.21.15.133:0/2223500911 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6dbc0a1350 0x7f6dbc0a3800 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:05.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.331+0000 7f6dd8bff700 1 --2- 172.21.15.133:0/2223500911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dd41037a0 0x7f6dd4070c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:05.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.331+0000 7f6dd8bff700 1 --2- 172.21.15.133:0/2223500911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6dd40fe480 0x7f6dd406c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:05.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.331+0000 7f6dd8bff700 1 --2- 172.21.15.133:0/2223500911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6dd4100de0 0x7f6dd406e5c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:05.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.331+0000 7f6dd8bff700 1 -- 172.21.15.133:0/2223500911 >> 172.21.15.133:0/2223500911 conn(0x7f6dd40fa130 msgr2=0x7f6dd40fc510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:30:05.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.344+0000 7f6dd8bff700 1 -- 172.21.15.133:0/2223500911 shutdown_connections 2023-07-14T18:30:05.384 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:05.347+0000 7f6dd8bff700 1 -- 172.21.15.133:0/2223500911 wait complete. 2023-07-14T18:30:05.445 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:30:05.861 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 640.0 s, 740.3 tps, lat 84.921 ms stddev 76.085 2023-07-14T18:30:05.894 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:05 smithi133 ceph-mon[126384]: pgmap v750: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 704 op/s 2023-07-14T18:30:06.091 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:05 smithi161 ceph-mon[128993]: pgmap v750: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 704 op/s 2023-07-14T18:30:06.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:05 smithi171 ceph-mon[129241]: pgmap v750: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 704 op/s 2023-07-14T18:30:07.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:07 smithi161 ceph-mon[128993]: pgmap v751: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 22 MiB/s wr, 1.36k op/s 2023-07-14T18:30:07.602 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:07 smithi171 ceph-mon[129241]: pgmap v751: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 22 MiB/s wr, 1.36k op/s 2023-07-14T18:30:07.643 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:07 smithi133 ceph-mon[126384]: pgmap v751: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 22 MiB/s wr, 1.36k op/s 2023-07-14T18:30:09.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:09 smithi161 ceph-mon[128993]: pgmap v752: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 22 MiB/s wr, 1.60k op/s 2023-07-14T18:30:09.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:09 smithi171 ceph-mon[129241]: pgmap v752: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 22 MiB/s wr, 1.60k op/s 2023-07-14T18:30:09.897 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:09 smithi133 ceph-mon[126384]: pgmap v752: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 22 MiB/s wr, 1.60k op/s 2023-07-14T18:30:10.860 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 645.0 s, 942.3 tps, lat 145.788 ms stddev 438.928 2023-07-14T18:30:11.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:11 smithi161 ceph-mon[128993]: pgmap v753: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 19 MiB/s wr, 1.49k op/s 2023-07-14T18:30:11.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:11 smithi171 ceph-mon[129241]: pgmap v753: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 19 MiB/s wr, 1.49k op/s 2023-07-14T18:30:11.894 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:11 smithi133 ceph-mon[126384]: pgmap v753: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 19 MiB/s wr, 1.49k op/s 2023-07-14T18:30:13.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:13 smithi133 ceph-mon[126384]: pgmap v754: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 23 MiB/s wr, 1.63k op/s 2023-07-14T18:30:14.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:13 smithi171 ceph-mon[129241]: pgmap v754: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 23 MiB/s wr, 1.63k op/s 2023-07-14T18:30:14.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:13 smithi161 ceph-mon[128993]: pgmap v754: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 23 MiB/s wr, 1.63k op/s 2023-07-14T18:30:15.860 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 650.0 s, 774.4 tps, lat 128.339 ms stddev 166.861 2023-07-14T18:30:15.904 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:15 smithi133 ceph-mon[126384]: pgmap v755: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 19 MiB/s wr, 1.53k op/s 2023-07-14T18:30:16.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:15 smithi161 ceph-mon[128993]: pgmap v755: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 19 MiB/s wr, 1.53k op/s 2023-07-14T18:30:16.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:15 smithi171 ceph-mon[129241]: pgmap v755: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 19 MiB/s wr, 1.53k op/s 2023-07-14T18:30:18.090 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:17 smithi161 ceph-mon[128993]: pgmap v756: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 22 MiB/s wr, 1.68k op/s 2023-07-14T18:30:18.090 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:17 smithi161 ceph-mon[128993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:30:18.090 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:17 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:30:18.106 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:17 smithi171 ceph-mon[129241]: pgmap v756: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 22 MiB/s wr, 1.68k op/s 2023-07-14T18:30:18.106 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:17 smithi171 ceph-mon[129241]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:30:18.106 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:17 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:30:18.142 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:17 smithi133 ceph-mon[126384]: pgmap v756: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 22 MiB/s wr, 1.68k op/s 2023-07-14T18:30:18.142 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:17 smithi133 ceph-mon[126384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:30:18.142 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:17 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:30:20.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:19 smithi161 ceph-mon[128993]: pgmap v757: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 15 MiB/s wr, 1.06k op/s 2023-07-14T18:30:20.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:19 smithi171 ceph-mon[129241]: pgmap v757: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 15 MiB/s wr, 1.06k op/s 2023-07-14T18:30:20.143 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:19 smithi133 ceph-mon[126384]: pgmap v757: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 15 MiB/s wr, 1.06k op/s 2023-07-14T18:30:20.862 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 655.0 s, 304.7 tps, lat 181.639 ms stddev 199.143 2023-07-14T18:30:22.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:21 smithi171 ceph-mon[129241]: pgmap v758: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 15 MiB/s wr, 914 op/s 2023-07-14T18:30:22.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:21 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:30:22.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:21 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:30:22.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:21 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:30:22.142 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:21 smithi133 ceph-mon[126384]: pgmap v758: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 15 MiB/s wr, 914 op/s 2023-07-14T18:30:22.142 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:21 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:30:22.142 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:21 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:30:22.143 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:21 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:30:22.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:21 smithi161 ceph-mon[128993]: pgmap v758: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 15 MiB/s wr, 914 op/s 2023-07-14T18:30:22.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:21 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:30:22.337 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:21 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:30:22.337 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:21 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:30:24.142 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:23 smithi133 ceph-mon[126384]: pgmap v759: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 20 MiB/s wr, 1.47k op/s 2023-07-14T18:30:24.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:23 smithi161 ceph-mon[128993]: pgmap v759: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 20 MiB/s wr, 1.47k op/s 2023-07-14T18:30:24.351 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:23 smithi171 ceph-mon[129241]: pgmap v759: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 20 MiB/s wr, 1.47k op/s 2023-07-14T18:30:25.860 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 660.0 s, 617.1 tps, lat 234.740 ms stddev 483.565 2023-07-14T18:30:26.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:25 smithi161 ceph-mon[128993]: pgmap v760: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 16 MiB/s wr, 1.40k op/s 2023-07-14T18:30:26.351 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:25 smithi171 ceph-mon[129241]: pgmap v760: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 16 MiB/s wr, 1.40k op/s 2023-07-14T18:30:26.392 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:25 smithi133 ceph-mon[126384]: pgmap v760: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 16 MiB/s wr, 1.40k op/s 2023-07-14T18:30:27.589 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:27 smithi161 ceph-mon[128993]: pgmap v761: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 18 MiB/s wr, 1.50k op/s 2023-07-14T18:30:27.602 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:27 smithi171 ceph-mon[129241]: pgmap v761: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 18 MiB/s wr, 1.50k op/s 2023-07-14T18:30:27.644 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:27 smithi133 ceph-mon[126384]: pgmap v761: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 18 MiB/s wr, 1.50k op/s 2023-07-14T18:30:29.893 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:29 smithi133 ceph-mon[126384]: pgmap v762: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 16 MiB/s wr, 1.42k op/s 2023-07-14T18:30:30.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:29 smithi161 ceph-mon[128993]: pgmap v762: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 16 MiB/s wr, 1.42k op/s 2023-07-14T18:30:30.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:29 smithi171 ceph-mon[129241]: pgmap v762: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 16 MiB/s wr, 1.42k op/s 2023-07-14T18:30:30.863 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 665.0 s, 1176.9 tps, lat 84.401 ms stddev 60.430 2023-07-14T18:30:32.101 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:31 smithi161 ceph-mon[128993]: pgmap v763: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 18 MiB/s wr, 1.41k op/s 2023-07-14T18:30:32.105 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:31 smithi171 ceph-mon[129241]: pgmap v763: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 18 MiB/s wr, 1.41k op/s 2023-07-14T18:30:32.143 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:31 smithi133 ceph-mon[126384]: pgmap v763: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 18 MiB/s wr, 1.41k op/s 2023-07-14T18:30:34.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:33 smithi161 ceph-mon[128993]: pgmap v764: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 19 MiB/s wr, 1.27k op/s 2023-07-14T18:30:34.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:33 smithi171 ceph-mon[129241]: pgmap v764: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 19 MiB/s wr, 1.27k op/s 2023-07-14T18:30:34.144 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:33 smithi133 ceph-mon[126384]: pgmap v764: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 19 MiB/s wr, 1.27k op/s 2023-07-14T18:30:35.447 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:30:35.867 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 670.0 s, 748.9 tps, lat 124.873 ms stddev 131.191 2023-07-14T18:30:35.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.968+0000 7f4077fff700 1 -- 172.21.15.133:0/3096475412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4070071960 msgr2=0x7f407010b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:30:35.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.968+0000 7f4077fff700 1 --2- 172.21.15.133:0/3096475412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4070071960 0x7f407010b900 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f406001a9a0 tx=0x7f406001acb0 comp rx=0 tx=0).stop 2023-07-14T18:30:35.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.968+0000 7f4077fff700 1 -- 172.21.15.133:0/3096475412 shutdown_connections 2023-07-14T18:30:35.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.968+0000 7f4077fff700 1 --2- 172.21.15.133:0/3096475412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f407010be90 0x7f407010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:35.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.968+0000 7f4077fff700 1 --2- 172.21.15.133:0/3096475412 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4070071010 0x7f4070071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:35.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.968+0000 7f4077fff700 1 --2- 172.21.15.133:0/3096475412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4070071960 0x7f407010b900 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:35.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.968+0000 7f4077fff700 1 -- 172.21.15.133:0/3096475412 >> 172.21.15.133:0/3096475412 conn(0x7f407006cbc0 msgr2=0x7f407006eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:30:36.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.970+0000 7f4077fff700 1 -- 172.21.15.133:0/3096475412 shutdown_connections 2023-07-14T18:30:36.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.970+0000 7f4077fff700 1 -- 172.21.15.133:0/3096475412 wait complete. 2023-07-14T18:30:36.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.972+0000 7f4077fff700 1 Processor -- start 2023-07-14T18:30:36.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.972+0000 7f4077fff700 1 -- start start 2023-07-14T18:30:36.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.972+0000 7f4077fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4070071010 0x7f407019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:36.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.972+0000 7f4077fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4070071960 0x7f40701a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:36.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.972+0000 7f4077fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f407010be90 0x7f40701a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:36.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.972+0000 7f4077fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40701a3bb0 con 0x7f4070071960 2023-07-14T18:30:36.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.972+0000 7f4077fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40701a3d20 con 0x7f407010be90 2023-07-14T18:30:36.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.972+0000 7f4077fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40701a3ee0 con 0x7f4070071010 2023-07-14T18:30:36.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.974+0000 7f407559a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4070071960 0x7f40701a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:30:36.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.974+0000 7f407559a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4070071960 0x7f40701a0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51278/0 (socket says 172.21.15.133:51278) 2023-07-14T18:30:36.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.974+0000 7f407559a700 1 -- 172.21.15.133:0/2284967714 learned_addr learned my addr 172.21.15.133:0/2284967714 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:30:36.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.975+0000 7f407559a700 1 -- 172.21.15.133:0/2284967714 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4070071010 msgr2=0x7f407019e8c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-14T18:30:36.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.975+0000 7f407559a700 1 --2- 172.21.15.133:0/2284967714 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4070071010 0x7f407019e8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.975+0000 7f407559a700 1 -- 172.21.15.133:0/2284967714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f407010be90 msgr2=0x7f40701a33e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-14T18:30:36.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.975+0000 7f407559a700 1 --2- 172.21.15.133:0/2284967714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f407010be90 0x7f40701a33e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.975+0000 7f407559a700 1 -- 172.21.15.133:0/2284967714 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406001a600 con 0x7f4070071960 2023-07-14T18:30:36.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.976+0000 7f407559a700 1 --2- 172.21.15.133:0/2284967714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4070071960 0x7f40701a0e10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f406001af50 tx=0x7f4060003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:30:36.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.982+0000 7f4066ffd700 1 -- 172.21.15.133:0/2284967714 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406002d070 con 0x7f4070071960 2023-07-14T18:30:36.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.982+0000 7f4066ffd700 1 -- 172.21.15.133:0/2284967714 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4060031410 con 0x7f4070071960 2023-07-14T18:30:36.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.982+0000 7f4066ffd700 1 -- 172.21.15.133:0/2284967714 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40600283f0 con 0x7f4070071960 2023-07-14T18:30:36.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.983+0000 7f4077fff700 1 -- 172.21.15.133:0/2284967714 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40701a4160 con 0x7f4070071960 2023-07-14T18:30:36.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.984+0000 7f4077fff700 1 -- 172.21.15.133:0/2284967714 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40701a4530 con 0x7f4070071960 2023-07-14T18:30:36.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.992+0000 7f408b8efb80 1 Processor -- start 2023-07-14T18:30:36.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.994+0000 7f408b8efb80 1 -- start start 2023-07-14T18:30:36.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.995+0000 7f408b8efb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ceb2f34f60 0x55ceb2f35370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:36.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.995+0000 7f408b8efb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceb2f358b0 0x55ceb2f35ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:36.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.995+0000 7f408b8efb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ceb2f364b0 0x55ceb2f36920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:36.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.995+0000 7f408b8efb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb2e39cb0 con 0x55ceb2f358b0 2023-07-14T18:30:36.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.995+0000 7f408b8efb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb2e8ca40 con 0x55ceb2f34f60 2023-07-14T18:30:36.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.995+0000 7f408b8efb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb2e85ed0 con 0x55ceb2f364b0 2023-07-14T18:30:36.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.995+0000 7f407559a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ceb2f364b0 0x55ceb2f36920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:30:36.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.995+0000 7f407559a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ceb2f364b0 0x55ceb2f36920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:37380/0 (socket says 172.21.15.133:37380) 2023-07-14T18:30:36.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.996+0000 7f407559a700 1 -- 172.21.15.133:0/3123362689 learned_addr learned my addr 172.21.15.133:0/3123362689 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:30:36.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.996+0000 7f407659c700 1 --2- 172.21.15.133:0/3123362689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ceb2f34f60 0x55ceb2f35370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:30:36.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.996+0000 7f4066ffd700 1 -- 172.21.15.133:0/2284967714 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4060004030 con 0x7f4070071960 2023-07-14T18:30:36.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.996+0000 7f407559a700 1 -- 172.21.15.133:0/3123362689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ceb2f34f60 msgr2=0x55ceb2f35370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:30:36.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.996+0000 7f407559a700 1 --2- 172.21.15.133:0/3123362689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ceb2f34f60 0x55ceb2f35370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.996+0000 7f407559a700 1 -- 172.21.15.133:0/3123362689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceb2f358b0 msgr2=0x55ceb2f35ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:30:36.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.996+0000 7f407559a700 1 --2- 172.21.15.133:0/3123362689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceb2f358b0 0x55ceb2f35ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.996+0000 7f407559a700 1 -- 172.21.15.133:0/3123362689 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ceb2cb3de0 con 0x55ceb2f364b0 2023-07-14T18:30:36.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.997+0000 7f407559a700 1 --2- 172.21.15.133:0/3123362689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ceb2f364b0 0x55ceb2f36920 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f40600e45d0 tx=0x7f40600e48e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63a86d937ec3d59b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:30:36.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.999+0000 7f4066ffd700 1 --2- 172.21.15.133:0/2284967714 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f405c0a1320 0x7f405c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:36.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:35.999+0000 7f4066ffd700 1 -- 172.21.15.133:0/2284967714 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f40600de580 con 0x7f4070071960 2023-07-14T18:30:36.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.003+0000 7f407659c700 1 --2- 172.21.15.133:0/2284967714 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f405c0a1320 0x7f405c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:30:36.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.004+0000 7f405bfff700 1 -- 172.21.15.133:0/3123362689 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40600ded80 con 0x55ceb2f364b0 2023-07-14T18:30:36.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.004+0000 7f405bfff700 1 -- 172.21.15.133:0/3123362689 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40600e0590 con 0x55ceb2f364b0 2023-07-14T18:30:36.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.004+0000 7f405bfff700 1 -- 172.21.15.133:0/3123362689 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40600ea980 con 0x55ceb2f364b0 2023-07-14T18:30:36.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.005+0000 7f408b8efb80 1 -- 172.21.15.133:0/3123362689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ceb2f364b0 msgr2=0x55ceb2f36920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:30:36.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.005+0000 7f408b8efb80 1 --2- 172.21.15.133:0/3123362689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ceb2f364b0 0x55ceb2f36920 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f40600e45d0 tx=0x7f40600e48e0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.006+0000 7f407659c700 1 --2- 172.21.15.133:0/2284967714 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f405c0a1320 0x7f405c0a37d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4070117580 tx=0x7f406c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:30:36.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.006+0000 7f408b8efb80 1 -- 172.21.15.133:0/3123362689 shutdown_connections 2023-07-14T18:30:36.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.006+0000 7f408b8efb80 1 --2- 172.21.15.133:0/3123362689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ceb2f34f60 0x55ceb2f35370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.006+0000 7f408b8efb80 1 --2- 172.21.15.133:0/3123362689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ceb2f364b0 0x55ceb2f36920 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.006+0000 7f408b8efb80 1 --2- 172.21.15.133:0/3123362689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceb2f358b0 0x55ceb2f35ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.006+0000 7f408b8efb80 1 -- 172.21.15.133:0/3123362689 >> 172.21.15.133:0/3123362689 conn(0x55ceb2f2a680 msgr2=0x55ceb2f2f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:30:36.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.008+0000 7f408b8efb80 1 -- 172.21.15.133:0/3123362689 shutdown_connections 2023-07-14T18:30:36.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.008+0000 7f408b8efb80 1 -- 172.21.15.133:0/3123362689 wait complete. 2023-07-14T18:30:36.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.009+0000 7f408b8efb80 1 -- start start 2023-07-14T18:30:36.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.009+0000 7f408b8efb80 1 Processor -- start 2023-07-14T18:30:36.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.016+0000 7f408b8efb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceb2f34f60 0x55ceb2f3e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:36.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.016+0000 7f408b8efb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ceb2f358b0 0x55ceb2f3ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:36.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.017+0000 7f408b8efb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ceb2f364b0 0x55ceb2f45bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:36.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.017+0000 7f408b8efb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb2e25ac0 con 0x55ceb2f34f60 2023-07-14T18:30:36.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.017+0000 7f408b8efb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb2e257c0 con 0x55ceb2f358b0 2023-07-14T18:30:36.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.017+0000 7f408b8efb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb2d09920 con 0x55ceb2f364b0 2023-07-14T18:30:36.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.017+0000 7f408b8efb80 10 client.0 _ll_register_callbacks cb 0x55ceb2e23760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:30:36.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.021+0000 7f407559a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ceb2f358b0 0x55ceb2f3ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:30:36.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.021+0000 7f407559a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ceb2f358b0 0x55ceb2f3ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:52288/0 (socket says 172.21.15.133:52288) 2023-07-14T18:30:36.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.021+0000 7f407559a700 1 -- 172.21.15.133:0/1855651818 learned_addr learned my addr 172.21.15.133:0/1855651818 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:30:36.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.023+0000 7f407559a700 1 -- 172.21.15.133:0/1855651818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ceb2f364b0 msgr2=0x55ceb2f45bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T18:30:36.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.023+0000 7f407559a700 1 --2- 172.21.15.133:0/1855651818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ceb2f364b0 0x55ceb2f45bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.023+0000 7f407559a700 1 -- 172.21.15.133:0/1855651818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceb2f34f60 msgr2=0x55ceb2f3e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:30:36.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.023+0000 7f407559a700 1 --2- 172.21.15.133:0/1855651818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceb2f34f60 0x55ceb2f3e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.023+0000 7f407559a700 1 -- 172.21.15.133:0/1855651818 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40701a4530 con 0x55ceb2f358b0 2023-07-14T18:30:36.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.024+0000 7f407559a700 1 --2- 172.21.15.133:0/1855651818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ceb2f358b0 0x55ceb2f3ebb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f40600a8100 tx=0x7f406001a2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=183d8763fe11a96c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:30:36.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.024+0000 7f40597fa700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:30:36.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.024+0000 7f40597fa700 1 -- 172.21.15.133:0/1855651818 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40600a79d0 con 0x55ceb2f358b0 2023-07-14T18:30:36.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.024+0000 7f40597fa700 1 -- 172.21.15.133:0/1855651818 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f40600cb0d0 con 0x55ceb2f358b0 2023-07-14T18:30:36.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.024+0000 7f408b8efb80 1 -- 172.21.15.133:0/1855651818 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ceb2ddf4f0 con 0x55ceb2f358b0 2023-07-14T18:30:36.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.025+0000 7f40597fa700 1 -- 172.21.15.133:0/1855651818 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40600ab460 con 0x55ceb2f358b0 2023-07-14T18:30:36.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.025+0000 7f408b8efb80 10 client.25979 fetch_fsmap learned FSMap version 46 2023-07-14T18:30:36.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.025+0000 7f408b8efb80 1 -- 172.21.15.133:0/1855651818 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55ceb2c9cc60 con 0x55ceb2f358b0 2023-07-14T18:30:36.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.025+0000 7f40597fa700 1 -- 172.21.15.133:0/1855651818 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ceb2ddf4f0 con 0x55ceb2f358b0 2023-07-14T18:30:36.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.026+0000 7f40597fa700 1 -- 172.21.15.133:0/1855651818 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f40600a7da0 con 0x55ceb2f358b0 2023-07-14T18:30:36.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.026+0000 7f408b8efb80 10 client.25979 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T18:30:36.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.026+0000 7f408b8efb80 10 client.25979 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:30:36.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.026+0000 7f408b8efb80 20 client.25979 populate_metadata read hostname 'smithi133' 2023-07-14T18:30:36.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.026+0000 7f408b8efb80 1 --2- 172.21.15.133:0/1855651818 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55ceb2f49f70 0x55ceb2f4c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:36.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.026+0000 7f408b8efb80 4 client.25979 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:30:36.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.026+0000 7f408b8efb80 1 -- 172.21.15.133:0/1855651818 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ceb2f03da0 con 0x55ceb2f49f70 2023-07-14T18:30:36.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.026+0000 7f4075d9b700 1 --2- 172.21.15.133:0/1855651818 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55ceb2f49f70 0x55ceb2f4c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:30:36.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.040+0000 7f4075d9b700 1 --2- 172.21.15.133:0/1855651818 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55ceb2f49f70 0x55ceb2f4c380 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3932c11a817d1f21 server_cookie=4cfbb4140de99f58 in_seq=0 out_seq=0 2023-07-14T18:30:36.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.040+0000 7f40597fa700 10 client.25979 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:30:36.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.044+0000 7f40597fa700 1 -- 172.21.15.133:0/1855651818 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55ceb2f03da0 con 0x55ceb2f49f70 2023-07-14T18:30:36.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.044+0000 7f40597fa700 10 client.25979 handle_command_reply: tid=0 2023-07-14T18:30:36.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.052+0000 7f408b8efb80 1 client.25979 shutdown 2023-07-14T18:30:36.092 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:35 smithi161 ceph-mon[128993]: pgmap v765: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 14 MiB/s wr, 683 op/s 2023-07-14T18:30:36.103 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:35 smithi171 ceph-mon[129241]: pgmap v765: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 14 MiB/s wr, 683 op/s 2023-07-14T18:30:36.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.079+0000 7f408b8efb80 1 -- 172.21.15.133:0/1855651818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ceb2f358b0 msgr2=0x55ceb2f3ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:30:36.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.079+0000 7f408b8efb80 1 --2- 172.21.15.133:0/1855651818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ceb2f358b0 0x55ceb2f3ebb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f40600a8100 tx=0x7f406001a2c0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.097+0000 7f408b8efb80 1 -- 172.21.15.133:0/1855651818 shutdown_connections 2023-07-14T18:30:36.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.097+0000 7f408b8efb80 1 --2- 172.21.15.133:0/1855651818 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55ceb2f49f70 0x55ceb2f4c380 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.097+0000 7f408b8efb80 1 --2- 172.21.15.133:0/1855651818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ceb2f364b0 0x55ceb2f45bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.097+0000 7f408b8efb80 1 --2- 172.21.15.133:0/1855651818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceb2f34f60 0x55ceb2f3e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.097+0000 7f408b8efb80 1 --2- 172.21.15.133:0/1855651818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ceb2f358b0 0x55ceb2f3ebb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.097+0000 7f408b8efb80 1 -- 172.21.15.133:0/1855651818 >> 172.21.15.133:0/1855651818 conn(0x55ceb2f2a680 msgr2=0x55ceb2f2e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:30:36.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.099+0000 7f40597fa700 0 client.25979 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:30:36.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:35 smithi133 ceph-mon[126384]: pgmap v765: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 14 MiB/s wr, 683 op/s 2023-07-14T18:30:36.151 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.116+0000 7f408b8efb80 1 -- 172.21.15.133:0/1855651818 shutdown_connections 2023-07-14T18:30:36.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.126+0000 7f408b8efb80 1 -- 172.21.15.133:0/1855651818 wait complete. 2023-07-14T18:30:36.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.130+0000 7f408b8efb80 20 client.25979 trim_cache size 0 max 16384 2023-07-14T18:30:36.206 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.176+0000 7f408b8efb80 1 Processor -- start 2023-07-14T18:30:36.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.182+0000 7f408b8efb80 1 -- start start 2023-07-14T18:30:36.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.183+0000 7f408b8efb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ceb2f34f60 0x55ceb2f50ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:36.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.183+0000 7f408b8efb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ceb2f358b0 0x55ceb2f45bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:36.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.183+0000 7f408b8efb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceb2f364b0 0x55ceb2f3d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:36.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.183+0000 7f408b8efb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb2f36330 con 0x55ceb2f364b0 2023-07-14T18:30:36.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.183+0000 7f408b8efb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb2f3f570 con 0x55ceb2f34f60 2023-07-14T18:30:36.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.183+0000 7f408b8efb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb2f3db20 con 0x55ceb2f358b0 2023-07-14T18:30:36.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.185+0000 7f407659c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceb2f364b0 0x55ceb2f3d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:30:36.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.185+0000 7f407659c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceb2f364b0 0x55ceb2f3d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51308/0 (socket says 172.21.15.133:51308) 2023-07-14T18:30:36.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.185+0000 7f407659c700 1 -- 172.21.15.133:0/1932422224 learned_addr learned my addr 172.21.15.133:0/1932422224 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:30:36.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.186+0000 7f407659c700 1 -- 172.21.15.133:0/1932422224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ceb2f358b0 msgr2=0x55ceb2f45bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:30:36.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.186+0000 7f407659c700 1 --2- 172.21.15.133:0/1932422224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ceb2f358b0 0x55ceb2f45bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.186+0000 7f407659c700 1 -- 172.21.15.133:0/1932422224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ceb2f34f60 msgr2=0x55ceb2f50ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T18:30:36.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.186+0000 7f407659c700 1 --2- 172.21.15.133:0/1932422224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ceb2f34f60 0x55ceb2f50ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.186+0000 7f407659c700 1 -- 172.21.15.133:0/1932422224 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ceb2cda250 con 0x55ceb2f364b0 2023-07-14T18:30:36.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.186+0000 7f407659c700 1 --2- 172.21.15.133:0/1932422224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceb2f364b0 0x55ceb2f3d3a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f406c007e40 tx=0x7f406c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a98fe213a455997 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:30:36.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.190+0000 7f4064ff9700 1 -- 172.21.15.133:0/1932422224 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406c007e70 con 0x55ceb2f364b0 2023-07-14T18:30:36.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.190+0000 7f4064ff9700 1 -- 172.21.15.133:0/1932422224 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f406c0218c0 con 0x55ceb2f364b0 2023-07-14T18:30:36.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.190+0000 7f4064ff9700 1 -- 172.21.15.133:0/1932422224 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406c014990 con 0x55ceb2f364b0 2023-07-14T18:30:36.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.191+0000 7f408b8efb80 1 -- 172.21.15.133:0/1932422224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceb2f364b0 msgr2=0x55ceb2f3d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:30:36.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.191+0000 7f408b8efb80 1 --2- 172.21.15.133:0/1932422224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceb2f364b0 0x55ceb2f3d3a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f406c007e40 tx=0x7f406c012000 comp rx=0 tx=0).stop 2023-07-14T18:30:36.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.195+0000 7f408b8efb80 1 -- 172.21.15.133:0/1932422224 shutdown_connections 2023-07-14T18:30:36.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.195+0000 7f408b8efb80 1 --2- 172.21.15.133:0/1932422224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ceb2f34f60 0x55ceb2f50ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.195+0000 7f408b8efb80 1 --2- 172.21.15.133:0/1932422224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceb2f364b0 0x55ceb2f3d3a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.195+0000 7f408b8efb80 1 --2- 172.21.15.133:0/1932422224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ceb2f358b0 0x55ceb2f45bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.195+0000 7f408b8efb80 1 -- 172.21.15.133:0/1932422224 >> 172.21.15.133:0/1932422224 conn(0x55ceb2f2a680 msgr2=0x55ceb2f2f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:30:36.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.207+0000 7f408b8efb80 1 -- 172.21.15.133:0/1932422224 shutdown_connections 2023-07-14T18:30:36.238 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.208+0000 7f408b8efb80 1 -- 172.21.15.133:0/1932422224 wait complete. 2023-07-14T18:30:36.238 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.208+0000 7f408b8efb80 1 -- start start 2023-07-14T18:30:36.238 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.209+0000 7f408b8efb80 1 Processor -- start 2023-07-14T18:30:36.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.209+0000 7f408b8efb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ceb2f34f60 0x55ceb2f454a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:36.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.209+0000 7f408b8efb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceb2f358b0 0x55ceb2f459e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:36.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.210+0000 7f408b8efb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ceb2f364b0 0x55ceb2f41f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:36.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.210+0000 7f408b8efb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb2f42510 con 0x55ceb2f358b0 2023-07-14T18:30:36.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.210+0000 7f408b8efb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb2f42680 con 0x55ceb2f364b0 2023-07-14T18:30:36.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.210+0000 7f408b8efb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb2f42840 con 0x55ceb2f34f60 2023-07-14T18:30:36.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.210+0000 7f408b8efb80 10 client.0 _ll_register_callbacks cb 0x7f40600e43a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:30:36.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.214+0000 7f407659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ceb2f364b0 0x55ceb2f41f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:30:36.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.214+0000 7f4075d9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ceb2f34f60 0x55ceb2f454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:30:36.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.214+0000 7f407659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ceb2f364b0 0x55ceb2f41f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:52304/0 (socket says 172.21.15.133:52304) 2023-07-14T18:30:36.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.214+0000 7f407659c700 1 -- 172.21.15.133:0/2868219389 learned_addr learned my addr 172.21.15.133:0/2868219389 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:30:36.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.217+0000 7f407559a700 1 --2- 172.21.15.133:0/2868219389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceb2f358b0 0x55ceb2f459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:30:36.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.218+0000 7f407559a700 1 -- 172.21.15.133:0/2868219389 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ceb2f34f60 msgr2=0x55ceb2f454a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:30:36.251 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.218+0000 7f407559a700 1 --2- 172.21.15.133:0/2868219389 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ceb2f34f60 0x55ceb2f454a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.251 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.218+0000 7f407559a700 1 -- 172.21.15.133:0/2868219389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ceb2f364b0 msgr2=0x55ceb2f41f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:30:36.251 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.218+0000 7f407559a700 1 --2- 172.21.15.133:0/2868219389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ceb2f364b0 0x55ceb2f41f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.251 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.218+0000 7f407559a700 1 -- 172.21.15.133:0/2868219389 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f406c007b80 con 0x55ceb2f358b0 2023-07-14T18:30:36.251 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.218+0000 7f407559a700 1 --2- 172.21.15.133:0/2868219389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceb2f358b0 0x55ceb2f459e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f40600a99e0 tx=0x7f40600de4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672a15245afd3a78 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:30:36.252 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.222+0000 7f40597fa700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T18:30:36.252 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.222+0000 7f40597fa700 1 -- 172.21.15.133:0/2868219389 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406001a2c0 con 0x55ceb2f358b0 2023-07-14T18:30:36.252 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.222+0000 7f40597fa700 1 -- 172.21.15.133:0/2868219389 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f40600cb0d0 con 0x55ceb2f358b0 2023-07-14T18:30:36.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.222+0000 7f408b8efb80 1 -- 172.21.15.133:0/2868219389 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40600e0590 con 0x55ceb2f358b0 2023-07-14T18:30:36.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.223+0000 7f40597fa700 1 -- 172.21.15.133:0/2868219389 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40600aaff0 con 0x55ceb2f358b0 2023-07-14T18:30:36.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.228+0000 7f408b8efb80 10 client.15894 fetch_fsmap learned FSMap version 46 2023-07-14T18:30:36.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.228+0000 7f408b8efb80 1 -- 172.21.15.133:0/2868219389 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55ceb2cb22b0 con 0x55ceb2f358b0 2023-07-14T18:30:36.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.239+0000 7f40597fa700 1 -- 172.21.15.133:0/2868219389 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40600e0590 con 0x55ceb2f358b0 2023-07-14T18:30:36.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.239+0000 7f40597fa700 1 -- 172.21.15.133:0/2868219389 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f40600b35c0 con 0x55ceb2f358b0 2023-07-14T18:30:36.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.241+0000 7f408b8efb80 10 client.15894 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T18:30:36.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.241+0000 7f408b8efb80 10 client.15894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:30:36.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.241+0000 7f408b8efb80 20 client.15894 populate_metadata read hostname 'smithi133' 2023-07-14T18:30:36.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.242+0000 7f408b8efb80 1 --2- 172.21.15.133:0/2868219389 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55ceb2f9eb20 0x55ceb2fa0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:30:36.272 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.242+0000 7f408b8efb80 4 client.15894 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:30:36.272 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.242+0000 7f408b8efb80 1 -- 172.21.15.133:0/2868219389 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ceb2e7eeb0 con 0x55ceb2f9eb20 2023-07-14T18:30:36.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.243+0000 7f4075d9b700 1 --2- 172.21.15.133:0/2868219389 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55ceb2f9eb20 0x55ceb2fa0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:30:36.296 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.267+0000 7f4075d9b700 1 --2- 172.21.15.133:0/2868219389 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55ceb2f9eb20 0x55ceb2fa0f00 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ba4cef0d38982ac server_cookie=bec6cb8b5fa209b4 in_seq=0 out_seq=0 2023-07-14T18:30:36.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.282+0000 7f40597fa700 10 client.15894 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:30:36.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.282+0000 7f40597fa700 1 -- 172.21.15.133:0/2868219389 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55ceb2e7eeb0 con 0x55ceb2f9eb20 2023-07-14T18:30:36.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.282+0000 7f40597fa700 10 client.15894 handle_command_reply: tid=0 2023-07-14T18:30:36.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.282+0000 7f408b8efb80 1 client.15894 shutdown 2023-07-14T18:30:36.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.288+0000 7f408b8efb80 1 -- 172.21.15.133:0/2868219389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceb2f358b0 msgr2=0x55ceb2f459e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:30:36.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.288+0000 7f408b8efb80 1 --2- 172.21.15.133:0/2868219389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceb2f358b0 0x55ceb2f459e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f40600a99e0 tx=0x7f40600de4b0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.292+0000 7f408b8efb80 1 -- 172.21.15.133:0/2868219389 shutdown_connections 2023-07-14T18:30:36.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.292+0000 7f408b8efb80 1 --2- 172.21.15.133:0/2868219389 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55ceb2f9eb20 0x55ceb2fa0f00 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.292+0000 7f408b8efb80 1 --2- 172.21.15.133:0/2868219389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ceb2f364b0 0x55ceb2f41f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.292+0000 7f408b8efb80 1 --2- 172.21.15.133:0/2868219389 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ceb2f34f60 0x55ceb2f454a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.292+0000 7f408b8efb80 1 --2- 172.21.15.133:0/2868219389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceb2f358b0 0x55ceb2f459e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.292+0000 7f408b8efb80 1 -- 172.21.15.133:0/2868219389 >> 172.21.15.133:0/2868219389 conn(0x55ceb2f2a680 msgr2=0x55ceb2f39370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:30:36.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.293+0000 7f40597fa700 0 client.15894 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:30:36.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.294+0000 7f408b8efb80 1 -- 172.21.15.133:0/2868219389 shutdown_connections 2023-07-14T18:30:36.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.294+0000 7f408b8efb80 1 -- 172.21.15.133:0/2868219389 wait complete. 2023-07-14T18:30:36.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.296+0000 7f408b8efb80 20 client.15894 trim_cache size 0 max 16384 2023-07-14T18:30:36.333 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:30:36.333 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T18:30:36.333 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:30:36.334 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:30:36.334 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:30:36.334 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:30:36.334 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:30:36.334 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:30:36.335 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:30:36.335 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:30:36.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.307+0000 7f4077fff700 1 -- 172.21.15.133:0/2284967714 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f405c0a1320 msgr2=0x7f405c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:30:36.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.307+0000 7f4077fff700 1 --2- 172.21.15.133:0/2284967714 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f405c0a1320 0x7f405c0a37d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4070117580 tx=0x7f406c009040 comp rx=0 tx=0).stop 2023-07-14T18:30:36.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.307+0000 7f4077fff700 1 -- 172.21.15.133:0/2284967714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4070071960 msgr2=0x7f40701a0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:30:36.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.307+0000 7f4077fff700 1 --2- 172.21.15.133:0/2284967714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4070071960 0x7f40701a0e10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f406001af50 tx=0x7f4060003c00 comp rx=0 tx=0).stop 2023-07-14T18:30:36.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.312+0000 7f4077fff700 1 -- 172.21.15.133:0/2284967714 shutdown_connections 2023-07-14T18:30:36.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.312+0000 7f4077fff700 1 --2- 172.21.15.133:0/2284967714 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f405c0a1320 0x7f405c0a37d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.312+0000 7f4077fff700 1 --2- 172.21.15.133:0/2284967714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f407010be90 0x7f40701a33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.312+0000 7f4077fff700 1 --2- 172.21.15.133:0/2284967714 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4070071010 0x7f407019e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:30:36.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.312+0000 7f4077fff700 1 --2- 172.21.15.133:0/2284967714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4070071960 0x7f40701a0e10 secure :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f406001af50 tx=0x7f4060003c00 comp rx=0 tx=0).stop 2023-07-14T18:30:36.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.312+0000 7f4077fff700 1 -- 172.21.15.133:0/2284967714 >> 172.21.15.133:0/2284967714 conn(0x7f407006cbc0 msgr2=0x7f407006ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:30:36.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.316+0000 7f4077fff700 1 -- 172.21.15.133:0/2284967714 shutdown_connections 2023-07-14T18:30:36.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:30:36.316+0000 7f4077fff700 1 -- 172.21.15.133:0/2284967714 wait complete. 2023-07-14T18:30:36.396 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:30:37.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:37 smithi161 ceph-mon[128993]: pgmap v766: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 1.03k op/s 2023-07-14T18:30:37.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:37 smithi171 ceph-mon[129241]: pgmap v766: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 1.03k op/s 2023-07-14T18:30:37.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:37 smithi133 ceph-mon[126384]: pgmap v766: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 1.03k op/s 2023-07-14T18:30:39.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:39 smithi171 ceph-mon[129241]: pgmap v767: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 18 MiB/s wr, 1.16k op/s 2023-07-14T18:30:39.895 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:39 smithi133 ceph-mon[126384]: pgmap v767: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 18 MiB/s wr, 1.16k op/s 2023-07-14T18:30:40.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:39 smithi161 ceph-mon[128993]: pgmap v767: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 18 MiB/s wr, 1.16k op/s 2023-07-14T18:30:40.859 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 675.0 s, 779.2 tps, lat 138.042 ms stddev 159.746 2023-07-14T18:30:41.894 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:41 smithi133 ceph-mon[126384]: pgmap v768: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 19 MiB/s wr, 1.32k op/s 2023-07-14T18:30:42.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:41 smithi161 ceph-mon[128993]: pgmap v768: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 19 MiB/s wr, 1.32k op/s 2023-07-14T18:30:42.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:41 smithi171 ceph-mon[129241]: pgmap v768: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 19 MiB/s wr, 1.32k op/s 2023-07-14T18:30:44.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:43 smithi161 ceph-mon[128993]: pgmap v769: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 20 MiB/s wr, 1.56k op/s 2023-07-14T18:30:44.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:43 smithi171 ceph-mon[129241]: pgmap v769: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 20 MiB/s wr, 1.56k op/s 2023-07-14T18:30:44.142 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:43 smithi133 ceph-mon[126384]: pgmap v769: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 20 MiB/s wr, 1.56k op/s 2023-07-14T18:30:45.859 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 680.0 s, 1284.8 tps, lat 77.328 ms stddev 58.053 2023-07-14T18:30:46.090 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:45 smithi161 ceph-mon[128993]: pgmap v770: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 18 MiB/s wr, 1.44k op/s 2023-07-14T18:30:46.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:45 smithi171 ceph-mon[129241]: pgmap v770: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 18 MiB/s wr, 1.44k op/s 2023-07-14T18:30:46.145 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:45 smithi133 ceph-mon[126384]: pgmap v770: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 18 MiB/s wr, 1.44k op/s 2023-07-14T18:30:47.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:47 smithi161 ceph-mon[128993]: pgmap v771: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 1.57k op/s 2023-07-14T18:30:47.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:47 smithi171 ceph-mon[129241]: pgmap v771: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 1.57k op/s 2023-07-14T18:30:47.641 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:47 smithi133 ceph-mon[126384]: pgmap v771: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 1.57k op/s 2023-07-14T18:30:49.895 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:49 smithi133 ceph-mon[126384]: pgmap v772: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 19 MiB/s wr, 1.21k op/s 2023-07-14T18:30:50.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:49 smithi161 ceph-mon[128993]: pgmap v772: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 19 MiB/s wr, 1.21k op/s 2023-07-14T18:30:50.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:49 smithi171 ceph-mon[129241]: pgmap v772: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 19 MiB/s wr, 1.21k op/s 2023-07-14T18:30:50.860 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 685.0 s, 1057.4 tps, lat 94.571 ms stddev 100.031 2023-07-14T18:30:52.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:51 smithi161 ceph-mon[128993]: pgmap v773: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 970 op/s 2023-07-14T18:30:52.104 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:51 smithi171 ceph-mon[129241]: pgmap v773: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 970 op/s 2023-07-14T18:30:52.146 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:51 smithi133 ceph-mon[126384]: pgmap v773: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 970 op/s 2023-07-14T18:30:54.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:53 smithi161 ceph-mon[128993]: pgmap v774: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 948 op/s 2023-07-14T18:30:54.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:53 smithi171 ceph-mon[129241]: pgmap v774: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 948 op/s 2023-07-14T18:30:54.143 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:53 smithi133 ceph-mon[126384]: pgmap v774: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 948 op/s 2023-07-14T18:30:55.858 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 690.0 s, 355.3 tps, lat 239.700 ms stddev 556.329 2023-07-14T18:30:56.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:55 smithi161 ceph-mon[128993]: pgmap v775: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 14 MiB/s wr, 625 op/s 2023-07-14T18:30:56.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:55 smithi171 ceph-mon[129241]: pgmap v775: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 14 MiB/s wr, 625 op/s 2023-07-14T18:30:56.143 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:55 smithi133 ceph-mon[126384]: pgmap v775: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 14 MiB/s wr, 625 op/s 2023-07-14T18:30:57.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:57 smithi161 ceph-mon[128993]: pgmap v776: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 23 MiB/s wr, 1.70k op/s 2023-07-14T18:30:57.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:57 smithi171 ceph-mon[129241]: pgmap v776: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 23 MiB/s wr, 1.70k op/s 2023-07-14T18:30:57.644 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:57 smithi133 ceph-mon[126384]: pgmap v776: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 23 MiB/s wr, 1.70k op/s 2023-07-14T18:31:00.084 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:30:59 smithi161 ceph-mon[128993]: pgmap v777: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 22 MiB/s wr, 1.81k op/s 2023-07-14T18:31:00.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:30:59 smithi171 ceph-mon[129241]: pgmap v777: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 22 MiB/s wr, 1.81k op/s 2023-07-14T18:31:00.142 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:30:59 smithi133 ceph-mon[126384]: pgmap v777: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 22 MiB/s wr, 1.81k op/s 2023-07-14T18:31:00.859 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 695.0 s, 380.1 tps, lat 301.638 ms stddev 389.189 2023-07-14T18:31:02.084 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:01 smithi161 ceph-mon[128993]: pgmap v778: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 21 MiB/s wr, 1.84k op/s 2023-07-14T18:31:02.106 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:01 smithi171 ceph-mon[129241]: pgmap v778: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 21 MiB/s wr, 1.84k op/s 2023-07-14T18:31:02.141 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:01 smithi133 ceph-mon[126384]: pgmap v778: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 21 MiB/s wr, 1.84k op/s 2023-07-14T18:31:04.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:03 smithi161 ceph-mon[128993]: pgmap v779: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 21 MiB/s wr, 2.06k op/s 2023-07-14T18:31:04.104 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:03 smithi171 ceph-mon[129241]: pgmap v779: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 21 MiB/s wr, 2.06k op/s 2023-07-14T18:31:04.142 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:03 smithi133 ceph-mon[126384]: pgmap v779: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 21 MiB/s wr, 2.06k op/s 2023-07-14T18:31:05.860 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 700.0 s, 520.1 tps, lat 179.987 ms stddev 109.671 2023-07-14T18:31:06.093 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:05 smithi161 ceph-mon[128993]: pgmap v780: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 19 MiB/s wr, 1.90k op/s 2023-07-14T18:31:06.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:05 smithi171 ceph-mon[129241]: pgmap v780: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 19 MiB/s wr, 1.90k op/s 2023-07-14T18:31:06.142 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:05 smithi133 ceph-mon[126384]: pgmap v780: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 19 MiB/s wr, 1.90k op/s 2023-07-14T18:31:06.397 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:31:07.569 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:07 smithi133 ceph-mon[126384]: pgmap v781: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 21 MiB/s wr, 2.00k op/s 2023-07-14T18:31:07.584 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:07 smithi161 ceph-mon[128993]: pgmap v781: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 21 MiB/s wr, 2.00k op/s 2023-07-14T18:31:07.599 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:07 smithi171 ceph-mon[129241]: pgmap v781: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 21 MiB/s wr, 2.00k op/s 2023-07-14T18:31:07.865 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.830+0000 7fe3a95ff700 1 -- 172.21.15.133:0/3874119366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3a41037a0 msgr2=0x7fe3a4103c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:31:07.865 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.830+0000 7fe3a95ff700 1 --2- 172.21.15.133:0/3874119366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3a41037a0 0x7fe3a4103c10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00b0a0 tx=0x7fe39c00b3b0 comp rx=0 tx=0).stop 2023-07-14T18:31:07.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.830+0000 7fe3a95ff700 1 -- 172.21.15.133:0/3874119366 shutdown_connections 2023-07-14T18:31:07.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.830+0000 7fe3a95ff700 1 --2- 172.21.15.133:0/3874119366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3a41037a0 0x7fe3a4103c10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:07.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.830+0000 7fe3a95ff700 1 --2- 172.21.15.133:0/3874119366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3a40fe480 0x7fe3a41008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:07.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.830+0000 7fe3a95ff700 1 --2- 172.21.15.133:0/3874119366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3a4100de0 0x7fe3a4103260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:07.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.831+0000 7fe3a95ff700 1 -- 172.21.15.133:0/3874119366 >> 172.21.15.133:0/3874119366 conn(0x7fe3a40fa130 msgr2=0x7fe3a40fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:31:07.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.839+0000 7fe3a95ff700 1 -- 172.21.15.133:0/3874119366 shutdown_connections 2023-07-14T18:31:07.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.847+0000 7fe3a95ff700 1 -- 172.21.15.133:0/3874119366 wait complete. 2023-07-14T18:31:07.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.848+0000 7fe3a95ff700 1 Processor -- start 2023-07-14T18:31:07.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.850+0000 7fe3a95ff700 1 -- start start 2023-07-14T18:31:07.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.850+0000 7fe3a95ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3a40fe480 0x7fe3a41098f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:07.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.850+0000 7fe3a95ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3a4100de0 0x7fe3a4109e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:07.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.850+0000 7fe3a95ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3a41037a0 0x7fe3a41ac290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:07.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.850+0000 7fe3a95ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a410cf10 con 0x7fe3a41037a0 2023-07-14T18:31:07.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.850+0000 7fe3a95ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a410d080 con 0x7fe3a4100de0 2023-07-14T18:31:07.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.850+0000 7fe3a95ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a41ac7d0 con 0x7fe3a40fe480 2023-07-14T18:31:07.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.852+0000 7fe39bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3a4100de0 0x7fe3a4109e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:31:07.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.852+0000 7fe39bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3a4100de0 0x7fe3a4109e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:52406/0 (socket says 172.21.15.133:52406) 2023-07-14T18:31:07.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.852+0000 7fe39bfff700 1 -- 172.21.15.133:0/2651747001 learned_addr learned my addr 172.21.15.133:0/2651747001 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:31:07.890 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.854+0000 7fe39bfff700 1 -- 172.21.15.133:0/2651747001 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3a40fe480 msgr2=0x7fe3a41098f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T18:31:07.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.854+0000 7fe39bfff700 1 --2- 172.21.15.133:0/2651747001 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3a40fe480 0x7fe3a41098f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:07.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.854+0000 7fe39bfff700 1 -- 172.21.15.133:0/2651747001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3a41037a0 msgr2=0x7fe3a41ac290 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T18:31:07.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.854+0000 7fe39bfff700 1 --2- 172.21.15.133:0/2651747001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3a41037a0 0x7fe3a41ac290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:07.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.854+0000 7fe39bfff700 1 -- 172.21.15.133:0/2651747001 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe39c009d00 con 0x7fe3a4100de0 2023-07-14T18:31:07.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.856+0000 7fe39bfff700 1 --2- 172.21.15.133:0/2651747001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3a4100de0 0x7fe3a4109e30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00b550 tx=0x7fe38c00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:31:07.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.866+0000 7fe3a0ff9700 1 -- 172.21.15.133:0/2651747001 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38c011830 con 0x7fe3a4100de0 2023-07-14T18:31:07.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.866+0000 7fe3a0ff9700 1 -- 172.21.15.133:0/2651747001 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe38c011e50 con 0x7fe3a4100de0 2023-07-14T18:31:07.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.866+0000 7fe3a0ff9700 1 -- 172.21.15.133:0/2651747001 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38c010e10 con 0x7fe3a4100de0 2023-07-14T18:31:07.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.874+0000 7fe3a95ff700 1 -- 172.21.15.133:0/2651747001 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a41aca30 con 0x7fe3a4100de0 2023-07-14T18:31:07.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.874+0000 7fe3a95ff700 1 -- 172.21.15.133:0/2651747001 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a41acf40 con 0x7fe3a4100de0 2023-07-14T18:31:07.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.883+0000 7fe3bcf03b80 1 Processor -- start 2023-07-14T18:31:07.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.884+0000 7fe3bcf03b80 1 -- start start 2023-07-14T18:31:07.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.884+0000 7fe3bcf03b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5614c21c4f20 0x5614c21c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:07.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.884+0000 7fe3bcf03b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614c21c5870 0x5614c21c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:07.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.884+0000 7fe3bcf03b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614c21c64a0 0x5614c21c6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:07.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.884+0000 7fe3bcf03b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c20b8cb0 con 0x5614c21c5870 2023-07-14T18:31:07.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.884+0000 7fe3bcf03b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c210ba40 con 0x5614c21c64a0 2023-07-14T18:31:07.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.884+0000 7fe3bcf03b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c2104ed0 con 0x5614c21c4f20 2023-07-14T18:31:07.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.885+0000 7fe3a2ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614c21c5870 0x5614c21c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:31:07.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.885+0000 7fe3a2ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614c21c5870 0x5614c21c5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57248/0 (socket says 172.21.15.133:57248) 2023-07-14T18:31:07.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.885+0000 7fe3a2ffd700 1 -- 172.21.15.133:0/3562333102 learned_addr learned my addr 172.21.15.133:0/3562333102 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:31:07.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.885+0000 7fe3a37fe700 1 --2- 172.21.15.133:0/3562333102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5614c21c4f20 0x5614c21c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:31:07.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.886+0000 7fe3a37fe700 1 -- 172.21.15.133:0/3562333102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614c21c64a0 msgr2=0x5614c21c6910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T18:31:07.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.886+0000 7fe3a37fe700 1 --2- 172.21.15.133:0/3562333102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614c21c64a0 0x5614c21c6910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:07.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.887+0000 7fe3a37fe700 1 -- 172.21.15.133:0/3562333102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614c21c5870 msgr2=0x5614c21c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:31:07.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.887+0000 7fe3a37fe700 1 --2- 172.21.15.133:0/3562333102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614c21c5870 0x5614c21c5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:07.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.887+0000 7fe3a37fe700 1 -- 172.21.15.133:0/3562333102 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614c1f32de0 con 0x5614c21c4f20 2023-07-14T18:31:07.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.887+0000 7fe3a37fe700 1 --2- 172.21.15.133:0/3562333102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5614c21c4f20 0x5614c21c5330 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00b070 tx=0x7fe39c008db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96c1b0cd3178aa16 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:31:07.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.888+0000 7fe3997fa700 1 -- 172.21.15.133:0/3562333102 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c00e040 con 0x5614c21c4f20 2023-07-14T18:31:07.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.888+0000 7fe3997fa700 1 -- 172.21.15.133:0/3562333102 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe39c0093b0 con 0x5614c21c4f20 2023-07-14T18:31:07.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.888+0000 7fe3997fa700 1 -- 172.21.15.133:0/3562333102 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c01b6c0 con 0x5614c21c4f20 2023-07-14T18:31:07.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.889+0000 7fe3bcf03b80 1 -- 172.21.15.133:0/3562333102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5614c21c4f20 msgr2=0x5614c21c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:31:07.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.889+0000 7fe3bcf03b80 1 --2- 172.21.15.133:0/3562333102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5614c21c4f20 0x5614c21c5330 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00b070 tx=0x7fe39c008db0 comp rx=0 tx=0).stop 2023-07-14T18:31:07.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.889+0000 7fe3bcf03b80 1 -- 172.21.15.133:0/3562333102 shutdown_connections 2023-07-14T18:31:07.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.889+0000 7fe3bcf03b80 1 --2- 172.21.15.133:0/3562333102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614c21c64a0 0x5614c21c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:07.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.889+0000 7fe3bcf03b80 1 --2- 172.21.15.133:0/3562333102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5614c21c4f20 0x5614c21c5330 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:07.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.889+0000 7fe3bcf03b80 1 --2- 172.21.15.133:0/3562333102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614c21c5870 0x5614c21c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:07.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.889+0000 7fe3bcf03b80 1 -- 172.21.15.133:0/3562333102 >> 172.21.15.133:0/3562333102 conn(0x5614c21a9680 msgr2=0x5614c21ae9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:31:07.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.900+0000 7fe3bcf03b80 1 -- 172.21.15.133:0/3562333102 shutdown_connections 2023-07-14T18:31:07.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.906+0000 7fe3a0ff9700 1 -- 172.21.15.133:0/2651747001 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe38c010510 con 0x7fe3a4100de0 2023-07-14T18:31:07.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.908+0000 7fe3a0ff9700 1 --2- 172.21.15.133:0/2651747001 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe3900a1260 0x7fe3900a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:07.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.908+0000 7fe3a0ff9700 1 -- 172.21.15.133:0/2651747001 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe38c0cc500 con 0x7fe3a4100de0 2023-07-14T18:31:07.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.908+0000 7fe3bcf03b80 1 -- 172.21.15.133:0/3562333102 wait complete. 2023-07-14T18:31:07.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.909+0000 7fe3bcf03b80 1 -- start start 2023-07-14T18:31:07.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.910+0000 7fe3bcf03b80 1 Processor -- start 2023-07-14T18:31:07.938 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.910+0000 7fe3bcf03b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614c21c4f20 0x5614c21b5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:07.938 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.910+0000 7fe3bcf03b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614c21c5870 0x5614c21b5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:07.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.910+0000 7fe3bcf03b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5614c21ba8b0 0x5614c21bcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:07.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.910+0000 7fe3bcf03b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c20a4ac0 con 0x5614c21c4f20 2023-07-14T18:31:07.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.910+0000 7fe3bcf03b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c20a47c0 con 0x5614c21c5870 2023-07-14T18:31:07.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.910+0000 7fe3bcf03b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c1f88920 con 0x5614c21ba8b0 2023-07-14T18:31:07.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.910+0000 7fe3bcf03b80 10 client.0 _ll_register_callbacks cb 0x5614c20a2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:31:07.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.911+0000 7fe3a37fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5614c21ba8b0 0x5614c21bcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:31:07.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.911+0000 7fe3a37fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5614c21ba8b0 0x5614c21bcd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:59758/0 (socket says 172.21.15.133:59758) 2023-07-14T18:31:07.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.911+0000 7fe3a37fe700 1 -- 172.21.15.133:0/4227670180 learned_addr learned my addr 172.21.15.133:0/4227670180 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:31:07.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.915+0000 7fe3a2ffd700 1 --2- 172.21.15.133:0/4227670180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614c21c4f20 0x5614c21b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:31:07.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.915+0000 7fe3a37fe700 1 -- 172.21.15.133:0/4227670180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614c21c5870 msgr2=0x5614c21b5880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T18:31:07.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.915+0000 7fe3a37fe700 1 --2- 172.21.15.133:0/4227670180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614c21c5870 0x5614c21b5880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:07.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.915+0000 7fe3a37fe700 1 -- 172.21.15.133:0/4227670180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614c21c4f20 msgr2=0x5614c21b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:31:07.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.915+0000 7fe3a37fe700 1 --2- 172.21.15.133:0/4227670180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614c21c4f20 0x5614c21b5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:07.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.915+0000 7fe3a37fe700 1 -- 172.21.15.133:0/4227670180 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a4104740 con 0x5614c21ba8b0 2023-07-14T18:31:07.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.915+0000 7fe3a2ffd700 1 --2- 172.21.15.133:0/4227670180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614c21c4f20 0x5614c21b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T18:31:07.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.930+0000 7fe3a2ffd700 1 --2- 172.21.15.133:0/2651747001 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe3900a1260 0x7fe3900a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:31:07.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.930+0000 7fe3a37fe700 1 --2- 172.21.15.133:0/4227670180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5614c21ba8b0 0x5614c21bcd30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00b3f0 tx=0x7fe39c004c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9cc8eb0a2294210 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:31:07.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.931+0000 7fe38a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:31:07.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.931+0000 7fe38a7fc700 1 -- 172.21.15.133:0/4227670180 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c01b670 con 0x5614c21ba8b0 2023-07-14T18:31:07.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.931+0000 7fe38a7fc700 1 -- 172.21.15.133:0/4227670180 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe39c01b7d0 con 0x5614c21ba8b0 2023-07-14T18:31:07.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.931+0000 7fe3bcf03b80 1 -- 172.21.15.133:0/4227670180 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614c205e4f0 con 0x5614c21ba8b0 2023-07-14T18:31:07.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.936+0000 7fe38a7fc700 1 -- 172.21.15.133:0/4227670180 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe39c01b990 con 0x5614c21ba8b0 2023-07-14T18:31:07.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.941+0000 7fe3bcf03b80 10 client.26020 fetch_fsmap learned FSMap version 46 2023-07-14T18:31:07.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.941+0000 7fe3bcf03b80 1 -- 172.21.15.133:0/4227670180 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5614c1f1bc60 con 0x5614c21ba8b0 2023-07-14T18:31:07.977 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.949+0000 7fe38a7fc700 1 -- 172.21.15.133:0/4227670180 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5614c205e4f0 con 0x5614c21ba8b0 2023-07-14T18:31:07.977 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.949+0000 7fe38a7fc700 1 -- 172.21.15.133:0/4227670180 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe39c019040 con 0x5614c21ba8b0 2023-07-14T18:31:07.977 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.949+0000 7fe3bcf03b80 10 client.26020 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T18:31:07.977 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.949+0000 7fe3bcf03b80 10 client.26020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:31:07.977 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.949+0000 7fe3bcf03b80 20 client.26020 populate_metadata read hostname 'smithi133' 2023-07-14T18:31:07.978 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.949+0000 7fe3bcf03b80 1 --2- 172.21.15.133:0/4227670180 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5614c21c1100 0x5614c21c3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:07.978 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.949+0000 7fe3bcf03b80 4 client.26020 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:31:07.978 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.949+0000 7fe3bcf03b80 1 -- 172.21.15.133:0/4227670180 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614c2182da0 con 0x5614c21c1100 2023-07-14T18:31:07.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.966+0000 7fe3a2ffd700 1 --2- 172.21.15.133:0/4227670180 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5614c21c1100 0x5614c21c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:31:08.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.978+0000 7fe3a2ffd700 1 --2- 172.21.15.133:0/2651747001 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe3900a1260 0x7fe3900a3710 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe39402ba40 tx=0x7fe394004750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:31:08.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.978+0000 7fe3a2ffd700 1 --2- 172.21.15.133:0/4227670180 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5614c21c1100 0x5614c21c3510 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfbde7055a200e15 server_cookie=ab0251209e31d17f in_seq=0 out_seq=0 2023-07-14T18:31:08.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.981+0000 7fe38a7fc700 10 client.26020 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:31:08.011 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.984+0000 7fe38a7fc700 1 -- 172.21.15.133:0/4227670180 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5614c2182da0 con 0x5614c21c1100 2023-07-14T18:31:08.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.984+0000 7fe38a7fc700 10 client.26020 handle_command_reply: tid=0 2023-07-14T18:31:08.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.984+0000 7fe3bcf03b80 1 client.26020 shutdown 2023-07-14T18:31:08.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.995+0000 7fe3bcf03b80 1 -- 172.21.15.133:0/4227670180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5614c21ba8b0 msgr2=0x5614c21bcd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:31:08.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.995+0000 7fe3bcf03b80 1 --2- 172.21.15.133:0/4227670180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5614c21ba8b0 0x5614c21bcd30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00b3f0 tx=0x7fe39c004c40 comp rx=0 tx=0).stop 2023-07-14T18:31:08.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.996+0000 7fe3bcf03b80 1 -- 172.21.15.133:0/4227670180 shutdown_connections 2023-07-14T18:31:08.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.996+0000 7fe3bcf03b80 1 --2- 172.21.15.133:0/4227670180 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5614c21c1100 0x5614c21c3510 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:08.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.996+0000 7fe3bcf03b80 1 --2- 172.21.15.133:0/4227670180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5614c21ba8b0 0x5614c21bcd30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:08.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.996+0000 7fe3bcf03b80 1 --2- 172.21.15.133:0/4227670180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614c21c4f20 0x5614c21b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:08.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.996+0000 7fe3bcf03b80 1 --2- 172.21.15.133:0/4227670180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614c21c5870 0x5614c21b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:08.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.996+0000 7fe3bcf03b80 1 -- 172.21.15.133:0/4227670180 >> 172.21.15.133:0/4227670180 conn(0x5614c21a9680 msgr2=0x5614c21ad650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:31:08.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:07.996+0000 7fe38a7fc700 0 client.26020 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:31:08.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.003+0000 7fe3bcf03b80 1 -- 172.21.15.133:0/4227670180 shutdown_connections 2023-07-14T18:31:08.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.009+0000 7fe3bcf03b80 1 -- 172.21.15.133:0/4227670180 wait complete. 2023-07-14T18:31:08.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.014+0000 7fe3bcf03b80 20 client.26020 trim_cache size 0 max 16384 2023-07-14T18:31:08.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.050+0000 7fe3bcf03b80 1 Processor -- start 2023-07-14T18:31:08.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.051+0000 7fe3bcf03b80 1 -- start start 2023-07-14T18:31:08.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.052+0000 7fe3bcf03b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614c21ba8b0 0x5614c21c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:08.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.052+0000 7fe3bcf03b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614c21c4f20 0x5614c21d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:08.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.052+0000 7fe3bcf03b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5614c21c5870 0x5614c21ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:08.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.052+0000 7fe3bcf03b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c21bd130 con 0x5614c21c4f20 2023-07-14T18:31:08.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.052+0000 7fe3bcf03b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c21d18c0 con 0x5614c21ba8b0 2023-07-14T18:31:08.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.052+0000 7fe3bcf03b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c21afcc0 con 0x5614c21c5870 2023-07-14T18:31:08.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.053+0000 7fe3a2ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614c21ba8b0 0x5614c21c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:31:08.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.053+0000 7fe3a2ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614c21ba8b0 0x5614c21c5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:52416/0 (socket says 172.21.15.133:52416) 2023-07-14T18:31:08.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.053+0000 7fe3a2ffd700 1 -- 172.21.15.133:0/3214567473 learned_addr learned my addr 172.21.15.133:0/3214567473 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:31:08.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.054+0000 7fe3a2ffd700 1 -- 172.21.15.133:0/3214567473 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5614c21c5870 msgr2=0x5614c21ae0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:31:08.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.054+0000 7fe3a2ffd700 1 --2- 172.21.15.133:0/3214567473 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5614c21c5870 0x5614c21ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:08.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.054+0000 7fe3a2ffd700 1 -- 172.21.15.133:0/3214567473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614c21c4f20 msgr2=0x5614c21d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:31:08.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.054+0000 7fe3a2ffd700 1 --2- 172.21.15.133:0/3214567473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614c21c4f20 0x5614c21d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:08.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.054+0000 7fe3a2ffd700 1 -- 172.21.15.133:0/3214567473 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614c1f59250 con 0x5614c21ba8b0 2023-07-14T18:31:08.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.054+0000 7fe3a2ffd700 1 --2- 172.21.15.133:0/3214567473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614c21ba8b0 0x5614c21c5e70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe394045ab0 tx=0x7fe39403c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4457f71044dbe753 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:31:08.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.055+0000 7fe399ffb700 1 -- 172.21.15.133:0/3214567473 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe394031b70 con 0x5614c21ba8b0 2023-07-14T18:31:08.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.055+0000 7fe399ffb700 1 -- 172.21.15.133:0/3214567473 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe394031cd0 con 0x5614c21ba8b0 2023-07-14T18:31:08.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.055+0000 7fe399ffb700 1 -- 172.21.15.133:0/3214567473 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe394037cc0 con 0x5614c21ba8b0 2023-07-14T18:31:08.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.056+0000 7fe3bcf03b80 1 -- 172.21.15.133:0/3214567473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614c21ba8b0 msgr2=0x5614c21c5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:31:08.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.056+0000 7fe3bcf03b80 1 --2- 172.21.15.133:0/3214567473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614c21ba8b0 0x5614c21c5e70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe394045ab0 tx=0x7fe39403c000 comp rx=0 tx=0).stop 2023-07-14T18:31:08.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.056+0000 7fe3bcf03b80 1 -- 172.21.15.133:0/3214567473 shutdown_connections 2023-07-14T18:31:08.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.056+0000 7fe3bcf03b80 1 --2- 172.21.15.133:0/3214567473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614c21ba8b0 0x5614c21c5e70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:08.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.056+0000 7fe3bcf03b80 1 --2- 172.21.15.133:0/3214567473 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5614c21c5870 0x5614c21ae0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:08.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.056+0000 7fe3bcf03b80 1 --2- 172.21.15.133:0/3214567473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614c21c4f20 0x5614c21d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:08.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.056+0000 7fe3bcf03b80 1 -- 172.21.15.133:0/3214567473 >> 172.21.15.133:0/3214567473 conn(0x5614c21a9680 msgr2=0x5614c21b3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:31:08.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.057+0000 7fe3bcf03b80 1 -- 172.21.15.133:0/3214567473 shutdown_connections 2023-07-14T18:31:08.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.057+0000 7fe3bcf03b80 1 -- 172.21.15.133:0/3214567473 wait complete. 2023-07-14T18:31:08.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.057+0000 7fe3bcf03b80 1 -- start start 2023-07-14T18:31:08.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.058+0000 7fe3bcf03b80 1 Processor -- start 2023-07-14T18:31:08.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.058+0000 7fe3bcf03b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614c21ba8b0 0x5614c21cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:08.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.058+0000 7fe3bcf03b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614c21c4f20 0x5614c21cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:08.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.058+0000 7fe3bcf03b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5614c21c5870 0x5614c21cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:08.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.058+0000 7fe3bcf03b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c21d0330 con 0x5614c21ba8b0 2023-07-14T18:31:08.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.058+0000 7fe3bcf03b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c21d04a0 con 0x5614c21c4f20 2023-07-14T18:31:08.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.058+0000 7fe3bcf03b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c21d0660 con 0x5614c21c5870 2023-07-14T18:31:08.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.058+0000 7fe3bcf03b80 10 client.0 _ll_register_callbacks cb 0x7fe39402b0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:31:08.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.059+0000 7fe39bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614c21c4f20 0x5614c21cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:31:08.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.059+0000 7fe39bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614c21c4f20 0x5614c21cefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:52428/0 (socket says 172.21.15.133:52428) 2023-07-14T18:31:08.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.059+0000 7fe39bfff700 1 -- 172.21.15.133:0/1235585352 learned_addr learned my addr 172.21.15.133:0/1235585352 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:31:08.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.059+0000 7fe3a37fe700 1 --2- 172.21.15.133:0/1235585352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5614c21c5870 0x5614c21cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:31:08.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.059+0000 7fe3a2ffd700 1 --2- 172.21.15.133:0/1235585352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614c21ba8b0 0x5614c21cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:31:08.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.059+0000 7fe3a37fe700 1 -- 172.21.15.133:0/1235585352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614c21c4f20 msgr2=0x5614c21cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:31:08.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.059+0000 7fe3a37fe700 1 --2- 172.21.15.133:0/1235585352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614c21c4f20 0x5614c21cefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:08.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.059+0000 7fe3a37fe700 1 -- 172.21.15.133:0/1235585352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614c21ba8b0 msgr2=0x5614c21cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:31:08.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.059+0000 7fe3a37fe700 1 --2- 172.21.15.133:0/1235585352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614c21ba8b0 0x5614c21cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:08.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.060+0000 7fe3a37fe700 1 -- 172.21.15.133:0/1235585352 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe39402f040 con 0x5614c21c5870 2023-07-14T18:31:08.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.060+0000 7fe39bfff700 1 --2- 172.21.15.133:0/1235585352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614c21c4f20 0x5614c21cefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:31:08.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.060+0000 7fe3a37fe700 1 --2- 172.21.15.133:0/1235585352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5614c21c5870 0x5614c21cfdf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe39c024d90 tx=0x7fe39c0247b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c94ae33d01f14599 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:31:08.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.060+0000 7fe38a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:31:08.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.060+0000 7fe38a7fc700 1 -- 172.21.15.133:0/1235585352 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c0096a0 con 0x5614c21c5870 2023-07-14T18:31:08.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.060+0000 7fe3bcf03b80 1 -- 172.21.15.133:0/1235585352 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe39c0093b0 con 0x5614c21c5870 2023-07-14T18:31:08.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.060+0000 7fe38a7fc700 1 -- 172.21.15.133:0/1235585352 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe39c02e3f0 con 0x5614c21c5870 2023-07-14T18:31:08.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.061+0000 7fe38a7fc700 1 -- 172.21.15.133:0/1235585352 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe39c024920 con 0x5614c21c5870 2023-07-14T18:31:08.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.061+0000 7fe3bcf03b80 10 client.26023 fetch_fsmap learned FSMap version 46 2023-07-14T18:31:08.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.061+0000 7fe3bcf03b80 1 -- 172.21.15.133:0/1235585352 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5614c1f312b0 con 0x5614c21c5870 2023-07-14T18:31:08.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.061+0000 7fe38a7fc700 1 -- 172.21.15.133:0/1235585352 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe39c0093b0 con 0x5614c21c5870 2023-07-14T18:31:08.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.061+0000 7fe38a7fc700 1 -- 172.21.15.133:0/1235585352 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe39c019070 con 0x5614c21c5870 2023-07-14T18:31:08.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.062+0000 7fe3bcf03b80 10 client.26023 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T18:31:08.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.062+0000 7fe3bcf03b80 10 client.26023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:31:08.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.062+0000 7fe3bcf03b80 20 client.26023 populate_metadata read hostname 'smithi133' 2023-07-14T18:31:08.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.062+0000 7fe3bcf03b80 1 --2- 172.21.15.133:0/1235585352 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5614c221e5f0 0x5614c22209d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:08.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.062+0000 7fe3bcf03b80 4 client.26023 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:31:08.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.062+0000 7fe3bcf03b80 1 -- 172.21.15.133:0/1235585352 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5614c20fdeb0 con 0x5614c221e5f0 2023-07-14T18:31:08.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.062+0000 7fe3a2ffd700 1 --2- 172.21.15.133:0/1235585352 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5614c221e5f0 0x5614c22209d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:31:08.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.064+0000 7fe3a2ffd700 1 --2- 172.21.15.133:0/1235585352 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5614c221e5f0 0x5614c22209d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13baf31cde727346 server_cookie=9446732e69507a94 in_seq=0 out_seq=0 2023-07-14T18:31:08.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.064+0000 7fe38a7fc700 10 client.26023 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:31:08.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.064+0000 7fe38a7fc700 1 -- 172.21.15.133:0/1235585352 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5614c20fdeb0 con 0x5614c221e5f0 2023-07-14T18:31:08.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.064+0000 7fe38a7fc700 10 client.26023 handle_command_reply: tid=0 2023-07-14T18:31:08.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.064+0000 7fe3bcf03b80 1 client.26023 shutdown 2023-07-14T18:31:08.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.065+0000 7fe3bcf03b80 1 -- 172.21.15.133:0/1235585352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5614c21c5870 msgr2=0x5614c21cfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:31:08.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.065+0000 7fe3bcf03b80 1 --2- 172.21.15.133:0/1235585352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5614c21c5870 0x5614c21cfdf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe39c024d90 tx=0x7fe39c0247b0 comp rx=0 tx=0).stop 2023-07-14T18:31:08.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.065+0000 7fe3bcf03b80 1 -- 172.21.15.133:0/1235585352 shutdown_connections 2023-07-14T18:31:08.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.065+0000 7fe3bcf03b80 1 --2- 172.21.15.133:0/1235585352 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5614c221e5f0 0x5614c22209d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:08.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.065+0000 7fe3bcf03b80 1 --2- 172.21.15.133:0/1235585352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5614c21c5870 0x5614c21cfdf0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:08.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.065+0000 7fe3bcf03b80 1 --2- 172.21.15.133:0/1235585352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614c21ba8b0 0x5614c21cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:08.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.065+0000 7fe3bcf03b80 1 --2- 172.21.15.133:0/1235585352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5614c21c4f20 0x5614c21cefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:08.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.065+0000 7fe3bcf03b80 1 -- 172.21.15.133:0/1235585352 >> 172.21.15.133:0/1235585352 conn(0x5614c21a9680 msgr2=0x5614c21ae0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:31:08.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.065+0000 7fe38a7fc700 0 client.26023 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:31:08.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.065+0000 7fe3bcf03b80 1 -- 172.21.15.133:0/1235585352 shutdown_connections 2023-07-14T18:31:08.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.065+0000 7fe3bcf03b80 1 -- 172.21.15.133:0/1235585352 wait complete. 2023-07-14T18:31:08.097 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:31:08.098 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T18:31:08.098 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:31:08.098 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:31:08.098 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:31:08.098 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:31:08.098 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:31:08.098 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:31:08.099 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:31:08.099 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:31:08.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.068+0000 7fe3bcf03b80 20 client.26023 trim_cache size 0 max 16384 2023-07-14T18:31:08.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.072+0000 7fe3a95ff700 1 -- 172.21.15.133:0/2651747001 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe3900a1260 msgr2=0x7fe3900a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:31:08.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.072+0000 7fe3a95ff700 1 --2- 172.21.15.133:0/2651747001 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe3900a1260 0x7fe3900a3710 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe39402ba40 tx=0x7fe394004750 comp rx=0 tx=0).stop 2023-07-14T18:31:08.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.072+0000 7fe3a95ff700 1 -- 172.21.15.133:0/2651747001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3a4100de0 msgr2=0x7fe3a4109e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:31:08.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.072+0000 7fe3a95ff700 1 --2- 172.21.15.133:0/2651747001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3a4100de0 0x7fe3a4109e30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00b550 tx=0x7fe38c00b910 comp rx=0 tx=0).stop 2023-07-14T18:31:08.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.072+0000 7fe3a95ff700 1 -- 172.21.15.133:0/2651747001 shutdown_connections 2023-07-14T18:31:08.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.072+0000 7fe3a95ff700 1 --2- 172.21.15.133:0/2651747001 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fe3900a1260 0x7fe3900a3710 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:08.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.072+0000 7fe3a95ff700 1 --2- 172.21.15.133:0/2651747001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3a41037a0 0x7fe3a41ac290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:08.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.072+0000 7fe3a95ff700 1 --2- 172.21.15.133:0/2651747001 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3a40fe480 0x7fe3a41098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:08.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.072+0000 7fe3a95ff700 1 --2- 172.21.15.133:0/2651747001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3a4100de0 0x7fe3a4109e30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:08.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.072+0000 7fe3a95ff700 1 -- 172.21.15.133:0/2651747001 >> 172.21.15.133:0/2651747001 conn(0x7fe3a40fa130 msgr2=0x7fe3a40ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:31:08.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.073+0000 7fe3a95ff700 1 -- 172.21.15.133:0/2651747001 shutdown_connections 2023-07-14T18:31:08.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:08.073+0000 7fe3a95ff700 1 -- 172.21.15.133:0/2651747001 wait complete. 2023-07-14T18:31:08.133 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:31:10.083 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:09 smithi161 ceph-mon[128993]: pgmap v782: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 13 MiB/s wr, 1.18k op/s 2023-07-14T18:31:10.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:09 smithi171 ceph-mon[129241]: pgmap v782: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 13 MiB/s wr, 1.18k op/s 2023-07-14T18:31:10.161 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:09 smithi133 ceph-mon[126384]: pgmap v782: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 13 MiB/s wr, 1.18k op/s 2023-07-14T18:31:10.858 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 705.0 s, 513.3 tps, lat 207.564 ms stddev 450.290 2023-07-14T18:31:12.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:11 smithi161 ceph-mon[128993]: pgmap v783: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 12 MiB/s wr, 1.02k op/s 2023-07-14T18:31:12.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:11 smithi171 ceph-mon[129241]: pgmap v783: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 12 MiB/s wr, 1.02k op/s 2023-07-14T18:31:12.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:11 smithi133 ceph-mon[126384]: pgmap v783: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 12 MiB/s wr, 1.02k op/s 2023-07-14T18:31:14.083 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:13 smithi161 ceph-mon[128993]: pgmap v784: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 988 op/s 2023-07-14T18:31:14.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:13 smithi171 ceph-mon[129241]: pgmap v784: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 988 op/s 2023-07-14T18:31:14.140 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:13 smithi133 ceph-mon[126384]: pgmap v784: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 988 op/s 2023-07-14T18:31:15.859 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 710.0 s, 716.2 tps, lat 138.956 ms stddev 327.466 2023-07-14T18:31:16.140 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:15 smithi133 ceph-mon[126384]: pgmap v785: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 12 MiB/s wr, 814 op/s 2023-07-14T18:31:16.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:15 smithi161 ceph-mon[128993]: pgmap v785: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 12 MiB/s wr, 814 op/s 2023-07-14T18:31:16.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:15 smithi171 ceph-mon[129241]: pgmap v785: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 12 MiB/s wr, 814 op/s 2023-07-14T18:31:17.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:17 smithi161 ceph-mon[128993]: pgmap v786: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 13 MiB/s wr, 905 op/s 2023-07-14T18:31:17.604 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:17 smithi171 ceph-mon[129241]: pgmap v786: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 13 MiB/s wr, 905 op/s 2023-07-14T18:31:17.647 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:17 smithi133 ceph-mon[126384]: pgmap v786: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 13 MiB/s wr, 905 op/s 2023-07-14T18:31:20.083 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:19 smithi161 ceph-mon[128993]: pgmap v787: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 14 MiB/s wr, 1.02k op/s 2023-07-14T18:31:20.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:19 smithi171 ceph-mon[129241]: pgmap v787: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 14 MiB/s wr, 1.02k op/s 2023-07-14T18:31:20.139 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:19 smithi133 ceph-mon[126384]: pgmap v787: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 14 MiB/s wr, 1.02k op/s 2023-07-14T18:31:20.858 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 715.0 s, 639.9 tps, lat 154.261 ms stddev 356.656 2023-07-14T18:31:22.084 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:21 smithi161 ceph-mon[128993]: pgmap v788: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 15 MiB/s wr, 1.01k op/s 2023-07-14T18:31:22.103 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:21 smithi171 ceph-mon[129241]: pgmap v788: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 15 MiB/s wr, 1.01k op/s 2023-07-14T18:31:22.142 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:21 smithi133 ceph-mon[126384]: pgmap v788: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 15 MiB/s wr, 1.01k op/s 2023-07-14T18:31:24.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:23 smithi171 ceph-mon[129241]: pgmap v789: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 15 MiB/s wr, 1.06k op/s 2023-07-14T18:31:24.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:23 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:31:24.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:23 smithi161 ceph-mon[128993]: pgmap v789: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 15 MiB/s wr, 1.06k op/s 2023-07-14T18:31:24.100 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:23 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:31:24.145 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:23 smithi133 ceph-mon[126384]: pgmap v789: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 15 MiB/s wr, 1.06k op/s 2023-07-14T18:31:24.145 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:23 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:31:25.857 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 720.0 s, 601.6 tps, lat 169.265 ms stddev 374.702 2023-07-14T18:31:26.083 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:25 smithi161 ceph-mon[128993]: pgmap v790: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 15 MiB/s wr, 1.07k op/s 2023-07-14T18:31:26.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:25 smithi171 ceph-mon[129241]: pgmap v790: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 15 MiB/s wr, 1.07k op/s 2023-07-14T18:31:26.139 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:25 smithi133 ceph-mon[126384]: pgmap v790: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 15 MiB/s wr, 1.07k op/s 2023-07-14T18:31:27.142 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:26 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:31:27.142 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:26 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:31:27.142 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:26 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:31:27.333 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:26 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:31:27.333 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:26 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:31:27.333 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:26 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:31:27.349 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:26 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:31:27.349 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:26 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:31:27.349 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:26 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:31:28.333 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:27 smithi161 ceph-mon[128993]: pgmap v791: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 16 MiB/s wr, 1.18k op/s 2023-07-14T18:31:28.348 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:27 smithi171 ceph-mon[129241]: pgmap v791: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 16 MiB/s wr, 1.18k op/s 2023-07-14T18:31:28.390 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:27 smithi133 ceph-mon[126384]: pgmap v791: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 16 MiB/s wr, 1.18k op/s 2023-07-14T18:31:30.333 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:29 smithi161 ceph-mon[128993]: pgmap v792: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 16 MiB/s wr, 1.22k op/s 2023-07-14T18:31:30.352 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:29 smithi171 ceph-mon[129241]: pgmap v792: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 16 MiB/s wr, 1.22k op/s 2023-07-14T18:31:30.389 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:29 smithi133 ceph-mon[126384]: pgmap v792: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 16 MiB/s wr, 1.22k op/s 2023-07-14T18:31:30.856 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 725.0 s, 830.9 tps, lat 120.177 ms stddev 119.328 2023-07-14T18:31:32.334 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:31 smithi161 ceph-mon[128993]: pgmap v793: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 17 MiB/s wr, 1.24k op/s 2023-07-14T18:31:32.347 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:31 smithi171 ceph-mon[129241]: pgmap v793: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 17 MiB/s wr, 1.24k op/s 2023-07-14T18:31:32.389 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:31 smithi133 ceph-mon[126384]: pgmap v793: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 17 MiB/s wr, 1.24k op/s 2023-07-14T18:31:34.333 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:33 smithi161 ceph-mon[128993]: pgmap v794: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 15 MiB/s wr, 1.08k op/s 2023-07-14T18:31:34.347 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:33 smithi171 ceph-mon[129241]: pgmap v794: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 15 MiB/s wr, 1.08k op/s 2023-07-14T18:31:34.391 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:33 smithi133 ceph-mon[126384]: pgmap v794: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 15 MiB/s wr, 1.08k op/s 2023-07-14T18:31:35.856 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 730.0 s, 1331.4 tps, lat 74.332 ms stddev 59.204 2023-07-14T18:31:36.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:36 smithi161 ceph-mon[128993]: pgmap v795: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 14 MiB/s wr, 960 op/s 2023-07-14T18:31:36.350 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:36 smithi171 ceph-mon[129241]: pgmap v795: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 14 MiB/s wr, 960 op/s 2023-07-14T18:31:36.394 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:35 smithi133 ceph-mon[126384]: pgmap v795: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 14 MiB/s wr, 960 op/s 2023-07-14T18:31:37.590 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:37 smithi161 ceph-mon[128993]: pgmap v796: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 989 op/s 2023-07-14T18:31:37.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:37 smithi171 ceph-mon[129241]: pgmap v796: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 989 op/s 2023-07-14T18:31:37.639 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:37 smithi133 ceph-mon[126384]: pgmap v796: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 989 op/s 2023-07-14T18:31:38.134 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:31:38.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.607+0000 7f19437ff700 1 -- 172.21.15.133:0/447602660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f193c071960 msgr2=0x7f193c10b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:31:38.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.607+0000 7f19437ff700 1 --2- 172.21.15.133:0/447602660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f193c071960 0x7f193c10b900 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f193400b0a0 tx=0x7f193400b3b0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.610+0000 7f19437ff700 1 -- 172.21.15.133:0/447602660 shutdown_connections 2023-07-14T18:31:38.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.610+0000 7f19437ff700 1 --2- 172.21.15.133:0/447602660 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f193c10be90 0x7f193c10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.610+0000 7f19437ff700 1 --2- 172.21.15.133:0/447602660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f193c071010 0x7f193c071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.610+0000 7f19437ff700 1 --2- 172.21.15.133:0/447602660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f193c071960 0x7f193c10b900 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.610+0000 7f19437ff700 1 -- 172.21.15.133:0/447602660 >> 172.21.15.133:0/447602660 conn(0x7f193c06cbc0 msgr2=0x7f193c06eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:31:38.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.610+0000 7f19437ff700 1 -- 172.21.15.133:0/447602660 shutdown_connections 2023-07-14T18:31:38.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.610+0000 7f19437ff700 1 -- 172.21.15.133:0/447602660 wait complete. 2023-07-14T18:31:38.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.612+0000 7f19437ff700 1 Processor -- start 2023-07-14T18:31:38.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.612+0000 7f19437ff700 1 -- start start 2023-07-14T18:31:38.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.612+0000 7f19437ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f193c071010 0x7f193c114d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:38.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.612+0000 7f19437ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f193c071960 0x7f193c115270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:38.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.612+0000 7f19437ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f193c10be90 0x7f193c1162b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:38.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.612+0000 7f19437ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c116880 con 0x7f193c071960 2023-07-14T18:31:38.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.612+0000 7f19437ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c1169f0 con 0x7f193c10be90 2023-07-14T18:31:38.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.612+0000 7f19437ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c116bb0 con 0x7f193c071010 2023-07-14T18:31:38.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.613+0000 7f1941d9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f193c10be90 0x7f193c1162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:31:38.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.613+0000 7f1941d9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f193c10be90 0x7f193c1162b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:45582/0 (socket says 172.21.15.133:45582) 2023-07-14T18:31:38.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.613+0000 7f1941d9c700 1 -- 172.21.15.133:0/3416693683 learned_addr learned my addr 172.21.15.133:0/3416693683 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:31:38.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.614+0000 7f1941d9c700 1 -- 172.21.15.133:0/3416693683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f193c071010 msgr2=0x7f193c114d30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-14T18:31:38.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.614+0000 7f1941d9c700 1 --2- 172.21.15.133:0/3416693683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f193c071010 0x7f193c114d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.614+0000 7f1941d9c700 1 -- 172.21.15.133:0/3416693683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f193c071960 msgr2=0x7f193c115270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:31:38.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.614+0000 7f1941d9c700 1 --2- 172.21.15.133:0/3416693683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f193c071960 0x7f193c115270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.614+0000 7f1941d9c700 1 -- 172.21.15.133:0/3416693683 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1934009d00 con 0x7f193c10be90 2023-07-14T18:31:38.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.614+0000 7f1941d9c700 1 --2- 172.21.15.133:0/3416693683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f193c10be90 0x7f193c1162b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f193800e940 tx=0x7f193800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:31:38.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.616+0000 7f19327fc700 1 -- 172.21.15.133:0/3416693683 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193800cbd0 con 0x7f193c10be90 2023-07-14T18:31:38.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.616+0000 7f19327fc700 1 -- 172.21.15.133:0/3416693683 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f193800cd30 con 0x7f193c10be90 2023-07-14T18:31:38.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.616+0000 7f19327fc700 1 -- 172.21.15.133:0/3416693683 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1938005650 con 0x7f193c10be90 2023-07-14T18:31:38.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.617+0000 7f19437ff700 1 -- 172.21.15.133:0/3416693683 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f193c116e30 con 0x7f193c10be90 2023-07-14T18:31:38.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.617+0000 7f19437ff700 1 -- 172.21.15.133:0/3416693683 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f193c117370 con 0x7f193c10be90 2023-07-14T18:31:38.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.625+0000 7f195708ab80 1 Processor -- start 2023-07-14T18:31:38.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.626+0000 7f195708ab80 1 -- start start 2023-07-14T18:31:38.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.626+0000 7f195708ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7802bcf60 0x55c7802bd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:38.653 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.626+0000 7f195708ab80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7802bd8b0 0x55c7802bded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:38.653 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.626+0000 7f195708ab80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7802be4b0 0x55c7802be920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:38.653 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.626+0000 7f195708ab80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7801c1cb0 con 0x55c7802bd8b0 2023-07-14T18:31:38.653 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.626+0000 7f195708ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c780214a40 con 0x55c7802bcf60 2023-07-14T18:31:38.653 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.626+0000 7f195708ab80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c78020ded0 con 0x55c7802be4b0 2023-07-14T18:31:38.655 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.628+0000 7f1940d9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7802bcf60 0x55c7802bd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:31:38.655 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.628+0000 7f1940d9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7802bcf60 0x55c7802bd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:45592/0 (socket says 172.21.15.133:45592) 2023-07-14T18:31:38.655 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.628+0000 7f1940d9a700 1 -- 172.21.15.133:0/2087112627 learned_addr learned my addr 172.21.15.133:0/2087112627 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:31:38.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.629+0000 7f1940d9a700 1 --2- 172.21.15.133:0/2087112627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7802be4b0 0x55c7802be920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:31:38.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.629+0000 7f1940d9a700 1 -- 172.21.15.133:0/2087112627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7802be4b0 msgr2=0x55c7802be920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:31:38.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.629+0000 7f1940d9a700 1 --2- 172.21.15.133:0/2087112627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7802be4b0 0x55c7802be920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.629+0000 7f1940d9a700 1 -- 172.21.15.133:0/2087112627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7802bd8b0 msgr2=0x55c7802bded0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T18:31:38.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.629+0000 7f1940d9a700 1 --2- 172.21.15.133:0/2087112627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7802bd8b0 0x55c7802bded0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.629+0000 7f1940d9a700 1 -- 172.21.15.133:0/2087112627 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c78003bde0 con 0x55c7802bcf60 2023-07-14T18:31:38.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.630+0000 7f1940d9a700 1 --2- 172.21.15.133:0/2087112627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7802bcf60 0x55c7802bd370 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1934009610 tx=0x7f1934006210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55176c1395149604 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:31:38.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.631+0000 7f19277fe700 1 -- 172.21.15.133:0/2087112627 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193400e070 con 0x55c7802bcf60 2023-07-14T18:31:38.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.631+0000 7f19277fe700 1 -- 172.21.15.133:0/2087112627 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f193400bbc0 con 0x55c7802bcf60 2023-07-14T18:31:38.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.631+0000 7f19277fe700 1 -- 172.21.15.133:0/2087112627 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193401c470 con 0x55c7802bcf60 2023-07-14T18:31:38.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.632+0000 7f195708ab80 1 -- 172.21.15.133:0/2087112627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7802bcf60 msgr2=0x55c7802bd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:31:38.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.632+0000 7f195708ab80 1 --2- 172.21.15.133:0/2087112627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7802bcf60 0x55c7802bd370 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1934009610 tx=0x7f1934006210 comp rx=0 tx=0).stop 2023-07-14T18:31:38.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.637+0000 7f195708ab80 1 -- 172.21.15.133:0/2087112627 shutdown_connections 2023-07-14T18:31:38.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.637+0000 7f195708ab80 1 --2- 172.21.15.133:0/2087112627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7802bcf60 0x55c7802bd370 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.637+0000 7f195708ab80 1 --2- 172.21.15.133:0/2087112627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7802be4b0 0x55c7802be920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.637+0000 7f195708ab80 1 --2- 172.21.15.133:0/2087112627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7802bd8b0 0x55c7802bded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.637+0000 7f195708ab80 1 -- 172.21.15.133:0/2087112627 >> 172.21.15.133:0/2087112627 conn(0x55c7802b2680 msgr2=0x55c7802b79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:31:38.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.637+0000 7f19327fc700 1 -- 172.21.15.133:0/3416693683 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f1938004750 con 0x7f193c10be90 2023-07-14T18:31:38.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.639+0000 7f19327fc700 1 --2- 172.21.15.133:0/3416693683 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f19280a1350 0x7f19280a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:38.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.640+0000 7f19327fc700 1 -- 172.21.15.133:0/3416693683 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1938014070 con 0x7f193c10be90 2023-07-14T18:31:38.673 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.645+0000 7f195708ab80 1 -- 172.21.15.133:0/2087112627 shutdown_connections 2023-07-14T18:31:38.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.645+0000 7f195708ab80 1 -- 172.21.15.133:0/2087112627 wait complete. 2023-07-14T18:31:38.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.646+0000 7f195708ab80 1 -- start start 2023-07-14T18:31:38.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.646+0000 7f195708ab80 1 Processor -- start 2023-07-14T18:31:38.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.647+0000 7f194159b700 1 --2- 172.21.15.133:0/3416693683 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f19280a1350 0x7f19280a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:31:38.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.648+0000 7f194159b700 1 --2- 172.21.15.133:0/3416693683 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f19280a1350 0x7f19280a3800 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x55c7802bdcc0 tx=0x7f192c006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:31:38.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.649+0000 7f195708ab80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7802bcf60 0x55c7802c6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:38.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.649+0000 7f195708ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7802bd8b0 0x55c7802c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:38.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.649+0000 7f195708ab80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7802be4b0 0x55c7802cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:38.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.649+0000 7f195708ab80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7801adac0 con 0x55c7802bcf60 2023-07-14T18:31:38.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.649+0000 7f195708ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7801ad7c0 con 0x55c7802bd8b0 2023-07-14T18:31:38.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.649+0000 7f195708ab80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c780091920 con 0x55c7802be4b0 2023-07-14T18:31:38.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.649+0000 7f195708ab80 10 client.0 _ll_register_callbacks cb 0x55c7801ab760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:31:38.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.650+0000 7f1940d9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7802bd8b0 0x55c7802c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:31:38.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.650+0000 7f1940d9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7802bd8b0 0x55c7802c6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:45600/0 (socket says 172.21.15.133:45600) 2023-07-14T18:31:38.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.650+0000 7f1940d9a700 1 -- 172.21.15.133:0/2983522035 learned_addr learned my addr 172.21.15.133:0/2983522035 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:31:38.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.650+0000 7f1941d9c700 1 --2- 172.21.15.133:0/2983522035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7802be4b0 0x55c7802cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:31:38.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.650+0000 7f194159b700 1 --2- 172.21.15.133:0/2983522035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7802bcf60 0x55c7802c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:31:38.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.650+0000 7f1941d9c700 1 -- 172.21.15.133:0/2983522035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7802bd8b0 msgr2=0x55c7802c6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:31:38.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.650+0000 7f1941d9c700 1 --2- 172.21.15.133:0/2983522035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7802bd8b0 0x55c7802c6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.650+0000 7f1941d9c700 1 -- 172.21.15.133:0/2983522035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7802bcf60 msgr2=0x55c7802c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:31:38.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.650+0000 7f1941d9c700 1 --2- 172.21.15.133:0/2983522035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7802bcf60 0x55c7802c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.650+0000 7f1941d9c700 1 -- 172.21.15.133:0/2983522035 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f193c117770 con 0x55c7802be4b0 2023-07-14T18:31:38.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.651+0000 7f1941d9c700 1 --2- 172.21.15.133:0/2983522035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7802be4b0 0x55c7802cde80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1938009fd0 tx=0x7f193800cea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de04b9a51860ce91 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:31:38.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.651+0000 7f1924ff9700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2023-07-14T18:31:38.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.653+0000 7f1924ff9700 1 -- 172.21.15.133:0/2983522035 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193809dff0 con 0x55c7802be4b0 2023-07-14T18:31:38.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.653+0000 7f1924ff9700 1 -- 172.21.15.133:0/2983522035 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f19380a2410 con 0x55c7802be4b0 2023-07-14T18:31:38.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.653+0000 7f195708ab80 1 -- 172.21.15.133:0/2983522035 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7801674f0 con 0x55c7802be4b0 2023-07-14T18:31:38.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.654+0000 7f1924ff9700 1 -- 172.21.15.133:0/2983522035 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19380b8250 con 0x55c7802be4b0 2023-07-14T18:31:38.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.654+0000 7f195708ab80 10 client.26032 fetch_fsmap learned FSMap version 46 2023-07-14T18:31:38.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.654+0000 7f195708ab80 1 -- 172.21.15.133:0/2983522035 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55c780024c60 con 0x55c7802be4b0 2023-07-14T18:31:38.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.655+0000 7f1924ff9700 1 -- 172.21.15.133:0/2983522035 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c7801674f0 con 0x55c7802be4b0 2023-07-14T18:31:38.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.655+0000 7f1924ff9700 1 -- 172.21.15.133:0/2983522035 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1938094bd0 con 0x55c7802be4b0 2023-07-14T18:31:38.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.655+0000 7f195708ab80 10 client.26032 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T18:31:38.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.655+0000 7f195708ab80 10 client.26032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:31:38.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.655+0000 7f195708ab80 20 client.26032 populate_metadata read hostname 'smithi133' 2023-07-14T18:31:38.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.655+0000 7f195708ab80 1 --2- 172.21.15.133:0/2983522035 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55c7802d2690 0x55c7802d4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:38.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.655+0000 7f195708ab80 4 client.26032 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:31:38.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.655+0000 7f195708ab80 1 -- 172.21.15.133:0/2983522035 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c78028bda0 con 0x55c7802d2690 2023-07-14T18:31:38.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.655+0000 7f194159b700 1 --2- 172.21.15.133:0/2983522035 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55c7802d2690 0x55c7802d4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:31:38.683 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.656+0000 7f1924ff9700 10 client.26032 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:31:38.683 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.656+0000 7f194159b700 1 --2- 172.21.15.133:0/2983522035 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55c7802d2690 0x55c7802d4aa0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c2f694ba8981b99 server_cookie=de4d8ba9f0e6550 in_seq=0 out_seq=0 2023-07-14T18:31:38.683 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.657+0000 7f1924ff9700 1 -- 172.21.15.133:0/2983522035 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55c78028bda0 con 0x55c7802d2690 2023-07-14T18:31:38.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.657+0000 7f1924ff9700 10 client.26032 handle_command_reply: tid=0 2023-07-14T18:31:38.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.657+0000 7f195708ab80 1 client.26032 shutdown 2023-07-14T18:31:38.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.657+0000 7f195708ab80 1 -- 172.21.15.133:0/2983522035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7802be4b0 msgr2=0x55c7802cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:31:38.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.657+0000 7f195708ab80 1 --2- 172.21.15.133:0/2983522035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7802be4b0 0x55c7802cde80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1938009fd0 tx=0x7f193800cea0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.658+0000 7f195708ab80 1 -- 172.21.15.133:0/2983522035 shutdown_connections 2023-07-14T18:31:38.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.658+0000 7f195708ab80 1 --2- 172.21.15.133:0/2983522035 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55c7802d2690 0x55c7802d4aa0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.658+0000 7f195708ab80 1 --2- 172.21.15.133:0/2983522035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7802be4b0 0x55c7802cde80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.658+0000 7f195708ab80 1 --2- 172.21.15.133:0/2983522035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7802bcf60 0x55c7802c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.658+0000 7f195708ab80 1 --2- 172.21.15.133:0/2983522035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7802bd8b0 0x55c7802c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.658+0000 7f195708ab80 1 -- 172.21.15.133:0/2983522035 >> 172.21.15.133:0/2983522035 conn(0x55c7802b2680 msgr2=0x55c7802b6650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:31:38.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.658+0000 7f1924ff9700 0 client.26032 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:31:38.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.658+0000 7f195708ab80 1 -- 172.21.15.133:0/2983522035 shutdown_connections 2023-07-14T18:31:38.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.658+0000 7f195708ab80 1 -- 172.21.15.133:0/2983522035 wait complete. 2023-07-14T18:31:38.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.658+0000 7f195708ab80 20 client.26032 trim_cache size 0 max 16384 2023-07-14T18:31:38.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.691+0000 7f195708ab80 1 Processor -- start 2023-07-14T18:31:38.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.691+0000 7f195708ab80 1 -- start start 2023-07-14T18:31:38.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.691+0000 7f195708ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7802bcf60 0x55c7802b70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:38.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.692+0000 7f195708ab80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7802bd8b0 0x55c7802cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:38.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.692+0000 7f195708ab80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7802be4b0 0x55c7802d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:38.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.692+0000 7f195708ab80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7802cfb70 con 0x55c7802bd8b0 2023-07-14T18:31:38.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.692+0000 7f195708ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7802c5dd0 con 0x55c7802bcf60 2023-07-14T18:31:38.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.692+0000 7f195708ab80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7802c6470 con 0x55c7802be4b0 2023-07-14T18:31:38.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.692+0000 7f1940d9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7802bd8b0 0x55c7802cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:31:38.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.692+0000 7f1940d9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7802bd8b0 0x55c7802cdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50884/0 (socket says 172.21.15.133:50884) 2023-07-14T18:31:38.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.692+0000 7f1940d9a700 1 -- 172.21.15.133:0/521664468 learned_addr learned my addr 172.21.15.133:0/521664468 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:31:38.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.693+0000 7f1941d9c700 1 --2- 172.21.15.133:0/521664468 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7802be4b0 0x55c7802d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:31:38.721 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.693+0000 7f1940d9a700 1 -- 172.21.15.133:0/521664468 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7802be4b0 msgr2=0x55c7802d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:31:38.721 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.693+0000 7f1940d9a700 1 --2- 172.21.15.133:0/521664468 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7802be4b0 0x55c7802d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.721 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.693+0000 7f1940d9a700 1 -- 172.21.15.133:0/521664468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7802bcf60 msgr2=0x55c7802b70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:31:38.721 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.693+0000 7f1940d9a700 1 --2- 172.21.15.133:0/521664468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7802bcf60 0x55c7802b70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.721 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.693+0000 7f1940d9a700 1 -- 172.21.15.133:0/521664468 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c780062250 con 0x55c7802bd8b0 2023-07-14T18:31:38.721 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.694+0000 7f1940d9a700 1 --2- 172.21.15.133:0/521664468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7802bd8b0 0x55c7802cdda0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f193400b070 tx=0x7f1934009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2f229c8e58c529 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:31:38.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.694+0000 7f1927fff700 1 -- 172.21.15.133:0/521664468 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193400e050 con 0x55c7802bd8b0 2023-07-14T18:31:38.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.694+0000 7f1927fff700 1 -- 172.21.15.133:0/521664468 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1934003d30 con 0x55c7802bd8b0 2023-07-14T18:31:38.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.694+0000 7f1927fff700 1 -- 172.21.15.133:0/521664468 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1934005900 con 0x55c7802bd8b0 2023-07-14T18:31:38.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.695+0000 7f195708ab80 1 -- 172.21.15.133:0/521664468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7802bd8b0 msgr2=0x55c7802cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:31:38.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.695+0000 7f195708ab80 1 --2- 172.21.15.133:0/521664468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7802bd8b0 0x55c7802cdda0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f193400b070 tx=0x7f1934009040 comp rx=0 tx=0).stop 2023-07-14T18:31:38.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.712+0000 7f195708ab80 1 -- 172.21.15.133:0/521664468 shutdown_connections 2023-07-14T18:31:38.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.712+0000 7f195708ab80 1 --2- 172.21.15.133:0/521664468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7802bcf60 0x55c7802b70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.712+0000 7f195708ab80 1 --2- 172.21.15.133:0/521664468 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7802be4b0 0x55c7802d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.712+0000 7f195708ab80 1 --2- 172.21.15.133:0/521664468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7802bd8b0 0x55c7802cdda0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.712+0000 7f195708ab80 1 -- 172.21.15.133:0/521664468 >> 172.21.15.133:0/521664468 conn(0x55c7802b2680 msgr2=0x55c7802c5640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:31:38.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.729+0000 7f195708ab80 1 -- 172.21.15.133:0/521664468 shutdown_connections 2023-07-14T18:31:38.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.745+0000 7f195708ab80 1 -- 172.21.15.133:0/521664468 wait complete. 2023-07-14T18:31:38.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.745+0000 7f195708ab80 1 -- start start 2023-07-14T18:31:38.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.745+0000 7f195708ab80 1 Processor -- start 2023-07-14T18:31:38.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.763+0000 7f195708ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7802bcf60 0x55c7802c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:38.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.763+0000 7f195708ab80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7802bd8b0 0x55c7802ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:38.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.763+0000 7f195708ab80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7802be4b0 0x55c7802cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:38.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.763+0000 7f195708ab80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7802cd840 con 0x55c7802be4b0 2023-07-14T18:31:38.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.763+0000 7f195708ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7802cd9b0 con 0x55c7802bcf60 2023-07-14T18:31:38.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.763+0000 7f195708ab80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7802cdb70 con 0x55c7802bd8b0 2023-07-14T18:31:38.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.763+0000 7f195708ab80 10 client.0 _ll_register_callbacks cb 0x7f1938099b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:31:38.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.764+0000 7f1940d9a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7802bd8b0 0x55c7802ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:31:38.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.765+0000 7f1940d9a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7802bd8b0 0x55c7802ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:35278/0 (socket says 172.21.15.133:35278) 2023-07-14T18:31:38.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.765+0000 7f1940d9a700 1 -- 172.21.15.133:0/1380432533 learned_addr learned my addr 172.21.15.133:0/1380432533 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:31:38.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.770+0000 7f194159b700 1 --2- 172.21.15.133:0/1380432533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7802bcf60 0x55c7802c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:31:38.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.771+0000 7f194159b700 1 -- 172.21.15.133:0/1380432533 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7802bd8b0 msgr2=0x55c7802ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:31:38.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.771+0000 7f194159b700 1 --2- 172.21.15.133:0/1380432533 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7802bd8b0 0x55c7802ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.771+0000 7f194159b700 1 -- 172.21.15.133:0/1380432533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7802be4b0 msgr2=0x55c7802cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:31:38.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.771+0000 7f194159b700 1 --2- 172.21.15.133:0/1380432533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7802be4b0 0x55c7802cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.771+0000 7f194159b700 1 -- 172.21.15.133:0/1380432533 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c78003bde0 con 0x55c7802bcf60 2023-07-14T18:31:38.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.771+0000 7f194159b700 1 --2- 172.21.15.133:0/1380432533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7802bcf60 0x55c7802c9da0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1938098310 tx=0x7f192c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfc51b69cc03fc8a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:31:38.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.772+0000 7f1924ff9700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:31:38.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.772+0000 7f1924ff9700 1 -- 172.21.15.133:0/1380432533 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c022d40 con 0x55c7802bcf60 2023-07-14T18:31:38.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.772+0000 7f1924ff9700 1 -- 172.21.15.133:0/1380432533 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f192c022040 con 0x55c7802bcf60 2023-07-14T18:31:38.816 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.779+0000 7f195708ab80 1 -- 172.21.15.133:0/1380432533 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f193400bbc0 con 0x55c7802bcf60 2023-07-14T18:31:38.816 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.780+0000 7f1924ff9700 1 -- 172.21.15.133:0/1380432533 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f192c022ea0 con 0x55c7802bcf60 2023-07-14T18:31:38.816 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.781+0000 7f195708ab80 10 client.26006 fetch_fsmap learned FSMap version 46 2023-07-14T18:31:38.817 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.781+0000 7f195708ab80 1 -- 172.21.15.133:0/1380432533 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55c78003a2b0 con 0x55c7802bcf60 2023-07-14T18:31:38.817 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.781+0000 7f1924ff9700 1 -- 172.21.15.133:0/1380432533 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f193400bbc0 con 0x55c7802bcf60 2023-07-14T18:31:38.817 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.781+0000 7f1924ff9700 1 -- 172.21.15.133:0/1380432533 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f192c034420 con 0x55c7802bcf60 2023-07-14T18:31:38.817 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.782+0000 7f195708ab80 10 client.26006 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T18:31:38.817 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.782+0000 7f195708ab80 10 client.26006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:31:38.817 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.782+0000 7f195708ab80 20 client.26006 populate_metadata read hostname 'smithi133' 2023-07-14T18:31:38.818 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.782+0000 7f195708ab80 1 --2- 172.21.15.133:0/1380432533 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55c780327430 0x55c780329810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:31:38.818 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.782+0000 7f195708ab80 4 client.26006 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:31:38.818 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.782+0000 7f195708ab80 1 -- 172.21.15.133:0/1380432533 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c780206eb0 con 0x55c780327430 2023-07-14T18:31:38.818 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.790+0000 7f1940d9a700 1 --2- 172.21.15.133:0/1380432533 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55c780327430 0x55c780329810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:31:38.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.795+0000 7f1940d9a700 1 --2- 172.21.15.133:0/1380432533 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55c780327430 0x55c780329810 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a26e8c6785f468b server_cookie=20c5ecff417da3a0 in_seq=0 out_seq=0 2023-07-14T18:31:38.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.797+0000 7f1924ff9700 10 client.26006 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:31:38.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.799+0000 7f1924ff9700 1 -- 172.21.15.133:0/1380432533 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55c780206eb0 con 0x55c780327430 2023-07-14T18:31:38.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.799+0000 7f1924ff9700 10 client.26006 handle_command_reply: tid=0 2023-07-14T18:31:38.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.799+0000 7f195708ab80 1 client.26006 shutdown 2023-07-14T18:31:38.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.815+0000 7f195708ab80 1 -- 172.21.15.133:0/1380432533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7802bcf60 msgr2=0x55c7802c9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:31:38.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.815+0000 7f195708ab80 1 --2- 172.21.15.133:0/1380432533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7802bcf60 0x55c7802c9da0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1938098310 tx=0x7f192c004750 comp rx=0 tx=0).stop 2023-07-14T18:31:38.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.816+0000 7f195708ab80 1 -- 172.21.15.133:0/1380432533 shutdown_connections 2023-07-14T18:31:38.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.816+0000 7f195708ab80 1 --2- 172.21.15.133:0/1380432533 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x55c780327430 0x55c780329810 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.816+0000 7f195708ab80 1 --2- 172.21.15.133:0/1380432533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7802bcf60 0x55c7802c9da0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.816+0000 7f195708ab80 1 --2- 172.21.15.133:0/1380432533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7802be4b0 0x55c7802cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.816+0000 7f195708ab80 1 --2- 172.21.15.133:0/1380432533 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7802bd8b0 0x55c7802ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.816+0000 7f195708ab80 1 -- 172.21.15.133:0/1380432533 >> 172.21.15.133:0/1380432533 conn(0x55c7802b2680 msgr2=0x55c7802b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:31:38.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.816+0000 7f1924ff9700 0 client.26006 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:31:38.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.816+0000 7f195708ab80 1 -- 172.21.15.133:0/1380432533 shutdown_connections 2023-07-14T18:31:38.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.817+0000 7f195708ab80 1 -- 172.21.15.133:0/1380432533 wait complete. 2023-07-14T18:31:38.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.818+0000 7f195708ab80 20 client.26006 trim_cache size 0 max 16384 2023-07-14T18:31:38.845 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:31:38.845 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T18:31:38.845 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:31:38.845 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:31:38.845 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:31:38.846 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:31:38.846 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:31:38.846 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:31:38.846 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:31:38.846 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:31:38.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.821+0000 7f19437ff700 1 -- 172.21.15.133:0/3416693683 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f19280a1350 msgr2=0x7f19280a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:31:38.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.821+0000 7f19437ff700 1 --2- 172.21.15.133:0/3416693683 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f19280a1350 0x7f19280a3800 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x55c7802bdcc0 tx=0x7f192c006c90 comp rx=0 tx=0).stop 2023-07-14T18:31:38.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.821+0000 7f19437ff700 1 -- 172.21.15.133:0/3416693683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f193c10be90 msgr2=0x7f193c1162b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:31:38.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.822+0000 7f19437ff700 1 --2- 172.21.15.133:0/3416693683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f193c10be90 0x7f193c1162b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f193800e940 tx=0x7f193800ed00 comp rx=0 tx=0).stop 2023-07-14T18:31:38.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.826+0000 7f19437ff700 1 -- 172.21.15.133:0/3416693683 shutdown_connections 2023-07-14T18:31:38.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.826+0000 7f19437ff700 1 --2- 172.21.15.133:0/3416693683 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f19280a1350 0x7f19280a3800 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.826+0000 7f19437ff700 1 --2- 172.21.15.133:0/3416693683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f193c10be90 0x7f193c1162b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.826+0000 7f19437ff700 1 --2- 172.21.15.133:0/3416693683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f193c071010 0x7f193c114d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.826+0000 7f19437ff700 1 --2- 172.21.15.133:0/3416693683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f193c071960 0x7f193c115270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:31:38.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.826+0000 7f19437ff700 1 -- 172.21.15.133:0/3416693683 >> 172.21.15.133:0/3416693683 conn(0x7f193c06cbc0 msgr2=0x7f193c06ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:31:38.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.827+0000 7f19437ff700 1 -- 172.21.15.133:0/3416693683 shutdown_connections 2023-07-14T18:31:38.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:31:38.827+0000 7f19437ff700 1 -- 172.21.15.133:0/3416693683 wait complete. 2023-07-14T18:31:38.882 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:31:40.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:39 smithi161 ceph-mon[128993]: pgmap v797: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 853 op/s 2023-07-14T18:31:40.097 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:39 smithi171 ceph-mon[129241]: pgmap v797: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 853 op/s 2023-07-14T18:31:40.146 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:39 smithi133 ceph-mon[126384]: pgmap v797: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 853 op/s 2023-07-14T18:31:40.856 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 735.0 s, 1195.0 tps, lat 85.327 ms stddev 65.150 2023-07-14T18:31:42.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:41 smithi161 ceph-mon[128993]: pgmap v798: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 816 op/s 2023-07-14T18:31:42.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:41 smithi171 ceph-mon[129241]: pgmap v798: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 816 op/s 2023-07-14T18:31:42.144 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:41 smithi133 ceph-mon[126384]: pgmap v798: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 816 op/s 2023-07-14T18:31:44.090 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:43 smithi161 ceph-mon[128993]: pgmap v799: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 686 op/s 2023-07-14T18:31:44.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:43 smithi171 ceph-mon[129241]: pgmap v799: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 686 op/s 2023-07-14T18:31:44.138 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:43 smithi133 ceph-mon[126384]: pgmap v799: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 686 op/s 2023-07-14T18:31:45.858 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 740.0 s, 807.8 tps, lat 117.809 ms stddev 133.531 2023-07-14T18:31:46.139 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:45 smithi133 ceph-mon[126384]: pgmap v800: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 599 op/s 2023-07-14T18:31:46.339 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:45 smithi161 ceph-mon[128993]: pgmap v800: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 599 op/s 2023-07-14T18:31:46.347 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:45 smithi171 ceph-mon[129241]: pgmap v800: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 599 op/s 2023-07-14T18:31:47.582 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:47 smithi161 ceph-mon[128993]: pgmap v801: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 701 op/s 2023-07-14T18:31:47.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:47 smithi171 ceph-mon[129241]: pgmap v801: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 701 op/s 2023-07-14T18:31:47.639 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:47 smithi133 ceph-mon[126384]: pgmap v801: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 701 op/s 2023-07-14T18:31:50.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:49 smithi161 ceph-mon[128993]: pgmap v802: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 677 op/s 2023-07-14T18:31:50.097 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:49 smithi171 ceph-mon[129241]: pgmap v802: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 677 op/s 2023-07-14T18:31:50.139 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:49 smithi133 ceph-mon[126384]: pgmap v802: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 677 op/s 2023-07-14T18:31:50.862 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 745.0 s, 686.8 tps, lat 150.838 ms stddev 126.507 2023-07-14T18:31:52.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:51 smithi161 ceph-mon[128993]: pgmap v803: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 730 op/s 2023-07-14T18:31:52.097 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:51 smithi171 ceph-mon[129241]: pgmap v803: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 730 op/s 2023-07-14T18:31:52.138 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:51 smithi133 ceph-mon[126384]: pgmap v803: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 730 op/s 2023-07-14T18:31:54.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:53 smithi161 ceph-mon[128993]: pgmap v804: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 13 MiB/s wr, 665 op/s 2023-07-14T18:31:54.097 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:53 smithi171 ceph-mon[129241]: pgmap v804: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 13 MiB/s wr, 665 op/s 2023-07-14T18:31:54.138 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:53 smithi133 ceph-mon[126384]: pgmap v804: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 13 MiB/s wr, 665 op/s 2023-07-14T18:31:55.858 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 750.0 s, 972.9 tps, lat 95.322 ms stddev 84.461 2023-07-14T18:31:56.137 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:55 smithi133 ceph-mon[126384]: pgmap v805: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 12 MiB/s wr, 596 op/s 2023-07-14T18:31:56.332 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:55 smithi161 ceph-mon[128993]: pgmap v805: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 12 MiB/s wr, 596 op/s 2023-07-14T18:31:56.347 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:55 smithi171 ceph-mon[129241]: pgmap v805: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 12 MiB/s wr, 596 op/s 2023-07-14T18:31:57.637 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:57 smithi133 ceph-mon[126384]: pgmap v806: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 13 MiB/s wr, 759 op/s 2023-07-14T18:31:57.831 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:57 smithi161 ceph-mon[128993]: pgmap v806: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 13 MiB/s wr, 759 op/s 2023-07-14T18:31:57.850 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:57 smithi171 ceph-mon[129241]: pgmap v806: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 13 MiB/s wr, 759 op/s 2023-07-14T18:32:00.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:31:59 smithi161 ceph-mon[128993]: pgmap v807: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 16 MiB/s wr, 1.05k op/s 2023-07-14T18:32:00.097 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:31:59 smithi171 ceph-mon[129241]: pgmap v807: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 16 MiB/s wr, 1.05k op/s 2023-07-14T18:32:00.140 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:31:59 smithi133 ceph-mon[126384]: pgmap v807: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 16 MiB/s wr, 1.05k op/s 2023-07-14T18:32:00.863 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 755.0 s, 1008.4 tps, lat 106.376 ms stddev 212.190 2023-07-14T18:32:02.082 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:01 smithi161 ceph-mon[128993]: pgmap v808: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 18 MiB/s wr, 1.26k op/s 2023-07-14T18:32:02.096 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:01 smithi171 ceph-mon[129241]: pgmap v808: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 18 MiB/s wr, 1.26k op/s 2023-07-14T18:32:02.138 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:01 smithi133 ceph-mon[126384]: pgmap v808: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 18 MiB/s wr, 1.26k op/s 2023-07-14T18:32:04.331 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:03 smithi161 ceph-mon[128993]: pgmap v809: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 18 MiB/s wr, 1.29k op/s 2023-07-14T18:32:04.346 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:03 smithi171 ceph-mon[129241]: pgmap v809: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 18 MiB/s wr, 1.29k op/s 2023-07-14T18:32:04.388 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:03 smithi133 ceph-mon[126384]: pgmap v809: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 18 MiB/s wr, 1.29k op/s 2023-07-14T18:32:05.857 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 760.0 s, 497.8 tps, lat 174.429 ms stddev 269.961 2023-07-14T18:32:06.333 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:05 smithi161 ceph-mon[128993]: pgmap v810: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 18 MiB/s wr, 1.25k op/s 2023-07-14T18:32:06.346 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:05 smithi171 ceph-mon[129241]: pgmap v810: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 18 MiB/s wr, 1.25k op/s 2023-07-14T18:32:06.391 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:05 smithi133 ceph-mon[126384]: pgmap v810: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 18 MiB/s wr, 1.25k op/s 2023-07-14T18:32:07.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:07 smithi161 ceph-mon[128993]: pgmap v811: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 22 MiB/s wr, 1.67k op/s 2023-07-14T18:32:07.846 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:07 smithi171 ceph-mon[129241]: pgmap v811: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 22 MiB/s wr, 1.67k op/s 2023-07-14T18:32:07.888 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:07 smithi133 ceph-mon[126384]: pgmap v811: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 22 MiB/s wr, 1.67k op/s 2023-07-14T18:32:08.883 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T18:32:09.892 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:09 smithi133 ceph-mon[126384]: pgmap v812: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 28 MiB/s wr, 2.27k op/s 2023-07-14T18:32:09.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.899+0000 7ff9bb5ff700 1 -- 172.21.15.133:0/4046833148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9b40ffc10 msgr2=0x7ff9b4100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:32:09.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.899+0000 7ff9bb5ff700 1 --2- 172.21.15.133:0/4046833148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9b40ffc10 0x7ff9b4100020 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4009990 tx=0x7ff9a4009ca0 comp rx=0 tx=0).stop 2023-07-14T18:32:09.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.899+0000 7ff9bb5ff700 1 -- 172.21.15.133:0/4046833148 shutdown_connections 2023-07-14T18:32:09.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.899+0000 7ff9bb5ff700 1 --2- 172.21.15.133:0/4046833148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b4101790 0x7ff9b4108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:09.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.899+0000 7ff9bb5ff700 1 --2- 172.21.15.133:0/4046833148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9b40ffc10 0x7ff9b4100020 secure :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4009990 tx=0x7ff9a4009ca0 comp rx=0 tx=0).stop 2023-07-14T18:32:09.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.899+0000 7ff9bb5ff700 1 --2- 172.21.15.133:0/4046833148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9b4100e00 0x7ff9b4101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:09.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.899+0000 7ff9bb5ff700 1 -- 172.21.15.133:0/4046833148 >> 172.21.15.133:0/4046833148 conn(0x7ff9b40fb250 msgr2=0x7ff9b40fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:32:09.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.899+0000 7ff9bb5ff700 1 -- 172.21.15.133:0/4046833148 shutdown_connections 2023-07-14T18:32:09.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.899+0000 7ff9bb5ff700 1 -- 172.21.15.133:0/4046833148 wait complete. 2023-07-14T18:32:09.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.901+0000 7ff9bb5ff700 1 Processor -- start 2023-07-14T18:32:09.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.901+0000 7ff9bb5ff700 1 -- start start 2023-07-14T18:32:09.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.901+0000 7ff9bb5ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9b4100e00 0x7ff9b406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:32:09.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.902+0000 7ff9bb5ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b4101790 0x7ff9b406b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:32:09.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.902+0000 7ff9bb5ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9b4070560 0x7ff9b40709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:32:09.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.902+0000 7ff9bb5ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b4070f10 con 0x7ff9b4070560 2023-07-14T18:32:09.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.902+0000 7ff9bb5ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b4071050 con 0x7ff9b4101790 2023-07-14T18:32:09.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.902+0000 7ff9bb5ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b4071210 con 0x7ff9b4100e00 2023-07-14T18:32:09.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.902+0000 7ff9b8b9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b4101790 0x7ff9b406b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:32:09.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.902+0000 7ff9b8b9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b4101790 0x7ff9b406b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:37418/0 (socket says 172.21.15.133:37418) 2023-07-14T18:32:09.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.902+0000 7ff9b8b9a700 1 -- 172.21.15.133:0/3477287156 learned_addr learned my addr 172.21.15.133:0/3477287156 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:32:09.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.902+0000 7ff9b939b700 1 --2- 172.21.15.133:0/3477287156 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9b4100e00 0x7ff9b406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:32:09.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.902+0000 7ff9b9b9c700 1 --2- 172.21.15.133:0/3477287156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9b4070560 0x7ff9b40709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:32:09.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.903+0000 7ff9b8b9a700 1 -- 172.21.15.133:0/3477287156 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9b4100e00 msgr2=0x7ff9b406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:32:09.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.903+0000 7ff9b8b9a700 1 --2- 172.21.15.133:0/3477287156 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9b4100e00 0x7ff9b406afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:09.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.903+0000 7ff9b8b9a700 1 -- 172.21.15.133:0/3477287156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9b4070560 msgr2=0x7ff9b40709d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:32:09.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.903+0000 7ff9b8b9a700 1 --2- 172.21.15.133:0/3477287156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9b4070560 0x7ff9b40709d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:09.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.903+0000 7ff9b8b9a700 1 -- 172.21.15.133:0/3477287156 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a40095f0 con 0x7ff9b4101790 2023-07-14T18:32:09.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.903+0000 7ff9b8b9a700 1 --2- 172.21.15.133:0/3477287156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b4101790 0x7ff9b406b500 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff9b000b550 tx=0x7ff9b000b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:32:09.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.906+0000 7ff9aa7fc700 1 -- 172.21.15.133:0/3477287156 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0011830 con 0x7ff9b4101790 2023-07-14T18:32:09.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.906+0000 7ff9bb5ff700 1 -- 172.21.15.133:0/3477287156 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b40714f0 con 0x7ff9b4101790 2023-07-14T18:32:09.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.906+0000 7ff9bb5ff700 1 -- 172.21.15.133:0/3477287156 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b4071a80 con 0x7ff9b4101790 2023-07-14T18:32:09.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.906+0000 7ff9aa7fc700 1 -- 172.21.15.133:0/3477287156 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9b0011e50 con 0x7ff9b4101790 2023-07-14T18:32:09.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.906+0000 7ff9aa7fc700 1 -- 172.21.15.133:0/3477287156 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0010e40 con 0x7ff9b4101790 2023-07-14T18:32:09.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.914+0000 7ff9aa7fc700 1 -- 172.21.15.133:0/3477287156 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff9b0011990 con 0x7ff9b4101790 2023-07-14T18:32:09.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.915+0000 7ff9cefe0b80 1 Processor -- start 2023-07-14T18:32:09.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.917+0000 7ff9aa7fc700 1 --2- 172.21.15.133:0/3477287156 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff9a00a11d0 0x7ff9a00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:32:09.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.917+0000 7ff9aa7fc700 1 -- 172.21.15.133:0/3477287156 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff9b00cc5e0 con 0x7ff9b4101790 2023-07-14T18:32:09.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.918+0000 7ff9cefe0b80 1 -- start start 2023-07-14T18:32:09.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.918+0000 7ff9cefe0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596fa323f60 0x5596fa324370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:32:09.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.918+0000 7ff9cefe0b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596fa3248b0 0x5596fa324ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:32:09.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.918+0000 7ff9cefe0b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5596fa3254b0 0x5596fa325920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:32:09.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.918+0000 7ff9cefe0b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596fa228cb0 con 0x5596fa3248b0 2023-07-14T18:32:09.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.918+0000 7ff9cefe0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596fa27ba40 con 0x5596fa323f60 2023-07-14T18:32:09.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.918+0000 7ff9cefe0b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596fa274ed0 con 0x5596fa3254b0 2023-07-14T18:32:09.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.919+0000 7ff9b939b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596fa323f60 0x5596fa324370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:32:09.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.919+0000 7ff9b939b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596fa323f60 0x5596fa324370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:37428/0 (socket says 172.21.15.133:37428) 2023-07-14T18:32:09.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.919+0000 7ff9b939b700 1 -- 172.21.15.133:0/769303783 learned_addr learned my addr 172.21.15.133:0/769303783 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:32:09.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.920+0000 7ff9b939b700 1 -- 172.21.15.133:0/769303783 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5596fa3254b0 msgr2=0x5596fa325920 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T18:32:09.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.920+0000 7ff9b939b700 1 --2- 172.21.15.133:0/769303783 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5596fa3254b0 0x5596fa325920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:09.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.920+0000 7ff9b939b700 1 -- 172.21.15.133:0/769303783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596fa3248b0 msgr2=0x5596fa324ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:32:09.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.920+0000 7ff9b939b700 1 --2- 172.21.15.133:0/769303783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596fa3248b0 0x5596fa324ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:09.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.920+0000 7ff9b939b700 1 -- 172.21.15.133:0/769303783 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596fa0a2de0 con 0x5596fa323f60 2023-07-14T18:32:09.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.920+0000 7ff9b9b9c700 1 --2- 172.21.15.133:0/3477287156 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff9a00a11d0 0x7ff9a00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:32:09.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.920+0000 7ff9b939b700 1 --2- 172.21.15.133:0/769303783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596fa323f60 0x5596fa324370 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400b6b0 tx=0x7ff9a400b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22edb6f8bd58b475 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:32:09.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.921+0000 7ff9b9b9c700 1 --2- 172.21.15.133:0/3477287156 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff9a00a11d0 0x7ff9a00a3680 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009ec0 tx=0x7ff9ac00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:32:09.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.922+0000 7ff99f7fe700 1 -- 172.21.15.133:0/769303783 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a401d070 con 0x5596fa323f60 2023-07-14T18:32:09.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.922+0000 7ff99f7fe700 1 -- 172.21.15.133:0/769303783 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9a401e040 con 0x5596fa323f60 2023-07-14T18:32:09.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.922+0000 7ff99f7fe700 1 -- 172.21.15.133:0/769303783 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4004bb0 con 0x5596fa323f60 2023-07-14T18:32:09.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.923+0000 7ff9cefe0b80 1 -- 172.21.15.133:0/769303783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596fa323f60 msgr2=0x5596fa324370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:32:09.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.923+0000 7ff9cefe0b80 1 --2- 172.21.15.133:0/769303783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596fa323f60 0x5596fa324370 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400b6b0 tx=0x7ff9a400b860 comp rx=0 tx=0).stop 2023-07-14T18:32:09.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.924+0000 7ff9cefe0b80 1 -- 172.21.15.133:0/769303783 shutdown_connections 2023-07-14T18:32:09.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.924+0000 7ff9cefe0b80 1 --2- 172.21.15.133:0/769303783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596fa323f60 0x5596fa324370 secure :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400b6b0 tx=0x7ff9a400b860 comp rx=0 tx=0).stop 2023-07-14T18:32:09.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.924+0000 7ff9cefe0b80 1 --2- 172.21.15.133:0/769303783 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5596fa3254b0 0x5596fa325920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:09.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.924+0000 7ff9cefe0b80 1 --2- 172.21.15.133:0/769303783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596fa3248b0 0x5596fa324ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:09.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.924+0000 7ff9cefe0b80 1 -- 172.21.15.133:0/769303783 >> 172.21.15.133:0/769303783 conn(0x5596fa319680 msgr2=0x5596fa31e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:32:09.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.934+0000 7ff9cefe0b80 1 -- 172.21.15.133:0/769303783 shutdown_connections 2023-07-14T18:32:09.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.934+0000 7ff9cefe0b80 1 -- 172.21.15.133:0/769303783 wait complete. 2023-07-14T18:32:09.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.935+0000 7ff9cefe0b80 1 -- start start 2023-07-14T18:32:09.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.935+0000 7ff9cefe0b80 1 Processor -- start 2023-07-14T18:32:09.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.936+0000 7ff9cefe0b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596fa3248b0 0x5596fa32db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:32:09.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.936+0000 7ff9cefe0b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5596fa3254b0 0x5596fa32e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:32:09.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.936+0000 7ff9cefe0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596fa3330d0 0x5596fa3354b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:32:09.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.936+0000 7ff9cefe0b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596fa214ac0 con 0x5596fa3248b0 2023-07-14T18:32:09.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.936+0000 7ff9cefe0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596fa2147c0 con 0x5596fa3330d0 2023-07-14T18:32:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.936+0000 7ff9cefe0b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596fa0f8920 con 0x5596fa3254b0 2023-07-14T18:32:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.936+0000 7ff9cefe0b80 10 client.0 _ll_register_callbacks cb 0x5596fa212760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:32:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.937+0000 7ff9b9b9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596fa3330d0 0x5596fa3354b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:32:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.937+0000 7ff9b9b9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596fa3330d0 0x5596fa3354b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:37440/0 (socket says 172.21.15.133:37440) 2023-07-14T18:32:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.937+0000 7ff9b9b9c700 1 -- 172.21.15.133:0/1213248143 learned_addr learned my addr 172.21.15.133:0/1213248143 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:32:09.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.937+0000 7ff9b8b9a700 1 --2- 172.21.15.133:0/1213248143 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5596fa3254b0 0x5596fa32e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:32:09.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.937+0000 7ff9b939b700 1 --2- 172.21.15.133:0/1213248143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596fa3248b0 0x5596fa32db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:32:09.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.937+0000 7ff9b9b9c700 1 -- 172.21.15.133:0/1213248143 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5596fa3254b0 msgr2=0x5596fa32e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:32:09.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.937+0000 7ff9b9b9c700 1 --2- 172.21.15.133:0/1213248143 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5596fa3254b0 0x5596fa32e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:09.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.937+0000 7ff9b9b9c700 1 -- 172.21.15.133:0/1213248143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596fa3248b0 msgr2=0x5596fa32db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:32:09.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.937+0000 7ff9b9b9c700 1 --2- 172.21.15.133:0/1213248143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596fa3248b0 0x5596fa32db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:09.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.937+0000 7ff9b9b9c700 1 -- 172.21.15.133:0/1213248143 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b4108980 con 0x5596fa3330d0 2023-07-14T18:32:09.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.938+0000 7ff9b939b700 1 --2- 172.21.15.133:0/1213248143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596fa3248b0 0x5596fa32db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T18:32:09.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.938+0000 7ff9b9b9c700 1 --2- 172.21.15.133:0/1213248143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596fa3330d0 0x5596fa3354b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac008a90 tx=0x7ff9ac004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a86c5d66a3212d14 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:32:09.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.938+0000 7ff99cff9700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T18:32:09.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.938+0000 7ff99cff9700 1 -- 172.21.15.133:0/1213248143 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac01c040 con 0x5596fa3330d0 2023-07-14T18:32:09.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.938+0000 7ff99cff9700 1 -- 172.21.15.133:0/1213248143 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff9ac017ce0 con 0x5596fa3330d0 2023-07-14T18:32:09.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.939+0000 7ff9cefe0b80 1 -- 172.21.15.133:0/1213248143 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596fa1ce4f0 con 0x5596fa3330d0 2023-07-14T18:32:09.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.939+0000 7ff99cff9700 1 -- 172.21.15.133:0/1213248143 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9ac00e040 con 0x5596fa3330d0 2023-07-14T18:32:09.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.939+0000 7ff9cefe0b80 10 client.26018 fetch_fsmap learned FSMap version 46 2023-07-14T18:32:09.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.939+0000 7ff9cefe0b80 1 -- 172.21.15.133:0/1213248143 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5596fa08bc60 con 0x5596fa3330d0 2023-07-14T18:32:09.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.940+0000 7ff99cff9700 1 -- 172.21.15.133:0/1213248143 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5596fa1ce4f0 con 0x5596fa3330d0 2023-07-14T18:32:09.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.940+0000 7ff99cff9700 1 -- 172.21.15.133:0/1213248143 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff9ac015410 con 0x5596fa3330d0 2023-07-14T18:32:09.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.941+0000 7ff9cefe0b80 10 client.26018 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T18:32:09.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.941+0000 7ff9cefe0b80 10 client.26018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:32:09.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.941+0000 7ff9cefe0b80 20 client.26018 populate_metadata read hostname 'smithi133' 2023-07-14T18:32:09.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.941+0000 7ff9cefe0b80 1 --2- 172.21.15.133:0/1213248143 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5596fa339d70 0x5596fa33c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:32:09.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.941+0000 7ff9cefe0b80 4 client.26018 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T18:32:09.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.941+0000 7ff9cefe0b80 1 -- 172.21.15.133:0/1213248143 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596fa2f2da0 con 0x5596fa339d70 2023-07-14T18:32:09.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.941+0000 7ff9b939b700 1 --2- 172.21.15.133:0/1213248143 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5596fa339d70 0x5596fa33c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:32:09.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.943+0000 7ff9b939b700 1 --2- 172.21.15.133:0/1213248143 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5596fa339d70 0x5596fa33c180 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=492ea3bc85461ba7 server_cookie=46c55df7f42460e1 in_seq=0 out_seq=0 2023-07-14T18:32:09.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.943+0000 7ff99cff9700 10 client.26018 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:32:09.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.944+0000 7ff99cff9700 1 -- 172.21.15.133:0/1213248143 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5596fa2f2da0 con 0x5596fa339d70 2023-07-14T18:32:09.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.944+0000 7ff99cff9700 10 client.26018 handle_command_reply: tid=0 2023-07-14T18:32:09.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.944+0000 7ff9cefe0b80 1 client.26018 shutdown 2023-07-14T18:32:09.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.945+0000 7ff9cefe0b80 1 -- 172.21.15.133:0/1213248143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596fa3330d0 msgr2=0x5596fa3354b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:32:09.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.945+0000 7ff9cefe0b80 1 --2- 172.21.15.133:0/1213248143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596fa3330d0 0x5596fa3354b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac008a90 tx=0x7ff9ac004ad0 comp rx=0 tx=0).stop 2023-07-14T18:32:09.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.945+0000 7ff9cefe0b80 1 -- 172.21.15.133:0/1213248143 shutdown_connections 2023-07-14T18:32:09.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.945+0000 7ff9cefe0b80 1 --2- 172.21.15.133:0/1213248143 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5596fa339d70 0x5596fa33c180 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:09.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.946+0000 7ff9cefe0b80 1 --2- 172.21.15.133:0/1213248143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596fa3330d0 0x5596fa3354b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:09.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.946+0000 7ff9cefe0b80 1 --2- 172.21.15.133:0/1213248143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596fa3248b0 0x5596fa32db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:09.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.946+0000 7ff9cefe0b80 1 --2- 172.21.15.133:0/1213248143 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5596fa3254b0 0x5596fa32e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:09.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.946+0000 7ff9cefe0b80 1 -- 172.21.15.133:0/1213248143 >> 172.21.15.133:0/1213248143 conn(0x5596fa319680 msgr2=0x5596fa31e980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:32:09.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.946+0000 7ff99cff9700 0 client.26018 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:32:09.974 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.946+0000 7ff9cefe0b80 1 -- 172.21.15.133:0/1213248143 shutdown_connections 2023-07-14T18:32:09.974 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.946+0000 7ff9cefe0b80 1 -- 172.21.15.133:0/1213248143 wait complete. 2023-07-14T18:32:09.976 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.947+0000 7ff9cefe0b80 20 client.26018 trim_cache size 0 max 16384 2023-07-14T18:32:10.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.979+0000 7ff9cefe0b80 1 Processor -- start 2023-07-14T18:32:10.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.979+0000 7ff9cefe0b80 1 -- start start 2023-07-14T18:32:10.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.979+0000 7ff9cefe0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596fa3248b0 0x5596fa340ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:32:10.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.979+0000 7ff9cefe0b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596fa3254b0 0x5596fa325880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:32:10.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.979+0000 7ff9cefe0b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5596fa3330d0 0x5596fa3497e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:32:10.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.979+0000 7ff9cefe0b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596fa1ce4f0 con 0x5596fa3254b0 2023-07-14T18:32:10.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.979+0000 7ff9cefe0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596fa31dd70 con 0x5596fa3248b0 2023-07-14T18:32:10.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.979+0000 7ff9cefe0b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596fa3192a0 con 0x5596fa3330d0 2023-07-14T18:32:10.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.980+0000 7ff9b9b9c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5596fa3330d0 0x5596fa3497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:32:10.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.980+0000 7ff9b9b9c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5596fa3330d0 0x5596fa3497e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:39920/0 (socket says 172.21.15.133:39920) 2023-07-14T18:32:10.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.980+0000 7ff9b9b9c700 1 -- 172.21.15.133:0/1513553974 learned_addr learned my addr 172.21.15.133:0/1513553974 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:32:10.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.980+0000 7ff9b8b9a700 1 --2- 172.21.15.133:0/1513553974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596fa3254b0 0x5596fa325880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:32:10.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.980+0000 7ff9b9b9c700 1 -- 172.21.15.133:0/1513553974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596fa3248b0 msgr2=0x5596fa340ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T18:32:10.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.980+0000 7ff9b9b9c700 1 --2- 172.21.15.133:0/1513553974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596fa3248b0 0x5596fa340ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:10.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.980+0000 7ff9b9b9c700 1 -- 172.21.15.133:0/1513553974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596fa3254b0 msgr2=0x5596fa325880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:32:10.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.980+0000 7ff9b9b9c700 1 --2- 172.21.15.133:0/1513553974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596fa3254b0 0x5596fa325880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:10.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.980+0000 7ff9b9b9c700 1 -- 172.21.15.133:0/1513553974 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596fa0c9250 con 0x5596fa3330d0 2023-07-14T18:32:10.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.981+0000 7ff9b9b9c700 1 --2- 172.21.15.133:0/1513553974 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5596fa3330d0 0x5596fa3497e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009fd0 tx=0x7ff9ac009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17956596c42bfc7e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:32:10.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.981+0000 7ff99ffff700 1 -- 172.21.15.133:0/1513553974 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac03a070 con 0x5596fa3330d0 2023-07-14T18:32:10.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.982+0000 7ff99ffff700 1 -- 172.21.15.133:0/1513553974 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9ac00f040 con 0x5596fa3330d0 2023-07-14T18:32:10.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.982+0000 7ff99ffff700 1 -- 172.21.15.133:0/1513553974 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac00e040 con 0x5596fa3330d0 2023-07-14T18:32:10.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.983+0000 7ff9cefe0b80 1 -- 172.21.15.133:0/1513553974 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5596fa3330d0 msgr2=0x5596fa3497e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:32:10.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.983+0000 7ff9cefe0b80 1 --2- 172.21.15.133:0/1513553974 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5596fa3330d0 0x5596fa3497e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009fd0 tx=0x7ff9ac009920 comp rx=0 tx=0).stop 2023-07-14T18:32:10.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.983+0000 7ff9cefe0b80 1 -- 172.21.15.133:0/1513553974 shutdown_connections 2023-07-14T18:32:10.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.983+0000 7ff9cefe0b80 1 --2- 172.21.15.133:0/1513553974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596fa3248b0 0x5596fa340ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:10.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.983+0000 7ff9cefe0b80 1 --2- 172.21.15.133:0/1513553974 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5596fa3330d0 0x5596fa3497e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:10.010 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.983+0000 7ff9cefe0b80 1 --2- 172.21.15.133:0/1513553974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596fa3254b0 0x5596fa325880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:10.010 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.983+0000 7ff9cefe0b80 1 -- 172.21.15.133:0/1513553974 >> 172.21.15.133:0/1513553974 conn(0x5596fa319680 msgr2=0x5596fa32ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:32:10.010 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.984+0000 7ff9cefe0b80 1 -- 172.21.15.133:0/1513553974 shutdown_connections 2023-07-14T18:32:10.011 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.984+0000 7ff9cefe0b80 1 -- 172.21.15.133:0/1513553974 wait complete. 2023-07-14T18:32:10.011 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.984+0000 7ff9cefe0b80 1 -- start start 2023-07-14T18:32:10.011 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.985+0000 7ff9cefe0b80 1 Processor -- start 2023-07-14T18:32:10.011 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.985+0000 7ff9cefe0b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596fa3248b0 0x5596fa346c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:32:10.011 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.985+0000 7ff9cefe0b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5596fa3254b0 0x5596fa347160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:32:10.011 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.985+0000 7ff9cefe0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596fa3330d0 0x5596fa348170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:32:10.011 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.985+0000 7ff9cefe0b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596fa3486b0 con 0x5596fa3248b0 2023-07-14T18:32:10.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.985+0000 7ff9cefe0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596fa348820 con 0x5596fa3330d0 2023-07-14T18:32:10.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.985+0000 7ff9cefe0b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596fa3489e0 con 0x5596fa3254b0 2023-07-14T18:32:10.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.986+0000 7ff9cefe0b80 10 client.0 _ll_register_callbacks cb 0x7ff9ac0093e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T18:32:10.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.986+0000 7ff9b8b9a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5596fa3254b0 0x5596fa347160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:32:10.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.986+0000 7ff9b8b9a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5596fa3254b0 0x5596fa347160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:39936/0 (socket says 172.21.15.133:39936) 2023-07-14T18:32:10.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.986+0000 7ff9b8b9a700 1 -- 172.21.15.133:0/1444466870 learned_addr learned my addr 172.21.15.133:0/1444466870 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:32:10.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.986+0000 7ff9b9b9c700 1 --2- 172.21.15.133:0/1444466870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596fa3330d0 0x5596fa348170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:32:10.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.986+0000 7ff9b939b700 1 --2- 172.21.15.133:0/1444466870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596fa3248b0 0x5596fa346c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:32:10.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.987+0000 7ff9b939b700 1 -- 172.21.15.133:0/1444466870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5596fa3254b0 msgr2=0x5596fa347160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:32:10.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.987+0000 7ff9b939b700 1 --2- 172.21.15.133:0/1444466870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5596fa3254b0 0x5596fa347160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:10.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.987+0000 7ff9b939b700 1 -- 172.21.15.133:0/1444466870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596fa3330d0 msgr2=0x5596fa348170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T18:32:10.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.987+0000 7ff9b939b700 1 --2- 172.21.15.133:0/1444466870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596fa3330d0 0x5596fa348170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:10.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.987+0000 7ff9b939b700 1 -- 172.21.15.133:0/1444466870 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596fa08bc60 con 0x5596fa3248b0 2023-07-14T18:32:10.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.987+0000 7ff9b939b700 1 --2- 172.21.15.133:0/1444466870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596fa3248b0 0x5596fa346c20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4018000 tx=0x7ff9a4005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=605d44e8153a4e27 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:32:10.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.988+0000 7ff99cff9700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T18:32:10.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.988+0000 7ff99cff9700 1 -- 172.21.15.133:0/1444466870 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a401d040 con 0x5596fa3248b0 2023-07-14T18:32:10.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.988+0000 7ff99cff9700 1 -- 172.21.15.133:0/1444466870 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff9a40054d0 con 0x5596fa3248b0 2023-07-14T18:32:10.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.989+0000 7ff9cefe0b80 1 -- 172.21.15.133:0/1444466870 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9a401e040 con 0x5596fa3248b0 2023-07-14T18:32:10.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.989+0000 7ff99cff9700 1 -- 172.21.15.133:0/1444466870 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9a40056b0 con 0x5596fa3248b0 2023-07-14T18:32:10.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.989+0000 7ff9cefe0b80 10 client.15918 fetch_fsmap learned FSMap version 46 2023-07-14T18:32:10.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.989+0000 7ff9cefe0b80 1 -- 172.21.15.133:0/1444466870 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5596fa0a12b0 con 0x5596fa3248b0 2023-07-14T18:32:10.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.990+0000 7ff99cff9700 1 -- 172.21.15.133:0/1444466870 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9a401e040 con 0x5596fa3248b0 2023-07-14T18:32:10.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.990+0000 7ff99cff9700 1 -- 172.21.15.133:0/1444466870 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff9a403b070 con 0x5596fa3248b0 2023-07-14T18:32:10.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.990+0000 7ff9cefe0b80 10 client.15918 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T18:32:10.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.990+0000 7ff9cefe0b80 10 client.15918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T18:32:10.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.990+0000 7ff9cefe0b80 20 client.15918 populate_metadata read hostname 'smithi133' 2023-07-14T18:32:10.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.990+0000 7ff9cefe0b80 1 --2- 172.21.15.133:0/1444466870 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5596fa38e1d0 0x5596fa3905b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:32:10.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.990+0000 7ff9cefe0b80 4 client.15918 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2023-07-14T18:32:10.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.990+0000 7ff9cefe0b80 1 -- 172.21.15.133:0/1444466870 --> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5596fa26deb0 con 0x5596fa38e1d0 2023-07-14T18:32:10.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.991+0000 7ff9b8b9a700 1 --2- 172.21.15.133:0/1444466870 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5596fa38e1d0 0x5596fa3905b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:32:10.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.994+0000 7ff9b8b9a700 1 --2- 172.21.15.133:0/1444466870 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5596fa38e1d0 0x5596fa3905b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa0f936d20f42023 server_cookie=7ba0351fb780824e in_seq=0 out_seq=0 2023-07-14T18:32:10.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.996+0000 7ff99cff9700 10 client.15918 ms_handle_connect on v2:172.21.15.161:6838/2503865871 2023-07-14T18:32:10.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.996+0000 7ff99cff9700 1 -- 172.21.15.133:0/1444466870 <== mds.0 v2:172.21.15.161:6838/2503865871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5596fa26deb0 con 0x5596fa38e1d0 2023-07-14T18:32:10.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.996+0000 7ff99cff9700 10 client.15918 handle_command_reply: tid=0 2023-07-14T18:32:10.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.996+0000 7ff9cefe0b80 1 client.15918 shutdown 2023-07-14T18:32:10.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.997+0000 7ff9cefe0b80 1 -- 172.21.15.133:0/1444466870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596fa3248b0 msgr2=0x5596fa346c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:32:10.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.997+0000 7ff9cefe0b80 1 --2- 172.21.15.133:0/1444466870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596fa3248b0 0x5596fa346c20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4018000 tx=0x7ff9a4005da0 comp rx=0 tx=0).stop 2023-07-14T18:32:10.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.998+0000 7ff9cefe0b80 1 -- 172.21.15.133:0/1444466870 shutdown_connections 2023-07-14T18:32:10.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.998+0000 7ff9cefe0b80 1 --2- 172.21.15.133:0/1444466870 >> [v2:172.21.15.161:6838/2503865871,v1:172.21.15.161:6839/2503865871] conn(0x5596fa38e1d0 0x5596fa3905b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:10.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.998+0000 7ff9cefe0b80 1 --2- 172.21.15.133:0/1444466870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596fa3330d0 0x5596fa348170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:10.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.998+0000 7ff9cefe0b80 1 --2- 172.21.15.133:0/1444466870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596fa3248b0 0x5596fa346c20 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:10.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.998+0000 7ff9cefe0b80 1 --2- 172.21.15.133:0/1444466870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5596fa3254b0 0x5596fa347160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:10.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.998+0000 7ff9cefe0b80 1 -- 172.21.15.133:0/1444466870 >> 172.21.15.133:0/1444466870 conn(0x5596fa319680 msgr2=0x5596fa32ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:32:10.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.998+0000 7ff99cff9700 0 client.15918 ms_handle_reset on v2:172.21.15.161:6838/2503865871 2023-07-14T18:32:10.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.998+0000 7ff9cefe0b80 1 -- 172.21.15.133:0/1444466870 shutdown_connections 2023-07-14T18:32:10.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.999+0000 7ff9cefe0b80 1 -- 172.21.15.133:0/1444466870 wait complete. 2023-07-14T18:32:10.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:09.999+0000 7ff9cefe0b80 20 client.15918 trim_cache size 0 max 16384 2023-07-14T18:32:10.025 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:32:10.026 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-14T18:32:10.026 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-07-14T18:32:10.026 INFO:teuthology.orchestra.run.smithi133.stdout: "57ccb48c-b320-4fdf-b110-1c3062ac52dc": { 2023-07-14T18:32:10.026 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-07-14T18:32:10.026 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "57ccb48c-b320-4fdf-b110-1c3062ac52dc", 2023-07-14T18:32:10.026 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-07-14T18:32:10.026 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:32:10.027 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:32:10.027 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:32:10.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:10.000+0000 7ff9bb5ff700 1 -- 172.21.15.133:0/3477287156 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff9a00a11d0 msgr2=0x7ff9a00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:32:10.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:10.000+0000 7ff9bb5ff700 1 --2- 172.21.15.133:0/3477287156 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff9a00a11d0 0x7ff9a00a3680 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009ec0 tx=0x7ff9ac00b040 comp rx=0 tx=0).stop 2023-07-14T18:32:10.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:10.000+0000 7ff9bb5ff700 1 -- 172.21.15.133:0/3477287156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b4101790 msgr2=0x7ff9b406b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:32:10.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:10.000+0000 7ff9bb5ff700 1 --2- 172.21.15.133:0/3477287156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b4101790 0x7ff9b406b500 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff9b000b550 tx=0x7ff9b000b860 comp rx=0 tx=0).stop 2023-07-14T18:32:10.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:10.000+0000 7ff9bb5ff700 1 -- 172.21.15.133:0/3477287156 shutdown_connections 2023-07-14T18:32:10.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:10.000+0000 7ff9bb5ff700 1 --2- 172.21.15.133:0/3477287156 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff9a00a11d0 0x7ff9a00a3680 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:10.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:10.000+0000 7ff9bb5ff700 1 --2- 172.21.15.133:0/3477287156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9b4070560 0x7ff9b40709d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:10.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:10.000+0000 7ff9bb5ff700 1 --2- 172.21.15.133:0/3477287156 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9b4100e00 0x7ff9b406afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:10.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:10.000+0000 7ff9bb5ff700 1 --2- 172.21.15.133:0/3477287156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b4101790 0x7ff9b406b500 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:32:10.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:10.001+0000 7ff9bb5ff700 1 -- 172.21.15.133:0/3477287156 >> 172.21.15.133:0/3477287156 conn(0x7ff9b40fb250 msgr2=0x7ff9b40fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:32:10.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:10.001+0000 7ff9bb5ff700 1 -- 172.21.15.133:0/3477287156 shutdown_connections 2023-07-14T18:32:10.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:32:10.001+0000 7ff9bb5ff700 1 -- 172.21.15.133:0/3477287156 wait complete. 2023-07-14T18:32:10.060 INFO:tasks.cephfs.filesystem:scrub status for tag:57ccb48c-b320-4fdf-b110-1c3062ac52dc - {'path': '/', 'tag': '57ccb48c-b320-4fdf-b110-1c3062ac52dc', 'options': 'recursive,force'} 2023-07-14T18:32:10.061 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_81c9387d2e7576da065747ec9c16d1c7d69016ec/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_81c9387d2e7576da065747ec9c16d1c7d69016ec/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_81c9387d2e7576da065747ec9c16d1c7d69016ec/qa/tasks/fwd_scrub.py", line 76, in _scrub done = self.fs.wait_until_scrub_complete(tag=tag, sleep=30, timeout=self.scrub_timeout) File "/home/teuthworker/src/git.ceph.com_ceph-c_81c9387d2e7576da065747ec9c16d1c7d69016ec/qa/tasks/cephfs/filesystem.py", line 1683, in wait_until_scrub_complete while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (31) after waiting for 900 seconds 2023-07-14T18:32:10.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:09 smithi161 ceph-mon[128993]: pgmap v812: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 28 MiB/s wr, 2.27k op/s 2023-07-14T18:32:10.097 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:09 smithi171 ceph-mon[129241]: pgmap v812: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 28 MiB/s wr, 2.27k op/s 2023-07-14T18:32:10.855 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 765.0 s, 226.4 tps, lat 504.722 ms stddev 809.548 2023-07-14T18:32:12.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:11 smithi161 ceph-mon[128993]: pgmap v813: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 28 MiB/s wr, 2.63k op/s 2023-07-14T18:32:12.096 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:11 smithi171 ceph-mon[129241]: pgmap v813: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 28 MiB/s wr, 2.63k op/s 2023-07-14T18:32:12.137 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:11 smithi133 ceph-mon[126384]: pgmap v813: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 28 MiB/s wr, 2.63k op/s 2023-07-14T18:32:14.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:13 smithi161 ceph-mon[128993]: pgmap v814: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 26 MiB/s wr, 2.54k op/s 2023-07-14T18:32:14.096 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:13 smithi171 ceph-mon[129241]: pgmap v814: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 26 MiB/s wr, 2.54k op/s 2023-07-14T18:32:14.137 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:13 smithi133 ceph-mon[126384]: pgmap v814: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 26 MiB/s wr, 2.54k op/s 2023-07-14T18:32:15.855 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 770.0 s, 1102.8 tps, lat 88.898 ms stddev 74.916 2023-07-14T18:32:16.333 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:15 smithi161 ceph-mon[128993]: pgmap v815: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 27 MiB/s wr, 2.68k op/s 2023-07-14T18:32:16.346 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:15 smithi171 ceph-mon[129241]: pgmap v815: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 27 MiB/s wr, 2.68k op/s 2023-07-14T18:32:16.387 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:15 smithi133 ceph-mon[126384]: pgmap v815: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 27 MiB/s wr, 2.68k op/s 2023-07-14T18:32:17.638 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:17 smithi133 ceph-mon[126384]: pgmap v816: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 29 MiB/s wr, 2.74k op/s 2023-07-14T18:32:17.831 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:17 smithi161 ceph-mon[128993]: pgmap v816: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 29 MiB/s wr, 2.74k op/s 2023-07-14T18:32:17.850 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:17 smithi171 ceph-mon[129241]: pgmap v816: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 29 MiB/s wr, 2.74k op/s 2023-07-14T18:32:20.096 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:19 smithi171 ceph-mon[129241]: pgmap v817: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 24 MiB/s wr, 2.29k op/s 2023-07-14T18:32:20.137 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:19 smithi133 ceph-mon[126384]: pgmap v817: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 24 MiB/s wr, 2.29k op/s 2023-07-14T18:32:20.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:19 smithi161 ceph-mon[128993]: pgmap v817: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 24 MiB/s wr, 2.29k op/s 2023-07-14T18:32:20.855 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 775.0 s, 503.0 tps, lat 125.777 ms stddev 274.878 2023-07-14T18:32:22.137 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:21 smithi133 ceph-mon[126384]: pgmap v818: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 23 MiB/s wr, 2.12k op/s 2023-07-14T18:32:22.330 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:21 smithi161 ceph-mon[128993]: pgmap v818: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 23 MiB/s wr, 2.12k op/s 2023-07-14T18:32:22.346 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:21 smithi171 ceph-mon[129241]: pgmap v818: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 23 MiB/s wr, 2.12k op/s 2023-07-14T18:32:24.333 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:23 smithi161 ceph-mon[128993]: pgmap v819: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 19 MiB/s wr, 1.57k op/s 2023-07-14T18:32:24.346 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:23 smithi171 ceph-mon[129241]: pgmap v819: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 19 MiB/s wr, 1.57k op/s 2023-07-14T18:32:24.388 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:23 smithi133 ceph-mon[126384]: pgmap v819: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 19 MiB/s wr, 1.57k op/s 2023-07-14T18:32:25.856 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 780.0 s, 659.0 tps, lat 208.857 ms stddev 477.563 2023-07-14T18:32:26.330 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:25 smithi161 ceph-mon[128993]: pgmap v820: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 17 MiB/s wr, 1.41k op/s 2023-07-14T18:32:26.345 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:25 smithi171 ceph-mon[129241]: pgmap v820: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 17 MiB/s wr, 1.41k op/s 2023-07-14T18:32:26.387 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:25 smithi133 ceph-mon[126384]: pgmap v820: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 17 MiB/s wr, 1.41k op/s 2023-07-14T18:32:27.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:27 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:32:27.831 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:27 smithi161 ceph-mon[128993]: pgmap v821: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 18 MiB/s wr, 1.46k op/s 2023-07-14T18:32:27.847 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:27 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:32:27.847 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:27 smithi171 ceph-mon[129241]: pgmap v821: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 18 MiB/s wr, 1.46k op/s 2023-07-14T18:32:27.896 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:27 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:32:27.897 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:27 smithi133 ceph-mon[126384]: pgmap v821: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 18 MiB/s wr, 1.46k op/s 2023-07-14T18:32:28.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:28 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:32:28.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:28 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:32:28.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:28 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:32:28.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:32:28.850 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:32:28.850 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:28 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:32:28.887 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:28 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:32:28.888 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:28 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:32:28.888 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:28 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:32:29.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:29 smithi161 ceph-mon[128993]: pgmap v822: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 15 MiB/s wr, 1.35k op/s 2023-07-14T18:32:29.846 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:29 smithi171 ceph-mon[129241]: pgmap v822: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 15 MiB/s wr, 1.35k op/s 2023-07-14T18:32:29.890 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:29 smithi133 ceph-mon[126384]: pgmap v822: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 15 MiB/s wr, 1.35k op/s 2023-07-14T18:32:30.860 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 785.0 s, 483.6 tps, lat 198.521 ms stddev 486.317 2023-07-14T18:32:32.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:31 smithi161 ceph-mon[128993]: pgmap v823: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 18 MiB/s wr, 1.70k op/s 2023-07-14T18:32:32.096 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:31 smithi171 ceph-mon[129241]: pgmap v823: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 18 MiB/s wr, 1.70k op/s 2023-07-14T18:32:32.140 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:31 smithi133 ceph-mon[126384]: pgmap v823: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 18 MiB/s wr, 1.70k op/s 2023-07-14T18:32:34.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:33 smithi161 ceph-mon[128993]: pgmap v824: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 15 MiB/s wr, 1.23k op/s 2023-07-14T18:32:34.096 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:33 smithi171 ceph-mon[129241]: pgmap v824: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 15 MiB/s wr, 1.23k op/s 2023-07-14T18:32:34.137 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:33 smithi133 ceph-mon[126384]: pgmap v824: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 15 MiB/s wr, 1.23k op/s 2023-07-14T18:32:35.858 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 790.0 s, 721.2 tps, lat 92.191 ms stddev 95.414 2023-07-14T18:32:36.330 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:35 smithi161 ceph-mon[128993]: pgmap v825: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 12 MiB/s wr, 975 op/s 2023-07-14T18:32:36.345 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:35 smithi171 ceph-mon[129241]: pgmap v825: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 12 MiB/s wr, 975 op/s 2023-07-14T18:32:36.387 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:35 smithi133 ceph-mon[126384]: pgmap v825: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 12 MiB/s wr, 975 op/s 2023-07-14T18:32:37.596 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:37 smithi171 ceph-mon[129241]: pgmap v826: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 18 MiB/s wr, 1.36k op/s 2023-07-14T18:32:37.644 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:37 smithi133 ceph-mon[126384]: pgmap v826: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 18 MiB/s wr, 1.36k op/s 2023-07-14T18:32:37.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:37 smithi161 ceph-mon[128993]: pgmap v826: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 18 MiB/s wr, 1.36k op/s 2023-07-14T18:32:40.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:39 smithi161 ceph-mon[128993]: pgmap v827: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 13 MiB/s wr, 1.00k op/s 2023-07-14T18:32:40.095 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:39 smithi171 ceph-mon[129241]: pgmap v827: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 13 MiB/s wr, 1.00k op/s 2023-07-14T18:32:40.136 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:39 smithi133 ceph-mon[126384]: pgmap v827: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 13 MiB/s wr, 1.00k op/s 2023-07-14T18:32:40.858 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 795.0 s, 424.3 tps, lat 322.474 ms stddev 703.470 2023-07-14T18:32:42.331 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:41 smithi161 ceph-mon[128993]: pgmap v828: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 1.19k op/s 2023-07-14T18:32:42.347 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:41 smithi171 ceph-mon[129241]: pgmap v828: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 1.19k op/s 2023-07-14T18:32:42.395 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:41 smithi133 ceph-mon[126384]: pgmap v828: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 1.19k op/s 2023-07-14T18:32:44.331 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:43 smithi161 ceph-mon[128993]: pgmap v829: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 13 MiB/s wr, 939 op/s 2023-07-14T18:32:44.346 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:43 smithi171 ceph-mon[129241]: pgmap v829: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 13 MiB/s wr, 939 op/s 2023-07-14T18:32:44.387 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:43 smithi133 ceph-mon[126384]: pgmap v829: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 13 MiB/s wr, 939 op/s 2023-07-14T18:32:45.853 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 800.0 s, 465.4 tps, lat 187.239 ms stddev 257.893 2023-07-14T18:32:46.331 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:45 smithi161 ceph-mon[128993]: pgmap v830: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 11 MiB/s wr, 812 op/s 2023-07-14T18:32:46.352 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:45 smithi171 ceph-mon[129241]: pgmap v830: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 11 MiB/s wr, 812 op/s 2023-07-14T18:32:46.387 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:45 smithi133 ceph-mon[126384]: pgmap v830: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 11 MiB/s wr, 812 op/s 2023-07-14T18:32:47.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:47 smithi161 ceph-mon[128993]: pgmap v831: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 15 MiB/s wr, 1.18k op/s 2023-07-14T18:32:47.845 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:47 smithi171 ceph-mon[129241]: pgmap v831: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 15 MiB/s wr, 1.18k op/s 2023-07-14T18:32:47.889 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:47 smithi133 ceph-mon[126384]: pgmap v831: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 15 MiB/s wr, 1.18k op/s 2023-07-14T18:32:50.094 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:49 smithi171 ceph-mon[129241]: pgmap v832: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.6 MiB/s wr, 797 op/s 2023-07-14T18:32:50.136 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:49 smithi133 ceph-mon[126384]: pgmap v832: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.6 MiB/s wr, 797 op/s 2023-07-14T18:32:50.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:49 smithi161 ceph-mon[128993]: pgmap v832: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.6 MiB/s wr, 797 op/s 2023-07-14T18:32:50.857 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 805.0 s, 750.4 tps, lat 150.627 ms stddev 329.869 2023-07-14T18:32:52.637 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:52 smithi133 ceph-mon[126384]: pgmap v833: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 14 MiB/s wr, 1.05k op/s 2023-07-14T18:32:52.829 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:52 smithi161 ceph-mon[128993]: pgmap v833: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 14 MiB/s wr, 1.05k op/s 2023-07-14T18:32:52.844 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:52 smithi171 ceph-mon[129241]: pgmap v833: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 14 MiB/s wr, 1.05k op/s 2023-07-14T18:32:53.637 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:53 smithi133 ceph-mon[126384]: pgmap v834: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 14 MiB/s wr, 1.05k op/s 2023-07-14T18:32:53.831 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:53 smithi161 ceph-mon[128993]: pgmap v834: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 14 MiB/s wr, 1.05k op/s 2023-07-14T18:32:53.844 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:53 smithi171 ceph-mon[129241]: pgmap v834: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 14 MiB/s wr, 1.05k op/s 2023-07-14T18:32:55.856 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 810.0 s, 758.4 tps, lat 94.564 ms stddev 70.398 2023-07-14T18:32:56.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:55 smithi161 ceph-mon[128993]: pgmap v835: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 12 MiB/s wr, 941 op/s 2023-07-14T18:32:56.349 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:55 smithi171 ceph-mon[129241]: pgmap v835: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 12 MiB/s wr, 941 op/s 2023-07-14T18:32:56.387 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:55 smithi133 ceph-mon[126384]: pgmap v835: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 12 MiB/s wr, 941 op/s 2023-07-14T18:32:57.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:57 smithi161 ceph-mon[128993]: pgmap v836: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 17 MiB/s wr, 1.35k op/s 2023-07-14T18:32:57.846 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:57 smithi171 ceph-mon[129241]: pgmap v836: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 17 MiB/s wr, 1.35k op/s 2023-07-14T18:32:57.886 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:57 smithi133 ceph-mon[126384]: pgmap v836: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 17 MiB/s wr, 1.35k op/s 2023-07-14T18:33:00.139 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:32:59 smithi133 ceph-mon[126384]: pgmap v837: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 14 MiB/s wr, 1.05k op/s 2023-07-14T18:33:00.332 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:32:59 smithi161 ceph-mon[128993]: pgmap v837: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 14 MiB/s wr, 1.05k op/s 2023-07-14T18:33:00.353 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:32:59 smithi171 ceph-mon[129241]: pgmap v837: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 14 MiB/s wr, 1.05k op/s 2023-07-14T18:33:00.855 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 815.0 s, 668.5 tps, lat 163.095 ms stddev 398.079 2023-07-14T18:33:02.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:01 smithi161 ceph-mon[128993]: pgmap v838: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 14 MiB/s wr, 1.05k op/s 2023-07-14T18:33:02.347 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:01 smithi171 ceph-mon[129241]: pgmap v838: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 14 MiB/s wr, 1.05k op/s 2023-07-14T18:33:02.388 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:01 smithi133 ceph-mon[126384]: pgmap v838: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 14 MiB/s wr, 1.05k op/s 2023-07-14T18:33:04.345 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:03 smithi171 ceph-mon[129241]: pgmap v839: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 15 MiB/s wr, 1.20k op/s 2023-07-14T18:33:04.346 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:03 smithi161 ceph-mon[128993]: pgmap v839: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 15 MiB/s wr, 1.20k op/s 2023-07-14T18:33:04.393 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:03 smithi133 ceph-mon[126384]: pgmap v839: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 15 MiB/s wr, 1.20k op/s 2023-07-14T18:33:05.853 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 820.0 s, 531.6 tps, lat 218.175 ms stddev 353.219 2023-07-14T18:33:06.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:06 smithi161 ceph-mon[128993]: pgmap v840: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 13 MiB/s wr, 999 op/s 2023-07-14T18:33:06.344 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:06 smithi171 ceph-mon[129241]: pgmap v840: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 13 MiB/s wr, 999 op/s 2023-07-14T18:33:06.395 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:06 smithi133 ceph-mon[126384]: pgmap v840: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 13 MiB/s wr, 999 op/s 2023-07-14T18:33:07.831 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:07 smithi161 ceph-mon[128993]: pgmap v841: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 18 MiB/s wr, 1.34k op/s 2023-07-14T18:33:07.847 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:07 smithi171 ceph-mon[129241]: pgmap v841: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 18 MiB/s wr, 1.34k op/s 2023-07-14T18:33:07.886 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:07 smithi133 ceph-mon[126384]: pgmap v841: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 18 MiB/s wr, 1.34k op/s 2023-07-14T18:33:10.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:09 smithi161 ceph-mon[128993]: pgmap v842: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 14 MiB/s wr, 984 op/s 2023-07-14T18:33:10.345 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:09 smithi171 ceph-mon[129241]: pgmap v842: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 14 MiB/s wr, 984 op/s 2023-07-14T18:33:10.395 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:09 smithi133 ceph-mon[126384]: pgmap v842: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 14 MiB/s wr, 984 op/s 2023-07-14T18:33:10.857 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 825.0 s, 895.7 tps, lat 99.915 ms stddev 119.844 2023-07-14T18:33:12.332 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:11 smithi161 ceph-mon[128993]: pgmap v843: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 12 MiB/s wr, 938 op/s 2023-07-14T18:33:12.344 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:11 smithi171 ceph-mon[129241]: pgmap v843: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 12 MiB/s wr, 938 op/s 2023-07-14T18:33:12.386 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:11 smithi133 ceph-mon[126384]: pgmap v843: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 12 MiB/s wr, 938 op/s 2023-07-14T18:33:14.331 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:14 smithi161 ceph-mon[128993]: pgmap v844: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 19 MiB/s wr, 1.44k op/s 2023-07-14T18:33:14.346 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:14 smithi171 ceph-mon[129241]: pgmap v844: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 19 MiB/s wr, 1.44k op/s 2023-07-14T18:33:14.385 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:14 smithi133 ceph-mon[126384]: pgmap v844: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 19 MiB/s wr, 1.44k op/s 2023-07-14T18:33:15.859 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 830.0 s, 379.4 tps, lat 274.184 ms stddev 524.743 2023-07-14T18:33:16.386 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:16 smithi133 ceph-mon[126384]: pgmap v845: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 13 MiB/s wr, 1.03k op/s 2023-07-14T18:33:16.579 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:16 smithi161 ceph-mon[128993]: pgmap v845: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 13 MiB/s wr, 1.03k op/s 2023-07-14T18:33:16.601 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:16 smithi171 ceph-mon[129241]: pgmap v845: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 13 MiB/s wr, 1.03k op/s 2023-07-14T18:33:17.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:17 smithi161 ceph-mon[128993]: pgmap v846: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 16 MiB/s wr, 1.31k op/s 2023-07-14T18:33:17.843 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:17 smithi171 ceph-mon[129241]: pgmap v846: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 16 MiB/s wr, 1.31k op/s 2023-07-14T18:33:17.889 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:17 smithi133 ceph-mon[126384]: pgmap v846: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 16 MiB/s wr, 1.31k op/s 2023-07-14T18:33:20.331 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:19 smithi161 ceph-mon[128993]: pgmap v847: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 13 MiB/s wr, 1.10k op/s 2023-07-14T18:33:20.343 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:19 smithi171 ceph-mon[129241]: pgmap v847: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 13 MiB/s wr, 1.10k op/s 2023-07-14T18:33:20.385 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:19 smithi133 ceph-mon[126384]: pgmap v847: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 13 MiB/s wr, 1.10k op/s 2023-07-14T18:33:20.853 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 835.0 s, 753.9 tps, lat 97.402 ms stddev 152.905 2023-07-14T18:33:22.828 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:22 smithi161 ceph-mon[128993]: pgmap v848: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 13 MiB/s wr, 1.06k op/s 2023-07-14T18:33:22.843 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:22 smithi171 ceph-mon[129241]: pgmap v848: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 13 MiB/s wr, 1.06k op/s 2023-07-14T18:33:22.890 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:22 smithi133 ceph-mon[126384]: pgmap v848: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 13 MiB/s wr, 1.06k op/s 2023-07-14T18:33:24.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:24 smithi161 ceph-mon[128993]: pgmap v849: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 19 MiB/s wr, 1.52k op/s 2023-07-14T18:33:24.844 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:24 smithi171 ceph-mon[129241]: pgmap v849: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 19 MiB/s wr, 1.52k op/s 2023-07-14T18:33:24.885 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:24 smithi133 ceph-mon[126384]: pgmap v849: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 19 MiB/s wr, 1.52k op/s 2023-07-14T18:33:25.828 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:25 smithi161 ceph-mon[128993]: pgmap v850: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 12 MiB/s wr, 932 op/s 2023-07-14T18:33:25.843 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:25 smithi171 ceph-mon[129241]: pgmap v850: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 12 MiB/s wr, 932 op/s 2023-07-14T18:33:25.854 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 840.0 s, 691.6 tps, lat 192.775 ms stddev 467.524 2023-07-14T18:33:25.891 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:25 smithi133 ceph-mon[126384]: pgmap v850: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 12 MiB/s wr, 932 op/s 2023-07-14T18:33:28.828 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:28 smithi161 ceph-mon[128993]: pgmap v851: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-07-14T18:33:28.843 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:28 smithi171 ceph-mon[129241]: pgmap v851: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-07-14T18:33:28.891 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:28 smithi133 ceph-mon[126384]: pgmap v851: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-07-14T18:33:29.834 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:29 smithi161 ceph-mon[128993]: pgmap v852: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 1.12k op/s 2023-07-14T18:33:29.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:29 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:33:29.844 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:29 smithi171 ceph-mon[129241]: pgmap v852: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 1.12k op/s 2023-07-14T18:33:29.845 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:29 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:33:29.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:29 smithi133 ceph-mon[126384]: pgmap v852: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 1.12k op/s 2023-07-14T18:33:29.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:29 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:33:30.843 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:30 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:33:30.843 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:30 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:33:30.844 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:30 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:33:30.844 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:30 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:33:30.844 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:30 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:33:30.844 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:30 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:33:30.844 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:30 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:33:30.852 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 845.0 s, 932.3 tps, lat 112.090 ms stddev 148.596 2023-07-14T18:33:30.891 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:30 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:33:30.892 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:30 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:33:30.892 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:30 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:33:30.892 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:30 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:33:30.892 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:30 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:33:30.892 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:30 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:33:30.893 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:30 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:33:31.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:30 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:33:31.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:30 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:33:31.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:30 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:33:31.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:30 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:33:31.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:30 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:33:31.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:30 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:33:31.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:30 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:33:32.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:31 smithi161 ceph-mon[128993]: Adjusting osd_memory_target on smithi133 to 220.9M 2023-07-14T18:33:32.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:31 smithi161 ceph-mon[128993]: Unable to set osd_memory_target on smithi133 to 231661158: error parsing value: Value '231661158' is below minimum 939524096 2023-07-14T18:33:32.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:31 smithi161 ceph-mon[128993]: pgmap v853: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 14 MiB/s wr, 1.00k op/s 2023-07-14T18:33:32.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:31 smithi171 ceph-mon[129241]: Adjusting osd_memory_target on smithi133 to 220.9M 2023-07-14T18:33:32.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:31 smithi171 ceph-mon[129241]: Unable to set osd_memory_target on smithi133 to 231661158: error parsing value: Value '231661158' is below minimum 939524096 2023-07-14T18:33:32.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:31 smithi171 ceph-mon[129241]: pgmap v853: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 14 MiB/s wr, 1.00k op/s 2023-07-14T18:33:32.135 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:31 smithi133 ceph-mon[126384]: Adjusting osd_memory_target on smithi133 to 220.9M 2023-07-14T18:33:32.135 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:31 smithi133 ceph-mon[126384]: Unable to set osd_memory_target on smithi133 to 231661158: error parsing value: Value '231661158' is below minimum 939524096 2023-07-14T18:33:32.135 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:31 smithi133 ceph-mon[126384]: pgmap v853: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 14 MiB/s wr, 1.00k op/s 2023-07-14T18:33:34.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:33 smithi161 ceph-mon[128993]: pgmap v854: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 17 MiB/s wr, 1.21k op/s 2023-07-14T18:33:34.343 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:33 smithi171 ceph-mon[129241]: pgmap v854: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 17 MiB/s wr, 1.21k op/s 2023-07-14T18:33:34.384 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:33 smithi133 ceph-mon[126384]: pgmap v854: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 17 MiB/s wr, 1.21k op/s 2023-07-14T18:33:35.861 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 850.0 s, 223.6 tps, lat 443.269 ms stddev 680.711 2023-07-14T18:33:36.328 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:35 smithi161 ceph-mon[128993]: pgmap v855: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 13 MiB/s wr, 964 op/s 2023-07-14T18:33:36.343 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:35 smithi171 ceph-mon[129241]: pgmap v855: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 13 MiB/s wr, 964 op/s 2023-07-14T18:33:36.388 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:35 smithi133 ceph-mon[126384]: pgmap v855: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 13 MiB/s wr, 964 op/s 2023-07-14T18:33:38.828 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:38 smithi161 ceph-mon[128993]: pgmap v856: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 15 MiB/s wr, 1.12k op/s 2023-07-14T18:33:38.843 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:38 smithi171 ceph-mon[129241]: pgmap v856: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 15 MiB/s wr, 1.12k op/s 2023-07-14T18:33:38.889 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:38 smithi133 ceph-mon[126384]: pgmap v856: 129 pgs: 129 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 15 MiB/s wr, 1.12k op/s 2023-07-14T18:33:39.828 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:39 smithi161 ceph-mon[128993]: pgmap v857: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 14 MiB/s wr, 1.02k op/s 2023-07-14T18:33:39.845 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:39 smithi171 ceph-mon[129241]: pgmap v857: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 14 MiB/s wr, 1.02k op/s 2023-07-14T18:33:39.884 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:39 smithi133 ceph-mon[126384]: pgmap v857: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 14 MiB/s wr, 1.02k op/s 2023-07-14T18:33:40.857 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 855.0 s, 825.3 tps, lat 121.101 ms stddev 78.416 2023-07-14T18:33:42.827 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:42 smithi161 ceph-mon[128993]: pgmap v858: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 12 MiB/s wr, 932 op/s 2023-07-14T18:33:42.848 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:42 smithi171 ceph-mon[129241]: pgmap v858: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 12 MiB/s wr, 932 op/s 2023-07-14T18:33:42.884 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:42 smithi133 ceph-mon[126384]: pgmap v858: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 12 MiB/s wr, 932 op/s 2023-07-14T18:33:43.827 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:43 smithi161 ceph-mon[128993]: pgmap v859: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 15 MiB/s wr, 1.14k op/s 2023-07-14T18:33:43.842 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:43 smithi171 ceph-mon[129241]: pgmap v859: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 15 MiB/s wr, 1.14k op/s 2023-07-14T18:33:43.889 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:43 smithi133 ceph-mon[126384]: pgmap v859: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 15 MiB/s wr, 1.14k op/s 2023-07-14T18:33:45.853 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 860.0 s, 632.8 tps, lat 101.487 ms stddev 64.514 2023-07-14T18:33:46.327 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:45 smithi161 ceph-mon[128993]: pgmap v860: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 13 MiB/s wr, 959 op/s 2023-07-14T18:33:46.342 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:45 smithi171 ceph-mon[129241]: pgmap v860: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 13 MiB/s wr, 959 op/s 2023-07-14T18:33:46.385 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:45 smithi133 ceph-mon[126384]: pgmap v860: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 13 MiB/s wr, 959 op/s 2023-07-14T18:33:48.827 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:48 smithi161 ceph-mon[128993]: pgmap v861: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 12 MiB/s wr, 920 op/s 2023-07-14T18:33:48.842 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:48 smithi171 ceph-mon[129241]: pgmap v861: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 12 MiB/s wr, 920 op/s 2023-07-14T18:33:48.885 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:48 smithi133 ceph-mon[126384]: pgmap v861: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 12 MiB/s wr, 920 op/s 2023-07-14T18:33:50.827 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:50 smithi161 ceph-mon[128993]: pgmap v862: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 15 MiB/s wr, 1.19k op/s 2023-07-14T18:33:50.842 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:50 smithi171 ceph-mon[129241]: pgmap v862: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 15 MiB/s wr, 1.19k op/s 2023-07-14T18:33:50.857 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 865.0 s, 459.2 tps, lat 252.473 ms stddev 645.109 2023-07-14T18:33:50.887 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:50 smithi133 ceph-mon[126384]: pgmap v862: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 15 MiB/s wr, 1.19k op/s 2023-07-14T18:33:52.829 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:52 smithi161 ceph-mon[128993]: pgmap v863: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 14 MiB/s wr, 1.07k op/s 2023-07-14T18:33:52.848 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:52 smithi171 ceph-mon[129241]: pgmap v863: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 14 MiB/s wr, 1.07k op/s 2023-07-14T18:33:52.886 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:52 smithi133 ceph-mon[126384]: pgmap v863: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 14 MiB/s wr, 1.07k op/s 2023-07-14T18:33:53.883 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:53 smithi133 ceph-mon[126384]: pgmap v864: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 17 MiB/s wr, 1.28k op/s 2023-07-14T18:33:54.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:53 smithi161 ceph-mon[128993]: pgmap v864: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 17 MiB/s wr, 1.28k op/s 2023-07-14T18:33:54.097 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:53 smithi171 ceph-mon[129241]: pgmap v864: 129 pgs: 129 active+clean; 8.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 17 MiB/s wr, 1.28k op/s 2023-07-14T18:33:55.851 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 870.0 s, 963.1 tps, lat 124.261 ms stddev 225.185 2023-07-14T18:33:56.333 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:55 smithi161 ceph-mon[128993]: pgmap v865: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 14 MiB/s wr, 1.15k op/s 2023-07-14T18:33:56.344 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:55 smithi171 ceph-mon[129241]: pgmap v865: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 14 MiB/s wr, 1.15k op/s 2023-07-14T18:33:56.383 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:55 smithi133 ceph-mon[126384]: pgmap v865: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 14 MiB/s wr, 1.15k op/s 2023-07-14T18:33:58.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:33:58 smithi161 ceph-mon[128993]: pgmap v866: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 15 MiB/s wr, 1.16k op/s 2023-07-14T18:33:58.844 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:33:58 smithi171 ceph-mon[129241]: pgmap v866: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 15 MiB/s wr, 1.16k op/s 2023-07-14T18:33:58.884 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:33:58 smithi133 ceph-mon[126384]: pgmap v866: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 15 MiB/s wr, 1.16k op/s 2023-07-14T18:34:00.827 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:00 smithi161 ceph-mon[128993]: pgmap v867: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 15 MiB/s wr, 1.19k op/s 2023-07-14T18:34:00.842 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:00 smithi171 ceph-mon[129241]: pgmap v867: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 15 MiB/s wr, 1.19k op/s 2023-07-14T18:34:00.853 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 875.0 s, 555.4 tps, lat 162.155 ms stddev 304.625 2023-07-14T18:34:00.883 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:00 smithi133 ceph-mon[126384]: pgmap v867: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 15 MiB/s wr, 1.19k op/s 2023-07-14T18:34:02.827 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:02 smithi161 ceph-mon[128993]: pgmap v868: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 13 MiB/s wr, 886 op/s 2023-07-14T18:34:02.842 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:02 smithi171 ceph-mon[129241]: pgmap v868: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 13 MiB/s wr, 886 op/s 2023-07-14T18:34:02.888 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:02 smithi133 ceph-mon[126384]: pgmap v868: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 13 MiB/s wr, 886 op/s 2023-07-14T18:34:04.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:04 smithi161 ceph-mon[128993]: pgmap v869: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 16 MiB/s wr, 1.17k op/s 2023-07-14T18:34:04.842 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:04 smithi171 ceph-mon[129241]: pgmap v869: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 16 MiB/s wr, 1.17k op/s 2023-07-14T18:34:04.893 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:04 smithi133 ceph-mon[126384]: pgmap v869: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 16 MiB/s wr, 1.17k op/s 2023-07-14T18:34:05.852 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 880.0 s, 1097.7 tps, lat 99.344 ms stddev 154.593 2023-07-14T18:34:06.828 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:06 smithi161 ceph-mon[128993]: pgmap v870: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 16 MiB/s wr, 1.17k op/s 2023-07-14T18:34:06.842 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:06 smithi171 ceph-mon[129241]: pgmap v870: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 16 MiB/s wr, 1.17k op/s 2023-07-14T18:34:06.883 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:06 smithi133 ceph-mon[126384]: pgmap v870: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 16 MiB/s wr, 1.17k op/s 2023-07-14T18:34:08.827 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:08 smithi161 ceph-mon[128993]: pgmap v871: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 17 MiB/s wr, 1.17k op/s 2023-07-14T18:34:08.842 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:08 smithi171 ceph-mon[129241]: pgmap v871: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 17 MiB/s wr, 1.17k op/s 2023-07-14T18:34:08.883 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:08 smithi133 ceph-mon[126384]: pgmap v871: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 17 MiB/s wr, 1.17k op/s 2023-07-14T18:34:10.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:09 smithi161 ceph-mon[128993]: pgmap v872: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 1.28k op/s 2023-07-14T18:34:10.092 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:09 smithi171 ceph-mon[129241]: pgmap v872: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 1.28k op/s 2023-07-14T18:34:10.133 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:09 smithi133 ceph-mon[126384]: pgmap v872: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 1.28k op/s 2023-07-14T18:34:10.852 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 885.0 s, 742.6 tps, lat 95.008 ms stddev 76.949 2023-07-14T18:34:12.826 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:12 smithi161 ceph-mon[128993]: pgmap v873: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 17 MiB/s wr, 1.20k op/s 2023-07-14T18:34:12.841 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:12 smithi171 ceph-mon[129241]: pgmap v873: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 17 MiB/s wr, 1.20k op/s 2023-07-14T18:34:12.882 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:12 smithi133 ceph-mon[126384]: pgmap v873: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 17 MiB/s wr, 1.20k op/s 2023-07-14T18:34:14.829 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:14 smithi161 ceph-mon[128993]: pgmap v874: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 19 MiB/s wr, 1.45k op/s 2023-07-14T18:34:14.843 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:14 smithi171 ceph-mon[129241]: pgmap v874: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 19 MiB/s wr, 1.45k op/s 2023-07-14T18:34:14.886 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:14 smithi133 ceph-mon[126384]: pgmap v874: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 19 MiB/s wr, 1.45k op/s 2023-07-14T18:34:15.852 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 890.0 s, 709.6 tps, lat 161.882 ms stddev 364.891 2023-07-14T18:34:16.884 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:16 smithi133 ceph-mon[126384]: pgmap v875: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 17 MiB/s wr, 1.31k op/s 2023-07-14T18:34:17.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:16 smithi161 ceph-mon[128993]: pgmap v875: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 17 MiB/s wr, 1.31k op/s 2023-07-14T18:34:17.092 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:16 smithi171 ceph-mon[129241]: pgmap v875: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 17 MiB/s wr, 1.31k op/s 2023-07-14T18:34:18.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:17 smithi161 ceph-mon[128993]: pgmap v876: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 16 MiB/s wr, 1.24k op/s 2023-07-14T18:34:18.091 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:17 smithi171 ceph-mon[129241]: pgmap v876: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 16 MiB/s wr, 1.24k op/s 2023-07-14T18:34:18.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:17 smithi133 ceph-mon[126384]: pgmap v876: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 16 MiB/s wr, 1.24k op/s 2023-07-14T18:34:20.326 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:19 smithi161 ceph-mon[128993]: pgmap v877: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 19 MiB/s wr, 1.65k op/s 2023-07-14T18:34:20.341 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:19 smithi171 ceph-mon[129241]: pgmap v877: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 19 MiB/s wr, 1.65k op/s 2023-07-14T18:34:20.388 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:19 smithi133 ceph-mon[126384]: pgmap v877: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 19 MiB/s wr, 1.65k op/s 2023-07-14T18:34:20.852 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 895.0 s, 358.6 tps, lat 215.493 ms stddev 463.604 2023-07-14T18:34:22.828 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:22 smithi161 ceph-mon[128993]: pgmap v878: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 17 MiB/s wr, 1.52k op/s 2023-07-14T18:34:22.841 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:22 smithi171 ceph-mon[129241]: pgmap v878: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 17 MiB/s wr, 1.52k op/s 2023-07-14T18:34:22.883 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:22 smithi133 ceph-mon[126384]: pgmap v878: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 17 MiB/s wr, 1.52k op/s 2023-07-14T18:34:24.841 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:24 smithi171 ceph-mon[129241]: pgmap v879: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 18 MiB/s wr, 1.57k op/s 2023-07-14T18:34:24.882 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:24 smithi133 ceph-mon[126384]: pgmap v879: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 18 MiB/s wr, 1.57k op/s 2023-07-14T18:34:25.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:24 smithi161 ceph-mon[128993]: pgmap v879: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 18 MiB/s wr, 1.57k op/s 2023-07-14T18:34:25.858 INFO:teuthology.orchestra.run.smithi133.stderr:progress: 900.0 s, 717.5 tps, lat 165.629 ms stddev 411.569 2023-07-14T18:34:26.596 INFO:teuthology.orchestra.run.smithi133.stdout:transaction type: 2023-07-14T18:34:26.596 INFO:teuthology.orchestra.run.smithi133.stdout:scaling factor: 500 2023-07-14T18:34:26.596 INFO:teuthology.orchestra.run.smithi133.stdout:query mode: simple 2023-07-14T18:34:26.596 INFO:teuthology.orchestra.run.smithi133.stdout:number of clients: 100 2023-07-14T18:34:26.597 INFO:teuthology.orchestra.run.smithi133.stdout:number of threads: 4 2023-07-14T18:34:26.597 INFO:teuthology.orchestra.run.smithi133.stdout:duration: 900 s 2023-07-14T18:34:26.597 INFO:teuthology.orchestra.run.smithi133.stdout:number of transactions actually processed: 758032 2023-07-14T18:34:26.597 INFO:teuthology.orchestra.run.smithi133.stdout:latency average = 118.717 ms 2023-07-14T18:34:26.597 INFO:teuthology.orchestra.run.smithi133.stdout:latency stddev = 264.818 ms 2023-07-14T18:34:26.597 INFO:teuthology.orchestra.run.smithi133.stdout:tps = 841.561080 (including connections establishing) 2023-07-14T18:34:26.597 INFO:teuthology.orchestra.run.smithi133.stdout:tps = 841.576252 (excluding connections establishing) 2023-07-14T18:34:26.605 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2023-07-14T18:34:26.884 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:26 smithi133 ceph-mon[126384]: pgmap v880: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 15 MiB/s wr, 1.31k op/s 2023-07-14T18:34:27.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:26 smithi161 ceph-mon[128993]: pgmap v880: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 15 MiB/s wr, 1.31k op/s 2023-07-14T18:34:27.093 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:26 smithi171 ceph-mon[129241]: pgmap v880: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 15 MiB/s wr, 1.31k op/s 2023-07-14T18:34:28.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:27 smithi161 ceph-mon[128993]: pgmap v881: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 14 MiB/s wr, 1.20k op/s 2023-07-14T18:34:28.092 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:27 smithi171 ceph-mon[129241]: pgmap v881: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 14 MiB/s wr, 1.20k op/s 2023-07-14T18:34:28.135 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:27 smithi133 ceph-mon[126384]: pgmap v881: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 14 MiB/s wr, 1.20k op/s 2023-07-14T18:34:30.327 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:30 smithi161 ceph-mon[128993]: pgmap v882: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 14 MiB/s wr, 1.20k op/s 2023-07-14T18:34:30.341 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:30 smithi171 ceph-mon[129241]: pgmap v882: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 14 MiB/s wr, 1.20k op/s 2023-07-14T18:34:30.389 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:30 smithi133 ceph-mon[126384]: pgmap v882: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 14 MiB/s wr, 1.20k op/s 2023-07-14T18:34:31.582 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:31 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:34:31.595 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:31 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:34:31.632 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:31 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:34:32.883 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:32 smithi133 ceph-mon[126384]: pgmap v883: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 11 MiB/s wr, 952 op/s 2023-07-14T18:34:32.883 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:32 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:34:32.883 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:32 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:34:32.883 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:32 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:34:32.883 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:32 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:34:32.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:32 smithi171 ceph-mon[129241]: pgmap v883: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 11 MiB/s wr, 952 op/s 2023-07-14T18:34:32.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:32 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:34:32.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:32 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:34:32.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:32 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:34:32.885 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:32 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:34:33.090 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:32 smithi161 ceph-mon[128993]: pgmap v883: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 11 MiB/s wr, 952 op/s 2023-07-14T18:34:33.090 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:32 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:34:33.091 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:32 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:34:33.091 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:32 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:34:33.091 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:32 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:34:33.885 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:33 smithi133 ceph-mon[126384]: Adjusting osd_memory_target on smithi161 to 196.2M 2023-07-14T18:34:33.885 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:33 smithi133 ceph-mon[126384]: Unable to set osd_memory_target on smithi161 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-14T18:34:34.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:33 smithi161 ceph-mon[128993]: Adjusting osd_memory_target on smithi161 to 196.2M 2023-07-14T18:34:34.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:33 smithi161 ceph-mon[128993]: Unable to set osd_memory_target on smithi161 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-14T18:34:34.091 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:33 smithi171 ceph-mon[129241]: Adjusting osd_memory_target on smithi161 to 196.2M 2023-07-14T18:34:34.091 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:33 smithi171 ceph-mon[129241]: Unable to set osd_memory_target on smithi161 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-14T18:34:35.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:34 smithi161 ceph-mon[128993]: pgmap v884: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 12 MiB/s wr, 1.05k op/s 2023-07-14T18:34:35.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:34 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:34:35.092 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:34 smithi171 ceph-mon[129241]: pgmap v884: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 12 MiB/s wr, 1.05k op/s 2023-07-14T18:34:35.093 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:34 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:34:35.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:34 smithi133 ceph-mon[126384]: pgmap v884: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 12 MiB/s wr, 1.05k op/s 2023-07-14T18:34:35.133 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:34 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:34:36.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:35 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:34:36.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:35 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:34:36.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:35 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:34:36.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:35 smithi161 ceph-mon[128993]: Adjusting osd_memory_target on smithi171 to 196.2M 2023-07-14T18:34:36.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:35 smithi161 ceph-mon[128993]: Unable to set osd_memory_target on smithi171 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-14T18:34:36.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:35 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:34:36.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:35 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:34:36.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:35 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:34:36.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:35 smithi161 ceph-mon[128993]: pgmap v885: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 1.01k op/s 2023-07-14T18:34:36.091 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:35 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:34:36.091 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:35 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:34:36.091 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:35 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:34:36.091 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:35 smithi171 ceph-mon[129241]: Adjusting osd_memory_target on smithi171 to 196.2M 2023-07-14T18:34:36.092 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:35 smithi171 ceph-mon[129241]: Unable to set osd_memory_target on smithi171 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-14T18:34:36.092 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:35 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:34:36.092 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:35 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:34:36.092 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:35 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:34:36.092 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:35 smithi171 ceph-mon[129241]: pgmap v885: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 1.01k op/s 2023-07-14T18:34:36.131 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:35 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:34:36.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:35 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:34:36.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:35 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T18:34:36.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:35 smithi133 ceph-mon[126384]: Adjusting osd_memory_target on smithi171 to 196.2M 2023-07-14T18:34:36.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:35 smithi133 ceph-mon[126384]: Unable to set osd_memory_target on smithi171 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-14T18:34:36.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:35 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:34:36.133 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:35 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:34:36.133 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:35 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:34:36.133 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:35 smithi133 ceph-mon[126384]: pgmap v885: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 1.01k op/s 2023-07-14T18:34:38.079 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:37 smithi161 ceph-mon[128993]: pgmap v886: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.4 MiB/s wr, 960 op/s 2023-07-14T18:34:38.093 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:37 smithi171 ceph-mon[129241]: pgmap v886: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.4 MiB/s wr, 960 op/s 2023-07-14T18:34:38.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:37 smithi133 ceph-mon[126384]: pgmap v886: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.4 MiB/s wr, 960 op/s 2023-07-14T18:34:40.326 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:40 smithi161 ceph-mon[128993]: pgmap v887: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.3 MiB/s wr, 879 op/s 2023-07-14T18:34:40.341 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:40 smithi171 ceph-mon[129241]: pgmap v887: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.3 MiB/s wr, 879 op/s 2023-07-14T18:34:40.381 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:40 smithi133 ceph-mon[126384]: pgmap v887: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.3 MiB/s wr, 879 op/s 2023-07-14T18:34:42.776 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-07-14T18:34:42.825 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:42 smithi161 ceph-mon[128993]: pgmap v888: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 7.5 MiB/s wr, 896 op/s 2023-07-14T18:34:42.841 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:42 smithi171 ceph-mon[129241]: pgmap v888: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 7.5 MiB/s wr, 896 op/s 2023-07-14T18:34:42.841 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:42 smithi133 ceph-mon[126384]: pgmap v888: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 7.5 MiB/s wr, 896 op/s 2023-07-14T18:34:42.944 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/: 2023-07-14T18:34:42.944 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:34:42.944 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 . 2023-07-14T18:34:42.944 INFO:teuthology.orchestra.run.smithi133.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 .. 2023-07-14T18:34:42.945 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 14 18:34 data 2023-07-14T18:34:42.945 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:42.945 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data: 2023-07-14T18:34:42.945 INFO:teuthology.orchestra.run.smithi133.stdout:total 31 2023-07-14T18:34:42.945 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 14 18:34 . 2023-07-14T18:34:42.945 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 .. 2023-07-14T18:34:42.945 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jul 14 18:18 base 2023-07-14T18:34:42.946 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Jul 14 18:16 current_logfiles 2023-07-14T18:34:42.946 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 59 Jul 14 18:16 global 2023-07-14T18:34:42.946 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 log 2023-07-14T18:34:42.946 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 pg_commit_ts 2023-07-14T18:34:42.946 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 pg_dynshmem 2023-07-14T18:34:42.946 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Jul 14 18:16 pg_hba.conf 2023-07-14T18:34:42.946 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Jul 14 18:16 pg_ident.conf 2023-07-14T18:34:42.946 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 14 18:34 pg_logical 2023-07-14T18:34:42.947 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jul 14 18:16 pg_multixact 2023-07-14T18:34:42.947 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 pg_notify 2023-07-14T18:34:42.947 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 pg_replslot 2023-07-14T18:34:42.947 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 pg_serial 2023-07-14T18:34:42.947 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 pg_snapshots 2023-07-14T18:34:42.947 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 14 18:34 pg_stat 2023-07-14T18:34:42.947 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:34 pg_stat_tmp 2023-07-14T18:34:42.948 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:34 pg_subtrans 2023-07-14T18:34:42.948 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 pg_tblspc 2023-07-14T18:34:42.948 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 pg_twophase 2023-07-14T18:34:42.948 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 14 18:16 PG_VERSION 2023-07-14T18:34:42.948 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 64 Jul 14 18:34 pg_wal 2023-07-14T18:34:42.948 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 pg_xact 2023-07-14T18:34:42.948 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Jul 14 18:16 postgresql.auto.conf 2023-07-14T18:34:42.948 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Jul 14 18:16 postgresql.conf 2023-07-14T18:34:42.949 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Jul 14 18:16 postmaster.opts 2023-07-14T18:34:42.949 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:42.949 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/base: 2023-07-14T18:34:42.949 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:34:42.949 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jul 14 18:18 . 2023-07-14T18:34:42.949 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 14 18:34 .. 2023-07-14T18:34:42.949 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jul 14 18:16 1 2023-07-14T18:34:42.950 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jul 14 18:16 12346 2023-07-14T18:34:42.950 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 315 Jul 14 18:19 12347 2023-07-14T18:34:42.950 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:19 pgsql_tmp 2023-07-14T18:34:42.950 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:42.950 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/base/1: 2023-07-14T18:34:42.950 INFO:teuthology.orchestra.run.smithi133.stdout:total 7193 2023-07-14T18:34:42.951 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jul 14 18:16 . 2023-07-14T18:34:42.951 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jul 14 18:18 .. 2023-07-14T18:34:42.951 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 112 2023-07-14T18:34:42.951 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 113 2023-07-14T18:34:42.951 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jul 14 18:16 12182 2023-07-14T18:34:42.951 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12182_fsm 2023-07-14T18:34:42.951 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12182_vm 2023-07-14T18:34:42.951 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12184 2023-07-14T18:34:42.952 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12186 2023-07-14T18:34:42.952 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12187 2023-07-14T18:34:42.952 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12187_fsm 2023-07-14T18:34:42.952 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12187_vm 2023-07-14T18:34:42.952 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12189 2023-07-14T18:34:42.952 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12191 2023-07-14T18:34:42.952 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12192 2023-07-14T18:34:42.953 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12192_fsm 2023-07-14T18:34:42.953 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12192_vm 2023-07-14T18:34:42.953 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12194 2023-07-14T18:34:42.953 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12196 2023-07-14T18:34:42.953 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12197 2023-07-14T18:34:42.953 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12197_fsm 2023-07-14T18:34:42.953 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12197_vm 2023-07-14T18:34:42.954 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12199 2023-07-14T18:34:42.954 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12201 2023-07-14T18:34:42.954 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12202 2023-07-14T18:34:42.954 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12202_fsm 2023-07-14T18:34:42.954 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12202_vm 2023-07-14T18:34:42.954 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12204 2023-07-14T18:34:42.954 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12206 2023-07-14T18:34:42.954 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12207 2023-07-14T18:34:42.955 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12207_fsm 2023-07-14T18:34:42.955 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12207_vm 2023-07-14T18:34:42.955 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12209 2023-07-14T18:34:42.955 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12211 2023-07-14T18:34:42.955 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12212 2023-07-14T18:34:42.955 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12214 2023-07-14T18:34:42.955 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12216 2023-07-14T18:34:42.955 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jul 14 18:16 1247 2023-07-14T18:34:42.956 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1247_fsm 2023-07-14T18:34:42.956 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1247_vm 2023-07-14T18:34:42.956 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jul 14 18:16 1249 2023-07-14T18:34:42.956 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1249_fsm 2023-07-14T18:34:42.956 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1249_vm 2023-07-14T18:34:42.956 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jul 14 18:16 1255 2023-07-14T18:34:42.956 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1255_fsm 2023-07-14T18:34:42.956 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1255_vm 2023-07-14T18:34:42.957 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jul 14 18:16 1259 2023-07-14T18:34:42.957 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1259_fsm 2023-07-14T18:34:42.957 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1259_vm 2023-07-14T18:34:42.957 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 1417 2023-07-14T18:34:42.957 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 1417_vm 2023-07-14T18:34:42.957 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 1418 2023-07-14T18:34:42.957 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 1418_vm 2023-07-14T18:34:42.958 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 174 2023-07-14T18:34:42.958 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 175 2023-07-14T18:34:42.958 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2187 2023-07-14T18:34:42.958 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2224 2023-07-14T18:34:42.958 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2224_vm 2023-07-14T18:34:42.958 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2328 2023-07-14T18:34:42.958 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2328_vm 2023-07-14T18:34:42.958 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2336 2023-07-14T18:34:42.959 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2336_vm 2023-07-14T18:34:42.959 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2337 2023-07-14T18:34:42.959 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2600 2023-07-14T18:34:42.959 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2600_fsm 2023-07-14T18:34:42.959 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2600_vm 2023-07-14T18:34:42.959 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2601 2023-07-14T18:34:42.959 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2601_fsm 2023-07-14T18:34:42.959 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2601_vm 2023-07-14T18:34:42.960 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 14 18:16 2602 2023-07-14T18:34:42.960 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2602_fsm 2023-07-14T18:34:42.960 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2602_vm 2023-07-14T18:34:42.960 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2603 2023-07-14T18:34:42.960 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2603_fsm 2023-07-14T18:34:42.960 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2603_vm 2023-07-14T18:34:42.961 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2604 2023-07-14T18:34:42.961 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2604_vm 2023-07-14T18:34:42.961 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2605 2023-07-14T18:34:42.961 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2605_fsm 2023-07-14T18:34:42.961 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2605_vm 2023-07-14T18:34:42.961 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2606 2023-07-14T18:34:42.961 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2606_fsm 2023-07-14T18:34:42.961 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2606_vm 2023-07-14T18:34:42.962 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2607 2023-07-14T18:34:42.962 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2607_fsm 2023-07-14T18:34:42.962 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2607_vm 2023-07-14T18:34:42.962 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Jul 14 18:16 2608 2023-07-14T18:34:42.962 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2608_fsm 2023-07-14T18:34:42.962 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2608_vm 2023-07-14T18:34:42.962 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jul 14 18:16 2609 2023-07-14T18:34:42.962 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2609_fsm 2023-07-14T18:34:42.963 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2609_vm 2023-07-14T18:34:42.963 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2610 2023-07-14T18:34:42.963 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2610_fsm 2023-07-14T18:34:42.963 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2610_vm 2023-07-14T18:34:42.963 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2611 2023-07-14T18:34:42.963 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2611_vm 2023-07-14T18:34:42.963 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2612 2023-07-14T18:34:42.963 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2612_fsm 2023-07-14T18:34:42.964 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2612_vm 2023-07-14T18:34:42.964 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2613 2023-07-14T18:34:42.964 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2613_vm 2023-07-14T18:34:42.964 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2615 2023-07-14T18:34:42.964 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2615_fsm 2023-07-14T18:34:42.964 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2615_vm 2023-07-14T18:34:42.964 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2616 2023-07-14T18:34:42.965 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2616_fsm 2023-07-14T18:34:42.965 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2616_vm 2023-07-14T18:34:42.965 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jul 14 18:16 2617 2023-07-14T18:34:42.965 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2617_fsm 2023-07-14T18:34:42.965 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2617_vm 2023-07-14T18:34:42.965 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jul 14 18:16 2618 2023-07-14T18:34:42.965 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2618_fsm 2023-07-14T18:34:42.965 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2618_vm 2023-07-14T18:34:42.966 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jul 14 18:16 2619 2023-07-14T18:34:42.966 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2619_fsm 2023-07-14T18:34:42.966 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2619_vm 2023-07-14T18:34:42.966 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2620 2023-07-14T18:34:42.966 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2620_vm 2023-07-14T18:34:42.966 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2650 2023-07-14T18:34:42.966 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2651 2023-07-14T18:34:42.966 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2652 2023-07-14T18:34:42.967 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 2653 2023-07-14T18:34:42.967 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 2654 2023-07-14T18:34:42.967 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2655 2023-07-14T18:34:42.967 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2656 2023-07-14T18:34:42.967 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2657 2023-07-14T18:34:42.967 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jul 14 18:16 2658 2023-07-14T18:34:42.967 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 14 18:16 2659 2023-07-14T18:34:42.967 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2660 2023-07-14T18:34:42.968 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2661 2023-07-14T18:34:42.968 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2662 2023-07-14T18:34:42.968 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 14 18:16 2663 2023-07-14T18:34:42.968 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2664 2023-07-14T18:34:42.968 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2665 2023-07-14T18:34:42.968 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2666 2023-07-14T18:34:42.968 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2667 2023-07-14T18:34:42.969 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2668 2023-07-14T18:34:42.969 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2669 2023-07-14T18:34:42.969 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2670 2023-07-14T18:34:42.969 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 Jul 14 18:16 2673 2023-07-14T18:34:42.969 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Jul 14 18:16 2674 2023-07-14T18:34:42.969 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jul 14 18:16 2675 2023-07-14T18:34:42.969 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2678 2023-07-14T18:34:42.969 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2679 2023-07-14T18:34:42.970 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2680 2023-07-14T18:34:42.970 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2681 2023-07-14T18:34:42.970 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2682 2023-07-14T18:34:42.970 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2683 2023-07-14T18:34:42.970 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2684 2023-07-14T18:34:42.970 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2685 2023-07-14T18:34:42.970 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2686 2023-07-14T18:34:42.971 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2687 2023-07-14T18:34:42.971 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 2688 2023-07-14T18:34:42.971 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 14 18:16 2689 2023-07-14T18:34:42.971 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 14 18:16 2690 2023-07-14T18:34:42.971 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jul 14 18:16 2691 2023-07-14T18:34:42.971 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2692 2023-07-14T18:34:42.972 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2693 2023-07-14T18:34:42.972 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2696 2023-07-14T18:34:42.972 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2699 2023-07-14T18:34:42.972 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2701 2023-07-14T18:34:42.972 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2702 2023-07-14T18:34:42.972 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2703 2023-07-14T18:34:42.972 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 2704 2023-07-14T18:34:42.973 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2753 2023-07-14T18:34:42.973 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2753_fsm 2023-07-14T18:34:42.973 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2753_vm 2023-07-14T18:34:42.973 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2754 2023-07-14T18:34:42.973 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2755 2023-07-14T18:34:42.973 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2756 2023-07-14T18:34:42.973 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2757 2023-07-14T18:34:42.973 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2830 2023-07-14T18:34:42.974 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2830_vm 2023-07-14T18:34:42.974 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2831 2023-07-14T18:34:42.974 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2832 2023-07-14T18:34:42.974 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2832_vm 2023-07-14T18:34:42.974 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2833 2023-07-14T18:34:42.974 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2834 2023-07-14T18:34:42.974 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2834_vm 2023-07-14T18:34:42.974 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2835 2023-07-14T18:34:42.975 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2836 2023-07-14T18:34:42.975 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2836_vm 2023-07-14T18:34:42.975 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2837 2023-07-14T18:34:42.975 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jul 14 18:16 2838 2023-07-14T18:34:42.975 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2838_fsm 2023-07-14T18:34:42.975 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2838_vm 2023-07-14T18:34:42.975 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2839 2023-07-14T18:34:42.975 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2840 2023-07-14T18:34:42.976 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2840_fsm 2023-07-14T18:34:42.976 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2840_vm 2023-07-14T18:34:42.976 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2841 2023-07-14T18:34:42.976 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2995 2023-07-14T18:34:42.976 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2995_vm 2023-07-14T18:34:42.976 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2996 2023-07-14T18:34:42.976 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3079 2023-07-14T18:34:42.977 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3079_fsm 2023-07-14T18:34:42.977 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3079_vm 2023-07-14T18:34:42.977 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3080 2023-07-14T18:34:42.977 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3081 2023-07-14T18:34:42.977 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3085 2023-07-14T18:34:42.977 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3118 2023-07-14T18:34:42.977 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3118_vm 2023-07-14T18:34:42.978 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3119 2023-07-14T18:34:42.978 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3164 2023-07-14T18:34:42.978 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3256 2023-07-14T18:34:42.978 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3256_vm 2023-07-14T18:34:42.978 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3257 2023-07-14T18:34:42.978 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3258 2023-07-14T18:34:42.978 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3350 2023-07-14T18:34:42.978 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3350_vm 2023-07-14T18:34:42.979 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3351 2023-07-14T18:34:42.979 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3379 2023-07-14T18:34:42.979 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3380 2023-07-14T18:34:42.979 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3381 2023-07-14T18:34:42.979 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3381_vm 2023-07-14T18:34:42.980 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3394 2023-07-14T18:34:42.980 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3394_fsm 2023-07-14T18:34:42.980 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3394_vm 2023-07-14T18:34:42.980 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3395 2023-07-14T18:34:42.980 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3439 2023-07-14T18:34:42.980 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3439_vm 2023-07-14T18:34:42.980 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3440 2023-07-14T18:34:42.981 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 3455 2023-07-14T18:34:42.981 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3456 2023-07-14T18:34:42.981 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3456_fsm 2023-07-14T18:34:42.981 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3456_vm 2023-07-14T18:34:42.981 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3466 2023-07-14T18:34:42.981 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3466_vm 2023-07-14T18:34:42.982 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3467 2023-07-14T18:34:42.982 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3468 2023-07-14T18:34:42.982 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3501 2023-07-14T18:34:42.982 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3501_vm 2023-07-14T18:34:42.982 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3502 2023-07-14T18:34:42.982 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3503 2023-07-14T18:34:42.982 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3534 2023-07-14T18:34:42.982 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3541 2023-07-14T18:34:42.983 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3541_fsm 2023-07-14T18:34:42.983 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3541_vm 2023-07-14T18:34:42.983 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3542 2023-07-14T18:34:42.983 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3574 2023-07-14T18:34:42.983 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3575 2023-07-14T18:34:42.983 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3576 2023-07-14T18:34:42.983 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3576_vm 2023-07-14T18:34:42.983 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3596 2023-07-14T18:34:42.984 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3596_vm 2023-07-14T18:34:42.984 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3597 2023-07-14T18:34:42.984 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3598 2023-07-14T18:34:42.984 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3598_vm 2023-07-14T18:34:42.984 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3599 2023-07-14T18:34:42.984 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3600 2023-07-14T18:34:42.984 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3600_fsm 2023-07-14T18:34:42.985 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3600_vm 2023-07-14T18:34:42.985 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3601 2023-07-14T18:34:42.985 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3601_fsm 2023-07-14T18:34:42.985 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3601_vm 2023-07-14T18:34:42.985 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3602 2023-07-14T18:34:42.985 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3602_fsm 2023-07-14T18:34:42.985 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3602_vm 2023-07-14T18:34:42.985 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3603 2023-07-14T18:34:42.986 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3603_fsm 2023-07-14T18:34:42.986 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3603_vm 2023-07-14T18:34:42.986 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3604 2023-07-14T18:34:42.987 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3605 2023-07-14T18:34:42.987 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3606 2023-07-14T18:34:42.987 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3607 2023-07-14T18:34:42.987 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3608 2023-07-14T18:34:42.987 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 3609 2023-07-14T18:34:42.987 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3712 2023-07-14T18:34:42.987 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3764 2023-07-14T18:34:42.988 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3764_fsm 2023-07-14T18:34:42.988 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3764_vm 2023-07-14T18:34:42.988 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3766 2023-07-14T18:34:42.988 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3767 2023-07-14T18:34:42.988 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3997 2023-07-14T18:34:42.988 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 5002 2023-07-14T18:34:42.988 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 548 2023-07-14T18:34:42.989 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 549 2023-07-14T18:34:42.989 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6102 2023-07-14T18:34:42.989 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6102_vm 2023-07-14T18:34:42.989 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6104 2023-07-14T18:34:42.989 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6104_vm 2023-07-14T18:34:42.989 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6106 2023-07-14T18:34:42.989 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6106_vm 2023-07-14T18:34:42.989 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6110 2023-07-14T18:34:42.990 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6111 2023-07-14T18:34:42.990 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6112 2023-07-14T18:34:42.990 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6113 2023-07-14T18:34:42.990 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6117 2023-07-14T18:34:42.990 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 826 2023-07-14T18:34:42.990 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 826_vm 2023-07-14T18:34:42.990 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 827 2023-07-14T18:34:42.991 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 828 2023-07-14T18:34:42.991 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jul 14 18:16 pg_filenode.map 2023-07-14T18:34:42.991 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 14 18:16 PG_VERSION 2023-07-14T18:34:42.991 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:42.991 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-07-14T18:34:42.991 INFO:teuthology.orchestra.run.smithi133.stdout:total 7193 2023-07-14T18:34:42.991 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jul 14 18:16 . 2023-07-14T18:34:42.991 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jul 14 18:18 .. 2023-07-14T18:34:42.992 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 112 2023-07-14T18:34:42.992 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 113 2023-07-14T18:34:42.992 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jul 14 18:16 12182 2023-07-14T18:34:42.992 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12182_fsm 2023-07-14T18:34:42.992 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12182_vm 2023-07-14T18:34:42.993 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12184 2023-07-14T18:34:42.993 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12186 2023-07-14T18:34:42.993 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12187 2023-07-14T18:34:42.993 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12187_fsm 2023-07-14T18:34:42.993 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12187_vm 2023-07-14T18:34:42.993 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12189 2023-07-14T18:34:42.993 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12191 2023-07-14T18:34:42.994 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12192 2023-07-14T18:34:42.994 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12192_fsm 2023-07-14T18:34:42.994 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12192_vm 2023-07-14T18:34:42.994 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12194 2023-07-14T18:34:42.994 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12196 2023-07-14T18:34:42.994 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12197 2023-07-14T18:34:42.994 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12197_fsm 2023-07-14T18:34:42.994 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12197_vm 2023-07-14T18:34:42.995 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12199 2023-07-14T18:34:42.995 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12201 2023-07-14T18:34:42.995 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12202 2023-07-14T18:34:42.995 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12202_fsm 2023-07-14T18:34:42.995 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12202_vm 2023-07-14T18:34:42.995 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12204 2023-07-14T18:34:42.995 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12206 2023-07-14T18:34:42.995 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12207 2023-07-14T18:34:42.996 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12207_fsm 2023-07-14T18:34:42.996 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12207_vm 2023-07-14T18:34:42.996 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12209 2023-07-14T18:34:42.996 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12211 2023-07-14T18:34:42.996 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12212 2023-07-14T18:34:42.996 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12214 2023-07-14T18:34:42.996 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12216 2023-07-14T18:34:42.996 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jul 14 18:16 1247 2023-07-14T18:34:42.997 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1247_fsm 2023-07-14T18:34:42.997 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1247_vm 2023-07-14T18:34:42.997 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jul 14 18:16 1249 2023-07-14T18:34:42.997 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1249_fsm 2023-07-14T18:34:42.997 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1249_vm 2023-07-14T18:34:42.997 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jul 14 18:16 1255 2023-07-14T18:34:42.997 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1255_fsm 2023-07-14T18:34:42.997 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1255_vm 2023-07-14T18:34:42.998 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jul 14 18:16 1259 2023-07-14T18:34:42.998 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1259_fsm 2023-07-14T18:34:42.998 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1259_vm 2023-07-14T18:34:42.998 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 1417 2023-07-14T18:34:42.998 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 1417_vm 2023-07-14T18:34:42.998 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 1418 2023-07-14T18:34:42.998 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 1418_vm 2023-07-14T18:34:42.999 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 174 2023-07-14T18:34:42.999 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 175 2023-07-14T18:34:42.999 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2187 2023-07-14T18:34:42.999 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2224 2023-07-14T18:34:42.999 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2224_vm 2023-07-14T18:34:42.999 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2328 2023-07-14T18:34:42.999 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2328_vm 2023-07-14T18:34:42.999 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2336 2023-07-14T18:34:43.000 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2336_vm 2023-07-14T18:34:43.000 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2337 2023-07-14T18:34:43.000 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2600 2023-07-14T18:34:43.000 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2600_fsm 2023-07-14T18:34:43.000 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2600_vm 2023-07-14T18:34:43.000 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2601 2023-07-14T18:34:43.000 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2601_fsm 2023-07-14T18:34:43.000 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2601_vm 2023-07-14T18:34:43.001 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 14 18:16 2602 2023-07-14T18:34:43.001 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2602_fsm 2023-07-14T18:34:43.001 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2602_vm 2023-07-14T18:34:43.001 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2603 2023-07-14T18:34:43.001 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2603_fsm 2023-07-14T18:34:43.001 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2603_vm 2023-07-14T18:34:43.001 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2604 2023-07-14T18:34:43.001 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2604_vm 2023-07-14T18:34:43.002 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2605 2023-07-14T18:34:43.002 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2605_fsm 2023-07-14T18:34:43.002 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2605_vm 2023-07-14T18:34:43.002 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2606 2023-07-14T18:34:43.002 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2606_fsm 2023-07-14T18:34:43.002 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2606_vm 2023-07-14T18:34:43.003 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2607 2023-07-14T18:34:43.003 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2607_fsm 2023-07-14T18:34:43.003 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2607_vm 2023-07-14T18:34:43.003 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Jul 14 18:16 2608 2023-07-14T18:34:43.003 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2608_fsm 2023-07-14T18:34:43.004 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2608_vm 2023-07-14T18:34:43.004 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jul 14 18:16 2609 2023-07-14T18:34:43.004 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2609_fsm 2023-07-14T18:34:43.004 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2609_vm 2023-07-14T18:34:43.004 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2610 2023-07-14T18:34:43.004 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2610_fsm 2023-07-14T18:34:43.004 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2610_vm 2023-07-14T18:34:43.004 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2611 2023-07-14T18:34:43.005 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2611_vm 2023-07-14T18:34:43.005 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2612 2023-07-14T18:34:43.005 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2612_fsm 2023-07-14T18:34:43.005 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2612_vm 2023-07-14T18:34:43.005 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2613 2023-07-14T18:34:43.005 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2613_vm 2023-07-14T18:34:43.005 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2615 2023-07-14T18:34:43.005 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2615_fsm 2023-07-14T18:34:43.006 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2615_vm 2023-07-14T18:34:43.006 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2616 2023-07-14T18:34:43.006 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2616_fsm 2023-07-14T18:34:43.006 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2616_vm 2023-07-14T18:34:43.006 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jul 14 18:16 2617 2023-07-14T18:34:43.006 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2617_fsm 2023-07-14T18:34:43.006 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2617_vm 2023-07-14T18:34:43.006 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jul 14 18:16 2618 2023-07-14T18:34:43.007 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2618_fsm 2023-07-14T18:34:43.007 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2618_vm 2023-07-14T18:34:43.007 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jul 14 18:16 2619 2023-07-14T18:34:43.007 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2619_fsm 2023-07-14T18:34:43.007 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2619_vm 2023-07-14T18:34:43.007 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2620 2023-07-14T18:34:43.007 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2620_vm 2023-07-14T18:34:43.007 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2650 2023-07-14T18:34:43.008 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2651 2023-07-14T18:34:43.008 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2652 2023-07-14T18:34:43.008 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 2653 2023-07-14T18:34:43.008 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 2654 2023-07-14T18:34:43.008 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2655 2023-07-14T18:34:43.008 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2656 2023-07-14T18:34:43.008 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2657 2023-07-14T18:34:43.008 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jul 14 18:16 2658 2023-07-14T18:34:43.009 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 14 18:16 2659 2023-07-14T18:34:43.009 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2660 2023-07-14T18:34:43.009 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2661 2023-07-14T18:34:43.009 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2662 2023-07-14T18:34:43.009 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 14 18:16 2663 2023-07-14T18:34:43.009 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2664 2023-07-14T18:34:43.009 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2665 2023-07-14T18:34:43.009 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2666 2023-07-14T18:34:43.010 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2667 2023-07-14T18:34:43.010 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2668 2023-07-14T18:34:43.010 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2669 2023-07-14T18:34:43.010 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2670 2023-07-14T18:34:43.010 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 Jul 14 18:16 2673 2023-07-14T18:34:43.010 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Jul 14 18:16 2674 2023-07-14T18:34:43.010 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jul 14 18:16 2675 2023-07-14T18:34:43.011 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2678 2023-07-14T18:34:43.011 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2679 2023-07-14T18:34:43.011 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2680 2023-07-14T18:34:43.011 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2681 2023-07-14T18:34:43.011 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2682 2023-07-14T18:34:43.011 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2683 2023-07-14T18:34:43.011 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2684 2023-07-14T18:34:43.011 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2685 2023-07-14T18:34:43.012 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2686 2023-07-14T18:34:43.012 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2687 2023-07-14T18:34:43.012 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 2688 2023-07-14T18:34:43.012 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 14 18:16 2689 2023-07-14T18:34:43.012 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 14 18:16 2690 2023-07-14T18:34:43.012 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jul 14 18:16 2691 2023-07-14T18:34:43.012 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2692 2023-07-14T18:34:43.012 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2693 2023-07-14T18:34:43.013 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2696 2023-07-14T18:34:43.013 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2699 2023-07-14T18:34:43.013 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2701 2023-07-14T18:34:43.013 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2702 2023-07-14T18:34:43.013 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2703 2023-07-14T18:34:43.013 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 2704 2023-07-14T18:34:43.013 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2753 2023-07-14T18:34:43.014 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2753_fsm 2023-07-14T18:34:43.014 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2753_vm 2023-07-14T18:34:43.014 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2754 2023-07-14T18:34:43.014 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2755 2023-07-14T18:34:43.014 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2756 2023-07-14T18:34:43.014 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2757 2023-07-14T18:34:43.014 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2830 2023-07-14T18:34:43.015 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2830_vm 2023-07-14T18:34:43.015 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2831 2023-07-14T18:34:43.015 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2832 2023-07-14T18:34:43.015 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2832_vm 2023-07-14T18:34:43.015 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2833 2023-07-14T18:34:43.015 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2834 2023-07-14T18:34:43.015 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2834_vm 2023-07-14T18:34:43.015 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2835 2023-07-14T18:34:43.016 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2836 2023-07-14T18:34:43.016 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2836_vm 2023-07-14T18:34:43.016 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2837 2023-07-14T18:34:43.016 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jul 14 18:16 2838 2023-07-14T18:34:43.016 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2838_fsm 2023-07-14T18:34:43.016 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2838_vm 2023-07-14T18:34:43.016 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2839 2023-07-14T18:34:43.017 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2840 2023-07-14T18:34:43.017 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2840_fsm 2023-07-14T18:34:43.017 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2840_vm 2023-07-14T18:34:43.017 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2841 2023-07-14T18:34:43.017 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2995 2023-07-14T18:34:43.017 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2995_vm 2023-07-14T18:34:43.017 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2996 2023-07-14T18:34:43.017 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3079 2023-07-14T18:34:43.018 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3079_fsm 2023-07-14T18:34:43.018 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3079_vm 2023-07-14T18:34:43.018 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3080 2023-07-14T18:34:43.018 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3081 2023-07-14T18:34:43.018 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3085 2023-07-14T18:34:43.018 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3118 2023-07-14T18:34:43.018 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3118_vm 2023-07-14T18:34:43.018 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3119 2023-07-14T18:34:43.019 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3164 2023-07-14T18:34:43.019 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3256 2023-07-14T18:34:43.019 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3256_vm 2023-07-14T18:34:43.019 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3257 2023-07-14T18:34:43.019 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3258 2023-07-14T18:34:43.019 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3350 2023-07-14T18:34:43.019 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3350_vm 2023-07-14T18:34:43.019 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3351 2023-07-14T18:34:43.020 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3379 2023-07-14T18:34:43.020 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3380 2023-07-14T18:34:43.020 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3381 2023-07-14T18:34:43.020 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3381_vm 2023-07-14T18:34:43.020 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3394 2023-07-14T18:34:43.020 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3394_fsm 2023-07-14T18:34:43.020 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3394_vm 2023-07-14T18:34:43.020 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3395 2023-07-14T18:34:43.021 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3439 2023-07-14T18:34:43.021 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3439_vm 2023-07-14T18:34:43.021 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3440 2023-07-14T18:34:43.021 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 3455 2023-07-14T18:34:43.021 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3456 2023-07-14T18:34:43.021 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3456_fsm 2023-07-14T18:34:43.021 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3456_vm 2023-07-14T18:34:43.021 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3466 2023-07-14T18:34:43.022 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3466_vm 2023-07-14T18:34:43.022 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3467 2023-07-14T18:34:43.022 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3468 2023-07-14T18:34:43.022 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3501 2023-07-14T18:34:43.022 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3501_vm 2023-07-14T18:34:43.022 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3502 2023-07-14T18:34:43.022 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3503 2023-07-14T18:34:43.022 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3534 2023-07-14T18:34:43.023 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3541 2023-07-14T18:34:43.023 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3541_fsm 2023-07-14T18:34:43.023 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3541_vm 2023-07-14T18:34:43.023 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3542 2023-07-14T18:34:43.023 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3574 2023-07-14T18:34:43.023 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3575 2023-07-14T18:34:43.023 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3576 2023-07-14T18:34:43.023 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3576_vm 2023-07-14T18:34:43.024 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3596 2023-07-14T18:34:43.024 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3596_vm 2023-07-14T18:34:43.024 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3597 2023-07-14T18:34:43.024 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3598 2023-07-14T18:34:43.024 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3598_vm 2023-07-14T18:34:43.025 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3599 2023-07-14T18:34:43.025 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3600 2023-07-14T18:34:43.025 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3600_fsm 2023-07-14T18:34:43.025 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3600_vm 2023-07-14T18:34:43.025 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3601 2023-07-14T18:34:43.025 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3601_fsm 2023-07-14T18:34:43.025 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3601_vm 2023-07-14T18:34:43.025 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3602 2023-07-14T18:34:43.026 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3602_fsm 2023-07-14T18:34:43.026 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3602_vm 2023-07-14T18:34:43.026 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3603 2023-07-14T18:34:43.026 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3603_fsm 2023-07-14T18:34:43.026 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3603_vm 2023-07-14T18:34:43.026 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3604 2023-07-14T18:34:43.026 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3605 2023-07-14T18:34:43.026 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3606 2023-07-14T18:34:43.027 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3607 2023-07-14T18:34:43.027 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3608 2023-07-14T18:34:43.029 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 3609 2023-07-14T18:34:43.029 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3712 2023-07-14T18:34:43.029 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3764 2023-07-14T18:34:43.029 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3764_fsm 2023-07-14T18:34:43.029 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3764_vm 2023-07-14T18:34:43.029 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3766 2023-07-14T18:34:43.030 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3767 2023-07-14T18:34:43.030 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3997 2023-07-14T18:34:43.030 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 5002 2023-07-14T18:34:43.030 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 548 2023-07-14T18:34:43.030 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 549 2023-07-14T18:34:43.030 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6102 2023-07-14T18:34:43.030 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6102_vm 2023-07-14T18:34:43.030 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6104 2023-07-14T18:34:43.031 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6104_vm 2023-07-14T18:34:43.031 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6106 2023-07-14T18:34:43.031 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6106_vm 2023-07-14T18:34:43.031 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6110 2023-07-14T18:34:43.031 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6111 2023-07-14T18:34:43.031 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6112 2023-07-14T18:34:43.031 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6113 2023-07-14T18:34:43.031 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6117 2023-07-14T18:34:43.032 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 826 2023-07-14T18:34:43.032 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 826_vm 2023-07-14T18:34:43.032 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 827 2023-07-14T18:34:43.032 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 828 2023-07-14T18:34:43.032 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jul 14 18:16 pg_filenode.map 2023-07-14T18:34:43.032 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 14 18:16 PG_VERSION 2023-07-14T18:34:43.032 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:43.032 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-07-14T18:34:43.033 INFO:teuthology.orchestra.run.smithi133.stdout:total 7774256 2023-07-14T18:34:43.033 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 315 Jul 14 18:19 . 2023-07-14T18:34:43.033 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jul 14 18:18 .. 2023-07-14T18:34:43.033 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 112 2023-07-14T18:34:43.033 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 113 2023-07-14T18:34:43.033 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jul 14 18:16 12182 2023-07-14T18:34:43.033 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12182_fsm 2023-07-14T18:34:43.033 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12182_vm 2023-07-14T18:34:43.034 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12184 2023-07-14T18:34:43.034 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12186 2023-07-14T18:34:43.034 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12187 2023-07-14T18:34:43.034 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12187_fsm 2023-07-14T18:34:43.034 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12187_vm 2023-07-14T18:34:43.034 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12189 2023-07-14T18:34:43.034 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12191 2023-07-14T18:34:43.035 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12192 2023-07-14T18:34:43.035 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12192_fsm 2023-07-14T18:34:43.035 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12192_vm 2023-07-14T18:34:43.035 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12194 2023-07-14T18:34:43.035 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12196 2023-07-14T18:34:43.035 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12197 2023-07-14T18:34:43.036 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12197_fsm 2023-07-14T18:34:43.036 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12197_vm 2023-07-14T18:34:43.036 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12199 2023-07-14T18:34:43.036 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12201 2023-07-14T18:34:43.036 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12202 2023-07-14T18:34:43.036 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12202_fsm 2023-07-14T18:34:43.036 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12202_vm 2023-07-14T18:34:43.036 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12204 2023-07-14T18:34:43.037 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12206 2023-07-14T18:34:43.037 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12207 2023-07-14T18:34:43.037 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 12207_fsm 2023-07-14T18:34:43.037 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12207_vm 2023-07-14T18:34:43.037 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12209 2023-07-14T18:34:43.037 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12211 2023-07-14T18:34:43.037 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12212 2023-07-14T18:34:43.037 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 12214 2023-07-14T18:34:43.038 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 12216 2023-07-14T18:34:43.038 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jul 14 18:16 1247 2023-07-14T18:34:43.038 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1247_fsm 2023-07-14T18:34:43.038 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1247_vm 2023-07-14T18:34:43.038 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jul 14 18:19 1249 2023-07-14T18:34:43.038 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1249_fsm 2023-07-14T18:34:43.038 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1249_vm 2023-07-14T18:34:43.038 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jul 14 18:16 1255 2023-07-14T18:34:43.039 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1255_fsm 2023-07-14T18:34:43.039 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1255_vm 2023-07-14T18:34:43.039 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jul 14 18:34 1259 2023-07-14T18:34:43.039 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1259_fsm 2023-07-14T18:34:43.039 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1259_vm 2023-07-14T18:34:43.039 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 1417 2023-07-14T18:34:43.039 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 1417_vm 2023-07-14T18:34:43.039 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 1418 2023-07-14T18:34:43.040 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 1418_vm 2023-07-14T18:34:43.040 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4063232 Jul 14 18:34 16387 2023-07-14T18:34:43.040 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:34 16387_fsm 2023-07-14T18:34:43.040 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:34 16387_vm 2023-07-14T18:34:43.040 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Jul 14 18:34 16393 2023-07-14T18:34:43.040 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:34 16393_fsm 2023-07-14T18:34:43.040 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:34 16393_vm 2023-07-14T18:34:43.040 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jul 14 18:34 16396 2023-07-14T18:34:43.041 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jul 14 18:34 16396.1 2023-07-14T18:34:43.041 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jul 14 18:34 16396.2 2023-07-14T18:34:43.041 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jul 14 18:34 16396.3 2023-07-14T18:34:43.041 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jul 14 18:34 16396.4 2023-07-14T18:34:43.041 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jul 14 18:34 16396.5 2023-07-14T18:34:43.041 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 339623936 Jul 14 18:34 16396.6 2023-07-14T18:34:43.041 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1687552 Jul 14 18:34 16396_fsm 2023-07-14T18:34:43.041 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 212992 Jul 14 18:34 16396_vm 2023-07-14T18:34:43.042 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 14 18:34 16397 2023-07-14T18:34:43.042 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 237568 Jul 14 18:34 16399 2023-07-14T18:34:43.042 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jul 14 18:34 16401 2023-07-14T18:34:43.042 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49373184 Jul 14 18:34 16401.1 2023-07-14T18:34:43.042 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 41451520 Jul 14 18:34 16403 2023-07-14T18:34:43.042 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:34 16403_fsm 2023-07-14T18:34:43.042 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 174 2023-07-14T18:34:43.042 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 175 2023-07-14T18:34:43.043 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2187 2023-07-14T18:34:43.043 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2224 2023-07-14T18:34:43.043 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2224_vm 2023-07-14T18:34:43.043 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2328 2023-07-14T18:34:43.043 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2328_vm 2023-07-14T18:34:43.043 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2336 2023-07-14T18:34:43.043 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2336_vm 2023-07-14T18:34:43.043 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2337 2023-07-14T18:34:43.044 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2600 2023-07-14T18:34:43.044 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2600_fsm 2023-07-14T18:34:43.044 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2600_vm 2023-07-14T18:34:43.044 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2601 2023-07-14T18:34:43.044 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2601_fsm 2023-07-14T18:34:43.044 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2601_vm 2023-07-14T18:34:43.044 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 14 18:16 2602 2023-07-14T18:34:43.044 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2602_fsm 2023-07-14T18:34:43.045 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2602_vm 2023-07-14T18:34:43.045 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2603 2023-07-14T18:34:43.045 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2603_fsm 2023-07-14T18:34:43.045 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2603_vm 2023-07-14T18:34:43.045 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2604 2023-07-14T18:34:43.045 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2604_vm 2023-07-14T18:34:43.046 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2605 2023-07-14T18:34:43.046 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2605_fsm 2023-07-14T18:34:43.046 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2605_vm 2023-07-14T18:34:43.046 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:19 2606 2023-07-14T18:34:43.046 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2606_fsm 2023-07-14T18:34:43.046 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:19 2606_vm 2023-07-14T18:34:43.046 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2607 2023-07-14T18:34:43.047 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2607_fsm 2023-07-14T18:34:43.047 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2607_vm 2023-07-14T18:34:43.047 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 450560 Jul 14 18:19 2608 2023-07-14T18:34:43.047 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:19 2608_fsm 2023-07-14T18:34:43.047 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:19 2608_vm 2023-07-14T18:34:43.047 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jul 14 18:16 2609 2023-07-14T18:34:43.047 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2609_fsm 2023-07-14T18:34:43.047 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2609_vm 2023-07-14T18:34:43.048 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:19 2610 2023-07-14T18:34:43.048 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:19 2610_fsm 2023-07-14T18:34:43.048 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:19 2610_vm 2023-07-14T18:34:43.048 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2611 2023-07-14T18:34:43.048 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2611_vm 2023-07-14T18:34:43.048 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2612 2023-07-14T18:34:43.048 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2612_fsm 2023-07-14T18:34:43.049 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2612_vm 2023-07-14T18:34:43.049 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2613 2023-07-14T18:34:43.049 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2613_vm 2023-07-14T18:34:43.049 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2615 2023-07-14T18:34:43.049 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2615_fsm 2023-07-14T18:34:43.049 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2615_vm 2023-07-14T18:34:43.050 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2616 2023-07-14T18:34:43.050 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2616_fsm 2023-07-14T18:34:43.050 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2616_vm 2023-07-14T18:34:43.050 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jul 14 18:16 2617 2023-07-14T18:34:43.050 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2617_fsm 2023-07-14T18:34:43.050 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2617_vm 2023-07-14T18:34:43.050 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jul 14 18:16 2618 2023-07-14T18:34:43.050 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2618_fsm 2023-07-14T18:34:43.051 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2618_vm 2023-07-14T18:34:43.051 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 155648 Jul 14 18:34 2619 2023-07-14T18:34:43.051 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:22 2619_fsm 2023-07-14T18:34:43.051 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:19 2619_vm 2023-07-14T18:34:43.051 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2620 2023-07-14T18:34:43.051 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2620_vm 2023-07-14T18:34:43.051 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2650 2023-07-14T18:34:43.051 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2651 2023-07-14T18:34:43.052 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2652 2023-07-14T18:34:43.052 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 2653 2023-07-14T18:34:43.052 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 2654 2023-07-14T18:34:43.052 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2655 2023-07-14T18:34:43.052 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2656 2023-07-14T18:34:43.052 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2657 2023-07-14T18:34:43.052 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Jul 14 18:19 2658 2023-07-14T18:34:43.053 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 14 18:19 2659 2023-07-14T18:34:43.053 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2660 2023-07-14T18:34:43.053 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2661 2023-07-14T18:34:43.053 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:19 2662 2023-07-14T18:34:43.053 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 14 18:19 2663 2023-07-14T18:34:43.053 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:19 2664 2023-07-14T18:34:43.053 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:19 2665 2023-07-14T18:34:43.053 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:19 2666 2023-07-14T18:34:43.054 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:19 2667 2023-07-14T18:34:43.054 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2668 2023-07-14T18:34:43.054 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2669 2023-07-14T18:34:43.054 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2670 2023-07-14T18:34:43.054 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 Jul 14 18:19 2673 2023-07-14T18:34:43.054 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Jul 14 18:19 2674 2023-07-14T18:34:43.054 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jul 14 18:16 2675 2023-07-14T18:34:43.054 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:19 2678 2023-07-14T18:34:43.055 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:19 2679 2023-07-14T18:34:43.055 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2680 2023-07-14T18:34:43.055 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2681 2023-07-14T18:34:43.055 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2682 2023-07-14T18:34:43.055 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2683 2023-07-14T18:34:43.055 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2684 2023-07-14T18:34:43.055 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2685 2023-07-14T18:34:43.055 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2686 2023-07-14T18:34:43.056 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2687 2023-07-14T18:34:43.056 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 2688 2023-07-14T18:34:43.056 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 14 18:16 2689 2023-07-14T18:34:43.056 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 14 18:16 2690 2023-07-14T18:34:43.056 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jul 14 18:16 2691 2023-07-14T18:34:43.057 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2692 2023-07-14T18:34:43.057 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2693 2023-07-14T18:34:43.057 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:23 2696 2023-07-14T18:34:43.057 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2699 2023-07-14T18:34:43.057 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2701 2023-07-14T18:34:43.057 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2702 2023-07-14T18:34:43.057 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2703 2023-07-14T18:34:43.057 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:16 2704 2023-07-14T18:34:43.058 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2753 2023-07-14T18:34:43.058 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2753_fsm 2023-07-14T18:34:43.058 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2753_vm 2023-07-14T18:34:43.058 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2754 2023-07-14T18:34:43.058 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2755 2023-07-14T18:34:43.058 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2756 2023-07-14T18:34:43.058 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 2757 2023-07-14T18:34:43.058 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2830 2023-07-14T18:34:43.059 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2830_vm 2023-07-14T18:34:43.059 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2831 2023-07-14T18:34:43.059 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2832 2023-07-14T18:34:43.059 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2832_vm 2023-07-14T18:34:43.059 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2833 2023-07-14T18:34:43.059 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2834 2023-07-14T18:34:43.059 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2834_vm 2023-07-14T18:34:43.059 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2835 2023-07-14T18:34:43.060 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2836 2023-07-14T18:34:43.060 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2836_vm 2023-07-14T18:34:43.060 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2837 2023-07-14T18:34:43.060 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jul 14 18:16 2838 2023-07-14T18:34:43.060 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2838_fsm 2023-07-14T18:34:43.060 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2838_vm 2023-07-14T18:34:43.060 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2839 2023-07-14T18:34:43.060 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2840 2023-07-14T18:34:43.061 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2840_fsm 2023-07-14T18:34:43.061 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2840_vm 2023-07-14T18:34:43.061 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2841 2023-07-14T18:34:43.061 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2995 2023-07-14T18:34:43.061 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2995_vm 2023-07-14T18:34:43.061 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2996 2023-07-14T18:34:43.061 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3079 2023-07-14T18:34:43.061 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3079_fsm 2023-07-14T18:34:43.062 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3079_vm 2023-07-14T18:34:43.062 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3080 2023-07-14T18:34:43.062 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3081 2023-07-14T18:34:43.062 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3085 2023-07-14T18:34:43.062 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3118 2023-07-14T18:34:43.062 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3118_vm 2023-07-14T18:34:43.062 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3119 2023-07-14T18:34:43.063 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3164 2023-07-14T18:34:43.063 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3256 2023-07-14T18:34:43.063 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3256_vm 2023-07-14T18:34:43.063 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3257 2023-07-14T18:34:43.063 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3258 2023-07-14T18:34:43.063 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3350 2023-07-14T18:34:43.063 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3350_vm 2023-07-14T18:34:43.063 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3351 2023-07-14T18:34:43.063 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3379 2023-07-14T18:34:43.064 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3380 2023-07-14T18:34:43.064 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3381 2023-07-14T18:34:43.064 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3381_vm 2023-07-14T18:34:43.064 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3394 2023-07-14T18:34:43.064 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3394_fsm 2023-07-14T18:34:43.064 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3394_vm 2023-07-14T18:34:43.064 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3395 2023-07-14T18:34:43.065 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3439 2023-07-14T18:34:43.065 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3439_vm 2023-07-14T18:34:43.065 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3440 2023-07-14T18:34:43.065 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 14 18:19 3455 2023-07-14T18:34:43.065 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3456 2023-07-14T18:34:43.065 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3456_fsm 2023-07-14T18:34:43.065 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3456_vm 2023-07-14T18:34:43.065 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3466 2023-07-14T18:34:43.066 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3466_vm 2023-07-14T18:34:43.066 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3467 2023-07-14T18:34:43.066 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3468 2023-07-14T18:34:43.066 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3501 2023-07-14T18:34:43.066 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3501_vm 2023-07-14T18:34:43.066 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3502 2023-07-14T18:34:43.066 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3503 2023-07-14T18:34:43.067 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3534 2023-07-14T18:34:43.067 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3541 2023-07-14T18:34:43.067 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3541_fsm 2023-07-14T18:34:43.067 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3541_vm 2023-07-14T18:34:43.067 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3542 2023-07-14T18:34:43.067 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3574 2023-07-14T18:34:43.067 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3575 2023-07-14T18:34:43.068 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3576 2023-07-14T18:34:43.068 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3576_vm 2023-07-14T18:34:43.068 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3596 2023-07-14T18:34:43.068 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3596_vm 2023-07-14T18:34:43.068 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3597 2023-07-14T18:34:43.068 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3598 2023-07-14T18:34:43.068 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3598_vm 2023-07-14T18:34:43.069 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3599 2023-07-14T18:34:43.069 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3600 2023-07-14T18:34:43.069 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3600_fsm 2023-07-14T18:34:43.069 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3600_vm 2023-07-14T18:34:43.069 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3601 2023-07-14T18:34:43.069 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3601_fsm 2023-07-14T18:34:43.069 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3601_vm 2023-07-14T18:34:43.069 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3602 2023-07-14T18:34:43.070 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3602_fsm 2023-07-14T18:34:43.070 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3602_vm 2023-07-14T18:34:43.070 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3603 2023-07-14T18:34:43.070 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3603_fsm 2023-07-14T18:34:43.070 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3603_vm 2023-07-14T18:34:43.070 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3604 2023-07-14T18:34:43.070 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3605 2023-07-14T18:34:43.070 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3606 2023-07-14T18:34:43.071 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3607 2023-07-14T18:34:43.071 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3608 2023-07-14T18:34:43.071 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 14 18:16 3609 2023-07-14T18:34:43.071 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3712 2023-07-14T18:34:43.071 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3764 2023-07-14T18:34:43.071 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 3764_fsm 2023-07-14T18:34:43.071 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3764_vm 2023-07-14T18:34:43.071 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3766 2023-07-14T18:34:43.072 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 3767 2023-07-14T18:34:43.072 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3997 2023-07-14T18:34:43.072 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 5002 2023-07-14T18:34:43.072 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 548 2023-07-14T18:34:43.072 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 549 2023-07-14T18:34:43.072 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6102 2023-07-14T18:34:43.073 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6102_vm 2023-07-14T18:34:43.073 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6104 2023-07-14T18:34:43.073 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6104_vm 2023-07-14T18:34:43.073 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6106 2023-07-14T18:34:43.073 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6106_vm 2023-07-14T18:34:43.073 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6110 2023-07-14T18:34:43.073 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6111 2023-07-14T18:34:43.074 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6112 2023-07-14T18:34:43.074 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6113 2023-07-14T18:34:43.074 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6117 2023-07-14T18:34:43.074 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 826 2023-07-14T18:34:43.074 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 826_vm 2023-07-14T18:34:43.074 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 827 2023-07-14T18:34:43.074 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 828 2023-07-14T18:34:43.075 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jul 14 18:16 pg_filenode.map 2023-07-14T18:34:43.075 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 137860 Jul 14 18:16 pg_internal.init 2023-07-14T18:34:43.075 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 14 18:16 PG_VERSION 2023-07-14T18:34:43.075 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:43.075 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2023-07-14T18:34:43.075 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:34:43.075 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:19 . 2023-07-14T18:34:43.076 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jul 14 18:18 .. 2023-07-14T18:34:43.076 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:43.076 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/global: 2023-07-14T18:34:43.076 INFO:teuthology.orchestra.run.smithi133.stdout:total 574 2023-07-14T18:34:43.076 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 59 Jul 14 18:16 . 2023-07-14T18:34:43.076 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 14 18:34 .. 2023-07-14T18:34:43.076 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1136 2023-07-14T18:34:43.077 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1136_fsm 2023-07-14T18:34:43.077 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1136_vm 2023-07-14T18:34:43.077 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 1137 2023-07-14T18:34:43.077 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1213 2023-07-14T18:34:43.077 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1213_fsm 2023-07-14T18:34:43.077 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1213_vm 2023-07-14T18:34:43.078 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1214 2023-07-14T18:34:43.078 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1214_fsm 2023-07-14T18:34:43.078 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1214_vm 2023-07-14T18:34:43.078 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 1232 2023-07-14T18:34:43.078 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 1233 2023-07-14T18:34:43.078 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1260 2023-07-14T18:34:43.078 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1260_fsm 2023-07-14T18:34:43.079 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1260_vm 2023-07-14T18:34:43.079 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1261 2023-07-14T18:34:43.079 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1261_fsm 2023-07-14T18:34:43.079 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1261_vm 2023-07-14T18:34:43.079 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1262 2023-07-14T18:34:43.079 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 1262_fsm 2023-07-14T18:34:43.079 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 1262_vm 2023-07-14T18:34:43.080 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2396 2023-07-14T18:34:43.080 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 14 18:16 2396_fsm 2023-07-14T18:34:43.080 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2396_vm 2023-07-14T18:34:43.080 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2397 2023-07-14T18:34:43.080 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2671 2023-07-14T18:34:43.080 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2672 2023-07-14T18:34:43.080 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2676 2023-07-14T18:34:43.081 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2677 2023-07-14T18:34:43.081 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2694 2023-07-14T18:34:43.081 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2695 2023-07-14T18:34:43.081 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2697 2023-07-14T18:34:43.081 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 14 18:16 2698 2023-07-14T18:34:43.081 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2846 2023-07-14T18:34:43.081 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2846_vm 2023-07-14T18:34:43.081 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2847 2023-07-14T18:34:43.082 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2964 2023-07-14T18:34:43.082 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2964_vm 2023-07-14T18:34:43.082 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2965 2023-07-14T18:34:43.082 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2966 2023-07-14T18:34:43.082 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 2966_vm 2023-07-14T18:34:43.083 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 2967 2023-07-14T18:34:43.083 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3592 2023-07-14T18:34:43.083 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 3592_vm 2023-07-14T18:34:43.083 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 3593 2023-07-14T18:34:43.084 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 4060 2023-07-14T18:34:43.084 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 4060_vm 2023-07-14T18:34:43.084 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 4061 2023-07-14T18:34:43.085 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6000 2023-07-14T18:34:43.085 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6000_vm 2023-07-14T18:34:43.085 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6001 2023-07-14T18:34:43.085 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6002 2023-07-14T18:34:43.086 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6100 2023-07-14T18:34:43.086 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 6100_vm 2023-07-14T18:34:43.086 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6114 2023-07-14T18:34:43.086 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 6115 2023-07-14T18:34:43.087 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:34 pg_control 2023-07-14T18:34:43.087 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jul 14 18:16 pg_filenode.map 2023-07-14T18:34:43.087 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 21644 Jul 14 18:16 pg_internal.init 2023-07-14T18:34:43.087 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:43.088 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/log: 2023-07-14T18:34:43.088 INFO:teuthology.orchestra.run.smithi133.stdout:total 6 2023-07-14T18:34:43.088 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 . 2023-07-14T18:34:43.088 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 14 18:34 .. 2023-07-14T18:34:43.089 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 5437 Jul 14 18:34 postgresql-Fri.log 2023-07-14T18:34:43.089 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:43.089 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-07-14T18:34:43.089 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:34:43.089 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 . 2023-07-14T18:34:43.090 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 14 18:34 .. 2023-07-14T18:34:43.090 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:43.090 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-07-14T18:34:43.090 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:34:43.090 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 . 2023-07-14T18:34:43.090 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 14 18:34 .. 2023-07-14T18:34:43.090 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:43.091 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-07-14T18:34:43.091 INFO:teuthology.orchestra.run.smithi133.stdout:total 1 2023-07-14T18:34:43.091 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 14 18:34 . 2023-07-14T18:34:43.091 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 14 18:34 .. 2023-07-14T18:34:43.091 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 mappings 2023-07-14T18:34:43.091 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jul 14 18:34 replorigin_checkpoint 2023-07-14T18:34:43.091 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 snapshots 2023-07-14T18:34:43.091 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:43.092 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-07-14T18:34:43.092 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:34:43.092 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 . 2023-07-14T18:34:43.092 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 14 18:34 .. 2023-07-14T18:34:43.092 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:43.092 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-07-14T18:34:43.092 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:34:43.093 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 . 2023-07-14T18:34:43.093 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 14 18:34 .. 2023-07-14T18:34:43.093 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:43.093 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-07-14T18:34:43.093 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:34:43.093 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jul 14 18:16 . 2023-07-14T18:34:43.093 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 14 18:34 .. 2023-07-14T18:34:43.094 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 members 2023-07-14T18:34:43.094 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 offsets 2023-07-14T18:34:43.094 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:43.094 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-07-14T18:34:43.095 INFO:teuthology.orchestra.run.smithi133.stdout:total 8 2023-07-14T18:34:43.095 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 . 2023-07-14T18:34:43.095 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jul 14 18:16 .. 2023-07-14T18:34:43.095 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 0000 2023-07-14T18:34:43.095 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:43.095 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-07-14T18:34:43.095 INFO:teuthology.orchestra.run.smithi133.stdout:total 8 2023-07-14T18:34:43.096 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 . 2023-07-14T18:34:43.096 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jul 14 18:16 .. 2023-07-14T18:34:43.096 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 0000 2023-07-14T18:34:43.096 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:43.096 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-07-14T18:34:43.096 INFO:teuthology.orchestra.run.smithi133.stdout:total 8 2023-07-14T18:34:43.096 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 . 2023-07-14T18:34:43.096 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 14 18:34 .. 2023-07-14T18:34:43.097 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 14 18:16 0000 2023-07-14T18:34:43.097 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:43.097 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-07-14T18:34:43.097 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:34:43.097 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 . 2023-07-14T18:34:43.097 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 14 18:34 .. 2023-07-14T18:34:43.097 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:43.097 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-07-14T18:34:43.098 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:34:43.098 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 . 2023-07-14T18:34:43.098 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 14 18:34 .. 2023-07-14T18:34:43.098 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:43.098 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-07-14T18:34:43.098 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:34:43.098 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 . 2023-07-14T18:34:43.098 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 14 18:34 .. 2023-07-14T18:34:43.099 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:43.099 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-07-14T18:34:43.099 INFO:teuthology.orchestra.run.smithi133.stdout:total 13 2023-07-14T18:34:43.099 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 14 18:34 . 2023-07-14T18:34:43.099 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 14 18:34 .. 2023-07-14T18:34:43.099 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1864 Jul 14 18:34 db_0.stat 2023-07-14T18:34:43.099 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 9300 Jul 14 18:34 db_12347.stat 2023-07-14T18:34:43.099 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 607 Jul 14 18:34 global.stat 2023-07-14T18:34:43.100 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:43.100 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-07-14T18:34:43.100 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:34:43.100 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:34 . 2023-07-14T18:34:43.100 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 14 18:34 .. 2023-07-14T18:34:43.100 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:43.100 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-07-14T18:34:43.100 INFO:teuthology.orchestra.run.smithi133.stdout:total 152 2023-07-14T18:34:43.101 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:34 . 2023-07-14T18:34:43.101 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 14 18:34 .. 2023-07-14T18:34:43.101 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 155648 Jul 14 18:34 000B 2023-07-14T18:34:43.101 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:43.101 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-07-14T18:34:43.101 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:34:43.101 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 . 2023-07-14T18:34:43.101 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 14 18:34 .. 2023-07-14T18:34:43.102 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:43.102 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-07-14T18:34:43.102 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:34:43.102 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 . 2023-07-14T18:34:43.102 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 14 18:34 .. 2023-07-14T18:34:43.102 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:43.102 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-07-14T18:34:43.102 INFO:teuthology.orchestra.run.smithi133.stdout:total 1032192 2023-07-14T18:34:43.103 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 64 Jul 14 18:34 . 2023-07-14T18:34:43.103 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 14 18:34 .. 2023-07-14T18:34:43.103 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:34 0000000100000003000000D0 2023-07-14T18:34:43.103 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:31 0000000100000003000000D1 2023-07-14T18:34:43.103 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:31 0000000100000003000000D2 2023-07-14T18:34:43.103 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:32 0000000100000003000000D3 2023-07-14T18:34:43.103 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:32 0000000100000003000000D4 2023-07-14T18:34:43.103 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:31 0000000100000003000000D5 2023-07-14T18:34:43.104 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:32 0000000100000003000000D6 2023-07-14T18:34:43.104 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:31 0000000100000003000000D7 2023-07-14T18:34:43.104 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:32 0000000100000003000000D8 2023-07-14T18:34:43.104 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:32 0000000100000003000000D9 2023-07-14T18:34:43.104 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:32 0000000100000003000000DA 2023-07-14T18:34:43.104 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:32 0000000100000003000000DB 2023-07-14T18:34:43.104 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:32 0000000100000003000000DC 2023-07-14T18:34:43.104 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:31 0000000100000003000000DD 2023-07-14T18:34:43.105 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:33 0000000100000003000000DE 2023-07-14T18:34:43.105 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:33 0000000100000003000000DF 2023-07-14T18:34:43.105 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:32 0000000100000003000000E0 2023-07-14T18:34:43.105 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:32 0000000100000003000000E1 2023-07-14T18:34:43.105 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:32 0000000100000003000000E2 2023-07-14T18:34:43.105 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:32 0000000100000003000000E3 2023-07-14T18:34:43.105 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:33 0000000100000003000000E4 2023-07-14T18:34:43.105 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:32 0000000100000003000000E5 2023-07-14T18:34:43.106 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:32 0000000100000003000000E6 2023-07-14T18:34:43.106 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:33 0000000100000003000000E7 2023-07-14T18:34:43.106 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:32 0000000100000003000000E8 2023-07-14T18:34:43.106 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:32 0000000100000003000000E9 2023-07-14T18:34:43.106 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:33 0000000100000003000000EA 2023-07-14T18:34:43.106 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:33 0000000100000003000000EB 2023-07-14T18:34:43.106 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:33 0000000100000003000000EC 2023-07-14T18:34:43.107 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:33 0000000100000003000000ED 2023-07-14T18:34:43.107 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:33 0000000100000003000000EE 2023-07-14T18:34:43.107 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:32 0000000100000003000000EF 2023-07-14T18:34:43.107 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:33 0000000100000003000000F0 2023-07-14T18:34:43.107 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:33 0000000100000003000000F1 2023-07-14T18:34:43.107 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:33 0000000100000003000000F2 2023-07-14T18:34:43.107 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:32 0000000100000003000000F3 2023-07-14T18:34:43.107 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:33 0000000100000003000000F4 2023-07-14T18:34:43.107 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:33 0000000100000003000000F5 2023-07-14T18:34:43.108 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:32 0000000100000003000000F6 2023-07-14T18:34:43.108 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:33 0000000100000003000000F7 2023-07-14T18:34:43.108 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:33 0000000100000003000000F8 2023-07-14T18:34:43.108 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:34 0000000100000003000000F9 2023-07-14T18:34:43.108 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:33 0000000100000003000000FA 2023-07-14T18:34:43.108 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:34 0000000100000003000000FB 2023-07-14T18:34:43.108 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:33 0000000100000003000000FC 2023-07-14T18:34:43.109 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:33 0000000100000003000000FD 2023-07-14T18:34:43.109 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:34 0000000100000003000000FE 2023-07-14T18:34:43.109 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:34 0000000100000003000000FF 2023-07-14T18:34:43.109 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:33 000000010000000400000000 2023-07-14T18:34:43.109 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:34 000000010000000400000001 2023-07-14T18:34:43.109 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:33 000000010000000400000002 2023-07-14T18:34:43.109 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:33 000000010000000400000003 2023-07-14T18:34:43.109 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:33 000000010000000400000004 2023-07-14T18:34:43.109 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:33 000000010000000400000005 2023-07-14T18:34:43.110 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:33 000000010000000400000006 2023-07-14T18:34:43.110 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:34 000000010000000400000007 2023-07-14T18:34:43.110 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:34 000000010000000400000008 2023-07-14T18:34:43.110 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:34 000000010000000400000009 2023-07-14T18:34:43.110 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:34 00000001000000040000000A 2023-07-14T18:34:43.110 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:34 00000001000000040000000B 2023-07-14T18:34:43.110 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:34 00000001000000040000000C 2023-07-14T18:34:43.110 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:34 00000001000000040000000D 2023-07-14T18:34:43.111 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 14 18:34 00000001000000040000000E 2023-07-14T18:34:43.111 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 archive_status 2023-07-14T18:34:43.111 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:43.111 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-07-14T18:34:43.111 INFO:teuthology.orchestra.run.smithi133.stdout:total 0 2023-07-14T18:34:43.111 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 14 18:16 . 2023-07-14T18:34:43.111 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 64 Jul 14 18:34 .. 2023-07-14T18:34:43.111 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:43.112 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-07-14T18:34:43.112 INFO:teuthology.orchestra.run.smithi133.stdout:total 192 2023-07-14T18:34:43.112 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 14 18:16 . 2023-07-14T18:34:43.112 INFO:teuthology.orchestra.run.smithi133.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 14 18:34 .. 2023-07-14T18:34:43.112 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Jul 14 18:34 0000 2023-07-14T18:34:43.113 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-07-14T18:34:43.123 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-07-14T18:34:43.124 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_81c9387d2e7576da065747ec9c16d1c7d69016ec/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_81c9387d2e7576da065747ec9c16d1c7d69016ec/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds 2023-07-14T18:34:43.124 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-07-14T18:34:43.133 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T18:34:43.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.321+0000 7ff46e3ff700 1 -- 172.21.15.133:0/3605269260 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4680730a0 msgr2=0x7ff468073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:43.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.321+0000 7ff46e3ff700 1 --2- 172.21.15.133:0/3605269260 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4680730a0 0x7ff468073510 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009990 tx=0x7ff45c009ca0 comp rx=0 tx=0).stop 2023-07-14T18:34:43.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.321+0000 7ff46e3ff700 1 -- 172.21.15.133:0/3605269260 shutdown_connections 2023-07-14T18:34:43.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.321+0000 7ff46e3ff700 1 --2- 172.21.15.133:0/3605269260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff468074680 0x7ff468072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:43.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.321+0000 7ff46e3ff700 1 --2- 172.21.15.133:0/3605269260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff468107af0 0x7ff468109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:43.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.321+0000 7ff46e3ff700 1 --2- 172.21.15.133:0/3605269260 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4680730a0 0x7ff468073510 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:43.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.321+0000 7ff46e3ff700 1 -- 172.21.15.133:0/3605269260 >> 172.21.15.133:0/3605269260 conn(0x7ff4680fb210 msgr2=0x7ff4680fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:34:43.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.321+0000 7ff46e3ff700 1 -- 172.21.15.133:0/3605269260 shutdown_connections 2023-07-14T18:34:43.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.321+0000 7ff46e3ff700 1 -- 172.21.15.133:0/3605269260 wait complete. 2023-07-14T18:34:43.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.323+0000 7ff46e3ff700 1 Processor -- start 2023-07-14T18:34:43.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.323+0000 7ff46e3ff700 1 -- start start 2023-07-14T18:34:43.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.323+0000 7ff46e3ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4680730a0 0x7ff468197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:34:43.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.323+0000 7ff46e3ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff468074680 0x7ff4681980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:34:43.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.324+0000 7ff46e3ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff468107af0 0x7ff46819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:34:43.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.324+0000 7ff46e3ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46819f710 con 0x7ff4680730a0 2023-07-14T18:34:43.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.324+0000 7ff46e3ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46819f850 con 0x7ff468107af0 2023-07-14T18:34:43.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.324+0000 7ff46e3ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46819f9e0 con 0x7ff468074680 2023-07-14T18:34:43.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.324+0000 7ff4677fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff468074680 0x7ff4681980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:34:43.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.324+0000 7ff4677fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff468074680 0x7ff4681980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:51584/0 (socket says 172.21.15.133:51584) 2023-07-14T18:34:43.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.324+0000 7ff4677fe700 1 -- 172.21.15.133:0/3297102629 learned_addr learned my addr 172.21.15.133:0/3297102629 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:34:43.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.324+0000 7ff467fff700 1 --2- 172.21.15.133:0/3297102629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4680730a0 0x7ff468197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:34:43.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.324+0000 7ff46c99c700 1 --2- 172.21.15.133:0/3297102629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff468107af0 0x7ff46819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:34:43.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.324+0000 7ff4677fe700 1 -- 172.21.15.133:0/3297102629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff468107af0 msgr2=0x7ff46819f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:43.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.324+0000 7ff4677fe700 1 --2- 172.21.15.133:0/3297102629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff468107af0 0x7ff46819f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:43.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.324+0000 7ff4677fe700 1 -- 172.21.15.133:0/3297102629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4680730a0 msgr2=0x7ff468197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:43.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.324+0000 7ff4677fe700 1 --2- 172.21.15.133:0/3297102629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4680730a0 0x7ff468197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:43.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.324+0000 7ff4677fe700 1 -- 172.21.15.133:0/3297102629 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45c0095f0 con 0x7ff468074680 2023-07-14T18:34:43.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.324+0000 7ff4677fe700 1 --2- 172.21.15.133:0/3297102629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff468074680 0x7ff4681980d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff45c004710 tx=0x7ff45c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:34:43.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.325+0000 7ff4657fa700 1 -- 172.21.15.133:0/3297102629 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c01d070 con 0x7ff468074680 2023-07-14T18:34:43.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.325+0000 7ff4657fa700 1 -- 172.21.15.133:0/3297102629 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff45c021410 con 0x7ff468074680 2023-07-14T18:34:43.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.325+0000 7ff46e3ff700 1 -- 172.21.15.133:0/3297102629 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff46819fc60 con 0x7ff468074680 2023-07-14T18:34:43.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.325+0000 7ff4657fa700 1 -- 172.21.15.133:0/3297102629 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c017650 con 0x7ff468074680 2023-07-14T18:34:43.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.325+0000 7ff46e3ff700 1 -- 172.21.15.133:0/3297102629 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4681a00e0 con 0x7ff468074680 2023-07-14T18:34:43.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.326+0000 7ff46e3ff700 1 -- 172.21.15.133:0/3297102629 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4681061a0 con 0x7ff468074680 2023-07-14T18:34:43.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.328+0000 7ff4657fa700 1 -- 172.21.15.133:0/3297102629 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff45c021580 con 0x7ff468074680 2023-07-14T18:34:43.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.329+0000 7ff4657fa700 1 --2- 172.21.15.133:0/3297102629 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff4500a1220 0x7ff4500a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:34:43.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.329+0000 7ff4657fa700 1 -- 172.21.15.133:0/3297102629 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff45c0ce630 con 0x7ff468074680 2023-07-14T18:34:43.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.329+0000 7ff467fff700 1 --2- 172.21.15.133:0/3297102629 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff4500a1220 0x7ff4500a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:34:43.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.330+0000 7ff467fff700 1 --2- 172.21.15.133:0/3297102629 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff4500a1220 0x7ff4500a36d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff468074a50 tx=0x7ff458009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:34:43.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.331+0000 7ff4657fa700 1 -- 172.21.15.133:0/3297102629 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7ff45c097a80 con 0x7ff468074680 2023-07-14T18:34:43.553 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.534+0000 7ff46e3ff700 1 -- 172.21.15.133:0/3297102629 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff46802c8e0 con 0x7ff468074680 2023-07-14T18:34:43.555 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.536+0000 7ff4657fa700 1 -- 172.21.15.133:0/3297102629 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24520 (secure 0 0 0) 0x7ff45c09b0a0 con 0x7ff468074680 2023-07-14T18:34:43.556 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:43.557 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":92,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","created":"2023-07-14T18:06:49.561582+0000","modified":"2023-07-14T18:15:49.951050+0000","last_up_change":"2023-07-14T18:13:29.836099+0000","last_in_change":"2023-07-14T18:13:11.606710+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T18:10:03.945039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T18:14:15.451340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T18:14:16.892215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T18:14:20.493175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"85a29de7-d712-422a-b647-d9e90091aaf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6803","nonce":3076510801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6805","nonce":3076510801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6809","nonce":3076510801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6807","nonce":3076510801}]},"public_addr":"172.21.15.133:6803/3076510801","cluster_addr":"172.21.15.133:6805/3076510801","heartbeat_back_addr":"172.21.15.133:6809/3076510801","heartbeat_front_addr":"172.21.15.133:6807/3076510801","state":["exists","up"]},{"osd":1,"uuid":"80ad4e78-7917-495e-b119-99f7c7c36243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6801","nonce":1841901852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6803","nonce":1841901852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6807","nonce":1841901852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6805","nonce":1841901852}]},"public_addr":"172.21.15.161:6801/1841901852","cluster_addr":"172.21.15.161:6803/1841901852","heartbeat_back_addr":"172.21.15.161:6807/1841901852","heartbeat_front_addr":"172.21.15.161:6805/1841901852","state":["exists","up"]},{"osd":2,"uuid":"fc4bfd37-1fdd-47c6-a48e-6d910daf9e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6801","nonce":346103099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6803","nonce":346103099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6807","nonce":346103099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6805","nonce":346103099}]},"public_addr":"172.21.15.171:6801/346103099","cluster_addr":"172.21.15.171:6803/346103099","heartbeat_back_addr":"172.21.15.171:6807/346103099","heartbeat_front_addr":"172.21.15.171:6805/346103099","state":["exists","up"]},{"osd":3,"uuid":"4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6811","nonce":2152994143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6813","nonce":2152994143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6817","nonce":2152994143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6815","nonce":2152994143}]},"public_addr":"172.21.15.133:6811/2152994143","cluster_addr":"172.21.15.133:6813/2152994143","heartbeat_back_addr":"172.21.15.133:6817/2152994143","heartbeat_front_addr":"172.21.15.133:6815/2152994143","state":["exists","up"]},{"osd":4,"uuid":"4601a1c2-254b-4c4d-9b15-8fc952b3d2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6809","nonce":47213291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6811","nonce":47213291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6815","nonce":47213291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6813","nonce":47213291}]},"public_addr":"172.21.15.161:6809/47213291","cluster_addr":"172.21.15.161:6811/47213291","heartbeat_back_addr":"172.21.15.161:6815/47213291","heartbeat_front_addr":"172.21.15.161:6813/47213291","state":["exists","up"]},{"osd":5,"uuid":"6cca180a-9140-4e49-835b-86842dae0574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6809","nonce":777556751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6811","nonce":777556751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6815","nonce":777556751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6813","nonce":777556751}]},"public_addr":"172.21.15.171:6809/777556751","cluster_addr":"172.21.15.171:6811/777556751","heartbeat_back_addr":"172.21.15.171:6815/777556751","heartbeat_front_addr":"172.21.15.171:6813/777556751","state":["exists","up"]},{"osd":6,"uuid":"ea0299ff-7740-4494-a3a9-1503b3fc0ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6819","nonce":2884152358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6821","nonce":2884152358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6825","nonce":2884152358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6823","nonce":2884152358}]},"public_addr":"172.21.15.133:6819/2884152358","cluster_addr":"172.21.15.133:6821/2884152358","heartbeat_back_addr":"172.21.15.133:6825/2884152358","heartbeat_front_addr":"172.21.15.133:6823/2884152358","state":["exists","up"]},{"osd":7,"uuid":"b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6817","nonce":4183705749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6819","nonce":4183705749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6823","nonce":4183705749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6821","nonce":4183705749}]},"public_addr":"172.21.15.161:6817/4183705749","cluster_addr":"172.21.15.161:6819/4183705749","heartbeat_back_addr":"172.21.15.161:6823/4183705749","heartbeat_front_addr":"172.21.15.161:6821/4183705749","state":["exists","up"]},{"osd":8,"uuid":"87dffc07-bf48-48c8-b418-5c4b509686d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6817","nonce":2771494483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6819","nonce":2771494483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6823","nonce":2771494483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6821","nonce":2771494483}]},"public_addr":"172.21.15.171:6817/2771494483","cluster_addr":"172.21.15.171:6819/2771494483","heartbeat_back_addr":"172.21.15.171:6823/2771494483","heartbeat_front_addr":"172.21.15.171:6821/2771494483","state":["exists","up"]},{"osd":9,"uuid":"e0832870-79ce-4bce-8045-58b7a7f6dc69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6827","nonce":2066164149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6829","nonce":2066164149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6833","nonce":2066164149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6831","nonce":2066164149}]},"public_addr":"172.21.15.133:6827/2066164149","cluster_addr":"172.21.15.133:6829/2066164149","heartbeat_back_addr":"172.21.15.133:6833/2066164149","heartbeat_front_addr":"172.21.15.133:6831/2066164149","state":["exists","up"]},{"osd":10,"uuid":"081f0484-d134-4b2e-9c48-ef8e6a646422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6825","nonce":2162333626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6827","nonce":2162333626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6831","nonce":2162333626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6829","nonce":2162333626}]},"public_addr":"172.21.15.161:6825/2162333626","cluster_addr":"172.21.15.161:6827/2162333626","heartbeat_back_addr":"172.21.15.161:6831/2162333626","heartbeat_front_addr":"172.21.15.161:6829/2162333626","state":["exists","up"]},{"osd":11,"uuid":"4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6825","nonce":3403894313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6827","nonce":3403894313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6831","nonce":3403894313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6829","nonce":3403894313}]},"public_addr":"172.21.15.171:6825/3403894313","cluster_addr":"172.21.15.171:6827/3403894313","heartbeat_back_addr":"172.21.15.171:6831/3403894313","heartbeat_front_addr":"172.21.15.171:6829/3403894313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:17.816229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:39.638543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:00.543317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:26.988921+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:48.644694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:10.049860+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:35.986508+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:57.753846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:19.018245+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:44.460450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:06.279960+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:27.955006+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/1832569241":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6801/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:0/4219815304":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:6801/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/432253615":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1840700480":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1807775439":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1089030734":"2023-07-15T18:07:47.346704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T18:34:43.558 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.539+0000 7ff46e3ff700 1 -- 172.21.15.133:0/3297102629 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff4500a1220 msgr2=0x7ff4500a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:43.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.539+0000 7ff46e3ff700 1 --2- 172.21.15.133:0/3297102629 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff4500a1220 0x7ff4500a36d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff468074a50 tx=0x7ff458009260 comp rx=0 tx=0).stop 2023-07-14T18:34:43.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.539+0000 7ff46e3ff700 1 -- 172.21.15.133:0/3297102629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff468074680 msgr2=0x7ff4681980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:43.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.539+0000 7ff46e3ff700 1 --2- 172.21.15.133:0/3297102629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff468074680 0x7ff4681980d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff45c004710 tx=0x7ff45c009580 comp rx=0 tx=0).stop 2023-07-14T18:34:43.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.539+0000 7ff46e3ff700 1 -- 172.21.15.133:0/3297102629 shutdown_connections 2023-07-14T18:34:43.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.539+0000 7ff46e3ff700 1 --2- 172.21.15.133:0/3297102629 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff4500a1220 0x7ff4500a36d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:43.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.539+0000 7ff46e3ff700 1 --2- 172.21.15.133:0/3297102629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff468107af0 0x7ff46819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:43.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.539+0000 7ff46e3ff700 1 --2- 172.21.15.133:0/3297102629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4680730a0 0x7ff468197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:43.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.539+0000 7ff46e3ff700 1 --2- 172.21.15.133:0/3297102629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff468074680 0x7ff4681980d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:43.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.539+0000 7ff46e3ff700 1 -- 172.21.15.133:0/3297102629 >> 172.21.15.133:0/3297102629 conn(0x7ff4680fb210 msgr2=0x7ff4680fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:34:43.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.539+0000 7ff46e3ff700 1 -- 172.21.15.133:0/3297102629 shutdown_connections 2023-07-14T18:34:43.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.539+0000 7ff46e3ff700 1 -- 172.21.15.133:0/3297102629 wait complete. 2023-07-14T18:34:43.581 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T18:10:03.945039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T18:14:15.451340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T18:14:16.892215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T18:14:20.493175+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T18:34:43.582 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T18:34:43.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.765+0000 7f09359ff700 1 -- 172.21.15.133:0/2576183953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09301017a0 msgr2=0x7f0930108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:43.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.765+0000 7f09359ff700 1 --2- 172.21.15.133:0/2576183953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09301017a0 0x7f0930108060 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0928009f90 tx=0x7f092800b330 comp rx=0 tx=0).stop 2023-07-14T18:34:43.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.765+0000 7f09359ff700 1 -- 172.21.15.133:0/2576183953 shutdown_connections 2023-07-14T18:34:43.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.765+0000 7f09359ff700 1 --2- 172.21.15.133:0/2576183953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09301017a0 0x7f0930108060 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:43.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.765+0000 7f09359ff700 1 --2- 172.21.15.133:0/2576183953 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09300ffc20 0x7f0930100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:43.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.765+0000 7f09359ff700 1 --2- 172.21.15.133:0/2576183953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0930100e10 0x7f0930101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:43.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.765+0000 7f09359ff700 1 -- 172.21.15.133:0/2576183953 >> 172.21.15.133:0/2576183953 conn(0x7f09300fb260 msgr2=0x7f09300fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:34:43.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.765+0000 7f09359ff700 1 -- 172.21.15.133:0/2576183953 shutdown_connections 2023-07-14T18:34:43.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.765+0000 7f09359ff700 1 -- 172.21.15.133:0/2576183953 wait complete. 2023-07-14T18:34:43.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.766+0000 7f09359ff700 1 Processor -- start 2023-07-14T18:34:43.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.767+0000 7f09359ff700 1 -- start start 2023-07-14T18:34:43.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.767+0000 7f09359ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09300ffc20 0x7f0930195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:34:43.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.767+0000 7f09359ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0930100e10 0x7f0930195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:34:43.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.767+0000 7f09359ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09301017a0 0x7f093019cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:34:43.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.767+0000 7f09359ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093019d580 con 0x7f09301017a0 2023-07-14T18:34:43.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.767+0000 7f09359ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093019d6c0 con 0x7f09300ffc20 2023-07-14T18:34:43.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.767+0000 7f09359ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093019d850 con 0x7f0930100e10 2023-07-14T18:34:43.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.767+0000 7f092e7fc700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0930100e10 0x7f0930195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:34:43.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.767+0000 7f092effd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09300ffc20 0x7f0930195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:34:43.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.767+0000 7f092e7fc700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0930100e10 0x7f0930195f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:51604/0 (socket says 172.21.15.133:51604) 2023-07-14T18:34:43.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.767+0000 7f092e7fc700 1 -- 172.21.15.133:0/3759286489 learned_addr learned my addr 172.21.15.133:0/3759286489 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:34:43.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.767+0000 7f092f7fe700 1 --2- 172.21.15.133:0/3759286489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09301017a0 0x7f093019cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:34:43.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.768+0000 7f092effd700 1 -- 172.21.15.133:0/3759286489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0930100e10 msgr2=0x7f0930195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:43.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.768+0000 7f092effd700 1 --2- 172.21.15.133:0/3759286489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0930100e10 0x7f0930195f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:43.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.768+0000 7f092effd700 1 -- 172.21.15.133:0/3759286489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09301017a0 msgr2=0x7f093019cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:43.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.768+0000 7f092effd700 1 --2- 172.21.15.133:0/3759286489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09301017a0 0x7f093019cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:43.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.768+0000 7f092effd700 1 -- 172.21.15.133:0/3759286489 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09200095f0 con 0x7f09300ffc20 2023-07-14T18:34:43.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.768+0000 7f092e7fc700 1 --2- 172.21.15.133:0/3759286489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0930100e10 0x7f0930195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:34:43.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.768+0000 7f092effd700 1 --2- 172.21.15.133:0/3759286489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09300ffc20 0x7f0930195a00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0920005fd0 tx=0x7f092000ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:34:43.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.768+0000 7f09349fd700 1 -- 172.21.15.133:0/3759286489 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092000cbd0 con 0x7f09300ffc20 2023-07-14T18:34:43.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.768+0000 7f09349fd700 1 -- 172.21.15.133:0/3759286489 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f092000cd30 con 0x7f09300ffc20 2023-07-14T18:34:43.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.768+0000 7f09349fd700 1 -- 172.21.15.133:0/3759286489 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0920018710 con 0x7f09300ffc20 2023-07-14T18:34:43.793 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.768+0000 7f09359ff700 1 -- 172.21.15.133:0/3759286489 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0928009c40 con 0x7f09300ffc20 2023-07-14T18:34:43.793 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.768+0000 7f09359ff700 1 -- 172.21.15.133:0/3759286489 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f093019de60 con 0x7f09300ffc20 2023-07-14T18:34:43.793 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.769+0000 7f09359ff700 1 -- 172.21.15.133:0/3759286489 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0930107110 con 0x7f09300ffc20 2023-07-14T18:34:43.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.771+0000 7f09349fd700 1 -- 172.21.15.133:0/3759286489 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f0920010980 con 0x7f09300ffc20 2023-07-14T18:34:43.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.773+0000 7f09349fd700 1 --2- 172.21.15.133:0/3759286489 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f09180a11d0 0x7f09180a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:34:43.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.773+0000 7f09349fd700 1 -- 172.21.15.133:0/3759286489 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f0920014070 con 0x7f09300ffc20 2023-07-14T18:34:43.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.773+0000 7f092e7fc700 1 --2- 172.21.15.133:0/3759286489 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f09180a11d0 0x7f09180a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:34:43.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.773+0000 7f092e7fc700 1 --2- 172.21.15.133:0/3759286489 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f09180a11d0 0x7f09180a3680 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0924011550 tx=0x7f0924009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:34:43.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.777+0000 7f09349fd700 1 -- 172.21.15.133:0/3759286489 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f0920096830 con 0x7f09300ffc20 2023-07-14T18:34:44.010 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.990+0000 7f09359ff700 1 -- 172.21.15.133:0/3759286489 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f093002ca90 con 0x7f09300ffc20 2023-07-14T18:34:44.010 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.991+0000 7f09349fd700 1 -- 172.21.15.133:0/3759286489 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0920099e50 con 0x7f09300ffc20 2023-07-14T18:34:44.011 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-07-14T18:34:44.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.993+0000 7f09359ff700 1 -- 172.21.15.133:0/3759286489 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f09180a11d0 msgr2=0x7f09180a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:44.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.993+0000 7f09359ff700 1 --2- 172.21.15.133:0/3759286489 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f09180a11d0 0x7f09180a3680 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0924011550 tx=0x7f0924009260 comp rx=0 tx=0).stop 2023-07-14T18:34:44.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.993+0000 7f09359ff700 1 -- 172.21.15.133:0/3759286489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09300ffc20 msgr2=0x7f0930195a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:44.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.993+0000 7f09359ff700 1 --2- 172.21.15.133:0/3759286489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09300ffc20 0x7f0930195a00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0920005fd0 tx=0x7f092000ee70 comp rx=0 tx=0).stop 2023-07-14T18:34:44.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.993+0000 7f09359ff700 1 -- 172.21.15.133:0/3759286489 shutdown_connections 2023-07-14T18:34:44.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.993+0000 7f09359ff700 1 --2- 172.21.15.133:0/3759286489 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f09180a11d0 0x7f09180a3680 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:44.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.993+0000 7f09359ff700 1 --2- 172.21.15.133:0/3759286489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09300ffc20 0x7f0930195a00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:44.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.993+0000 7f09359ff700 1 --2- 172.21.15.133:0/3759286489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09301017a0 0x7f093019cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:44.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.994+0000 7f09359ff700 1 --2- 172.21.15.133:0/3759286489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0930100e10 0x7f0930195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:44.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.994+0000 7f09359ff700 1 -- 172.21.15.133:0/3759286489 >> 172.21.15.133:0/3759286489 conn(0x7f09300fb260 msgr2=0x7f0930104250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:34:44.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.994+0000 7f09359ff700 1 -- 172.21.15.133:0/3759286489 shutdown_connections 2023-07-14T18:34:44.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:43.994+0000 7f09359ff700 1 -- 172.21.15.133:0/3759286489 wait complete. 2023-07-14T18:34:44.033 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T18:34:44.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.216+0000 7f76ba7ff700 1 -- 172.21.15.133:0/535170108 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76b4100e00 msgr2=0x7f76b4101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:44.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.216+0000 7f76ba7ff700 1 --2- 172.21.15.133:0/535170108 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76b4100e00 0x7f76b4101250 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f76a80098f0 tx=0x7f76a8009c00 comp rx=0 tx=0).stop 2023-07-14T18:34:44.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.216+0000 7f76ba7ff700 1 -- 172.21.15.133:0/535170108 shutdown_connections 2023-07-14T18:34:44.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.216+0000 7f76ba7ff700 1 --2- 172.21.15.133:0/535170108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76b4101790 0x7f76b4108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:44.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.216+0000 7f76ba7ff700 1 --2- 172.21.15.133:0/535170108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b40ffc10 0x7f76b4100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:44.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.216+0000 7f76ba7ff700 1 --2- 172.21.15.133:0/535170108 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76b4100e00 0x7f76b4101250 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:44.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.216+0000 7f76ba7ff700 1 -- 172.21.15.133:0/535170108 >> 172.21.15.133:0/535170108 conn(0x7f76b40fb210 msgr2=0x7f76b40fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:34:44.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.216+0000 7f76ba7ff700 1 -- 172.21.15.133:0/535170108 shutdown_connections 2023-07-14T18:34:44.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.216+0000 7f76ba7ff700 1 -- 172.21.15.133:0/535170108 wait complete. 2023-07-14T18:34:44.238 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.217+0000 7f76ba7ff700 1 Processor -- start 2023-07-14T18:34:44.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.217+0000 7f76ba7ff700 1 -- start start 2023-07-14T18:34:44.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.217+0000 7f76ba7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b40ffc10 0x7f76b4195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:34:44.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.217+0000 7f76ba7ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76b4100e00 0x7f76b4195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:34:44.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.217+0000 7f76ba7ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76b4101790 0x7f76b419cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:34:44.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.218+0000 7f76ba7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b419d500 con 0x7f76b40ffc10 2023-07-14T18:34:44.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.218+0000 7f76ba7ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b419d640 con 0x7f76b4100e00 2023-07-14T18:34:44.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.218+0000 7f76ba7ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b419d7d0 con 0x7f76b4101790 2023-07-14T18:34:44.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.218+0000 7f76b37fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76b4100e00 0x7f76b4195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:34:44.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.218+0000 7f76b37fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76b4100e00 0x7f76b4195ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:51928/0 (socket says 172.21.15.133:51928) 2023-07-14T18:34:44.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.218+0000 7f76b37fe700 1 -- 172.21.15.133:0/2949096540 learned_addr learned my addr 172.21.15.133:0/2949096540 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:34:44.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.218+0000 7f76b3fff700 1 --2- 172.21.15.133:0/2949096540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b40ffc10 0x7f76b4195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:34:44.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.218+0000 7f76b8d9c700 1 --2- 172.21.15.133:0/2949096540 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76b4101790 0x7f76b419cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:34:44.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.218+0000 7f76b37fe700 1 -- 172.21.15.133:0/2949096540 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76b4101790 msgr2=0x7f76b419cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:44.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.218+0000 7f76b37fe700 1 --2- 172.21.15.133:0/2949096540 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76b4101790 0x7f76b419cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:44.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.218+0000 7f76b37fe700 1 -- 172.21.15.133:0/2949096540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b40ffc10 msgr2=0x7f76b4195980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:44.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.218+0000 7f76b37fe700 1 --2- 172.21.15.133:0/2949096540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b40ffc10 0x7f76b4195980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:44.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.218+0000 7f76b37fe700 1 -- 172.21.15.133:0/2949096540 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a8009520 con 0x7f76b4100e00 2023-07-14T18:34:44.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.218+0000 7f76b37fe700 1 --2- 172.21.15.133:0/2949096540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76b4100e00 0x7f76b4195ec0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009fd0 tx=0x7f76a800b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:34:44.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.218+0000 7f76b17fa700 1 -- 172.21.15.133:0/2949096540 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a801c070 con 0x7f76b4100e00 2023-07-14T18:34:44.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.219+0000 7f76b17fa700 1 -- 172.21.15.133:0/2949096540 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76a800f7d0 con 0x7f76b4100e00 2023-07-14T18:34:44.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.219+0000 7f76ba7ff700 1 -- 172.21.15.133:0/2949096540 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b419da50 con 0x7f76b4100e00 2023-07-14T18:34:44.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.219+0000 7f76b17fa700 1 -- 172.21.15.133:0/2949096540 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a8004ad0 con 0x7f76b4100e00 2023-07-14T18:34:44.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.219+0000 7f76ba7ff700 1 -- 172.21.15.133:0/2949096540 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b419df20 con 0x7f76b4100e00 2023-07-14T18:34:44.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.220+0000 7f76ba7ff700 1 -- 172.21.15.133:0/2949096540 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76b410b3d0 con 0x7f76b4100e00 2023-07-14T18:34:44.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.222+0000 7f76b17fa700 1 -- 172.21.15.133:0/2949096540 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f76a8020070 con 0x7f76b4100e00 2023-07-14T18:34:44.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.223+0000 7f76b17fa700 1 --2- 172.21.15.133:0/2949096540 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f769c0a11d0 0x7f769c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:34:44.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.223+0000 7f76b17fa700 1 -- 172.21.15.133:0/2949096540 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f76a80cdcd0 con 0x7f76b4100e00 2023-07-14T18:34:44.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.223+0000 7f76b3fff700 1 --2- 172.21.15.133:0/2949096540 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f769c0a11d0 0x7f769c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:34:44.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.224+0000 7f76b3fff700 1 --2- 172.21.15.133:0/2949096540 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f769c0a11d0 0x7f769c0a3680 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f76b4100c60 tx=0x7f76a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:34:44.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.226+0000 7f76b17fa700 1 -- 172.21.15.133:0/2949096540 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f76a8097120 con 0x7f76b4100e00 2023-07-14T18:34:44.457 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.437+0000 7f76ba7ff700 1 -- 172.21.15.133:0/2949096540 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f76b4106b40 con 0x7f76b4100e00 2023-07-14T18:34:44.457 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.438+0000 7f76b17fa700 1 -- 172.21.15.133:0/2949096540 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f76a809a740 con 0x7f76b4100e00 2023-07-14T18:34:44.457 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-07-14T18:34:44.460 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.440+0000 7f76ba7ff700 1 -- 172.21.15.133:0/2949096540 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f769c0a11d0 msgr2=0x7f769c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:44.460 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.440+0000 7f76ba7ff700 1 --2- 172.21.15.133:0/2949096540 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f769c0a11d0 0x7f769c0a3680 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f76b4100c60 tx=0x7f76a4009260 comp rx=0 tx=0).stop 2023-07-14T18:34:44.460 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.440+0000 7f76ba7ff700 1 -- 172.21.15.133:0/2949096540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76b4100e00 msgr2=0x7f76b4195ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:44.460 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.440+0000 7f76ba7ff700 1 --2- 172.21.15.133:0/2949096540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76b4100e00 0x7f76b4195ec0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009fd0 tx=0x7f76a800b790 comp rx=0 tx=0).stop 2023-07-14T18:34:44.460 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.440+0000 7f76ba7ff700 1 -- 172.21.15.133:0/2949096540 shutdown_connections 2023-07-14T18:34:44.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.440+0000 7f76ba7ff700 1 --2- 172.21.15.133:0/2949096540 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f769c0a11d0 0x7f769c0a3680 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:44.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.440+0000 7f76ba7ff700 1 --2- 172.21.15.133:0/2949096540 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76b4101790 0x7f76b419cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:44.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.440+0000 7f76ba7ff700 1 --2- 172.21.15.133:0/2949096540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b40ffc10 0x7f76b4195980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:44.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.440+0000 7f76ba7ff700 1 --2- 172.21.15.133:0/2949096540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76b4100e00 0x7f76b4195ec0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:44.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.440+0000 7f76ba7ff700 1 -- 172.21.15.133:0/2949096540 >> 172.21.15.133:0/2949096540 conn(0x7f76b40fb210 msgr2=0x7f76b40fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:34:44.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.440+0000 7f76ba7ff700 1 -- 172.21.15.133:0/2949096540 shutdown_connections 2023-07-14T18:34:44.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.440+0000 7f76ba7ff700 1 -- 172.21.15.133:0/2949096540 wait complete. 2023-07-14T18:34:44.479 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T18:34:44.683 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.664+0000 7f01f1dff700 1 -- 172.21.15.133:0/318333312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01ec0ffc20 msgr2=0x7f01ec100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:44.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.664+0000 7f01f1dff700 1 --2- 172.21.15.133:0/318333312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01ec0ffc20 0x7f01ec100030 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f01dc009990 tx=0x7f01dc009ca0 comp rx=0 tx=0).stop 2023-07-14T18:34:44.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.664+0000 7f01f1dff700 1 -- 172.21.15.133:0/318333312 shutdown_connections 2023-07-14T18:34:44.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.664+0000 7f01f1dff700 1 --2- 172.21.15.133:0/318333312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ec1017a0 0x7f01ec108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:44.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.664+0000 7f01f1dff700 1 --2- 172.21.15.133:0/318333312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01ec0ffc20 0x7f01ec100030 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:44.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.664+0000 7f01f1dff700 1 --2- 172.21.15.133:0/318333312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01ec100e10 0x7f01ec101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:44.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.664+0000 7f01f1dff700 1 -- 172.21.15.133:0/318333312 >> 172.21.15.133:0/318333312 conn(0x7f01ec0fb260 msgr2=0x7f01ec0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:34:44.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.664+0000 7f01f1dff700 1 -- 172.21.15.133:0/318333312 shutdown_connections 2023-07-14T18:34:44.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.664+0000 7f01f1dff700 1 -- 172.21.15.133:0/318333312 wait complete. 2023-07-14T18:34:44.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.665+0000 7f01f1dff700 1 Processor -- start 2023-07-14T18:34:44.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.665+0000 7f01f1dff700 1 -- start start 2023-07-14T18:34:44.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.665+0000 7f01f1dff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01ec0ffc20 0x7f01ec197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:34:44.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.665+0000 7f01f1dff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01ec100e10 0x7f01ec1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:34:44.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.665+0000 7f01f1dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ec1017a0 0x7f01ec19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:34:44.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.665+0000 7f01f1dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec19f6e0 con 0x7f01ec1017a0 2023-07-14T18:34:44.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.665+0000 7f01f1dff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec19f820 con 0x7f01ec100e10 2023-07-14T18:34:44.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.665+0000 7f01f1dff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec19f9b0 con 0x7f01ec0ffc20 2023-07-14T18:34:44.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.666+0000 7f01eb7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01ec0ffc20 0x7f01ec197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:34:44.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.666+0000 7f01eaffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01ec100e10 0x7f01ec1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:34:44.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.666+0000 7f01eb7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01ec0ffc20 0x7f01ec197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:51648/0 (socket says 172.21.15.133:51648) 2023-07-14T18:34:44.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.666+0000 7f01eb7fe700 1 -- 172.21.15.133:0/3247759941 learned_addr learned my addr 172.21.15.133:0/3247759941 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:34:44.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.666+0000 7f01ebfff700 1 --2- 172.21.15.133:0/3247759941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ec1017a0 0x7f01ec19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:34:44.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.666+0000 7f01eb7fe700 1 -- 172.21.15.133:0/3247759941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01ec100e10 msgr2=0x7f01ec1980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:44.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.666+0000 7f01eb7fe700 1 --2- 172.21.15.133:0/3247759941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01ec100e10 0x7f01ec1980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:44.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.666+0000 7f01eb7fe700 1 -- 172.21.15.133:0/3247759941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ec1017a0 msgr2=0x7f01ec19f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:44.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.666+0000 7f01eb7fe700 1 --2- 172.21.15.133:0/3247759941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ec1017a0 0x7f01ec19f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:44.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.666+0000 7f01eb7fe700 1 -- 172.21.15.133:0/3247759941 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01dc0095f0 con 0x7f01ec0ffc20 2023-07-14T18:34:44.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.666+0000 7f01eaffd700 1 --2- 172.21.15.133:0/3247759941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01ec100e10 0x7f01ec1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:34:44.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.666+0000 7f01eb7fe700 1 --2- 172.21.15.133:0/3247759941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01ec0ffc20 0x7f01ec197b60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f01dc004710 tx=0x7f01dc009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:34:44.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.666+0000 7f01e8ff9700 1 -- 172.21.15.133:0/3247759941 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc01d070 con 0x7f01ec0ffc20 2023-07-14T18:34:44.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.666+0000 7f01e8ff9700 1 -- 172.21.15.133:0/3247759941 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01dc021410 con 0x7f01ec0ffc20 2023-07-14T18:34:44.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.666+0000 7f01f1dff700 1 -- 172.21.15.133:0/3247759941 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ec19fc30 con 0x7f01ec0ffc20 2023-07-14T18:34:44.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.667+0000 7f01e8ff9700 1 -- 172.21.15.133:0/3247759941 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc017650 con 0x7f01ec0ffc20 2023-07-14T18:34:44.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.667+0000 7f01f1dff700 1 -- 172.21.15.133:0/3247759941 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ec1a00b0 con 0x7f01ec0ffc20 2023-07-14T18:34:44.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.668+0000 7f01f1dff700 1 -- 172.21.15.133:0/3247759941 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ec10b3e0 con 0x7f01ec0ffc20 2023-07-14T18:34:44.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.669+0000 7f01e8ff9700 1 -- 172.21.15.133:0/3247759941 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f01dc021580 con 0x7f01ec0ffc20 2023-07-14T18:34:44.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.671+0000 7f01e8ff9700 1 --2- 172.21.15.133:0/3247759941 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f01d40a1150 0x7f01d40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:34:44.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.671+0000 7f01e8ff9700 1 -- 172.21.15.133:0/3247759941 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f01dc0cdfd0 con 0x7f01ec0ffc20 2023-07-14T18:34:44.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.671+0000 7f01eaffd700 1 --2- 172.21.15.133:0/3247759941 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f01d40a1150 0x7f01d40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:34:44.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.672+0000 7f01eaffd700 1 --2- 172.21.15.133:0/3247759941 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f01d40a1150 0x7f01d40a3600 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f01e0011490 tx=0x7f01e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:34:44.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.674+0000 7f01e8ff9700 1 -- 172.21.15.133:0/3247759941 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f01dc097420 con 0x7f01ec0ffc20 2023-07-14T18:34:44.825 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:44 smithi161 ceph-mon[128993]: pgmap v889: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.6 MiB/s wr, 701 op/s 2023-07-14T18:34:44.826 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:44 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3297102629' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:34:44.826 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:44 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3759286489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:34:44.826 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:44 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2949096540' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:34:44.841 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:44 smithi171 ceph-mon[129241]: pgmap v889: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.6 MiB/s wr, 701 op/s 2023-07-14T18:34:44.841 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:44 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3297102629' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:34:44.841 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:44 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3759286489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:34:44.841 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:44 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2949096540' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:34:44.881 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:44 smithi133 ceph-mon[126384]: pgmap v889: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.6 MiB/s wr, 701 op/s 2023-07-14T18:34:44.881 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:44 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3297102629' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:34:44.882 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:44 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3759286489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:34:44.882 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:44 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2949096540' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:34:44.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.878+0000 7f01f1dff700 1 -- 172.21.15.133:0/3247759941 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f01ec1a0660 con 0x7f01ec0ffc20 2023-07-14T18:34:44.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.879+0000 7f01e8ff9700 1 -- 172.21.15.133:0/3247759941 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f01dc026020 con 0x7f01ec0ffc20 2023-07-14T18:34:44.898 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2023-07-14T18:34:44.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.881+0000 7f01f1dff700 1 -- 172.21.15.133:0/3247759941 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f01d40a1150 msgr2=0x7f01d40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:44.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.881+0000 7f01f1dff700 1 --2- 172.21.15.133:0/3247759941 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f01d40a1150 0x7f01d40a3600 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f01e0011490 tx=0x7f01e0009260 comp rx=0 tx=0).stop 2023-07-14T18:34:44.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.881+0000 7f01f1dff700 1 -- 172.21.15.133:0/3247759941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01ec0ffc20 msgr2=0x7f01ec197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:44.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.881+0000 7f01f1dff700 1 --2- 172.21.15.133:0/3247759941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01ec0ffc20 0x7f01ec197b60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f01dc004710 tx=0x7f01dc009580 comp rx=0 tx=0).stop 2023-07-14T18:34:44.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.881+0000 7f01f1dff700 1 -- 172.21.15.133:0/3247759941 shutdown_connections 2023-07-14T18:34:44.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.881+0000 7f01f1dff700 1 --2- 172.21.15.133:0/3247759941 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f01d40a1150 0x7f01d40a3600 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:44.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.881+0000 7f01f1dff700 1 --2- 172.21.15.133:0/3247759941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ec1017a0 0x7f01ec19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:44.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.881+0000 7f01f1dff700 1 --2- 172.21.15.133:0/3247759941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01ec0ffc20 0x7f01ec197b60 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:44.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.881+0000 7f01f1dff700 1 --2- 172.21.15.133:0/3247759941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01ec100e10 0x7f01ec1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:44.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.881+0000 7f01f1dff700 1 -- 172.21.15.133:0/3247759941 >> 172.21.15.133:0/3247759941 conn(0x7f01ec0fb260 msgr2=0x7f01ec0fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:34:44.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.881+0000 7f01f1dff700 1 -- 172.21.15.133:0/3247759941 shutdown_connections 2023-07-14T18:34:44.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:44.881+0000 7f01f1dff700 1 -- 172.21.15.133:0/3247759941 wait complete. 2023-07-14T18:34:44.920 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T18:34:45.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.103+0000 7f03477ff700 1 -- 172.21.15.133:0/2074494160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03400ffc20 msgr2=0x7f0340100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:45.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.103+0000 7f03477ff700 1 --2- 172.21.15.133:0/2074494160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03400ffc20 0x7f0340100030 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f033c009990 tx=0x7f033c009ca0 comp rx=0 tx=0).stop 2023-07-14T18:34:45.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.104+0000 7f03477ff700 1 -- 172.21.15.133:0/2074494160 shutdown_connections 2023-07-14T18:34:45.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.104+0000 7f03477ff700 1 --2- 172.21.15.133:0/2074494160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03400ffc20 0x7f0340100030 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:45.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.104+0000 7f03477ff700 1 --2- 172.21.15.133:0/2074494160 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03401017a0 0x7f0340108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:45.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.104+0000 7f03477ff700 1 --2- 172.21.15.133:0/2074494160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0340100e10 0x7f0340101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:45.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.104+0000 7f03477ff700 1 -- 172.21.15.133:0/2074494160 >> 172.21.15.133:0/2074494160 conn(0x7f03400fb260 msgr2=0x7f03400fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:34:45.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.104+0000 7f03477ff700 1 -- 172.21.15.133:0/2074494160 shutdown_connections 2023-07-14T18:34:45.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.104+0000 7f03477ff700 1 -- 172.21.15.133:0/2074494160 wait complete. 2023-07-14T18:34:45.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.105+0000 7f03477ff700 1 Processor -- start 2023-07-14T18:34:45.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.105+0000 7f03477ff700 1 -- start start 2023-07-14T18:34:45.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.105+0000 7f03477ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03400ffc20 0x7f0340197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:34:45.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.105+0000 7f03477ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0340100e10 0x7f0340198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:34:45.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.105+0000 7f03477ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03401017a0 0x7f034019f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:34:45.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.105+0000 7f03477ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034019f750 con 0x7f0340100e10 2023-07-14T18:34:45.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.105+0000 7f03477ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034019f890 con 0x7f03401017a0 2023-07-14T18:34:45.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.105+0000 7f03477ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034019fa20 con 0x7f03400ffc20 2023-07-14T18:34:45.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.105+0000 7f034559b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03400ffc20 0x7f0340197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:34:45.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.105+0000 7f034559b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03400ffc20 0x7f0340197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:51662/0 (socket says 172.21.15.133:51662) 2023-07-14T18:34:45.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.105+0000 7f034559b700 1 -- 172.21.15.133:0/3489409416 learned_addr learned my addr 172.21.15.133:0/3489409416 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:34:45.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.105+0000 7f0345d9c700 1 --2- 172.21.15.133:0/3489409416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03401017a0 0x7f034019f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:34:45.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.105+0000 7f0344d9a700 1 --2- 172.21.15.133:0/3489409416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0340100e10 0x7f0340198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:34:45.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.106+0000 7f034559b700 1 -- 172.21.15.133:0/3489409416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03401017a0 msgr2=0x7f034019f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:45.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.106+0000 7f034559b700 1 --2- 172.21.15.133:0/3489409416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03401017a0 0x7f034019f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:45.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.106+0000 7f034559b700 1 -- 172.21.15.133:0/3489409416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0340100e10 msgr2=0x7f0340198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:45.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.106+0000 7f034559b700 1 --2- 172.21.15.133:0/3489409416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0340100e10 0x7f0340198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:45.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.106+0000 7f034559b700 1 -- 172.21.15.133:0/3489409416 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0338009c40 con 0x7f03400ffc20 2023-07-14T18:34:45.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.106+0000 7f034559b700 1 --2- 172.21.15.133:0/3489409416 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03400ffc20 0x7f0340197bd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f033c005740 tx=0x7f033c009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:34:45.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.106+0000 7f03367fc700 1 -- 172.21.15.133:0/3489409416 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c01d070 con 0x7f03400ffc20 2023-07-14T18:34:45.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.106+0000 7f03477ff700 1 -- 172.21.15.133:0/3489409416 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f033c0095f0 con 0x7f03400ffc20 2023-07-14T18:34:45.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.106+0000 7f03367fc700 1 -- 172.21.15.133:0/3489409416 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f033c022470 con 0x7f03400ffc20 2023-07-14T18:34:45.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.106+0000 7f03367fc700 1 -- 172.21.15.133:0/3489409416 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c017650 con 0x7f03400ffc20 2023-07-14T18:34:45.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.106+0000 7f03477ff700 1 -- 172.21.15.133:0/3489409416 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034019ff90 con 0x7f03400ffc20 2023-07-14T18:34:45.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.107+0000 7f03477ff700 1 -- 172.21.15.133:0/3489409416 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0340192300 con 0x7f03400ffc20 2023-07-14T18:34:45.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.109+0000 7f03367fc700 1 -- 172.21.15.133:0/3489409416 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f033c022850 con 0x7f03400ffc20 2023-07-14T18:34:45.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.110+0000 7f03367fc700 1 --2- 172.21.15.133:0/3489409416 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f032c0a1150 0x7f032c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:34:45.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.110+0000 7f03367fc700 1 -- 172.21.15.133:0/3489409416 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f033c0ce340 con 0x7f03400ffc20 2023-07-14T18:34:45.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.111+0000 7f0344d9a700 1 --2- 172.21.15.133:0/3489409416 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f032c0a1150 0x7f032c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:34:45.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.111+0000 7f0344d9a700 1 --2- 172.21.15.133:0/3489409416 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f032c0a1150 0x7f032c0a3600 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0330009bf0 tx=0x7f0330009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:34:45.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.113+0000 7f03367fc700 1 -- 172.21.15.133:0/3489409416 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f033c097790 con 0x7f03400ffc20 2023-07-14T18:34:45.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.314+0000 7f03477ff700 1 -- 172.21.15.133:0/3489409416 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f034002cd60 con 0x7f03400ffc20 2023-07-14T18:34:45.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.315+0000 7f03367fc700 1 -- 172.21.15.133:0/3489409416 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f034002cd60 con 0x7f03400ffc20 2023-07-14T18:34:45.335 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-07-14T18:34:45.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.317+0000 7f03477ff700 1 -- 172.21.15.133:0/3489409416 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f032c0a1150 msgr2=0x7f032c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:45.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.317+0000 7f03477ff700 1 --2- 172.21.15.133:0/3489409416 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f032c0a1150 0x7f032c0a3600 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0330009bf0 tx=0x7f0330009260 comp rx=0 tx=0).stop 2023-07-14T18:34:45.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.317+0000 7f03477ff700 1 -- 172.21.15.133:0/3489409416 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03400ffc20 msgr2=0x7f0340197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:45.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.317+0000 7f03477ff700 1 --2- 172.21.15.133:0/3489409416 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03400ffc20 0x7f0340197bd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f033c005740 tx=0x7f033c009550 comp rx=0 tx=0).stop 2023-07-14T18:34:45.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.318+0000 7f03477ff700 1 -- 172.21.15.133:0/3489409416 shutdown_connections 2023-07-14T18:34:45.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.318+0000 7f03477ff700 1 --2- 172.21.15.133:0/3489409416 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f032c0a1150 0x7f032c0a3600 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:45.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.318+0000 7f03477ff700 1 --2- 172.21.15.133:0/3489409416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03401017a0 0x7f034019f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:45.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.318+0000 7f03477ff700 1 --2- 172.21.15.133:0/3489409416 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03400ffc20 0x7f0340197bd0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:45.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.318+0000 7f03477ff700 1 --2- 172.21.15.133:0/3489409416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0340100e10 0x7f0340198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:45.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.318+0000 7f03477ff700 1 -- 172.21.15.133:0/3489409416 >> 172.21.15.133:0/3489409416 conn(0x7f03400fb260 msgr2=0x7f0340106740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:34:45.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.318+0000 7f03477ff700 1 -- 172.21.15.133:0/3489409416 shutdown_connections 2023-07-14T18:34:45.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.318+0000 7f03477ff700 1 -- 172.21.15.133:0/3489409416 wait complete. 2023-07-14T18:34:45.356 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-07-14T18:34:45.554 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.534+0000 7fc58a1ff700 1 -- 172.21.15.133:0/2255980257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc584073040 msgr2=0x7fc5840734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:45.554 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.534+0000 7fc58a1ff700 1 --2- 172.21.15.133:0/2255980257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc584073040 0x7fc5840734b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc578009990 tx=0x7fc578009ca0 comp rx=0 tx=0).stop 2023-07-14T18:34:45.554 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.534+0000 7fc58a1ff700 1 -- 172.21.15.133:0/2255980257 shutdown_connections 2023-07-14T18:34:45.555 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.534+0000 7fc58a1ff700 1 --2- 172.21.15.133:0/2255980257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc584107aa0 0x7fc584109f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:45.555 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.534+0000 7fc58a1ff700 1 --2- 172.21.15.133:0/2255980257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc584074620 0x7fc584072a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:45.555 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.534+0000 7fc58a1ff700 1 --2- 172.21.15.133:0/2255980257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc584073040 0x7fc5840734b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:45.555 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.534+0000 7fc58a1ff700 1 -- 172.21.15.133:0/2255980257 >> 172.21.15.133:0/2255980257 conn(0x7fc5840fb230 msgr2=0x7fc5840fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:34:45.555 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.534+0000 7fc58a1ff700 1 -- 172.21.15.133:0/2255980257 shutdown_connections 2023-07-14T18:34:45.555 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.534+0000 7fc58a1ff700 1 -- 172.21.15.133:0/2255980257 wait complete. 2023-07-14T18:34:45.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.535+0000 7fc58a1ff700 1 Processor -- start 2023-07-14T18:34:45.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.536+0000 7fc58a1ff700 1 -- start start 2023-07-14T18:34:45.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.536+0000 7fc58a1ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc584073040 0x7fc584197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:34:45.558 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.536+0000 7fc58a1ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc584074620 0x7fc584198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:34:45.558 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.536+0000 7fc58a1ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc584107aa0 0x7fc58419f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:34:45.558 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.536+0000 7fc58a1ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58419f8c0 con 0x7fc584074620 2023-07-14T18:34:45.558 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.536+0000 7fc58a1ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58419fa00 con 0x7fc584107aa0 2023-07-14T18:34:45.558 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.536+0000 7fc58a1ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58419fb90 con 0x7fc584073040 2023-07-14T18:34:45.558 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.536+0000 7fc5837fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc584073040 0x7fc584197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:34:45.558 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.536+0000 7fc582ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc584074620 0x7fc584198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:34:45.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.536+0000 7fc583fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc584107aa0 0x7fc58419f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:34:45.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.536+0000 7fc582ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc584074620 0x7fc584198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41174/0 (socket says 172.21.15.133:41174) 2023-07-14T18:34:45.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.536+0000 7fc582ffd700 1 -- 172.21.15.133:0/2958468735 learned_addr learned my addr 172.21.15.133:0/2958468735 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:34:45.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.536+0000 7fc5837fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc584073040 0x7fc584197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:51676/0 (socket says 172.21.15.133:51676) 2023-07-14T18:34:45.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.537+0000 7fc583fff700 1 -- 172.21.15.133:0/2958468735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc584073040 msgr2=0x7fc584197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:45.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.537+0000 7fc583fff700 1 --2- 172.21.15.133:0/2958468735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc584073040 0x7fc584197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:45.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.537+0000 7fc583fff700 1 -- 172.21.15.133:0/2958468735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc584074620 msgr2=0x7fc584198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:45.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.537+0000 7fc583fff700 1 --2- 172.21.15.133:0/2958468735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc584074620 0x7fc584198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:45.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.537+0000 7fc583fff700 1 -- 172.21.15.133:0/2958468735 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5780095f0 con 0x7fc584107aa0 2023-07-14T18:34:45.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.537+0000 7fc5837fe700 1 --2- 172.21.15.133:0/2958468735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc584073040 0x7fc584197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:34:45.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.537+0000 7fc582ffd700 1 --2- 172.21.15.133:0/2958468735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc584074620 0x7fc584198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:34:45.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.537+0000 7fc583fff700 1 --2- 172.21.15.133:0/2958468735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc584107aa0 0x7fc58419f2f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00efc0 tx=0x7fc57c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:34:45.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.537+0000 7fc580ff9700 1 -- 172.21.15.133:0/2958468735 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c018070 con 0x7fc584107aa0 2023-07-14T18:34:45.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.537+0000 7fc580ff9700 1 -- 172.21.15.133:0/2958468735 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc57c00f040 con 0x7fc584107aa0 2023-07-14T18:34:45.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.537+0000 7fc58a1ff700 1 -- 172.21.15.133:0/2958468735 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc58419fe10 con 0x7fc584107aa0 2023-07-14T18:34:45.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.537+0000 7fc580ff9700 1 -- 172.21.15.133:0/2958468735 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c00cd40 con 0x7fc584107aa0 2023-07-14T18:34:45.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.537+0000 7fc58a1ff700 1 -- 172.21.15.133:0/2958468735 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5841a0240 con 0x7fc584107aa0 2023-07-14T18:34:45.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.538+0000 7fc58a1ff700 1 -- 172.21.15.133:0/2958468735 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc58410d610 con 0x7fc584107aa0 2023-07-14T18:34:45.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.540+0000 7fc580ff9700 1 -- 172.21.15.133:0/2958468735 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fc57c003a00 con 0x7fc584107aa0 2023-07-14T18:34:45.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.541+0000 7fc580ff9700 1 --2- 172.21.15.133:0/2958468735 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc56c0a11a0 0x7fc56c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:34:45.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.541+0000 7fc580ff9700 1 -- 172.21.15.133:0/2958468735 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc57c0cc880 con 0x7fc584107aa0 2023-07-14T18:34:45.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.541+0000 7fc5837fe700 1 --2- 172.21.15.133:0/2958468735 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc56c0a11a0 0x7fc56c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:34:45.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.542+0000 7fc5837fe700 1 --2- 172.21.15.133:0/2958468735 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc56c0a11a0 0x7fc56c0a3650 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc574011490 tx=0x7fc574009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:34:45.566 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.547+0000 7fc580ff9700 1 -- 172.21.15.133:0/2958468735 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fc57c095cd0 con 0x7fc584107aa0 2023-07-14T18:34:45.631 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:45 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3247759941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:34:45.631 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:45 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3489409416' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:34:45.821 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.802+0000 7fc58a1ff700 1 -- 172.21.15.133:0/2958468735 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7fc5841a0ab0 con 0x7fc584107aa0 2023-07-14T18:34:45.825 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:45 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3247759941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:34:45.825 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:45 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3489409416' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:34:45.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.811+0000 7fc580ff9700 1 -- 172.21.15.133:0/2958468735 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+733247 (secure 0 0 0) 0x7fc57c0992f0 con 0x7fc584107aa0 2023-07-14T18:34:45.841 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:34:45.842 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:34:45.842 INFO:teuthology.orchestra.run.smithi133.stdout: "epoch": 18, 2023-07-14T18:34:45.842 INFO:teuthology.orchestra.run.smithi133.stdout: "active_gid": 14152, 2023-07-14T18:34:45.842 INFO:teuthology.orchestra.run.smithi133.stdout: "active_name": "x", 2023-07-14T18:34:45.842 INFO:teuthology.orchestra.run.smithi133.stdout: "active_addrs": { 2023-07-14T18:34:45.842 INFO:teuthology.orchestra.run.smithi133.stdout: "addrvec": [ 2023-07-14T18:34:45.843 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:45.843 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v2", 2023-07-14T18:34:45.843 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:6800", 2023-07-14T18:34:45.843 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 196197529 2023-07-14T18:34:45.843 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.843 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:45.843 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2023-07-14T18:34:45.844 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:6801", 2023-07-14T18:34:45.844 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 196197529 2023-07-14T18:34:45.844 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:45.844 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2023-07-14T18:34:45.844 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.844 INFO:teuthology.orchestra.run.smithi133.stdout: "active_addr": "172.21.15.133:6801/196197529", 2023-07-14T18:34:45.845 INFO:teuthology.orchestra.run.smithi133.stdout: "active_change": "2023-07-14T18:07:47.346809+0000", 2023-07-14T18:34:45.845 INFO:teuthology.orchestra.run.smithi133.stdout: "active_mgr_features": 4540138322906710015, 2023-07-14T18:34:45.845 INFO:teuthology.orchestra.run.smithi133.stdout: "available": true, 2023-07-14T18:34:45.845 INFO:teuthology.orchestra.run.smithi133.stdout: "standbys": [ 2023-07-14T18:34:45.845 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:45.845 INFO:teuthology.orchestra.run.smithi133.stdout: "gid": 24109, 2023-07-14T18:34:45.845 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "z", 2023-07-14T18:34:45.846 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr_features": 4540138322906710015, 2023-07-14T18:34:45.846 INFO:teuthology.orchestra.run.smithi133.stdout: "available_modules": [ 2023-07-14T18:34:45.846 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:45.846 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "alerts", 2023-07-14T18:34:45.846 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:45.846 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:45.846 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:45.847 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-07-14T18:34:45.847 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-07-14T18:34:45.847 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:45.847 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.847 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.847 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-07-14T18:34:45.847 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.847 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.848 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.848 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How frequently to reexamine health status", 2023-07-14T18:34:45.848 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.848 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.848 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.848 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.848 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:45.849 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:45.849 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.849 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.849 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.849 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:45.849 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.849 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.849 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:45.850 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:45.850 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:45.850 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:45.850 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:45.850 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:45.850 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:45.850 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:45.851 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:45.851 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.851 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.851 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.851 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.851 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:45.851 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:45.851 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:45.852 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.852 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.852 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:45.852 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.852 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.852 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.852 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:45.852 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.853 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.853 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.853 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.853 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:45.853 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:45.853 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.853 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.853 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.854 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:45.854 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.854 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.854 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:45.854 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:45.854 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:45.854 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:45.854 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:45.855 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:45.855 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:45.855 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:45.855 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:45.855 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.855 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.855 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.856 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.856 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:45.856 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:45.856 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:45.856 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.856 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.856 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:45.857 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.857 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.857 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.857 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:45.857 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.857 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.857 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.858 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.858 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_destination": { 2023-07-14T18:34:45.858 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_destination", 2023-07-14T18:34:45.858 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.858 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.858 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.858 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:45.858 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.859 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.859 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.859 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Email address to send alerts to", 2023-07-14T18:34:45.859 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.859 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.859 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.859 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.859 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_from_name": { 2023-07-14T18:34:45.860 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_from_name", 2023-07-14T18:34:45.860 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.860 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.860 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.860 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "Ceph", 2023-07-14T18:34:45.860 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.860 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.860 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.861 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Email From: name", 2023-07-14T18:34:45.861 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.861 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.861 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.861 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.861 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_host": { 2023-07-14T18:34:45.862 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_host", 2023-07-14T18:34:45.862 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.862 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.862 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.862 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:45.862 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.862 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.862 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.863 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP server", 2023-07-14T18:34:45.863 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.863 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.863 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.863 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.863 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_password": { 2023-07-14T18:34:45.863 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_password", 2023-07-14T18:34:45.863 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.864 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.864 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.864 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:45.864 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.864 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.864 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.864 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Password to authenticate with", 2023-07-14T18:34:45.864 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.865 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.865 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.865 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.865 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_port": { 2023-07-14T18:34:45.865 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_port", 2023-07-14T18:34:45.865 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:45.865 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.865 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.866 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "465", 2023-07-14T18:34:45.866 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.866 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.866 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.866 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP port", 2023-07-14T18:34:45.866 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.866 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.866 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.867 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.867 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_sender": { 2023-07-14T18:34:45.867 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_sender", 2023-07-14T18:34:45.867 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.867 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.867 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.867 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:45.867 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.868 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.868 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.868 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP envelope sender", 2023-07-14T18:34:45.868 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.868 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.868 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.869 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.869 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_ssl": { 2023-07-14T18:34:45.869 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_ssl", 2023-07-14T18:34:45.869 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:45.869 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.869 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.869 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:45.870 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.870 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.870 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.870 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-14T18:34:45.870 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.870 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.870 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.871 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.871 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_user": { 2023-07-14T18:34:45.871 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_user", 2023-07-14T18:34:45.871 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.871 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.871 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.871 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:45.871 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.872 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.872 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.872 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "User to authenticate as", 2023-07-14T18:34:45.872 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.872 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.872 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.872 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:45.873 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:45.873 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.873 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:45.873 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "balancer", 2023-07-14T18:34:45.873 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:45.873 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:45.873 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:45.873 INFO:teuthology.orchestra.run.smithi133.stdout: "active": { 2023-07-14T18:34:45.874 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "active", 2023-07-14T18:34:45.874 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:45.874 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.874 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.874 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:45.874 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.874 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.874 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.875 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "automatically balance PGs across cluster", 2023-07-14T18:34:45.875 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.875 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.875 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.875 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.875 INFO:teuthology.orchestra.run.smithi133.stdout: "begin_time": { 2023-07-14T18:34:45.875 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "begin_time", 2023-07-14T18:34:45.875 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.876 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.876 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.876 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0000", 2023-07-14T18:34:45.876 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.876 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.876 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.876 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "beginning time of day to automatically balance", 2023-07-14T18:34:45.877 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-14T18:34:45.877 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.877 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.877 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.877 INFO:teuthology.orchestra.run.smithi133.stdout: "begin_weekday": { 2023-07-14T18:34:45.877 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "begin_weekday", 2023-07-14T18:34:45.877 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-07-14T18:34:45.878 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.878 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.878 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-07-14T18:34:45.878 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0", 2023-07-14T18:34:45.878 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "6", 2023-07-14T18:34:45.878 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.878 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-14T18:34:45.879 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-14T18:34:45.879 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.879 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.879 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.879 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_max_iterations": { 2023-07-14T18:34:45.879 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_max_iterations", 2023-07-14T18:34:45.880 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-07-14T18:34:45.880 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.880 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.880 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "25", 2023-07-14T18:34:45.880 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2023-07-14T18:34:45.880 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "250", 2023-07-14T18:34:45.880 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.881 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-14T18:34:45.881 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.881 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.881 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.881 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.881 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_metrics": { 2023-07-14T18:34:45.881 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_metrics", 2023-07-14T18:34:45.881 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.882 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.882 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.882 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "pgs,objects,bytes", 2023-07-14T18:34:45.882 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.882 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.882 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.883 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-14T18:34:45.883 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-14T18:34:45.883 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.883 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.883 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.884 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_step": { 2023-07-14T18:34:45.884 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_step", 2023-07-14T18:34:45.884 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-07-14T18:34:45.884 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.884 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.884 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0.5", 2023-07-14T18:34:45.884 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0.001", 2023-07-14T18:34:45.884 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "0.999", 2023-07-14T18:34:45.885 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.885 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "aggressiveness of optimization", 2023-07-14T18:34:45.885 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-14T18:34:45.885 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.885 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.885 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.885 INFO:teuthology.orchestra.run.smithi133.stdout: "end_time": { 2023-07-14T18:34:45.886 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "end_time", 2023-07-14T18:34:45.886 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.886 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.886 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.886 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "2359", 2023-07-14T18:34:45.886 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.886 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.887 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.887 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "ending time of day to automatically balance", 2023-07-14T18:34:45.887 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-14T18:34:45.887 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.887 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.887 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.887 INFO:teuthology.orchestra.run.smithi133.stdout: "end_weekday": { 2023-07-14T18:34:45.887 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "end_weekday", 2023-07-14T18:34:45.888 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-07-14T18:34:45.888 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.888 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.888 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-07-14T18:34:45.888 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0", 2023-07-14T18:34:45.888 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "6", 2023-07-14T18:34:45.888 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.889 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-14T18:34:45.889 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-14T18:34:45.889 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.889 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.889 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.889 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:45.889 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:45.889 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.890 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.890 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.890 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:45.890 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.890 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.890 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:45.890 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:45.890 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:45.891 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:45.891 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:45.891 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:45.891 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:45.891 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:45.891 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:45.891 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.891 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.892 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.892 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.892 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:45.892 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:45.892 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:45.892 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.892 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.892 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:45.893 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.893 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.893 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.893 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:45.893 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.893 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.894 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.894 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.894 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:45.894 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:45.894 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.894 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.894 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.894 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:45.895 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.895 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.895 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:45.895 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:45.895 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:45.896 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:45.896 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:45.896 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:45.896 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:45.896 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:45.896 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:45.896 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.897 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.897 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.897 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.897 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:45.897 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:45.897 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:45.897 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.898 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.898 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:45.898 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.898 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.898 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.898 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:45.898 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.898 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.899 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.899 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.899 INFO:teuthology.orchestra.run.smithi133.stdout: "min_score": { 2023-07-14T18:34:45.899 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "min_score", 2023-07-14T18:34:45.899 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-07-14T18:34:45.899 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.899 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.899 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-07-14T18:34:45.900 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.900 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.900 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.900 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-14T18:34:45.900 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.901 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.901 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.901 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.901 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": { 2023-07-14T18:34:45.901 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mode", 2023-07-14T18:34:45.901 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.901 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.902 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.902 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "upmap", 2023-07-14T18:34:45.902 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.902 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.902 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:45.902 INFO:teuthology.orchestra.run.smithi133.stdout: "crush-compat", 2023-07-14T18:34:45.902 INFO:teuthology.orchestra.run.smithi133.stdout: "none", 2023-07-14T18:34:45.902 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap" 2023-07-14T18:34:45.903 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:45.903 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Balancer mode", 2023-07-14T18:34:45.903 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.903 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.903 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.903 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.903 INFO:teuthology.orchestra.run.smithi133.stdout: "pool_ids": { 2023-07-14T18:34:45.903 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pool_ids", 2023-07-14T18:34:45.904 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.904 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.904 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.904 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:45.904 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.904 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.904 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.904 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-14T18:34:45.905 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.905 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.905 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.905 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.905 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-07-14T18:34:45.905 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-07-14T18:34:45.905 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:45.906 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.906 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.906 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-07-14T18:34:45.906 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.906 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.906 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.906 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-14T18:34:45.906 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.907 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.907 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.907 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.907 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap_max_deviation": { 2023-07-14T18:34:45.907 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "upmap_max_deviation", 2023-07-14T18:34:45.907 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:45.907 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.907 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.908 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2023-07-14T18:34:45.908 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2023-07-14T18:34:45.908 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.908 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.908 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-14T18:34:45.908 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-14T18:34:45.908 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.908 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.909 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.909 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap_max_optimizations": { 2023-07-14T18:34:45.909 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "upmap_max_optimizations", 2023-07-14T18:34:45.909 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-07-14T18:34:45.909 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.909 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.909 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-07-14T18:34:45.909 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.910 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.910 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.910 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-14T18:34:45.910 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.910 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.910 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.910 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:45.911 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:45.911 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.911 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:45.911 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cephadm", 2023-07-14T18:34:45.911 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:45.911 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:45.911 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:45.911 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_down_multiplier": { 2023-07-14T18:34:45.912 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_down_multiplier", 2023-07-14T18:34:45.912 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-07-14T18:34:45.912 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.912 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.912 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3.0", 2023-07-14T18:34:45.912 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.912 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.912 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.913 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-14T18:34:45.913 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.913 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.913 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.913 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.913 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_refresh_rate": { 2023-07-14T18:34:45.913 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_refresh_rate", 2023-07-14T18:34:45.913 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:45.914 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.914 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.914 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "20", 2023-07-14T18:34:45.914 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.914 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.914 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.914 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-14T18:34:45.915 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.915 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.915 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.915 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.915 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_starting_port": { 2023-07-14T18:34:45.915 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_starting_port", 2023-07-14T18:34:45.915 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:45.916 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.916 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.916 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "4721", 2023-07-14T18:34:45.916 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.916 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.916 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.917 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-14T18:34:45.917 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.917 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.917 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.917 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.918 INFO:teuthology.orchestra.run.smithi133.stdout: "alertmanager_web_password": { 2023-07-14T18:34:45.918 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "alertmanager_web_password", 2023-07-14T18:34:45.918 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.918 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.919 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.919 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-07-14T18:34:45.919 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.919 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.920 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.920 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Alertmanager web password", 2023-07-14T18:34:45.920 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.921 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.921 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.921 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.921 INFO:teuthology.orchestra.run.smithi133.stdout: "alertmanager_web_user": { 2023-07-14T18:34:45.922 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "alertmanager_web_user", 2023-07-14T18:34:45.922 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.922 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.922 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.923 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-07-14T18:34:45.923 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.923 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.923 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.924 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Alertmanager web user", 2023-07-14T18:34:45.924 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.924 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.924 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.924 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.925 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_ptrace": { 2023-07-14T18:34:45.925 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_ptrace", 2023-07-14T18:34:45.925 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:45.925 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.925 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.925 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:45.925 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.925 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.926 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.926 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-14T18:34:45.926 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-14T18:34:45.926 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.926 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.926 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.926 INFO:teuthology.orchestra.run.smithi133.stdout: "autotune_interval": { 2023-07-14T18:34:45.926 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "autotune_interval", 2023-07-14T18:34:45.927 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:45.927 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.927 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.927 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-07-14T18:34:45.927 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.927 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.927 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.927 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-14T18:34:45.928 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.928 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.928 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.928 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.928 INFO:teuthology.orchestra.run.smithi133.stdout: "autotune_memory_target_ratio": { 2023-07-14T18:34:45.928 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "autotune_memory_target_ratio", 2023-07-14T18:34:45.928 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-07-14T18:34:45.929 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.929 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.929 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0.7", 2023-07-14T18:34:45.929 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.929 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.929 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.929 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-14T18:34:45.929 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.930 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.930 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.930 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.930 INFO:teuthology.orchestra.run.smithi133.stdout: "cgroups_split": { 2023-07-14T18:34:45.930 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cgroups_split", 2023-07-14T18:34:45.930 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:45.930 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.930 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.931 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:45.931 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.931 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.931 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.931 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-14T18:34:45.931 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.931 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.931 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.932 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.932 INFO:teuthology.orchestra.run.smithi133.stdout: "config_checks_enabled": { 2023-07-14T18:34:45.932 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_checks_enabled", 2023-07-14T18:34:45.932 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:45.932 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.932 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.932 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:45.932 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.933 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.933 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.933 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-14T18:34:45.933 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.933 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.933 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.934 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.934 INFO:teuthology.orchestra.run.smithi133.stdout: "config_dashboard": { 2023-07-14T18:34:45.934 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_dashboard", 2023-07-14T18:34:45.934 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:45.934 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.934 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.935 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:45.935 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.935 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.935 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.935 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-14T18:34:45.935 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.935 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.936 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.936 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.936 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_alertmanager": { 2023-07-14T18:34:45.936 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_alertmanager", 2023-07-14T18:34:45.936 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.936 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.936 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.936 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-14T18:34:45.937 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.937 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.937 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.937 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2023-07-14T18:34:45.937 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.937 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.937 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.937 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.938 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_base": { 2023-07-14T18:34:45.938 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_base", 2023-07-14T18:34:45.938 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.938 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.938 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.938 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-14T18:34:45.938 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.938 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.939 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.939 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Container image name, without the tag", 2023-07-14T18:34:45.939 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.939 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.939 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.939 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.939 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_elasticsearch": { 2023-07-14T18:34:45.939 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_elasticsearch", 2023-07-14T18:34:45.940 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.940 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.940 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.940 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-14T18:34:45.940 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.940 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.940 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.941 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "elasticsearch container image", 2023-07-14T18:34:45.941 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.941 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.941 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.941 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.941 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_grafana": { 2023-07-14T18:34:45.941 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_grafana", 2023-07-14T18:34:45.941 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.942 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.942 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.942 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-14T18:34:45.942 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.942 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.942 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.942 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2023-07-14T18:34:45.943 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.943 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.943 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.943 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.943 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_haproxy": { 2023-07-14T18:34:45.943 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_haproxy", 2023-07-14T18:34:45.943 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.944 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.944 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.944 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-14T18:34:45.944 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.944 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.944 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.944 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "HAproxy container image", 2023-07-14T18:34:45.945 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.945 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.945 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.945 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.945 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_agent": { 2023-07-14T18:34:45.945 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_agent", 2023-07-14T18:34:45.945 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.945 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.946 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.946 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-14T18:34:45.946 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.946 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.946 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.946 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger agent container image", 2023-07-14T18:34:45.946 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.946 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.947 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.947 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.947 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_collector": { 2023-07-14T18:34:45.947 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_collector", 2023-07-14T18:34:45.947 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.947 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.947 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.948 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-14T18:34:45.948 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.948 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.948 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.948 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger collector container image", 2023-07-14T18:34:45.948 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.948 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.949 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.949 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.949 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_query": { 2023-07-14T18:34:45.949 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_query", 2023-07-14T18:34:45.949 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.949 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.949 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.949 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-14T18:34:45.950 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.950 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.950 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.950 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger query container image", 2023-07-14T18:34:45.950 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.950 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.950 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.951 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.951 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_keepalived": { 2023-07-14T18:34:45.951 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_keepalived", 2023-07-14T18:34:45.951 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.951 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.951 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.951 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-14T18:34:45.952 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.952 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.952 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.952 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Keepalived container image", 2023-07-14T18:34:45.952 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.952 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.952 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.953 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.953 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_loki": { 2023-07-14T18:34:45.953 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_loki", 2023-07-14T18:34:45.953 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.953 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.953 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.954 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-14T18:34:45.954 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.954 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.954 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.954 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Loki container image", 2023-07-14T18:34:45.954 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.954 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.954 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.955 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.955 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_node_exporter": { 2023-07-14T18:34:45.955 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_node_exporter", 2023-07-14T18:34:45.955 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.955 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.955 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.955 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-14T18:34:45.955 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.956 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.956 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.956 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2023-07-14T18:34:45.956 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.956 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.956 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.956 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.957 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_prometheus": { 2023-07-14T18:34:45.957 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_prometheus", 2023-07-14T18:34:45.957 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.957 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.957 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.957 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-14T18:34:45.957 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.957 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.958 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.958 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2023-07-14T18:34:45.958 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.958 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.958 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.958 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.958 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_promtail": { 2023-07-14T18:34:45.958 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_promtail", 2023-07-14T18:34:45.959 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.959 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.959 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.959 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-14T18:34:45.959 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.959 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.959 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.959 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Promtail container image", 2023-07-14T18:34:45.960 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.960 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.960 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.960 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.960 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_snmp_gateway": { 2023-07-14T18:34:45.960 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_snmp_gateway", 2023-07-14T18:34:45.960 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.961 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.961 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.961 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-14T18:34:45.961 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.961 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.961 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.961 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SNMP Gateway container image", 2023-07-14T18:34:45.962 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.962 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.962 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.962 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.963 INFO:teuthology.orchestra.run.smithi133.stdout: "container_init": { 2023-07-14T18:34:45.963 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_init", 2023-07-14T18:34:45.963 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:45.963 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.964 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.964 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:45.964 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.965 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.965 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.965 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Run podman/docker with `--init`", 2023-07-14T18:34:45.965 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.966 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.966 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.966 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.966 INFO:teuthology.orchestra.run.smithi133.stdout: "daemon_cache_timeout": { 2023-07-14T18:34:45.966 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "daemon_cache_timeout", 2023-07-14T18:34:45.967 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:45.967 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.967 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.967 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-07-14T18:34:45.968 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.968 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.968 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.968 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-14T18:34:45.968 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.968 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.968 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.968 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.969 INFO:teuthology.orchestra.run.smithi133.stdout: "default_cephadm_command_timeout": { 2023-07-14T18:34:45.969 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "default_cephadm_command_timeout", 2023-07-14T18:34:45.969 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:45.969 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.969 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.969 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "900", 2023-07-14T18:34:45.970 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.970 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.970 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.970 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-14T18:34:45.970 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.970 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.970 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.970 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.971 INFO:teuthology.orchestra.run.smithi133.stdout: "default_registry": { 2023-07-14T18:34:45.971 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "default_registry", 2023-07-14T18:34:45.971 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.971 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.971 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.971 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io", 2023-07-14T18:34:45.971 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.971 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.972 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.972 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-14T18:34:45.972 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.972 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.972 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.972 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.972 INFO:teuthology.orchestra.run.smithi133.stdout: "device_cache_timeout": { 2023-07-14T18:34:45.972 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_cache_timeout", 2023-07-14T18:34:45.973 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:45.973 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.973 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.973 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1800", 2023-07-14T18:34:45.973 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.973 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.973 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.973 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache device inventory", 2023-07-14T18:34:45.974 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.974 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.974 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.974 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.974 INFO:teuthology.orchestra.run.smithi133.stdout: "device_enhanced_scan": { 2023-07-14T18:34:45.974 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_enhanced_scan", 2023-07-14T18:34:45.974 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:45.974 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.975 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.975 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:45.975 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.975 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.975 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.975 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-14T18:34:45.975 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.976 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.976 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.976 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.976 INFO:teuthology.orchestra.run.smithi133.stdout: "facts_cache_timeout": { 2023-07-14T18:34:45.976 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "facts_cache_timeout", 2023-07-14T18:34:45.976 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:45.976 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.976 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.977 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-07-14T18:34:45.977 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.977 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.977 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.977 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache host facts data", 2023-07-14T18:34:45.977 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.977 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.977 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.978 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.978 INFO:teuthology.orchestra.run.smithi133.stdout: "host_check_interval": { 2023-07-14T18:34:45.978 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "host_check_interval", 2023-07-14T18:34:45.978 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:45.978 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.978 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.978 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-07-14T18:34:45.978 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.979 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.979 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.979 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to perform a host check", 2023-07-14T18:34:45.979 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.979 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.979 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.979 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.979 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:45.980 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:45.980 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.980 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.980 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.980 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:45.980 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.980 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.980 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:45.981 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:45.981 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:45.981 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:45.981 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:45.981 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:45.981 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:45.981 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:45.981 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:45.982 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.982 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.982 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.982 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.982 INFO:teuthology.orchestra.run.smithi133.stdout: "log_refresh_metadata": { 2023-07-14T18:34:45.982 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_refresh_metadata", 2023-07-14T18:34:45.982 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:45.982 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.983 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.983 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:45.983 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.983 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.983 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.983 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-14T18:34:45.983 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.983 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.984 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.984 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.984 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:45.984 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:45.984 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:45.984 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.984 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.985 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:45.985 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.985 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.985 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.985 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-14T18:34:45.985 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.985 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.985 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.986 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.986 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:45.986 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:45.986 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.986 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.986 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.986 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:45.986 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.987 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.987 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:45.987 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:45.987 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:45.987 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:45.987 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:45.987 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:45.987 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:45.988 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:45.988 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:45.988 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.988 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.988 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.988 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.988 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:45.988 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:45.989 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:45.989 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.989 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:45.989 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:45.989 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.989 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.989 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.989 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:45.990 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.990 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.990 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.990 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.990 INFO:teuthology.orchestra.run.smithi133.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-14T18:34:45.990 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-14T18:34:45.990 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:45.990 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.991 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.991 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:45.991 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.991 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.991 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.991 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-14T18:34:45.992 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.992 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.992 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.992 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.992 INFO:teuthology.orchestra.run.smithi133.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-14T18:34:45.992 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-14T18:34:45.992 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.992 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.993 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.993 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "*", 2023-07-14T18:34:45.993 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.993 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.993 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.993 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-14T18:34:45.993 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.993 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.994 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.994 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.994 INFO:teuthology.orchestra.run.smithi133.stdout: "max_count_per_host": { 2023-07-14T18:34:45.994 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_count_per_host", 2023-07-14T18:34:45.994 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:45.994 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.994 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.994 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-07-14T18:34:45.995 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.995 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.995 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.995 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "max number of daemons per service per host", 2023-07-14T18:34:45.995 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.995 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.995 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.995 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.996 INFO:teuthology.orchestra.run.smithi133.stdout: "max_osd_draining_count": { 2023-07-14T18:34:45.996 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_osd_draining_count", 2023-07-14T18:34:45.996 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:45.996 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.996 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.996 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-07-14T18:34:45.996 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.996 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.997 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.997 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-14T18:34:45.997 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.997 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.997 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.997 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.997 INFO:teuthology.orchestra.run.smithi133.stdout: "migration_current": { 2023-07-14T18:34:45.997 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "migration_current", 2023-07-14T18:34:45.998 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:45.998 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:45.998 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:45.998 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:45.998 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:45.998 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:45.998 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:45.998 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "internal - do not modify", 2023-07-14T18:34:45.999 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:45.999 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:45.999 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:45.999 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:45.999 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": { 2023-07-14T18:34:45.999 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mode", 2023-07-14T18:34:45.999 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:45.999 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.000 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.000 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "root", 2023-07-14T18:34:46.000 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.000 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.000 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.000 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm-package", 2023-07-14T18:34:46.000 INFO:teuthology.orchestra.run.smithi133.stdout: "root" 2023-07-14T18:34:46.000 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.001 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "mode for remote execution of cephadm", 2023-07-14T18:34:46.001 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.001 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.001 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.001 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.001 INFO:teuthology.orchestra.run.smithi133.stdout: "prometheus_alerts_path": { 2023-07-14T18:34:46.001 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus_alerts_path", 2023-07-14T18:34:46.002 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.002 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.002 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.002 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-14T18:34:46.002 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.002 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.002 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.002 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-14T18:34:46.003 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.003 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.003 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.003 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.003 INFO:teuthology.orchestra.run.smithi133.stdout: "prometheus_web_password": { 2023-07-14T18:34:46.003 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus_web_password", 2023-07-14T18:34:46.003 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.003 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.004 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.004 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-07-14T18:34:46.004 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.004 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.004 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.004 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus web password", 2023-07-14T18:34:46.004 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.004 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.005 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.005 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.005 INFO:teuthology.orchestra.run.smithi133.stdout: "prometheus_web_user": { 2023-07-14T18:34:46.005 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus_web_user", 2023-07-14T18:34:46.005 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.005 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.005 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.005 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-07-14T18:34:46.006 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.006 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.006 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.006 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus web user", 2023-07-14T18:34:46.006 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.006 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.006 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.006 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.007 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_insecure": { 2023-07-14T18:34:46.007 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_insecure", 2023-07-14T18:34:46.007 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.007 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.007 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.007 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.007 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.008 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.008 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.008 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-14T18:34:46.008 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.008 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.008 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.008 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.008 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_password": { 2023-07-14T18:34:46.009 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_password", 2023-07-14T18:34:46.009 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.009 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.009 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.009 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.009 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.009 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.009 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.010 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-14T18:34:46.010 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.010 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.010 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.010 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.010 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_url": { 2023-07-14T18:34:46.010 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_url", 2023-07-14T18:34:46.010 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.011 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.011 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.011 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.011 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.011 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.011 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.011 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-14T18:34:46.011 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.012 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.012 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.012 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.012 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_username": { 2023-07-14T18:34:46.012 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_username", 2023-07-14T18:34:46.012 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.012 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.012 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.013 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.013 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.013 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.013 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.013 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-14T18:34:46.013 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.013 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.014 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.014 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.014 INFO:teuthology.orchestra.run.smithi133.stdout: "secure_monitoring_stack": { 2023-07-14T18:34:46.014 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "secure_monitoring_stack", 2023-07-14T18:34:46.014 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.014 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.014 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.014 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.015 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.015 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.015 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.015 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-14T18:34:46.015 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.015 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.015 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.016 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.016 INFO:teuthology.orchestra.run.smithi133.stdout: "service_discovery_port": { 2023-07-14T18:34:46.016 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "service_discovery_port", 2023-07-14T18:34:46.016 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.016 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.016 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.016 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8765", 2023-07-14T18:34:46.016 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.017 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.017 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.017 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "cephadm service discovery port", 2023-07-14T18:34:46.017 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.017 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.017 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.017 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.017 INFO:teuthology.orchestra.run.smithi133.stdout: "ssh_config_file": { 2023-07-14T18:34:46.018 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssh_config_file", 2023-07-14T18:34:46.018 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.018 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.018 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.018 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.018 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.018 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.018 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.019 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-14T18:34:46.019 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.019 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.019 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.019 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.019 INFO:teuthology.orchestra.run.smithi133.stdout: "use_agent": { 2023-07-14T18:34:46.019 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "use_agent", 2023-07-14T18:34:46.019 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.020 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.020 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.020 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.020 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.020 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.020 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.020 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-14T18:34:46.021 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.021 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.021 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.021 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.021 INFO:teuthology.orchestra.run.smithi133.stdout: "use_repo_digest": { 2023-07-14T18:34:46.021 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "use_repo_digest", 2023-07-14T18:34:46.021 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.021 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.022 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.022 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.022 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.022 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.022 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.022 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-14T18:34:46.022 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.022 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.023 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.023 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.023 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_failed_host_check": { 2023-07-14T18:34:46.023 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_failed_host_check", 2023-07-14T18:34:46.023 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.023 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.023 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.023 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.024 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.024 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.024 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.024 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if the host check fails", 2023-07-14T18:34:46.024 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.024 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.024 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.024 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.025 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_stray_daemons": { 2023-07-14T18:34:46.025 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_stray_daemons", 2023-07-14T18:34:46.025 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.025 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.025 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.025 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.025 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.025 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.026 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.026 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-14T18:34:46.026 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.026 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.026 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.026 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.026 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_stray_hosts": { 2023-07-14T18:34:46.026 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_stray_hosts", 2023-07-14T18:34:46.027 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.027 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.027 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.027 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.027 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.027 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.027 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.027 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-14T18:34:46.028 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.028 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.028 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.028 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.028 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.028 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.028 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.028 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crash", 2023-07-14T18:34:46.029 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.029 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.029 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.029 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.029 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.029 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.029 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.029 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.030 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.030 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.030 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.030 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.030 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.030 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.030 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.030 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.031 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.031 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.031 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.031 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.031 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.031 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.031 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.031 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.032 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.032 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.032 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.032 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.032 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.032 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.032 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.033 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.033 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.033 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.033 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.033 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.033 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.033 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.033 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.034 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.034 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.034 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.034 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.034 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.034 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.034 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.034 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.035 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.035 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.035 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.035 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.035 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.035 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.035 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.035 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.036 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.036 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.036 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.036 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.036 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.036 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.036 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.037 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.037 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.037 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.037 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.037 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.037 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.037 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.037 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.038 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.038 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.038 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.038 INFO:teuthology.orchestra.run.smithi133.stdout: "retain_interval": { 2023-07-14T18:34:46.038 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "retain_interval", 2023-07-14T18:34:46.038 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.038 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.038 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.039 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "31536000", 2023-07-14T18:34:46.039 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.039 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.039 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.039 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-14T18:34:46.039 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.039 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.039 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.040 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.040 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_recent_interval": { 2023-07-14T18:34:46.040 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_recent_interval", 2023-07-14T18:34:46.040 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.040 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.040 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.040 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1209600", 2023-07-14T18:34:46.040 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.041 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.041 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.041 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-14T18:34:46.041 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.041 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.041 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.041 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.041 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.042 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.042 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.042 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "dashboard", 2023-07-14T18:34:46.042 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.042 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.042 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.042 INFO:teuthology.orchestra.run.smithi133.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-14T18:34:46.042 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-14T18:34:46.043 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.043 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.043 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.043 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-07-14T18:34:46.043 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.043 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.043 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.043 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.044 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.044 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.044 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.044 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.044 INFO:teuthology.orchestra.run.smithi133.stdout: "ALERTMANAGER_API_HOST": { 2023-07-14T18:34:46.044 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-14T18:34:46.044 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.045 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.045 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.045 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.045 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.045 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.045 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.045 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.045 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.046 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.046 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.046 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.046 INFO:teuthology.orchestra.run.smithi133.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-14T18:34:46.046 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-14T18:34:46.046 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.046 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.046 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.047 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.047 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.047 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.047 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.047 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.047 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.047 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.047 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.048 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.048 INFO:teuthology.orchestra.run.smithi133.stdout: "AUDIT_API_ENABLED": { 2023-07-14T18:34:46.048 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "AUDIT_API_ENABLED", 2023-07-14T18:34:46.048 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.048 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.048 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.048 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.048 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.049 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.049 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.049 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.049 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.049 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.049 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.049 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.049 INFO:teuthology.orchestra.run.smithi133.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-14T18:34:46.050 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-14T18:34:46.050 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.050 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.050 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.050 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.050 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.050 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.050 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.051 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.051 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.051 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.051 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.051 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.051 INFO:teuthology.orchestra.run.smithi133.stdout: "ENABLE_BROWSABLE_API": { 2023-07-14T18:34:46.051 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-14T18:34:46.051 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.052 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.052 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.052 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.052 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.052 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.052 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.052 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.053 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.053 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.053 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.053 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.053 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-14T18:34:46.053 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-14T18:34:46.053 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.053 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.054 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.054 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.054 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.054 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.054 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.054 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.054 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.054 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.055 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.055 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.055 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-14T18:34:46.055 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-14T18:34:46.055 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.055 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.055 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.055 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.056 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.056 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.056 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.056 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.056 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.056 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.056 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.056 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.057 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-14T18:34:46.057 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-14T18:34:46.057 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.057 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.057 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.057 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.057 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.057 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.058 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.058 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.058 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.058 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.058 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.058 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.058 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-14T18:34:46.058 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-14T18:34:46.059 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.059 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.059 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.059 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.059 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.059 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.059 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.059 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.060 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.060 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.060 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.060 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.060 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-14T18:34:46.060 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-14T18:34:46.061 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.061 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.061 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.061 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.061 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.061 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.061 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.061 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.062 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.062 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.062 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.062 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.062 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-14T18:34:46.062 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-14T18:34:46.062 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.062 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.063 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.063 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.063 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.063 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.063 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.063 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.063 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.063 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.064 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.064 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.064 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-14T18:34:46.064 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-14T18:34:46.064 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.064 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.064 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.064 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.065 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.065 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.065 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.065 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.065 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.065 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.065 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.065 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.066 INFO:teuthology.orchestra.run.smithi133.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-14T18:34:46.066 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-14T18:34:46.066 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.066 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.066 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.066 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.066 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.066 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.067 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.067 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.067 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.067 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.067 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.067 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.067 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_PASSWORD": { 2023-07-14T18:34:46.067 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-14T18:34:46.068 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.068 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.068 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.068 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-07-14T18:34:46.068 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.068 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.068 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.068 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.069 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.069 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.069 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.069 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.069 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-14T18:34:46.069 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-14T18:34:46.069 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.069 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.070 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.070 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.070 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.070 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.070 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.070 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.070 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.070 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.071 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.071 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.071 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_URL": { 2023-07-14T18:34:46.071 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_URL", 2023-07-14T18:34:46.071 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.071 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.071 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.071 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.072 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.072 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.072 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.072 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.072 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.072 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.072 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.073 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.073 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_USERNAME": { 2023-07-14T18:34:46.073 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-14T18:34:46.073 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.073 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.073 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.073 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-07-14T18:34:46.073 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.074 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.074 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.074 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.074 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.074 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.074 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.074 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.074 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-14T18:34:46.075 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-14T18:34:46.075 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.075 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.075 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.075 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.075 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.075 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.075 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.076 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.076 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.076 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.076 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.076 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.076 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-14T18:34:46.076 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-14T18:34:46.076 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.077 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.077 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.077 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.077 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.077 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.077 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.077 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.077 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.078 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.078 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.078 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.078 INFO:teuthology.orchestra.run.smithi133.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-14T18:34:46.078 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-14T18:34:46.078 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.078 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.078 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.079 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.079 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.079 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.079 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.079 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.079 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.079 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.079 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.080 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.080 INFO:teuthology.orchestra.run.smithi133.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-14T18:34:46.080 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-14T18:34:46.080 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.080 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.080 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.080 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.080 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.081 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.081 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.081 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.081 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.081 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.081 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.081 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.081 INFO:teuthology.orchestra.run.smithi133.stdout: "PROMETHEUS_API_HOST": { 2023-07-14T18:34:46.082 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-14T18:34:46.082 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.082 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.082 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.082 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.082 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.082 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.082 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.083 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.083 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.083 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.083 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.083 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.083 INFO:teuthology.orchestra.run.smithi133.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-14T18:34:46.083 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-14T18:34:46.084 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.084 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.084 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.084 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.084 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.084 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.084 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.084 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.085 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.085 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.085 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.085 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.085 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-14T18:34:46.085 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-14T18:34:46.085 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.086 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.086 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.086 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.086 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.086 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.086 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.086 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.086 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.087 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.087 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.087 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.087 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-14T18:34:46.087 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-14T18:34:46.087 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.087 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.087 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.088 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.088 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.088 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.088 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.088 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.088 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.088 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.088 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.089 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.089 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-14T18:34:46.089 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-14T18:34:46.089 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.089 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.089 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.089 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.089 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.090 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.090 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.090 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.090 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.090 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.090 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.090 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.090 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-14T18:34:46.091 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-14T18:34:46.091 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.091 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.091 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.091 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.091 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.091 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.091 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.092 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.092 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.092 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.092 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.092 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.092 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-14T18:34:46.092 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-14T18:34:46.092 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.093 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.093 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.093 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.093 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.093 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.093 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.093 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.094 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.094 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.094 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.094 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.094 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-14T18:34:46.094 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-14T18:34:46.094 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.094 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.095 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.095 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.095 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.095 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.095 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.095 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.095 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.095 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.096 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.096 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.096 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-14T18:34:46.096 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-14T18:34:46.096 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.096 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.096 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.096 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.097 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.097 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.097 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.097 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.097 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.097 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.097 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.097 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.098 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_ENABLED": { 2023-07-14T18:34:46.098 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-14T18:34:46.098 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.098 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.098 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.098 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.098 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.098 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.099 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.099 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.099 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.099 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.099 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.099 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.099 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-14T18:34:46.099 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-14T18:34:46.100 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.100 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.100 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.100 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-14T18:34:46.100 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.100 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.100 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.100 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.101 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.101 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.101 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.101 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.101 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-14T18:34:46.101 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-14T18:34:46.101 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.101 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.102 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.102 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-07-14T18:34:46.102 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.102 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.102 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.102 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.102 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.102 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.103 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.103 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.103 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-14T18:34:46.103 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-14T18:34:46.103 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.103 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.103 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.103 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8", 2023-07-14T18:34:46.104 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.104 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.104 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.104 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.104 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.104 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.104 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.104 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.105 INFO:teuthology.orchestra.run.smithi133.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-14T18:34:46.105 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-14T18:34:46.105 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.105 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.105 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.105 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "45", 2023-07-14T18:34:46.105 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.105 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.106 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.106 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.106 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.106 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.106 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.106 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.106 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_ACCESS_KEY": { 2023-07-14T18:34:46.107 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-14T18:34:46.107 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.107 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.107 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.107 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.107 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.107 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.107 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.108 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.108 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.108 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.108 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.108 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.108 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-14T18:34:46.108 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-14T18:34:46.109 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.109 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.109 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.109 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-07-14T18:34:46.109 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.109 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.109 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.109 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.110 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.110 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.110 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.110 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.110 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_SECRET_KEY": { 2023-07-14T18:34:46.110 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-14T18:34:46.110 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.110 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.111 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.111 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.111 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.111 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.111 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.111 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.111 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.111 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.112 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.112 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.112 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_SSL_VERIFY": { 2023-07-14T18:34:46.112 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-14T18:34:46.112 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.112 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.112 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.112 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.113 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.113 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.113 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.113 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.113 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.113 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.113 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.113 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.114 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-14T18:34:46.114 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-14T18:34:46.114 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.114 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.114 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.114 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-07-14T18:34:46.114 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.114 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.115 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.115 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.115 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.115 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.115 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.115 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.115 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-14T18:34:46.115 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-14T18:34:46.116 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.116 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.116 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.116 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-07-14T18:34:46.116 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.116 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.116 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.116 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.117 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.117 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.117 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.117 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.117 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-14T18:34:46.117 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-14T18:34:46.117 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.117 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.118 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.118 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2023-07-14T18:34:46.118 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.118 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.118 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.118 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.118 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.118 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.119 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.119 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.119 INFO:teuthology.orchestra.run.smithi133.stdout: "cross_origin_url": { 2023-07-14T18:34:46.119 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cross_origin_url", 2023-07-14T18:34:46.119 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.119 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.119 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.119 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.120 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.120 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.120 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.120 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.120 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.120 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.120 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.120 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.121 INFO:teuthology.orchestra.run.smithi133.stdout: "crt_file": { 2023-07-14T18:34:46.121 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crt_file", 2023-07-14T18:34:46.121 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.121 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.121 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.121 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.121 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.121 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.122 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.122 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.122 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.122 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.122 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.122 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.122 INFO:teuthology.orchestra.run.smithi133.stdout: "debug": { 2023-07-14T18:34:46.122 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "debug", 2023-07-14T18:34:46.123 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.123 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.123 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.123 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.123 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.123 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.123 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.123 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable/disable debug options", 2023-07-14T18:34:46.124 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.124 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.124 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.124 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.124 INFO:teuthology.orchestra.run.smithi133.stdout: "jwt_token_ttl": { 2023-07-14T18:34:46.124 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "jwt_token_ttl", 2023-07-14T18:34:46.124 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.124 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.125 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.125 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "28800", 2023-07-14T18:34:46.125 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.125 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.125 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.125 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.125 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.125 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.126 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.126 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.126 INFO:teuthology.orchestra.run.smithi133.stdout: "key_file": { 2023-07-14T18:34:46.126 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "key_file", 2023-07-14T18:34:46.126 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.126 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.126 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.126 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.127 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.127 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.127 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.127 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.127 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.127 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.127 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.127 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.128 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.128 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.128 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.128 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.128 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.128 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.128 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.128 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.129 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.129 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.129 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.129 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.129 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.129 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.129 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.129 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.130 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.130 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.130 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.130 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.130 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.130 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.130 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.131 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.131 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.131 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.131 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.131 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.131 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.131 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.131 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.132 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.132 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.132 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.132 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.132 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.132 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.132 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.133 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.133 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.133 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.133 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.133 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.133 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.133 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.133 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.134 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.134 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.134 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.134 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.134 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.134 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.134 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.134 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.135 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.135 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.135 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.135 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.135 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.135 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.135 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.135 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.136 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.136 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.136 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.136 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.136 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.136 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.136 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.136 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.137 INFO:teuthology.orchestra.run.smithi133.stdout: "motd": { 2023-07-14T18:34:46.137 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "motd", 2023-07-14T18:34:46.137 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.137 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.137 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.137 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.137 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.137 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.138 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.138 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "The message of the day", 2023-07-14T18:34:46.138 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.138 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.138 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.138 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.138 INFO:teuthology.orchestra.run.smithi133.stdout: "redirect_resolve_ip_addr": { 2023-07-14T18:34:46.138 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "redirect_resolve_ip_addr", 2023-07-14T18:34:46.139 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.139 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.139 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.139 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.139 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.139 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.139 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.139 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.140 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.140 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.140 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.140 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.140 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2023-07-14T18:34:46.140 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2023-07-14T18:34:46.140 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.140 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.141 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.141 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "::", 2023-07-14T18:34:46.141 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.141 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.141 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.141 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.141 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.141 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.142 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.142 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.142 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2023-07-14T18:34:46.142 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2023-07-14T18:34:46.142 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.142 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.142 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.143 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8080", 2023-07-14T18:34:46.143 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.143 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.143 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.143 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.143 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.143 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.143 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.144 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.144 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl": { 2023-07-14T18:34:46.144 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl", 2023-07-14T18:34:46.144 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.144 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.144 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.144 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.144 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.145 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.145 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.145 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.145 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.145 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.145 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.145 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.145 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl_server_port": { 2023-07-14T18:34:46.146 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl_server_port", 2023-07-14T18:34:46.146 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.146 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.146 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.146 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8443", 2023-07-14T18:34:46.146 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.146 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.146 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.147 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.147 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.147 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.147 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.147 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.147 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_behaviour": { 2023-07-14T18:34:46.147 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_behaviour", 2023-07-14T18:34:46.147 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.148 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.148 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.148 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "redirect", 2023-07-14T18:34:46.148 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.148 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.148 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.148 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.148 INFO:teuthology.orchestra.run.smithi133.stdout: "redirect" 2023-07-14T18:34:46.149 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.149 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.149 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.149 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.149 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.149 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.149 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_error_status_code": { 2023-07-14T18:34:46.149 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_error_status_code", 2023-07-14T18:34:46.150 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.150 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.150 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.150 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "500", 2023-07-14T18:34:46.150 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "400", 2023-07-14T18:34:46.150 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "599", 2023-07-14T18:34:46.150 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.150 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.151 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.151 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.151 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.151 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.151 INFO:teuthology.orchestra.run.smithi133.stdout: "url_prefix": { 2023-07-14T18:34:46.151 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "url_prefix", 2023-07-14T18:34:46.151 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.151 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.152 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.152 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.152 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.152 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.152 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.152 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.152 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.152 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.153 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.153 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.153 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.153 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.153 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.153 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "devicehealth", 2023-07-14T18:34:46.153 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.153 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.154 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.154 INFO:teuthology.orchestra.run.smithi133.stdout: "enable_monitoring": { 2023-07-14T18:34:46.154 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enable_monitoring", 2023-07-14T18:34:46.154 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.154 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.154 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.154 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.155 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.155 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.155 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.155 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "monitor device health metrics", 2023-07-14T18:34:46.155 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.155 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.155 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.155 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.156 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.156 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.156 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.156 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.156 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.156 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.156 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.156 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.157 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.157 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.157 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.157 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.157 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.157 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.157 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.157 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.158 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.158 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.158 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.158 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.158 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.158 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.158 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.158 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.159 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.159 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.159 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.159 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.159 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.159 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.159 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.159 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.160 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.160 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.160 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.160 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.160 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.160 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.160 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.160 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.161 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.161 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.161 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.161 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.161 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.161 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.161 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.161 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.162 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.162 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.162 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.162 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.162 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.162 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.162 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.162 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.163 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.163 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.163 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.163 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.163 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.163 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.163 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.164 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.164 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.164 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.164 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.164 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.164 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.164 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.164 INFO:teuthology.orchestra.run.smithi133.stdout: "mark_out_threshold": { 2023-07-14T18:34:46.165 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mark_out_threshold", 2023-07-14T18:34:46.165 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.165 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.165 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.165 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "2419200", 2023-07-14T18:34:46.165 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.165 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.165 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.166 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-14T18:34:46.166 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.166 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.166 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.166 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.166 INFO:teuthology.orchestra.run.smithi133.stdout: "pool_name": { 2023-07-14T18:34:46.166 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pool_name", 2023-07-14T18:34:46.166 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.167 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.167 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.167 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "device_health_metrics", 2023-07-14T18:34:46.167 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.167 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.167 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.167 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-14T18:34:46.167 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.168 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.168 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.168 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.168 INFO:teuthology.orchestra.run.smithi133.stdout: "retention_period": { 2023-07-14T18:34:46.168 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "retention_period", 2023-07-14T18:34:46.168 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.168 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.168 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.169 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15552000", 2023-07-14T18:34:46.169 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.169 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.169 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.169 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long to retain device health metrics", 2023-07-14T18:34:46.169 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.169 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.169 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.170 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.170 INFO:teuthology.orchestra.run.smithi133.stdout: "scrape_frequency": { 2023-07-14T18:34:46.170 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "scrape_frequency", 2023-07-14T18:34:46.170 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.170 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.170 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.170 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "86400", 2023-07-14T18:34:46.170 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.171 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.171 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.171 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-14T18:34:46.171 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.171 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.171 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.171 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.171 INFO:teuthology.orchestra.run.smithi133.stdout: "self_heal": { 2023-07-14T18:34:46.172 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "self_heal", 2023-07-14T18:34:46.172 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.172 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.172 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.172 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.172 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.172 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.172 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.173 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-14T18:34:46.173 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.173 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.173 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.173 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.173 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-07-14T18:34:46.173 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-07-14T18:34:46.173 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.174 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.174 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.174 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-07-14T18:34:46.174 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.174 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.174 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.174 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to wake up and check device health", 2023-07-14T18:34:46.174 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.175 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.175 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.175 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.175 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_threshold": { 2023-07-14T18:34:46.175 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_threshold", 2023-07-14T18:34:46.175 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.175 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.175 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.176 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "7257600", 2023-07-14T18:34:46.176 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.176 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.176 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.176 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-14T18:34:46.176 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.176 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.176 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.177 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.177 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.177 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.177 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.177 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "diskprediction_local", 2023-07-14T18:34:46.177 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.177 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.178 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.178 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.178 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.178 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.178 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.178 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.178 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.178 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.179 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.179 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.179 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.179 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.179 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.179 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.179 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.179 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.180 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.180 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.180 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.180 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.180 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.180 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.180 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.180 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.181 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.181 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.181 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.181 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.181 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.181 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.181 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.181 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.182 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.182 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.182 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.182 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.182 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.182 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.182 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.182 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.183 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.183 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.183 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.183 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.183 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.183 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.183 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.183 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.184 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.184 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.184 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.184 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.184 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.184 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.184 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.185 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.185 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.185 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.185 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.185 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.185 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.185 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.185 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.186 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.186 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.186 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.186 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.186 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.186 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.186 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.186 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.187 INFO:teuthology.orchestra.run.smithi133.stdout: "predict_interval": { 2023-07-14T18:34:46.187 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "predict_interval", 2023-07-14T18:34:46.187 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.187 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.187 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.187 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "86400", 2023-07-14T18:34:46.187 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.187 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.188 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.188 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.188 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.188 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.188 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.188 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.188 INFO:teuthology.orchestra.run.smithi133.stdout: "predictor_model": { 2023-07-14T18:34:46.188 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "predictor_model", 2023-07-14T18:34:46.189 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.189 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.189 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.189 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "prophetstor", 2023-07-14T18:34:46.189 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.189 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.189 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.189 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.190 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.190 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.190 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.190 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.190 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-07-14T18:34:46.190 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-07-14T18:34:46.190 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.190 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.191 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.191 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-07-14T18:34:46.191 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.191 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.191 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.191 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.191 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.191 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.192 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.192 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.192 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.192 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.192 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.192 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "influx", 2023-07-14T18:34:46.192 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": false, 2023-07-14T18:34:46.192 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "influxdb python module not found", 2023-07-14T18:34:46.193 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.193 INFO:teuthology.orchestra.run.smithi133.stdout: "batch_size": { 2023-07-14T18:34:46.193 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "batch_size", 2023-07-14T18:34:46.193 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.193 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.193 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.193 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5000", 2023-07-14T18:34:46.193 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.194 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.194 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.194 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-14T18:34:46.194 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.194 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.194 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.194 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.194 INFO:teuthology.orchestra.run.smithi133.stdout: "database": { 2023-07-14T18:34:46.195 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "database", 2023-07-14T18:34:46.195 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.195 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.195 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.195 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "ceph", 2023-07-14T18:34:46.195 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.195 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.195 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.196 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-14T18:34:46.196 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.196 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.196 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.196 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.196 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": { 2023-07-14T18:34:46.196 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "hostname", 2023-07-14T18:34:46.196 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.197 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.197 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.197 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.197 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.197 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.197 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.197 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB server hostname", 2023-07-14T18:34:46.197 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.198 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.198 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.198 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.198 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-07-14T18:34:46.198 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-07-14T18:34:46.198 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.198 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.198 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.199 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "30", 2023-07-14T18:34:46.199 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "5", 2023-07-14T18:34:46.199 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.199 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.199 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-14T18:34:46.199 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.199 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.199 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.200 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.200 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.200 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.200 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.200 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.200 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.200 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.200 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.201 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.201 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.201 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.201 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.201 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.201 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.201 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.202 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.202 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.202 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.202 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.202 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.202 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.202 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.202 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.203 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.203 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.203 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.203 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.203 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.203 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.203 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.203 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.204 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.204 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.204 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.204 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.204 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.204 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.204 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.205 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.205 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.205 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.205 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.205 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.205 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.205 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.205 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.206 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.206 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.206 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.206 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.206 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.206 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.206 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.206 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.207 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.207 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.207 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.207 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.207 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.207 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.207 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.207 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.208 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.208 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.208 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.208 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.208 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.208 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.208 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.208 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.209 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.209 INFO:teuthology.orchestra.run.smithi133.stdout: "password": { 2023-07-14T18:34:46.209 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "password", 2023-07-14T18:34:46.209 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.209 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.209 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.209 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.209 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.210 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.210 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.210 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "password of InfluxDB server user", 2023-07-14T18:34:46.210 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.210 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.210 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.210 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.210 INFO:teuthology.orchestra.run.smithi133.stdout: "port": { 2023-07-14T18:34:46.211 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "port", 2023-07-14T18:34:46.211 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.211 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.211 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.211 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8086", 2023-07-14T18:34:46.211 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.211 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.211 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.212 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB server port", 2023-07-14T18:34:46.212 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.212 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.212 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.212 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.212 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl": { 2023-07-14T18:34:46.212 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl", 2023-07-14T18:34:46.212 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.213 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.213 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.213 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "false", 2023-07-14T18:34:46.213 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.213 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.213 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.213 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-14T18:34:46.213 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.214 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.214 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.214 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.214 INFO:teuthology.orchestra.run.smithi133.stdout: "threads": { 2023-07-14T18:34:46.214 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "threads", 2023-07-14T18:34:46.214 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.214 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.214 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.215 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2023-07-14T18:34:46.215 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2023-07-14T18:34:46.215 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "32", 2023-07-14T18:34:46.215 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.215 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-14T18:34:46.215 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.215 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.215 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.216 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.216 INFO:teuthology.orchestra.run.smithi133.stdout: "username": { 2023-07-14T18:34:46.216 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "username", 2023-07-14T18:34:46.216 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.216 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.216 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.216 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.216 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.217 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.217 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.217 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "username of InfluxDB server user", 2023-07-14T18:34:46.217 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.217 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.217 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.217 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.217 INFO:teuthology.orchestra.run.smithi133.stdout: "verify_ssl": { 2023-07-14T18:34:46.218 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "verify_ssl", 2023-07-14T18:34:46.218 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.218 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.218 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.218 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "true", 2023-07-14T18:34:46.218 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.218 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.218 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.219 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-14T18:34:46.219 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.219 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.219 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.219 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.219 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.219 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.219 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.220 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "insights", 2023-07-14T18:34:46.220 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.220 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.220 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.220 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.220 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.220 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.220 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.221 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.221 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.221 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.221 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.221 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.221 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.221 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.221 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.222 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.222 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.222 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.222 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.222 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.222 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.222 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.222 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.223 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.223 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.223 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.223 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.223 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.223 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.223 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.223 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.224 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.224 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.224 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.224 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.224 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.224 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.224 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.225 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.225 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.225 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.225 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.225 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.225 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.225 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.226 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.226 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.226 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.226 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.226 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.226 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.226 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.226 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.227 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.227 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.227 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.227 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.227 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.227 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.227 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.227 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.228 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.228 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.228 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.228 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.228 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.228 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.228 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.228 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.229 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.229 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.229 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.229 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.229 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.229 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.229 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.229 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "iostat", 2023-07-14T18:34:46.230 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.230 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.230 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.230 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.230 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.230 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.230 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.230 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.231 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.231 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.231 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.231 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.231 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.231 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.231 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.231 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.232 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.232 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.232 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.232 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.232 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.232 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.232 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.232 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.233 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.233 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.233 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.233 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.233 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.233 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.233 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.233 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.234 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.234 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.234 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.234 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.234 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.234 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.234 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.234 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.235 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.235 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.235 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.235 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.235 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.235 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.235 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.235 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.236 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.236 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.236 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.236 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.236 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.236 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.236 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.236 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.237 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.237 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.237 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.237 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.237 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.237 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.237 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.237 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.238 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.238 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.238 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.238 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.238 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.238 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.238 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.238 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.239 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.239 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.239 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.239 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.239 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "k8sevents", 2023-07-14T18:34:46.239 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.239 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.239 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.240 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_event_retention_days": { 2023-07-14T18:34:46.240 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ceph_event_retention_days", 2023-07-14T18:34:46.240 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.240 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.240 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.240 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "7", 2023-07-14T18:34:46.240 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.240 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.241 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.241 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-14T18:34:46.241 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.241 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.241 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.241 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.241 INFO:teuthology.orchestra.run.smithi133.stdout: "config_check_secs": { 2023-07-14T18:34:46.241 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_check_secs", 2023-07-14T18:34:46.242 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.242 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.242 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.242 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-07-14T18:34:46.242 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "10", 2023-07-14T18:34:46.242 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.242 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.242 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-14T18:34:46.243 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.243 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.243 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.243 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.243 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.243 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.243 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.243 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.244 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.244 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.244 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.244 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.244 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.244 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.244 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.244 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.245 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.245 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.245 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.245 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.245 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.245 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.245 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.245 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.246 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.246 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.246 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.246 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.246 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.246 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.246 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.246 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.247 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.247 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.247 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.247 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.247 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.247 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.247 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.247 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.248 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.248 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.248 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.248 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.248 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.248 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.248 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.248 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.249 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.249 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.249 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.249 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.249 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.249 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.249 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.250 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.250 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.250 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.250 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.250 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.250 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.250 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.250 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.251 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.251 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.251 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.251 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.251 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.251 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.251 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.251 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.252 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.252 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.252 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.252 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.252 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.252 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.252 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "localpool", 2023-07-14T18:34:46.252 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.253 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.253 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.253 INFO:teuthology.orchestra.run.smithi133.stdout: "failure_domain": { 2023-07-14T18:34:46.253 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "failure_domain", 2023-07-14T18:34:46.253 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.253 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.253 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.253 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "host", 2023-07-14T18:34:46.254 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.254 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.254 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.254 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "failure domain for any created local pool", 2023-07-14T18:34:46.254 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-14T18:34:46.254 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.254 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.254 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.255 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.255 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.255 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.255 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.255 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.255 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.255 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.255 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.256 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.256 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.256 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.256 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.256 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.256 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.256 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.257 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.257 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.257 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.257 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.257 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.257 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.257 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.257 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.258 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.258 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.258 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.258 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.258 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.258 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.258 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.258 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.259 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.259 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.259 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.259 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.259 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.259 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.259 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.259 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.260 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.260 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.260 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.260 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.260 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.260 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.260 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.260 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.261 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.261 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.261 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.261 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.261 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.261 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.261 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.261 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.262 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.262 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.262 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.262 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.262 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.262 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.262 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.262 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.263 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.263 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.263 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.263 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.263 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.263 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.263 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.263 INFO:teuthology.orchestra.run.smithi133.stdout: "min_size": { 2023-07-14T18:34:46.264 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "min_size", 2023-07-14T18:34:46.264 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.264 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.264 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.264 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.264 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.264 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.264 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.265 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default min_size for any created local pool", 2023-07-14T18:34:46.265 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-14T18:34:46.265 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.265 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.265 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.265 INFO:teuthology.orchestra.run.smithi133.stdout: "num_rep": { 2023-07-14T18:34:46.265 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "num_rep", 2023-07-14T18:34:46.266 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.266 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.266 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.266 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3", 2023-07-14T18:34:46.266 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.266 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.266 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.266 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default replica count for any created local pool", 2023-07-14T18:34:46.267 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.267 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.267 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.267 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.267 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_num": { 2023-07-14T18:34:46.267 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pg_num", 2023-07-14T18:34:46.267 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.267 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.268 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.268 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "128", 2023-07-14T18:34:46.268 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.268 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.268 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.268 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default pg_num for any created local pool", 2023-07-14T18:34:46.268 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.268 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.269 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.269 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.269 INFO:teuthology.orchestra.run.smithi133.stdout: "prefix": { 2023-07-14T18:34:46.269 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prefix", 2023-07-14T18:34:46.269 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.269 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.269 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.269 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.270 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.270 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.270 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.270 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "name prefix for any created local pool", 2023-07-14T18:34:46.270 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.270 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.270 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.270 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.271 INFO:teuthology.orchestra.run.smithi133.stdout: "subtree": { 2023-07-14T18:34:46.271 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "subtree", 2023-07-14T18:34:46.271 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.271 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.271 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.271 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "rack", 2023-07-14T18:34:46.271 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.271 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.272 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.272 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-14T18:34:46.272 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-14T18:34:46.272 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.272 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.272 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.272 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.272 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.273 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.273 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mds_autoscaler", 2023-07-14T18:34:46.273 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.273 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.273 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.273 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.273 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.274 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.274 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.274 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.274 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.274 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.274 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.274 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.274 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.275 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.275 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.275 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.275 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.275 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.275 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.275 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.275 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.276 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.276 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.276 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.276 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.276 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.276 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.276 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.276 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.277 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.277 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.277 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.277 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.277 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.277 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.277 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.277 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.278 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.278 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.278 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.278 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.278 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.278 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.278 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.278 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.279 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.279 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.279 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.279 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.279 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.279 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.279 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.279 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.280 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.280 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.280 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.280 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.280 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.280 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.280 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.281 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.281 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.281 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.281 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.281 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.281 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.281 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.281 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.282 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.282 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.282 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.282 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.282 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.282 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.282 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.282 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.283 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mirroring", 2023-07-14T18:34:46.283 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.283 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.283 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.283 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.283 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.283 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.283 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.283 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.284 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.284 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.284 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.284 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.284 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.284 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.284 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.285 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.285 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.285 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.285 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.285 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.285 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.285 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.285 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.286 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.286 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.286 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.286 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.286 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.286 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.286 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.286 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.287 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.287 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.287 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.287 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.287 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.287 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.287 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.287 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.288 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.288 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.288 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.288 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.288 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.288 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.288 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.288 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.289 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.289 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.289 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.289 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.289 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.289 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.289 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.289 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.290 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.290 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.290 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.290 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.290 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.290 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.290 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.290 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.291 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.291 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.291 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.291 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.291 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.291 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.291 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.291 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.292 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.292 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.292 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.292 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.292 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.292 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "nfs", 2023-07-14T18:34:46.292 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.292 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.293 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.293 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.293 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.293 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.293 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.293 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.293 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.293 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.294 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.294 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.294 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.294 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.294 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.294 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.294 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.294 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.295 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.295 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.295 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.295 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.295 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.295 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.295 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.295 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.296 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.296 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.296 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.296 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.296 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.296 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.296 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.296 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.297 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.297 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.297 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.297 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.297 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.297 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.297 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.297 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.298 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.298 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.298 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.298 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.298 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.298 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.298 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.299 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.299 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.299 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.299 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.299 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.299 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.299 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.299 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.300 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.300 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.300 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.300 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.300 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.300 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.300 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.300 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.301 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.301 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.301 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.301 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.301 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.301 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.301 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.301 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.302 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.302 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.302 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.302 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "orchestrator", 2023-07-14T18:34:46.302 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.302 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.302 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.302 INFO:teuthology.orchestra.run.smithi133.stdout: "fail_fs": { 2023-07-14T18:34:46.303 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "fail_fs", 2023-07-14T18:34:46.303 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.303 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.303 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.303 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.303 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.303 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.303 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.304 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-14T18:34:46.304 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.304 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.304 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.304 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.304 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.304 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.304 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.305 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.305 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.305 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.305 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.305 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.305 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.305 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.305 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.306 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.306 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.306 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.306 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.306 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.306 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.306 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.306 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.307 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.307 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.307 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.307 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.307 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.307 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.307 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.307 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.308 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.308 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.308 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.308 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.308 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.308 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.308 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.309 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.309 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.309 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.309 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.309 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.309 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.309 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.309 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.310 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.310 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.310 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.310 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.310 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.310 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.310 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.310 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.311 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.311 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.311 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.311 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.311 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.311 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.311 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.311 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.312 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.312 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.312 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.312 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.312 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.312 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.312 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.312 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.313 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.313 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.313 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.313 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.313 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator": { 2023-07-14T18:34:46.313 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "orchestrator", 2023-07-14T18:34:46.313 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.313 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.314 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.314 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.314 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.314 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.314 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.314 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm", 2023-07-14T18:34:46.314 INFO:teuthology.orchestra.run.smithi133.stdout: "rook", 2023-07-14T18:34:46.314 INFO:teuthology.orchestra.run.smithi133.stdout: "test_orchestrator" 2023-07-14T18:34:46.315 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.315 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Orchestrator backend", 2023-07-14T18:34:46.315 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.315 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.315 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.315 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.315 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.315 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.315 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.316 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "osd_perf_query", 2023-07-14T18:34:46.316 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.316 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.316 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.316 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.316 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.316 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.317 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.317 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.317 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.317 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.317 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.317 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.317 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.317 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.318 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.318 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.318 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.318 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.318 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.318 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.318 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.318 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.319 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.319 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.319 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.319 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.319 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.319 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.319 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.319 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.320 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.320 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.320 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.320 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.320 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.320 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.320 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.320 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.321 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.321 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.321 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.321 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.321 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.321 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.321 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.321 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.322 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.322 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.322 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.322 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.322 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.322 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.322 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.322 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.323 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.323 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.323 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.323 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.323 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.323 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.323 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.324 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.324 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.324 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.324 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.324 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.324 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.324 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.324 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.325 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.325 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.325 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.325 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.325 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.325 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.325 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.325 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "osd_support", 2023-07-14T18:34:46.326 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.326 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.326 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.326 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.326 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.326 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.326 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.326 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.327 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.327 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.327 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.327 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.327 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.327 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.327 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.327 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.328 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.328 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.328 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.328 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.328 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.328 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.328 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.329 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.329 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.329 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.329 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.329 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.329 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.329 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.329 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.330 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.330 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.330 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.330 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.330 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.330 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.330 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.330 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.331 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.331 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.331 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.331 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.331 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.331 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.331 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.331 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.332 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.332 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.332 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.332 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.332 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.332 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.332 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.332 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.333 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.333 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.333 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.333 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.333 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.333 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.333 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.333 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.334 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.334 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.334 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.334 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.334 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.334 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.334 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.334 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.335 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.335 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.335 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.335 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.335 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.335 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pg_autoscaler", 2023-07-14T18:34:46.335 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.335 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.336 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.336 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.336 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.336 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.336 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.336 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.336 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.336 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.337 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.337 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.337 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.337 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.337 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.337 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.337 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.337 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.338 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.338 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.338 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.338 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.338 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.338 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.338 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.338 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.339 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.339 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.339 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.339 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.339 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.339 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.339 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.339 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.340 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.340 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.340 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.340 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.340 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.340 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.340 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.340 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.341 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.341 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.341 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.341 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.341 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.341 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.341 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.341 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.342 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.342 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.342 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.342 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.342 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.342 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.342 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.342 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.343 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.343 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.343 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.343 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.343 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.343 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.343 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.343 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.344 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.344 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.344 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.344 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.344 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.344 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.344 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.344 INFO:teuthology.orchestra.run.smithi133.stdout: "noautoscale": { 2023-07-14T18:34:46.345 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "noautoscale", 2023-07-14T18:34:46.345 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.345 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.345 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.345 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.345 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.345 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.345 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.346 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "global autoscale flag", 2023-07-14T18:34:46.346 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-14T18:34:46.346 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.346 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.346 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.346 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-07-14T18:34:46.346 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-07-14T18:34:46.346 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.347 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.347 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.347 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-07-14T18:34:46.347 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.347 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.347 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.347 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.347 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.348 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.348 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.348 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.348 INFO:teuthology.orchestra.run.smithi133.stdout: "threshold": { 2023-07-14T18:34:46.348 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "threshold", 2023-07-14T18:34:46.348 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-07-14T18:34:46.348 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.349 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.349 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3.0", 2023-07-14T18:34:46.349 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1.0", 2023-07-14T18:34:46.349 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.349 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.349 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "scaling threshold", 2023-07-14T18:34:46.349 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-14T18:34:46.349 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.350 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.350 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.350 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.350 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.350 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.350 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "progress", 2023-07-14T18:34:46.350 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.350 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.351 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.351 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_pg_recovery_event": { 2023-07-14T18:34:46.351 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_pg_recovery_event", 2023-07-14T18:34:46.351 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.351 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.351 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.351 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.351 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.352 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.352 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.352 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-14T18:34:46.352 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.352 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.352 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.352 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.352 INFO:teuthology.orchestra.run.smithi133.stdout: "enabled": { 2023-07-14T18:34:46.353 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enabled", 2023-07-14T18:34:46.353 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.353 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.353 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.353 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.353 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.353 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.353 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.354 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.354 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.354 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.354 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.354 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.354 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.354 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.354 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.355 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.355 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.355 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.355 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.355 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.355 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.355 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.355 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.356 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.356 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.356 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.356 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.356 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.356 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.356 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.356 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.357 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.357 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.357 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.357 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.357 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.357 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.357 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.357 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.358 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.358 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.358 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.358 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.358 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.358 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.358 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.358 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.359 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.359 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.359 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.359 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.359 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.359 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.359 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.359 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.360 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.360 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.360 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.360 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.360 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.360 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.360 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.361 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.361 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.361 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.361 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.361 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.361 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.361 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.361 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.362 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.362 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.362 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.362 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.362 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.362 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.362 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.362 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.363 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.363 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.363 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.363 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.363 INFO:teuthology.orchestra.run.smithi133.stdout: "max_completed_events": { 2023-07-14T18:34:46.363 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_completed_events", 2023-07-14T18:34:46.363 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.363 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.364 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.364 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "50", 2023-07-14T18:34:46.364 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.364 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.364 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.364 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "number of past completed events to remember", 2023-07-14T18:34:46.364 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.364 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.365 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.365 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.365 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-07-14T18:34:46.365 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-07-14T18:34:46.365 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.365 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.365 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.365 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2023-07-14T18:34:46.366 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.366 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.366 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.366 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long the module is going to sleep", 2023-07-14T18:34:46.366 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.366 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.366 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.366 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.367 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.367 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.367 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.367 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus", 2023-07-14T18:34:46.367 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.367 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.367 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.367 INFO:teuthology.orchestra.run.smithi133.stdout: "cache": { 2023-07-14T18:34:46.368 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cache", 2023-07-14T18:34:46.368 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.368 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.368 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.368 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.368 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.368 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.368 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.369 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.369 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.369 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.369 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.369 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.369 INFO:teuthology.orchestra.run.smithi133.stdout: "exclude_perf_counters": { 2023-07-14T18:34:46.369 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "exclude_perf_counters", 2023-07-14T18:34:46.369 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.370 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.370 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.370 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.370 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.370 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.370 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.370 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-14T18:34:46.371 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-14T18:34:46.371 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.371 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.371 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.371 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.371 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.371 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.371 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.372 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.372 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.372 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.372 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.372 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.372 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.372 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.372 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.373 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.373 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.373 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.373 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.373 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.373 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.373 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.373 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.374 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.374 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.374 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.374 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.374 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.374 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.374 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.374 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.375 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.375 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.375 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.375 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.375 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.375 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.375 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.375 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.376 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.376 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.376 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.376 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.376 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.376 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.376 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.377 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.377 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.377 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.377 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.377 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.377 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.377 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.377 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.378 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.378 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.378 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.378 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.378 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.378 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.378 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.378 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.379 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.379 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.379 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.379 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.379 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.379 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.379 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.379 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.380 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.380 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.380 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.380 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_stats_pools": { 2023-07-14T18:34:46.380 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_stats_pools", 2023-07-14T18:34:46.380 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.380 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.380 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.381 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.381 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.381 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.381 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.381 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.381 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.381 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.381 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.382 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.382 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-14T18:34:46.382 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-14T18:34:46.382 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.382 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.382 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.382 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "300", 2023-07-14T18:34:46.382 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.383 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.383 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.383 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.383 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.383 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.383 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.383 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.383 INFO:teuthology.orchestra.run.smithi133.stdout: "scrape_interval": { 2023-07-14T18:34:46.384 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "scrape_interval", 2023-07-14T18:34:46.384 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-07-14T18:34:46.384 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.384 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.384 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15.0", 2023-07-14T18:34:46.384 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.384 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.384 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.385 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.385 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.385 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.385 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.385 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.385 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2023-07-14T18:34:46.385 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2023-07-14T18:34:46.385 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.386 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.386 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.386 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "::", 2023-07-14T18:34:46.386 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.386 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.386 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.386 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-14T18:34:46.386 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.387 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.387 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.387 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.387 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2023-07-14T18:34:46.387 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2023-07-14T18:34:46.387 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.387 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.387 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.388 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "9283", 2023-07-14T18:34:46.388 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.388 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.388 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.388 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-14T18:34:46.388 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.388 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.389 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.389 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.389 INFO:teuthology.orchestra.run.smithi133.stdout: "stale_cache_strategy": { 2023-07-14T18:34:46.389 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "stale_cache_strategy", 2023-07-14T18:34:46.389 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.389 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.389 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.389 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "log", 2023-07-14T18:34:46.390 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.390 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.390 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.390 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.390 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.390 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.390 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.390 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.391 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_behaviour": { 2023-07-14T18:34:46.391 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_behaviour", 2023-07-14T18:34:46.391 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.391 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.391 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.391 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "default", 2023-07-14T18:34:46.391 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.391 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.392 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.392 INFO:teuthology.orchestra.run.smithi133.stdout: "default", 2023-07-14T18:34:46.392 INFO:teuthology.orchestra.run.smithi133.stdout: "error" 2023-07-14T18:34:46.392 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.392 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.392 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.392 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.392 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.393 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.393 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_error_status_code": { 2023-07-14T18:34:46.393 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_error_status_code", 2023-07-14T18:34:46.393 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.393 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.393 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.393 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "500", 2023-07-14T18:34:46.393 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "400", 2023-07-14T18:34:46.394 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "599", 2023-07-14T18:34:46.394 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.394 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.394 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.394 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.394 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.394 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.395 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.395 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.395 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.395 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_support", 2023-07-14T18:34:46.395 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.395 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.395 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.395 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.396 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.396 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.396 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.396 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.396 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.396 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.396 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.396 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.397 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.397 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.397 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.397 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.397 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.397 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.397 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.397 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.398 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.398 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.398 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.398 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.398 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.398 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.398 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.398 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.399 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.399 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.399 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.399 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.399 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.399 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.399 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.399 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.400 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.400 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.400 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.400 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.400 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.400 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.400 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.400 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.401 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.401 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.401 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.401 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.401 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.401 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.401 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.401 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.402 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.402 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.402 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.402 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.402 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.402 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.402 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.403 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.403 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.403 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.403 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.403 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.403 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.403 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.403 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.404 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.404 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.404 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.404 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.404 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.404 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.404 INFO:teuthology.orchestra.run.smithi133.stdout: "max_concurrent_snap_create": { 2023-07-14T18:34:46.404 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_concurrent_snap_create", 2023-07-14T18:34:46.405 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.405 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.405 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.405 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-07-14T18:34:46.405 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.405 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.405 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.405 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.406 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.406 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.406 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.406 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.406 INFO:teuthology.orchestra.run.smithi133.stdout: "mirror_snapshot_schedule": { 2023-07-14T18:34:46.406 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mirror_snapshot_schedule", 2023-07-14T18:34:46.406 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.406 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.407 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.407 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.407 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.407 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.407 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.407 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.407 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.407 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.408 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.408 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.408 INFO:teuthology.orchestra.run.smithi133.stdout: "trash_purge_schedule": { 2023-07-14T18:34:46.408 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "trash_purge_schedule", 2023-07-14T18:34:46.408 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.408 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.408 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.408 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.409 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.409 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.409 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.409 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.409 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.409 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.409 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.409 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.410 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.410 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.410 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.410 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "restful", 2023-07-14T18:34:46.410 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.410 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.410 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.410 INFO:teuthology.orchestra.run.smithi133.stdout: "enable_auth": { 2023-07-14T18:34:46.411 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enable_auth", 2023-07-14T18:34:46.411 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.411 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.411 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.411 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.411 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.411 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.411 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.412 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.412 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.412 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.412 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.412 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.412 INFO:teuthology.orchestra.run.smithi133.stdout: "key_file": { 2023-07-14T18:34:46.412 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "key_file", 2023-07-14T18:34:46.412 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.413 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.413 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.413 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.413 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.413 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.413 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.413 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.413 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.414 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.414 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.414 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.414 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.414 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.414 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.414 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.414 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.415 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.415 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.415 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.415 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.415 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.415 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.415 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.415 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.416 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.416 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.416 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.416 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.416 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.416 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.416 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.416 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.417 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.417 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.417 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.417 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.417 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.417 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.417 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.417 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.418 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.418 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.418 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.418 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.418 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.418 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.418 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.418 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.419 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.419 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.419 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.419 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.419 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.420 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.420 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.420 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.420 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.420 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.420 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.420 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.420 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.421 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.421 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.421 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.421 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.421 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.421 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.421 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.421 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.422 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.422 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.422 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.422 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.422 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.422 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.422 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.422 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.423 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.423 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.423 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.423 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.423 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2023-07-14T18:34:46.423 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2023-07-14T18:34:46.423 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.423 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.424 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.424 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.424 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.424 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.424 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.424 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.424 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.424 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.425 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.425 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.425 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2023-07-14T18:34:46.425 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2023-07-14T18:34:46.425 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.425 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.425 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.425 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.426 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.426 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.426 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.426 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.426 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.426 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.426 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.426 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.427 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.427 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.427 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.427 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rgw", 2023-07-14T18:34:46.427 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.427 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.427 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.427 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.428 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.428 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.428 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.428 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.428 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.428 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.428 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.429 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.429 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.429 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.429 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.429 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.429 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.429 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.429 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.430 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.430 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.430 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.430 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.430 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.430 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.430 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.430 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.431 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.431 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.431 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.431 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.431 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.431 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.431 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.431 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.432 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.432 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.432 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.432 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.432 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.432 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.432 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.432 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.433 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.433 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.433 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.433 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.433 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.433 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.433 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.433 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.434 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.434 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.434 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.434 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.434 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.434 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.434 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.434 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.435 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.435 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.435 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.435 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.435 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.435 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.435 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.435 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.436 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.436 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.436 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.436 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.436 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.436 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.436 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.436 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.437 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.437 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rook", 2023-07-14T18:34:46.437 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.437 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.437 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.437 INFO:teuthology.orchestra.run.smithi133.stdout: "drive_group_interval": { 2023-07-14T18:34:46.437 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "drive_group_interval", 2023-07-14T18:34:46.437 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-07-14T18:34:46.438 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.438 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.438 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "300.0", 2023-07-14T18:34:46.438 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.438 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.438 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.438 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-14T18:34:46.439 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.439 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.439 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.439 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.439 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.439 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.439 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.439 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.440 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.440 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.440 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.440 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.440 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.440 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.440 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.440 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.441 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.441 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.441 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.441 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.441 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.441 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.441 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.441 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.442 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.442 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.442 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.442 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.442 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.442 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.442 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.442 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.443 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.443 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.443 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.443 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.443 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.443 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.443 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.443 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.444 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.444 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.444 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.444 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.444 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.444 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.444 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.445 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.445 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.445 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.445 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.445 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.445 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.445 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.445 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.446 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.446 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.446 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.446 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.446 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.446 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.446 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.446 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.447 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.447 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.447 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.447 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.447 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.447 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.447 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.447 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.448 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.448 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.448 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.448 INFO:teuthology.orchestra.run.smithi133.stdout: "storage_class": { 2023-07-14T18:34:46.448 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "storage_class", 2023-07-14T18:34:46.448 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.448 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.448 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.449 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "local", 2023-07-14T18:34:46.449 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.449 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.449 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.449 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-14T18:34:46.449 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.449 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.449 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.450 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.450 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.450 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.450 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.450 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "selftest", 2023-07-14T18:34:46.450 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.450 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.451 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.451 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.451 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.451 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.451 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.451 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.451 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.451 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.452 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.452 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.452 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.452 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.452 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.452 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.452 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.452 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.453 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.453 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.453 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.453 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.453 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.453 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.453 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.453 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.454 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.454 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.454 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.454 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.454 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.454 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.454 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.454 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.455 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.455 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.455 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.455 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.455 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.455 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.455 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.455 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.456 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.456 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.456 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.456 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.456 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.456 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.456 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.456 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.457 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.457 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.457 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.457 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.457 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.457 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.457 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.457 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.458 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.458 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.458 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.458 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.458 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.458 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.458 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.458 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.459 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.459 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.459 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.459 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.459 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.459 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.459 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.459 INFO:teuthology.orchestra.run.smithi133.stdout: "roption1": { 2023-07-14T18:34:46.460 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "roption1", 2023-07-14T18:34:46.460 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.460 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.460 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.460 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.460 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.460 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.461 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.461 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.461 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.461 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.461 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.461 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.461 INFO:teuthology.orchestra.run.smithi133.stdout: "roption2": { 2023-07-14T18:34:46.461 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "roption2", 2023-07-14T18:34:46.462 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.462 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.462 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.462 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "xyz", 2023-07-14T18:34:46.462 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.462 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.462 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.462 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.463 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.463 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.463 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.463 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.463 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption1": { 2023-07-14T18:34:46.463 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption1", 2023-07-14T18:34:46.463 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.463 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.464 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.464 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.464 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.464 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.464 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.464 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.464 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.464 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.465 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.465 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.465 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption2": { 2023-07-14T18:34:46.465 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption2", 2023-07-14T18:34:46.465 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.465 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.465 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.465 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.466 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.466 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.466 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.466 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.466 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.466 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.466 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.466 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.467 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption3": { 2023-07-14T18:34:46.467 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption3", 2023-07-14T18:34:46.467 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-07-14T18:34:46.467 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.467 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.467 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.467 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.467 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.468 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.468 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.468 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.468 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.468 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.468 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.468 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption4": { 2023-07-14T18:34:46.469 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption4", 2023-07-14T18:34:46.469 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.469 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.469 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.469 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.469 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.469 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.470 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.470 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.470 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.470 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.470 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.470 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.470 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption5": { 2023-07-14T18:34:46.470 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption5", 2023-07-14T18:34:46.471 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.471 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.471 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.471 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.471 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.471 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.471 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.471 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.472 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.472 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.472 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.472 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.472 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption6": { 2023-07-14T18:34:46.472 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption6", 2023-07-14T18:34:46.472 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.472 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.473 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.473 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.473 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.473 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.473 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.473 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.473 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.473 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.474 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.474 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.474 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption7": { 2023-07-14T18:34:46.474 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption7", 2023-07-14T18:34:46.474 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.474 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.474 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.474 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.475 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2023-07-14T18:34:46.475 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "42", 2023-07-14T18:34:46.475 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.475 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.475 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.475 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.475 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.475 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.476 INFO:teuthology.orchestra.run.smithi133.stdout: "testkey": { 2023-07-14T18:34:46.476 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testkey", 2023-07-14T18:34:46.476 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.476 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.476 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.476 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.476 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.476 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.477 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.477 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.477 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.477 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.477 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.477 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.477 INFO:teuthology.orchestra.run.smithi133.stdout: "testlkey": { 2023-07-14T18:34:46.477 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testlkey", 2023-07-14T18:34:46.478 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.478 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.478 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.478 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.478 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.478 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.478 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.478 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.479 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.479 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.479 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.479 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.479 INFO:teuthology.orchestra.run.smithi133.stdout: "testnewline": { 2023-07-14T18:34:46.479 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testnewline", 2023-07-14T18:34:46.479 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.479 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.480 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.480 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.480 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.480 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.480 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.480 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.480 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.480 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.481 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.481 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.481 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.481 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.481 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.481 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snap_schedule", 2023-07-14T18:34:46.481 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.481 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.482 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.482 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_m_granularity": { 2023-07-14T18:34:46.482 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_m_granularity", 2023-07-14T18:34:46.482 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.482 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.482 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.482 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.483 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.483 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.483 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.483 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow minute scheduled snapshots", 2023-07-14T18:34:46.483 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.483 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.483 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.483 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.484 INFO:teuthology.orchestra.run.smithi133.stdout: "dump_on_update": { 2023-07-14T18:34:46.484 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "dump_on_update", 2023-07-14T18:34:46.484 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.484 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.484 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.484 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.484 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.484 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.485 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.485 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "dump database to debug log on update", 2023-07-14T18:34:46.485 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.485 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.485 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.485 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.485 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.485 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.486 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.486 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.486 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.486 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.486 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.486 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.486 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.486 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.487 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.487 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.487 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.487 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.487 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.487 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.487 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.487 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.488 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.488 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.488 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.488 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.488 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.488 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.488 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.488 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.489 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.489 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.489 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.489 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.489 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.489 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.489 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.489 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.490 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.490 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.490 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.490 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.490 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.490 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.490 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.490 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.491 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.491 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.491 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.491 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.491 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.491 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.491 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.491 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.492 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.492 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.492 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.492 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.492 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.492 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.492 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.493 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.493 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.493 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.493 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.493 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.493 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.493 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.493 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.494 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.494 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.494 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.494 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.494 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.494 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.494 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.494 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.495 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "stats", 2023-07-14T18:34:46.495 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.495 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.495 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.495 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.495 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.495 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.495 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.496 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.496 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.496 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.496 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.496 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.496 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.496 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.497 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.497 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.497 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.497 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.497 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.497 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.497 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.497 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.498 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.498 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.498 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.498 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.498 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.498 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.498 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.498 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.499 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.499 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.499 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.499 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.499 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.499 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.499 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.499 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.500 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.500 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.500 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.500 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.500 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.500 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.500 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.500 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.501 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.501 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.501 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.501 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.501 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.501 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.501 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.501 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.502 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.502 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.502 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.502 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.502 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.502 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.502 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.502 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.503 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.503 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.503 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.503 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.503 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.503 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.503 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.503 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.504 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.504 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.504 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.504 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.504 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.504 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.504 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "status", 2023-07-14T18:34:46.504 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.505 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.505 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.505 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.505 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.505 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.505 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.505 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.505 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.506 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.506 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.506 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.506 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.506 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.506 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.506 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.506 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.507 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.507 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.507 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.507 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.507 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.507 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.507 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.507 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.508 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.508 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.508 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.508 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.508 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.508 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.508 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.508 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.509 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.509 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.509 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.509 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.509 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.509 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.509 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.509 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.510 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.510 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.510 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.510 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.510 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.510 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.510 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.510 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.511 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.511 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.511 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.511 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.511 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.511 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.511 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.511 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.512 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.512 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.512 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.512 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.512 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.512 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.512 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.513 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.513 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.513 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.513 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.513 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.513 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.513 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.513 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.514 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.514 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.514 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.514 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.514 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "telegraf", 2023-07-14T18:34:46.514 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.514 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.514 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.515 INFO:teuthology.orchestra.run.smithi133.stdout: "address": { 2023-07-14T18:34:46.515 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "address", 2023-07-14T18:34:46.515 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.515 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.515 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.515 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-14T18:34:46.515 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.515 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.516 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.516 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.516 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.516 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.516 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.516 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.516 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-07-14T18:34:46.516 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-07-14T18:34:46.517 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.517 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.517 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.517 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15", 2023-07-14T18:34:46.517 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.517 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.517 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.517 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.518 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.518 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.518 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.518 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.518 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.518 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.518 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.519 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.519 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.519 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.519 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.519 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.519 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.519 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.519 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.520 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.520 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.520 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.520 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.520 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.520 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.520 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.520 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.521 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.521 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.521 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.521 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.521 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.521 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.521 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.521 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.522 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.522 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.522 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.522 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.522 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.522 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.522 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.522 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.523 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.523 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.523 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.523 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.523 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.523 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.523 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.524 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.524 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.524 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.524 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.524 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.524 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.524 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.524 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.525 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.525 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.525 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.525 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.525 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.525 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.525 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.525 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.526 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.526 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.526 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.526 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.526 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.526 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.526 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.526 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.527 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.527 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.527 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.527 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.527 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.527 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.527 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.527 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "telemetry", 2023-07-14T18:34:46.528 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.528 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.528 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.528 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_basic": { 2023-07-14T18:34:46.528 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_basic", 2023-07-14T18:34:46.528 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.528 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.528 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.529 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.529 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.529 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.529 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.529 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-14T18:34:46.529 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.529 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.529 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.530 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.530 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_crash": { 2023-07-14T18:34:46.530 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_crash", 2023-07-14T18:34:46.530 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.530 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.530 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.530 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.530 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.531 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.531 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.531 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-14T18:34:46.531 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.531 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.531 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.531 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.531 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_device": { 2023-07-14T18:34:46.532 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_device", 2023-07-14T18:34:46.532 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.532 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.532 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.532 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.532 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.532 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.532 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.533 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-14T18:34:46.533 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.533 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.533 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.533 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.533 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_ident": { 2023-07-14T18:34:46.533 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_ident", 2023-07-14T18:34:46.533 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.534 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.534 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.534 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.534 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.534 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.534 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.534 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-14T18:34:46.534 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.535 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.535 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.535 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.535 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_perf": { 2023-07-14T18:34:46.535 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_perf", 2023-07-14T18:34:46.535 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.535 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.535 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.536 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.536 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.536 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.536 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.536 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-14T18:34:46.536 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.536 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.536 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.537 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.537 INFO:teuthology.orchestra.run.smithi133.stdout: "contact": { 2023-07-14T18:34:46.537 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "contact", 2023-07-14T18:34:46.537 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.537 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.537 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.537 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.538 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.538 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.538 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.538 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.538 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.538 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.538 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.538 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.539 INFO:teuthology.orchestra.run.smithi133.stdout: "description": { 2023-07-14T18:34:46.539 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "description", 2023-07-14T18:34:46.539 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.539 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.539 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.539 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.539 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.539 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.540 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.540 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.540 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.540 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.540 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.540 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.540 INFO:teuthology.orchestra.run.smithi133.stdout: "device_url": { 2023-07-14T18:34:46.540 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_url", 2023-07-14T18:34:46.541 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.541 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.541 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.541 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-14T18:34:46.541 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.541 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.541 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.541 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.542 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.542 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.542 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.542 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.542 INFO:teuthology.orchestra.run.smithi133.stdout: "enabled": { 2023-07-14T18:34:46.542 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enabled", 2023-07-14T18:34:46.542 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.543 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.543 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.543 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.543 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.543 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.543 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.543 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.543 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.544 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.544 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.544 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.544 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-07-14T18:34:46.544 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-07-14T18:34:46.544 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.544 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.544 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.545 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "24", 2023-07-14T18:34:46.545 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "8", 2023-07-14T18:34:46.545 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.545 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.545 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.545 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.545 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.545 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.546 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.546 INFO:teuthology.orchestra.run.smithi133.stdout: "last_opt_revision": { 2023-07-14T18:34:46.546 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "last_opt_revision", 2023-07-14T18:34:46.546 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.546 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.546 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.546 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1", 2023-07-14T18:34:46.546 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.547 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.547 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.547 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.547 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.547 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.547 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.547 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.547 INFO:teuthology.orchestra.run.smithi133.stdout: "leaderboard": { 2023-07-14T18:34:46.548 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "leaderboard", 2023-07-14T18:34:46.548 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.548 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.548 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.548 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.548 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.548 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.549 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.549 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.549 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.549 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.549 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.549 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.549 INFO:teuthology.orchestra.run.smithi133.stdout: "leaderboard_description": { 2023-07-14T18:34:46.549 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "leaderboard_description", 2023-07-14T18:34:46.550 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.550 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.550 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.550 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.550 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.550 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.550 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.550 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.551 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.551 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.551 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.551 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.551 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.551 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.551 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.551 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.551 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.552 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.552 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.552 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.552 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.552 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.552 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.552 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.553 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.553 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.553 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.553 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.553 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.553 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.553 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.553 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.554 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.554 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.554 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.554 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.554 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.554 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.554 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.554 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.555 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.555 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.555 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.555 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.555 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.555 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.555 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.555 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.556 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.556 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.556 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.556 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.556 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.556 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.556 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.556 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.557 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.557 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.557 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.557 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.557 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.557 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.557 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.557 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.558 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.558 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.558 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.558 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.558 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.558 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.558 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.558 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.559 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.559 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.559 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.559 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.559 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.559 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.559 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.559 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.560 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.560 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.560 INFO:teuthology.orchestra.run.smithi133.stdout: "organization": { 2023-07-14T18:34:46.560 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "organization", 2023-07-14T18:34:46.560 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.560 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.560 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.560 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.561 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.561 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.561 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.561 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.561 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.561 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.561 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.561 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.562 INFO:teuthology.orchestra.run.smithi133.stdout: "proxy": { 2023-07-14T18:34:46.562 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "proxy", 2023-07-14T18:34:46.562 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.562 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.562 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.562 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.562 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.562 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.563 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.563 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.563 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.563 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.563 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.563 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.563 INFO:teuthology.orchestra.run.smithi133.stdout: "url": { 2023-07-14T18:34:46.563 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "url", 2023-07-14T18:34:46.564 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.564 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.564 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.564 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-14T18:34:46.564 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.564 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.564 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.564 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.565 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.565 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.565 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.565 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.565 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.565 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.565 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.565 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "test_orchestrator", 2023-07-14T18:34:46.566 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.566 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.566 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.566 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.566 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.566 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.566 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.566 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.567 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.567 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.567 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.567 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.567 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.567 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.567 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.568 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.568 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.568 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.568 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.568 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.568 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.568 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.569 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.569 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.569 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.569 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.569 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.569 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.569 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.569 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.570 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.570 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.570 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.570 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.570 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.570 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.570 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.570 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.571 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.571 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.571 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.571 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.571 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.571 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.571 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.571 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.572 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.572 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.572 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.572 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.572 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.572 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.572 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.572 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.573 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.573 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.573 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.573 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.573 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.573 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.573 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.573 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.574 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.574 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.574 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.574 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.574 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.574 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.574 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.574 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.575 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.575 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.575 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.575 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.575 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.575 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.575 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "volumes", 2023-07-14T18:34:46.575 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.576 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.576 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.576 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.576 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.576 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.576 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.576 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.576 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.577 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.577 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.577 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.577 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.577 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.577 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.577 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.577 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.578 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.578 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.578 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.578 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.578 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.578 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.578 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.578 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.579 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.579 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.579 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.579 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.579 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.579 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.579 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.580 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.580 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.580 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.580 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.580 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.580 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.580 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.580 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.581 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.581 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.581 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.581 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.581 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.581 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.581 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.581 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.582 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.582 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.582 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.582 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.582 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.582 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.582 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.582 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.583 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.583 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.583 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.583 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.583 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.583 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.583 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.583 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.584 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.584 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.584 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.584 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.584 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.584 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.584 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.584 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.585 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.585 INFO:teuthology.orchestra.run.smithi133.stdout: "max_concurrent_clones": { 2023-07-14T18:34:46.585 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_concurrent_clones", 2023-07-14T18:34:46.585 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.585 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.585 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.585 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "4", 2023-07-14T18:34:46.585 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.586 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.586 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.586 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-14T18:34:46.586 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.586 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.586 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.586 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.586 INFO:teuthology.orchestra.run.smithi133.stdout: "snapshot_clone_delay": { 2023-07-14T18:34:46.587 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snapshot_clone_delay", 2023-07-14T18:34:46.587 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.587 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.587 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.587 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-07-14T18:34:46.587 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.587 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.587 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.588 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-14T18:34:46.588 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.588 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.588 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.588 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.588 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.588 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.589 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.589 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix", 2023-07-14T18:34:46.589 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.589 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.589 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.589 INFO:teuthology.orchestra.run.smithi133.stdout: "discovery_interval": { 2023-07-14T18:34:46.589 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "discovery_interval", 2023-07-14T18:34:46.589 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-07-14T18:34:46.590 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.590 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.590 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "100", 2023-07-14T18:34:46.590 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.590 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.590 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.590 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.590 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.591 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.591 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.591 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.591 INFO:teuthology.orchestra.run.smithi133.stdout: "identifier": { 2023-07-14T18:34:46.591 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "identifier", 2023-07-14T18:34:46.591 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.591 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.591 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.592 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.592 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.592 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.592 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.592 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.592 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.592 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.593 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.593 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.593 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-07-14T18:34:46.593 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-07-14T18:34:46.593 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.593 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.593 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.593 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-07-14T18:34:46.594 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.594 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.594 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.594 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.594 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.594 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.594 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.594 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.595 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.595 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.595 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.595 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.595 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.595 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.595 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.595 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.596 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.596 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.596 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.596 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.596 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.596 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.596 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.596 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.597 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.597 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.597 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.597 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.597 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.597 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.597 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.597 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.598 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.598 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.598 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.598 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.598 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.598 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.598 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.598 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.599 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.599 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.599 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.599 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.599 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.599 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.599 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.599 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.600 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.600 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.600 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.600 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.600 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.600 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.600 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.600 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.601 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.601 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.601 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.601 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.601 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.601 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.601 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.601 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.602 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.602 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.602 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.602 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.602 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.602 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.602 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.602 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.603 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.603 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.603 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.603 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.603 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.603 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.603 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_host": { 2023-07-14T18:34:46.603 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_host", 2023-07-14T18:34:46.604 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.604 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.604 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.604 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.604 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.604 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.604 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.604 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.605 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.605 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.605 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.605 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.605 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_port": { 2023-07-14T18:34:46.605 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_port", 2023-07-14T18:34:46.605 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.605 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.606 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.606 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10051", 2023-07-14T18:34:46.606 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.606 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.606 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.606 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.606 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.606 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.607 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.607 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.607 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_sender": { 2023-07-14T18:34:46.607 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_sender", 2023-07-14T18:34:46.607 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.607 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.607 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.607 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-14T18:34:46.608 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.608 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.608 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.608 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.608 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.608 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.608 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.608 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.609 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.609 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.609 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2023-07-14T18:34:46.609 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.609 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.609 INFO:teuthology.orchestra.run.smithi133.stdout: "gid": 24110, 2023-07-14T18:34:46.609 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "y", 2023-07-14T18:34:46.609 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr_features": 4540138322906710015, 2023-07-14T18:34:46.610 INFO:teuthology.orchestra.run.smithi133.stdout: "available_modules": [ 2023-07-14T18:34:46.610 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.610 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "alerts", 2023-07-14T18:34:46.610 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.610 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.610 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.610 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-07-14T18:34:46.610 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-07-14T18:34:46.611 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.611 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.611 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.611 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-07-14T18:34:46.611 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.611 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.611 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.611 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How frequently to reexamine health status", 2023-07-14T18:34:46.612 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.612 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.612 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.612 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.612 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.612 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.612 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.612 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.613 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.613 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.613 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.613 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.613 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.613 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.613 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.613 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.614 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.614 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.614 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.614 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.614 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.614 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.614 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.614 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.615 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.615 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.615 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.615 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.615 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.615 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.615 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.615 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.616 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.616 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.616 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.616 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.616 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.616 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.616 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.617 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.617 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.617 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.617 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.617 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.617 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.617 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.617 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.618 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.618 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.618 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.618 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.618 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.618 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.618 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.618 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.619 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.619 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.619 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.619 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.619 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.619 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.619 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.619 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.620 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.620 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.620 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.620 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.620 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.620 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.620 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.620 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.621 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.621 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.621 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.621 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_destination": { 2023-07-14T18:34:46.621 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_destination", 2023-07-14T18:34:46.621 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.621 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.621 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.622 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.622 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.622 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.622 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.622 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Email address to send alerts to", 2023-07-14T18:34:46.622 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.622 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.622 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.623 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.623 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_from_name": { 2023-07-14T18:34:46.623 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_from_name", 2023-07-14T18:34:46.623 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.623 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.623 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.623 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "Ceph", 2023-07-14T18:34:46.623 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.624 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.624 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.624 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Email From: name", 2023-07-14T18:34:46.624 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.624 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.624 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.624 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.625 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_host": { 2023-07-14T18:34:46.625 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_host", 2023-07-14T18:34:46.625 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.625 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.625 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.625 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.625 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.625 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.626 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.626 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP server", 2023-07-14T18:34:46.626 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.626 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.626 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.626 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.626 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_password": { 2023-07-14T18:34:46.626 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_password", 2023-07-14T18:34:46.627 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.627 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.627 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.627 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.627 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.627 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.627 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.627 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Password to authenticate with", 2023-07-14T18:34:46.628 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.628 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.628 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.628 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.628 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_port": { 2023-07-14T18:34:46.628 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_port", 2023-07-14T18:34:46.628 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.628 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.629 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.629 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "465", 2023-07-14T18:34:46.629 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.629 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.629 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.629 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP port", 2023-07-14T18:34:46.629 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.629 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.630 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.630 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.630 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_sender": { 2023-07-14T18:34:46.630 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_sender", 2023-07-14T18:34:46.630 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.630 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.630 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.630 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.631 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.631 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.631 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.631 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP envelope sender", 2023-07-14T18:34:46.631 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.631 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.631 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.631 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.632 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_ssl": { 2023-07-14T18:34:46.632 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_ssl", 2023-07-14T18:34:46.632 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.632 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.632 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.632 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.632 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.632 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.633 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.633 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-14T18:34:46.633 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.633 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.633 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.633 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.633 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_user": { 2023-07-14T18:34:46.633 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_user", 2023-07-14T18:34:46.634 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.634 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.634 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.634 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.634 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.634 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.634 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.634 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "User to authenticate as", 2023-07-14T18:34:46.635 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.635 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.635 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.635 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.635 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.635 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.635 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.635 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "balancer", 2023-07-14T18:34:46.636 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.636 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.636 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.636 INFO:teuthology.orchestra.run.smithi133.stdout: "active": { 2023-07-14T18:34:46.636 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "active", 2023-07-14T18:34:46.636 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.636 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.637 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.637 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.637 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.637 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.637 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.637 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "automatically balance PGs across cluster", 2023-07-14T18:34:46.637 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.637 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.638 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.638 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.638 INFO:teuthology.orchestra.run.smithi133.stdout: "begin_time": { 2023-07-14T18:34:46.638 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "begin_time", 2023-07-14T18:34:46.638 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.638 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.638 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.638 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0000", 2023-07-14T18:34:46.639 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.639 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.639 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.639 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "beginning time of day to automatically balance", 2023-07-14T18:34:46.639 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-14T18:34:46.639 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.639 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.639 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.640 INFO:teuthology.orchestra.run.smithi133.stdout: "begin_weekday": { 2023-07-14T18:34:46.640 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "begin_weekday", 2023-07-14T18:34:46.640 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-07-14T18:34:46.640 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.640 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.640 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-07-14T18:34:46.640 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0", 2023-07-14T18:34:46.641 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "6", 2023-07-14T18:34:46.641 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.641 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-14T18:34:46.641 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-14T18:34:46.641 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.641 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.641 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.641 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_max_iterations": { 2023-07-14T18:34:46.642 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_max_iterations", 2023-07-14T18:34:46.642 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-07-14T18:34:46.642 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.642 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.642 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "25", 2023-07-14T18:34:46.642 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2023-07-14T18:34:46.642 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "250", 2023-07-14T18:34:46.642 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.643 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-14T18:34:46.643 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.643 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.643 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.643 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.643 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_metrics": { 2023-07-14T18:34:46.643 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_metrics", 2023-07-14T18:34:46.643 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.644 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.644 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.644 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "pgs,objects,bytes", 2023-07-14T18:34:46.644 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.644 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.644 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.644 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-14T18:34:46.645 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-14T18:34:46.645 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.645 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.645 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.645 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_step": { 2023-07-14T18:34:46.645 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_step", 2023-07-14T18:34:46.645 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-07-14T18:34:46.645 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.646 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.646 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0.5", 2023-07-14T18:34:46.646 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0.001", 2023-07-14T18:34:46.646 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "0.999", 2023-07-14T18:34:46.646 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.646 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "aggressiveness of optimization", 2023-07-14T18:34:46.646 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-14T18:34:46.646 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.647 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.647 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.647 INFO:teuthology.orchestra.run.smithi133.stdout: "end_time": { 2023-07-14T18:34:46.647 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "end_time", 2023-07-14T18:34:46.647 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.647 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.647 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.647 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "2359", 2023-07-14T18:34:46.648 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.648 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.648 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.648 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "ending time of day to automatically balance", 2023-07-14T18:34:46.648 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-14T18:34:46.648 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.648 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.648 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.649 INFO:teuthology.orchestra.run.smithi133.stdout: "end_weekday": { 2023-07-14T18:34:46.649 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "end_weekday", 2023-07-14T18:34:46.649 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-07-14T18:34:46.649 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.649 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.649 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-07-14T18:34:46.649 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0", 2023-07-14T18:34:46.649 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "6", 2023-07-14T18:34:46.650 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.650 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-14T18:34:46.650 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-14T18:34:46.650 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.650 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.650 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.650 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.650 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.651 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.651 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.651 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.651 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.651 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.651 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.651 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.651 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.652 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.652 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.652 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.652 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.652 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.652 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.652 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.652 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.653 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.653 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.653 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.653 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.653 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.653 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.653 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.653 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.654 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.654 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.654 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.654 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.654 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.654 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.654 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.654 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.655 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.655 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.655 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.655 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.655 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.655 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.655 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.655 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.656 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.656 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.656 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.656 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.656 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.656 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.656 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.657 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.657 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.657 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.657 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.657 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.657 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.657 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.657 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.658 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.658 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.658 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.658 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.658 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.658 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.658 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.658 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.659 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.659 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.659 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.659 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.659 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.659 INFO:teuthology.orchestra.run.smithi133.stdout: "min_score": { 2023-07-14T18:34:46.659 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "min_score", 2023-07-14T18:34:46.659 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-07-14T18:34:46.660 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.660 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.660 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-07-14T18:34:46.660 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.660 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.660 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.660 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-14T18:34:46.660 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.661 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.661 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.661 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.661 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": { 2023-07-14T18:34:46.661 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mode", 2023-07-14T18:34:46.661 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.661 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.661 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.662 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "upmap", 2023-07-14T18:34:46.662 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.662 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.662 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.662 INFO:teuthology.orchestra.run.smithi133.stdout: "crush-compat", 2023-07-14T18:34:46.662 INFO:teuthology.orchestra.run.smithi133.stdout: "none", 2023-07-14T18:34:46.662 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap" 2023-07-14T18:34:46.663 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.663 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Balancer mode", 2023-07-14T18:34:46.663 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.663 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.663 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.663 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.663 INFO:teuthology.orchestra.run.smithi133.stdout: "pool_ids": { 2023-07-14T18:34:46.663 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pool_ids", 2023-07-14T18:34:46.664 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.664 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.664 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.664 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.664 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.664 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.664 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.664 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-14T18:34:46.665 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.665 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.665 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.665 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.665 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-07-14T18:34:46.665 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-07-14T18:34:46.665 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.665 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.666 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.666 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-07-14T18:34:46.666 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.666 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.666 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.666 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-14T18:34:46.666 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.666 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.667 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.667 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.667 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap_max_deviation": { 2023-07-14T18:34:46.667 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "upmap_max_deviation", 2023-07-14T18:34:46.667 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.667 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.667 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.667 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2023-07-14T18:34:46.668 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2023-07-14T18:34:46.668 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.668 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.668 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-14T18:34:46.668 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-14T18:34:46.668 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.668 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.668 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.669 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap_max_optimizations": { 2023-07-14T18:34:46.669 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "upmap_max_optimizations", 2023-07-14T18:34:46.669 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-07-14T18:34:46.669 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.669 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.669 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-07-14T18:34:46.669 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.669 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.670 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.670 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-14T18:34:46.670 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.670 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.670 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.670 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.670 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.670 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.671 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.671 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cephadm", 2023-07-14T18:34:46.671 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.671 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.671 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.671 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_down_multiplier": { 2023-07-14T18:34:46.671 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_down_multiplier", 2023-07-14T18:34:46.671 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-07-14T18:34:46.672 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.672 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.672 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3.0", 2023-07-14T18:34:46.672 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.672 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.672 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.672 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-14T18:34:46.673 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.673 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.673 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.673 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.673 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_refresh_rate": { 2023-07-14T18:34:46.673 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_refresh_rate", 2023-07-14T18:34:46.673 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.673 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.673 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.674 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "20", 2023-07-14T18:34:46.674 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.674 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.674 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.674 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-14T18:34:46.674 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.674 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.675 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.675 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.675 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_starting_port": { 2023-07-14T18:34:46.675 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_starting_port", 2023-07-14T18:34:46.675 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.675 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.675 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.675 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "4721", 2023-07-14T18:34:46.675 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.676 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.676 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.676 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-14T18:34:46.676 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.676 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.676 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.676 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.677 INFO:teuthology.orchestra.run.smithi133.stdout: "alertmanager_web_password": { 2023-07-14T18:34:46.677 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "alertmanager_web_password", 2023-07-14T18:34:46.677 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.677 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.677 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.677 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-07-14T18:34:46.677 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.677 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.678 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.678 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Alertmanager web password", 2023-07-14T18:34:46.678 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.678 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.678 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.678 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.678 INFO:teuthology.orchestra.run.smithi133.stdout: "alertmanager_web_user": { 2023-07-14T18:34:46.678 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "alertmanager_web_user", 2023-07-14T18:34:46.679 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.679 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.679 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.679 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-07-14T18:34:46.679 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.679 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.679 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.679 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Alertmanager web user", 2023-07-14T18:34:46.679 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.680 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.680 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.680 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.680 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_ptrace": { 2023-07-14T18:34:46.680 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_ptrace", 2023-07-14T18:34:46.680 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.680 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.681 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.681 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.681 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.681 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.681 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.681 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-14T18:34:46.681 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-14T18:34:46.681 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.682 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.682 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.682 INFO:teuthology.orchestra.run.smithi133.stdout: "autotune_interval": { 2023-07-14T18:34:46.682 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "autotune_interval", 2023-07-14T18:34:46.682 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.682 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.682 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.682 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-07-14T18:34:46.683 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.683 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.683 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.683 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-14T18:34:46.683 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.683 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.683 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.683 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.684 INFO:teuthology.orchestra.run.smithi133.stdout: "autotune_memory_target_ratio": { 2023-07-14T18:34:46.684 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "autotune_memory_target_ratio", 2023-07-14T18:34:46.684 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-07-14T18:34:46.684 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.684 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.684 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0.7", 2023-07-14T18:34:46.684 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.684 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.685 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.685 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-14T18:34:46.685 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.685 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.685 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.685 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.685 INFO:teuthology.orchestra.run.smithi133.stdout: "cgroups_split": { 2023-07-14T18:34:46.685 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cgroups_split", 2023-07-14T18:34:46.686 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.686 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.686 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.686 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.686 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.686 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.686 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.686 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-14T18:34:46.687 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.687 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.687 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.687 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.687 INFO:teuthology.orchestra.run.smithi133.stdout: "config_checks_enabled": { 2023-07-14T18:34:46.687 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_checks_enabled", 2023-07-14T18:34:46.687 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.688 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.688 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.688 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.688 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.688 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.688 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.688 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-14T18:34:46.688 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.689 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.689 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.689 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.689 INFO:teuthology.orchestra.run.smithi133.stdout: "config_dashboard": { 2023-07-14T18:34:46.689 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_dashboard", 2023-07-14T18:34:46.689 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.689 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.689 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.690 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.690 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.690 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.690 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.690 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-14T18:34:46.690 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.690 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.690 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.691 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.691 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_alertmanager": { 2023-07-14T18:34:46.691 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_alertmanager", 2023-07-14T18:34:46.691 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.691 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.691 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.691 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-14T18:34:46.691 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.692 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.692 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.692 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2023-07-14T18:34:46.692 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.692 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.692 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.692 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.692 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_base": { 2023-07-14T18:34:46.693 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_base", 2023-07-14T18:34:46.693 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.693 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.693 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.693 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-14T18:34:46.693 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.693 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.693 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.694 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Container image name, without the tag", 2023-07-14T18:34:46.694 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.694 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.694 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.694 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.694 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_elasticsearch": { 2023-07-14T18:34:46.694 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_elasticsearch", 2023-07-14T18:34:46.694 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.695 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.695 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.695 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-14T18:34:46.695 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.695 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.695 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.695 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "elasticsearch container image", 2023-07-14T18:34:46.695 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.696 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.696 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.696 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.696 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_grafana": { 2023-07-14T18:34:46.696 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_grafana", 2023-07-14T18:34:46.696 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.696 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.696 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.697 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-14T18:34:46.697 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.697 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.697 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.697 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2023-07-14T18:34:46.697 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.697 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.697 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.698 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.698 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_haproxy": { 2023-07-14T18:34:46.698 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_haproxy", 2023-07-14T18:34:46.698 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.698 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.698 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.698 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-14T18:34:46.698 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.699 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.699 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.699 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "HAproxy container image", 2023-07-14T18:34:46.699 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.699 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.699 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.699 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.699 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_agent": { 2023-07-14T18:34:46.700 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_agent", 2023-07-14T18:34:46.700 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.700 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.700 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.700 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-14T18:34:46.700 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.700 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.700 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.701 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger agent container image", 2023-07-14T18:34:46.701 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.701 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.701 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.701 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.701 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_collector": { 2023-07-14T18:34:46.701 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_collector", 2023-07-14T18:34:46.701 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.702 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.702 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.702 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-14T18:34:46.702 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.702 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.702 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.702 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger collector container image", 2023-07-14T18:34:46.703 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.703 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.703 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.703 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.703 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_query": { 2023-07-14T18:34:46.703 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_query", 2023-07-14T18:34:46.703 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.703 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.704 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.704 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-14T18:34:46.704 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.704 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.704 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.704 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger query container image", 2023-07-14T18:34:46.704 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.704 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.705 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.705 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.705 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_keepalived": { 2023-07-14T18:34:46.705 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_keepalived", 2023-07-14T18:34:46.705 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.705 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.705 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.706 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-14T18:34:46.706 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.706 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.706 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.706 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Keepalived container image", 2023-07-14T18:34:46.706 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.706 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.706 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.707 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.707 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_loki": { 2023-07-14T18:34:46.707 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_loki", 2023-07-14T18:34:46.707 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.707 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.707 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.707 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-14T18:34:46.707 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.708 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.708 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.708 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Loki container image", 2023-07-14T18:34:46.708 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.708 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.708 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.708 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.708 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_node_exporter": { 2023-07-14T18:34:46.709 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_node_exporter", 2023-07-14T18:34:46.709 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.709 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.709 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.709 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-14T18:34:46.709 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.709 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.709 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.710 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2023-07-14T18:34:46.710 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.710 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.710 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.710 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.710 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_prometheus": { 2023-07-14T18:34:46.710 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_prometheus", 2023-07-14T18:34:46.710 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.711 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.711 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.711 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-14T18:34:46.711 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.711 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.711 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.711 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2023-07-14T18:34:46.711 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.712 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.712 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.712 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.712 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_promtail": { 2023-07-14T18:34:46.712 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_promtail", 2023-07-14T18:34:46.712 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.712 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.713 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.713 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-14T18:34:46.713 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.713 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.713 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.713 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Promtail container image", 2023-07-14T18:34:46.713 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.713 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.714 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.714 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.714 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_snmp_gateway": { 2023-07-14T18:34:46.714 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_snmp_gateway", 2023-07-14T18:34:46.714 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.714 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.714 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.714 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-14T18:34:46.715 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.715 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.715 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.715 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SNMP Gateway container image", 2023-07-14T18:34:46.715 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.715 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.715 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.715 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.716 INFO:teuthology.orchestra.run.smithi133.stdout: "container_init": { 2023-07-14T18:34:46.716 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_init", 2023-07-14T18:34:46.716 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.716 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.716 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.716 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.716 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.716 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.717 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.717 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Run podman/docker with `--init`", 2023-07-14T18:34:46.717 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.717 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.717 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.717 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.717 INFO:teuthology.orchestra.run.smithi133.stdout: "daemon_cache_timeout": { 2023-07-14T18:34:46.717 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "daemon_cache_timeout", 2023-07-14T18:34:46.718 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.718 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.718 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.718 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-07-14T18:34:46.718 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.718 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.718 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.718 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-14T18:34:46.719 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.719 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.719 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.719 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.719 INFO:teuthology.orchestra.run.smithi133.stdout: "default_cephadm_command_timeout": { 2023-07-14T18:34:46.719 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "default_cephadm_command_timeout", 2023-07-14T18:34:46.719 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.719 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.720 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.720 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "900", 2023-07-14T18:34:46.720 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.720 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.720 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.720 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-14T18:34:46.720 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.720 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.721 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.721 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.721 INFO:teuthology.orchestra.run.smithi133.stdout: "default_registry": { 2023-07-14T18:34:46.721 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "default_registry", 2023-07-14T18:34:46.721 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.721 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.721 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.721 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io", 2023-07-14T18:34:46.722 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.722 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.722 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.722 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-14T18:34:46.722 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.722 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.722 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.722 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.723 INFO:teuthology.orchestra.run.smithi133.stdout: "device_cache_timeout": { 2023-07-14T18:34:46.723 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_cache_timeout", 2023-07-14T18:34:46.723 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.723 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.723 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.723 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1800", 2023-07-14T18:34:46.723 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.723 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.724 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.724 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache device inventory", 2023-07-14T18:34:46.724 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.724 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.724 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.724 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.724 INFO:teuthology.orchestra.run.smithi133.stdout: "device_enhanced_scan": { 2023-07-14T18:34:46.724 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_enhanced_scan", 2023-07-14T18:34:46.725 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.725 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.725 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.725 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.725 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.725 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.725 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.725 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-14T18:34:46.726 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.726 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.726 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.726 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.726 INFO:teuthology.orchestra.run.smithi133.stdout: "facts_cache_timeout": { 2023-07-14T18:34:46.726 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "facts_cache_timeout", 2023-07-14T18:34:46.726 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.727 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.727 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.727 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-07-14T18:34:46.727 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.727 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.727 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.727 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache host facts data", 2023-07-14T18:34:46.727 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.728 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.728 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.728 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.728 INFO:teuthology.orchestra.run.smithi133.stdout: "host_check_interval": { 2023-07-14T18:34:46.728 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "host_check_interval", 2023-07-14T18:34:46.728 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.728 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.729 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.729 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-07-14T18:34:46.729 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.729 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.729 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.729 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to perform a host check", 2023-07-14T18:34:46.729 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.729 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.730 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.730 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.730 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.730 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.730 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.730 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.730 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.730 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.731 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.731 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.731 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.731 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.731 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.731 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.731 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.731 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.732 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.732 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.732 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.732 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.732 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.732 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.732 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.732 INFO:teuthology.orchestra.run.smithi133.stdout: "log_refresh_metadata": { 2023-07-14T18:34:46.733 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_refresh_metadata", 2023-07-14T18:34:46.733 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.733 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.733 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.733 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.733 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.733 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.733 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.734 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-14T18:34:46.734 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.734 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.734 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.734 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.734 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.734 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.734 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.735 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.735 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.735 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.735 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.735 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.735 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.735 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-14T18:34:46.735 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.736 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.736 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.736 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.736 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.736 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.736 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.736 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.737 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.737 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.737 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.737 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.737 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.737 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.737 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.737 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.738 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.738 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.738 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.738 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.738 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.738 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.738 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.738 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.739 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.739 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.739 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.739 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.739 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.739 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.739 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.739 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.740 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.740 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.740 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.740 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.740 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.740 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.740 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.740 INFO:teuthology.orchestra.run.smithi133.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-14T18:34:46.741 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-14T18:34:46.741 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.741 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.741 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.741 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.741 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.741 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.741 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.742 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-14T18:34:46.742 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.742 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.742 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.742 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.742 INFO:teuthology.orchestra.run.smithi133.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-14T18:34:46.742 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-14T18:34:46.742 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.743 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.743 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.743 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "*", 2023-07-14T18:34:46.743 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.743 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.743 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.743 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-14T18:34:46.743 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.744 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.744 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.744 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.744 INFO:teuthology.orchestra.run.smithi133.stdout: "max_count_per_host": { 2023-07-14T18:34:46.744 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_count_per_host", 2023-07-14T18:34:46.744 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.744 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.744 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.745 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-07-14T18:34:46.745 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.745 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.745 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.745 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "max number of daemons per service per host", 2023-07-14T18:34:46.745 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.745 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.745 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.746 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.746 INFO:teuthology.orchestra.run.smithi133.stdout: "max_osd_draining_count": { 2023-07-14T18:34:46.746 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_osd_draining_count", 2023-07-14T18:34:46.746 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.746 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.746 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.746 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-07-14T18:34:46.747 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.747 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.747 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.747 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-14T18:34:46.747 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.747 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.747 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.747 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.748 INFO:teuthology.orchestra.run.smithi133.stdout: "migration_current": { 2023-07-14T18:34:46.748 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "migration_current", 2023-07-14T18:34:46.748 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.748 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.748 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.748 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.748 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.748 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.749 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.749 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "internal - do not modify", 2023-07-14T18:34:46.749 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.749 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.749 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.749 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.749 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": { 2023-07-14T18:34:46.750 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mode", 2023-07-14T18:34:46.750 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.750 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.750 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.750 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "root", 2023-07-14T18:34:46.750 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.750 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.750 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.751 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm-package", 2023-07-14T18:34:46.751 INFO:teuthology.orchestra.run.smithi133.stdout: "root" 2023-07-14T18:34:46.751 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.751 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "mode for remote execution of cephadm", 2023-07-14T18:34:46.751 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.751 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.751 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.751 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.752 INFO:teuthology.orchestra.run.smithi133.stdout: "prometheus_alerts_path": { 2023-07-14T18:34:46.752 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus_alerts_path", 2023-07-14T18:34:46.752 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.752 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.752 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.752 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-14T18:34:46.752 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.752 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.753 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.753 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-14T18:34:46.753 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.753 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.753 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.753 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.753 INFO:teuthology.orchestra.run.smithi133.stdout: "prometheus_web_password": { 2023-07-14T18:34:46.753 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus_web_password", 2023-07-14T18:34:46.754 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.754 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.754 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.754 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-07-14T18:34:46.754 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.754 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.754 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.754 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus web password", 2023-07-14T18:34:46.755 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.755 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.755 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.755 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.755 INFO:teuthology.orchestra.run.smithi133.stdout: "prometheus_web_user": { 2023-07-14T18:34:46.755 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus_web_user", 2023-07-14T18:34:46.755 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.755 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.756 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.756 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-07-14T18:34:46.756 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.756 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.756 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.756 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus web user", 2023-07-14T18:34:46.756 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.757 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.757 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.757 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.757 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_insecure": { 2023-07-14T18:34:46.757 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_insecure", 2023-07-14T18:34:46.757 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.757 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.757 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.758 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.758 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.758 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.758 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.758 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-14T18:34:46.758 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.758 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.758 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.759 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.759 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_password": { 2023-07-14T18:34:46.759 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_password", 2023-07-14T18:34:46.759 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.759 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.759 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.759 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.759 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.760 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.760 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.760 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-14T18:34:46.760 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.760 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.760 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.760 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.760 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_url": { 2023-07-14T18:34:46.761 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_url", 2023-07-14T18:34:46.761 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.761 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.761 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.761 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.761 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.761 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.761 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.762 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-14T18:34:46.762 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.762 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.762 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.762 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.762 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_username": { 2023-07-14T18:34:46.762 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_username", 2023-07-14T18:34:46.762 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.763 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.763 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.763 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.763 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.763 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.763 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.763 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-14T18:34:46.763 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.764 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.764 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.764 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.764 INFO:teuthology.orchestra.run.smithi133.stdout: "secure_monitoring_stack": { 2023-07-14T18:34:46.764 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "secure_monitoring_stack", 2023-07-14T18:34:46.764 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.764 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.765 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.765 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.765 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.765 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.765 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.765 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-14T18:34:46.765 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.765 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.766 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.766 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.766 INFO:teuthology.orchestra.run.smithi133.stdout: "service_discovery_port": { 2023-07-14T18:34:46.766 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "service_discovery_port", 2023-07-14T18:34:46.766 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.766 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.766 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.767 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8765", 2023-07-14T18:34:46.767 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.767 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.767 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.767 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "cephadm service discovery port", 2023-07-14T18:34:46.768 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.768 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.768 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.768 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.768 INFO:teuthology.orchestra.run.smithi133.stdout: "ssh_config_file": { 2023-07-14T18:34:46.768 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssh_config_file", 2023-07-14T18:34:46.768 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.768 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.769 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.769 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.769 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.769 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.769 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.769 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-14T18:34:46.769 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.769 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.770 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.770 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.770 INFO:teuthology.orchestra.run.smithi133.stdout: "use_agent": { 2023-07-14T18:34:46.770 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "use_agent", 2023-07-14T18:34:46.770 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.770 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.770 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.770 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.771 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.771 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.771 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.771 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-14T18:34:46.771 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.771 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.771 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.771 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.772 INFO:teuthology.orchestra.run.smithi133.stdout: "use_repo_digest": { 2023-07-14T18:34:46.772 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "use_repo_digest", 2023-07-14T18:34:46.772 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.772 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.772 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.772 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.772 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.773 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.773 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.773 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-14T18:34:46.773 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.773 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.773 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.773 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.773 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_failed_host_check": { 2023-07-14T18:34:46.774 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_failed_host_check", 2023-07-14T18:34:46.774 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.774 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.774 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.774 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.774 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.774 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.774 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.775 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if the host check fails", 2023-07-14T18:34:46.775 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.775 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.775 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.775 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.775 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_stray_daemons": { 2023-07-14T18:34:46.775 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_stray_daemons", 2023-07-14T18:34:46.775 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.776 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.776 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.776 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.776 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.776 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.776 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.776 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-14T18:34:46.777 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.777 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.777 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.777 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.777 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_stray_hosts": { 2023-07-14T18:34:46.777 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_stray_hosts", 2023-07-14T18:34:46.777 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.777 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.778 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.778 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.778 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.778 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.778 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.778 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-14T18:34:46.778 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.778 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.779 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.779 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.779 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.779 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.779 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.779 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crash", 2023-07-14T18:34:46.779 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.779 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.780 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.780 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.780 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.780 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.780 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.780 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.780 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.780 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.781 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.781 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.781 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.781 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.781 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.781 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.781 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.781 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.782 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.782 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.782 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.782 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.782 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.782 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.782 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.782 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.783 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.783 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.783 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.783 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.783 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.783 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.783 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.783 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.784 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.784 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.784 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.784 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.784 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.784 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.784 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.784 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.785 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.785 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.785 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.785 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.785 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.785 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.785 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.785 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.786 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.786 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.786 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.786 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.786 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.786 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.786 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.786 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.787 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.787 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.787 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.787 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.787 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.787 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.787 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.787 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.788 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.788 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.788 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.788 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.788 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.788 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.788 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.788 INFO:teuthology.orchestra.run.smithi133.stdout: "retain_interval": { 2023-07-14T18:34:46.789 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "retain_interval", 2023-07-14T18:34:46.789 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.789 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.789 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.789 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "31536000", 2023-07-14T18:34:46.789 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.789 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.789 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.790 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-14T18:34:46.790 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.790 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.790 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.790 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.790 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_recent_interval": { 2023-07-14T18:34:46.790 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_recent_interval", 2023-07-14T18:34:46.790 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.791 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.791 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.791 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1209600", 2023-07-14T18:34:46.791 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.791 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.791 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.791 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-14T18:34:46.791 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.792 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.792 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.792 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.792 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.792 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.792 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.792 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "dashboard", 2023-07-14T18:34:46.792 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.793 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.793 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.793 INFO:teuthology.orchestra.run.smithi133.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-14T18:34:46.793 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-14T18:34:46.793 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.793 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.793 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.793 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-07-14T18:34:46.794 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.794 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.794 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.794 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.794 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.794 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.794 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.794 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.795 INFO:teuthology.orchestra.run.smithi133.stdout: "ALERTMANAGER_API_HOST": { 2023-07-14T18:34:46.795 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-14T18:34:46.795 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.795 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.795 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.795 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.795 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.795 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.796 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.796 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.796 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.796 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.796 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.796 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.796 INFO:teuthology.orchestra.run.smithi133.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-14T18:34:46.797 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-14T18:34:46.797 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.797 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.797 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.797 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.797 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.797 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.797 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.798 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.798 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.798 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.798 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.798 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.798 INFO:teuthology.orchestra.run.smithi133.stdout: "AUDIT_API_ENABLED": { 2023-07-14T18:34:46.798 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "AUDIT_API_ENABLED", 2023-07-14T18:34:46.798 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.799 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.799 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.799 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.799 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.799 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.799 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.799 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.799 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.800 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.800 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.800 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.800 INFO:teuthology.orchestra.run.smithi133.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-14T18:34:46.800 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-14T18:34:46.800 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.800 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.800 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.801 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.801 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.801 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.801 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.801 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.801 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.801 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.801 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.802 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.802 INFO:teuthology.orchestra.run.smithi133.stdout: "ENABLE_BROWSABLE_API": { 2023-07-14T18:34:46.802 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-14T18:34:46.802 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.802 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.802 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.802 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.802 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.803 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.803 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.803 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.803 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.803 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.803 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.803 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.803 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-14T18:34:46.804 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-14T18:34:46.804 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.804 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.804 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.804 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.804 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.804 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.804 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.805 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.805 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.805 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.805 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.805 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.805 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-14T18:34:46.805 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-14T18:34:46.805 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.806 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.806 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.806 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.806 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.806 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.806 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.806 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.806 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.807 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.807 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.807 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.807 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-14T18:34:46.807 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-14T18:34:46.807 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.807 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.807 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.808 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.808 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.808 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.808 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.808 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.808 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.808 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.809 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.809 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.809 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-14T18:34:46.809 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-14T18:34:46.809 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.809 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.809 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.809 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.810 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.810 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.810 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.810 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.810 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.810 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.810 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.810 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.811 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-14T18:34:46.811 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-14T18:34:46.811 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.811 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.811 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.811 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.811 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.811 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.812 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.812 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.812 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.812 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.812 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.812 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.812 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-14T18:34:46.812 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-14T18:34:46.813 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.813 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.813 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.813 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.813 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.813 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.813 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.813 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.814 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.814 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.814 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.814 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.814 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-14T18:34:46.814 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-14T18:34:46.814 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.814 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.815 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.815 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.815 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.815 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.815 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.815 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.815 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.815 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.816 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.816 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.816 INFO:teuthology.orchestra.run.smithi133.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-14T18:34:46.816 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-14T18:34:46.816 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.816 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.816 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.817 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.817 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.817 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.817 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.817 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.817 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.817 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.817 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.818 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.818 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_PASSWORD": { 2023-07-14T18:34:46.818 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-14T18:34:46.818 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.818 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.818 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.818 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-07-14T18:34:46.818 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.819 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.819 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.819 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.819 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.819 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.819 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.819 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.819 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-14T18:34:46.820 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-14T18:34:46.820 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.820 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.820 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.820 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.820 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.820 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.821 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.821 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.821 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.821 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.821 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.821 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.821 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_URL": { 2023-07-14T18:34:46.821 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_URL", 2023-07-14T18:34:46.822 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.822 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.822 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.822 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.822 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.822 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.822 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.822 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.823 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.823 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.823 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.823 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.823 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_USERNAME": { 2023-07-14T18:34:46.823 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-14T18:34:46.823 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.823 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.824 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.824 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-07-14T18:34:46.824 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.824 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.824 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.824 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.824 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.824 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.825 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.825 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.825 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-14T18:34:46.825 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-14T18:34:46.825 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.825 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.825 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.825 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.826 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.826 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.826 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.826 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.826 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.826 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.826 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.826 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.827 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-14T18:34:46.827 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-14T18:34:46.827 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.827 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.827 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.827 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.827 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.827 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.828 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.828 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.828 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.828 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.828 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.828 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.828 INFO:teuthology.orchestra.run.smithi133.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-14T18:34:46.828 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-14T18:34:46.829 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.829 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.829 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.829 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.829 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.829 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.829 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.829 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.830 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.830 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.830 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.830 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.830 INFO:teuthology.orchestra.run.smithi133.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-14T18:34:46.830 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-14T18:34:46.830 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.830 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.831 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.831 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.831 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.831 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.831 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.831 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.831 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.831 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.832 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.832 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.832 INFO:teuthology.orchestra.run.smithi133.stdout: "PROMETHEUS_API_HOST": { 2023-07-14T18:34:46.832 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-14T18:34:46.832 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.832 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.832 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.832 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.833 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.833 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.833 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.833 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.833 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.833 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.833 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.833 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.834 INFO:teuthology.orchestra.run.smithi133.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-14T18:34:46.834 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-14T18:34:46.834 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.834 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.834 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.834 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.834 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.834 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.835 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.835 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.835 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.835 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.835 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.835 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.835 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-14T18:34:46.835 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-14T18:34:46.836 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.836 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.836 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.836 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.836 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.836 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.836 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.837 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.837 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.837 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.837 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.837 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.837 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-14T18:34:46.837 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-14T18:34:46.837 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.838 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.838 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.838 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.838 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.838 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.838 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.838 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.838 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.839 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.839 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.839 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.839 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-14T18:34:46.839 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-14T18:34:46.839 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.839 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.839 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.840 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.840 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.840 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.840 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.840 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.840 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.840 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.840 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.841 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.841 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-14T18:34:46.841 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-14T18:34:46.841 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.841 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.841 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.841 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.841 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.842 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.842 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.842 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.842 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.842 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.842 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.842 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.843 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-14T18:34:46.843 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-14T18:34:46.843 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.843 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.843 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.843 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.843 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.843 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.844 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.844 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.844 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.844 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.844 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.844 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.844 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-14T18:34:46.845 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-14T18:34:46.845 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.845 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.845 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.845 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.845 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.845 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.845 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.846 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.846 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.846 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.846 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.846 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.846 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-14T18:34:46.846 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-14T18:34:46.846 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.847 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.847 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.847 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.847 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.847 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.847 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.847 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.847 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.848 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.848 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.848 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.848 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_ENABLED": { 2023-07-14T18:34:46.848 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-14T18:34:46.848 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.848 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.848 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.849 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.849 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.849 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.849 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.849 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.849 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.849 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.849 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.850 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.850 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-14T18:34:46.850 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-14T18:34:46.850 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.850 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.850 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.850 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-14T18:34:46.850 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.851 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.851 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.851 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.851 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.851 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.851 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.851 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.851 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-14T18:34:46.852 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-14T18:34:46.852 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.852 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.852 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.852 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-07-14T18:34:46.852 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.852 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.852 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.853 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.853 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.853 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.853 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.853 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.853 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-14T18:34:46.853 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-14T18:34:46.853 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.854 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.854 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.854 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8", 2023-07-14T18:34:46.854 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.854 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.854 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.854 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.854 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.855 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.855 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.855 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.855 INFO:teuthology.orchestra.run.smithi133.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-14T18:34:46.855 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-14T18:34:46.855 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.855 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.855 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.856 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "45", 2023-07-14T18:34:46.856 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.856 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.856 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.856 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.856 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.856 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.856 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.857 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.857 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_ACCESS_KEY": { 2023-07-14T18:34:46.857 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-14T18:34:46.857 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.857 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.857 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.857 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.857 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.858 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.858 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.858 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.858 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.858 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.858 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.858 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.858 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-14T18:34:46.859 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-14T18:34:46.859 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.859 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.859 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.859 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-07-14T18:34:46.859 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.859 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.859 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.860 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.860 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.860 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.860 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.860 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.860 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_SECRET_KEY": { 2023-07-14T18:34:46.860 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-14T18:34:46.860 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.861 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.861 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.861 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.861 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.861 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.861 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.861 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.861 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.862 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.862 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.862 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.862 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_SSL_VERIFY": { 2023-07-14T18:34:46.862 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-14T18:34:46.862 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.862 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.862 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.863 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.863 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.863 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.863 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.863 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.863 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.863 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.863 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.864 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.864 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-14T18:34:46.864 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-14T18:34:46.864 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.864 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.864 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.864 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-07-14T18:34:46.864 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.865 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.865 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.865 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.865 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.865 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.865 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.865 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.866 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-14T18:34:46.866 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-14T18:34:46.866 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.866 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.866 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.866 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-07-14T18:34:46.866 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.866 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.867 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.867 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.867 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.867 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.867 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.867 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.867 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-14T18:34:46.867 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-14T18:34:46.868 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.868 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.868 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.868 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2023-07-14T18:34:46.868 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.868 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.868 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.868 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.869 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.869 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.869 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.869 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.869 INFO:teuthology.orchestra.run.smithi133.stdout: "cross_origin_url": { 2023-07-14T18:34:46.869 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cross_origin_url", 2023-07-14T18:34:46.869 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.869 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.870 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.870 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.870 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.870 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.870 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.870 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.870 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.870 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.871 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.871 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.871 INFO:teuthology.orchestra.run.smithi133.stdout: "crt_file": { 2023-07-14T18:34:46.871 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crt_file", 2023-07-14T18:34:46.871 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.871 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.871 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.871 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.872 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.872 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.872 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.872 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.872 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.872 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.872 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.872 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.873 INFO:teuthology.orchestra.run.smithi133.stdout: "debug": { 2023-07-14T18:34:46.873 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "debug", 2023-07-14T18:34:46.873 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.873 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.873 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.873 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.873 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.873 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.874 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.874 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable/disable debug options", 2023-07-14T18:34:46.874 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.874 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.874 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.874 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.874 INFO:teuthology.orchestra.run.smithi133.stdout: "jwt_token_ttl": { 2023-07-14T18:34:46.874 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "jwt_token_ttl", 2023-07-14T18:34:46.875 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.875 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.875 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.875 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "28800", 2023-07-14T18:34:46.875 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.875 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.875 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.875 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.876 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.876 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.876 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.876 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.876 INFO:teuthology.orchestra.run.smithi133.stdout: "key_file": { 2023-07-14T18:34:46.876 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "key_file", 2023-07-14T18:34:46.876 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.877 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.877 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.877 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.877 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.877 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.877 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.877 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.877 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.878 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.878 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.878 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.878 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.878 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.878 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.878 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.878 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.879 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.879 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.879 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.879 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.879 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.879 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.879 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.879 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.880 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.880 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.880 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.880 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.880 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.880 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.880 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.880 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.881 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.881 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.881 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.881 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.881 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.881 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.881 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.881 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.882 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.882 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.882 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.882 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.882 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.882 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.882 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.882 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.883 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.883 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.883 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.883 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.883 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.883 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.883 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.883 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.884 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.884 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.884 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.884 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.884 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.884 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.884 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.884 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.885 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.885 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.885 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.885 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.885 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.885 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.885 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.885 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.886 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.886 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.886 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.886 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.886 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.886 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.886 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.886 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.887 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.887 INFO:teuthology.orchestra.run.smithi133.stdout: "motd": { 2023-07-14T18:34:46.887 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "motd", 2023-07-14T18:34:46.887 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.887 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.887 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.887 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.887 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.888 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.888 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.888 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "The message of the day", 2023-07-14T18:34:46.888 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.888 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.888 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.888 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.888 INFO:teuthology.orchestra.run.smithi133.stdout: "redirect_resolve_ip_addr": { 2023-07-14T18:34:46.889 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "redirect_resolve_ip_addr", 2023-07-14T18:34:46.889 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.889 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.889 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.889 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.889 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.889 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.890 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.890 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.890 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.890 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.890 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.890 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.890 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2023-07-14T18:34:46.890 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2023-07-14T18:34:46.891 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.891 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.891 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.891 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "::", 2023-07-14T18:34:46.891 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.891 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.891 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.891 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.892 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.892 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.892 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.892 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.892 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2023-07-14T18:34:46.892 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2023-07-14T18:34:46.892 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.892 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.893 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.893 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8080", 2023-07-14T18:34:46.893 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.893 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.893 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.893 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.893 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.894 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.894 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.894 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.894 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl": { 2023-07-14T18:34:46.894 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl", 2023-07-14T18:34:46.894 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.894 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.894 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.895 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.895 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.895 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.895 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.895 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.895 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.895 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.895 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.896 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.896 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl_server_port": { 2023-07-14T18:34:46.896 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl_server_port", 2023-07-14T18:34:46.896 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.896 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.896 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.896 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8443", 2023-07-14T18:34:46.896 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.897 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.897 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.897 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.897 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.897 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.897 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.897 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.897 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_behaviour": { 2023-07-14T18:34:46.898 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_behaviour", 2023-07-14T18:34:46.898 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.898 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.898 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.898 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "redirect", 2023-07-14T18:34:46.898 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.898 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.898 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.899 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.899 INFO:teuthology.orchestra.run.smithi133.stdout: "redirect" 2023-07-14T18:34:46.899 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.899 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.899 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.899 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.899 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.899 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.900 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_error_status_code": { 2023-07-14T18:34:46.900 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_error_status_code", 2023-07-14T18:34:46.900 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.900 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.900 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.900 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "500", 2023-07-14T18:34:46.900 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "400", 2023-07-14T18:34:46.900 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "599", 2023-07-14T18:34:46.901 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.901 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.901 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.901 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.901 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.901 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.901 INFO:teuthology.orchestra.run.smithi133.stdout: "url_prefix": { 2023-07-14T18:34:46.901 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "url_prefix", 2023-07-14T18:34:46.902 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.902 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.902 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.902 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.902 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.902 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.902 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.902 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.903 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.903 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.903 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.903 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.903 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.903 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.903 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.903 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "devicehealth", 2023-07-14T18:34:46.904 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.904 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.904 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.904 INFO:teuthology.orchestra.run.smithi133.stdout: "enable_monitoring": { 2023-07-14T18:34:46.904 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enable_monitoring", 2023-07-14T18:34:46.904 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.904 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.904 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.905 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.905 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.905 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.905 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.905 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "monitor device health metrics", 2023-07-14T18:34:46.905 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.905 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.905 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.906 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.906 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.906 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.906 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.906 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.906 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.906 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.906 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.907 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.907 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.907 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.907 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.907 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.907 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.907 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.907 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.908 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.908 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.908 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.908 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.908 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.908 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.908 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.909 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.909 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.909 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.909 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.909 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.909 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.909 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.909 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.910 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.910 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.910 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.910 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.910 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.910 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.910 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.910 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.911 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.911 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.911 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.911 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.911 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.911 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.911 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.911 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.912 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.912 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.912 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.912 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.912 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.912 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.912 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.912 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.913 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.913 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.913 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.913 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.913 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.913 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.913 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.914 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.914 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.914 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.914 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.914 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.914 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.914 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.914 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.915 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.915 INFO:teuthology.orchestra.run.smithi133.stdout: "mark_out_threshold": { 2023-07-14T18:34:46.915 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mark_out_threshold", 2023-07-14T18:34:46.915 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.915 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.915 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.915 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "2419200", 2023-07-14T18:34:46.915 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.916 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.916 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.916 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-14T18:34:46.916 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.916 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.916 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.916 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.916 INFO:teuthology.orchestra.run.smithi133.stdout: "pool_name": { 2023-07-14T18:34:46.917 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pool_name", 2023-07-14T18:34:46.917 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.917 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.917 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.917 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "device_health_metrics", 2023-07-14T18:34:46.917 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.917 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.917 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.918 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-14T18:34:46.918 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.918 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.918 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.918 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.918 INFO:teuthology.orchestra.run.smithi133.stdout: "retention_period": { 2023-07-14T18:34:46.918 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "retention_period", 2023-07-14T18:34:46.918 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.919 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.919 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.919 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15552000", 2023-07-14T18:34:46.919 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.919 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.919 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.919 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long to retain device health metrics", 2023-07-14T18:34:46.919 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.920 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.920 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.920 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.920 INFO:teuthology.orchestra.run.smithi133.stdout: "scrape_frequency": { 2023-07-14T18:34:46.920 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "scrape_frequency", 2023-07-14T18:34:46.920 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.920 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.920 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.921 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "86400", 2023-07-14T18:34:46.921 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.921 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.921 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.921 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-14T18:34:46.921 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.921 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.921 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.922 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.922 INFO:teuthology.orchestra.run.smithi133.stdout: "self_heal": { 2023-07-14T18:34:46.922 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "self_heal", 2023-07-14T18:34:46.922 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.922 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.922 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.922 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:46.922 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.923 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.923 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.923 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-14T18:34:46.923 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.923 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.923 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.923 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.923 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-07-14T18:34:46.924 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-07-14T18:34:46.924 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.924 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.924 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.924 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-07-14T18:34:46.924 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.924 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.924 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.925 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to wake up and check device health", 2023-07-14T18:34:46.925 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.925 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.925 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.925 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.925 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_threshold": { 2023-07-14T18:34:46.925 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_threshold", 2023-07-14T18:34:46.925 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.926 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.926 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.926 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "7257600", 2023-07-14T18:34:46.926 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.926 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.926 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.926 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-14T18:34:46.926 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.927 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.927 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.927 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.927 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.927 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.927 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.927 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "diskprediction_local", 2023-07-14T18:34:46.927 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.928 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.928 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.928 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.928 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.928 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.928 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.928 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.929 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.929 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.929 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.929 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.929 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.929 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.929 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.929 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.930 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.930 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.930 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.930 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.930 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.930 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.930 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.930 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.931 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.931 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.931 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.931 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.931 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.931 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.931 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.931 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.932 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.932 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.932 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.932 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.932 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.932 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.932 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.932 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.933 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.933 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.933 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.933 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.933 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.933 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.933 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.933 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.934 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.934 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.934 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.934 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.934 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.934 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.934 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.934 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.935 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.935 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.935 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.935 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.935 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.935 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.935 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.935 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.936 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.936 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.936 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.936 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.936 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.936 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.936 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.936 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.937 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.937 INFO:teuthology.orchestra.run.smithi133.stdout: "predict_interval": { 2023-07-14T18:34:46.937 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "predict_interval", 2023-07-14T18:34:46.937 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.937 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.937 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.937 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "86400", 2023-07-14T18:34:46.937 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.938 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.938 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.938 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.938 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.938 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.938 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.938 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.938 INFO:teuthology.orchestra.run.smithi133.stdout: "predictor_model": { 2023-07-14T18:34:46.939 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "predictor_model", 2023-07-14T18:34:46.939 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.939 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.939 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.939 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "prophetstor", 2023-07-14T18:34:46.939 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.939 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.939 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.940 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.940 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.940 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.940 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.940 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.940 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-07-14T18:34:46.940 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-07-14T18:34:46.941 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.941 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.941 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.941 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-07-14T18:34:46.941 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.941 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.941 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.941 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.942 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.942 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.942 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.942 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.942 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.942 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.942 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.942 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "influx", 2023-07-14T18:34:46.943 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": false, 2023-07-14T18:34:46.943 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "influxdb python module not found", 2023-07-14T18:34:46.943 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.943 INFO:teuthology.orchestra.run.smithi133.stdout: "batch_size": { 2023-07-14T18:34:46.943 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "batch_size", 2023-07-14T18:34:46.943 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.943 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.943 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.944 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5000", 2023-07-14T18:34:46.944 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.944 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.944 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.944 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-14T18:34:46.944 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.944 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.944 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.945 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.945 INFO:teuthology.orchestra.run.smithi133.stdout: "database": { 2023-07-14T18:34:46.945 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "database", 2023-07-14T18:34:46.945 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.945 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.945 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.945 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "ceph", 2023-07-14T18:34:46.945 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.946 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.946 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.946 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-14T18:34:46.946 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.946 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.946 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.946 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.946 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": { 2023-07-14T18:34:46.947 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "hostname", 2023-07-14T18:34:46.947 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.947 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.947 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.947 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.947 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.947 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.947 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.948 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB server hostname", 2023-07-14T18:34:46.948 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.948 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.948 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.948 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.948 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-07-14T18:34:46.948 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-07-14T18:34:46.948 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:46.949 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.949 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.949 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "30", 2023-07-14T18:34:46.949 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "5", 2023-07-14T18:34:46.949 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.949 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.949 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-14T18:34:46.949 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.950 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.950 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.950 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.950 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.950 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.950 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.950 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.951 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.951 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.951 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.951 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.951 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.951 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.951 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.951 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.952 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.952 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.952 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.952 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.952 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.952 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.952 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.952 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.953 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.953 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.953 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.953 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.953 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.953 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.953 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.953 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.954 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.954 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.954 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.954 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.954 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.954 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.954 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.954 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.955 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.955 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.955 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.955 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.955 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.955 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.955 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.955 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.956 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.956 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.956 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.956 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.956 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.956 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.956 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.956 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.957 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.957 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.957 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.957 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.957 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.957 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.957 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.957 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.958 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.958 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.958 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.958 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.958 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.958 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.958 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.958 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.959 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.959 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.959 INFO:teuthology.orchestra.run.smithi133.stdout: "password": { 2023-07-14T18:34:46.959 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "password", 2023-07-14T18:34:46.959 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.959 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.959 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.959 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.960 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.960 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.960 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.960 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "password of InfluxDB server user", 2023-07-14T18:34:46.960 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.960 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.960 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.961 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.961 INFO:teuthology.orchestra.run.smithi133.stdout: "port": { 2023-07-14T18:34:46.961 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "port", 2023-07-14T18:34:46.961 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.961 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.961 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.961 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8086", 2023-07-14T18:34:46.962 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.962 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.962 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.962 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB server port", 2023-07-14T18:34:46.962 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.962 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.962 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.962 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.963 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl": { 2023-07-14T18:34:46.963 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl", 2023-07-14T18:34:46.963 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.963 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.963 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.963 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "false", 2023-07-14T18:34:46.963 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.963 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.964 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.964 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-14T18:34:46.964 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.964 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.964 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.964 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.964 INFO:teuthology.orchestra.run.smithi133.stdout: "threads": { 2023-07-14T18:34:46.964 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "threads", 2023-07-14T18:34:46.965 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.965 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.965 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.965 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2023-07-14T18:34:46.965 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2023-07-14T18:34:46.965 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "32", 2023-07-14T18:34:46.965 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.965 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-14T18:34:46.966 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.966 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.966 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.966 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.966 INFO:teuthology.orchestra.run.smithi133.stdout: "username": { 2023-07-14T18:34:46.966 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "username", 2023-07-14T18:34:46.966 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.966 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.967 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.967 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.967 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.967 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.967 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.967 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "username of InfluxDB server user", 2023-07-14T18:34:46.967 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.967 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.968 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.968 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.968 INFO:teuthology.orchestra.run.smithi133.stdout: "verify_ssl": { 2023-07-14T18:34:46.968 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "verify_ssl", 2023-07-14T18:34:46.968 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.968 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.968 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.969 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "true", 2023-07-14T18:34:46.969 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.969 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.969 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.969 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-14T18:34:46.969 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.969 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.969 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.970 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.970 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.970 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.970 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.970 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "insights", 2023-07-14T18:34:46.970 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.970 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.970 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.971 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.971 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.971 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.971 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.971 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.971 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.971 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.971 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.972 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.972 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.972 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.972 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.972 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.972 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.972 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.972 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.973 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.973 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.973 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.973 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.973 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.973 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.973 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.973 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.974 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.974 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.974 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.974 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.974 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.974 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.974 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.974 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.975 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.975 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.975 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.975 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.975 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.975 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.975 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.975 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.976 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.976 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.976 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.976 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.976 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.976 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.976 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.976 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.977 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.977 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.977 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.977 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.977 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.977 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.977 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.977 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.978 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.978 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.978 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.978 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.978 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.978 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.978 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.978 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.979 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.979 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.979 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.979 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.979 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.979 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.979 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.979 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.980 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.980 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "iostat", 2023-07-14T18:34:46.980 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.980 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.980 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.980 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.980 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.980 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.981 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.981 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.981 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.981 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.981 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.981 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.981 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.982 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.982 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.982 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.982 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.982 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.982 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.982 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.982 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.983 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.983 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.983 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.983 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.983 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.983 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.983 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.983 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.984 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.984 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.984 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.984 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.984 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.984 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.984 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.984 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.985 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.985 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.985 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.985 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.985 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.985 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.985 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.986 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.986 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.986 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.986 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.986 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.986 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.986 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.986 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.987 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.987 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.987 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.987 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.987 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.987 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.987 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.987 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:46.988 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:46.988 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.988 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.988 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.988 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.988 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.988 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.988 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.989 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.989 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.989 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.989 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.989 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.989 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:46.989 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.989 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:46.990 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "k8sevents", 2023-07-14T18:34:46.990 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:46.990 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:46.990 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:46.990 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_event_retention_days": { 2023-07-14T18:34:46.990 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ceph_event_retention_days", 2023-07-14T18:34:46.990 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.990 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.991 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.991 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "7", 2023-07-14T18:34:46.991 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.991 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.991 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.991 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-14T18:34:46.991 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.991 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.992 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.992 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.992 INFO:teuthology.orchestra.run.smithi133.stdout: "config_check_secs": { 2023-07-14T18:34:46.992 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_check_secs", 2023-07-14T18:34:46.992 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:46.992 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.992 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:46.993 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-07-14T18:34:46.993 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "10", 2023-07-14T18:34:46.993 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.993 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.993 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-14T18:34:46.993 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.993 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.993 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.994 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.994 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:46.994 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:46.994 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.994 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.994 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.994 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:46.994 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.995 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.995 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.995 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.995 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.995 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:46.995 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:46.995 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:46.995 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:46.996 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:46.996 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.996 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.996 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.996 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.996 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.996 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:46.996 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:46.997 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:46.997 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.997 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.997 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:46.997 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.997 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.997 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:46.997 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:46.998 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:46.998 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:46.998 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:46.998 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:46.998 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:46.998 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:46.998 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:46.998 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:46.999 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:46.999 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:46.999 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:46.999 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:46.999 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:46.999 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:46.999 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:46.999 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.000 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.000 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.000 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.000 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.000 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.000 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.000 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.000 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.001 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.001 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.001 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.001 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.001 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.001 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.001 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.001 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.002 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.002 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.002 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.002 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.002 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.002 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.002 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.002 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.003 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.003 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.003 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "localpool", 2023-07-14T18:34:47.003 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.003 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.003 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.003 INFO:teuthology.orchestra.run.smithi133.stdout: "failure_domain": { 2023-07-14T18:34:47.003 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "failure_domain", 2023-07-14T18:34:47.004 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.004 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.004 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.004 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "host", 2023-07-14T18:34:47.004 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.004 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.004 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.004 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "failure domain for any created local pool", 2023-07-14T18:34:47.005 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-14T18:34:47.005 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.005 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.005 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.005 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.005 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.005 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.005 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.006 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.006 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.006 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.006 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.006 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.006 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.006 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.006 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.007 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.007 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.007 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.007 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.007 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.007 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.007 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.007 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.008 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.008 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.008 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.008 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.008 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.008 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.008 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.008 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.009 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.009 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.009 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.009 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.009 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.009 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.009 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.010 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.010 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.010 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.010 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.010 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.010 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.010 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.010 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.011 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.011 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.011 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.011 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.011 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.011 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.011 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.011 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.012 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.012 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.012 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.012 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.012 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.012 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.012 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.012 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.013 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.013 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.013 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.013 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.013 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.013 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.013 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.013 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.014 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.014 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.014 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.014 INFO:teuthology.orchestra.run.smithi133.stdout: "min_size": { 2023-07-14T18:34:47.014 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "min_size", 2023-07-14T18:34:47.014 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.014 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.014 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.015 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.015 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.015 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.015 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.015 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default min_size for any created local pool", 2023-07-14T18:34:47.015 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-14T18:34:47.015 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.015 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.016 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.016 INFO:teuthology.orchestra.run.smithi133.stdout: "num_rep": { 2023-07-14T18:34:47.016 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "num_rep", 2023-07-14T18:34:47.016 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.016 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.016 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.016 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3", 2023-07-14T18:34:47.017 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.017 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.017 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.017 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default replica count for any created local pool", 2023-07-14T18:34:47.017 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.017 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.017 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.017 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.018 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_num": { 2023-07-14T18:34:47.018 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pg_num", 2023-07-14T18:34:47.018 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.018 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.018 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.018 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "128", 2023-07-14T18:34:47.018 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.018 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.019 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.019 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default pg_num for any created local pool", 2023-07-14T18:34:47.019 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.019 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.019 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.019 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.019 INFO:teuthology.orchestra.run.smithi133.stdout: "prefix": { 2023-07-14T18:34:47.019 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prefix", 2023-07-14T18:34:47.020 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.020 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.020 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.020 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.020 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.020 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.020 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.020 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "name prefix for any created local pool", 2023-07-14T18:34:47.021 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.021 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.021 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.021 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.021 INFO:teuthology.orchestra.run.smithi133.stdout: "subtree": { 2023-07-14T18:34:47.021 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "subtree", 2023-07-14T18:34:47.021 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.021 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.022 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.022 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "rack", 2023-07-14T18:34:47.022 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.022 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.022 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.022 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-14T18:34:47.022 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-14T18:34:47.022 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.023 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.023 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.023 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.023 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.023 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.023 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mds_autoscaler", 2023-07-14T18:34:47.023 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.023 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.024 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.024 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.024 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.024 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.024 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.024 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.024 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.024 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.025 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.025 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.025 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.025 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.025 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.025 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.025 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.025 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.026 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.026 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.026 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.026 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.026 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.026 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.026 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.026 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.027 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.027 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.027 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.027 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.027 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.027 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.027 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.027 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.028 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.028 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.028 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.028 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.028 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.028 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.028 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.029 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.029 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.029 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.029 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.029 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.029 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.029 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.029 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.030 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.030 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.030 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.030 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.030 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.030 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.030 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.030 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.031 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.031 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.031 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.031 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.031 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.031 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.031 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.031 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.031 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.032 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.032 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.032 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.032 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.032 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.032 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.033 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.033 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.033 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.033 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.033 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mirroring", 2023-07-14T18:34:47.033 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.033 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.034 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.034 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.034 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.034 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.034 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.034 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.034 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.034 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.035 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.035 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.035 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.035 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.035 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.035 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.035 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.035 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.036 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.036 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.036 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.036 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.036 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.036 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.036 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.037 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.037 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.037 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.037 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.037 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.037 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.037 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.037 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.038 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.038 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.038 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.038 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.038 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.038 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.038 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.038 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.039 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.039 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.039 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.039 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.039 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.039 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.039 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.039 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.040 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.040 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.040 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.040 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.040 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.040 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.040 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.040 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.041 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.041 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.041 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.041 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.041 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.041 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.041 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.041 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.042 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.042 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.042 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.042 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.042 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.042 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.042 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.042 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.043 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.043 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.043 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.043 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "nfs", 2023-07-14T18:34:47.043 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.043 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.043 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.043 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.044 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.044 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.044 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.044 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.044 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.044 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.044 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.044 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.045 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.045 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.045 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.045 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.045 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.045 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.045 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.045 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.046 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.046 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.046 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.046 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.046 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.046 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.046 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.046 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.047 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.047 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.047 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.047 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.047 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.047 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.047 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.047 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.048 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.048 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.048 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.048 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.048 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.048 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.048 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.049 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.049 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.049 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.049 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.049 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.049 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.049 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.049 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.050 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.050 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.050 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.050 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.050 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.050 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.050 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.050 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.051 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.051 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.051 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.051 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.051 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.051 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.051 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.051 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.052 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.052 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.052 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.052 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.052 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.052 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.052 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.052 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.053 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.053 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "orchestrator", 2023-07-14T18:34:47.053 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.053 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.053 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.053 INFO:teuthology.orchestra.run.smithi133.stdout: "fail_fs": { 2023-07-14T18:34:47.053 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "fail_fs", 2023-07-14T18:34:47.053 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.054 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.054 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.054 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.054 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.054 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.054 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.054 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-14T18:34:47.054 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.055 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.055 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.055 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.055 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.055 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.055 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.055 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.055 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.056 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.056 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.056 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.056 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.056 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.056 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.056 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.056 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.057 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.057 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.057 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.057 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.057 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.057 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.057 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.057 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.058 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.058 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.058 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.058 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.058 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.058 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.058 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.059 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.059 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.059 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.059 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.059 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.059 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.059 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.059 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.060 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.060 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.060 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.060 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.060 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.060 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.060 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.060 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.061 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.061 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.061 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.061 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.061 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.061 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.061 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.061 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.062 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.062 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.062 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.062 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.062 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.062 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.062 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.062 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.063 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.063 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.063 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.063 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.063 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.063 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.063 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.063 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.064 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.064 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.064 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator": { 2023-07-14T18:34:47.064 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "orchestrator", 2023-07-14T18:34:47.064 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.064 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.064 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.064 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.065 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.065 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.065 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.065 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm", 2023-07-14T18:34:47.065 INFO:teuthology.orchestra.run.smithi133.stdout: "rook", 2023-07-14T18:34:47.065 INFO:teuthology.orchestra.run.smithi133.stdout: "test_orchestrator" 2023-07-14T18:34:47.065 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.065 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Orchestrator backend", 2023-07-14T18:34:47.066 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.066 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.066 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.066 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.066 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.066 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.066 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.066 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "osd_perf_query", 2023-07-14T18:34:47.067 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.067 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.067 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.067 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.067 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.067 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.067 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.067 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.068 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.068 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.068 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.068 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.068 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.068 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.068 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.068 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.069 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.069 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.069 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.069 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.069 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.069 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.069 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.069 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.070 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.070 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.070 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.070 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.070 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.070 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.070 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.070 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.071 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.071 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.071 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.071 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.071 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.071 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.071 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.072 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.072 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.072 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.072 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.072 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.072 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.072 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.072 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.073 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.073 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.073 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.073 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.073 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.073 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.073 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.073 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.074 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.074 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.074 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.074 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.074 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.074 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.074 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.074 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.075 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.075 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.075 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.075 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.075 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.075 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.075 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.075 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.076 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.076 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.076 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.076 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.076 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.076 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "osd_support", 2023-07-14T18:34:47.076 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.076 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.077 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.077 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.077 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.077 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.077 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.077 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.077 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.077 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.078 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.078 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.078 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.078 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.078 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.078 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.078 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.078 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.079 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.079 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.079 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.079 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.079 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.079 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.079 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.079 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.080 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.080 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.080 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.080 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.080 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.080 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.080 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.081 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.081 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.081 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.081 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.081 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.081 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.081 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.081 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.082 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.082 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.082 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.082 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.082 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.082 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.082 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.082 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.083 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.083 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.083 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.083 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.083 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.083 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.083 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.083 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.084 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.084 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.084 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.084 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.084 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.084 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.084 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.085 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.085 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.085 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.085 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.085 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.085 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.085 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.085 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.086 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.086 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.086 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.086 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.086 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pg_autoscaler", 2023-07-14T18:34:47.086 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.086 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.086 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.087 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.087 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.087 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.087 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.087 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.087 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.087 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.087 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.088 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.088 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.088 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.088 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.088 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.088 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.088 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.088 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.089 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.089 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.089 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.089 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.089 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.089 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.089 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.089 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.090 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.090 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.090 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.090 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.090 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.090 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.090 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.090 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.091 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.091 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.091 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.091 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.091 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.091 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.091 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.091 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.092 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.092 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.092 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.092 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.092 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.092 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.092 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.092 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.093 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.093 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.093 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.093 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.093 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.093 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.093 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.093 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.094 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.094 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.094 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.094 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.094 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.094 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.094 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.094 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.095 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.095 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.095 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.095 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.095 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.095 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.095 INFO:teuthology.orchestra.run.smithi133.stdout: "noautoscale": { 2023-07-14T18:34:47.096 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "noautoscale", 2023-07-14T18:34:47.096 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.096 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.096 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.096 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.097 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.097 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.097 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.097 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "global autoscale flag", 2023-07-14T18:34:47.097 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-14T18:34:47.097 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.097 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.098 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.098 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-07-14T18:34:47.098 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-07-14T18:34:47.098 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:47.098 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.098 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.099 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-07-14T18:34:47.099 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.099 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.099 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.099 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.099 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.100 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.100 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.100 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.100 INFO:teuthology.orchestra.run.smithi133.stdout: "threshold": { 2023-07-14T18:34:47.100 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "threshold", 2023-07-14T18:34:47.101 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-07-14T18:34:47.101 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.101 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.101 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3.0", 2023-07-14T18:34:47.101 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1.0", 2023-07-14T18:34:47.101 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.102 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.102 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "scaling threshold", 2023-07-14T18:34:47.102 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-14T18:34:47.102 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.102 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.102 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.102 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.103 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.103 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.103 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "progress", 2023-07-14T18:34:47.103 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.103 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.103 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.103 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_pg_recovery_event": { 2023-07-14T18:34:47.103 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_pg_recovery_event", 2023-07-14T18:34:47.104 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.104 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.104 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.104 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.104 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.104 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.104 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.104 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-14T18:34:47.105 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.105 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.105 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.105 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.105 INFO:teuthology.orchestra.run.smithi133.stdout: "enabled": { 2023-07-14T18:34:47.105 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enabled", 2023-07-14T18:34:47.105 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.105 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.106 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.106 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.106 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.106 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.106 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.106 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.106 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.106 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.107 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.107 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.107 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.107 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.107 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.107 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.107 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.107 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.108 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.108 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.108 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.108 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.108 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.108 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.108 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.109 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.109 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.109 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.109 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.109 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.109 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.109 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.109 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.110 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.110 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.110 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.110 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.110 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.110 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.110 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.110 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.111 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.111 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.111 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.111 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.111 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.111 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.111 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.111 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.112 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.112 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.112 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.112 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.112 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.112 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.112 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.112 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.113 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.113 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.113 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.113 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.113 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.113 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.113 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.113 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.114 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.114 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.114 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.114 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.114 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.114 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.114 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.114 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.115 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.115 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.115 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.115 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.115 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.115 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.115 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.115 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.116 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.116 INFO:teuthology.orchestra.run.smithi133.stdout: "max_completed_events": { 2023-07-14T18:34:47.116 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_completed_events", 2023-07-14T18:34:47.116 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.116 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.116 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.116 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "50", 2023-07-14T18:34:47.116 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.117 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.117 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.117 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "number of past completed events to remember", 2023-07-14T18:34:47.117 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.117 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.117 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.117 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.117 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-07-14T18:34:47.118 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-07-14T18:34:47.118 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:47.118 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.118 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.118 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2023-07-14T18:34:47.118 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.118 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.118 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.119 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long the module is going to sleep", 2023-07-14T18:34:47.119 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.119 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.119 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.119 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.119 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.119 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.119 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.120 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus", 2023-07-14T18:34:47.120 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.120 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.120 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.120 INFO:teuthology.orchestra.run.smithi133.stdout: "cache": { 2023-07-14T18:34:47.120 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cache", 2023-07-14T18:34:47.120 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.121 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.121 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.121 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.121 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.121 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.121 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.121 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.121 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.122 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.122 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.122 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.122 INFO:teuthology.orchestra.run.smithi133.stdout: "exclude_perf_counters": { 2023-07-14T18:34:47.122 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "exclude_perf_counters", 2023-07-14T18:34:47.122 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.122 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.122 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.123 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.123 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.123 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.123 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.123 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-14T18:34:47.123 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-14T18:34:47.123 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.123 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.124 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.124 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.124 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.124 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.124 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.124 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.124 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.124 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.125 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.125 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.125 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.125 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.125 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.125 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.125 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.125 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.126 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.126 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.126 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.126 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.126 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.126 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.126 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.126 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.127 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.127 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.127 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.127 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.127 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.127 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.127 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.127 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.128 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.128 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.128 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.128 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.128 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.128 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.128 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.128 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.129 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.129 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.129 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.129 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.129 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.129 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.129 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.129 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.130 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.130 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.130 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.130 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.130 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.130 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.130 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.130 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.131 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.131 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.131 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.131 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.131 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.131 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.131 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.131 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.132 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.132 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.132 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.132 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.132 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.132 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.132 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.133 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_stats_pools": { 2023-07-14T18:34:47.133 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_stats_pools", 2023-07-14T18:34:47.133 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.133 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.133 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.133 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.133 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.133 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.134 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.134 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.134 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.134 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.134 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.134 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.134 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-14T18:34:47.134 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-14T18:34:47.135 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.135 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.135 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.135 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "300", 2023-07-14T18:34:47.135 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.135 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.135 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.135 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.136 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.136 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.136 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.136 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.136 INFO:teuthology.orchestra.run.smithi133.stdout: "scrape_interval": { 2023-07-14T18:34:47.136 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "scrape_interval", 2023-07-14T18:34:47.136 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-07-14T18:34:47.136 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.137 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.137 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15.0", 2023-07-14T18:34:47.137 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.137 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.137 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.137 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.137 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.137 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.138 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.138 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.138 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2023-07-14T18:34:47.138 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2023-07-14T18:34:47.138 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.138 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.138 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.138 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "::", 2023-07-14T18:34:47.139 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.139 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.139 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.139 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-14T18:34:47.139 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.139 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.139 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.139 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.140 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2023-07-14T18:34:47.140 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2023-07-14T18:34:47.140 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.140 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.140 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.140 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "9283", 2023-07-14T18:34:47.140 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.141 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.141 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.141 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-14T18:34:47.141 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.141 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.141 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.141 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.141 INFO:teuthology.orchestra.run.smithi133.stdout: "stale_cache_strategy": { 2023-07-14T18:34:47.142 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "stale_cache_strategy", 2023-07-14T18:34:47.142 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.142 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.142 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.142 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "log", 2023-07-14T18:34:47.142 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.142 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.142 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.143 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.143 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.143 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.143 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.143 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.143 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_behaviour": { 2023-07-14T18:34:47.143 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_behaviour", 2023-07-14T18:34:47.143 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.144 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.144 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.144 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "default", 2023-07-14T18:34:47.144 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.144 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.144 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.144 INFO:teuthology.orchestra.run.smithi133.stdout: "default", 2023-07-14T18:34:47.144 INFO:teuthology.orchestra.run.smithi133.stdout: "error" 2023-07-14T18:34:47.145 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.145 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.145 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.145 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.145 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.145 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.145 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_error_status_code": { 2023-07-14T18:34:47.145 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_error_status_code", 2023-07-14T18:34:47.146 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.146 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.146 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.146 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "500", 2023-07-14T18:34:47.146 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "400", 2023-07-14T18:34:47.146 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "599", 2023-07-14T18:34:47.146 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.146 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.147 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.147 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.147 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.147 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.147 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.147 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.147 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.147 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_support", 2023-07-14T18:34:47.148 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.148 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.148 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.148 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.148 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.148 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.148 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.149 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.149 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.149 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.149 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.149 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.149 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.149 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.149 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.150 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.150 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.150 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.150 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.150 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.150 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.150 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.150 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.151 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.151 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.151 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.151 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.151 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.151 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.151 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.151 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.152 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.152 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.152 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.152 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.152 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.152 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.152 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.152 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.153 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.153 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.153 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.153 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.153 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.153 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.153 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.153 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.154 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.154 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.154 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.154 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.154 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.154 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.154 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.154 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.155 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.155 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.155 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.155 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.155 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.155 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.155 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.155 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.156 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.156 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.156 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.156 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.156 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.156 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.156 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.157 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.157 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.157 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.157 INFO:teuthology.orchestra.run.smithi133.stdout: "max_concurrent_snap_create": { 2023-07-14T18:34:47.157 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_concurrent_snap_create", 2023-07-14T18:34:47.157 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.157 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.157 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.158 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-07-14T18:34:47.158 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.158 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.158 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.158 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.158 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.158 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.158 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.159 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.159 INFO:teuthology.orchestra.run.smithi133.stdout: "mirror_snapshot_schedule": { 2023-07-14T18:34:47.159 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mirror_snapshot_schedule", 2023-07-14T18:34:47.159 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.159 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.159 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.159 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.159 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.160 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.160 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.160 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.160 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.160 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.160 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.160 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.160 INFO:teuthology.orchestra.run.smithi133.stdout: "trash_purge_schedule": { 2023-07-14T18:34:47.161 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "trash_purge_schedule", 2023-07-14T18:34:47.161 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.161 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.161 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.161 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.161 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.161 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.161 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.162 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.162 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.162 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.162 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.162 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.162 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.162 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.162 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.163 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "restful", 2023-07-14T18:34:47.163 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.163 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.163 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.163 INFO:teuthology.orchestra.run.smithi133.stdout: "enable_auth": { 2023-07-14T18:34:47.163 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enable_auth", 2023-07-14T18:34:47.163 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.163 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.164 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.164 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.164 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.164 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.164 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.164 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.164 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.164 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.165 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.165 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.165 INFO:teuthology.orchestra.run.smithi133.stdout: "key_file": { 2023-07-14T18:34:47.165 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "key_file", 2023-07-14T18:34:47.165 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.165 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.165 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.165 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.166 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.166 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.166 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.166 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.166 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.166 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.166 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.166 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.167 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.167 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.167 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.167 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.167 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.167 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.167 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.167 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.168 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.168 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.168 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.168 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.168 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.168 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.168 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.169 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.169 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.169 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.169 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.169 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.169 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.169 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.169 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.169 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.170 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.170 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.170 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.170 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.170 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.170 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.170 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.170 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.171 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.171 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.171 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.171 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.171 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.171 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.171 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.171 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.172 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.172 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.172 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.172 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.172 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.172 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.172 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.173 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.173 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.173 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.173 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.173 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.173 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.173 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.173 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.174 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.174 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.174 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.174 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.174 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.174 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.174 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.174 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.175 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.175 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.175 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.175 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.175 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.175 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.175 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.175 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2023-07-14T18:34:47.176 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2023-07-14T18:34:47.176 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.176 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.176 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.176 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.176 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.176 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.176 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.177 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.177 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.177 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.177 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.177 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.177 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2023-07-14T18:34:47.177 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2023-07-14T18:34:47.177 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.178 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.178 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.178 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.178 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.178 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.178 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.178 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.178 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.179 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.179 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.179 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.179 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.179 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.179 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.179 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rgw", 2023-07-14T18:34:47.179 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.180 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.180 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.180 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.180 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.180 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.180 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.180 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.180 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.181 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.181 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.181 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.181 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.181 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.181 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.181 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.182 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.182 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.182 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.182 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.182 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.182 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.182 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.182 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.183 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.183 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.183 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.183 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.183 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.183 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.183 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.183 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.184 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.184 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.184 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.184 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.184 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.184 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.184 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.184 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.185 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.185 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.185 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.185 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.185 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.185 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.185 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.185 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.186 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.186 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.186 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.186 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.186 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.186 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.186 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.186 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.187 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.187 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.187 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.187 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.187 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.187 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.187 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.187 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.188 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.188 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.188 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.188 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.188 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.188 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.188 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.188 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.189 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.189 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.189 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.189 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.189 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rook", 2023-07-14T18:34:47.189 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.189 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.189 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.190 INFO:teuthology.orchestra.run.smithi133.stdout: "drive_group_interval": { 2023-07-14T18:34:47.190 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "drive_group_interval", 2023-07-14T18:34:47.190 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-07-14T18:34:47.190 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.190 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.190 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "300.0", 2023-07-14T18:34:47.190 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.190 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.191 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.191 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-14T18:34:47.191 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.191 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.191 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.191 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.191 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.191 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.192 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.192 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.192 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.192 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.192 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.192 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.192 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.192 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.193 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.193 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.193 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.193 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.193 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.193 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.193 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.193 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.194 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.194 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.194 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.194 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.194 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.194 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.194 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.194 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.195 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.195 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.195 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.195 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.195 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.195 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.195 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.195 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.196 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.196 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.196 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.196 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.196 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.196 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.196 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.197 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.197 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.197 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.197 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.197 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.197 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.197 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.197 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.198 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.198 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.198 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.198 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.198 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.198 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.198 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.198 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.198 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.199 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.199 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.199 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.199 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.199 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.199 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.199 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.200 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.200 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.200 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.200 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.200 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.200 INFO:teuthology.orchestra.run.smithi133.stdout: "storage_class": { 2023-07-14T18:34:47.200 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "storage_class", 2023-07-14T18:34:47.200 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.201 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.201 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.201 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "local", 2023-07-14T18:34:47.201 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.201 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.201 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.201 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-14T18:34:47.201 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.202 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.202 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.202 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.202 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.202 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.202 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.202 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "selftest", 2023-07-14T18:34:47.202 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.203 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.203 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.203 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.203 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.203 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.203 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.203 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.203 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.204 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.204 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.204 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.204 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.204 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.204 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.204 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.204 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.205 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.205 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.205 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.205 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.205 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.205 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.205 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.206 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.206 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.206 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.206 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.206 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.206 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.206 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.206 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.207 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.207 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.207 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.207 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.207 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.207 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.207 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.207 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.208 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.208 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.208 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.208 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.208 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.208 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.208 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.208 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.209 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.209 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.209 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.209 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.209 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.209 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.209 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.209 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.210 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.210 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.210 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.210 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.210 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.210 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.210 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.210 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.211 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.211 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.211 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.211 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.211 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.211 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.211 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.211 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.212 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.212 INFO:teuthology.orchestra.run.smithi133.stdout: "roption1": { 2023-07-14T18:34:47.212 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "roption1", 2023-07-14T18:34:47.212 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.212 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.212 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.212 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.212 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.213 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.213 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.213 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.213 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.213 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.213 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.213 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.213 INFO:teuthology.orchestra.run.smithi133.stdout: "roption2": { 2023-07-14T18:34:47.214 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "roption2", 2023-07-14T18:34:47.214 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.214 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.214 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.214 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "xyz", 2023-07-14T18:34:47.214 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.214 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.214 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.215 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.215 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.215 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.215 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.215 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.215 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption1": { 2023-07-14T18:34:47.215 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption1", 2023-07-14T18:34:47.215 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.216 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.216 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.216 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.216 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.216 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.216 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.216 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.217 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.217 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.217 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.217 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.217 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption2": { 2023-07-14T18:34:47.217 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption2", 2023-07-14T18:34:47.217 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.217 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.218 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.218 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.218 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.218 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.218 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.218 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.218 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.218 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.219 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.219 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.219 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption3": { 2023-07-14T18:34:47.219 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption3", 2023-07-14T18:34:47.219 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-07-14T18:34:47.219 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.219 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.219 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.220 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.220 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.220 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.220 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.220 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.220 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.220 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.220 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.221 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption4": { 2023-07-14T18:34:47.221 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption4", 2023-07-14T18:34:47.221 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.221 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.221 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.221 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.221 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.221 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.222 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.222 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.222 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.222 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.222 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.222 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.222 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption5": { 2023-07-14T18:34:47.222 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption5", 2023-07-14T18:34:47.223 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.223 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.223 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.223 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.223 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.223 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.223 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.223 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.224 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.224 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.224 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.224 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.224 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption6": { 2023-07-14T18:34:47.224 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption6", 2023-07-14T18:34:47.224 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.224 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.225 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.225 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.225 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.225 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.225 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.225 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.225 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.225 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.226 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.226 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.226 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption7": { 2023-07-14T18:34:47.226 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption7", 2023-07-14T18:34:47.226 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.226 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.226 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.226 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.227 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2023-07-14T18:34:47.227 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "42", 2023-07-14T18:34:47.227 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.227 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.227 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.227 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.227 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.227 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.228 INFO:teuthology.orchestra.run.smithi133.stdout: "testkey": { 2023-07-14T18:34:47.228 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testkey", 2023-07-14T18:34:47.228 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.228 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.228 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.228 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.228 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.228 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.229 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.229 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.229 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.229 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.229 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.229 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.229 INFO:teuthology.orchestra.run.smithi133.stdout: "testlkey": { 2023-07-14T18:34:47.229 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testlkey", 2023-07-14T18:34:47.230 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.230 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.230 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.230 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.230 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.230 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.230 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.231 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.231 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.231 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.231 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.231 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.231 INFO:teuthology.orchestra.run.smithi133.stdout: "testnewline": { 2023-07-14T18:34:47.231 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testnewline", 2023-07-14T18:34:47.231 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.232 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.232 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.232 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.232 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.232 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.232 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.232 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.232 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.233 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.233 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.233 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.233 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.233 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.233 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.233 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snap_schedule", 2023-07-14T18:34:47.233 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.234 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.234 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.234 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_m_granularity": { 2023-07-14T18:34:47.234 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_m_granularity", 2023-07-14T18:34:47.234 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.234 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.234 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.234 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.235 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.235 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.235 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.235 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow minute scheduled snapshots", 2023-07-14T18:34:47.235 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.235 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.235 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.235 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.236 INFO:teuthology.orchestra.run.smithi133.stdout: "dump_on_update": { 2023-07-14T18:34:47.236 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "dump_on_update", 2023-07-14T18:34:47.236 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.236 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.236 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.236 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.236 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.236 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.237 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.237 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "dump database to debug log on update", 2023-07-14T18:34:47.237 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.237 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.237 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.237 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.237 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.238 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.238 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.238 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.238 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.238 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.238 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.238 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.238 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.238 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.239 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.239 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.239 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.239 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.239 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.239 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.239 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.239 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.240 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.240 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.240 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.240 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.240 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.240 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.240 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.241 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.241 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.241 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.241 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.241 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.241 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.241 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.241 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.241 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.242 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.242 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.242 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.242 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.242 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.242 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.242 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.242 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.243 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.243 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.243 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.243 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.243 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.243 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.243 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.243 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.244 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.244 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.244 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.244 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.244 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.244 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.244 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.245 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.245 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.245 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.245 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.245 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.245 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.245 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.245 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.246 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.246 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.246 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.246 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.246 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.246 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.246 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.246 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.247 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "stats", 2023-07-14T18:34:47.247 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.247 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.247 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.247 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.247 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.247 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.247 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.248 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.248 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.248 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.248 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.248 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.248 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.248 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.248 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.249 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.249 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.249 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.249 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.249 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.249 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.249 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.249 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.250 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.250 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.250 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.250 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.250 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.250 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.250 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.250 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.251 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.251 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.251 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.251 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.251 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.251 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.251 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.251 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.252 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.252 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.252 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.252 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.252 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.252 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.252 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.252 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.253 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.253 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.253 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.253 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.253 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.253 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.253 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.254 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.254 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.254 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.254 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.254 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.254 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.254 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.254 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.255 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.255 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.255 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.255 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.255 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.255 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.255 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.255 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.256 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.256 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.256 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.256 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.256 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.256 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.256 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "status", 2023-07-14T18:34:47.257 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.257 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.257 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.257 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.257 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.257 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.257 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.257 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.258 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.258 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.258 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.258 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.258 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.258 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.258 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.258 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.259 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.259 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.259 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.259 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.259 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.259 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.259 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.259 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.260 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.260 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.260 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.260 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.260 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.260 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.260 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.260 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.261 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.261 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.261 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.261 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.261 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.261 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.261 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.261 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.262 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.262 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.262 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.262 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.262 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.262 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.262 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.262 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.263 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.263 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.263 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.263 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.263 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.263 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.263 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.263 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.264 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.264 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.264 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.264 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.264 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.264 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.264 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.264 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.265 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.265 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.265 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.265 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.265 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.265 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.265 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.265 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.266 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.266 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.266 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.266 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.266 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "telegraf", 2023-07-14T18:34:47.266 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.266 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.266 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.267 INFO:teuthology.orchestra.run.smithi133.stdout: "address": { 2023-07-14T18:34:47.267 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "address", 2023-07-14T18:34:47.267 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.267 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.267 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.267 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-14T18:34:47.267 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.267 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.268 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.268 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.268 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.268 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.268 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.268 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.268 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-07-14T18:34:47.269 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-07-14T18:34:47.269 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:47.269 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.269 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.269 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15", 2023-07-14T18:34:47.269 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.269 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.269 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.270 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.270 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.270 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.270 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.270 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.270 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.270 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.270 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.271 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.271 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.271 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.271 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.271 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.271 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.271 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.271 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.272 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.272 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.272 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.272 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.272 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.272 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.272 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.272 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.273 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.273 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.273 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.273 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.273 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.273 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.273 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.273 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.274 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.274 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.274 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.274 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.274 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.274 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.274 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.274 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.275 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.275 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.275 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.275 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.275 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.275 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.275 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.275 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.276 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.276 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.276 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.276 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.276 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.276 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.276 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.276 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.277 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.277 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.277 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.277 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.277 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.277 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.277 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.277 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.278 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.278 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.278 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.278 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.278 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.278 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.278 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.279 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.279 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.279 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.279 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.279 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.279 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.279 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.279 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "telemetry", 2023-07-14T18:34:47.280 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.280 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.280 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.280 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_basic": { 2023-07-14T18:34:47.280 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_basic", 2023-07-14T18:34:47.280 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.280 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.280 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.281 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.281 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.281 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.281 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.281 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-14T18:34:47.281 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.281 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.281 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.282 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.282 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_crash": { 2023-07-14T18:34:47.282 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_crash", 2023-07-14T18:34:47.282 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.282 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.282 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.282 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.282 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.283 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.283 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.283 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-14T18:34:47.283 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.283 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.283 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.283 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.283 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_device": { 2023-07-14T18:34:47.284 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_device", 2023-07-14T18:34:47.284 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.284 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.284 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.284 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.284 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.284 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.285 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.285 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-14T18:34:47.285 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.285 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.285 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.285 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.285 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_ident": { 2023-07-14T18:34:47.285 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_ident", 2023-07-14T18:34:47.286 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.286 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.286 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.286 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.286 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.286 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.286 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.286 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-14T18:34:47.287 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.287 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.287 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.287 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.287 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_perf": { 2023-07-14T18:34:47.287 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_perf", 2023-07-14T18:34:47.287 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.287 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.288 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.288 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.288 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.288 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.288 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.288 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-14T18:34:47.288 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.288 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.289 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.289 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.289 INFO:teuthology.orchestra.run.smithi133.stdout: "contact": { 2023-07-14T18:34:47.289 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "contact", 2023-07-14T18:34:47.289 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.289 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.289 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.289 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.290 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.290 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.290 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.290 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.290 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.290 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.290 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.290 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.291 INFO:teuthology.orchestra.run.smithi133.stdout: "description": { 2023-07-14T18:34:47.291 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "description", 2023-07-14T18:34:47.291 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.291 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.291 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.291 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.291 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.291 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.291 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.292 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.292 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.292 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.292 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.292 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.292 INFO:teuthology.orchestra.run.smithi133.stdout: "device_url": { 2023-07-14T18:34:47.292 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_url", 2023-07-14T18:34:47.293 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.293 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.293 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.293 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-14T18:34:47.293 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.293 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.293 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.293 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.294 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.294 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.294 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.294 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.294 INFO:teuthology.orchestra.run.smithi133.stdout: "enabled": { 2023-07-14T18:34:47.294 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enabled", 2023-07-14T18:34:47.294 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.294 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.295 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.295 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.295 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.295 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.295 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.295 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.295 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.295 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.296 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.296 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.296 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-07-14T18:34:47.296 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-07-14T18:34:47.296 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.296 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.296 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.296 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "24", 2023-07-14T18:34:47.297 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "8", 2023-07-14T18:34:47.297 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.297 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.297 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.297 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.297 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.297 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.297 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.298 INFO:teuthology.orchestra.run.smithi133.stdout: "last_opt_revision": { 2023-07-14T18:34:47.298 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "last_opt_revision", 2023-07-14T18:34:47.298 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.298 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.298 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.298 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1", 2023-07-14T18:34:47.298 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.298 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.299 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.299 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.299 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.299 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.299 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.299 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.299 INFO:teuthology.orchestra.run.smithi133.stdout: "leaderboard": { 2023-07-14T18:34:47.299 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "leaderboard", 2023-07-14T18:34:47.300 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.300 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.300 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.300 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.300 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.300 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.300 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.300 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.301 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.301 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.301 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.301 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.301 INFO:teuthology.orchestra.run.smithi133.stdout: "leaderboard_description": { 2023-07-14T18:34:47.301 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "leaderboard_description", 2023-07-14T18:34:47.301 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.301 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.302 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.302 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.302 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.302 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.302 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.302 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.302 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.302 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.303 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.303 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.303 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.303 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.303 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.303 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.303 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.303 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.304 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.304 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.304 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.304 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.304 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.304 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.304 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.305 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.305 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.305 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.305 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.305 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.305 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.305 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.305 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.306 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.306 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.306 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.306 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.306 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.306 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.306 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.306 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.307 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.307 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.307 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.307 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.307 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.307 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.307 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.307 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.308 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.308 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.308 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.308 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.308 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.308 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.308 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.308 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.309 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.309 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.309 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.309 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.309 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.309 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.309 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.309 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.310 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.310 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.310 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.310 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.310 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.310 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.310 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.310 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.311 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.311 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.311 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.311 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.311 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.311 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.311 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.311 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.312 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.312 INFO:teuthology.orchestra.run.smithi133.stdout: "organization": { 2023-07-14T18:34:47.312 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "organization", 2023-07-14T18:34:47.312 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.312 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.312 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.312 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.312 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.313 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.313 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.313 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.313 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.313 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.313 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.313 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.313 INFO:teuthology.orchestra.run.smithi133.stdout: "proxy": { 2023-07-14T18:34:47.314 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "proxy", 2023-07-14T18:34:47.314 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.315 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.315 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.316 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.316 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.316 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.316 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.316 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.316 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.316 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.316 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.317 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.317 INFO:teuthology.orchestra.run.smithi133.stdout: "url": { 2023-07-14T18:34:47.317 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "url", 2023-07-14T18:34:47.317 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.317 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.317 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.317 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-14T18:34:47.317 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.318 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.318 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.318 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.318 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.318 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.318 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.318 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.318 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.319 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.319 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.319 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "test_orchestrator", 2023-07-14T18:34:47.319 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.319 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.319 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.319 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.319 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.320 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.320 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.320 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.320 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.320 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.320 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.320 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.321 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.321 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.321 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.321 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.321 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.321 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.321 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.321 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.322 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.322 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.322 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.322 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.322 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.322 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.322 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.322 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.322 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.323 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.323 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.323 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.323 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.324 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.324 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.324 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.324 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.324 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.324 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.324 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.324 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.325 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.325 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.325 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.325 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.325 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.325 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.325 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.326 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.326 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.326 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.326 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.326 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.326 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.326 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.326 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.327 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.327 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.327 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.327 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.327 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.327 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.327 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.327 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.328 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.328 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.328 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.328 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.328 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.328 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.328 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.328 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.329 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.329 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.329 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.329 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.329 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "volumes", 2023-07-14T18:34:47.329 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.329 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.330 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.330 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.330 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.330 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.330 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.330 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.330 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.331 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.331 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.331 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.331 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.331 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.331 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.331 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.331 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.332 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.332 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.332 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.332 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.332 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.332 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.332 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.332 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.333 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.333 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.333 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.333 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.333 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.333 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.333 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.333 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.334 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.334 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.334 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.334 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.334 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.334 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.334 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.335 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.335 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.335 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.335 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.335 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.335 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.335 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.335 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.336 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.336 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.336 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.336 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.336 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.336 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.336 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.336 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.337 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.337 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.337 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.337 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.337 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.337 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.337 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.337 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.338 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.338 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.338 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.338 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.338 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.338 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.338 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.338 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.339 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.339 INFO:teuthology.orchestra.run.smithi133.stdout: "max_concurrent_clones": { 2023-07-14T18:34:47.339 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_concurrent_clones", 2023-07-14T18:34:47.339 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.339 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.339 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.339 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "4", 2023-07-14T18:34:47.339 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.340 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.340 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.340 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-14T18:34:47.340 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.340 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.340 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.340 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.341 INFO:teuthology.orchestra.run.smithi133.stdout: "snapshot_clone_delay": { 2023-07-14T18:34:47.341 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snapshot_clone_delay", 2023-07-14T18:34:47.341 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.341 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.341 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.341 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-07-14T18:34:47.341 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.341 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.342 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.342 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-14T18:34:47.342 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.342 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.342 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.342 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.342 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.342 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.343 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.343 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix", 2023-07-14T18:34:47.343 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.343 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.343 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.343 INFO:teuthology.orchestra.run.smithi133.stdout: "discovery_interval": { 2023-07-14T18:34:47.343 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "discovery_interval", 2023-07-14T18:34:47.343 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-07-14T18:34:47.344 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.344 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.344 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "100", 2023-07-14T18:34:47.344 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.344 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.344 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.344 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.344 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.345 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.345 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.345 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.345 INFO:teuthology.orchestra.run.smithi133.stdout: "identifier": { 2023-07-14T18:34:47.345 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "identifier", 2023-07-14T18:34:47.345 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.345 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.345 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.346 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.346 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.346 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.346 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.346 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.346 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.346 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.346 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.347 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.347 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-07-14T18:34:47.347 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-07-14T18:34:47.347 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:47.347 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.347 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.347 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-07-14T18:34:47.347 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.348 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.348 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.348 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.348 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.348 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.348 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.348 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.348 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.349 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.349 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.349 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.349 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.349 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.349 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.349 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.349 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.350 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.350 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.350 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.350 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.350 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.350 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.350 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.350 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.351 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.351 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.351 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.351 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.351 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.351 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.351 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.351 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.352 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.352 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.352 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.352 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.352 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.352 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.352 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.353 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.353 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.353 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.353 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.353 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.353 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.353 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.353 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.354 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.354 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.354 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.354 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.354 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.354 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.355 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.355 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.355 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.355 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.355 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.355 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.355 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.355 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.356 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.356 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.356 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.356 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.356 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.356 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.356 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.357 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.357 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.357 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.357 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.357 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.357 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.357 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.358 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.358 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.358 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_host": { 2023-07-14T18:34:47.358 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_host", 2023-07-14T18:34:47.358 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.358 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.358 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.358 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.359 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.359 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.359 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.359 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.359 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.359 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.359 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.360 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.360 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_port": { 2023-07-14T18:34:47.360 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_port", 2023-07-14T18:34:47.360 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.360 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.360 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.360 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10051", 2023-07-14T18:34:47.360 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.361 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.361 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.361 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.361 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.361 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.361 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.361 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.361 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_sender": { 2023-07-14T18:34:47.362 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_sender", 2023-07-14T18:34:47.362 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.362 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.362 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.362 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-14T18:34:47.362 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.362 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.362 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.363 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.363 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.363 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.363 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.363 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.363 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.363 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.363 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2023-07-14T18:34:47.364 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.364 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.364 INFO:teuthology.orchestra.run.smithi133.stdout: "modules": [ 2023-07-14T18:34:47.364 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm", 2023-07-14T18:34:47.364 INFO:teuthology.orchestra.run.smithi133.stdout: "dashboard", 2023-07-14T18:34:47.364 INFO:teuthology.orchestra.run.smithi133.stdout: "iostat", 2023-07-14T18:34:47.364 INFO:teuthology.orchestra.run.smithi133.stdout: "nfs", 2023-07-14T18:34:47.364 INFO:teuthology.orchestra.run.smithi133.stdout: "restful" 2023-07-14T18:34:47.365 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.365 INFO:teuthology.orchestra.run.smithi133.stdout: "available_modules": [ 2023-07-14T18:34:47.365 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.365 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "alerts", 2023-07-14T18:34:47.365 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.365 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.365 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.365 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-07-14T18:34:47.366 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-07-14T18:34:47.366 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:47.366 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.366 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.366 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-07-14T18:34:47.366 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.366 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.366 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.367 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How frequently to reexamine health status", 2023-07-14T18:34:47.367 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.367 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.367 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.367 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.367 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.367 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.368 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.368 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.368 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.368 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.368 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.368 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.368 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.368 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.369 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.369 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.369 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.369 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.369 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.369 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.369 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.369 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.370 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.370 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.370 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.370 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.370 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.370 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.370 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.370 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.371 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.371 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.371 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.371 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.371 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.371 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.371 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.371 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.372 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.372 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.372 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.372 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.372 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.372 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.372 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.372 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.373 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.373 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.373 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.373 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.373 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.373 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.373 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.373 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.374 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.374 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.374 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.374 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.374 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.374 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.374 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.374 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.375 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.375 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.375 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.375 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.375 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.375 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.375 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.375 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.376 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.376 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.376 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.376 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.376 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_destination": { 2023-07-14T18:34:47.376 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_destination", 2023-07-14T18:34:47.376 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.377 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.377 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.377 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.377 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.377 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.377 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.377 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Email address to send alerts to", 2023-07-14T18:34:47.377 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.378 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.378 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.378 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.378 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_from_name": { 2023-07-14T18:34:47.378 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_from_name", 2023-07-14T18:34:47.378 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.378 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.378 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.379 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "Ceph", 2023-07-14T18:34:47.379 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.379 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.379 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.379 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Email From: name", 2023-07-14T18:34:47.379 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.379 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.379 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.380 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.380 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_host": { 2023-07-14T18:34:47.380 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_host", 2023-07-14T18:34:47.380 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.380 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.380 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.380 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.380 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.381 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.381 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.381 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP server", 2023-07-14T18:34:47.381 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.381 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.381 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.381 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.381 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_password": { 2023-07-14T18:34:47.382 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_password", 2023-07-14T18:34:47.382 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.382 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.382 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.382 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.382 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.382 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.382 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.383 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Password to authenticate with", 2023-07-14T18:34:47.383 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.383 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.383 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.383 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.383 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_port": { 2023-07-14T18:34:47.383 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_port", 2023-07-14T18:34:47.384 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.384 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.384 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.384 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "465", 2023-07-14T18:34:47.384 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.384 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.384 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.384 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP port", 2023-07-14T18:34:47.385 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.385 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.385 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.385 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.385 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_sender": { 2023-07-14T18:34:47.385 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_sender", 2023-07-14T18:34:47.385 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.385 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.386 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.386 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.386 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.386 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.386 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.386 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP envelope sender", 2023-07-14T18:34:47.386 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.386 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.387 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.387 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.387 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_ssl": { 2023-07-14T18:34:47.387 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_ssl", 2023-07-14T18:34:47.387 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.387 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.387 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.387 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.388 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.388 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.388 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.388 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-14T18:34:47.388 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.388 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.388 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.389 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.389 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_user": { 2023-07-14T18:34:47.389 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_user", 2023-07-14T18:34:47.389 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.389 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.389 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.389 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.389 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.390 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.390 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.390 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "User to authenticate as", 2023-07-14T18:34:47.390 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.390 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.390 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.390 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.390 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.391 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.391 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.391 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "balancer", 2023-07-14T18:34:47.391 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.391 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.391 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.391 INFO:teuthology.orchestra.run.smithi133.stdout: "active": { 2023-07-14T18:34:47.391 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "active", 2023-07-14T18:34:47.392 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.392 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.392 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.392 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.392 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.392 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.392 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.392 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "automatically balance PGs across cluster", 2023-07-14T18:34:47.393 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.393 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.393 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.393 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.393 INFO:teuthology.orchestra.run.smithi133.stdout: "begin_time": { 2023-07-14T18:34:47.393 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "begin_time", 2023-07-14T18:34:47.393 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.394 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.394 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.394 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0000", 2023-07-14T18:34:47.394 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.394 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.394 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.394 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "beginning time of day to automatically balance", 2023-07-14T18:34:47.394 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-14T18:34:47.395 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.395 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.395 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.395 INFO:teuthology.orchestra.run.smithi133.stdout: "begin_weekday": { 2023-07-14T18:34:47.395 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "begin_weekday", 2023-07-14T18:34:47.395 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-07-14T18:34:47.395 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.395 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.396 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-07-14T18:34:47.396 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0", 2023-07-14T18:34:47.396 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "6", 2023-07-14T18:34:47.396 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.396 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-14T18:34:47.396 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-14T18:34:47.396 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.396 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.397 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.397 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_max_iterations": { 2023-07-14T18:34:47.397 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_max_iterations", 2023-07-14T18:34:47.397 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-07-14T18:34:47.397 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.397 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.397 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "25", 2023-07-14T18:34:47.397 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2023-07-14T18:34:47.398 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "250", 2023-07-14T18:34:47.398 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.398 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-14T18:34:47.398 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.398 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.398 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.398 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.398 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_metrics": { 2023-07-14T18:34:47.399 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_metrics", 2023-07-14T18:34:47.399 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.399 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.399 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.399 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "pgs,objects,bytes", 2023-07-14T18:34:47.399 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.399 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.399 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.400 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-14T18:34:47.400 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-14T18:34:47.400 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.400 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.400 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.400 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_step": { 2023-07-14T18:34:47.400 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_step", 2023-07-14T18:34:47.401 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-07-14T18:34:47.401 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.401 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.401 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0.5", 2023-07-14T18:34:47.401 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0.001", 2023-07-14T18:34:47.401 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "0.999", 2023-07-14T18:34:47.401 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.401 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "aggressiveness of optimization", 2023-07-14T18:34:47.402 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-14T18:34:47.402 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.402 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.402 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.402 INFO:teuthology.orchestra.run.smithi133.stdout: "end_time": { 2023-07-14T18:34:47.402 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "end_time", 2023-07-14T18:34:47.402 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.402 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.403 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.403 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "2359", 2023-07-14T18:34:47.403 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.403 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.403 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.403 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "ending time of day to automatically balance", 2023-07-14T18:34:47.403 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-14T18:34:47.403 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.404 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.404 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.404 INFO:teuthology.orchestra.run.smithi133.stdout: "end_weekday": { 2023-07-14T18:34:47.404 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "end_weekday", 2023-07-14T18:34:47.404 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-07-14T18:34:47.404 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.404 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.404 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-07-14T18:34:47.405 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0", 2023-07-14T18:34:47.405 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "6", 2023-07-14T18:34:47.405 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.405 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-14T18:34:47.405 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-14T18:34:47.405 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.405 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.405 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.406 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.406 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.406 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.406 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.406 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.406 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.406 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.406 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.407 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.407 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.407 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.407 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.407 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.407 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.407 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.407 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.408 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.408 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.408 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.408 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.408 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.408 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.408 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.409 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.409 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.409 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.409 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.409 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.409 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.409 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.409 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.410 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.410 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.410 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.410 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.410 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.410 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.410 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.410 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.411 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.411 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.411 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.411 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.411 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.411 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.411 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.412 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.412 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.412 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.412 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.412 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.412 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.412 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.412 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.413 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.413 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.413 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.413 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.413 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.413 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.413 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.413 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.414 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.414 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.414 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.414 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.414 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.414 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.414 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.414 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.415 INFO:teuthology.orchestra.run.smithi133.stdout: "min_score": { 2023-07-14T18:34:47.415 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "min_score", 2023-07-14T18:34:47.415 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-07-14T18:34:47.415 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.415 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.415 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-07-14T18:34:47.415 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.415 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.416 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.416 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-14T18:34:47.416 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.416 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.416 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.416 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.416 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": { 2023-07-14T18:34:47.416 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mode", 2023-07-14T18:34:47.417 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.417 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.417 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.417 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "upmap", 2023-07-14T18:34:47.417 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.417 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.417 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.417 INFO:teuthology.orchestra.run.smithi133.stdout: "crush-compat", 2023-07-14T18:34:47.418 INFO:teuthology.orchestra.run.smithi133.stdout: "none", 2023-07-14T18:34:47.418 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap" 2023-07-14T18:34:47.418 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.418 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Balancer mode", 2023-07-14T18:34:47.418 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.418 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.418 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.418 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.419 INFO:teuthology.orchestra.run.smithi133.stdout: "pool_ids": { 2023-07-14T18:34:47.419 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pool_ids", 2023-07-14T18:34:47.419 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.419 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.419 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.419 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.419 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.419 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.420 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.420 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-14T18:34:47.420 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.420 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.420 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.420 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.420 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-07-14T18:34:47.420 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-07-14T18:34:47.421 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:47.421 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.421 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.421 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-07-14T18:34:47.421 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.421 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.421 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.421 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-14T18:34:47.422 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.422 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.422 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.422 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.422 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap_max_deviation": { 2023-07-14T18:34:47.422 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "upmap_max_deviation", 2023-07-14T18:34:47.422 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.422 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.423 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.423 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2023-07-14T18:34:47.423 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2023-07-14T18:34:47.423 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.423 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.423 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-14T18:34:47.423 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-14T18:34:47.423 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.424 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.424 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.424 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap_max_optimizations": { 2023-07-14T18:34:47.424 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "upmap_max_optimizations", 2023-07-14T18:34:47.424 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-07-14T18:34:47.424 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.424 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.425 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-07-14T18:34:47.425 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.425 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.425 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.425 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-14T18:34:47.425 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.425 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.425 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.426 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.426 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.426 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.426 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.426 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cephadm", 2023-07-14T18:34:47.426 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.426 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.426 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.427 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_down_multiplier": { 2023-07-14T18:34:47.427 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_down_multiplier", 2023-07-14T18:34:47.427 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-07-14T18:34:47.427 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.427 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.427 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3.0", 2023-07-14T18:34:47.427 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.427 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.428 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.428 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-14T18:34:47.428 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.428 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.428 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.428 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.428 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_refresh_rate": { 2023-07-14T18:34:47.429 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_refresh_rate", 2023-07-14T18:34:47.429 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:47.429 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.429 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.429 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "20", 2023-07-14T18:34:47.429 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.429 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.429 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.430 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-14T18:34:47.430 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.430 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.430 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.430 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.430 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_starting_port": { 2023-07-14T18:34:47.430 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_starting_port", 2023-07-14T18:34:47.430 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.431 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.431 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.431 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "4721", 2023-07-14T18:34:47.431 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.431 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.431 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.431 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-14T18:34:47.431 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.432 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.432 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.432 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.432 INFO:teuthology.orchestra.run.smithi133.stdout: "alertmanager_web_password": { 2023-07-14T18:34:47.432 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "alertmanager_web_password", 2023-07-14T18:34:47.432 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.432 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.432 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.433 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-07-14T18:34:47.433 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.433 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.433 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.433 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Alertmanager web password", 2023-07-14T18:34:47.433 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.433 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.433 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.434 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.434 INFO:teuthology.orchestra.run.smithi133.stdout: "alertmanager_web_user": { 2023-07-14T18:34:47.434 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "alertmanager_web_user", 2023-07-14T18:34:47.434 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.434 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.434 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.434 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-07-14T18:34:47.435 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.435 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.435 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.435 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Alertmanager web user", 2023-07-14T18:34:47.435 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.435 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.435 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.435 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.436 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_ptrace": { 2023-07-14T18:34:47.436 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_ptrace", 2023-07-14T18:34:47.436 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.436 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.436 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.436 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.436 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.436 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.437 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.437 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-14T18:34:47.437 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-14T18:34:47.437 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.437 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.437 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.437 INFO:teuthology.orchestra.run.smithi133.stdout: "autotune_interval": { 2023-07-14T18:34:47.437 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "autotune_interval", 2023-07-14T18:34:47.438 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:47.438 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.438 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.438 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-07-14T18:34:47.438 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.438 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.438 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.439 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-14T18:34:47.439 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.439 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.439 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.439 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.439 INFO:teuthology.orchestra.run.smithi133.stdout: "autotune_memory_target_ratio": { 2023-07-14T18:34:47.439 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "autotune_memory_target_ratio", 2023-07-14T18:34:47.440 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-07-14T18:34:47.440 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.440 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.440 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0.7", 2023-07-14T18:34:47.440 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.440 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.441 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.441 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-14T18:34:47.441 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.441 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.441 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.442 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.442 INFO:teuthology.orchestra.run.smithi133.stdout: "cgroups_split": { 2023-07-14T18:34:47.442 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cgroups_split", 2023-07-14T18:34:47.442 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.442 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.442 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.443 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.443 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.443 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.443 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.443 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-14T18:34:47.444 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.444 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.444 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.444 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.444 INFO:teuthology.orchestra.run.smithi133.stdout: "config_checks_enabled": { 2023-07-14T18:34:47.445 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_checks_enabled", 2023-07-14T18:34:47.445 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.445 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.445 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.445 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.445 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.445 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.446 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.446 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-14T18:34:47.446 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.446 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.446 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.446 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.446 INFO:teuthology.orchestra.run.smithi133.stdout: "config_dashboard": { 2023-07-14T18:34:47.446 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_dashboard", 2023-07-14T18:34:47.447 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.447 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.447 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.447 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.447 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.447 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.447 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.447 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-14T18:34:47.448 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.448 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.448 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.448 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.448 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_alertmanager": { 2023-07-14T18:34:47.448 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_alertmanager", 2023-07-14T18:34:47.448 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.448 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.449 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.449 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-14T18:34:47.449 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.449 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.449 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.449 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2023-07-14T18:34:47.449 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.449 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.450 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.450 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.450 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_base": { 2023-07-14T18:34:47.450 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_base", 2023-07-14T18:34:47.450 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.450 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.450 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.450 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-14T18:34:47.451 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.451 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.451 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.451 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Container image name, without the tag", 2023-07-14T18:34:47.451 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.451 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.451 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.451 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.452 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_elasticsearch": { 2023-07-14T18:34:47.452 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_elasticsearch", 2023-07-14T18:34:47.452 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.452 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.452 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.452 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-14T18:34:47.452 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.452 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.453 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.453 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "elasticsearch container image", 2023-07-14T18:34:47.453 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.453 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.453 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.453 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.453 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_grafana": { 2023-07-14T18:34:47.453 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_grafana", 2023-07-14T18:34:47.454 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.454 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.454 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.454 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-14T18:34:47.454 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.454 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.454 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.454 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2023-07-14T18:34:47.455 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.455 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.455 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.455 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.455 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_haproxy": { 2023-07-14T18:34:47.455 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_haproxy", 2023-07-14T18:34:47.455 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.455 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.456 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.456 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-14T18:34:47.456 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.456 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.456 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.456 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "HAproxy container image", 2023-07-14T18:34:47.456 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.456 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.457 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.457 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.457 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_agent": { 2023-07-14T18:34:47.457 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_agent", 2023-07-14T18:34:47.457 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.457 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.457 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.457 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-14T18:34:47.458 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.458 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.458 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.458 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger agent container image", 2023-07-14T18:34:47.458 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.458 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.458 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.458 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.459 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_collector": { 2023-07-14T18:34:47.459 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_collector", 2023-07-14T18:34:47.459 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.459 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.459 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.459 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-14T18:34:47.459 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.459 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.460 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.460 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger collector container image", 2023-07-14T18:34:47.460 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.460 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.460 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.460 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.460 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_query": { 2023-07-14T18:34:47.460 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_query", 2023-07-14T18:34:47.461 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.461 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.461 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.461 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-14T18:34:47.461 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.461 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.461 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.461 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger query container image", 2023-07-14T18:34:47.462 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.462 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.462 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.462 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.462 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_keepalived": { 2023-07-14T18:34:47.462 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_keepalived", 2023-07-14T18:34:47.462 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.462 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.463 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.463 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-14T18:34:47.463 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.463 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.463 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.463 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Keepalived container image", 2023-07-14T18:34:47.463 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.463 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.464 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.464 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.464 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_loki": { 2023-07-14T18:34:47.464 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_loki", 2023-07-14T18:34:47.464 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.464 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.464 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.464 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-14T18:34:47.465 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.465 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.465 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.465 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Loki container image", 2023-07-14T18:34:47.465 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.465 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.465 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.466 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.466 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_node_exporter": { 2023-07-14T18:34:47.466 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_node_exporter", 2023-07-14T18:34:47.466 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.466 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.466 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.466 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-14T18:34:47.466 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.467 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.467 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.467 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2023-07-14T18:34:47.467 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.467 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.467 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.467 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.467 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_prometheus": { 2023-07-14T18:34:47.468 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_prometheus", 2023-07-14T18:34:47.468 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.468 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.468 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.468 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-14T18:34:47.468 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.468 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.468 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.469 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2023-07-14T18:34:47.469 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.469 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.469 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.469 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.469 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_promtail": { 2023-07-14T18:34:47.469 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_promtail", 2023-07-14T18:34:47.469 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.470 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.470 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.470 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-14T18:34:47.470 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.470 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.470 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.470 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Promtail container image", 2023-07-14T18:34:47.470 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.471 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.471 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.471 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.471 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_snmp_gateway": { 2023-07-14T18:34:47.471 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_snmp_gateway", 2023-07-14T18:34:47.471 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.471 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.472 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.472 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-14T18:34:47.472 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.472 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.472 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.472 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SNMP Gateway container image", 2023-07-14T18:34:47.472 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.472 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.473 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.473 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.473 INFO:teuthology.orchestra.run.smithi133.stdout: "container_init": { 2023-07-14T18:34:47.473 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_init", 2023-07-14T18:34:47.473 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.473 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.473 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.473 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.474 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.474 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.474 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.474 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Run podman/docker with `--init`", 2023-07-14T18:34:47.474 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.474 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.474 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.474 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.475 INFO:teuthology.orchestra.run.smithi133.stdout: "daemon_cache_timeout": { 2023-07-14T18:34:47.475 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "daemon_cache_timeout", 2023-07-14T18:34:47.475 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:47.475 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.475 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.475 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-07-14T18:34:47.475 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.475 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.476 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.476 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-14T18:34:47.476 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.476 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.476 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.476 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.476 INFO:teuthology.orchestra.run.smithi133.stdout: "default_cephadm_command_timeout": { 2023-07-14T18:34:47.476 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "default_cephadm_command_timeout", 2023-07-14T18:34:47.477 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:47.477 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.477 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.477 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "900", 2023-07-14T18:34:47.477 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.477 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.477 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.477 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-14T18:34:47.478 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.478 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.478 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.478 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.478 INFO:teuthology.orchestra.run.smithi133.stdout: "default_registry": { 2023-07-14T18:34:47.478 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "default_registry", 2023-07-14T18:34:47.478 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.478 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.479 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.479 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io", 2023-07-14T18:34:47.479 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.479 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.479 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.479 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-14T18:34:47.479 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.479 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.480 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.480 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.480 INFO:teuthology.orchestra.run.smithi133.stdout: "device_cache_timeout": { 2023-07-14T18:34:47.480 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_cache_timeout", 2023-07-14T18:34:47.480 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:47.480 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.480 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.480 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1800", 2023-07-14T18:34:47.481 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.481 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.481 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.481 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache device inventory", 2023-07-14T18:34:47.481 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.481 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.481 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.481 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.482 INFO:teuthology.orchestra.run.smithi133.stdout: "device_enhanced_scan": { 2023-07-14T18:34:47.482 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_enhanced_scan", 2023-07-14T18:34:47.482 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.482 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.482 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.482 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.482 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.482 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.483 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.483 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-14T18:34:47.483 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.483 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.483 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.483 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.483 INFO:teuthology.orchestra.run.smithi133.stdout: "facts_cache_timeout": { 2023-07-14T18:34:47.483 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "facts_cache_timeout", 2023-07-14T18:34:47.484 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:47.484 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.484 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.484 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-07-14T18:34:47.484 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.484 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.484 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.484 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache host facts data", 2023-07-14T18:34:47.485 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.485 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.485 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.485 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.485 INFO:teuthology.orchestra.run.smithi133.stdout: "host_check_interval": { 2023-07-14T18:34:47.485 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "host_check_interval", 2023-07-14T18:34:47.485 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:47.485 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.486 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.486 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-07-14T18:34:47.486 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.486 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.486 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.486 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to perform a host check", 2023-07-14T18:34:47.486 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.486 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.487 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.487 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.487 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.487 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.487 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.487 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.487 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.487 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.488 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.488 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.488 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.488 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.488 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.488 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.488 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.488 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.489 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.489 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.489 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.489 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.489 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.489 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.489 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.489 INFO:teuthology.orchestra.run.smithi133.stdout: "log_refresh_metadata": { 2023-07-14T18:34:47.490 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_refresh_metadata", 2023-07-14T18:34:47.490 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.490 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.490 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.490 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.490 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.490 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.490 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.491 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-14T18:34:47.491 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.491 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.491 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.491 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.491 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.491 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.491 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.492 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.492 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.492 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.492 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.492 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.492 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.492 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-14T18:34:47.493 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.493 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.493 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.493 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.493 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.493 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.493 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.493 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.494 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.494 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.494 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.494 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.494 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.494 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.494 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.494 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.495 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.495 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.495 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.495 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.495 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.495 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.495 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.495 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.496 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.496 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.496 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.496 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.496 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.496 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.497 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.497 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.497 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.497 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.497 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.497 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.497 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.497 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.498 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.498 INFO:teuthology.orchestra.run.smithi133.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-14T18:34:47.498 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-14T18:34:47.498 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.498 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.498 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.498 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.498 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.499 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.499 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.499 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-14T18:34:47.499 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.499 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.499 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.499 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.499 INFO:teuthology.orchestra.run.smithi133.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-14T18:34:47.500 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-14T18:34:47.500 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.500 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.500 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.500 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "*", 2023-07-14T18:34:47.500 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.500 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.500 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.501 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-14T18:34:47.501 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.501 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.501 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.501 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.501 INFO:teuthology.orchestra.run.smithi133.stdout: "max_count_per_host": { 2023-07-14T18:34:47.501 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_count_per_host", 2023-07-14T18:34:47.501 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.502 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.502 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.502 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-07-14T18:34:47.502 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.502 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.502 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.502 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "max number of daemons per service per host", 2023-07-14T18:34:47.502 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.503 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.503 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.503 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.503 INFO:teuthology.orchestra.run.smithi133.stdout: "max_osd_draining_count": { 2023-07-14T18:34:47.503 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_osd_draining_count", 2023-07-14T18:34:47.503 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.503 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.503 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.504 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-07-14T18:34:47.504 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.504 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.504 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.504 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-14T18:34:47.504 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.504 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.505 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.505 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.505 INFO:teuthology.orchestra.run.smithi133.stdout: "migration_current": { 2023-07-14T18:34:47.505 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "migration_current", 2023-07-14T18:34:47.505 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.505 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.505 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.505 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.506 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.506 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.506 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.506 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "internal - do not modify", 2023-07-14T18:34:47.506 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.506 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.506 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.506 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.507 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": { 2023-07-14T18:34:47.507 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mode", 2023-07-14T18:34:47.507 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.507 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.507 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.507 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "root", 2023-07-14T18:34:47.507 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.507 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.508 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.508 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm-package", 2023-07-14T18:34:47.508 INFO:teuthology.orchestra.run.smithi133.stdout: "root" 2023-07-14T18:34:47.508 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.508 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "mode for remote execution of cephadm", 2023-07-14T18:34:47.508 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.508 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.508 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.509 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.509 INFO:teuthology.orchestra.run.smithi133.stdout: "prometheus_alerts_path": { 2023-07-14T18:34:47.509 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus_alerts_path", 2023-07-14T18:34:47.509 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.509 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.509 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.509 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-14T18:34:47.509 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.510 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.510 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.510 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-14T18:34:47.510 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.510 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.510 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.510 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.510 INFO:teuthology.orchestra.run.smithi133.stdout: "prometheus_web_password": { 2023-07-14T18:34:47.511 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus_web_password", 2023-07-14T18:34:47.511 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.511 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.511 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.511 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-07-14T18:34:47.511 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.511 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.511 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.512 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus web password", 2023-07-14T18:34:47.512 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.512 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.512 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.512 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.512 INFO:teuthology.orchestra.run.smithi133.stdout: "prometheus_web_user": { 2023-07-14T18:34:47.512 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus_web_user", 2023-07-14T18:34:47.513 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.513 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.513 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.513 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-07-14T18:34:47.513 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.513 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.513 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.513 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus web user", 2023-07-14T18:34:47.514 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.514 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.514 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.514 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.514 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_insecure": { 2023-07-14T18:34:47.514 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_insecure", 2023-07-14T18:34:47.514 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.514 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.515 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.515 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.515 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.515 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.515 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.515 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-14T18:34:47.515 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.515 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.516 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.516 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.516 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_password": { 2023-07-14T18:34:47.516 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_password", 2023-07-14T18:34:47.516 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.516 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.516 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.516 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.517 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.517 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.517 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.517 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-14T18:34:47.517 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.517 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.517 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.517 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.518 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_url": { 2023-07-14T18:34:47.518 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_url", 2023-07-14T18:34:47.518 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.518 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.518 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.518 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.518 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.519 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.519 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.519 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-14T18:34:47.519 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.519 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.519 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.519 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.519 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_username": { 2023-07-14T18:34:47.520 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_username", 2023-07-14T18:34:47.520 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.520 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.520 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.520 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.520 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.520 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.520 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.521 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-14T18:34:47.521 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.521 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.521 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.521 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.521 INFO:teuthology.orchestra.run.smithi133.stdout: "secure_monitoring_stack": { 2023-07-14T18:34:47.521 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "secure_monitoring_stack", 2023-07-14T18:34:47.521 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.522 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.522 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.522 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.522 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.522 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.522 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.522 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-14T18:34:47.522 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.523 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.523 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.523 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.523 INFO:teuthology.orchestra.run.smithi133.stdout: "service_discovery_port": { 2023-07-14T18:34:47.523 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "service_discovery_port", 2023-07-14T18:34:47.523 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.523 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.524 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.524 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8765", 2023-07-14T18:34:47.524 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.524 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.524 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.524 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "cephadm service discovery port", 2023-07-14T18:34:47.524 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.524 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.525 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.525 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.525 INFO:teuthology.orchestra.run.smithi133.stdout: "ssh_config_file": { 2023-07-14T18:34:47.525 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssh_config_file", 2023-07-14T18:34:47.525 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.525 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.525 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.525 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.526 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.526 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.526 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.526 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-14T18:34:47.526 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.526 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.526 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.526 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.527 INFO:teuthology.orchestra.run.smithi133.stdout: "use_agent": { 2023-07-14T18:34:47.527 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "use_agent", 2023-07-14T18:34:47.527 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.527 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.527 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.527 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.527 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.527 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.528 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.528 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-14T18:34:47.528 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.528 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.528 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.528 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.528 INFO:teuthology.orchestra.run.smithi133.stdout: "use_repo_digest": { 2023-07-14T18:34:47.529 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "use_repo_digest", 2023-07-14T18:34:47.529 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.529 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.529 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.529 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.529 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.529 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.529 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.530 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-14T18:34:47.530 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.530 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.530 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.530 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.530 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_failed_host_check": { 2023-07-14T18:34:47.530 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_failed_host_check", 2023-07-14T18:34:47.530 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.531 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.531 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.531 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.531 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.531 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.531 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.531 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if the host check fails", 2023-07-14T18:34:47.531 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.532 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.532 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.532 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.532 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_stray_daemons": { 2023-07-14T18:34:47.532 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_stray_daemons", 2023-07-14T18:34:47.532 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.532 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.533 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.533 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.533 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.533 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.533 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.533 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-14T18:34:47.533 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.533 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.534 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.534 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.534 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_stray_hosts": { 2023-07-14T18:34:47.534 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_stray_hosts", 2023-07-14T18:34:47.534 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.534 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.534 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.534 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.535 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.535 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.535 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.535 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-14T18:34:47.535 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.535 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.535 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.535 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.536 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.536 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.536 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.536 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crash", 2023-07-14T18:34:47.536 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.536 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.536 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.536 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.537 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.537 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.537 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.537 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.537 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.537 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.537 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.538 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.538 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.538 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.538 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.538 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.538 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.538 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.538 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.539 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.539 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.539 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.539 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.539 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.539 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.539 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.539 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.540 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.540 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.540 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.540 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.540 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.540 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.540 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.540 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.541 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.541 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.541 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.541 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.541 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.541 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.541 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.541 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.542 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.542 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.542 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.542 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.542 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.542 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.542 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.542 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.543 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.543 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.543 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.543 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.543 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.543 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.543 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.543 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.544 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.544 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.544 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.544 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.544 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.544 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.544 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.544 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.545 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.545 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.545 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.545 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.545 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.545 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.545 INFO:teuthology.orchestra.run.smithi133.stdout: "retain_interval": { 2023-07-14T18:34:47.545 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "retain_interval", 2023-07-14T18:34:47.546 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:47.546 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.546 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.546 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "31536000", 2023-07-14T18:34:47.546 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.546 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.546 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.547 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-14T18:34:47.547 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.547 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.547 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.547 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.547 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_recent_interval": { 2023-07-14T18:34:47.547 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_recent_interval", 2023-07-14T18:34:47.547 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:47.548 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.548 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.548 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1209600", 2023-07-14T18:34:47.548 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.548 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.548 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.548 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-14T18:34:47.548 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.549 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.549 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.549 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.549 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.549 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.549 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.549 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "dashboard", 2023-07-14T18:34:47.549 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.550 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.550 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.550 INFO:teuthology.orchestra.run.smithi133.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-14T18:34:47.550 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-14T18:34:47.550 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.550 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.550 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.550 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-07-14T18:34:47.551 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.551 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.551 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.551 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.551 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.551 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.551 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.552 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.552 INFO:teuthology.orchestra.run.smithi133.stdout: "ALERTMANAGER_API_HOST": { 2023-07-14T18:34:47.552 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-14T18:34:47.552 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.552 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.552 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.552 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.552 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.553 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.553 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.553 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.553 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.553 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.553 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.553 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.553 INFO:teuthology.orchestra.run.smithi133.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-14T18:34:47.554 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-14T18:34:47.554 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.554 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.554 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.554 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.554 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.554 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.554 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.555 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.555 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.555 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.555 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.555 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.555 INFO:teuthology.orchestra.run.smithi133.stdout: "AUDIT_API_ENABLED": { 2023-07-14T18:34:47.555 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "AUDIT_API_ENABLED", 2023-07-14T18:34:47.555 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.556 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.556 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.556 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.556 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.556 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.556 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.556 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.556 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.557 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.557 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.557 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.557 INFO:teuthology.orchestra.run.smithi133.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-14T18:34:47.557 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-14T18:34:47.557 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.557 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.557 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.558 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.558 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.558 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.558 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.558 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.558 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.558 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.558 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.559 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.559 INFO:teuthology.orchestra.run.smithi133.stdout: "ENABLE_BROWSABLE_API": { 2023-07-14T18:34:47.559 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-14T18:34:47.559 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.559 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.559 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.559 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.559 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.560 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.560 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.560 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.560 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.560 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.560 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.560 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.561 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-14T18:34:47.561 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-14T18:34:47.561 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.561 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.561 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.561 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.561 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.561 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.562 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.562 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.562 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.562 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.562 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.562 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.562 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-14T18:34:47.562 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-14T18:34:47.563 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.563 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.563 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.563 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.563 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.563 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.563 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.563 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.564 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.564 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.564 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.564 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.564 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-14T18:34:47.564 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-14T18:34:47.564 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.564 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.565 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.565 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.565 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.565 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.565 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.565 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.565 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.565 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.566 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.566 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.566 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-14T18:34:47.566 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-14T18:34:47.566 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.566 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.566 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.566 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.567 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.567 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.567 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.567 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.567 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.567 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.567 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.567 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.568 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-14T18:34:47.568 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-14T18:34:47.568 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.568 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.568 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.568 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.568 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.568 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.569 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.569 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.569 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.569 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.569 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.569 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.569 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-14T18:34:47.569 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-14T18:34:47.570 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.570 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.570 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.570 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.570 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.570 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.570 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.570 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.571 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.571 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.571 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.571 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.571 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-14T18:34:47.571 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-14T18:34:47.571 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.572 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.572 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.572 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.572 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.572 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.572 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.572 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.572 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.573 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.573 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.573 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.573 INFO:teuthology.orchestra.run.smithi133.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-14T18:34:47.573 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-14T18:34:47.573 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.573 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.573 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.574 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.574 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.574 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.574 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.574 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.574 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.574 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.575 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.575 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.575 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_PASSWORD": { 2023-07-14T18:34:47.575 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-14T18:34:47.575 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.575 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.575 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.575 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-07-14T18:34:47.576 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.576 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.576 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.576 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.576 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.576 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.576 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.577 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.577 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-14T18:34:47.577 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-14T18:34:47.577 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.577 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.577 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.577 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.578 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.578 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.578 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.578 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.578 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.578 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.578 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.578 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.579 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_URL": { 2023-07-14T18:34:47.579 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_URL", 2023-07-14T18:34:47.579 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.579 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.579 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.579 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.579 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.580 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.580 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.580 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.580 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.580 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.580 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.580 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.581 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_USERNAME": { 2023-07-14T18:34:47.581 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-14T18:34:47.581 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.581 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.581 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.581 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-07-14T18:34:47.581 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.581 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.582 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.582 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.582 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.582 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.582 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.582 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.582 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-14T18:34:47.582 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-14T18:34:47.583 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.583 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.583 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.583 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.583 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.583 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.583 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.584 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.584 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.584 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.584 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.584 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.584 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-14T18:34:47.584 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-14T18:34:47.584 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.585 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.585 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.585 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.585 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.585 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.585 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.585 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.585 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.586 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.586 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.586 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.586 INFO:teuthology.orchestra.run.smithi133.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-14T18:34:47.586 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-14T18:34:47.586 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.586 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.586 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.587 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.587 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.587 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.587 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.587 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.587 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.587 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.587 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.588 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.588 INFO:teuthology.orchestra.run.smithi133.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-14T18:34:47.588 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-14T18:34:47.588 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.588 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.588 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.588 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.589 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.589 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.589 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.589 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.589 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.589 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.589 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.589 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.590 INFO:teuthology.orchestra.run.smithi133.stdout: "PROMETHEUS_API_HOST": { 2023-07-14T18:34:47.590 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-14T18:34:47.590 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.590 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.590 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.590 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.590 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.590 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.591 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.591 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.591 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.591 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.591 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.591 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.591 INFO:teuthology.orchestra.run.smithi133.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-14T18:34:47.591 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-14T18:34:47.592 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.592 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.592 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.592 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.592 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.592 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.592 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.592 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.593 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.593 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.593 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.593 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.593 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-14T18:34:47.593 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-14T18:34:47.593 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.593 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.594 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.594 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.594 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.594 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.594 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.594 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.594 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.594 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.595 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.595 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.595 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-14T18:34:47.595 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-14T18:34:47.595 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.595 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.595 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.595 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.596 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.596 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.596 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.596 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.596 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.596 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.596 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.596 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.597 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-14T18:34:47.597 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-14T18:34:47.597 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.597 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.597 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.597 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.597 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.597 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.598 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.598 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.598 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.598 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.598 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.598 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.598 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-14T18:34:47.598 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-14T18:34:47.599 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.599 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.599 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.599 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.599 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.599 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.599 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.599 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.600 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.600 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.600 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.600 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.600 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-14T18:34:47.600 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-14T18:34:47.600 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.600 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.601 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.601 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.601 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.601 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.601 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.601 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.601 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.601 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.602 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.602 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.602 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-14T18:34:47.602 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-14T18:34:47.602 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.602 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.602 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.602 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.603 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.603 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.603 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.603 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.603 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.603 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.603 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.604 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.604 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-14T18:34:47.604 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-14T18:34:47.604 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.604 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.604 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.604 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.604 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.605 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.605 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.605 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.605 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.605 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.605 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.605 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.605 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_ENABLED": { 2023-07-14T18:34:47.606 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-14T18:34:47.606 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.606 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.606 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.606 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.606 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.606 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.606 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.607 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.607 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.607 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.607 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.607 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.607 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-14T18:34:47.607 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-14T18:34:47.607 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.608 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.608 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.608 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-14T18:34:47.608 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.608 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.608 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.608 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.609 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.609 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.609 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.609 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.609 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-14T18:34:47.609 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-14T18:34:47.609 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.609 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.610 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.610 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-07-14T18:34:47.610 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.610 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.610 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.610 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.610 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.610 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.610 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.611 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.611 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-14T18:34:47.611 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-14T18:34:47.611 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.611 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.611 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.611 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8", 2023-07-14T18:34:47.612 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.612 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.612 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.612 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.612 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.612 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.612 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.612 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.613 INFO:teuthology.orchestra.run.smithi133.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-14T18:34:47.613 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-14T18:34:47.613 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.613 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.613 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.613 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "45", 2023-07-14T18:34:47.613 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.613 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.614 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.614 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.614 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.614 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.614 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.614 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.614 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_ACCESS_KEY": { 2023-07-14T18:34:47.614 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-14T18:34:47.615 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.615 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.615 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.615 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.615 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.615 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.615 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.615 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.616 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.616 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.616 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.616 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.616 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-14T18:34:47.616 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-14T18:34:47.616 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.616 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.617 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.617 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-07-14T18:34:47.617 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.617 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.617 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.617 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.617 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.617 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.618 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.618 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.618 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_SECRET_KEY": { 2023-07-14T18:34:47.618 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-14T18:34:47.618 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.618 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.618 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.618 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.619 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.619 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.619 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.619 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.619 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.619 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.619 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.619 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.620 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_SSL_VERIFY": { 2023-07-14T18:34:47.620 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-14T18:34:47.620 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.620 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.620 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.620 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.620 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.620 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.621 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.621 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.621 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.621 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.621 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.624 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.624 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-14T18:34:47.625 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-14T18:34:47.625 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.625 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.625 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.625 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-07-14T18:34:47.625 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.625 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.625 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.626 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.626 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.626 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.626 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.626 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.626 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-14T18:34:47.626 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-14T18:34:47.626 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.627 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.627 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.627 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-07-14T18:34:47.627 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.627 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.627 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.627 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.627 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.628 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.628 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.628 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.628 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-14T18:34:47.628 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-14T18:34:47.628 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.629 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.629 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.629 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2023-07-14T18:34:47.629 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.629 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.629 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.630 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.630 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.630 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.630 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.630 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.630 INFO:teuthology.orchestra.run.smithi133.stdout: "cross_origin_url": { 2023-07-14T18:34:47.631 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cross_origin_url", 2023-07-14T18:34:47.631 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.631 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.631 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.632 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.632 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.632 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.632 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.632 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.633 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.633 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.633 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.633 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.633 INFO:teuthology.orchestra.run.smithi133.stdout: "crt_file": { 2023-07-14T18:34:47.634 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crt_file", 2023-07-14T18:34:47.634 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.634 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.634 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.634 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.635 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.635 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.635 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.635 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.635 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.635 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.636 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.636 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.636 INFO:teuthology.orchestra.run.smithi133.stdout: "debug": { 2023-07-14T18:34:47.636 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "debug", 2023-07-14T18:34:47.636 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.636 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.637 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.637 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.637 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.637 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.637 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.637 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable/disable debug options", 2023-07-14T18:34:47.638 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.638 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.638 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.638 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.638 INFO:teuthology.orchestra.run.smithi133.stdout: "jwt_token_ttl": { 2023-07-14T18:34:47.639 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "jwt_token_ttl", 2023-07-14T18:34:47.639 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.639 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.639 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.639 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "28800", 2023-07-14T18:34:47.640 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.640 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.640 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.640 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.641 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.641 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.641 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.642 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.642 INFO:teuthology.orchestra.run.smithi133.stdout: "key_file": { 2023-07-14T18:34:47.642 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "key_file", 2023-07-14T18:34:47.642 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.643 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.643 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.643 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.643 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.644 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.644 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.644 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.644 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.645 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.645 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.645 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.645 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.645 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.645 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.646 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.646 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.646 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.646 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.646 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.646 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.646 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.647 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.647 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.647 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.647 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.647 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.647 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.648 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.648 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.648 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.648 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.648 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.648 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.648 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.649 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.649 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.649 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.649 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.649 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.649 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.650 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.650 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.650 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.650 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.650 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.651 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.651 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.651 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.651 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.652 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.652 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.652 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.652 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.653 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.653 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.653 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.653 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.653 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.654 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.654 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.654 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.654 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.655 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.655 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.655 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.655 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.656 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.656 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.656 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.656 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.656 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.657 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.657 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.657 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.657 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.657 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.657 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.658 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.658 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.658 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.658 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.658 INFO:teuthology.orchestra.run.smithi133.stdout: "motd": { 2023-07-14T18:34:47.659 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "motd", 2023-07-14T18:34:47.659 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.659 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.659 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.659 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.660 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.660 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.660 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.660 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "The message of the day", 2023-07-14T18:34:47.660 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.661 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.661 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.661 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.661 INFO:teuthology.orchestra.run.smithi133.stdout: "redirect_resolve_ip_addr": { 2023-07-14T18:34:47.661 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "redirect_resolve_ip_addr", 2023-07-14T18:34:47.662 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.662 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.662 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.662 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.662 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.662 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.663 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.663 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.663 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.663 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.663 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.664 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.664 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2023-07-14T18:34:47.664 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2023-07-14T18:34:47.664 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.664 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.665 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.665 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "::", 2023-07-14T18:34:47.665 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.665 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.665 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.666 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.666 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.666 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.666 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.666 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.666 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2023-07-14T18:34:47.667 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2023-07-14T18:34:47.667 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.667 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.667 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.667 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8080", 2023-07-14T18:34:47.668 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.668 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.668 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.668 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.668 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.669 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.669 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.669 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.669 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl": { 2023-07-14T18:34:47.669 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl", 2023-07-14T18:34:47.670 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.670 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.670 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.670 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.670 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.670 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.671 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.671 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.671 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.671 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.671 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.671 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.671 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl_server_port": { 2023-07-14T18:34:47.672 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl_server_port", 2023-07-14T18:34:47.672 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.672 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.672 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.672 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8443", 2023-07-14T18:34:47.672 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.672 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.672 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.673 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.673 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.673 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.673 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.673 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.673 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_behaviour": { 2023-07-14T18:34:47.673 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_behaviour", 2023-07-14T18:34:47.673 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.674 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.674 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.674 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "redirect", 2023-07-14T18:34:47.674 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.674 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.674 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.674 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.674 INFO:teuthology.orchestra.run.smithi133.stdout: "redirect" 2023-07-14T18:34:47.675 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.675 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.675 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.675 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.675 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.675 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.675 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_error_status_code": { 2023-07-14T18:34:47.675 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_error_status_code", 2023-07-14T18:34:47.676 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.676 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.676 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.676 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "500", 2023-07-14T18:34:47.676 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "400", 2023-07-14T18:34:47.676 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "599", 2023-07-14T18:34:47.676 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.677 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.677 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.677 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.677 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.677 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.677 INFO:teuthology.orchestra.run.smithi133.stdout: "url_prefix": { 2023-07-14T18:34:47.677 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "url_prefix", 2023-07-14T18:34:47.677 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.678 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.678 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.678 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.678 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.678 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.678 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.678 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.678 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.679 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.679 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.679 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.679 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.679 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.679 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.679 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "devicehealth", 2023-07-14T18:34:47.679 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.679 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.680 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.680 INFO:teuthology.orchestra.run.smithi133.stdout: "enable_monitoring": { 2023-07-14T18:34:47.680 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enable_monitoring", 2023-07-14T18:34:47.680 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.680 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.680 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.680 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.681 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.681 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.681 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.681 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "monitor device health metrics", 2023-07-14T18:34:47.681 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.681 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.681 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.681 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.682 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.682 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.682 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.682 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.682 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.682 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.682 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.682 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.683 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.683 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.683 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.683 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.683 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.683 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.683 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.683 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.684 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.684 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.684 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.684 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.684 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.684 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.684 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.685 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.685 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.685 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.685 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.685 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.685 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.685 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.685 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.686 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.686 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.686 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.686 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.686 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.686 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.686 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.686 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.687 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.687 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.687 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.687 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.687 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.687 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.687 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.687 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.688 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.688 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.688 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.688 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.688 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.688 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.688 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.688 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.689 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.689 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.689 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.689 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.689 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.689 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.689 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.689 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.690 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.690 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.690 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.690 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.690 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.690 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.690 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.691 INFO:teuthology.orchestra.run.smithi133.stdout: "mark_out_threshold": { 2023-07-14T18:34:47.691 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mark_out_threshold", 2023-07-14T18:34:47.691 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:47.691 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.691 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.691 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "2419200", 2023-07-14T18:34:47.691 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.691 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.692 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.692 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-14T18:34:47.692 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.692 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.692 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.692 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.692 INFO:teuthology.orchestra.run.smithi133.stdout: "pool_name": { 2023-07-14T18:34:47.692 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pool_name", 2023-07-14T18:34:47.693 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.693 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.693 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.693 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "device_health_metrics", 2023-07-14T18:34:47.693 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.693 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.693 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.694 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-14T18:34:47.694 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.694 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.694 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.694 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.695 INFO:teuthology.orchestra.run.smithi133.stdout: "retention_period": { 2023-07-14T18:34:47.695 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "retention_period", 2023-07-14T18:34:47.695 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:47.695 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.695 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.695 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15552000", 2023-07-14T18:34:47.695 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.695 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.696 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.696 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long to retain device health metrics", 2023-07-14T18:34:47.696 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.696 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.696 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.696 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.696 INFO:teuthology.orchestra.run.smithi133.stdout: "scrape_frequency": { 2023-07-14T18:34:47.696 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "scrape_frequency", 2023-07-14T18:34:47.697 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:47.697 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.697 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.697 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "86400", 2023-07-14T18:34:47.697 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.697 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.697 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.697 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-14T18:34:47.698 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.698 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.698 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.698 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.698 INFO:teuthology.orchestra.run.smithi133.stdout: "self_heal": { 2023-07-14T18:34:47.698 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "self_heal", 2023-07-14T18:34:47.698 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.698 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.699 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.699 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.699 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.699 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.699 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.699 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-14T18:34:47.699 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.699 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.700 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.700 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.700 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-07-14T18:34:47.700 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-07-14T18:34:47.700 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:47.700 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.700 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.700 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-07-14T18:34:47.701 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.701 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.701 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.701 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to wake up and check device health", 2023-07-14T18:34:47.701 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.701 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.701 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.701 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.702 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_threshold": { 2023-07-14T18:34:47.702 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_threshold", 2023-07-14T18:34:47.702 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:47.702 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.702 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.702 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "7257600", 2023-07-14T18:34:47.702 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.702 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.703 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.703 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-14T18:34:47.703 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.703 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.703 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.703 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.703 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.703 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.704 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.704 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "diskprediction_local", 2023-07-14T18:34:47.704 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.704 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.704 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.704 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.704 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.704 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.705 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.705 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.705 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.705 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.705 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.705 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.705 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.705 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.706 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.706 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.706 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.706 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.706 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.706 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.706 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.706 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.707 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.707 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.707 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.707 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.707 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.707 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.707 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.707 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.708 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.708 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.708 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.708 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.708 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.708 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.708 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.708 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.709 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.709 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.709 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.709 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.709 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.709 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.709 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.710 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.710 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.710 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.710 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.710 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.710 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.710 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.710 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.711 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.711 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.711 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.711 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.711 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.711 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.711 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.711 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.712 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.712 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.712 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.712 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.712 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.712 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.712 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.712 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.713 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.713 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.713 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.713 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.713 INFO:teuthology.orchestra.run.smithi133.stdout: "predict_interval": { 2023-07-14T18:34:47.713 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "predict_interval", 2023-07-14T18:34:47.713 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.713 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.714 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.714 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "86400", 2023-07-14T18:34:47.714 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.714 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.714 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.714 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.714 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.714 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.715 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.715 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.715 INFO:teuthology.orchestra.run.smithi133.stdout: "predictor_model": { 2023-07-14T18:34:47.715 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "predictor_model", 2023-07-14T18:34:47.715 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.715 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.715 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.715 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "prophetstor", 2023-07-14T18:34:47.716 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.716 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.716 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.716 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.716 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.716 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.716 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.717 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.717 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-07-14T18:34:47.717 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-07-14T18:34:47.717 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.717 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.717 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.717 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-07-14T18:34:47.717 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.718 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.718 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.718 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.718 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.718 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.718 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.718 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.718 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.719 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.719 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.719 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "influx", 2023-07-14T18:34:47.719 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": false, 2023-07-14T18:34:47.719 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "influxdb python module not found", 2023-07-14T18:34:47.719 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.719 INFO:teuthology.orchestra.run.smithi133.stdout: "batch_size": { 2023-07-14T18:34:47.719 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "batch_size", 2023-07-14T18:34:47.720 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.720 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.720 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.720 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5000", 2023-07-14T18:34:47.720 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.720 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.720 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.720 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-14T18:34:47.721 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.721 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.721 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.721 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.721 INFO:teuthology.orchestra.run.smithi133.stdout: "database": { 2023-07-14T18:34:47.721 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "database", 2023-07-14T18:34:47.721 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.721 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.722 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.722 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "ceph", 2023-07-14T18:34:47.722 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.722 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.722 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.722 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-14T18:34:47.722 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.722 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.723 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.723 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.723 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": { 2023-07-14T18:34:47.723 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "hostname", 2023-07-14T18:34:47.723 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.723 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.723 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.723 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.724 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.724 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.724 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.724 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB server hostname", 2023-07-14T18:34:47.724 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.724 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.724 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.725 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.725 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-07-14T18:34:47.725 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-07-14T18:34:47.725 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:47.725 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.725 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.725 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "30", 2023-07-14T18:34:47.725 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "5", 2023-07-14T18:34:47.726 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.726 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.726 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-14T18:34:47.726 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.726 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.726 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.726 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.726 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.727 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.727 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.727 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.727 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.727 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.727 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.727 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.727 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.728 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.728 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.728 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.728 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.728 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.728 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.728 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.728 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.729 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.729 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.729 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.729 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.729 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.729 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.729 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.729 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.730 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.730 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.730 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.730 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.730 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.730 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.730 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.730 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.731 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.731 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.731 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.731 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.731 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.731 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.731 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.731 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.732 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.732 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.732 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.732 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.732 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.732 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.732 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.732 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.733 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.733 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.733 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.733 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.733 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.733 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.733 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.733 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.734 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.734 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.734 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.734 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.734 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.734 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.734 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.734 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.735 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.735 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.735 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.735 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.735 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.735 INFO:teuthology.orchestra.run.smithi133.stdout: "password": { 2023-07-14T18:34:47.735 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "password", 2023-07-14T18:34:47.735 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.736 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.736 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.736 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.736 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.736 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.736 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.736 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "password of InfluxDB server user", 2023-07-14T18:34:47.736 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.737 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.737 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.737 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.737 INFO:teuthology.orchestra.run.smithi133.stdout: "port": { 2023-07-14T18:34:47.737 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "port", 2023-07-14T18:34:47.737 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.737 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.737 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.738 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8086", 2023-07-14T18:34:47.738 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.738 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.738 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.738 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB server port", 2023-07-14T18:34:47.739 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.739 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.739 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.739 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.739 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl": { 2023-07-14T18:34:47.739 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl", 2023-07-14T18:34:47.739 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.739 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.740 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.740 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "false", 2023-07-14T18:34:47.740 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.740 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.740 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.740 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-14T18:34:47.740 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.741 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.741 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.741 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.741 INFO:teuthology.orchestra.run.smithi133.stdout: "threads": { 2023-07-14T18:34:47.741 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "threads", 2023-07-14T18:34:47.741 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.741 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.741 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.742 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2023-07-14T18:34:47.742 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2023-07-14T18:34:47.742 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "32", 2023-07-14T18:34:47.742 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.742 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-14T18:34:47.742 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.742 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.742 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.743 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.743 INFO:teuthology.orchestra.run.smithi133.stdout: "username": { 2023-07-14T18:34:47.743 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "username", 2023-07-14T18:34:47.743 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.743 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.743 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.743 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.743 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.744 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.744 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.744 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "username of InfluxDB server user", 2023-07-14T18:34:47.744 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.744 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.744 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.744 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.744 INFO:teuthology.orchestra.run.smithi133.stdout: "verify_ssl": { 2023-07-14T18:34:47.745 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "verify_ssl", 2023-07-14T18:34:47.745 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.745 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.745 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.745 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "true", 2023-07-14T18:34:47.745 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.745 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.745 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.746 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-14T18:34:47.746 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.746 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.746 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.746 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.746 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.746 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.746 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.747 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "insights", 2023-07-14T18:34:47.747 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.747 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.747 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.747 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.747 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.747 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.747 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.748 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.748 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.748 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.748 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.748 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.748 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.748 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.748 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.749 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.749 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.749 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.749 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.749 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.749 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.749 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.749 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.750 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.750 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.750 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.750 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.750 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.750 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.750 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.750 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.751 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.751 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.751 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.751 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.751 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.751 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.751 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.751 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.752 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.752 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.752 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.752 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.752 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.752 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.752 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.752 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.753 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.753 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.753 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.753 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.753 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.753 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.753 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.753 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.754 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.754 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.754 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.754 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.754 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.754 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.754 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.754 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.755 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.755 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.755 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.755 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.755 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.755 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.755 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.755 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.756 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.756 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.756 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.756 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.756 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.756 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "iostat", 2023-07-14T18:34:47.756 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.756 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.757 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.757 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.757 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.757 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.757 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.757 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.757 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.757 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.758 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.758 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.758 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.758 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.758 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.758 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.758 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.758 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.759 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.759 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.759 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.759 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.759 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.759 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.759 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.759 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.760 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.760 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.760 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.760 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.760 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.760 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.760 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.760 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.761 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.761 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.761 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.761 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.761 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.761 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.761 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.761 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.762 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.762 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.762 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.762 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.762 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.762 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.762 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.762 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.763 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.763 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.763 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.763 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.763 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.763 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.763 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.764 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.764 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.764 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.764 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.764 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.764 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.764 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.764 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.765 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.765 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.765 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.765 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.765 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.765 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.765 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.765 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.766 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.766 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.766 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.766 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "k8sevents", 2023-07-14T18:34:47.766 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.766 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.766 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.766 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_event_retention_days": { 2023-07-14T18:34:47.767 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ceph_event_retention_days", 2023-07-14T18:34:47.767 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.767 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.767 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.767 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "7", 2023-07-14T18:34:47.767 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.767 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.767 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.768 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-14T18:34:47.768 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.768 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.768 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.768 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.768 INFO:teuthology.orchestra.run.smithi133.stdout: "config_check_secs": { 2023-07-14T18:34:47.768 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_check_secs", 2023-07-14T18:34:47.769 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.769 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.769 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.769 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-07-14T18:34:47.769 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "10", 2023-07-14T18:34:47.769 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.769 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.769 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-14T18:34:47.770 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.770 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.770 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.770 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.770 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.770 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.770 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.770 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.771 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.771 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.771 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.771 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.771 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.771 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.771 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.771 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.772 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.772 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.772 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.772 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.772 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.772 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.772 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.772 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.773 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.773 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.773 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.773 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.773 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.773 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.773 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.773 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.774 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.774 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.774 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.774 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.774 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.774 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.774 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.774 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.775 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.775 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.775 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.775 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.775 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.775 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.775 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.775 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.776 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.776 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.776 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.776 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.776 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.776 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.776 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.776 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.777 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.777 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.777 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.777 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.777 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.777 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.777 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.777 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.778 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.778 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.778 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.778 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.778 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.778 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.778 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.778 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.779 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.779 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.779 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.779 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.779 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.779 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "localpool", 2023-07-14T18:34:47.779 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.779 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.780 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.780 INFO:teuthology.orchestra.run.smithi133.stdout: "failure_domain": { 2023-07-14T18:34:47.780 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "failure_domain", 2023-07-14T18:34:47.780 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.780 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.780 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.780 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "host", 2023-07-14T18:34:47.781 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.781 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.781 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.781 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "failure domain for any created local pool", 2023-07-14T18:34:47.781 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-14T18:34:47.781 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.781 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.781 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.782 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.782 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.782 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.782 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.782 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.782 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.782 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.782 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.783 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.783 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.783 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.783 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.783 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.783 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.783 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.783 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.784 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.784 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.784 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.784 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.784 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.784 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.784 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.784 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.785 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.785 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.785 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.785 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.785 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.785 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.785 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.785 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.786 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.786 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.786 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.786 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.786 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.786 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.786 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.786 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.787 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.787 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.787 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.787 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.787 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.787 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.787 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.787 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.788 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.788 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.788 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.788 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.788 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.788 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.788 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.788 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.789 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.789 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.789 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.789 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.789 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.789 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.789 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.789 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.790 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.790 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.790 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.790 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.790 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.790 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.790 INFO:teuthology.orchestra.run.smithi133.stdout: "min_size": { 2023-07-14T18:34:47.790 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "min_size", 2023-07-14T18:34:47.791 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.791 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.791 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.791 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.791 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.791 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.791 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.791 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default min_size for any created local pool", 2023-07-14T18:34:47.792 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-14T18:34:47.792 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.792 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.792 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.792 INFO:teuthology.orchestra.run.smithi133.stdout: "num_rep": { 2023-07-14T18:34:47.792 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "num_rep", 2023-07-14T18:34:47.792 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.792 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.793 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.793 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3", 2023-07-14T18:34:47.793 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.793 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.793 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.793 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default replica count for any created local pool", 2023-07-14T18:34:47.793 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.793 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.794 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.794 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.794 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_num": { 2023-07-14T18:34:47.794 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pg_num", 2023-07-14T18:34:47.795 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.795 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.795 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.795 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "128", 2023-07-14T18:34:47.795 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.795 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.795 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.795 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default pg_num for any created local pool", 2023-07-14T18:34:47.796 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.796 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.796 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.796 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.796 INFO:teuthology.orchestra.run.smithi133.stdout: "prefix": { 2023-07-14T18:34:47.796 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prefix", 2023-07-14T18:34:47.796 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.797 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.797 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.797 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.797 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.797 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.797 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.798 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "name prefix for any created local pool", 2023-07-14T18:34:47.798 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.798 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.798 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.798 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.798 INFO:teuthology.orchestra.run.smithi133.stdout: "subtree": { 2023-07-14T18:34:47.798 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "subtree", 2023-07-14T18:34:47.798 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.799 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.799 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.799 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "rack", 2023-07-14T18:34:47.799 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.799 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.799 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.799 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-14T18:34:47.799 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-14T18:34:47.800 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.800 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.800 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.800 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.800 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.800 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.800 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mds_autoscaler", 2023-07-14T18:34:47.800 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.801 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.801 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.801 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.801 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.801 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.801 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.801 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.801 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.802 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.802 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.802 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.802 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.802 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.802 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.802 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.802 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.803 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.803 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.803 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.803 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.803 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.803 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.803 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.803 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.804 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.804 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.804 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.804 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.804 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.804 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.804 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.805 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.805 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.805 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.805 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.805 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.805 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.805 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.805 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.806 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.806 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.806 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.806 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.806 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.806 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.806 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.806 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.807 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.807 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.807 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.807 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.807 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.807 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.807 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.807 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.808 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.808 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.808 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.808 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.808 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.808 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.808 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.809 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.809 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.809 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.809 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.809 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.809 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.809 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.809 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.809 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.810 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.810 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.810 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.810 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.810 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mirroring", 2023-07-14T18:34:47.811 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.811 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.811 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.811 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.811 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.811 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.811 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.812 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.812 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.812 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.812 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.812 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.812 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.813 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.813 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.813 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.813 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.813 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.813 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.814 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.814 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.814 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.814 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.814 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.814 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.815 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.815 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.815 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.815 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.815 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.815 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.816 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.816 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.816 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.816 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.816 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.816 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.817 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.817 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.817 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.817 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.817 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.817 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.818 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.818 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.818 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.818 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.818 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.818 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.819 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.819 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.819 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.819 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.819 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.819 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.820 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.820 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.820 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.820 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.820 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.820 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.820 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.821 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.821 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.821 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.821 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.821 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.821 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.821 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.821 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.822 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.822 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.822 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.822 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.822 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.822 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.822 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "nfs", 2023-07-14T18:34:47.822 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.823 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.823 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.823 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.823 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.823 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.823 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.823 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.823 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.824 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.824 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.824 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.824 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.824 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.824 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.824 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.824 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.825 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.825 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.825 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.825 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.825 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.825 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.825 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.825 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.826 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.826 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.826 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.826 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.826 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.826 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.826 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.826 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.827 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.827 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.827 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.827 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.827 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.827 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.827 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.827 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.828 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.828 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.828 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.828 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.828 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.828 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.828 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.828 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.829 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.829 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.829 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.829 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.829 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.829 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.829 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.829 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.830 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.830 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.830 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.830 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.830 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.830 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.830 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.830 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.831 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.831 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.831 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.831 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.831 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.831 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.831 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.831 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.832 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.832 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.832 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.832 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "orchestrator", 2023-07-14T18:34:47.832 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.832 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.832 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.832 INFO:teuthology.orchestra.run.smithi133.stdout: "fail_fs": { 2023-07-14T18:34:47.833 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "fail_fs", 2023-07-14T18:34:47.833 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.833 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.833 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.833 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.833 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.833 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.833 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.834 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-14T18:34:47.834 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.834 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.834 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.834 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.834 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.834 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.834 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.835 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.835 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.835 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.835 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.835 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.835 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.835 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.835 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.836 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.836 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.836 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.836 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.836 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.836 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.836 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.837 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.837 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.837 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.837 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.837 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.837 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.837 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.837 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.838 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.838 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.838 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.838 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.838 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.838 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.838 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.838 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.838 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.839 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.839 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.839 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.839 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.839 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.839 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.839 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.839 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.840 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.840 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.840 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.840 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.840 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.840 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.840 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.840 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.841 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.841 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.841 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.841 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.841 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.841 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.841 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.841 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.842 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.842 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.842 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.842 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.842 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.842 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.842 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.842 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.843 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.843 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.843 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.843 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator": { 2023-07-14T18:34:47.843 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "orchestrator", 2023-07-14T18:34:47.843 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.843 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.843 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.844 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.844 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.844 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.844 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.844 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm", 2023-07-14T18:34:47.844 INFO:teuthology.orchestra.run.smithi133.stdout: "rook", 2023-07-14T18:34:47.844 INFO:teuthology.orchestra.run.smithi133.stdout: "test_orchestrator" 2023-07-14T18:34:47.844 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.845 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Orchestrator backend", 2023-07-14T18:34:47.845 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.845 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.845 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.845 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.845 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.846 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.846 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.846 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "osd_perf_query", 2023-07-14T18:34:47.846 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.846 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.846 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.846 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.846 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.847 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.847 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.847 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.847 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.847 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.847 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.847 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.847 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.848 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.848 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.848 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.848 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.848 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.848 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.848 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.848 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.849 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.849 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.849 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.849 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.849 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.849 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.849 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.849 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.850 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.850 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.850 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.850 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.850 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.850 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.850 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.851 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.851 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.851 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.851 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.851 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.851 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.851 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.851 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.852 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.852 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.852 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.852 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.852 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.852 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.852 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.852 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.853 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.853 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.853 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.853 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.853 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.853 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.853 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.853 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.854 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.854 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.854 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.854 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.854 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.854 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.854 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.854 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.855 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.855 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.855 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.855 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.855 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.855 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.855 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.855 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.856 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "osd_support", 2023-07-14T18:34:47.856 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.856 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.856 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.856 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.856 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.856 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.856 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.857 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.857 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.857 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.857 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.857 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.857 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.857 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.857 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.858 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.858 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.858 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.858 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.858 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.858 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.858 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.858 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.859 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.859 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.859 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.859 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.859 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.859 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.859 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.859 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.860 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.860 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.860 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.860 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.860 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.860 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.860 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.860 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.861 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.861 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.861 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.861 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.861 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.861 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.861 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.861 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.862 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.862 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.862 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.862 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.862 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.862 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.862 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.862 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.863 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.863 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.863 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.863 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.863 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.863 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.863 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.863 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.864 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.864 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.864 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.864 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.864 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.864 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.864 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.864 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.865 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.865 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.865 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.865 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.865 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.865 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pg_autoscaler", 2023-07-14T18:34:47.865 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.865 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.866 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.866 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.866 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.866 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.866 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.866 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.866 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.866 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.867 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.867 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.867 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.867 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.867 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.867 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.867 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.867 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.868 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.868 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.868 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.868 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.868 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.868 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.868 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.868 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.869 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.869 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.869 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.869 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.869 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.869 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.869 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.869 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.870 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.870 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.870 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.870 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.870 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.870 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.870 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.870 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.871 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.871 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.871 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.871 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.871 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.871 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.871 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.871 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.872 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.872 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.872 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.872 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.872 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.872 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.872 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.873 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.873 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.873 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.873 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.873 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.873 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.873 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.873 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.873 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.874 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.874 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.874 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.874 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.874 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.874 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.874 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.875 INFO:teuthology.orchestra.run.smithi133.stdout: "noautoscale": { 2023-07-14T18:34:47.875 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "noautoscale", 2023-07-14T18:34:47.875 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.875 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.875 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.875 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.875 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.875 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.876 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.876 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "global autoscale flag", 2023-07-14T18:34:47.876 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-14T18:34:47.876 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.876 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.876 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.876 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-07-14T18:34:47.876 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-07-14T18:34:47.877 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:47.877 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.877 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.877 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-07-14T18:34:47.877 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.877 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.877 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.877 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.878 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.878 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.878 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.878 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.878 INFO:teuthology.orchestra.run.smithi133.stdout: "threshold": { 2023-07-14T18:34:47.878 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "threshold", 2023-07-14T18:34:47.878 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-07-14T18:34:47.878 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.879 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.879 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3.0", 2023-07-14T18:34:47.879 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1.0", 2023-07-14T18:34:47.879 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.879 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.879 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "scaling threshold", 2023-07-14T18:34:47.879 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-14T18:34:47.879 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.880 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.880 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.880 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.880 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.880 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.880 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "progress", 2023-07-14T18:34:47.880 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.880 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.881 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.881 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_pg_recovery_event": { 2023-07-14T18:34:47.881 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_pg_recovery_event", 2023-07-14T18:34:47.881 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.881 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.881 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.881 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.881 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.882 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.882 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.882 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-14T18:34:47.882 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.882 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.882 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.882 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.882 INFO:teuthology.orchestra.run.smithi133.stdout: "enabled": { 2023-07-14T18:34:47.883 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enabled", 2023-07-14T18:34:47.883 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.883 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.883 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.883 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.883 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.883 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.883 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.884 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.884 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.884 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.884 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.884 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.884 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.884 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.885 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.885 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.885 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.885 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.885 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.885 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.885 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.885 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.886 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.886 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.886 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.886 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.886 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.886 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.886 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.886 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.886 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.887 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.887 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.887 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.887 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.887 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.887 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.887 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.887 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.888 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.888 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.888 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.888 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.888 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.888 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.888 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.889 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.889 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.889 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.889 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.889 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.889 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.889 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.889 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.890 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.890 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.890 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.890 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.890 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.890 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.890 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.890 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.890 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.891 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.891 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.891 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.891 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.891 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.891 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.891 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.892 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.892 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.892 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.892 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.892 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.892 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.892 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.892 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.893 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.893 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.893 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.893 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.893 INFO:teuthology.orchestra.run.smithi133.stdout: "max_completed_events": { 2023-07-14T18:34:47.893 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_completed_events", 2023-07-14T18:34:47.893 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.893 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.894 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.894 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "50", 2023-07-14T18:34:47.894 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.894 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.894 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.894 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "number of past completed events to remember", 2023-07-14T18:34:47.894 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.894 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.895 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.895 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.895 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-07-14T18:34:47.895 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-07-14T18:34:47.895 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:47.895 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.895 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.895 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2023-07-14T18:34:47.896 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.896 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.896 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.896 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long the module is going to sleep", 2023-07-14T18:34:47.896 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.896 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.896 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.896 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.897 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.897 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.897 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.897 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus", 2023-07-14T18:34:47.897 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.897 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.897 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.897 INFO:teuthology.orchestra.run.smithi133.stdout: "cache": { 2023-07-14T18:34:47.898 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cache", 2023-07-14T18:34:47.898 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.898 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.898 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.898 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.898 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.898 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.898 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.899 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.899 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.899 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.899 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.899 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.899 INFO:teuthology.orchestra.run.smithi133.stdout: "exclude_perf_counters": { 2023-07-14T18:34:47.899 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "exclude_perf_counters", 2023-07-14T18:34:47.899 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.900 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.900 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.900 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.900 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.900 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.900 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.900 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-14T18:34:47.901 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-14T18:34:47.901 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.901 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.901 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.901 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.902 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.902 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.902 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.902 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.902 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.902 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.902 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.902 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.903 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.903 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.903 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.903 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.903 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.903 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.904 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.904 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.904 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.904 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.905 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.905 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.905 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.905 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.905 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.905 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.906 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.906 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.906 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.906 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.906 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.906 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.906 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.906 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.907 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.907 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.907 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.907 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.907 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.907 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.907 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.907 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.908 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.908 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.908 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.908 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.908 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.908 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.908 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.908 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.909 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.909 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.909 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.909 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.909 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.909 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.909 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.909 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.910 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.910 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.910 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.910 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.910 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.910 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.910 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.910 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.911 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.911 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.911 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.911 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.911 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.911 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_stats_pools": { 2023-07-14T18:34:47.911 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_stats_pools", 2023-07-14T18:34:47.911 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.912 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.912 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.912 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.912 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.912 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.912 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.912 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.913 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.913 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.913 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.913 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.913 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-14T18:34:47.913 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-14T18:34:47.913 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.913 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.914 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.914 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "300", 2023-07-14T18:34:47.914 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.914 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.914 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.914 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.914 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.914 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.915 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.915 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.915 INFO:teuthology.orchestra.run.smithi133.stdout: "scrape_interval": { 2023-07-14T18:34:47.915 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "scrape_interval", 2023-07-14T18:34:47.915 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-07-14T18:34:47.915 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.915 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.915 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15.0", 2023-07-14T18:34:47.916 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.916 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.916 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.916 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.916 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.916 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.916 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.916 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.917 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2023-07-14T18:34:47.917 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2023-07-14T18:34:47.917 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.917 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.917 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.917 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "::", 2023-07-14T18:34:47.917 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.917 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.918 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.918 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-14T18:34:47.918 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.918 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.918 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.918 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.918 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2023-07-14T18:34:47.918 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2023-07-14T18:34:47.919 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.919 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.919 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.919 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "9283", 2023-07-14T18:34:47.919 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.919 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.919 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.919 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-14T18:34:47.920 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.920 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.920 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.920 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.920 INFO:teuthology.orchestra.run.smithi133.stdout: "stale_cache_strategy": { 2023-07-14T18:34:47.920 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "stale_cache_strategy", 2023-07-14T18:34:47.920 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.920 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.921 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.921 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "log", 2023-07-14T18:34:47.921 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.921 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.921 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.921 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.921 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.921 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.922 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.922 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.922 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_behaviour": { 2023-07-14T18:34:47.922 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_behaviour", 2023-07-14T18:34:47.922 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.922 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.922 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.922 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "default", 2023-07-14T18:34:47.923 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.923 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.923 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.923 INFO:teuthology.orchestra.run.smithi133.stdout: "default", 2023-07-14T18:34:47.923 INFO:teuthology.orchestra.run.smithi133.stdout: "error" 2023-07-14T18:34:47.923 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.923 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.923 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.924 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.924 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.924 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.924 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_error_status_code": { 2023-07-14T18:34:47.924 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_error_status_code", 2023-07-14T18:34:47.924 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.924 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.924 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.925 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "500", 2023-07-14T18:34:47.925 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "400", 2023-07-14T18:34:47.925 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "599", 2023-07-14T18:34:47.925 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.925 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.925 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.925 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.925 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.926 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.926 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.926 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.926 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.926 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_support", 2023-07-14T18:34:47.926 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.926 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.926 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.927 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.927 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.927 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.927 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.927 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.927 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.927 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.927 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.928 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.928 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.928 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.928 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.928 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.928 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.928 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.928 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.929 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.929 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.929 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.929 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.929 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.929 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.929 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.930 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.930 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.930 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.930 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.930 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.930 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.930 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.930 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.931 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.931 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.931 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.931 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.931 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.931 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.931 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.931 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.931 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.932 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.932 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.932 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.932 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.932 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.932 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.932 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.933 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.933 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.933 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.933 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.933 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.933 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.933 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.933 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.934 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.934 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.934 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.934 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.934 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.934 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.934 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.934 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.935 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.935 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.935 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.935 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.935 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.935 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.935 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.935 INFO:teuthology.orchestra.run.smithi133.stdout: "max_concurrent_snap_create": { 2023-07-14T18:34:47.936 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_concurrent_snap_create", 2023-07-14T18:34:47.936 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.936 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.936 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.936 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-07-14T18:34:47.936 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.936 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.936 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.937 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.937 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.937 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.937 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.937 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.937 INFO:teuthology.orchestra.run.smithi133.stdout: "mirror_snapshot_schedule": { 2023-07-14T18:34:47.937 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mirror_snapshot_schedule", 2023-07-14T18:34:47.937 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.938 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.938 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.938 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.938 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.938 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.938 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.938 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.938 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.939 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.939 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.939 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.939 INFO:teuthology.orchestra.run.smithi133.stdout: "trash_purge_schedule": { 2023-07-14T18:34:47.939 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "trash_purge_schedule", 2023-07-14T18:34:47.939 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.939 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.939 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.940 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.940 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.940 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.940 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.940 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.940 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.940 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.940 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.941 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.941 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.941 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.941 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.941 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "restful", 2023-07-14T18:34:47.941 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.941 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.941 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.942 INFO:teuthology.orchestra.run.smithi133.stdout: "enable_auth": { 2023-07-14T18:34:47.942 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enable_auth", 2023-07-14T18:34:47.942 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.942 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.942 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.942 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:47.942 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.942 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.943 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.943 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.943 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.943 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.943 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.943 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.943 INFO:teuthology.orchestra.run.smithi133.stdout: "key_file": { 2023-07-14T18:34:47.943 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "key_file", 2023-07-14T18:34:47.944 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.944 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.944 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.944 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.944 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.944 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.944 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.944 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.945 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.945 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.945 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.945 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.945 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.945 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.945 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.945 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.946 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.946 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.946 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.946 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.946 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.946 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.946 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.946 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.947 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.947 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.947 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.947 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.947 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.947 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.947 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.947 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.948 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.948 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.948 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.948 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.948 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.948 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.948 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.948 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.949 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.949 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.949 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.949 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.949 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.949 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.949 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.949 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.950 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.950 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.950 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.950 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.950 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.950 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.950 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.950 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.951 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.951 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.951 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.951 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.951 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.951 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.951 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.951 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.952 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.952 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.952 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.952 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.952 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.952 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.952 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.952 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.953 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.953 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.953 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.953 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.953 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.953 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.953 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.954 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.954 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.954 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.954 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2023-07-14T18:34:47.954 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2023-07-14T18:34:47.954 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.954 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.954 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.954 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.955 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.955 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.955 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.955 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.955 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.955 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.955 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.955 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.956 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2023-07-14T18:34:47.956 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2023-07-14T18:34:47.956 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.956 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.956 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.956 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.956 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.957 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.957 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.957 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.957 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.957 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.957 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.957 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.957 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.958 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.958 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.958 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rgw", 2023-07-14T18:34:47.958 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.958 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.958 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.958 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.958 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.959 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.959 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.959 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.959 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.959 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.959 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.959 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.959 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.960 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.960 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.960 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.960 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.960 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.960 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.960 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.960 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.961 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.961 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.961 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.961 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.961 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.961 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.961 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.961 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.962 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.962 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.962 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.962 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.962 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.962 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.962 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.962 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.963 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.963 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.963 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.963 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.963 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.963 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.963 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.963 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.964 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.964 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.964 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.964 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.964 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.964 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.964 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.964 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.965 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.965 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.965 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.965 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.965 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.965 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.965 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.965 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.966 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.966 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.966 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.966 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.966 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.966 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.966 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.966 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.967 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.967 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.967 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.967 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.967 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.967 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.967 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.967 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rook", 2023-07-14T18:34:47.968 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.968 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.968 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.968 INFO:teuthology.orchestra.run.smithi133.stdout: "drive_group_interval": { 2023-07-14T18:34:47.968 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "drive_group_interval", 2023-07-14T18:34:47.968 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-07-14T18:34:47.968 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.968 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.969 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "300.0", 2023-07-14T18:34:47.969 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.969 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.969 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.969 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-14T18:34:47.969 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.969 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.969 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.970 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.970 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.970 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.970 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.970 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.970 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.970 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.970 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.971 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.971 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.971 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.971 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.971 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.971 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.971 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.971 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.972 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.972 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.972 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.972 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.972 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.972 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.972 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.972 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.973 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.973 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.973 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.973 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.973 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.973 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.973 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.973 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.974 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.974 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.974 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.974 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.974 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.974 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.974 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.974 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.975 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.975 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.975 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.975 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.975 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.975 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.975 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.975 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.976 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.976 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.976 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.976 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.976 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.976 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.976 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.976 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.977 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.977 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.977 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.977 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.977 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.977 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.977 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.977 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.978 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.978 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.978 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.978 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.978 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.978 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.978 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.978 INFO:teuthology.orchestra.run.smithi133.stdout: "storage_class": { 2023-07-14T18:34:47.979 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "storage_class", 2023-07-14T18:34:47.979 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.979 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.979 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.979 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "local", 2023-07-14T18:34:47.979 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.979 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.979 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.980 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-14T18:34:47.980 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.980 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.980 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.980 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.980 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:47.980 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.981 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:47.981 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "selftest", 2023-07-14T18:34:47.981 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:47.981 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:47.981 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:47.981 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:47.981 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:47.981 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.982 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.982 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.982 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.982 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.982 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.982 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.982 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.982 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.983 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.983 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.983 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.983 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.983 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.983 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.983 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.983 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.984 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.984 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.984 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:47.984 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:47.984 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.984 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.984 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.984 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.985 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.985 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.985 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.985 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.985 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.985 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.985 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.986 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.986 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:47.986 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:47.986 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.986 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.986 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.986 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:47.986 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.987 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.987 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:47.987 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:47.987 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:47.987 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:47.987 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:47.987 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:47.987 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:47.988 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:47.988 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.988 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.988 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.988 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.988 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.988 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:47.988 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:47.989 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:47.989 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.989 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:47.989 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:47.989 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.989 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.989 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.989 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.990 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.990 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.990 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.990 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.990 INFO:teuthology.orchestra.run.smithi133.stdout: "roption1": { 2023-07-14T18:34:47.990 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "roption1", 2023-07-14T18:34:47.990 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.990 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.991 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.991 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.991 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.991 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.991 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.991 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.991 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.991 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.991 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.992 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.992 INFO:teuthology.orchestra.run.smithi133.stdout: "roption2": { 2023-07-14T18:34:47.992 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "roption2", 2023-07-14T18:34:47.992 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.992 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.992 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.992 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "xyz", 2023-07-14T18:34:47.993 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.993 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.993 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.993 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.993 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.993 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.993 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.993 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.994 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption1": { 2023-07-14T18:34:47.994 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption1", 2023-07-14T18:34:47.994 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.994 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.994 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.994 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.994 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.994 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.995 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.995 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.995 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.995 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.995 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.995 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.995 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption2": { 2023-07-14T18:34:47.995 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption2", 2023-07-14T18:34:47.995 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:47.996 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.996 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.996 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.996 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.996 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.996 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.996 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.997 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.997 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.997 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.997 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.997 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption3": { 2023-07-14T18:34:47.997 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption3", 2023-07-14T18:34:47.997 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-07-14T18:34:47.997 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.997 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.998 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.998 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:47.998 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:47.998 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:47.998 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:47.998 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:47.998 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:47.998 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:47.999 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:47.999 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption4": { 2023-07-14T18:34:47.999 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption4", 2023-07-14T18:34:47.999 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:47.999 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:47.999 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:47.999 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:47.999 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.000 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.000 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.000 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.000 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.000 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.000 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.000 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.000 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption5": { 2023-07-14T18:34:48.001 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption5", 2023-07-14T18:34:48.001 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:48.001 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.001 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.001 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:48.001 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.001 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.001 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.002 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.002 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.002 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.002 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.002 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.002 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption6": { 2023-07-14T18:34:48.002 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption6", 2023-07-14T18:34:48.002 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:48.003 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.003 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.003 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:48.003 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.003 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.003 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.003 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.003 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.004 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.004 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.004 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.004 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption7": { 2023-07-14T18:34:48.004 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption7", 2023-07-14T18:34:48.004 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:48.004 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.004 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.005 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:48.005 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2023-07-14T18:34:48.005 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "42", 2023-07-14T18:34:48.005 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.005 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.005 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.005 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.005 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.006 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.006 INFO:teuthology.orchestra.run.smithi133.stdout: "testkey": { 2023-07-14T18:34:48.006 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testkey", 2023-07-14T18:34:48.006 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.006 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.006 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.006 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:48.006 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.007 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.007 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.007 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.007 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.007 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.007 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.007 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.008 INFO:teuthology.orchestra.run.smithi133.stdout: "testlkey": { 2023-07-14T18:34:48.008 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testlkey", 2023-07-14T18:34:48.008 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.008 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.008 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.008 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:48.008 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.008 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.009 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.009 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.009 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.009 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.009 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.009 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.009 INFO:teuthology.orchestra.run.smithi133.stdout: "testnewline": { 2023-07-14T18:34:48.009 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testnewline", 2023-07-14T18:34:48.010 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.010 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.010 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.010 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:48.010 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.010 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.010 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.010 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.011 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.011 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.011 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.011 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:48.011 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:48.011 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.011 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:48.011 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snap_schedule", 2023-07-14T18:34:48.012 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:48.012 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:48.012 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:48.012 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_m_granularity": { 2023-07-14T18:34:48.012 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_m_granularity", 2023-07-14T18:34:48.012 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:48.012 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.012 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.013 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:48.013 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.013 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.013 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.013 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow minute scheduled snapshots", 2023-07-14T18:34:48.013 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.013 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.013 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.014 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.014 INFO:teuthology.orchestra.run.smithi133.stdout: "dump_on_update": { 2023-07-14T18:34:48.014 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "dump_on_update", 2023-07-14T18:34:48.014 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:48.014 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.014 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.014 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:48.014 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.015 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.015 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.015 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "dump database to debug log on update", 2023-07-14T18:34:48.015 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.015 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.015 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.015 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.015 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:48.016 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:48.016 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.016 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.016 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.016 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:48.016 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.016 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.016 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:48.017 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:48.017 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:48.017 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:48.017 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:48.017 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:48.017 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:48.017 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:48.017 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.018 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.018 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.018 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.018 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.018 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:48.018 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:48.018 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:48.018 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.019 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.019 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:48.019 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.019 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.019 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.019 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.019 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.019 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.020 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.020 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.020 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:48.020 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:48.020 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.020 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.020 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.020 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:48.021 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.021 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.021 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:48.021 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:48.021 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:48.021 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:48.021 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:48.021 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:48.022 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:48.022 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:48.022 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.022 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.022 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.022 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.022 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.022 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:48.023 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:48.023 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:48.023 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.023 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.023 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:48.023 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.023 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.023 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.024 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.024 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.024 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.024 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.024 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:48.024 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:48.024 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.025 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:48.025 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "stats", 2023-07-14T18:34:48.025 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:48.025 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:48.025 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:48.025 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:48.025 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:48.025 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.026 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.026 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.026 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:48.026 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.026 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.026 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:48.026 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:48.026 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:48.027 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:48.027 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:48.027 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:48.027 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:48.027 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:48.027 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.027 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.027 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.028 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.028 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.028 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:48.028 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:48.028 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:48.028 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.028 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.028 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:48.029 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.029 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.029 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.029 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.029 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.029 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.029 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.029 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.030 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:48.030 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:48.030 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.030 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.030 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.030 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:48.030 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.030 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.031 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:48.031 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:48.031 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:48.031 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:48.031 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:48.031 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:48.031 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:48.031 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:48.032 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.032 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.032 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.032 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.032 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.032 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:48.032 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:48.032 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:48.033 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.033 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.033 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:48.033 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.033 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.033 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.033 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.033 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.034 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.034 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.034 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:48.034 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:48.034 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.034 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:48.034 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "status", 2023-07-14T18:34:48.034 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:48.035 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:48.035 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:48.035 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:48.035 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:48.035 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.035 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.035 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.035 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:48.036 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.036 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.036 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:48.036 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:48.036 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:48.036 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:48.036 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:48.036 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:48.037 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:48.037 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:48.037 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.037 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.037 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.037 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.037 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.038 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:48.038 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:48.038 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:48.038 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.038 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.038 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:48.038 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.038 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.039 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.039 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.039 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.039 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.039 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.039 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.039 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:48.039 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:48.040 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.040 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.040 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.040 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:48.040 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.040 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.040 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:48.040 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:48.041 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:48.041 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:48.041 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:48.041 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:48.041 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:48.041 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:48.041 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.041 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.042 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.042 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.042 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.042 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:48.042 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:48.042 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:48.042 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.042 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.043 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:48.043 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.043 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.043 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.043 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.043 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.043 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.043 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.044 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:48.044 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:48.044 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.044 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:48.044 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "telegraf", 2023-07-14T18:34:48.044 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:48.044 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:48.044 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:48.045 INFO:teuthology.orchestra.run.smithi133.stdout: "address": { 2023-07-14T18:34:48.045 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "address", 2023-07-14T18:34:48.045 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.045 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.045 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.045 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-14T18:34:48.045 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.045 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.046 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.046 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.046 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.046 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.046 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.046 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.046 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-07-14T18:34:48.046 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-07-14T18:34:48.047 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:48.047 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.047 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.047 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15", 2023-07-14T18:34:48.047 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.047 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.047 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.047 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.048 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.048 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.048 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.048 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.048 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:48.048 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:48.048 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.049 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.049 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.049 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:48.049 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.049 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.049 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:48.049 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:48.049 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:48.050 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:48.050 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:48.050 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:48.050 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:48.050 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:48.050 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.050 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.050 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.051 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.051 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.051 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:48.051 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:48.051 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:48.051 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.051 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.051 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:48.052 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.052 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.052 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.052 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.052 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.052 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.052 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.052 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.053 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:48.053 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:48.053 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.053 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.053 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.053 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:48.053 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.053 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.054 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:48.054 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:48.054 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:48.054 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:48.054 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:48.054 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:48.054 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:48.054 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:48.055 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.055 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.055 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.055 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.055 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.055 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:48.055 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:48.055 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:48.056 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.056 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.056 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:48.056 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.056 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.056 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.056 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.056 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.057 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.057 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.057 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:48.057 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:48.057 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.057 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:48.057 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "telemetry", 2023-07-14T18:34:48.057 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:48.058 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:48.058 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:48.058 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_basic": { 2023-07-14T18:34:48.058 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_basic", 2023-07-14T18:34:48.058 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:48.058 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.058 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.058 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:48.059 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.059 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.059 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.059 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-14T18:34:48.059 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.059 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.059 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.059 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.060 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_crash": { 2023-07-14T18:34:48.060 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_crash", 2023-07-14T18:34:48.060 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:48.060 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.060 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.060 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:48.060 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.060 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.061 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.061 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-14T18:34:48.061 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.061 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.061 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.061 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.061 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_device": { 2023-07-14T18:34:48.061 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_device", 2023-07-14T18:34:48.062 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:48.062 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.062 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.062 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-07-14T18:34:48.062 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.062 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.062 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.062 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-14T18:34:48.063 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.063 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.063 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.063 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.063 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_ident": { 2023-07-14T18:34:48.063 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_ident", 2023-07-14T18:34:48.063 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:48.063 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.064 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.064 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:48.064 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.064 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.064 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.064 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-14T18:34:48.064 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.065 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.065 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.065 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.065 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_perf": { 2023-07-14T18:34:48.065 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_perf", 2023-07-14T18:34:48.065 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:48.065 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.065 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.066 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:48.066 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.066 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.066 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.066 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-14T18:34:48.066 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.066 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.066 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.067 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.067 INFO:teuthology.orchestra.run.smithi133.stdout: "contact": { 2023-07-14T18:34:48.067 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "contact", 2023-07-14T18:34:48.067 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.067 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.067 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.067 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:48.067 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.068 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.068 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.068 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.068 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.068 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.068 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.068 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.068 INFO:teuthology.orchestra.run.smithi133.stdout: "description": { 2023-07-14T18:34:48.069 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "description", 2023-07-14T18:34:48.069 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.069 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.069 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.069 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:48.069 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.069 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.069 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.070 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.070 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.070 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.070 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.070 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.070 INFO:teuthology.orchestra.run.smithi133.stdout: "device_url": { 2023-07-14T18:34:48.070 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_url", 2023-07-14T18:34:48.070 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.070 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.071 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.071 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-14T18:34:48.071 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.071 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.071 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.071 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.071 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.072 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.072 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.072 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.072 INFO:teuthology.orchestra.run.smithi133.stdout: "enabled": { 2023-07-14T18:34:48.072 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enabled", 2023-07-14T18:34:48.072 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:48.072 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.072 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.073 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:48.073 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.073 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.073 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.073 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.073 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.073 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.073 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.074 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.074 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-07-14T18:34:48.074 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-07-14T18:34:48.074 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:48.074 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.074 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.074 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "24", 2023-07-14T18:34:48.074 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "8", 2023-07-14T18:34:48.075 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.075 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.075 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.075 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.075 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.075 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.075 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.075 INFO:teuthology.orchestra.run.smithi133.stdout: "last_opt_revision": { 2023-07-14T18:34:48.076 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "last_opt_revision", 2023-07-14T18:34:48.076 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:48.076 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.076 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.076 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1", 2023-07-14T18:34:48.076 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.076 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.076 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.077 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.077 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.077 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.077 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.077 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.077 INFO:teuthology.orchestra.run.smithi133.stdout: "leaderboard": { 2023-07-14T18:34:48.077 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "leaderboard", 2023-07-14T18:34:48.077 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:48.078 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.078 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.078 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:48.078 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.078 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.078 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.078 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.078 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.078 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.079 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.079 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.079 INFO:teuthology.orchestra.run.smithi133.stdout: "leaderboard_description": { 2023-07-14T18:34:48.079 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "leaderboard_description", 2023-07-14T18:34:48.079 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.079 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.079 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.079 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:48.080 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.080 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.080 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.080 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.080 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.080 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.080 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.080 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.081 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:48.081 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:48.081 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.081 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.081 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.081 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:48.081 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.082 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.082 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:48.082 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:48.082 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:48.082 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:48.082 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:48.082 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:48.082 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:48.083 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:48.083 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.083 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.083 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.083 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.083 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.083 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:48.083 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:48.084 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:48.084 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.084 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.084 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:48.084 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.084 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.084 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.084 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.085 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.085 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.085 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.085 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.085 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:48.085 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:48.085 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.085 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.086 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.086 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:48.086 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.086 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.086 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:48.086 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:48.086 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:48.086 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:48.087 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:48.087 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:48.087 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:48.087 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:48.087 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.087 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.087 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.087 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.088 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.088 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:48.088 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:48.088 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:48.088 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.088 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.088 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:48.088 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.089 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.089 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.089 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.089 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.089 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.089 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.089 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.089 INFO:teuthology.orchestra.run.smithi133.stdout: "organization": { 2023-07-14T18:34:48.090 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "organization", 2023-07-14T18:34:48.090 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.090 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.090 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.090 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:48.090 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.090 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.090 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.091 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.091 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.091 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.091 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.091 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.091 INFO:teuthology.orchestra.run.smithi133.stdout: "proxy": { 2023-07-14T18:34:48.091 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "proxy", 2023-07-14T18:34:48.091 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.092 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.092 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.092 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:48.092 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.092 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.092 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.092 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.092 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.093 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.093 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.093 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.093 INFO:teuthology.orchestra.run.smithi133.stdout: "url": { 2023-07-14T18:34:48.093 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "url", 2023-07-14T18:34:48.093 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.093 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.093 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.094 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-14T18:34:48.094 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.094 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.094 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.094 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.094 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.094 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.094 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.095 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:48.095 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:48.095 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.095 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:48.095 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "test_orchestrator", 2023-07-14T18:34:48.095 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:48.095 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:48.095 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:48.096 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:48.096 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:48.096 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.096 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.096 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.096 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:48.096 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.096 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.097 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:48.097 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:48.097 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:48.097 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:48.097 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:48.097 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:48.097 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:48.097 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:48.098 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.098 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.098 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.098 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.098 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.098 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:48.098 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:48.098 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:48.099 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.099 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.099 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:48.099 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.099 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.099 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.099 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.099 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.100 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.100 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.100 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.100 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:48.100 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:48.100 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.100 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.100 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.101 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:48.101 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.101 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.101 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:48.101 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:48.101 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:48.101 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:48.102 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:48.102 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:48.102 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:48.102 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:48.102 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.102 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.102 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.102 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.103 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.103 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:48.103 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:48.103 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:48.103 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.103 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.103 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:48.103 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.104 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.104 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.104 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.104 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.104 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.104 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.104 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:48.104 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:48.105 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.105 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:48.105 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "volumes", 2023-07-14T18:34:48.105 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:48.105 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:48.105 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:48.105 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:48.105 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:48.106 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.106 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.106 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.106 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:48.106 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.106 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.106 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:48.106 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:48.107 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:48.107 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:48.107 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:48.107 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:48.107 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:48.107 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:48.107 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.107 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.108 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.108 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.108 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.108 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:48.108 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:48.108 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:48.108 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.108 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.109 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:48.109 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.109 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.109 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.109 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.109 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.109 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.109 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.110 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.110 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:48.110 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:48.110 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.110 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.110 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.110 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:48.110 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.111 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.111 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:48.111 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:48.111 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:48.111 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:48.111 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:48.111 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:48.111 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:48.112 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:48.112 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.112 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.112 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.112 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.112 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.112 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:48.112 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:48.113 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:48.113 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.113 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.113 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:48.113 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.113 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.113 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.113 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.114 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.114 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.114 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.114 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.114 INFO:teuthology.orchestra.run.smithi133.stdout: "max_concurrent_clones": { 2023-07-14T18:34:48.114 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_concurrent_clones", 2023-07-14T18:34:48.114 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:48.114 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.115 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.115 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "4", 2023-07-14T18:34:48.115 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.115 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.115 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.115 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-14T18:34:48.115 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.115 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.116 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.116 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.116 INFO:teuthology.orchestra.run.smithi133.stdout: "snapshot_clone_delay": { 2023-07-14T18:34:48.116 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snapshot_clone_delay", 2023-07-14T18:34:48.116 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:48.116 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.116 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.116 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-07-14T18:34:48.117 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.117 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.117 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.117 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-14T18:34:48.117 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.117 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.117 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.117 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:48.118 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:48.118 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.118 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:48.118 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix", 2023-07-14T18:34:48.118 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-07-14T18:34:48.118 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-07-14T18:34:48.118 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-07-14T18:34:48.118 INFO:teuthology.orchestra.run.smithi133.stdout: "discovery_interval": { 2023-07-14T18:34:48.119 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "discovery_interval", 2023-07-14T18:34:48.119 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-07-14T18:34:48.119 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.119 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.119 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "100", 2023-07-14T18:34:48.119 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.119 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.119 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.120 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.120 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.120 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.120 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.120 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.120 INFO:teuthology.orchestra.run.smithi133.stdout: "identifier": { 2023-07-14T18:34:48.120 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "identifier", 2023-07-14T18:34:48.120 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.121 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.121 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.121 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:48.121 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.121 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.121 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.121 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.121 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.122 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.122 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.122 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.122 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-07-14T18:34:48.122 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-07-14T18:34:48.122 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-07-14T18:34:48.122 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.122 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.123 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-07-14T18:34:48.123 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.123 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.123 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.123 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.123 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.123 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.123 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.124 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.124 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-07-14T18:34:48.124 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-07-14T18:34:48.124 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.124 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.124 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.124 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:48.124 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.125 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.125 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:48.125 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:48.125 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:48.125 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:48.125 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:48.125 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:48.125 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:48.126 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:48.126 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.126 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.126 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.126 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.126 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.126 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-07-14T18:34:48.126 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-07-14T18:34:48.127 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:48.127 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.127 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.127 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:48.127 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.127 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.127 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.127 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.128 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.128 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.128 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.128 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.128 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-07-14T18:34:48.128 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-07-14T18:34:48.128 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.129 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.129 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.129 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-07-14T18:34:48.129 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.129 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.129 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-07-14T18:34:48.129 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-07-14T18:34:48.129 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-07-14T18:34:48.129 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-07-14T18:34:48.130 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-07-14T18:34:48.130 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-07-14T18:34:48.130 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-07-14T18:34:48.130 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:48.130 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.130 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.130 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.130 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.131 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.131 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-07-14T18:34:48.131 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-07-14T18:34:48.131 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-07-14T18:34:48.131 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.131 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-07-14T18:34:48.131 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-07-14T18:34:48.131 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.132 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.132 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.132 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.132 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.132 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.132 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.132 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.132 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_host": { 2023-07-14T18:34:48.133 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_host", 2023-07-14T18:34:48.133 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.133 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.133 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.133 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-07-14T18:34:48.133 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.133 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.133 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.134 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.134 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.134 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.134 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.134 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.134 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_port": { 2023-07-14T18:34:48.134 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_port", 2023-07-14T18:34:48.134 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-07-14T18:34:48.135 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.135 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.135 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10051", 2023-07-14T18:34:48.135 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.135 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.135 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.135 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.135 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.136 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.136 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.136 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.136 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_sender": { 2023-07-14T18:34:48.136 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_sender", 2023-07-14T18:34:48.136 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-07-14T18:34:48.136 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-07-14T18:34:48.136 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-07-14T18:34:48.137 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-14T18:34:48.137 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-07-14T18:34:48.137 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-07-14T18:34:48.137 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-07-14T18:34:48.137 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-07-14T18:34:48.137 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-07-14T18:34:48.137 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-07-14T18:34:48.137 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-07-14T18:34:48.138 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:48.138 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:48.138 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:48.138 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:48.138 INFO:teuthology.orchestra.run.smithi133.stdout: "services": { 2023-07-14T18:34:48.138 INFO:teuthology.orchestra.run.smithi133.stdout: "dashboard": "https://172.21.15.133:8443/" 2023-07-14T18:34:48.138 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.138 INFO:teuthology.orchestra.run.smithi133.stdout: "always_on_modules": { 2023-07-14T18:34:48.139 INFO:teuthology.orchestra.run.smithi133.stdout: "octopus": [ 2023-07-14T18:34:48.139 INFO:teuthology.orchestra.run.smithi133.stdout: "balancer", 2023-07-14T18:34:48.139 INFO:teuthology.orchestra.run.smithi133.stdout: "crash", 2023-07-14T18:34:48.139 INFO:teuthology.orchestra.run.smithi133.stdout: "devicehealth", 2023-07-14T18:34:48.139 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator", 2023-07-14T18:34:48.139 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_autoscaler", 2023-07-14T18:34:48.139 INFO:teuthology.orchestra.run.smithi133.stdout: "progress", 2023-07-14T18:34:48.140 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_support", 2023-07-14T18:34:48.140 INFO:teuthology.orchestra.run.smithi133.stdout: "status", 2023-07-14T18:34:48.140 INFO:teuthology.orchestra.run.smithi133.stdout: "telemetry", 2023-07-14T18:34:48.140 INFO:teuthology.orchestra.run.smithi133.stdout: "volumes" 2023-07-14T18:34:48.140 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:48.140 INFO:teuthology.orchestra.run.smithi133.stdout: "pacific": [ 2023-07-14T18:34:48.140 INFO:teuthology.orchestra.run.smithi133.stdout: "balancer", 2023-07-14T18:34:48.140 INFO:teuthology.orchestra.run.smithi133.stdout: "crash", 2023-07-14T18:34:48.141 INFO:teuthology.orchestra.run.smithi133.stdout: "devicehealth", 2023-07-14T18:34:48.141 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator", 2023-07-14T18:34:48.141 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_autoscaler", 2023-07-14T18:34:48.141 INFO:teuthology.orchestra.run.smithi133.stdout: "progress", 2023-07-14T18:34:48.141 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_support", 2023-07-14T18:34:48.141 INFO:teuthology.orchestra.run.smithi133.stdout: "status", 2023-07-14T18:34:48.141 INFO:teuthology.orchestra.run.smithi133.stdout: "telemetry", 2023-07-14T18:34:48.141 INFO:teuthology.orchestra.run.smithi133.stdout: "volumes" 2023-07-14T18:34:48.142 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:48.142 INFO:teuthology.orchestra.run.smithi133.stdout: "quincy": [ 2023-07-14T18:34:48.142 INFO:teuthology.orchestra.run.smithi133.stdout: "balancer", 2023-07-14T18:34:48.142 INFO:teuthology.orchestra.run.smithi133.stdout: "crash", 2023-07-14T18:34:48.142 INFO:teuthology.orchestra.run.smithi133.stdout: "devicehealth", 2023-07-14T18:34:48.142 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator", 2023-07-14T18:34:48.142 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_autoscaler", 2023-07-14T18:34:48.142 INFO:teuthology.orchestra.run.smithi133.stdout: "progress", 2023-07-14T18:34:48.142 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_support", 2023-07-14T18:34:48.143 INFO:teuthology.orchestra.run.smithi133.stdout: "status", 2023-07-14T18:34:48.143 INFO:teuthology.orchestra.run.smithi133.stdout: "telemetry", 2023-07-14T18:34:48.143 INFO:teuthology.orchestra.run.smithi133.stdout: "volumes" 2023-07-14T18:34:48.143 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-07-14T18:34:48.143 INFO:teuthology.orchestra.run.smithi133.stdout: "reef": [ 2023-07-14T18:34:48.143 INFO:teuthology.orchestra.run.smithi133.stdout: "balancer", 2023-07-14T18:34:48.143 INFO:teuthology.orchestra.run.smithi133.stdout: "crash", 2023-07-14T18:34:48.143 INFO:teuthology.orchestra.run.smithi133.stdout: "devicehealth", 2023-07-14T18:34:48.144 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator", 2023-07-14T18:34:48.144 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_autoscaler", 2023-07-14T18:34:48.144 INFO:teuthology.orchestra.run.smithi133.stdout: "progress", 2023-07-14T18:34:48.144 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_support", 2023-07-14T18:34:48.144 INFO:teuthology.orchestra.run.smithi133.stdout: "status", 2023-07-14T18:34:48.144 INFO:teuthology.orchestra.run.smithi133.stdout: "telemetry", 2023-07-14T18:34:48.144 INFO:teuthology.orchestra.run.smithi133.stdout: "volumes" 2023-07-14T18:34:48.145 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2023-07-14T18:34:48.145 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.145 INFO:teuthology.orchestra.run.smithi133.stdout: "last_failure_osd_epoch": 3, 2023-07-14T18:34:48.145 INFO:teuthology.orchestra.run.smithi133.stdout: "active_clients": [ 2023-07-14T18:34:48.145 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:48.145 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "devicehealth", 2023-07-14T18:34:48.145 INFO:teuthology.orchestra.run.smithi133.stdout: "addrvec": [ 2023-07-14T18:34:48.145 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:48.146 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v2", 2023-07-14T18:34:48.146 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:0", 2023-07-14T18:34:48.146 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 624488340 2023-07-14T18:34:48.146 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:48.146 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2023-07-14T18:34:48.146 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.146 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:48.146 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "libcephsqlite", 2023-07-14T18:34:48.147 INFO:teuthology.orchestra.run.smithi133.stdout: "addrvec": [ 2023-07-14T18:34:48.147 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:48.147 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v2", 2023-07-14T18:34:48.147 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:0", 2023-07-14T18:34:48.147 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 2547348609 2023-07-14T18:34:48.147 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:48.147 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2023-07-14T18:34:48.147 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.148 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:48.148 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_support", 2023-07-14T18:34:48.148 INFO:teuthology.orchestra.run.smithi133.stdout: "addrvec": [ 2023-07-14T18:34:48.148 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:48.148 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v2", 2023-07-14T18:34:48.148 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:0", 2023-07-14T18:34:48.148 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 2817674038 2023-07-14T18:34:48.148 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:48.149 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2023-07-14T18:34:48.149 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:48.149 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:48.149 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "volumes", 2023-07-14T18:34:48.149 INFO:teuthology.orchestra.run.smithi133.stdout: "addrvec": [ 2023-07-14T18:34:48.149 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-07-14T18:34:48.149 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v2", 2023-07-14T18:34:48.149 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:0", 2023-07-14T18:34:48.150 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 605654724 2023-07-14T18:34:48.150 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:48.150 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2023-07-14T18:34:48.150 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:48.150 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2023-07-14T18:34:48.150 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:34:48.152 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:46 smithi161 ceph-mon[128993]: pgmap v890: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.8 MiB/s wr, 579 op/s 2023-07-14T18:34:48.152 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:46 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2958468735' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-14T18:34:48.152 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:47 smithi161 ceph-mon[128993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:34:48.152 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:45 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3247759941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:34:48.153 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:45 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3489409416' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:34:48.153 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:46 smithi171 ceph-mon[129241]: pgmap v890: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.8 MiB/s wr, 579 op/s 2023-07-14T18:34:48.153 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:46 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2958468735' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-14T18:34:48.153 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:47 smithi171 ceph-mon[129241]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:34:48.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.819+0000 7fc58a1ff700 1 -- 172.21.15.133:0/2958468735 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc56c0a11a0 msgr2=0x7fc56c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:48.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.819+0000 7fc58a1ff700 1 --2- 172.21.15.133:0/2958468735 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc56c0a11a0 0x7fc56c0a3650 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc574011490 tx=0x7fc574009260 comp rx=0 tx=0).stop 2023-07-14T18:34:48.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.819+0000 7fc58a1ff700 1 -- 172.21.15.133:0/2958468735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc584107aa0 msgr2=0x7fc58419f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:34:48.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.819+0000 7fc58a1ff700 1 --2- 172.21.15.133:0/2958468735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc584107aa0 0x7fc58419f2f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00efc0 tx=0x7fc57c00c3a0 comp rx=0 tx=0).stop 2023-07-14T18:34:48.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.819+0000 7fc58a1ff700 1 -- 172.21.15.133:0/2958468735 shutdown_connections 2023-07-14T18:34:48.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.819+0000 7fc58a1ff700 1 --2- 172.21.15.133:0/2958468735 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fc56c0a11a0 0x7fc56c0a3650 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:48.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.819+0000 7fc58a1ff700 1 --2- 172.21.15.133:0/2958468735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc584107aa0 0x7fc58419f2f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:48.155 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.819+0000 7fc58a1ff700 1 --2- 172.21.15.133:0/2958468735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc584073040 0x7fc584197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:48.155 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.819+0000 7fc58a1ff700 1 --2- 172.21.15.133:0/2958468735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc584074620 0x7fc584198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:34:48.155 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.819+0000 7fc58a1ff700 1 -- 172.21.15.133:0/2958468735 >> 172.21.15.133:0/2958468735 conn(0x7fc5840fb230 msgr2=0x7fc584109df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:34:48.155 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.819+0000 7fc58a1ff700 1 -- 172.21.15.133:0/2958468735 shutdown_connections 2023-07-14T18:34:48.155 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:34:45.819+0000 7fc58a1ff700 1 -- 172.21.15.133:0/2958468735 wait complete. 2023-07-14T18:34:48.162 DEBUG:tasks.check_counter:Getting stats from a 2023-07-14T18:34:48.162 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph daemon mds.a perf dump 2023-07-14T18:34:48.163 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:46 smithi133 ceph-mon[126384]: pgmap v890: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.8 MiB/s wr, 579 op/s 2023-07-14T18:34:48.164 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:46 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2958468735' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-14T18:34:48.164 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:47 smithi133 ceph-mon[126384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:34:48.825 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:48 smithi161 ceph-mon[128993]: pgmap v891: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 3.8 MiB/s wr, 463 op/s 2023-07-14T18:34:48.840 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:48 smithi171 ceph-mon[129241]: pgmap v891: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 3.8 MiB/s wr, 463 op/s 2023-07-14T18:34:48.881 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:48 smithi133 ceph-mon[126384]: pgmap v891: 129 pgs: 129 active+clean; 8.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 3.8 MiB/s wr, 463 op/s 2023-07-14T18:34:50.825 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:50 smithi161 ceph-mon[128993]: pgmap v892: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 3.2 MiB/s wr, 403 op/s 2023-07-14T18:34:50.840 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:50 smithi171 ceph-mon[129241]: pgmap v892: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 3.2 MiB/s wr, 403 op/s 2023-07-14T18:34:50.881 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:50 smithi133 ceph-mon[126384]: pgmap v892: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 3.2 MiB/s wr, 403 op/s 2023-07-14T18:34:50.918 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:34:52.825 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:52 smithi161 ceph-mon[128993]: pgmap v893: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 2.7 MiB/s wr, 341 op/s 2023-07-14T18:34:52.826 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:52 smithi161 ceph-mon[128993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:34:52.840 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:52 smithi171 ceph-mon[129241]: pgmap v893: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 2.7 MiB/s wr, 341 op/s 2023-07-14T18:34:52.841 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:52 smithi171 ceph-mon[129241]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:34:52.882 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:52 smithi133 ceph-mon[126384]: pgmap v893: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 2.7 MiB/s wr, 341 op/s 2023-07-14T18:34:52.882 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:52 smithi133 ceph-mon[126384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:34:53.090 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:34:53.091 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T18:34:53.091 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 0, 2023-07-14T18:34:53.091 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 2, 2023-07-14T18:34:53.091 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 0, 2023-07-14T18:34:53.091 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 0, 2023-07-14T18:34:53.092 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 4, 2023-07-14T18:34:53.092 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 0, 2023-07-14T18:34:53.092 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.003651396, 2023-07-14T18:34:53.092 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.000000000, 2023-07-14T18:34:53.092 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.000657011, 2023-07-14T18:34:53.092 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T18:34:53.092 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:34:53.093 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T18:34:53.093 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T18:34:53.093 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T18:34:53.093 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:53.093 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:34:53.093 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T18:34:53.094 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T18:34:53.094 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T18:34:53.094 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:53.094 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-14T18:34:53.094 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 0 2023-07-14T18:34:53.094 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:53.094 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T18:34:53.094 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 2, 2023-07-14T18:34:53.095 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 253, 2023-07-14T18:34:53.095 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 320, 2023-07-14T18:34:53.095 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 196272, 2023-07-14T18:34:53.095 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 4, 2023-07-14T18:34:53.095 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 2, 2023-07-14T18:34:53.095 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.024643532, 2023-07-14T18:34:53.096 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.019611752, 2023-07-14T18:34:53.096 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.002295309, 2023-07-14T18:34:53.096 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T18:34:53.096 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:34:53.096 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 251, 2023-07-14T18:34:53.096 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.058692967, 2023-07-14T18:34:53.096 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000233836 2023-07-14T18:34:53.096 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:53.097 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:34:53.097 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T18:34:53.097 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T18:34:53.097 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T18:34:53.097 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:53.097 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-14T18:34:53.097 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 196272 2023-07-14T18:34:53.098 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:53.098 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T18:34:53.098 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 337, 2023-07-14T18:34:53.098 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 332, 2023-07-14T18:34:53.098 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 507232, 2023-07-14T18:34:53.098 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 189584, 2023-07-14T18:34:53.098 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 2, 2023-07-14T18:34:53.098 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 1, 2023-07-14T18:34:53.099 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.061891882, 2023-07-14T18:34:53.099 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.027752693, 2023-07-14T18:34:53.099 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.021803937, 2023-07-14T18:34:53.099 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.009758061, 2023-07-14T18:34:53.099 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:34:53.099 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 332, 2023-07-14T18:34:53.099 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.126380130, 2023-07-14T18:34:53.099 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000380663 2023-07-14T18:34:53.100 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:53.100 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:34:53.100 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T18:34:53.100 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T18:34:53.100 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T18:34:53.100 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:53.100 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 507232, 2023-07-14T18:34:53.101 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 189584 2023-07-14T18:34:53.101 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:53.101 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2023-07-14T18:34:53.101 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 0, 2023-07-14T18:34:53.101 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2023-07-14T18:34:53.102 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:53.102 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2023-07-14T18:34:53.102 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2023-07-14T18:34:53.102 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2023-07-14T18:34:53.102 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T18:34:53.102 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2023-07-14T18:34:53.102 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T18:34:53.103 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2023-07-14T18:34:53.103 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T18:34:53.103 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T18:34:53.103 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T18:34:53.103 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T18:34:53.103 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T18:34:53.103 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2023-07-14T18:34:53.104 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T18:34:53.104 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T18:34:53.104 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2023-07-14T18:34:53.104 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2023-07-14T18:34:53.104 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2023-07-14T18:34:53.104 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2023-07-14T18:34:53.105 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T18:34:53.105 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T18:34:53.105 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T18:34:53.105 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T18:34:53.105 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T18:34:53.105 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2023-07-14T18:34:53.105 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2023-07-14T18:34:53.106 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2023-07-14T18:34:53.106 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T18:34:53.106 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T18:34:53.106 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2023-07-14T18:34:53.106 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2023-07-14T18:34:53.106 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2023-07-14T18:34:53.106 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2023-07-14T18:34:53.106 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T18:34:53.107 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2023-07-14T18:34:53.107 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T18:34:53.107 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2023-07-14T18:34:53.107 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 148704, 2023-07-14T18:34:53.107 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 29, 2023-07-14T18:34:53.107 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2023-07-14T18:34:53.107 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2023-07-14T18:34:53.107 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2023-07-14T18:34:53.108 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2023-07-14T18:34:53.108 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2023-07-14T18:34:53.108 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2023-07-14T18:34:53.108 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2023-07-14T18:34:53.108 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2023-07-14T18:34:53.108 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 0, 2023-07-14T18:34:53.109 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 0, 2023-07-14T18:34:53.109 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T18:34:53.109 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2023-07-14T18:34:53.109 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2023-07-14T18:34:53.109 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2023-07-14T18:34:53.109 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 928, 2023-07-14T18:34:53.109 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 2, 2023-07-14T18:34:53.109 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2023-07-14T18:34:53.110 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2023-07-14T18:34:53.110 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2023-07-14T18:34:53.110 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2023-07-14T18:34:53.110 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:53.110 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T18:34:53.110 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2023-07-14T18:34:53.110 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2023-07-14T18:34:53.110 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2023-07-14T18:34:53.111 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 335, 2023-07-14T18:34:53.111 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 457149, 2023-07-14T18:34:53.111 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2023-07-14T18:34:53.111 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 335, 2023-07-14T18:34:53.111 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2023-07-14T18:34:53.111 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2023-07-14T18:34:53.112 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 335, 2023-07-14T18:34:53.112 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 457149, 2023-07-14T18:34:53.112 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2023-07-14T18:34:53.112 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T18:34:53.112 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T18:34:53.112 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T18:34:53.113 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:53.113 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:53.113 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:34:53.614 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2023-07-14T18:34:53.614 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2023-07-14T18:34:53.614 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2023-07-14T18:34:53.614 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2023-07-14T18:34:53.615 DEBUG:tasks.check_counter:Getting stats from d 2023-07-14T18:34:53.615 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph daemon mds.d perf dump 2023-07-14T18:34:54.338 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:34:54.825 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:54 smithi161 ceph-mon[128993]: pgmap v894: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2023-07-14T18:34:54.840 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:54 smithi171 ceph-mon[129241]: pgmap v894: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2023-07-14T18:34:54.881 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:54 smithi133 ceph-mon[126384]: pgmap v894: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2023-07-14T18:34:55.905 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:34:55.906 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T18:34:55.906 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 0, 2023-07-14T18:34:55.906 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 2, 2023-07-14T18:34:55.906 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 0, 2023-07-14T18:34:55.906 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 0, 2023-07-14T18:34:55.906 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 4, 2023-07-14T18:34:55.907 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 0, 2023-07-14T18:34:55.907 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.003325926, 2023-07-14T18:34:55.907 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.000000000, 2023-07-14T18:34:55.907 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.000711610, 2023-07-14T18:34:55.907 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T18:34:55.907 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:34:55.907 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T18:34:55.908 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T18:34:55.908 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T18:34:55.908 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:55.908 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:34:55.908 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T18:34:55.908 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T18:34:55.908 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T18:34:55.909 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:55.909 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-14T18:34:55.909 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 0 2023-07-14T18:34:55.909 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:55.909 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T18:34:55.909 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 2, 2023-07-14T18:34:55.910 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 257, 2023-07-14T18:34:55.910 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 320, 2023-07-14T18:34:55.910 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 196976, 2023-07-14T18:34:55.910 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 4, 2023-07-14T18:34:55.911 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 2, 2023-07-14T18:34:55.911 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.025850596, 2023-07-14T18:34:55.911 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.021015710, 2023-07-14T18:34:55.912 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.002015117, 2023-07-14T18:34:55.912 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T18:34:55.912 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:34:55.912 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 255, 2023-07-14T18:34:55.913 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.037176097, 2023-07-14T18:34:55.913 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000145788 2023-07-14T18:34:55.913 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:55.913 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:34:55.914 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T18:34:55.914 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T18:34:55.914 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T18:34:55.915 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:55.915 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-14T18:34:55.915 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 196976 2023-07-14T18:34:55.915 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:55.916 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T18:34:55.916 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 343, 2023-07-14T18:34:55.917 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 338, 2023-07-14T18:34:55.917 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 509488, 2023-07-14T18:34:55.918 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 192880, 2023-07-14T18:34:55.918 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 2, 2023-07-14T18:34:55.918 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 1, 2023-07-14T18:34:55.918 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.064340068, 2023-07-14T18:34:55.919 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.028747871, 2023-07-14T18:34:55.919 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.023737709, 2023-07-14T18:34:55.919 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.009930228, 2023-07-14T18:34:55.919 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:34:55.919 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 338, 2023-07-14T18:34:55.919 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.147170481, 2023-07-14T18:34:55.919 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000435415 2023-07-14T18:34:55.920 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:55.920 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:34:55.920 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T18:34:55.920 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T18:34:55.920 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T18:34:55.920 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:55.920 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 509488, 2023-07-14T18:34:55.921 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 192880 2023-07-14T18:34:55.921 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:55.921 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2023-07-14T18:34:55.921 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 0, 2023-07-14T18:34:55.921 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2023-07-14T18:34:55.921 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:55.921 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2023-07-14T18:34:55.921 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2023-07-14T18:34:55.922 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2023-07-14T18:34:55.922 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T18:34:55.922 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2023-07-14T18:34:55.922 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T18:34:55.922 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2023-07-14T18:34:55.922 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T18:34:55.922 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T18:34:55.923 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T18:34:55.923 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T18:34:55.923 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T18:34:55.923 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2023-07-14T18:34:55.923 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T18:34:55.923 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T18:34:55.923 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2023-07-14T18:34:55.924 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2023-07-14T18:34:55.924 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2023-07-14T18:34:55.924 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2023-07-14T18:34:55.924 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T18:34:55.924 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T18:34:55.924 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T18:34:55.924 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T18:34:55.925 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T18:34:55.925 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2023-07-14T18:34:55.925 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2023-07-14T18:34:55.925 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2023-07-14T18:34:55.925 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T18:34:55.925 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T18:34:55.925 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2023-07-14T18:34:55.925 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2023-07-14T18:34:55.926 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2023-07-14T18:34:55.926 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2023-07-14T18:34:55.926 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T18:34:55.926 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2023-07-14T18:34:55.926 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T18:34:55.926 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2023-07-14T18:34:55.926 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 149735, 2023-07-14T18:34:55.927 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 30, 2023-07-14T18:34:55.927 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2023-07-14T18:34:55.927 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2023-07-14T18:34:55.927 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2023-07-14T18:34:55.927 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2023-07-14T18:34:55.928 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2023-07-14T18:34:55.928 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2023-07-14T18:34:55.928 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2023-07-14T18:34:55.928 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2023-07-14T18:34:55.928 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 0, 2023-07-14T18:34:55.928 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 0, 2023-07-14T18:34:55.928 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T18:34:55.929 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2023-07-14T18:34:55.929 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2023-07-14T18:34:55.929 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2023-07-14T18:34:55.929 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 928, 2023-07-14T18:34:55.929 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 2, 2023-07-14T18:34:55.929 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2023-07-14T18:34:55.929 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2023-07-14T18:34:55.929 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2023-07-14T18:34:55.930 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2023-07-14T18:34:55.930 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:55.930 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T18:34:55.930 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2023-07-14T18:34:55.930 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2023-07-14T18:34:55.930 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2023-07-14T18:34:55.930 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 341, 2023-07-14T18:34:55.930 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 458566, 2023-07-14T18:34:55.931 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2023-07-14T18:34:55.931 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 341, 2023-07-14T18:34:55.931 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2023-07-14T18:34:55.931 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2023-07-14T18:34:55.931 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 341, 2023-07-14T18:34:55.931 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 458566, 2023-07-14T18:34:55.931 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2023-07-14T18:34:55.931 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T18:34:55.932 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T18:34:55.932 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T18:34:55.932 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:55.932 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:55.932 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:34:56.430 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-07-14T18:34:56.430 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-07-14T18:34:56.430 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2023-07-14T18:34:56.431 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2023-07-14T18:34:56.431 DEBUG:tasks.check_counter:Getting stats from g 2023-07-14T18:34:56.431 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph daemon mds.g perf dump 2023-07-14T18:34:56.631 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:56 smithi133 ceph-mon[126384]: pgmap v895: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 687 KiB/s wr, 87 op/s 2023-07-14T18:34:56.825 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:56 smithi161 ceph-mon[128993]: pgmap v895: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 687 KiB/s wr, 87 op/s 2023-07-14T18:34:56.840 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:56 smithi171 ceph-mon[129241]: pgmap v895: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 687 KiB/s wr, 87 op/s 2023-07-14T18:34:57.162 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:34:57.794 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:57 smithi133 ceph-mon[126384]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:34:57.825 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:57 smithi161 ceph-mon[128993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:34:57.840 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:57 smithi171 ceph-mon[129241]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T18:34:58.781 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:34:58.781 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T18:34:58.781 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 0, 2023-07-14T18:34:58.782 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 2, 2023-07-14T18:34:58.782 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 0, 2023-07-14T18:34:58.782 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 0, 2023-07-14T18:34:58.782 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 4, 2023-07-14T18:34:58.782 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 0, 2023-07-14T18:34:58.782 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.003360660, 2023-07-14T18:34:58.783 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.000000000, 2023-07-14T18:34:58.783 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.000645116, 2023-07-14T18:34:58.783 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T18:34:58.783 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:34:58.783 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T18:34:58.783 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T18:34:58.783 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T18:34:58.784 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:58.784 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:34:58.784 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T18:34:58.784 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T18:34:58.784 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T18:34:58.784 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:58.785 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-14T18:34:58.785 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 0 2023-07-14T18:34:58.785 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:58.785 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T18:34:58.785 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 336, 2023-07-14T18:34:58.785 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 333, 2023-07-14T18:34:58.785 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 505056, 2023-07-14T18:34:58.786 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 190912, 2023-07-14T18:34:58.786 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 2, 2023-07-14T18:34:58.786 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 1, 2023-07-14T18:34:58.786 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.062518566, 2023-07-14T18:34:58.787 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.027948557, 2023-07-14T18:34:58.787 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.022570980, 2023-07-14T18:34:58.787 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.009985997, 2023-07-14T18:34:58.787 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:34:58.787 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 332, 2023-07-14T18:34:58.787 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.089301955, 2023-07-14T18:34:58.787 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000268981 2023-07-14T18:34:58.788 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:58.788 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:34:58.788 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T18:34:58.788 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T18:34:58.788 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T18:34:58.788 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:58.789 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 505056, 2023-07-14T18:34:58.789 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 190912 2023-07-14T18:34:58.789 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:58.789 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T18:34:58.789 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 6, 2023-07-14T18:34:58.789 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 257, 2023-07-14T18:34:58.789 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 3312, 2023-07-14T18:34:58.790 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 197072, 2023-07-14T18:34:58.790 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 4, 2023-07-14T18:34:58.790 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 2, 2023-07-14T18:34:58.790 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.026665625, 2023-07-14T18:34:58.790 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.020749732, 2023-07-14T18:34:58.790 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.002864497, 2023-07-14T18:34:58.790 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T18:34:58.790 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:34:58.791 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 256, 2023-07-14T18:34:58.791 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.068844104, 2023-07-14T18:34:58.791 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000268922 2023-07-14T18:34:58.791 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:58.792 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:34:58.792 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T18:34:58.792 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T18:34:58.793 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T18:34:58.793 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:58.793 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-07-14T18:34:58.794 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 197072 2023-07-14T18:34:58.794 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:58.794 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2023-07-14T18:34:58.794 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 0, 2023-07-14T18:34:58.795 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2023-07-14T18:34:58.795 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:58.795 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2023-07-14T18:34:58.795 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2023-07-14T18:34:58.796 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2023-07-14T18:34:58.796 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T18:34:58.796 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2023-07-14T18:34:58.796 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T18:34:58.796 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2023-07-14T18:34:58.796 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T18:34:58.797 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T18:34:58.797 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T18:34:58.797 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T18:34:58.798 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T18:34:58.798 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2023-07-14T18:34:58.798 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T18:34:58.798 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T18:34:58.798 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2023-07-14T18:34:58.798 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2023-07-14T18:34:58.799 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2023-07-14T18:34:58.799 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2023-07-14T18:34:58.799 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T18:34:58.799 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T18:34:58.799 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T18:34:58.799 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T18:34:58.799 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T18:34:58.800 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2023-07-14T18:34:58.800 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2023-07-14T18:34:58.800 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2023-07-14T18:34:58.800 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T18:34:58.800 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T18:34:58.800 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2023-07-14T18:34:58.801 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2023-07-14T18:34:58.801 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2023-07-14T18:34:58.801 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2023-07-14T18:34:58.801 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T18:34:58.801 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2023-07-14T18:34:58.802 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T18:34:58.802 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2023-07-14T18:34:58.802 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 148704, 2023-07-14T18:34:58.802 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 29, 2023-07-14T18:34:58.803 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2023-07-14T18:34:58.803 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2023-07-14T18:34:58.803 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2023-07-14T18:34:58.803 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2023-07-14T18:34:58.803 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2023-07-14T18:34:58.804 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2023-07-14T18:34:58.804 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2023-07-14T18:34:58.804 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2023-07-14T18:34:58.804 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 0, 2023-07-14T18:34:58.805 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 0, 2023-07-14T18:34:58.805 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T18:34:58.805 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2023-07-14T18:34:58.805 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2023-07-14T18:34:58.805 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2023-07-14T18:34:58.805 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 928, 2023-07-14T18:34:58.806 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 2, 2023-07-14T18:34:58.806 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2023-07-14T18:34:58.806 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2023-07-14T18:34:58.806 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2023-07-14T18:34:58.806 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2023-07-14T18:34:58.806 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:34:58.806 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T18:34:58.807 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2023-07-14T18:34:58.807 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2023-07-14T18:34:58.807 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2023-07-14T18:34:58.807 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 338, 2023-07-14T18:34:58.807 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 457539, 2023-07-14T18:34:58.807 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2023-07-14T18:34:58.807 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 338, 2023-07-14T18:34:58.808 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2023-07-14T18:34:58.808 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2023-07-14T18:34:58.808 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 338, 2023-07-14T18:34:58.808 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 457539, 2023-07-14T18:34:58.809 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2023-07-14T18:34:58.809 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T18:34:58.809 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T18:34:58.809 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T18:34:58.809 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:58.809 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:34:58.809 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:34:58.810 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:34:58 smithi133 ceph-mon[126384]: pgmap v896: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 327 KiB/s wr, 61 op/s 2023-07-14T18:34:58.840 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:34:58 smithi171 ceph-mon[129241]: pgmap v896: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 327 KiB/s wr, 61 op/s 2023-07-14T18:34:59.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:34:58 smithi161 ceph-mon[128993]: pgmap v896: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 327 KiB/s wr, 61 op/s 2023-07-14T18:34:59.254 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-07-14T18:34:59.254 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-07-14T18:34:59.255 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2023-07-14T18:34:59.255 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2023-07-14T18:34:59.255 DEBUG:tasks.check_counter:Getting stats from j 2023-07-14T18:34:59.255 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph daemon mds.j perf dump 2023-07-14T18:34:59.978 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T18:35:00.808 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:00 smithi133 ceph-mon[126384]: pgmap v897: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 10 op/s 2023-07-14T18:35:00.840 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:00 smithi171 ceph-mon[129241]: pgmap v897: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 10 op/s 2023-07-14T18:35:01.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:00 smithi161 ceph-mon[128993]: pgmap v897: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 10 op/s 2023-07-14T18:35:01.537 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T18:35:01.537 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T18:35:01.537 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 346, 2023-07-14T18:35:01.537 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 341, 2023-07-14T18:35:01.538 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 510304, 2023-07-14T18:35:01.538 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 194656, 2023-07-14T18:35:01.538 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 4, 2023-07-14T18:35:01.538 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 0, 2023-07-14T18:35:01.538 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.064638469, 2023-07-14T18:35:01.538 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.021384356, 2023-07-14T18:35:01.538 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.026568490, 2023-07-14T18:35:01.538 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.011924480, 2023-07-14T18:35:01.539 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:35:01.539 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 341, 2023-07-14T18:35:01.539 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.135272434, 2023-07-14T18:35:01.539 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000396693 2023-07-14T18:35:01.539 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:35:01.539 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:35:01.539 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T18:35:01.540 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T18:35:01.540 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T18:35:01.540 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:35:01.540 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 510304, 2023-07-14T18:35:01.540 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 194656 2023-07-14T18:35:01.540 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:35:01.540 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T18:35:01.541 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 2, 2023-07-14T18:35:01.541 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 259, 2023-07-14T18:35:01.541 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 320, 2023-07-14T18:35:01.541 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 197328, 2023-07-14T18:35:01.541 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 4, 2023-07-14T18:35:01.541 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 2, 2023-07-14T18:35:01.541 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.026177753, 2023-07-14T18:35:01.541 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.020552377, 2023-07-14T18:35:01.542 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.002687014, 2023-07-14T18:35:01.542 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T18:35:01.542 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:35:01.542 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 257, 2023-07-14T18:35:01.542 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.068866571, 2023-07-14T18:35:01.542 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000267963 2023-07-14T18:35:01.542 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:35:01.542 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:35:01.543 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T18:35:01.543 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T18:35:01.543 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T18:35:01.543 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:35:01.543 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-14T18:35:01.543 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 197328 2023-07-14T18:35:01.543 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:35:01.543 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T18:35:01.544 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 0, 2023-07-14T18:35:01.544 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 2, 2023-07-14T18:35:01.544 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 0, 2023-07-14T18:35:01.544 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 0, 2023-07-14T18:35:01.544 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 2, 2023-07-14T18:35:01.545 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 1, 2023-07-14T18:35:01.545 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.000881595, 2023-07-14T18:35:01.545 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.000000000, 2023-07-14T18:35:01.545 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.000712325, 2023-07-14T18:35:01.545 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T18:35:01.545 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:35:01.545 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T18:35:01.545 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T18:35:01.546 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T18:35:01.546 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:35:01.546 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:35:01.546 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T18:35:01.546 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T18:35:01.546 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T18:35:01.546 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:35:01.546 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-14T18:35:01.547 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 0 2023-07-14T18:35:01.547 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:35:01.547 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2023-07-14T18:35:01.547 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 0, 2023-07-14T18:35:01.547 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2023-07-14T18:35:01.547 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:35:01.547 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2023-07-14T18:35:01.547 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2023-07-14T18:35:01.548 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2023-07-14T18:35:01.548 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T18:35:01.548 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2023-07-14T18:35:01.548 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T18:35:01.548 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2023-07-14T18:35:01.548 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T18:35:01.548 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T18:35:01.549 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T18:35:01.549 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T18:35:01.549 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T18:35:01.549 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2023-07-14T18:35:01.549 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T18:35:01.549 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T18:35:01.549 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2023-07-14T18:35:01.549 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2023-07-14T18:35:01.550 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2023-07-14T18:35:01.550 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2023-07-14T18:35:01.550 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T18:35:01.550 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T18:35:01.550 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T18:35:01.550 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T18:35:01.550 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T18:35:01.550 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2023-07-14T18:35:01.551 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2023-07-14T18:35:01.551 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2023-07-14T18:35:01.551 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T18:35:01.551 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T18:35:01.551 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2023-07-14T18:35:01.551 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2023-07-14T18:35:01.551 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2023-07-14T18:35:01.551 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2023-07-14T18:35:01.552 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T18:35:01.552 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2023-07-14T18:35:01.552 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T18:35:01.552 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2023-07-14T18:35:01.552 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 148704, 2023-07-14T18:35:01.552 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 29, 2023-07-14T18:35:01.552 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2023-07-14T18:35:01.553 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2023-07-14T18:35:01.553 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2023-07-14T18:35:01.553 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2023-07-14T18:35:01.553 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2023-07-14T18:35:01.553 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2023-07-14T18:35:01.553 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2023-07-14T18:35:01.553 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2023-07-14T18:35:01.553 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 0, 2023-07-14T18:35:01.554 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 0, 2023-07-14T18:35:01.554 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T18:35:01.554 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2023-07-14T18:35:01.554 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2023-07-14T18:35:01.554 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2023-07-14T18:35:01.554 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 928, 2023-07-14T18:35:01.554 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 2, 2023-07-14T18:35:01.554 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2023-07-14T18:35:01.555 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2023-07-14T18:35:01.555 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2023-07-14T18:35:01.555 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2023-07-14T18:35:01.555 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T18:35:01.555 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T18:35:01.555 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2023-07-14T18:35:01.555 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2023-07-14T18:35:01.555 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2023-07-14T18:35:01.556 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 344, 2023-07-14T18:35:01.556 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 458956, 2023-07-14T18:35:01.556 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:01.556 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 344, 2023-07-14T18:35:01.556 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2023-07-14T18:35:01.556 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2023-07-14T18:35:01.556 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 344, 2023-07-14T18:35:01.557 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 458956, 2023-07-14T18:35:01.557 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2023-07-14T18:35:01.557 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T18:35:01.557 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T18:35:01.557 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T18:35:01.557 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:35:01.557 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T18:35:01.557 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T18:35:02.066 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-07-14T18:35:02.066 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-07-14T18:35:02.066 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2023-07-14T18:35:02.066 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2023-07-14T18:35:02.067 DEBUG:tasks.check_counter:Getting stats from b 2023-07-14T18:35:02.067 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph daemon mds.b perf dump 2023-07-14T18:35:02.840 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:02 smithi171 ceph-mon[129241]: pgmap v898: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-07-14T18:35:02.862 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:02 smithi133 ceph-mon[126384]: pgmap v898: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-07-14T18:35:03.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:02 smithi161 ceph-mon[128993]: pgmap v898: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-07-14T18:35:04.317 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T18:35:04.840 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:04 smithi171 ceph-mon[129241]: pgmap v899: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 KiB/s wr, 9 op/s 2023-07-14T18:35:04.881 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:04 smithi133 ceph-mon[126384]: pgmap v899: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 KiB/s wr, 9 op/s 2023-07-14T18:35:05.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:04 smithi161 ceph-mon[128993]: pgmap v899: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 KiB/s wr, 9 op/s 2023-07-14T18:35:06.881 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:06 smithi133 ceph-mon[126384]: pgmap v900: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 27 KiB/s wr, 9 op/s 2023-07-14T18:35:07.066 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:06 smithi161 ceph-mon[128993]: pgmap v900: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 27 KiB/s wr, 9 op/s 2023-07-14T18:35:07.090 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:06 smithi171 ceph-mon[129241]: pgmap v900: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 27 KiB/s wr, 9 op/s 2023-07-14T18:35:08.492 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T18:35:08.492 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T18:35:08.493 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 10923, 2023-07-14T18:35:08.493 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 11947, 2023-07-14T18:35:08.493 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 3148985, 2023-07-14T18:35:08.493 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 13698875, 2023-07-14T18:35:08.493 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 83, 2023-07-14T18:35:08.493 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 5, 2023-07-14T18:35:08.493 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.752631142, 2023-07-14T18:35:08.493 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.222559167, 2023-07-14T18:35:08.494 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.427101577, 2023-07-14T18:35:08.494 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.201526815, 2023-07-14T18:35:08.494 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:35:08.494 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 11945, 2023-07-14T18:35:08.494 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 1.732596111, 2023-07-14T18:35:08.494 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000145047 2023-07-14T18:35:08.494 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.495 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:35:08.495 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 8162, 2023-07-14T18:35:08.495 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.002485149, 2023-07-14T18:35:08.495 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000304 2023-07-14T18:35:08.495 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.495 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 56320, 2023-07-14T18:35:08.495 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 29328 2023-07-14T18:35:08.495 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.496 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T18:35:08.496 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 6127050, 2023-07-14T18:35:08.496 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 6129006, 2023-07-14T18:35:08.496 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 968652601, 2023-07-14T18:35:08.496 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 999246234, 2023-07-14T18:35:08.496 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 8, 2023-07-14T18:35:08.496 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 6, 2023-07-14T18:35:08.497 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 162.457442540, 2023-07-14T18:35:08.497 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 62.781630634, 2023-07-14T18:35:08.497 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 114.904311970, 2023-07-14T18:35:08.497 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.055764881, 2023-07-14T18:35:08.497 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:35:08.497 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 6129005, 2023-07-14T18:35:08.497 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 171.762714577, 2023-07-14T18:35:08.497 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000028024 2023-07-14T18:35:08.498 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.498 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:35:08.498 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 11848095, 2023-07-14T18:35:08.498 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 1.849410264, 2023-07-14T18:35:08.498 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000156 2023-07-14T18:35:08.498 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.498 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-07-14T18:35:08.498 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 532800 2023-07-14T18:35:08.499 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.499 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T18:35:08.499 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 15465, 2023-07-14T18:35:08.499 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 11842, 2023-07-14T18:35:08.499 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 4879178, 2023-07-14T18:35:08.499 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 5707483, 2023-07-14T18:35:08.499 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 13, 2023-07-14T18:35:08.500 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 5, 2023-07-14T18:35:08.500 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.677202745, 2023-07-14T18:35:08.500 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.240697320, 2023-07-14T18:35:08.500 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.490690676, 2023-07-14T18:35:08.500 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.057522044, 2023-07-14T18:35:08.500 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:35:08.500 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 11841, 2023-07-14T18:35:08.501 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 3.129060596, 2023-07-14T18:35:08.501 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000264256 2023-07-14T18:35:08.501 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.501 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:35:08.501 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 23276, 2023-07-14T18:35:08.501 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.003308642, 2023-07-14T18:35:08.501 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000142 2023-07-14T18:35:08.502 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.502 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 645168, 2023-07-14T18:35:08.502 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 202880 2023-07-14T18:35:08.502 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.502 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2023-07-14T18:35:08.502 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2023-07-14T18:35:08.502 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2023-07-14T18:35:08.502 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.503 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2023-07-14T18:35:08.503 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2023-07-14T18:35:08.503 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2023-07-14T18:35:08.503 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3986, 2023-07-14T18:35:08.503 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 2.567905957, 2023-07-14T18:35:08.503 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000644231 2023-07-14T18:35:08.503 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:08.503 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.504 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2023-07-14T18:35:08.504 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2023-07-14T18:35:08.504 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2023-07-14T18:35:08.504 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 539, 2023-07-14T18:35:08.504 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.023977666, 2023-07-14T18:35:08.504 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000044485 2023-07-14T18:35:08.504 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:08.504 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.505 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2023-07-14T18:35:08.505 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 3599, 2023-07-14T18:35:08.505 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 2568, 2023-07-14T18:35:08.505 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2023-07-14T18:35:08.505 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2568, 2023-07-14T18:35:08.505 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 51.040085386, 2023-07-14T18:35:08.505 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.019875422 2023-07-14T18:35:08.505 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.506 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2023-07-14T18:35:08.506 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 1031, 2023-07-14T18:35:08.506 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2023-07-14T18:35:08.506 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2023-07-14T18:35:08.506 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 39, 2023-07-14T18:35:08.506 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 3, 2023-07-14T18:35:08.506 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2023-07-14T18:35:08.506 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 519, 2023-07-14T18:35:08.507 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 0, 2023-07-14T18:35:08.507 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 0, 2023-07-14T18:35:08.507 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 519, 2023-07-14T18:35:08.507 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 519, 2023-07-14T18:35:08.507 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 805, 2023-07-14T18:35:08.507 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 465, 2023-07-14T18:35:08.507 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 465, 2023-07-14T18:35:08.507 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 22, 2023-07-14T18:35:08.508 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 4768, 2023-07-14T18:35:08.508 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 2973, 2023-07-14T18:35:08.508 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 1017, 2023-07-14T18:35:08.508 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 34, 2023-07-14T18:35:08.508 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2023-07-14T18:35:08.508 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 1, 2023-07-14T18:35:08.508 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 87, 2023-07-14T18:35:08.508 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 10775, 2023-07-14T18:35:08.509 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 6, 2023-07-14T18:35:08.509 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 30, 2023-07-14T18:35:08.509 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 774, 2023-07-14T18:35:08.509 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 10, 2023-07-14T18:35:08.509 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 213, 2023-07-14T18:35:08.509 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2023-07-14T18:35:08.509 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 548, 2023-07-14T18:35:08.510 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 1, 2023-07-14T18:35:08.510 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 1047, 2023-07-14T18:35:08.510 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 9033468741, 2023-07-14T18:35:08.510 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2023-07-14T18:35:08.510 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 548, 2023-07-14T18:35:08.510 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2023-07-14T18:35:08.510 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2023-07-14T18:35:08.510 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2023-07-14T18:35:08.511 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 36, 2023-07-14T18:35:08.511 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 2, 2023-07-14T18:35:08.511 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 34, 2023-07-14T18:35:08.511 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 512, 2023-07-14T18:35:08.511 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 11, 2023-07-14T18:35:08.511 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 225, 2023-07-14T18:35:08.511 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 1602, 2023-07-14T18:35:08.511 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2023-07-14T18:35:08.512 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-14T18:35:08.512 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-14T18:35:08.512 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 3010, 2023-07-14T18:35:08.512 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 2553, 2023-07-14T18:35:08.512 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 67, 2023-07-14T18:35:08.512 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 376 2023-07-14T18:35:08.512 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.512 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2023-07-14T18:35:08.513 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 103, 2023-07-14T18:35:08.513 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2023-07-14T18:35:08.513 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2023-07-14T18:35:08.513 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 35, 2023-07-14T18:35:08.513 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 419, 2023-07-14T18:35:08.513 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2023-07-14T18:35:08.513 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2023-07-14T18:35:08.513 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2023-07-14T18:35:08.514 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 64, 2023-07-14T18:35:08.514 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 59, 2023-07-14T18:35:08.514 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 20, 2023-07-14T18:35:08.514 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2023-07-14T18:35:08.514 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2023-07-14T18:35:08.514 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2023-07-14T18:35:08.514 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2023-07-14T18:35:08.514 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2023-07-14T18:35:08.515 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2023-07-14T18:35:08.515 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 2, 2023-07-14T18:35:08.515 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 30, 2023-07-14T18:35:08.515 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2023-07-14T18:35:08.515 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 3, 2023-07-14T18:35:08.515 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2023-07-14T18:35:08.515 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2023-07-14T18:35:08.515 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.516 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2023-07-14T18:35:08.516 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 5495, 2023-07-14T18:35:08.516 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2023-07-14T18:35:08.516 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2023-07-14T18:35:08.516 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 5495, 2023-07-14T18:35:08.516 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2023-07-14T18:35:08.516 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2023-07-14T18:35:08.516 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 7, 2023-07-14T18:35:08.517 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2023-07-14T18:35:08.517 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2023-07-14T18:35:08.517 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 7, 2023-07-14T18:35:08.517 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2023-07-14T18:35:08.517 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2023-07-14T18:35:08.517 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2023-07-14T18:35:08.517 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 19248195, 2023-07-14T18:35:08.517 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 0, 2023-07-14T18:35:08.518 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2023-07-14T18:35:08.518 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2135, 2023-07-14T18:35:08.518 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 246.220080115, 2023-07-14T18:35:08.518 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.115325564 2023-07-14T18:35:08.518 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.518 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 0 2023-07-14T18:35:08.518 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.518 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2023-07-14T18:35:08.519 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 503, 2023-07-14T18:35:08.519 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 1213, 2023-07-14T18:35:08.519 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 710, 2023-07-14T18:35:08.519 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 63, 2023-07-14T18:35:08.519 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 88, 2023-07-14T18:35:08.519 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 25, 2023-07-14T18:35:08.519 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 519, 2023-07-14T18:35:08.519 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 1678, 2023-07-14T18:35:08.520 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 1159, 2023-07-14T18:35:08.520 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 465, 2023-07-14T18:35:08.520 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 1776, 2023-07-14T18:35:08.520 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 1311, 2023-07-14T18:35:08.520 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 65488, 2023-07-14T18:35:08.520 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 207152 2023-07-14T18:35:08.520 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.520 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2023-07-14T18:35:08.521 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 4098, 2023-07-14T18:35:08.521 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 173, 2023-07-14T18:35:08.521 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 3599, 2023-07-14T18:35:08.521 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 1014, 2023-07-14T18:35:08.521 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 415, 2023-07-14T18:35:08.521 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2023-07-14T18:35:08.521 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1044, 2023-07-14T18:35:08.521 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 47.412454839, 2023-07-14T18:35:08.522 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.045414228 2023-07-14T18:35:08.522 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.522 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2023-07-14T18:35:08.522 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 119, 2023-07-14T18:35:08.522 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.407029380, 2023-07-14T18:35:08.522 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.003420414 2023-07-14T18:35:08.522 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.523 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2023-07-14T18:35:08.523 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:08.523 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:08.523 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:08.523 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.523 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2023-07-14T18:35:08.523 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 5, 2023-07-14T18:35:08.523 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.009826189, 2023-07-14T18:35:08.524 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001965237 2023-07-14T18:35:08.524 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.524 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2023-07-14T18:35:08.524 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 620, 2023-07-14T18:35:08.524 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.785662554, 2023-07-14T18:35:08.524 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001267197 2023-07-14T18:35:08.524 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.524 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2023-07-14T18:35:08.525 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:08.525 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:08.525 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:08.525 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.525 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2023-07-14T18:35:08.525 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:08.525 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:08.525 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:08.526 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.526 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2023-07-14T18:35:08.526 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:08.526 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:08.526 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:08.526 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.526 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2023-07-14T18:35:08.526 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:08.527 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:08.527 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:08.527 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.527 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2023-07-14T18:35:08.527 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:08.527 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:08.527 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:08.527 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.528 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2023-07-14T18:35:08.528 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:08.528 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:08.528 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:08.528 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.528 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2023-07-14T18:35:08.528 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 35, 2023-07-14T18:35:08.528 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.268810842, 2023-07-14T18:35:08.529 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.007680309 2023-07-14T18:35:08.529 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.529 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2023-07-14T18:35:08.529 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:08.529 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:08.529 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:08.529 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.529 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2023-07-14T18:35:08.530 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:08.530 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:08.530 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:08.530 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.530 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2023-07-14T18:35:08.530 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 560, 2023-07-14T18:35:08.530 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.880759296, 2023-07-14T18:35:08.531 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001572784 2023-07-14T18:35:08.531 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.531 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2023-07-14T18:35:08.531 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 80, 2023-07-14T18:35:08.531 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.151794019, 2023-07-14T18:35:08.531 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001897425 2023-07-14T18:35:08.531 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.531 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2023-07-14T18:35:08.531 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 64, 2023-07-14T18:35:08.532 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 1.029439418, 2023-07-14T18:35:08.532 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.016084990 2023-07-14T18:35:08.532 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.532 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2023-07-14T18:35:08.532 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:08.532 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:08.532 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:08.533 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.533 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2023-07-14T18:35:08.533 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:08.533 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:08.533 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:08.533 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.533 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2023-07-14T18:35:08.533 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:08.534 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:08.534 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:08.534 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.534 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2023-07-14T18:35:08.534 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:08.534 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:08.534 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:08.534 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.535 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2023-07-14T18:35:08.535 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:08.535 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:08.535 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:08.535 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.535 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2023-07-14T18:35:08.535 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2, 2023-07-14T18:35:08.535 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.001538289, 2023-07-14T18:35:08.536 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000769144 2023-07-14T18:35:08.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.536 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2023-07-14T18:35:08.536 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:08.536 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:08.536 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:08.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.536 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2023-07-14T18:35:08.537 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:08.537 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:08.537 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:08.537 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.537 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2023-07-14T18:35:08.537 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:08.537 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:08.537 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:08.538 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.538 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2023-07-14T18:35:08.538 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 7, 2023-07-14T18:35:08.538 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.019196110, 2023-07-14T18:35:08.538 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002742301 2023-07-14T18:35:08.538 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.538 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2023-07-14T18:35:08.538 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:08.539 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:08.539 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:08.539 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.539 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2023-07-14T18:35:08.539 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 32, 2023-07-14T18:35:08.539 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.073574450, 2023-07-14T18:35:08.539 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002299201 2023-07-14T18:35:08.539 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.540 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2023-07-14T18:35:08.540 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2023-07-14T18:35:08.540 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2023-07-14T18:35:08.540 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:08.540 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:08.540 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:08.540 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:08.541 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.541 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2023-07-14T18:35:08.541 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2023-07-14T18:35:08.541 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 3, 2023-07-14T18:35:08.541 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 2, 2023-07-14T18:35:08.541 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 1, 2023-07-14T18:35:08.541 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2023-07-14T18:35:08.541 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 102, 2023-07-14T18:35:08.542 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 102, 2023-07-14T18:35:08.542 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 1121 2023-07-14T18:35:08.542 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.542 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2023-07-14T18:35:08.542 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2023-07-14T18:35:08.542 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2023-07-14T18:35:08.542 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T18:35:08.542 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2023-07-14T18:35:08.543 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T18:35:08.543 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2023-07-14T18:35:08.543 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T18:35:08.543 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T18:35:08.543 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T18:35:08.543 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T18:35:08.543 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T18:35:08.543 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2023-07-14T18:35:08.544 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T18:35:08.544 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T18:35:08.544 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2023-07-14T18:35:08.544 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2023-07-14T18:35:08.544 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2023-07-14T18:35:08.544 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2023-07-14T18:35:08.544 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T18:35:08.544 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T18:35:08.545 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T18:35:08.545 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T18:35:08.545 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T18:35:08.545 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2023-07-14T18:35:08.545 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2023-07-14T18:35:08.545 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2023-07-14T18:35:08.545 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T18:35:08.545 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T18:35:08.546 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2023-07-14T18:35:08.546 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2023-07-14T18:35:08.546 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2023-07-14T18:35:08.546 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2023-07-14T18:35:08.546 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T18:35:08.546 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2023-07-14T18:35:08.546 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T18:35:08.546 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2023-07-14T18:35:08.547 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 326066, 2023-07-14T18:35:08.547 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 592, 2023-07-14T18:35:08.547 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 352, 2023-07-14T18:35:08.547 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 4, 2023-07-14T18:35:08.547 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2023-07-14T18:35:08.547 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2023-07-14T18:35:08.547 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2023-07-14T18:35:08.547 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2023-07-14T18:35:08.548 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2023-07-14T18:35:08.548 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2023-07-14T18:35:08.548 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 8792, 2023-07-14T18:35:08.548 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 168, 2023-07-14T18:35:08.548 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T18:35:08.548 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2023-07-14T18:35:08.548 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2023-07-14T18:35:08.548 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2023-07-14T18:35:08.549 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 1809483, 2023-07-14T18:35:08.549 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 21198, 2023-07-14T18:35:08.549 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2023-07-14T18:35:08.549 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2023-07-14T18:35:08.549 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2023-07-14T18:35:08.549 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2023-07-14T18:35:08.549 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.549 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2023-07-14T18:35:08.550 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2023-07-14T18:35:08.550 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2023-07-14T18:35:08.550 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 3939, 2023-07-14T18:35:08.550 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 15688654, 2023-07-14T18:35:08.550 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 37, 2023-07-14T18:35:08.550 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 3939, 2023-07-14T18:35:08.550 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2023-07-14T18:35:08.550 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3939, 2023-07-14T18:35:08.551 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 279.002508653, 2023-07-14T18:35:08.551 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.070830796 2023-07-14T18:35:08.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.551 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2023-07-14T18:35:08.551 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2023-07-14T18:35:08.551 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3939, 2023-07-14T18:35:08.551 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 4142 2023-07-14T18:35:08.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.552 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 3939, 2023-07-14T18:35:08.552 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 595, 2023-07-14T18:35:08.552 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 3344, 2023-07-14T18:35:08.552 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2023-07-14T18:35:08.552 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2023-07-14T18:35:08.552 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 0, 2023-07-14T18:35:08.552 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 11, 2023-07-14T18:35:08.553 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 45, 2023-07-14T18:35:08.553 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 2178, 2023-07-14T18:35:08.553 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 124, 2023-07-14T18:35:08.553 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2023-07-14T18:35:08.553 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2023-07-14T18:35:08.553 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2023-07-14T18:35:08.553 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 3, 2023-07-14T18:35:08.553 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 871, 2023-07-14T18:35:08.554 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2023-07-14T18:35:08.554 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2023-07-14T18:35:08.554 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2023-07-14T18:35:08.554 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 550, 2023-07-14T18:35:08.554 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 22, 2023-07-14T18:35:08.554 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2023-07-14T18:35:08.554 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2023-07-14T18:35:08.554 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2023-07-14T18:35:08.555 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2023-07-14T18:35:08.555 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2023-07-14T18:35:08.555 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2023-07-14T18:35:08.555 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2023-07-14T18:35:08.555 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2023-07-14T18:35:08.555 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2023-07-14T18:35:08.555 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2023-07-14T18:35:08.555 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2023-07-14T18:35:08.556 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2023-07-14T18:35:08.556 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2023-07-14T18:35:08.556 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2023-07-14T18:35:08.556 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2023-07-14T18:35:08.556 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2023-07-14T18:35:08.556 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2023-07-14T18:35:08.556 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2023-07-14T18:35:08.556 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2023-07-14T18:35:08.557 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2023-07-14T18:35:08.557 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2023-07-14T18:35:08.557 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2023-07-14T18:35:08.557 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2023-07-14T18:35:08.557 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2023-07-14T18:35:08.557 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2023-07-14T18:35:08.557 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2023-07-14T18:35:08.557 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 92, 2023-07-14T18:35:08.558 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2023-07-14T18:35:08.558 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 9, 2023-07-14T18:35:08.558 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 12, 2023-07-14T18:35:08.558 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 12, 2023-07-14T18:35:08.558 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2023-07-14T18:35:08.558 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2023-07-14T18:35:08.558 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 273, 2023-07-14T18:35:08.558 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 0, 2023-07-14T18:35:08.559 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 65 2023-07-14T18:35:08.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.559 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2023-07-14T18:35:08.559 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 1, 2023-07-14T18:35:08.559 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2023-07-14T18:35:08.559 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2023-07-14T18:35:08.559 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2023-07-14T18:35:08.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.560 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2023-07-14T18:35:08.560 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2023-07-14T18:35:08.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.560 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2023-07-14T18:35:08.560 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2023-07-14T18:35:08.560 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 240, 2023-07-14T18:35:08.560 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2023-07-14T18:35:08.561 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 8, 2023-07-14T18:35:08.561 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 59, 2023-07-14T18:35:08.561 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2023-07-14T18:35:08.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.561 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T18:35:08.561 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T18:35:08.561 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2023-07-14T18:35:08.561 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T18:35:08.562 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 6153436, 2023-07-14T18:35:08.562 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 422986905, 2023-07-14T18:35:08.562 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:08.562 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 6153436, 2023-07-14T18:35:08.562 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2023-07-14T18:35:08.562 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2023-07-14T18:35:08.562 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 6153436, 2023-07-14T18:35:08.562 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 422986905, 2023-07-14T18:35:08.563 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T18:35:08.563 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:08.563 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:08.563 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:08.563 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:08.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.563 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2023-07-14T18:35:08.563 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T18:35:08.564 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2023-07-14T18:35:08.564 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T18:35:08.564 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2023-07-14T18:35:08.564 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2023-07-14T18:35:08.564 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:08.564 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2023-07-14T18:35:08.564 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 3939, 2023-07-14T18:35:08.564 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 15695085, 2023-07-14T18:35:08.565 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 3939, 2023-07-14T18:35:08.565 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 15695085, 2023-07-14T18:35:08.565 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T18:35:08.565 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:08.565 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:08.565 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:08.565 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:08.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.566 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2023-07-14T18:35:08.566 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T18:35:08.566 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2023-07-14T18:35:08.566 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T18:35:08.566 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2023-07-14T18:35:08.566 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2023-07-14T18:35:08.566 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:08.566 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2023-07-14T18:35:08.567 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 3939, 2023-07-14T18:35:08.567 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 3939, 2023-07-14T18:35:08.567 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 3939, 2023-07-14T18:35:08.567 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 3939, 2023-07-14T18:35:08.567 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T18:35:08.567 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:08.567 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:08.567 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:08.568 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:08.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.568 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2023-07-14T18:35:08.568 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T18:35:08.568 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2023-07-14T18:35:08.568 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T18:35:08.568 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 59, 2023-07-14T18:35:08.569 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 6431, 2023-07-14T18:35:08.569 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:08.569 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 59, 2023-07-14T18:35:08.569 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2023-07-14T18:35:08.569 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2023-07-14T18:35:08.569 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 43, 2023-07-14T18:35:08.569 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 6431, 2023-07-14T18:35:08.569 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T18:35:08.570 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:08.570 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:08.570 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:08.570 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:08.570 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:08.570 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x5589a8c4e0a0": { 2023-07-14T18:35:08.570 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T18:35:08.570 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2023-07-14T18:35:08.571 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T18:35:08.571 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 5495, 2023-07-14T18:35:08.571 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 15053891, 2023-07-14T18:35:08.571 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:08.571 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 5495, 2023-07-14T18:35:08.571 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2023-07-14T18:35:08.571 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2023-07-14T18:35:08.571 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2135, 2023-07-14T18:35:08.572 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 15053891, 2023-07-14T18:35:08.572 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T18:35:08.572 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:08.572 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:08.572 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:08.572 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:08.572 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:08.572 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T18:35:08.631 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:08 smithi133 ceph-mon[126384]: pgmap v901: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-07-14T18:35:08.795 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:08 smithi161 ceph-mon[128993]: pgmap v901: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-07-14T18:35:08.796 INFO:tasks.check_counter:Daemon mds.b mds.exported=30 2023-07-14T18:35:08.796 INFO:tasks.check_counter:Daemon mds.b mds.imported=10 2023-07-14T18:35:08.797 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=103 2023-07-14T18:35:08.797 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2023-07-14T18:35:08.797 DEBUG:tasks.check_counter:Getting stats from e 2023-07-14T18:35:08.797 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph daemon mds.e perf dump 2023-07-14T18:35:09.090 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:08 smithi171 ceph-mon[129241]: pgmap v901: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-07-14T18:35:09.388 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T18:35:10.594 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T18:35:10.595 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T18:35:10.595 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 1401, 2023-07-14T18:35:10.595 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 1401, 2023-07-14T18:35:10.595 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 4558909, 2023-07-14T18:35:10.595 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 434979, 2023-07-14T18:35:10.595 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 7, 2023-07-14T18:35:10.595 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 2, 2023-07-14T18:35:10.596 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.122960225, 2023-07-14T18:35:10.596 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.037428226, 2023-07-14T18:35:10.596 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.032949534, 2023-07-14T18:35:10.596 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.042164827, 2023-07-14T18:35:10.596 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:35:10.596 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1400, 2023-07-14T18:35:10.596 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.479723504, 2023-07-14T18:35:10.596 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000342659 2023-07-14T18:35:10.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.597 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:35:10.597 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.597 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.597 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.597 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-07-14T18:35:10.598 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 448 2023-07-14T18:35:10.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.598 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T18:35:10.598 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 1067, 2023-07-14T18:35:10.598 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 1325, 2023-07-14T18:35:10.598 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 672075, 2023-07-14T18:35:10.598 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 925012, 2023-07-14T18:35:10.598 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 7, 2023-07-14T18:35:10.599 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2023-07-14T18:35:10.599 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.106301556, 2023-07-14T18:35:10.599 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.029279735, 2023-07-14T18:35:10.599 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.024255179, 2023-07-14T18:35:10.599 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.044032745, 2023-07-14T18:35:10.599 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:35:10.599 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1323, 2023-07-14T18:35:10.600 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.350553685, 2023-07-14T18:35:10.600 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000264968 2023-07-14T18:35:10.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.600 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:35:10.600 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.600 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.600 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.601 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-14T18:35:10.601 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 593792 2023-07-14T18:35:10.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.601 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T18:35:10.601 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 1581, 2023-07-14T18:35:10.601 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 1554, 2023-07-14T18:35:10.601 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 1016192, 2023-07-14T18:35:10.601 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 571345, 2023-07-14T18:35:10.602 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2023-07-14T18:35:10.602 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2023-07-14T18:35:10.602 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.162376368, 2023-07-14T18:35:10.602 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.047234855, 2023-07-14T18:35:10.602 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.049094119, 2023-07-14T18:35:10.603 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.055533615, 2023-07-14T18:35:10.603 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:35:10.603 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1553, 2023-07-14T18:35:10.603 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.579647430, 2023-07-14T18:35:10.603 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000373243 2023-07-14T18:35:10.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.603 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:35:10.603 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.604 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.604 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.604 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 616400, 2023-07-14T18:35:10.604 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 195968 2023-07-14T18:35:10.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.604 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2023-07-14T18:35:10.605 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2023-07-14T18:35:10.605 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2023-07-14T18:35:10.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.605 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2023-07-14T18:35:10.605 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2023-07-14T18:35:10.605 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2023-07-14T18:35:10.605 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 6255, 2023-07-14T18:35:10.605 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.165176426, 2023-07-14T18:35:10.606 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000026407 2023-07-14T18:35:10.606 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:10.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.606 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2023-07-14T18:35:10.606 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2023-07-14T18:35:10.606 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2023-07-14T18:35:10.606 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.606 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.607 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.607 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:10.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.607 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2023-07-14T18:35:10.607 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 0, 2023-07-14T18:35:10.607 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 0, 2023-07-14T18:35:10.607 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2023-07-14T18:35:10.608 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.608 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.608 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.608 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2023-07-14T18:35:10.608 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 0, 2023-07-14T18:35:10.608 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2023-07-14T18:35:10.609 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2023-07-14T18:35:10.609 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 0, 2023-07-14T18:35:10.609 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2023-07-14T18:35:10.609 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2023-07-14T18:35:10.609 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 825, 2023-07-14T18:35:10.609 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 0, 2023-07-14T18:35:10.609 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 0, 2023-07-14T18:35:10.609 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 825, 2023-07-14T18:35:10.610 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 825, 2023-07-14T18:35:10.610 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 2, 2023-07-14T18:35:10.610 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 0, 2023-07-14T18:35:10.610 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 0, 2023-07-14T18:35:10.610 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 21, 2023-07-14T18:35:10.610 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 0, 2023-07-14T18:35:10.610 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 0, 2023-07-14T18:35:10.610 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2023-07-14T18:35:10.611 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 0, 2023-07-14T18:35:10.611 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2023-07-14T18:35:10.611 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2023-07-14T18:35:10.611 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2023-07-14T18:35:10.611 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2023-07-14T18:35:10.611 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2023-07-14T18:35:10.611 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2023-07-14T18:35:10.611 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2023-07-14T18:35:10.612 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2023-07-14T18:35:10.612 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2023-07-14T18:35:10.612 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2023-07-14T18:35:10.612 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2023-07-14T18:35:10.612 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2023-07-14T18:35:10.612 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 0, 2023-07-14T18:35:10.612 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 0, 2023-07-14T18:35:10.612 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2023-07-14T18:35:10.613 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2023-07-14T18:35:10.613 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2023-07-14T18:35:10.613 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2023-07-14T18:35:10.613 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2023-07-14T18:35:10.613 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2023-07-14T18:35:10.613 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2023-07-14T18:35:10.613 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2023-07-14T18:35:10.613 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2023-07-14T18:35:10.614 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2023-07-14T18:35:10.614 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 0, 2023-07-14T18:35:10.614 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 0, 2023-07-14T18:35:10.614 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2023-07-14T18:35:10.614 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-14T18:35:10.614 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-14T18:35:10.614 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 0, 2023-07-14T18:35:10.614 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2023-07-14T18:35:10.615 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 0, 2023-07-14T18:35:10.615 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 0 2023-07-14T18:35:10.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.615 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2023-07-14T18:35:10.615 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2023-07-14T18:35:10.615 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2023-07-14T18:35:10.615 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2023-07-14T18:35:10.615 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 0, 2023-07-14T18:35:10.616 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 0, 2023-07-14T18:35:10.616 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2023-07-14T18:35:10.616 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2023-07-14T18:35:10.616 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2023-07-14T18:35:10.616 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 0, 2023-07-14T18:35:10.616 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 0, 2023-07-14T18:35:10.616 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2023-07-14T18:35:10.617 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2023-07-14T18:35:10.617 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2023-07-14T18:35:10.617 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2023-07-14T18:35:10.617 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2023-07-14T18:35:10.617 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2023-07-14T18:35:10.617 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2023-07-14T18:35:10.617 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2023-07-14T18:35:10.617 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2023-07-14T18:35:10.618 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2023-07-14T18:35:10.618 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2023-07-14T18:35:10.618 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2023-07-14T18:35:10.618 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2023-07-14T18:35:10.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.618 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2023-07-14T18:35:10.618 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 0, 2023-07-14T18:35:10.618 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2023-07-14T18:35:10.619 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2023-07-14T18:35:10.619 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 0, 2023-07-14T18:35:10.619 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2023-07-14T18:35:10.619 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2023-07-14T18:35:10.619 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 0, 2023-07-14T18:35:10.619 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2023-07-14T18:35:10.619 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2023-07-14T18:35:10.619 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 4, 2023-07-14T18:35:10.620 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2023-07-14T18:35:10.620 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2023-07-14T18:35:10.620 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2023-07-14T18:35:10.620 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 0, 2023-07-14T18:35:10.620 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 8796023, 2023-07-14T18:35:10.620 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2023-07-14T18:35:10.620 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.621 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.621 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.621 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 2949 2023-07-14T18:35:10.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.621 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2023-07-14T18:35:10.621 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 568, 2023-07-14T18:35:10.621 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 902, 2023-07-14T18:35:10.622 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 334, 2023-07-14T18:35:10.622 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 36, 2023-07-14T18:35:10.622 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 40, 2023-07-14T18:35:10.622 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 4, 2023-07-14T18:35:10.622 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 825, 2023-07-14T18:35:10.622 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 938, 2023-07-14T18:35:10.622 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 113, 2023-07-14T18:35:10.622 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 0, 2023-07-14T18:35:10.623 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 0, 2023-07-14T18:35:10.623 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 0, 2023-07-14T18:35:10.623 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 50632, 2023-07-14T18:35:10.623 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 207152 2023-07-14T18:35:10.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.623 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2023-07-14T18:35:10.623 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 0, 2023-07-14T18:35:10.623 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 0, 2023-07-14T18:35:10.624 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 0, 2023-07-14T18:35:10.624 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 0, 2023-07-14T18:35:10.624 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 0, 2023-07-14T18:35:10.624 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2023-07-14T18:35:10.624 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.624 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.624 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.625 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2023-07-14T18:35:10.625 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.625 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.625 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.625 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2023-07-14T18:35:10.625 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.625 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.626 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.626 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2023-07-14T18:35:10.626 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.626 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.626 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.626 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2023-07-14T18:35:10.627 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.627 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.627 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.627 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2023-07-14T18:35:10.627 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.627 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.627 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.628 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2023-07-14T18:35:10.628 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.628 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.628 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.628 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2023-07-14T18:35:10.629 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.629 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.629 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.629 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2023-07-14T18:35:10.629 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.629 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.629 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.630 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2023-07-14T18:35:10.630 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.630 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.630 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.630 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2023-07-14T18:35:10.630 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.631 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.631 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.631 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2023-07-14T18:35:10.631 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.631 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.631 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.632 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2023-07-14T18:35:10.632 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.632 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.632 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.632 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2023-07-14T18:35:10.632 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.632 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.633 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.633 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2023-07-14T18:35:10.633 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.633 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.633 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.633 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2023-07-14T18:35:10.634 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.634 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.634 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.634 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2023-07-14T18:35:10.634 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.634 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.634 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.635 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2023-07-14T18:35:10.635 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.635 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.635 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.635 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2023-07-14T18:35:10.636 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.636 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.636 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.636 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2023-07-14T18:35:10.636 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.636 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.636 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.637 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2023-07-14T18:35:10.637 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.637 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.637 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.637 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2023-07-14T18:35:10.637 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.638 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.638 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.638 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2023-07-14T18:35:10.638 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.638 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.638 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.639 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2023-07-14T18:35:10.639 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.639 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.639 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.639 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2023-07-14T18:35:10.639 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.640 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.640 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.640 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2023-07-14T18:35:10.640 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.640 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.640 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.641 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2023-07-14T18:35:10.641 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.641 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.641 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.641 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2023-07-14T18:35:10.641 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.641 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.642 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.642 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2023-07-14T18:35:10.642 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.642 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.642 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.642 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2023-07-14T18:35:10.643 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2023-07-14T18:35:10.643 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2023-07-14T18:35:10.643 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.643 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.643 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.643 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:10.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.643 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2023-07-14T18:35:10.644 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2023-07-14T18:35:10.644 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 1, 2023-07-14T18:35:10.644 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 0, 2023-07-14T18:35:10.644 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 1, 2023-07-14T18:35:10.644 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2023-07-14T18:35:10.644 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 0, 2023-07-14T18:35:10.644 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 0, 2023-07-14T18:35:10.645 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 3209 2023-07-14T18:35:10.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.645 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2023-07-14T18:35:10.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2023-07-14T18:35:10.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2023-07-14T18:35:10.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T18:35:10.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2023-07-14T18:35:10.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T18:35:10.646 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2023-07-14T18:35:10.646 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T18:35:10.646 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T18:35:10.646 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T18:35:10.646 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T18:35:10.646 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T18:35:10.646 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2023-07-14T18:35:10.646 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T18:35:10.647 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T18:35:10.647 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2023-07-14T18:35:10.647 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2023-07-14T18:35:10.647 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2023-07-14T18:35:10.647 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2023-07-14T18:35:10.647 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T18:35:10.647 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T18:35:10.647 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T18:35:10.648 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T18:35:10.648 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T18:35:10.648 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2023-07-14T18:35:10.648 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2023-07-14T18:35:10.648 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2023-07-14T18:35:10.648 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T18:35:10.648 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T18:35:10.648 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2023-07-14T18:35:10.649 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2023-07-14T18:35:10.649 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2023-07-14T18:35:10.649 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2023-07-14T18:35:10.649 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T18:35:10.649 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2023-07-14T18:35:10.649 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T18:35:10.649 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2023-07-14T18:35:10.649 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 219657, 2023-07-14T18:35:10.650 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 616, 2023-07-14T18:35:10.650 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2023-07-14T18:35:10.650 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2023-07-14T18:35:10.650 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2023-07-14T18:35:10.650 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2023-07-14T18:35:10.650 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2023-07-14T18:35:10.650 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2023-07-14T18:35:10.650 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2023-07-14T18:35:10.651 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2023-07-14T18:35:10.651 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 8792, 2023-07-14T18:35:10.651 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 168, 2023-07-14T18:35:10.651 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T18:35:10.651 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2023-07-14T18:35:10.651 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2023-07-14T18:35:10.651 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2023-07-14T18:35:10.651 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 1913888, 2023-07-14T18:35:10.652 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 17114, 2023-07-14T18:35:10.652 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2023-07-14T18:35:10.652 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2023-07-14T18:35:10.652 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2023-07-14T18:35:10.652 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2023-07-14T18:35:10.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.652 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2023-07-14T18:35:10.653 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2023-07-14T18:35:10.653 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2023-07-14T18:35:10.653 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 3669, 2023-07-14T18:35:10.653 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 21, 2023-07-14T18:35:10.653 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 3, 2023-07-14T18:35:10.653 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 3669, 2023-07-14T18:35:10.653 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2023-07-14T18:35:10.653 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3669, 2023-07-14T18:35:10.654 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 78.529465306, 2023-07-14T18:35:10.654 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.021403506 2023-07-14T18:35:10.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.654 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2023-07-14T18:35:10.654 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2023-07-14T18:35:10.654 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3669, 2023-07-14T18:35:10.654 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 3670 2023-07-14T18:35:10.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.655 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 3669, 2023-07-14T18:35:10.655 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 3669, 2023-07-14T18:35:10.655 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 0, 2023-07-14T18:35:10.655 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2023-07-14T18:35:10.655 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2023-07-14T18:35:10.655 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 2181, 2023-07-14T18:35:10.655 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 0, 2023-07-14T18:35:10.655 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 1486, 2023-07-14T18:35:10.656 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 0, 2023-07-14T18:35:10.656 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 0, 2023-07-14T18:35:10.656 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2023-07-14T18:35:10.656 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2023-07-14T18:35:10.656 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2023-07-14T18:35:10.656 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2023-07-14T18:35:10.656 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 0, 2023-07-14T18:35:10.656 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2023-07-14T18:35:10.657 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2023-07-14T18:35:10.657 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2023-07-14T18:35:10.657 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 1, 2023-07-14T18:35:10.657 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 0, 2023-07-14T18:35:10.657 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2023-07-14T18:35:10.657 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2023-07-14T18:35:10.657 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2023-07-14T18:35:10.657 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2023-07-14T18:35:10.658 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2023-07-14T18:35:10.658 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2023-07-14T18:35:10.658 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2023-07-14T18:35:10.658 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2023-07-14T18:35:10.658 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2023-07-14T18:35:10.658 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2023-07-14T18:35:10.658 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2023-07-14T18:35:10.658 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2023-07-14T18:35:10.659 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2023-07-14T18:35:10.659 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2023-07-14T18:35:10.659 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2023-07-14T18:35:10.659 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2023-07-14T18:35:10.659 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2023-07-14T18:35:10.659 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2023-07-14T18:35:10.659 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2023-07-14T18:35:10.659 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2023-07-14T18:35:10.660 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2023-07-14T18:35:10.660 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2023-07-14T18:35:10.660 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2023-07-14T18:35:10.660 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2023-07-14T18:35:10.660 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2023-07-14T18:35:10.660 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2023-07-14T18:35:10.660 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 92, 2023-07-14T18:35:10.660 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2023-07-14T18:35:10.661 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 2, 2023-07-14T18:35:10.661 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 6, 2023-07-14T18:35:10.661 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 8, 2023-07-14T18:35:10.661 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 2, 2023-07-14T18:35:10.661 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2023-07-14T18:35:10.661 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 0, 2023-07-14T18:35:10.661 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 2, 2023-07-14T18:35:10.661 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 0 2023-07-14T18:35:10.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.662 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2023-07-14T18:35:10.662 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 0, 2023-07-14T18:35:10.662 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2023-07-14T18:35:10.662 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2023-07-14T18:35:10.662 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2023-07-14T18:35:10.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.662 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2023-07-14T18:35:10.663 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2023-07-14T18:35:10.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.663 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2023-07-14T18:35:10.663 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2023-07-14T18:35:10.663 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 0, 2023-07-14T18:35:10.663 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2023-07-14T18:35:10.663 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 0, 2023-07-14T18:35:10.663 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 0, 2023-07-14T18:35:10.664 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2023-07-14T18:35:10.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.664 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T18:35:10.664 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T18:35:10.664 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2023-07-14T18:35:10.664 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T18:35:10.664 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 4045, 2023-07-14T18:35:10.665 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 5860911, 2023-07-14T18:35:10.665 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:10.665 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 4045, 2023-07-14T18:35:10.665 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2023-07-14T18:35:10.665 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2023-07-14T18:35:10.665 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 4045, 2023-07-14T18:35:10.665 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 5860911, 2023-07-14T18:35:10.665 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T18:35:10.666 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.666 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.666 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.666 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:10.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.666 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2023-07-14T18:35:10.666 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T18:35:10.666 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2023-07-14T18:35:10.667 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T18:35:10.667 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2023-07-14T18:35:10.667 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2023-07-14T18:35:10.667 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:10.667 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2023-07-14T18:35:10.667 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 3669, 2023-07-14T18:35:10.667 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 4602884, 2023-07-14T18:35:10.667 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 3669, 2023-07-14T18:35:10.668 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 4602884, 2023-07-14T18:35:10.668 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T18:35:10.668 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.668 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.668 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.668 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:10.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.668 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2023-07-14T18:35:10.669 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T18:35:10.669 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2023-07-14T18:35:10.669 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T18:35:10.669 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2023-07-14T18:35:10.669 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2023-07-14T18:35:10.669 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:10.669 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2023-07-14T18:35:10.670 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 3669, 2023-07-14T18:35:10.670 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 3669, 2023-07-14T18:35:10.670 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 3669, 2023-07-14T18:35:10.670 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 3669, 2023-07-14T18:35:10.670 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T18:35:10.670 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.670 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.670 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.671 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:10.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.671 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2023-07-14T18:35:10.671 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T18:35:10.671 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2023-07-14T18:35:10.671 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T18:35:10.671 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2023-07-14T18:35:10.671 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2023-07-14T18:35:10.672 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:10.672 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2023-07-14T18:35:10.672 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2023-07-14T18:35:10.672 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2023-07-14T18:35:10.672 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2023-07-14T18:35:10.672 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2023-07-14T18:35:10.672 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T18:35:10.672 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.673 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.673 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.673 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:10.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:10.673 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x55b5647a0460": { 2023-07-14T18:35:10.673 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T18:35:10.673 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2023-07-14T18:35:10.673 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T18:35:10.674 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2023-07-14T18:35:10.674 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2023-07-14T18:35:10.674 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:10.674 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2023-07-14T18:35:10.674 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2023-07-14T18:35:10.674 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2023-07-14T18:35:10.674 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2023-07-14T18:35:10.674 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2023-07-14T18:35:10.675 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T18:35:10.675 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:10.675 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:10.675 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:10.675 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:10.675 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:10.675 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T18:35:10.751 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:10 smithi161 ceph-mon[128993]: pgmap v902: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 19 KiB/s wr, 8 op/s 2023-07-14T18:35:10.880 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:10 smithi133 ceph-mon[126384]: pgmap v902: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 19 KiB/s wr, 8 op/s 2023-07-14T18:35:10.945 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-07-14T18:35:10.945 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-07-14T18:35:10.945 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2023-07-14T18:35:10.945 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2023-07-14T18:35:10.946 DEBUG:tasks.check_counter:Getting stats from h 2023-07-14T18:35:10.946 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph daemon mds.h perf dump 2023-07-14T18:35:11.090 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:10 smithi171 ceph-mon[129241]: pgmap v902: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 19 KiB/s wr, 8 op/s 2023-07-14T18:35:11.536 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T18:35:12.663 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T18:35:12.664 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T18:35:12.664 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 3321, 2023-07-14T18:35:12.664 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 3321, 2023-07-14T18:35:12.664 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 11413182, 2023-07-14T18:35:12.664 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 1032042, 2023-07-14T18:35:12.665 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 6, 2023-07-14T18:35:12.665 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 2, 2023-07-14T18:35:12.665 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.263934241, 2023-07-14T18:35:12.665 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.066283303, 2023-07-14T18:35:12.665 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.074663550, 2023-07-14T18:35:12.665 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.105368375, 2023-07-14T18:35:12.665 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:35:12.665 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3320, 2023-07-14T18:35:12.666 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.789346641, 2023-07-14T18:35:12.666 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000237755 2023-07-14T18:35:12.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.666 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:35:12.666 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.666 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.667 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.667 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-07-14T18:35:12.667 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 448 2023-07-14T18:35:12.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.667 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T18:35:12.667 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 321, 2023-07-14T18:35:12.667 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 578, 2023-07-14T18:35:12.668 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 78858, 2023-07-14T18:35:12.668 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 694439, 2023-07-14T18:35:12.668 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 6, 2023-07-14T18:35:12.668 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2023-07-14T18:35:12.668 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.042845536, 2023-07-14T18:35:12.669 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.015210993, 2023-07-14T18:35:12.669 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.009264882, 2023-07-14T18:35:12.669 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.014381049, 2023-07-14T18:35:12.669 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:35:12.669 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 576, 2023-07-14T18:35:12.669 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.164294847, 2023-07-14T18:35:12.669 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000285234 2023-07-14T18:35:12.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.670 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:35:12.670 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.670 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.670 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.670 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-14T18:35:12.670 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 595168 2023-07-14T18:35:12.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.671 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T18:35:12.671 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 377, 2023-07-14T18:35:12.671 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 347, 2023-07-14T18:35:12.671 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 625980, 2023-07-14T18:35:12.671 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 195687, 2023-07-14T18:35:12.671 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2023-07-14T18:35:12.671 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 2, 2023-07-14T18:35:12.671 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.048305313, 2023-07-14T18:35:12.672 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.017245233, 2023-07-14T18:35:12.672 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.020080562, 2023-07-14T18:35:12.672 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.009947474, 2023-07-14T18:35:12.672 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:35:12.672 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 346, 2023-07-14T18:35:12.672 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.113841807, 2023-07-14T18:35:12.672 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000329022 2023-07-14T18:35:12.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.673 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:35:12.673 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.673 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.673 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.673 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 625712, 2023-07-14T18:35:12.673 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 195376 2023-07-14T18:35:12.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.674 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2023-07-14T18:35:12.674 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2023-07-14T18:35:12.674 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2023-07-14T18:35:12.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.674 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2023-07-14T18:35:12.674 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2023-07-14T18:35:12.674 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2023-07-14T18:35:12.675 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 6161, 2023-07-14T18:35:12.675 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.150771092, 2023-07-14T18:35:12.675 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000024471 2023-07-14T18:35:12.675 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:12.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.675 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2023-07-14T18:35:12.675 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2023-07-14T18:35:12.675 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2023-07-14T18:35:12.676 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.676 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.676 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.676 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:12.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.676 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2023-07-14T18:35:12.676 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 0, 2023-07-14T18:35:12.676 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 0, 2023-07-14T18:35:12.677 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2023-07-14T18:35:12.677 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.677 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.677 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.677 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2023-07-14T18:35:12.677 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 0, 2023-07-14T18:35:12.677 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2023-07-14T18:35:12.678 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2023-07-14T18:35:12.678 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 0, 2023-07-14T18:35:12.678 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2023-07-14T18:35:12.678 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2023-07-14T18:35:12.678 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 1538, 2023-07-14T18:35:12.678 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 0, 2023-07-14T18:35:12.678 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 0, 2023-07-14T18:35:12.678 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 1538, 2023-07-14T18:35:12.679 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 1538, 2023-07-14T18:35:12.679 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 32, 2023-07-14T18:35:12.679 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 0, 2023-07-14T18:35:12.679 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 0, 2023-07-14T18:35:12.679 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 3, 2023-07-14T18:35:12.679 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 0, 2023-07-14T18:35:12.679 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 0, 2023-07-14T18:35:12.679 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2023-07-14T18:35:12.680 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 0, 2023-07-14T18:35:12.680 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2023-07-14T18:35:12.680 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2023-07-14T18:35:12.680 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2023-07-14T18:35:12.680 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2023-07-14T18:35:12.680 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2023-07-14T18:35:12.680 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2023-07-14T18:35:12.681 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2023-07-14T18:35:12.681 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2023-07-14T18:35:12.681 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2023-07-14T18:35:12.681 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2023-07-14T18:35:12.681 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2023-07-14T18:35:12.682 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2023-07-14T18:35:12.682 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 0, 2023-07-14T18:35:12.682 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 0, 2023-07-14T18:35:12.682 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2023-07-14T18:35:12.682 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2023-07-14T18:35:12.682 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2023-07-14T18:35:12.682 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2023-07-14T18:35:12.682 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2023-07-14T18:35:12.683 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2023-07-14T18:35:12.683 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2023-07-14T18:35:12.683 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2023-07-14T18:35:12.683 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2023-07-14T18:35:12.683 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2023-07-14T18:35:12.683 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 0, 2023-07-14T18:35:12.683 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 0, 2023-07-14T18:35:12.683 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2023-07-14T18:35:12.684 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-14T18:35:12.684 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-14T18:35:12.684 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 0, 2023-07-14T18:35:12.684 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2023-07-14T18:35:12.684 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 0, 2023-07-14T18:35:12.684 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 0 2023-07-14T18:35:12.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.685 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2023-07-14T18:35:12.685 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2023-07-14T18:35:12.685 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2023-07-14T18:35:12.685 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2023-07-14T18:35:12.685 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 0, 2023-07-14T18:35:12.685 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 0, 2023-07-14T18:35:12.685 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2023-07-14T18:35:12.685 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2023-07-14T18:35:12.686 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2023-07-14T18:35:12.686 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 0, 2023-07-14T18:35:12.686 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 0, 2023-07-14T18:35:12.686 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2023-07-14T18:35:12.686 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2023-07-14T18:35:12.686 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2023-07-14T18:35:12.686 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2023-07-14T18:35:12.686 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2023-07-14T18:35:12.687 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2023-07-14T18:35:12.687 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2023-07-14T18:35:12.687 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2023-07-14T18:35:12.687 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2023-07-14T18:35:12.687 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2023-07-14T18:35:12.687 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2023-07-14T18:35:12.687 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2023-07-14T18:35:12.687 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2023-07-14T18:35:12.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.688 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2023-07-14T18:35:12.688 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 0, 2023-07-14T18:35:12.688 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2023-07-14T18:35:12.688 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2023-07-14T18:35:12.688 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 0, 2023-07-14T18:35:12.688 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2023-07-14T18:35:12.689 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2023-07-14T18:35:12.689 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 0, 2023-07-14T18:35:12.689 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2023-07-14T18:35:12.689 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2023-07-14T18:35:12.689 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 8, 2023-07-14T18:35:12.689 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2023-07-14T18:35:12.689 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2023-07-14T18:35:12.689 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2023-07-14T18:35:12.690 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 0, 2023-07-14T18:35:12.690 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 14666917, 2023-07-14T18:35:12.690 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2023-07-14T18:35:12.690 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.690 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.690 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.690 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.690 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 5734 2023-07-14T18:35:12.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.691 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2023-07-14T18:35:12.691 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 82, 2023-07-14T18:35:12.691 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 281, 2023-07-14T18:35:12.691 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 199, 2023-07-14T18:35:12.691 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 18, 2023-07-14T18:35:12.691 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 31, 2023-07-14T18:35:12.691 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 13, 2023-07-14T18:35:12.692 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 1538, 2023-07-14T18:35:12.692 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 1747, 2023-07-14T18:35:12.692 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 209, 2023-07-14T18:35:12.692 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 0, 2023-07-14T18:35:12.692 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 0, 2023-07-14T18:35:12.692 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 0, 2023-07-14T18:35:12.692 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 50508, 2023-07-14T18:35:12.692 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 207152 2023-07-14T18:35:12.693 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.693 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2023-07-14T18:35:12.693 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 0, 2023-07-14T18:35:12.693 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 0, 2023-07-14T18:35:12.693 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 0, 2023-07-14T18:35:12.693 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 0, 2023-07-14T18:35:12.693 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 0, 2023-07-14T18:35:12.693 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2023-07-14T18:35:12.694 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.694 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.694 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.694 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.694 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2023-07-14T18:35:12.694 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.694 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.694 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.695 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2023-07-14T18:35:12.695 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.695 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.695 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.695 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2023-07-14T18:35:12.695 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.696 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.696 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.696 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2023-07-14T18:35:12.696 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.696 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.696 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.697 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2023-07-14T18:35:12.697 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.697 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.697 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.697 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2023-07-14T18:35:12.697 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.697 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.698 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.698 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2023-07-14T18:35:12.698 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.698 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.698 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.699 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2023-07-14T18:35:12.699 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.699 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.699 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.699 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2023-07-14T18:35:12.699 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.699 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.700 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.700 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2023-07-14T18:35:12.700 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.700 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.700 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.700 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2023-07-14T18:35:12.701 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.701 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.701 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.701 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.701 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2023-07-14T18:35:12.701 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.701 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.701 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.702 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2023-07-14T18:35:12.702 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.702 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.702 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.702 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2023-07-14T18:35:12.702 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.703 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.703 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.703 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2023-07-14T18:35:12.703 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.703 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.703 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.704 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2023-07-14T18:35:12.704 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.704 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.704 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.704 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2023-07-14T18:35:12.705 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.705 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.705 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.705 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.705 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2023-07-14T18:35:12.705 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.705 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.705 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.706 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.706 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2023-07-14T18:35:12.706 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.706 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.706 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.706 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.706 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2023-07-14T18:35:12.706 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.707 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.707 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.707 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.707 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2023-07-14T18:35:12.707 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.707 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.707 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.707 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.708 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2023-07-14T18:35:12.708 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.708 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.708 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.708 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2023-07-14T18:35:12.708 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.708 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.709 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.709 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2023-07-14T18:35:12.709 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.709 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.709 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.709 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2023-07-14T18:35:12.710 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.710 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.710 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.710 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2023-07-14T18:35:12.710 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.710 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.710 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.711 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.711 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2023-07-14T18:35:12.711 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.711 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.711 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.711 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.711 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2023-07-14T18:35:12.711 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.712 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.712 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.712 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.712 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2023-07-14T18:35:12.712 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2023-07-14T18:35:12.712 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2023-07-14T18:35:12.712 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.712 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.713 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.713 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:12.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.713 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2023-07-14T18:35:12.713 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2023-07-14T18:35:12.713 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 2, 2023-07-14T18:35:12.713 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 1, 2023-07-14T18:35:12.713 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 1, 2023-07-14T18:35:12.714 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2023-07-14T18:35:12.714 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 0, 2023-07-14T18:35:12.714 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 0, 2023-07-14T18:35:12.714 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 3211 2023-07-14T18:35:12.714 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.714 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2023-07-14T18:35:12.714 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2023-07-14T18:35:12.714 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2023-07-14T18:35:12.715 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T18:35:12.715 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2023-07-14T18:35:12.715 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T18:35:12.715 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2023-07-14T18:35:12.715 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T18:35:12.715 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T18:35:12.715 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T18:35:12.715 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T18:35:12.716 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T18:35:12.716 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2023-07-14T18:35:12.716 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T18:35:12.716 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T18:35:12.716 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2023-07-14T18:35:12.716 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2023-07-14T18:35:12.716 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2023-07-14T18:35:12.716 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2023-07-14T18:35:12.717 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T18:35:12.717 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T18:35:12.717 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T18:35:12.717 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T18:35:12.717 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T18:35:12.717 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2023-07-14T18:35:12.717 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2023-07-14T18:35:12.717 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2023-07-14T18:35:12.718 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T18:35:12.718 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T18:35:12.718 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2023-07-14T18:35:12.718 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2023-07-14T18:35:12.718 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2023-07-14T18:35:12.718 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2023-07-14T18:35:12.718 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T18:35:12.718 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2023-07-14T18:35:12.719 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T18:35:12.719 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2023-07-14T18:35:12.719 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 192663, 2023-07-14T18:35:12.719 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 120, 2023-07-14T18:35:12.719 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2023-07-14T18:35:12.719 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2023-07-14T18:35:12.719 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2023-07-14T18:35:12.719 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2023-07-14T18:35:12.720 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2023-07-14T18:35:12.720 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2023-07-14T18:35:12.720 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2023-07-14T18:35:12.720 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2023-07-14T18:35:12.720 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 8792, 2023-07-14T18:35:12.720 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 168, 2023-07-14T18:35:12.720 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T18:35:12.721 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2023-07-14T18:35:12.721 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2023-07-14T18:35:12.721 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2023-07-14T18:35:12.721 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 1326708, 2023-07-14T18:35:12.721 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 40622, 2023-07-14T18:35:12.721 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2023-07-14T18:35:12.721 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2023-07-14T18:35:12.721 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2023-07-14T18:35:12.722 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2023-07-14T18:35:12.722 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.722 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2023-07-14T18:35:12.722 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2023-07-14T18:35:12.722 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2023-07-14T18:35:12.722 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 3637, 2023-07-14T18:35:12.722 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 21, 2023-07-14T18:35:12.722 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 3, 2023-07-14T18:35:12.723 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 3637, 2023-07-14T18:35:12.723 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2023-07-14T18:35:12.723 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3637, 2023-07-14T18:35:12.723 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 121.057639193, 2023-07-14T18:35:12.723 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.033285025 2023-07-14T18:35:12.723 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.723 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2023-07-14T18:35:12.723 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2023-07-14T18:35:12.724 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3637, 2023-07-14T18:35:12.724 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 3638 2023-07-14T18:35:12.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.724 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 3637, 2023-07-14T18:35:12.724 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 3637, 2023-07-14T18:35:12.724 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 0, 2023-07-14T18:35:12.724 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2023-07-14T18:35:12.724 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2023-07-14T18:35:12.725 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 2174, 2023-07-14T18:35:12.725 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 0, 2023-07-14T18:35:12.725 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 1461, 2023-07-14T18:35:12.725 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 0, 2023-07-14T18:35:12.725 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 0, 2023-07-14T18:35:12.725 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2023-07-14T18:35:12.725 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2023-07-14T18:35:12.725 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2023-07-14T18:35:12.726 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2023-07-14T18:35:12.726 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 0, 2023-07-14T18:35:12.726 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2023-07-14T18:35:12.726 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2023-07-14T18:35:12.726 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2023-07-14T18:35:12.726 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 1, 2023-07-14T18:35:12.726 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 0, 2023-07-14T18:35:12.726 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2023-07-14T18:35:12.727 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2023-07-14T18:35:12.727 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2023-07-14T18:35:12.727 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2023-07-14T18:35:12.727 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2023-07-14T18:35:12.727 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2023-07-14T18:35:12.727 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2023-07-14T18:35:12.727 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2023-07-14T18:35:12.727 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2023-07-14T18:35:12.728 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2023-07-14T18:35:12.728 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2023-07-14T18:35:12.728 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2023-07-14T18:35:12.728 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2023-07-14T18:35:12.728 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2023-07-14T18:35:12.728 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2023-07-14T18:35:12.728 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2023-07-14T18:35:12.728 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2023-07-14T18:35:12.729 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2023-07-14T18:35:12.729 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2023-07-14T18:35:12.729 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2023-07-14T18:35:12.729 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2023-07-14T18:35:12.729 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2023-07-14T18:35:12.729 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2023-07-14T18:35:12.729 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2023-07-14T18:35:12.729 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2023-07-14T18:35:12.730 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2023-07-14T18:35:12.730 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 92, 2023-07-14T18:35:12.730 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2023-07-14T18:35:12.730 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 2, 2023-07-14T18:35:12.730 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 5, 2023-07-14T18:35:12.730 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 6, 2023-07-14T18:35:12.730 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 1, 2023-07-14T18:35:12.730 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2023-07-14T18:35:12.731 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 0, 2023-07-14T18:35:12.731 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 2, 2023-07-14T18:35:12.731 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 0 2023-07-14T18:35:12.731 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.731 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2023-07-14T18:35:12.731 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 0, 2023-07-14T18:35:12.731 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2023-07-14T18:35:12.731 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2023-07-14T18:35:12.732 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2023-07-14T18:35:12.732 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.732 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2023-07-14T18:35:12.732 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2023-07-14T18:35:12.732 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.732 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2023-07-14T18:35:12.732 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2023-07-14T18:35:12.732 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 0, 2023-07-14T18:35:12.733 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2023-07-14T18:35:12.733 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 0, 2023-07-14T18:35:12.733 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 0, 2023-07-14T18:35:12.733 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2023-07-14T18:35:12.733 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.733 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T18:35:12.733 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T18:35:12.733 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2023-07-14T18:35:12.734 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T18:35:12.734 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 4015, 2023-07-14T18:35:12.734 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 11734368, 2023-07-14T18:35:12.734 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:12.734 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 4015, 2023-07-14T18:35:12.734 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2023-07-14T18:35:12.734 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2023-07-14T18:35:12.734 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 4015, 2023-07-14T18:35:12.735 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 11734368, 2023-07-14T18:35:12.735 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T18:35:12.735 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.735 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.735 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.735 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:12.735 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.735 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2023-07-14T18:35:12.736 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T18:35:12.736 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2023-07-14T18:35:12.736 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T18:35:12.736 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2023-07-14T18:35:12.736 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2023-07-14T18:35:12.736 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:12.736 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2023-07-14T18:35:12.736 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 3637, 2023-07-14T18:35:12.737 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 10473192, 2023-07-14T18:35:12.737 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 3637, 2023-07-14T18:35:12.737 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 10473192, 2023-07-14T18:35:12.737 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T18:35:12.737 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.737 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.737 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.737 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:12.738 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.738 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2023-07-14T18:35:12.738 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T18:35:12.738 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2023-07-14T18:35:12.738 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T18:35:12.738 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2023-07-14T18:35:12.738 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2023-07-14T18:35:12.738 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:12.739 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2023-07-14T18:35:12.739 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 3637, 2023-07-14T18:35:12.739 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 3637, 2023-07-14T18:35:12.739 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 3637, 2023-07-14T18:35:12.739 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 3637, 2023-07-14T18:35:12.739 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T18:35:12.739 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.739 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.740 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.740 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:12.740 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.740 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2023-07-14T18:35:12.740 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T18:35:12.740 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2023-07-14T18:35:12.740 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T18:35:12.741 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2023-07-14T18:35:12.741 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2023-07-14T18:35:12.741 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:12.741 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2023-07-14T18:35:12.741 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2023-07-14T18:35:12.741 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2023-07-14T18:35:12.741 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2023-07-14T18:35:12.741 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2023-07-14T18:35:12.742 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T18:35:12.742 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.742 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.742 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.742 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:12.742 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:12.742 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x5634a60ba460": { 2023-07-14T18:35:12.742 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T18:35:12.743 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2023-07-14T18:35:12.743 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T18:35:12.743 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2023-07-14T18:35:12.743 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2023-07-14T18:35:12.743 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:12.743 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2023-07-14T18:35:12.743 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2023-07-14T18:35:12.743 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2023-07-14T18:35:12.744 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2023-07-14T18:35:12.744 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2023-07-14T18:35:12.744 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T18:35:12.744 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:12.744 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:12.744 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:12.744 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:12.744 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:12.745 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T18:35:12.745 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:12 smithi161 ceph-mon[128993]: pgmap v903: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 KiB/s wr, 10 op/s 2023-07-14T18:35:12.881 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:12 smithi133 ceph-mon[126384]: pgmap v903: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 KiB/s wr, 10 op/s 2023-07-14T18:35:13.090 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:12 smithi171 ceph-mon[129241]: pgmap v903: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 KiB/s wr, 10 op/s 2023-07-14T18:35:14.401 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2023-07-14T18:35:14.401 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2023-07-14T18:35:14.401 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2023-07-14T18:35:14.402 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2023-07-14T18:35:14.402 DEBUG:tasks.check_counter:Getting stats from k 2023-07-14T18:35:14.402 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph daemon mds.k perf dump 2023-07-14T18:35:14.824 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:14 smithi161 ceph-mon[128993]: pgmap v904: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 8 op/s 2023-07-14T18:35:14.880 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:14 smithi133 ceph-mon[126384]: pgmap v904: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 8 op/s 2023-07-14T18:35:14.992 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T18:35:15.090 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:14 smithi171 ceph-mon[129241]: pgmap v904: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 8 op/s 2023-07-14T18:35:16.226 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T18:35:16.226 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T18:35:16.226 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 346, 2023-07-14T18:35:16.227 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 343, 2023-07-14T18:35:16.227 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 508400, 2023-07-14T18:35:16.227 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 196560, 2023-07-14T18:35:16.227 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2023-07-14T18:35:16.227 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 0, 2023-07-14T18:35:16.227 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.047473330, 2023-07-14T18:35:16.227 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.016998019, 2023-07-14T18:35:16.228 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.018394803, 2023-07-14T18:35:16.228 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.009694093, 2023-07-14T18:35:16.228 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:35:16.228 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 342, 2023-07-14T18:35:16.228 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.075019947, 2023-07-14T18:35:16.228 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000219356 2023-07-14T18:35:16.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:16.229 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:35:16.229 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:16.229 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:16.229 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:16.229 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:16.229 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 508400, 2023-07-14T18:35:16.229 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 196560 2023-07-14T18:35:16.229 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:16.230 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T18:35:16.230 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 2, 2023-07-14T18:35:16.230 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 263, 2023-07-14T18:35:16.230 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 320, 2023-07-14T18:35:16.230 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 198032, 2023-07-14T18:35:16.230 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2023-07-14T18:35:16.230 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 2, 2023-07-14T18:35:16.231 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.011847657, 2023-07-14T18:35:16.231 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.008591842, 2023-07-14T18:35:16.231 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.001236420, 2023-07-14T18:35:16.231 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T18:35:16.231 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:35:16.231 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 261, 2023-07-14T18:35:16.231 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.106347584, 2023-07-14T18:35:16.231 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000407462 2023-07-14T18:35:16.232 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:16.232 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:35:16.232 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:16.232 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:16.232 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:16.232 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:16.232 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-14T18:35:16.232 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 198032 2023-07-14T18:35:16.233 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:16.233 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T18:35:16.233 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 4, 2023-07-14T18:35:16.233 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 4, 2023-07-14T18:35:16.233 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 2992, 2023-07-14T18:35:16.233 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 448, 2023-07-14T18:35:16.233 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 2, 2023-07-14T18:35:16.234 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 1, 2023-07-14T18:35:16.234 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.000793371, 2023-07-14T18:35:16.234 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.000043419, 2023-07-14T18:35:16.234 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.000752579, 2023-07-14T18:35:16.234 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T18:35:16.234 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:35:16.234 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3, 2023-07-14T18:35:16.234 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000728200, 2023-07-14T18:35:16.235 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000242733 2023-07-14T18:35:16.235 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:16.235 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:35:16.235 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:16.235 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:16.235 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:16.235 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:16.235 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-07-14T18:35:16.236 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 448 2023-07-14T18:35:16.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:16.236 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2023-07-14T18:35:16.236 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 0, 2023-07-14T18:35:16.236 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2023-07-14T18:35:16.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:16.236 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2023-07-14T18:35:16.237 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2023-07-14T18:35:16.237 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2023-07-14T18:35:16.237 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T18:35:16.237 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2023-07-14T18:35:16.237 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T18:35:16.237 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2023-07-14T18:35:16.237 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T18:35:16.237 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T18:35:16.238 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T18:35:16.238 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T18:35:16.238 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T18:35:16.238 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2023-07-14T18:35:16.238 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T18:35:16.238 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T18:35:16.238 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2023-07-14T18:35:16.239 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2023-07-14T18:35:16.239 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2023-07-14T18:35:16.239 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2023-07-14T18:35:16.239 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T18:35:16.239 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T18:35:16.239 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T18:35:16.239 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T18:35:16.239 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T18:35:16.240 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2023-07-14T18:35:16.240 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2023-07-14T18:35:16.240 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2023-07-14T18:35:16.240 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T18:35:16.240 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T18:35:16.240 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2023-07-14T18:35:16.240 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2023-07-14T18:35:16.240 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2023-07-14T18:35:16.241 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2023-07-14T18:35:16.241 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T18:35:16.241 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2023-07-14T18:35:16.241 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T18:35:16.241 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2023-07-14T18:35:16.241 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 148761, 2023-07-14T18:35:16.241 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 31, 2023-07-14T18:35:16.241 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2023-07-14T18:35:16.242 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2023-07-14T18:35:16.242 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2023-07-14T18:35:16.242 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2023-07-14T18:35:16.242 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2023-07-14T18:35:16.242 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2023-07-14T18:35:16.242 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2023-07-14T18:35:16.242 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2023-07-14T18:35:16.242 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 0, 2023-07-14T18:35:16.243 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 0, 2023-07-14T18:35:16.243 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T18:35:16.243 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2023-07-14T18:35:16.243 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2023-07-14T18:35:16.243 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2023-07-14T18:35:16.243 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 928, 2023-07-14T18:35:16.243 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 2, 2023-07-14T18:35:16.243 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2023-07-14T18:35:16.244 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2023-07-14T18:35:16.244 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2023-07-14T18:35:16.244 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2023-07-14T18:35:16.244 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T18:35:16.244 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T18:35:16.244 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T18:35:16.244 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2023-07-14T18:35:16.245 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T18:35:16.245 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 348, 2023-07-14T18:35:16.245 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 459476, 2023-07-14T18:35:16.245 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:16.245 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 348, 2023-07-14T18:35:16.245 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2023-07-14T18:35:16.245 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2023-07-14T18:35:16.245 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 348, 2023-07-14T18:35:16.246 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 459476, 2023-07-14T18:35:16.246 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T18:35:16.246 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T18:35:16.246 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T18:35:16.246 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T18:35:16.246 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:16.246 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T18:35:16.246 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T18:35:16.552 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2023-07-14T18:35:16.552 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2023-07-14T18:35:16.552 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2023-07-14T18:35:16.553 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2023-07-14T18:35:16.553 DEBUG:tasks.check_counter:Getting stats from c 2023-07-14T18:35:16.553 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph daemon mds.c perf dump 2023-07-14T18:35:16.824 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:16 smithi161 ceph-mon[128993]: pgmap v905: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 8 op/s 2023-07-14T18:35:16.880 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:16 smithi133 ceph-mon[126384]: pgmap v905: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 8 op/s 2023-07-14T18:35:17.090 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:16 smithi171 ceph-mon[129241]: pgmap v905: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 8 op/s 2023-07-14T18:35:17.402 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T18:35:18.840 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:18 smithi171 ceph-mon[129241]: pgmap v906: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.7 KiB/s wr, 10 op/s 2023-07-14T18:35:18.880 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:18 smithi133 ceph-mon[126384]: pgmap v906: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.7 KiB/s wr, 10 op/s 2023-07-14T18:35:19.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:18 smithi161 ceph-mon[128993]: pgmap v906: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.7 KiB/s wr, 10 op/s 2023-07-14T18:35:19.678 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2023-07-14T18:35:19.679 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T18:35:19.679 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 1729, 2023-07-14T18:35:19.679 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 1699, 2023-07-14T18:35:19.679 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 5243092, 2023-07-14T18:35:19.679 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 616574, 2023-07-14T18:35:19.679 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 5, 2023-07-14T18:35:19.679 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 1, 2023-07-14T18:35:19.680 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.167995144, 2023-07-14T18:35:19.680 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.047563232, 2023-07-14T18:35:19.680 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.050555883, 2023-07-14T18:35:19.680 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.057988465, 2023-07-14T18:35:19.680 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:35:19.680 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1698, 2023-07-14T18:35:19.680 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.443911435, 2023-07-14T18:35:19.681 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000261431 2023-07-14T18:35:19.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.681 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:35:19.681 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.681 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.681 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.681 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 626416, 2023-07-14T18:35:19.682 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 196720 2023-07-14T18:35:19.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.682 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T18:35:19.682 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 1445, 2023-07-14T18:35:19.682 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 1702, 2023-07-14T18:35:19.682 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 7053041, 2023-07-14T18:35:19.682 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 960402, 2023-07-14T18:35:19.682 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 5, 2023-07-14T18:35:19.683 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 4, 2023-07-14T18:35:19.683 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.134671141, 2023-07-14T18:35:19.683 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.038814394, 2023-07-14T18:35:19.683 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.036228839, 2023-07-14T18:35:19.683 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.049896766, 2023-07-14T18:35:19.683 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:35:19.683 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1701, 2023-07-14T18:35:19.684 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.549606255, 2023-07-14T18:35:19.684 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000323107 2023-07-14T18:35:19.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.684 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:35:19.684 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.684 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.684 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.685 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-07-14T18:35:19.685 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 512496 2023-07-14T18:35:19.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.685 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T18:35:19.685 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 621, 2023-07-14T18:35:19.685 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 623, 2023-07-14T18:35:19.685 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 4355779, 2023-07-14T18:35:19.685 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 193138, 2023-07-14T18:35:19.686 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 5, 2023-07-14T18:35:19.686 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 3, 2023-07-14T18:35:19.686 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.051761654, 2023-07-14T18:35:19.686 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.015806326, 2023-07-14T18:35:19.686 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.015099898, 2023-07-14T18:35:19.686 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.016781831, 2023-07-14T18:35:19.686 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:35:19.686 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 621, 2023-07-14T18:35:19.687 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.055191929, 2023-07-14T18:35:19.687 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000088875 2023-07-14T18:35:19.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.687 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:35:19.687 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.687 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.687 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.688 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-14T18:35:19.688 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 0 2023-07-14T18:35:19.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.688 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2023-07-14T18:35:19.688 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 1, 2023-07-14T18:35:19.688 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2023-07-14T18:35:19.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.689 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-MDSRank": { 2023-07-14T18:35:19.689 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2023-07-14T18:35:19.689 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2023-07-14T18:35:19.689 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 5793, 2023-07-14T18:35:19.689 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.144772420, 2023-07-14T18:35:19.689 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000024990 2023-07-14T18:35:19.689 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:19.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.690 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-PurgeQueue": { 2023-07-14T18:35:19.690 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2023-07-14T18:35:19.690 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2023-07-14T18:35:19.690 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.690 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.690 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.690 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:19.691 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.691 INFO:teuthology.orchestra.run.smithi171.stdout: "mds": { 2023-07-14T18:35:19.691 INFO:teuthology.orchestra.run.smithi171.stdout: "request": 0, 2023-07-14T18:35:19.691 INFO:teuthology.orchestra.run.smithi171.stdout: "reply": 0, 2023-07-14T18:35:19.691 INFO:teuthology.orchestra.run.smithi171.stdout: "reply_latency": { 2023-07-14T18:35:19.691 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.691 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.691 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.692 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.692 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_reply": 0, 2023-07-14T18:35:19.692 INFO:teuthology.orchestra.run.smithi171.stdout: "forward": 0, 2023-07-14T18:35:19.692 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_complete": 0, 2023-07-14T18:35:19.692 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_keys": 0, 2023-07-14T18:35:19.692 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_commit": 0, 2023-07-14T18:35:19.692 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_split": 0, 2023-07-14T18:35:19.692 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_merge": 0, 2023-07-14T18:35:19.693 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes": 558, 2023-07-14T18:35:19.693 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_top": 0, 2023-07-14T18:35:19.693 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_bottom": 0, 2023-07-14T18:35:19.693 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pin_tail": 558, 2023-07-14T18:35:19.693 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pinned": 558, 2023-07-14T18:35:19.693 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_expired": 32, 2023-07-14T18:35:19.693 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_with_caps": 0, 2023-07-14T18:35:19.693 INFO:teuthology.orchestra.run.smithi171.stdout: "caps": 0, 2023-07-14T18:35:19.694 INFO:teuthology.orchestra.run.smithi171.stdout: "subtrees": 22, 2023-07-14T18:35:19.694 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse": 0, 2023-07-14T18:35:19.694 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_hit": 0, 2023-07-14T18:35:19.694 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_forward": 0, 2023-07-14T18:35:19.694 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_discover": 0, 2023-07-14T18:35:19.694 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_dir_fetch": 0, 2023-07-14T18:35:19.694 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_remote_ino": 0, 2023-07-14T18:35:19.695 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_lock": 0, 2023-07-14T18:35:19.695 INFO:teuthology.orchestra.run.smithi171.stdout: "load_cent": 0, 2023-07-14T18:35:19.695 INFO:teuthology.orchestra.run.smithi171.stdout: "q": 0, 2023-07-14T18:35:19.695 INFO:teuthology.orchestra.run.smithi171.stdout: "exported": 0, 2023-07-14T18:35:19.695 INFO:teuthology.orchestra.run.smithi171.stdout: "exported_inodes": 0, 2023-07-14T18:35:19.695 INFO:teuthology.orchestra.run.smithi171.stdout: "imported": 0, 2023-07-14T18:35:19.695 INFO:teuthology.orchestra.run.smithi171.stdout: "imported_inodes": 0, 2023-07-14T18:35:19.695 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_dir_fetch": 0, 2023-07-14T18:35:19.696 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_backtrace_fetch": 0, 2023-07-14T18:35:19.696 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_peer_discover": 0, 2023-07-14T18:35:19.696 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rfiles": 1044, 2023-07-14T18:35:19.696 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rbytes": 9033542873, 2023-07-14T18:35:19.696 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rsnaps": 0, 2023-07-14T18:35:19.696 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_fetch": 0, 2023-07-14T18:35:19.696 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_set_tag": 0, 2023-07-14T18:35:19.696 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_repaired": 0, 2023-07-14T18:35:19.697 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_inotable_repaired": 0, 2023-07-14T18:35:19.697 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_inodes": 0, 2023-07-14T18:35:19.697 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_base_inodes": 0, 2023-07-14T18:35:19.697 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dirfrag_rstats": 0, 2023-07-14T18:35:19.697 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_file_inodes": 0, 2023-07-14T18:35:19.697 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_inode_file_caps": 0, 2023-07-14T18:35:19.697 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_revoke": 0, 2023-07-14T18:35:19.698 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_grant": 0, 2023-07-14T18:35:19.698 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_trunc": 0, 2023-07-14T18:35:19.698 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-14T18:35:19.698 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-14T18:35:19.698 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps": 0, 2023-07-14T18:35:19.698 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps_dirty": 0, 2023-07-14T18:35:19.698 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_cap_release": 0, 2023-07-14T18:35:19.698 INFO:teuthology.orchestra.run.smithi171.stdout: "process_request_cap_release": 0 2023-07-14T18:35:19.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.699 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache": { 2023-07-14T18:35:19.699 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update": 0, 2023-07-14T18:35:19.699 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update_receipt": 0, 2023-07-14T18:35:19.699 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_try_discover": 0, 2023-07-14T18:35:19.699 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_send_discover": 0, 2023-07-14T18:35:19.699 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_handle_discover": 0, 2023-07-14T18:35:19.700 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays": 0, 2023-07-14T18:35:19.700 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_delayed": 0, 2023-07-14T18:35:19.700 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_enqueuing": 0, 2023-07-14T18:35:19.700 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_created": 0, 2023-07-14T18:35:19.700 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_enqueued": 0, 2023-07-14T18:35:19.700 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_reintegrated": 0, 2023-07-14T18:35:19.700 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_migrated": 0, 2023-07-14T18:35:19.700 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_processing": 0, 2023-07-14T18:35:19.701 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_enqueued": 0, 2023-07-14T18:35:19.701 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_prioritized": 0, 2023-07-14T18:35:19.701 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_started": 0, 2023-07-14T18:35:19.701 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_completed": 0, 2023-07-14T18:35:19.701 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_enqueue_scrub": 0, 2023-07-14T18:35:19.701 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_exportdir": 0, 2023-07-14T18:35:19.701 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_flush": 0, 2023-07-14T18:35:19.701 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragmentdir": 0, 2023-07-14T18:35:19.702 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragstats": 0, 2023-07-14T18:35:19.702 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_inodestats": 0 2023-07-14T18:35:19.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.702 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log": { 2023-07-14T18:35:19.702 INFO:teuthology.orchestra.run.smithi171.stdout: "evadd": 0, 2023-07-14T18:35:19.702 INFO:teuthology.orchestra.run.smithi171.stdout: "evex": 0, 2023-07-14T18:35:19.702 INFO:teuthology.orchestra.run.smithi171.stdout: "evtrm": 0, 2023-07-14T18:35:19.703 INFO:teuthology.orchestra.run.smithi171.stdout: "ev": 0, 2023-07-14T18:35:19.703 INFO:teuthology.orchestra.run.smithi171.stdout: "evexg": 0, 2023-07-14T18:35:19.703 INFO:teuthology.orchestra.run.smithi171.stdout: "evexd": 0, 2023-07-14T18:35:19.703 INFO:teuthology.orchestra.run.smithi171.stdout: "segadd": 0, 2023-07-14T18:35:19.703 INFO:teuthology.orchestra.run.smithi171.stdout: "segex": 0, 2023-07-14T18:35:19.703 INFO:teuthology.orchestra.run.smithi171.stdout: "segtrm": 0, 2023-07-14T18:35:19.703 INFO:teuthology.orchestra.run.smithi171.stdout: "seg": 7, 2023-07-14T18:35:19.703 INFO:teuthology.orchestra.run.smithi171.stdout: "segexg": 0, 2023-07-14T18:35:19.704 INFO:teuthology.orchestra.run.smithi171.stdout: "segexd": 0, 2023-07-14T18:35:19.704 INFO:teuthology.orchestra.run.smithi171.stdout: "expos": 4194304, 2023-07-14T18:35:19.704 INFO:teuthology.orchestra.run.smithi171.stdout: "wrpos": 0, 2023-07-14T18:35:19.704 INFO:teuthology.orchestra.run.smithi171.stdout: "rdpos": 19263879, 2023-07-14T18:35:19.704 INFO:teuthology.orchestra.run.smithi171.stdout: "jlat": { 2023-07-14T18:35:19.704 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.704 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.704 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.705 INFO:teuthology.orchestra.run.smithi171.stdout: "replayed": 5502 2023-07-14T18:35:19.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.705 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_mem": { 2023-07-14T18:35:19.705 INFO:teuthology.orchestra.run.smithi171.stdout: "ino": 481, 2023-07-14T18:35:19.705 INFO:teuthology.orchestra.run.smithi171.stdout: "ino+": 1252, 2023-07-14T18:35:19.705 INFO:teuthology.orchestra.run.smithi171.stdout: "ino-": 771, 2023-07-14T18:35:19.705 INFO:teuthology.orchestra.run.smithi171.stdout: "dir": 59, 2023-07-14T18:35:19.706 INFO:teuthology.orchestra.run.smithi171.stdout: "dir+": 71, 2023-07-14T18:35:19.706 INFO:teuthology.orchestra.run.smithi171.stdout: "dir-": 12, 2023-07-14T18:35:19.706 INFO:teuthology.orchestra.run.smithi171.stdout: "dn": 558, 2023-07-14T18:35:19.706 INFO:teuthology.orchestra.run.smithi171.stdout: "dn+": 1333, 2023-07-14T18:35:19.706 INFO:teuthology.orchestra.run.smithi171.stdout: "dn-": 775, 2023-07-14T18:35:19.706 INFO:teuthology.orchestra.run.smithi171.stdout: "cap": 0, 2023-07-14T18:35:19.706 INFO:teuthology.orchestra.run.smithi171.stdout: "cap+": 0, 2023-07-14T18:35:19.706 INFO:teuthology.orchestra.run.smithi171.stdout: "cap-": 0, 2023-07-14T18:35:19.707 INFO:teuthology.orchestra.run.smithi171.stdout: "rss": 51564, 2023-07-14T18:35:19.707 INFO:teuthology.orchestra.run.smithi171.stdout: "heap": 207152 2023-07-14T18:35:19.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.707 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_server": { 2023-07-14T18:35:19.707 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_client_request": 0, 2023-07-14T18:35:19.707 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_server_request": 0, 2023-07-14T18:35:19.707 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_request": 0, 2023-07-14T18:35:19.708 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_session": 0, 2023-07-14T18:35:19.708 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_peer_request": 0, 2023-07-14T18:35:19.708 INFO:teuthology.orchestra.run.smithi171.stdout: "req_create_latency": { 2023-07-14T18:35:19.708 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.708 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.708 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.708 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getattr_latency": { 2023-07-14T18:35:19.709 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.709 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.709 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.709 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getfilelock_latency": { 2023-07-14T18:35:19.709 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.709 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.709 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.710 INFO:teuthology.orchestra.run.smithi171.stdout: "req_link_latency": { 2023-07-14T18:35:19.710 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.710 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.710 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.710 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookup_latency": { 2023-07-14T18:35:19.711 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.711 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.711 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.711 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookuphash_latency": { 2023-07-14T18:35:19.711 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.711 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.711 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.712 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupino_latency": { 2023-07-14T18:35:19.712 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.712 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.712 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.712 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupname_latency": { 2023-07-14T18:35:19.712 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.713 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.713 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.713 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupparent_latency": { 2023-07-14T18:35:19.713 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.713 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.713 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.714 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupsnap_latency": { 2023-07-14T18:35:19.714 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.714 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.714 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.714 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lssnap_latency": { 2023-07-14T18:35:19.714 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.715 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.715 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.715 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mkdir_latency": { 2023-07-14T18:35:19.715 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.715 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.715 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.716 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mknod_latency": { 2023-07-14T18:35:19.716 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.716 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.716 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.716 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mksnap_latency": { 2023-07-14T18:35:19.716 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.716 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.717 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.717 INFO:teuthology.orchestra.run.smithi171.stdout: "req_open_latency": { 2023-07-14T18:35:19.717 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.717 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.717 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.718 INFO:teuthology.orchestra.run.smithi171.stdout: "req_readdir_latency": { 2023-07-14T18:35:19.718 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.718 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.718 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.718 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rename_latency": { 2023-07-14T18:35:19.718 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.718 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.719 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.719 INFO:teuthology.orchestra.run.smithi171.stdout: "req_renamesnap_latency": { 2023-07-14T18:35:19.719 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.719 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.719 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.719 INFO:teuthology.orchestra.run.smithi171.stdout: "req_snapdiff_latency": { 2023-07-14T18:35:19.720 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.720 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.720 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.720 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmdir_latency": { 2023-07-14T18:35:19.720 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.720 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.721 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.721 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmsnap_latency": { 2023-07-14T18:35:19.721 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.721 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.721 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.721 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmxattr_latency": { 2023-07-14T18:35:19.722 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.722 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.722 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.722 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setattr_latency": { 2023-07-14T18:35:19.722 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.722 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.723 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.723 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setdirlayout_latency": { 2023-07-14T18:35:19.723 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.723 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.723 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.723 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setfilelock_latency": { 2023-07-14T18:35:19.724 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.724 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.724 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.724 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setlayout_latency": { 2023-07-14T18:35:19.724 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.724 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.724 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.725 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setxattr_latency": { 2023-07-14T18:35:19.725 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.725 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.725 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.725 INFO:teuthology.orchestra.run.smithi171.stdout: "req_symlink_latency": { 2023-07-14T18:35:19.725 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.726 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.726 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.726 INFO:teuthology.orchestra.run.smithi171.stdout: "req_unlink_latency": { 2023-07-14T18:35:19.726 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.726 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.726 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.727 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_revoke_eviction": 0, 2023-07-14T18:35:19.727 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_acquisition_throttle": 0, 2023-07-14T18:35:19.727 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getvxattr_latency": { 2023-07-14T18:35:19.727 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.727 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.727 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.727 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:19.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.728 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_sessions": { 2023-07-14T18:35:19.728 INFO:teuthology.orchestra.run.smithi171.stdout: "session_count": 1, 2023-07-14T18:35:19.728 INFO:teuthology.orchestra.run.smithi171.stdout: "session_add": 3, 2023-07-14T18:35:19.728 INFO:teuthology.orchestra.run.smithi171.stdout: "session_remove": 2, 2023-07-14T18:35:19.728 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_open": 1, 2023-07-14T18:35:19.728 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_stale": 0, 2023-07-14T18:35:19.728 INFO:teuthology.orchestra.run.smithi171.stdout: "total_load": 0, 2023-07-14T18:35:19.729 INFO:teuthology.orchestra.run.smithi171.stdout: "average_load": 0, 2023-07-14T18:35:19.729 INFO:teuthology.orchestra.run.smithi171.stdout: "avg_session_uptime": 3213 2023-07-14T18:35:19.729 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.729 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2023-07-14T18:35:19.729 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2023-07-14T18:35:19.729 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2023-07-14T18:35:19.729 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T18:35:19.729 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2023-07-14T18:35:19.730 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T18:35:19.730 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2023-07-14T18:35:19.730 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T18:35:19.730 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T18:35:19.730 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T18:35:19.730 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T18:35:19.730 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T18:35:19.730 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2023-07-14T18:35:19.731 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T18:35:19.731 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T18:35:19.731 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2023-07-14T18:35:19.731 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2023-07-14T18:35:19.731 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2023-07-14T18:35:19.731 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2023-07-14T18:35:19.731 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T18:35:19.731 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T18:35:19.732 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T18:35:19.732 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T18:35:19.732 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T18:35:19.732 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2023-07-14T18:35:19.732 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2023-07-14T18:35:19.732 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2023-07-14T18:35:19.732 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T18:35:19.732 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T18:35:19.733 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2023-07-14T18:35:19.733 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2023-07-14T18:35:19.733 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2023-07-14T18:35:19.733 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2023-07-14T18:35:19.733 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T18:35:19.733 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2023-07-14T18:35:19.733 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T18:35:19.733 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2023-07-14T18:35:19.734 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 207341, 2023-07-14T18:35:19.734 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 516, 2023-07-14T18:35:19.734 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2023-07-14T18:35:19.734 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2023-07-14T18:35:19.734 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2023-07-14T18:35:19.734 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2023-07-14T18:35:19.734 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2023-07-14T18:35:19.734 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2023-07-14T18:35:19.735 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2023-07-14T18:35:19.735 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2023-07-14T18:35:19.735 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 8792, 2023-07-14T18:35:19.735 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 168, 2023-07-14T18:35:19.735 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T18:35:19.735 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2023-07-14T18:35:19.735 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2023-07-14T18:35:19.735 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2023-07-14T18:35:19.736 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 1558525, 2023-07-14T18:35:19.736 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 13919, 2023-07-14T18:35:19.736 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2023-07-14T18:35:19.736 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2023-07-14T18:35:19.736 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2023-07-14T18:35:19.736 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2023-07-14T18:35:19.736 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.737 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter": { 2023-07-14T18:35:19.737 INFO:teuthology.orchestra.run.smithi171.stdout: "op_active": 0, 2023-07-14T18:35:19.737 INFO:teuthology.orchestra.run.smithi171.stdout: "op_laggy": 0, 2023-07-14T18:35:19.737 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send": 3411, 2023-07-14T18:35:19.737 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send_bytes": 26, 2023-07-14T18:35:19.737 INFO:teuthology.orchestra.run.smithi171.stdout: "op_resend": 4, 2023-07-14T18:35:19.737 INFO:teuthology.orchestra.run.smithi171.stdout: "op_reply": 3411, 2023-07-14T18:35:19.737 INFO:teuthology.orchestra.run.smithi171.stdout: "op_latency": { 2023-07-14T18:35:19.738 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 3411, 2023-07-14T18:35:19.738 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 258.150497089, 2023-07-14T18:35:19.738 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.075681764 2023-07-14T18:35:19.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.738 INFO:teuthology.orchestra.run.smithi171.stdout: "op_inflight": 0, 2023-07-14T18:35:19.738 INFO:teuthology.orchestra.run.smithi171.stdout: "oplen_avg": { 2023-07-14T18:35:19.738 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 3411, 2023-07-14T18:35:19.738 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 3412 2023-07-14T18:35:19.739 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.739 INFO:teuthology.orchestra.run.smithi171.stdout: "op": 3411, 2023-07-14T18:35:19.739 INFO:teuthology.orchestra.run.smithi171.stdout: "op_r": 3411, 2023-07-14T18:35:19.739 INFO:teuthology.orchestra.run.smithi171.stdout: "op_w": 0, 2023-07-14T18:35:19.739 INFO:teuthology.orchestra.run.smithi171.stdout: "op_rmw": 0, 2023-07-14T18:35:19.739 INFO:teuthology.orchestra.run.smithi171.stdout: "op_pg": 0, 2023-07-14T18:35:19.739 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_stat": 1950, 2023-07-14T18:35:19.739 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_create": 0, 2023-07-14T18:35:19.740 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_read": 1458, 2023-07-14T18:35:19.740 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_write": 0, 2023-07-14T18:35:19.740 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writefull": 0, 2023-07-14T18:35:19.740 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writesame": 0, 2023-07-14T18:35:19.740 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_append": 0, 2023-07-14T18:35:19.740 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_zero": 0, 2023-07-14T18:35:19.740 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_truncate": 0, 2023-07-14T18:35:19.740 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_delete": 0, 2023-07-14T18:35:19.741 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_mapext": 0, 2023-07-14T18:35:19.741 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_sparse_read": 0, 2023-07-14T18:35:19.741 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_clonerange": 0, 2023-07-14T18:35:19.741 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_getxattr": 2, 2023-07-14T18:35:19.741 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_setxattr": 0, 2023-07-14T18:35:19.741 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_cmpxattr": 0, 2023-07-14T18:35:19.741 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_rmxattr": 0, 2023-07-14T18:35:19.742 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_resetxattrs": 0, 2023-07-14T18:35:19.742 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_call": 0, 2023-07-14T18:35:19.742 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_watch": 0, 2023-07-14T18:35:19.742 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_notify": 0, 2023-07-14T18:35:19.742 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_src_cmpxattr": 0, 2023-07-14T18:35:19.742 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls": 0, 2023-07-14T18:35:19.742 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls_filter": 0, 2023-07-14T18:35:19.742 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_other": 0, 2023-07-14T18:35:19.743 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_active": 0, 2023-07-14T18:35:19.743 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_send": 0, 2023-07-14T18:35:19.743 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_resend": 0, 2023-07-14T18:35:19.743 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_ping": 0, 2023-07-14T18:35:19.743 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_active": 0, 2023-07-14T18:35:19.743 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_send": 0, 2023-07-14T18:35:19.743 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_resend": 0, 2023-07-14T18:35:19.743 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_active": 0, 2023-07-14T18:35:19.744 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_send": 0, 2023-07-14T18:35:19.744 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_resend": 0, 2023-07-14T18:35:19.744 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_active": 0, 2023-07-14T18:35:19.744 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_send": 0, 2023-07-14T18:35:19.744 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_resend": 0, 2023-07-14T18:35:19.744 INFO:teuthology.orchestra.run.smithi171.stdout: "command_active": 0, 2023-07-14T18:35:19.744 INFO:teuthology.orchestra.run.smithi171.stdout: "command_send": 0, 2023-07-14T18:35:19.744 INFO:teuthology.orchestra.run.smithi171.stdout: "command_resend": 0, 2023-07-14T18:35:19.745 INFO:teuthology.orchestra.run.smithi171.stdout: "map_epoch": 92, 2023-07-14T18:35:19.745 INFO:teuthology.orchestra.run.smithi171.stdout: "map_full": 0, 2023-07-14T18:35:19.745 INFO:teuthology.orchestra.run.smithi171.stdout: "map_inc": 2, 2023-07-14T18:35:19.745 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_sessions": 5, 2023-07-14T18:35:19.745 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_open": 6, 2023-07-14T18:35:19.745 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_close": 1, 2023-07-14T18:35:19.745 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_laggy": 0, 2023-07-14T18:35:19.745 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_wr": 0, 2023-07-14T18:35:19.746 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_rd": 2, 2023-07-14T18:35:19.746 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_del": 0 2023-07-14T18:35:19.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.746 INFO:teuthology.orchestra.run.smithi171.stdout: "oft": { 2023-07-14T18:35:19.746 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_objs": 0, 2023-07-14T18:35:19.746 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_kv_pairs": 0, 2023-07-14T18:35:19.746 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_updates": 0, 2023-07-14T18:35:19.746 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_removes": 0 2023-07-14T18:35:19.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.747 INFO:teuthology.orchestra.run.smithi171.stdout: "osd-slow-ops": { 2023-07-14T18:35:19.747 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_ops_count": 0 2023-07-14T18:35:19.747 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.747 INFO:teuthology.orchestra.run.smithi171.stdout: "purge_queue": { 2023-07-14T18:35:19.747 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops": 0, 2023-07-14T18:35:19.747 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops_high_water": 0, 2023-07-14T18:35:19.747 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing": 0, 2023-07-14T18:35:19.748 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_high_water": 0, 2023-07-14T18:35:19.748 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed": 0, 2023-07-14T18:35:19.748 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_item_in_journal": 0 2023-07-14T18:35:19.748 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.748 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T18:35:19.748 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2023-07-14T18:35:19.748 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2023-07-14T18:35:19.749 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2023-07-14T18:35:19.749 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 3791, 2023-07-14T18:35:19.749 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 16288313, 2023-07-14T18:35:19.749 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:19.749 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 3791, 2023-07-14T18:35:19.749 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2023-07-14T18:35:19.749 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2023-07-14T18:35:19.749 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 3791, 2023-07-14T18:35:19.750 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 16288313, 2023-07-14T18:35:19.750 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2023-07-14T18:35:19.750 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.750 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.750 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.750 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:19.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.750 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_bytes": { 2023-07-14T18:35:19.751 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2023-07-14T18:35:19.751 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2023-07-14T18:35:19.751 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2023-07-14T18:35:19.751 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2023-07-14T18:35:19.751 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2023-07-14T18:35:19.751 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:19.751 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2023-07-14T18:35:19.751 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 3411, 2023-07-14T18:35:19.752 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 15078358, 2023-07-14T18:35:19.752 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 3411, 2023-07-14T18:35:19.752 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 15078358, 2023-07-14T18:35:19.752 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2023-07-14T18:35:19.752 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.752 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.752 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.752 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:19.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.753 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_ops": { 2023-07-14T18:35:19.753 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2023-07-14T18:35:19.753 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 1024, 2023-07-14T18:35:19.753 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2023-07-14T18:35:19.753 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2023-07-14T18:35:19.753 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2023-07-14T18:35:19.753 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:19.754 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2023-07-14T18:35:19.754 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 3411, 2023-07-14T18:35:19.754 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 3411, 2023-07-14T18:35:19.754 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 3411, 2023-07-14T18:35:19.754 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 3411, 2023-07-14T18:35:19.754 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2023-07-14T18:35:19.754 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.755 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.755 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.755 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:19.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.755 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle": { 2023-07-14T18:35:19.755 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2023-07-14T18:35:19.755 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2023-07-14T18:35:19.755 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2023-07-14T18:35:19.756 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2023-07-14T18:35:19.756 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2023-07-14T18:35:19.756 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:19.756 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2023-07-14T18:35:19.756 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2023-07-14T18:35:19.756 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2023-07-14T18:35:19.756 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 0, 2023-07-14T18:35:19.756 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 0, 2023-07-14T18:35:19.757 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2023-07-14T18:35:19.757 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.757 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.757 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.757 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:19.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:19.757 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle-0x55d06b368460": { 2023-07-14T18:35:19.757 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2023-07-14T18:35:19.758 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2023-07-14T18:35:19.758 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2023-07-14T18:35:19.758 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2023-07-14T18:35:19.758 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2023-07-14T18:35:19.758 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:19.758 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2023-07-14T18:35:19.758 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2023-07-14T18:35:19.758 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2023-07-14T18:35:19.759 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 0, 2023-07-14T18:35:19.759 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 0, 2023-07-14T18:35:19.759 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2023-07-14T18:35:19.759 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:19.759 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:19.759 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:19.759 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:19.760 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:19.760 INFO:teuthology.orchestra.run.smithi171.stdout:} 2023-07-14T18:35:20.107 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-07-14T18:35:20.107 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2023-07-14T18:35:20.107 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2023-07-14T18:35:20.108 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2023-07-14T18:35:20.108 DEBUG:tasks.check_counter:Getting stats from f 2023-07-14T18:35:20.108 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph daemon mds.f perf dump 2023-07-14T18:35:20.773 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T18:35:20.880 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:20 smithi133 ceph-mon[126384]: pgmap v907: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2023-07-14T18:35:20.981 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:20 smithi171 ceph-mon[129241]: pgmap v907: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2023-07-14T18:35:21.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:20 smithi161 ceph-mon[128993]: pgmap v907: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2023-07-14T18:35:22.620 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2023-07-14T18:35:22.620 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T18:35:22.621 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 1620, 2023-07-14T18:35:22.621 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 1789, 2023-07-14T18:35:22.621 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 471207, 2023-07-14T18:35:22.621 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 4586557, 2023-07-14T18:35:22.622 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 16, 2023-07-14T18:35:22.622 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 5, 2023-07-14T18:35:22.622 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.133274890, 2023-07-14T18:35:22.622 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.049868179, 2023-07-14T18:35:22.623 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.049633778, 2023-07-14T18:35:22.623 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.031677395, 2023-07-14T18:35:22.623 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:35:22.624 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1785, 2023-07-14T18:35:22.624 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.635402232, 2023-07-14T18:35:22.624 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000355967 2023-07-14T18:35:22.624 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.625 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:35:22.625 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1335, 2023-07-14T18:35:22.625 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000260991, 2023-07-14T18:35:22.625 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000195 2023-07-14T18:35:22.626 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.626 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-14T18:35:22.626 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 0 2023-07-14T18:35:22.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.627 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T18:35:22.627 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 6243182, 2023-07-14T18:35:22.627 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 6241766, 2023-07-14T18:35:22.628 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 1012719336, 2023-07-14T18:35:22.628 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 987856711, 2023-07-14T18:35:22.628 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 9, 2023-07-14T18:35:22.629 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 5, 2023-07-14T18:35:22.629 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 176.116274976, 2023-07-14T18:35:22.629 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 69.016761705, 2023-07-14T18:35:22.630 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 121.371331686, 2023-07-14T18:35:22.630 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.056923786, 2023-07-14T18:35:22.630 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:35:22.630 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 6241767, 2023-07-14T18:35:22.630 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 141.557056217, 2023-07-14T18:35:22.631 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000022679 2023-07-14T18:35:22.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.631 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:35:22.631 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 11500687, 2023-07-14T18:35:22.631 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 1.452141674, 2023-07-14T18:35:22.631 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000126 2023-07-14T18:35:22.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.631 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-07-14T18:35:22.632 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 609760 2023-07-14T18:35:22.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.632 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T18:35:22.632 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 4062, 2023-07-14T18:35:22.632 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 5627, 2023-07-14T18:35:22.632 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 1761767, 2023-07-14T18:35:22.632 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 2740513, 2023-07-14T18:35:22.633 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 8, 2023-07-14T18:35:22.633 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 5, 2023-07-14T18:35:22.633 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.345979424, 2023-07-14T18:35:22.633 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.107802103, 2023-07-14T18:35:22.633 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.208532578, 2023-07-14T18:35:22.633 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.083826244, 2023-07-14T18:35:22.633 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:35:22.633 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 5626, 2023-07-14T18:35:22.634 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 1.547898871, 2023-07-14T18:35:22.634 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000275133 2023-07-14T18:35:22.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.634 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:35:22.634 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2821, 2023-07-14T18:35:22.634 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.001380465, 2023-07-14T18:35:22.634 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000489 2023-07-14T18:35:22.634 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.635 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 629424, 2023-07-14T18:35:22.635 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 205248 2023-07-14T18:35:22.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.635 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2023-07-14T18:35:22.635 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 1, 2023-07-14T18:35:22.635 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2023-07-14T18:35:22.635 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.636 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-MDSRank": { 2023-07-14T18:35:22.636 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2023-07-14T18:35:22.636 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2023-07-14T18:35:22.636 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1565, 2023-07-14T18:35:22.636 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.971152823, 2023-07-14T18:35:22.636 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000620544 2023-07-14T18:35:22.636 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:22.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.637 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-PurgeQueue": { 2023-07-14T18:35:22.637 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2023-07-14T18:35:22.637 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2023-07-14T18:35:22.637 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 816, 2023-07-14T18:35:22.637 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.041406247, 2023-07-14T18:35:22.637 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000050742 2023-07-14T18:35:22.638 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:22.638 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.638 INFO:teuthology.orchestra.run.smithi171.stdout: "mds": { 2023-07-14T18:35:22.638 INFO:teuthology.orchestra.run.smithi171.stdout: "request": 1625, 2023-07-14T18:35:22.639 INFO:teuthology.orchestra.run.smithi171.stdout: "reply": 1214, 2023-07-14T18:35:22.639 INFO:teuthology.orchestra.run.smithi171.stdout: "reply_latency": { 2023-07-14T18:35:22.639 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1214, 2023-07-14T18:35:22.639 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 3.827949225, 2023-07-14T18:35:22.640 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.003153170 2023-07-14T18:35:22.640 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.640 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_reply": 0, 2023-07-14T18:35:22.641 INFO:teuthology.orchestra.run.smithi171.stdout: "forward": 411, 2023-07-14T18:35:22.641 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_complete": 0, 2023-07-14T18:35:22.641 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_keys": 0, 2023-07-14T18:35:22.641 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_commit": 11, 2023-07-14T18:35:22.642 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_split": 0, 2023-07-14T18:35:22.642 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_merge": 0, 2023-07-14T18:35:22.642 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes": 596, 2023-07-14T18:35:22.642 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_top": 0, 2023-07-14T18:35:22.643 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_bottom": 0, 2023-07-14T18:35:22.643 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pin_tail": 596, 2023-07-14T18:35:22.643 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pinned": 596, 2023-07-14T18:35:22.643 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_expired": 164, 2023-07-14T18:35:22.644 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_with_caps": 525, 2023-07-14T18:35:22.644 INFO:teuthology.orchestra.run.smithi171.stdout: "caps": 525, 2023-07-14T18:35:22.644 INFO:teuthology.orchestra.run.smithi171.stdout: "subtrees": 21, 2023-07-14T18:35:22.644 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse": 2877, 2023-07-14T18:35:22.645 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_hit": 2110, 2023-07-14T18:35:22.645 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_forward": 411, 2023-07-14T18:35:22.645 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_discover": 61, 2023-07-14T18:35:22.645 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_dir_fetch": 0, 2023-07-14T18:35:22.646 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_remote_ino": 0, 2023-07-14T18:35:22.646 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_lock": 89, 2023-07-14T18:35:22.646 INFO:teuthology.orchestra.run.smithi171.stdout: "load_cent": 1045, 2023-07-14T18:35:22.646 INFO:teuthology.orchestra.run.smithi171.stdout: "q": 0, 2023-07-14T18:35:22.647 INFO:teuthology.orchestra.run.smithi171.stdout: "exported": 4, 2023-07-14T18:35:22.647 INFO:teuthology.orchestra.run.smithi171.stdout: "exported_inodes": 113, 2023-07-14T18:35:22.647 INFO:teuthology.orchestra.run.smithi171.stdout: "imported": 23, 2023-07-14T18:35:22.647 INFO:teuthology.orchestra.run.smithi171.stdout: "imported_inodes": 665, 2023-07-14T18:35:22.648 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_dir_fetch": 0, 2023-07-14T18:35:22.648 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_backtrace_fetch": 263, 2023-07-14T18:35:22.648 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_peer_discover": 0, 2023-07-14T18:35:22.648 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rfiles": 1047, 2023-07-14T18:35:22.649 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rbytes": 9049017265, 2023-07-14T18:35:22.649 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rsnaps": 0, 2023-07-14T18:35:22.649 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_fetch": 263, 2023-07-14T18:35:22.649 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_set_tag": 0, 2023-07-14T18:35:22.650 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_repaired": 0, 2023-07-14T18:35:22.650 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_inotable_repaired": 0, 2023-07-14T18:35:22.650 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_inodes": 0, 2023-07-14T18:35:22.650 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_base_inodes": 0, 2023-07-14T18:35:22.651 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dirfrag_rstats": 0, 2023-07-14T18:35:22.651 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_file_inodes": 263, 2023-07-14T18:35:22.651 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_inode_file_caps": 0, 2023-07-14T18:35:22.651 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_revoke": 18, 2023-07-14T18:35:22.652 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_grant": 803, 2023-07-14T18:35:22.652 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_trunc": 0, 2023-07-14T18:35:22.652 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-14T18:35:22.653 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-14T18:35:22.653 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps": 1302, 2023-07-14T18:35:22.653 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps_dirty": 1087, 2023-07-14T18:35:22.653 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_cap_release": 105, 2023-07-14T18:35:22.654 INFO:teuthology.orchestra.run.smithi171.stdout: "process_request_cap_release": 989 2023-07-14T18:35:22.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.654 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache": { 2023-07-14T18:35:22.654 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update": 98, 2023-07-14T18:35:22.655 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update_receipt": 304, 2023-07-14T18:35:22.655 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_try_discover": 164, 2023-07-14T18:35:22.655 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_send_discover": 304, 2023-07-14T18:35:22.655 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_handle_discover": 114, 2023-07-14T18:35:22.656 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays": 0, 2023-07-14T18:35:22.656 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_delayed": 0, 2023-07-14T18:35:22.656 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_enqueuing": 0, 2023-07-14T18:35:22.656 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_created": 236, 2023-07-14T18:35:22.657 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_enqueued": 236, 2023-07-14T18:35:22.657 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_reintegrated": 0, 2023-07-14T18:35:22.657 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_migrated": 0, 2023-07-14T18:35:22.658 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_processing": 0, 2023-07-14T18:35:22.658 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_enqueued": 0, 2023-07-14T18:35:22.658 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_prioritized": 0, 2023-07-14T18:35:22.658 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_started": 0, 2023-07-14T18:35:22.659 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_completed": 0, 2023-07-14T18:35:22.659 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_enqueue_scrub": 0, 2023-07-14T18:35:22.659 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_exportdir": 4, 2023-07-14T18:35:22.659 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_flush": 0, 2023-07-14T18:35:22.660 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragmentdir": 0, 2023-07-14T18:35:22.660 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragstats": 0, 2023-07-14T18:35:22.660 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_inodestats": 0 2023-07-14T18:35:22.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.661 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log": { 2023-07-14T18:35:22.661 INFO:teuthology.orchestra.run.smithi171.stdout: "evadd": 2975, 2023-07-14T18:35:22.661 INFO:teuthology.orchestra.run.smithi171.stdout: "evex": 0, 2023-07-14T18:35:22.661 INFO:teuthology.orchestra.run.smithi171.stdout: "evtrm": 0, 2023-07-14T18:35:22.661 INFO:teuthology.orchestra.run.smithi171.stdout: "ev": 2975, 2023-07-14T18:35:22.662 INFO:teuthology.orchestra.run.smithi171.stdout: "evexg": 0, 2023-07-14T18:35:22.662 INFO:teuthology.orchestra.run.smithi171.stdout: "evexd": 0, 2023-07-14T18:35:22.662 INFO:teuthology.orchestra.run.smithi171.stdout: "segadd": 4, 2023-07-14T18:35:22.662 INFO:teuthology.orchestra.run.smithi171.stdout: "segex": 0, 2023-07-14T18:35:22.662 INFO:teuthology.orchestra.run.smithi171.stdout: "segtrm": 0, 2023-07-14T18:35:22.662 INFO:teuthology.orchestra.run.smithi171.stdout: "seg": 4, 2023-07-14T18:35:22.662 INFO:teuthology.orchestra.run.smithi171.stdout: "segexg": 0, 2023-07-14T18:35:22.662 INFO:teuthology.orchestra.run.smithi171.stdout: "segexd": 0, 2023-07-14T18:35:22.663 INFO:teuthology.orchestra.run.smithi171.stdout: "expos": 4194304, 2023-07-14T18:35:22.663 INFO:teuthology.orchestra.run.smithi171.stdout: "wrpos": 8836309, 2023-07-14T18:35:22.663 INFO:teuthology.orchestra.run.smithi171.stdout: "rdpos": 0, 2023-07-14T18:35:22.663 INFO:teuthology.orchestra.run.smithi171.stdout: "jlat": { 2023-07-14T18:35:22.663 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 638, 2023-07-14T18:35:22.663 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 27.880201729, 2023-07-14T18:35:22.663 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.043699375 2023-07-14T18:35:22.663 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.664 INFO:teuthology.orchestra.run.smithi171.stdout: "replayed": 0 2023-07-14T18:35:22.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.664 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_mem": { 2023-07-14T18:35:22.664 INFO:teuthology.orchestra.run.smithi171.stdout: "ino": 576, 2023-07-14T18:35:22.664 INFO:teuthology.orchestra.run.smithi171.stdout: "ino+": 935, 2023-07-14T18:35:22.664 INFO:teuthology.orchestra.run.smithi171.stdout: "ino-": 359, 2023-07-14T18:35:22.664 INFO:teuthology.orchestra.run.smithi171.stdout: "dir": 56, 2023-07-14T18:35:22.665 INFO:teuthology.orchestra.run.smithi171.stdout: "dir+": 175, 2023-07-14T18:35:22.665 INFO:teuthology.orchestra.run.smithi171.stdout: "dir-": 119, 2023-07-14T18:35:22.665 INFO:teuthology.orchestra.run.smithi171.stdout: "dn": 596, 2023-07-14T18:35:22.665 INFO:teuthology.orchestra.run.smithi171.stdout: "dn+": 1000, 2023-07-14T18:35:22.665 INFO:teuthology.orchestra.run.smithi171.stdout: "dn-": 404, 2023-07-14T18:35:22.665 INFO:teuthology.orchestra.run.smithi171.stdout: "cap": 525, 2023-07-14T18:35:22.665 INFO:teuthology.orchestra.run.smithi171.stdout: "cap+": 1210, 2023-07-14T18:35:22.665 INFO:teuthology.orchestra.run.smithi171.stdout: "cap-": 685, 2023-07-14T18:35:22.666 INFO:teuthology.orchestra.run.smithi171.stdout: "rss": 60192, 2023-07-14T18:35:22.666 INFO:teuthology.orchestra.run.smithi171.stdout: "heap": 198960 2023-07-14T18:35:22.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.666 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_server": { 2023-07-14T18:35:22.666 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_client_request": 1919, 2023-07-14T18:35:22.666 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_server_request": 50, 2023-07-14T18:35:22.666 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_request": 1625, 2023-07-14T18:35:22.666 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_session": 32, 2023-07-14T18:35:22.667 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_peer_request": 110, 2023-07-14T18:35:22.667 INFO:teuthology.orchestra.run.smithi171.stdout: "req_create_latency": { 2023-07-14T18:35:22.667 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 240, 2023-07-14T18:35:22.667 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.590234107, 2023-07-14T18:35:22.667 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.002459308 2023-07-14T18:35:22.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.667 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getattr_latency": { 2023-07-14T18:35:22.667 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.668 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.668 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.668 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.668 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getfilelock_latency": { 2023-07-14T18:35:22.668 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.668 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.668 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.669 INFO:teuthology.orchestra.run.smithi171.stdout: "req_link_latency": { 2023-07-14T18:35:22.669 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.669 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.669 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.669 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookup_latency": { 2023-07-14T18:35:22.669 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 522, 2023-07-14T18:35:22.670 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 2.456478796, 2023-07-14T18:35:22.670 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.004705898 2023-07-14T18:35:22.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.670 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookuphash_latency": { 2023-07-14T18:35:22.670 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.670 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.670 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.671 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupino_latency": { 2023-07-14T18:35:22.671 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2, 2023-07-14T18:35:22.671 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000169800, 2023-07-14T18:35:22.671 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000084900 2023-07-14T18:35:22.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.671 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupname_latency": { 2023-07-14T18:35:22.671 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.672 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.672 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.672 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupparent_latency": { 2023-07-14T18:35:22.672 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.672 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.672 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.672 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.673 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupsnap_latency": { 2023-07-14T18:35:22.673 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.673 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.673 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.673 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.673 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lssnap_latency": { 2023-07-14T18:35:22.674 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.674 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.674 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.674 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mkdir_latency": { 2023-07-14T18:35:22.675 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.675 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.675 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.675 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mknod_latency": { 2023-07-14T18:35:22.675 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.675 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.675 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.676 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mksnap_latency": { 2023-07-14T18:35:22.676 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.676 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.676 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.676 INFO:teuthology.orchestra.run.smithi171.stdout: "req_open_latency": { 2023-07-14T18:35:22.677 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 192, 2023-07-14T18:35:22.677 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.104470877, 2023-07-14T18:35:22.677 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000544119 2023-07-14T18:35:22.677 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.677 INFO:teuthology.orchestra.run.smithi171.stdout: "req_readdir_latency": { 2023-07-14T18:35:22.677 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 20, 2023-07-14T18:35:22.678 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.017444979, 2023-07-14T18:35:22.678 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000872248 2023-07-14T18:35:22.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.678 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rename_latency": { 2023-07-14T18:35:22.678 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 230, 2023-07-14T18:35:22.678 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.628843412, 2023-07-14T18:35:22.678 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.002734101 2023-07-14T18:35:22.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.679 INFO:teuthology.orchestra.run.smithi171.stdout: "req_renamesnap_latency": { 2023-07-14T18:35:22.679 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.679 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.679 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.679 INFO:teuthology.orchestra.run.smithi171.stdout: "req_snapdiff_latency": { 2023-07-14T18:35:22.679 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.680 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.680 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.680 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmdir_latency": { 2023-07-14T18:35:22.680 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.681 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.681 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.681 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmsnap_latency": { 2023-07-14T18:35:22.681 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.681 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.681 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.681 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.682 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmxattr_latency": { 2023-07-14T18:35:22.682 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.682 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.682 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.682 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setattr_latency": { 2023-07-14T18:35:22.682 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.683 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.683 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.683 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setdirlayout_latency": { 2023-07-14T18:35:22.683 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.683 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.684 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.684 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setfilelock_latency": { 2023-07-14T18:35:22.684 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.684 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.685 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.685 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setlayout_latency": { 2023-07-14T18:35:22.685 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.685 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.686 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.686 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setxattr_latency": { 2023-07-14T18:35:22.686 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.686 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.686 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.686 INFO:teuthology.orchestra.run.smithi171.stdout: "req_symlink_latency": { 2023-07-14T18:35:22.687 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.687 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.687 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.687 INFO:teuthology.orchestra.run.smithi171.stdout: "req_unlink_latency": { 2023-07-14T18:35:22.687 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 8, 2023-07-14T18:35:22.687 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.030307254, 2023-07-14T18:35:22.688 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.003788406 2023-07-14T18:35:22.688 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.688 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_revoke_eviction": 0, 2023-07-14T18:35:22.688 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_acquisition_throttle": 0, 2023-07-14T18:35:22.688 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getvxattr_latency": { 2023-07-14T18:35:22.688 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.688 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.689 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.689 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:22.689 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.689 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_sessions": { 2023-07-14T18:35:22.689 INFO:teuthology.orchestra.run.smithi171.stdout: "session_count": 1, 2023-07-14T18:35:22.689 INFO:teuthology.orchestra.run.smithi171.stdout: "session_add": 1, 2023-07-14T18:35:22.689 INFO:teuthology.orchestra.run.smithi171.stdout: "session_remove": 0, 2023-07-14T18:35:22.689 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_open": 1, 2023-07-14T18:35:22.690 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_stale": 0, 2023-07-14T18:35:22.690 INFO:teuthology.orchestra.run.smithi171.stdout: "total_load": 110, 2023-07-14T18:35:22.690 INFO:teuthology.orchestra.run.smithi171.stdout: "average_load": 110, 2023-07-14T18:35:22.690 INFO:teuthology.orchestra.run.smithi171.stdout: "avg_session_uptime": 1125 2023-07-14T18:35:22.690 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.690 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2023-07-14T18:35:22.691 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2023-07-14T18:35:22.691 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2023-07-14T18:35:22.691 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T18:35:22.691 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2023-07-14T18:35:22.691 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T18:35:22.691 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2023-07-14T18:35:22.691 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T18:35:22.691 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T18:35:22.692 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T18:35:22.692 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T18:35:22.692 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T18:35:22.692 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2023-07-14T18:35:22.692 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T18:35:22.692 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T18:35:22.692 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2023-07-14T18:35:22.692 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2023-07-14T18:35:22.693 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2023-07-14T18:35:22.693 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2023-07-14T18:35:22.693 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T18:35:22.693 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T18:35:22.693 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T18:35:22.693 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T18:35:22.693 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T18:35:22.694 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2023-07-14T18:35:22.694 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2023-07-14T18:35:22.694 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2023-07-14T18:35:22.694 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T18:35:22.694 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T18:35:22.694 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2023-07-14T18:35:22.694 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2023-07-14T18:35:22.694 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2023-07-14T18:35:22.695 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2023-07-14T18:35:22.695 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T18:35:22.695 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2023-07-14T18:35:22.695 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T18:35:22.695 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2023-07-14T18:35:22.695 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 477537, 2023-07-14T18:35:22.695 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 721, 2023-07-14T18:35:22.695 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 704, 2023-07-14T18:35:22.695 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 8, 2023-07-14T18:35:22.696 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2023-07-14T18:35:22.696 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2023-07-14T18:35:22.696 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2023-07-14T18:35:22.696 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2023-07-14T18:35:22.696 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2023-07-14T18:35:22.697 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2023-07-14T18:35:22.697 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 8792, 2023-07-14T18:35:22.697 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 168, 2023-07-14T18:35:22.697 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T18:35:22.697 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2023-07-14T18:35:22.697 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2023-07-14T18:35:22.697 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2023-07-14T18:35:22.698 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 2043873, 2023-07-14T18:35:22.698 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 21313, 2023-07-14T18:35:22.698 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2023-07-14T18:35:22.698 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2023-07-14T18:35:22.698 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2023-07-14T18:35:22.698 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2023-07-14T18:35:22.698 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.698 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter": { 2023-07-14T18:35:22.699 INFO:teuthology.orchestra.run.smithi171.stdout: "op_active": 0, 2023-07-14T18:35:22.699 INFO:teuthology.orchestra.run.smithi171.stdout: "op_laggy": 0, 2023-07-14T18:35:22.699 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send": 1873, 2023-07-14T18:35:22.699 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send_bytes": 4764598, 2023-07-14T18:35:22.699 INFO:teuthology.orchestra.run.smithi171.stdout: "op_resend": 33, 2023-07-14T18:35:22.699 INFO:teuthology.orchestra.run.smithi171.stdout: "op_reply": 1873, 2023-07-14T18:35:22.699 INFO:teuthology.orchestra.run.smithi171.stdout: "op_latency": { 2023-07-14T18:35:22.699 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1873, 2023-07-14T18:35:22.700 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 71.299748681, 2023-07-14T18:35:22.700 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.038067137 2023-07-14T18:35:22.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.700 INFO:teuthology.orchestra.run.smithi171.stdout: "op_inflight": 0, 2023-07-14T18:35:22.700 INFO:teuthology.orchestra.run.smithi171.stdout: "oplen_avg": { 2023-07-14T18:35:22.700 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1873, 2023-07-14T18:35:22.700 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 2023 2023-07-14T18:35:22.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.701 INFO:teuthology.orchestra.run.smithi171.stdout: "op": 1873, 2023-07-14T18:35:22.701 INFO:teuthology.orchestra.run.smithi171.stdout: "op_r": 365, 2023-07-14T18:35:22.701 INFO:teuthology.orchestra.run.smithi171.stdout: "op_w": 1508, 2023-07-14T18:35:22.701 INFO:teuthology.orchestra.run.smithi171.stdout: "op_rmw": 0, 2023-07-14T18:35:22.701 INFO:teuthology.orchestra.run.smithi171.stdout: "op_pg": 0, 2023-07-14T18:35:22.701 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_stat": 0, 2023-07-14T18:35:22.701 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_create": 10, 2023-07-14T18:35:22.701 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_read": 102, 2023-07-14T18:35:22.702 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_write": 740, 2023-07-14T18:35:22.702 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writefull": 155, 2023-07-14T18:35:22.702 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writesame": 0, 2023-07-14T18:35:22.702 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_append": 0, 2023-07-14T18:35:22.702 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_zero": 0, 2023-07-14T18:35:22.702 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_truncate": 0, 2023-07-14T18:35:22.702 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_delete": 485, 2023-07-14T18:35:22.703 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_mapext": 0, 2023-07-14T18:35:22.703 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_sparse_read": 0, 2023-07-14T18:35:22.703 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_clonerange": 0, 2023-07-14T18:35:22.703 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_getxattr": 263, 2023-07-14T18:35:22.703 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_setxattr": 20, 2023-07-14T18:35:22.703 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_cmpxattr": 0, 2023-07-14T18:35:22.703 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_rmxattr": 0, 2023-07-14T18:35:22.703 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_resetxattrs": 0, 2023-07-14T18:35:22.704 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_call": 0, 2023-07-14T18:35:22.704 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_watch": 0, 2023-07-14T18:35:22.704 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_notify": 0, 2023-07-14T18:35:22.704 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_src_cmpxattr": 0, 2023-07-14T18:35:22.704 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls": 0, 2023-07-14T18:35:22.704 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls_filter": 0, 2023-07-14T18:35:22.704 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_other": 0, 2023-07-14T18:35:22.704 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_active": 0, 2023-07-14T18:35:22.705 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_send": 0, 2023-07-14T18:35:22.705 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_resend": 0, 2023-07-14T18:35:22.705 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_ping": 0, 2023-07-14T18:35:22.705 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_active": 0, 2023-07-14T18:35:22.705 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_send": 0, 2023-07-14T18:35:22.705 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_resend": 0, 2023-07-14T18:35:22.705 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_active": 0, 2023-07-14T18:35:22.705 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_send": 0, 2023-07-14T18:35:22.706 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_resend": 0, 2023-07-14T18:35:22.706 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_active": 0, 2023-07-14T18:35:22.706 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_send": 0, 2023-07-14T18:35:22.706 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_resend": 0, 2023-07-14T18:35:22.706 INFO:teuthology.orchestra.run.smithi171.stdout: "command_active": 0, 2023-07-14T18:35:22.706 INFO:teuthology.orchestra.run.smithi171.stdout: "command_send": 0, 2023-07-14T18:35:22.706 INFO:teuthology.orchestra.run.smithi171.stdout: "command_resend": 0, 2023-07-14T18:35:22.706 INFO:teuthology.orchestra.run.smithi171.stdout: "map_epoch": 92, 2023-07-14T18:35:22.707 INFO:teuthology.orchestra.run.smithi171.stdout: "map_full": 0, 2023-07-14T18:35:22.707 INFO:teuthology.orchestra.run.smithi171.stdout: "map_inc": 2, 2023-07-14T18:35:22.707 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_sessions": 12, 2023-07-14T18:35:22.707 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_open": 12, 2023-07-14T18:35:22.707 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_close": 0, 2023-07-14T18:35:22.707 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_laggy": 0, 2023-07-14T18:35:22.707 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_wr": 187, 2023-07-14T18:35:22.707 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_rd": 0, 2023-07-14T18:35:22.708 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_del": 61 2023-07-14T18:35:22.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.708 INFO:teuthology.orchestra.run.smithi171.stdout: "oft": { 2023-07-14T18:35:22.708 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_objs": 1, 2023-07-14T18:35:22.708 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_kv_pairs": 0, 2023-07-14T18:35:22.708 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_updates": 0, 2023-07-14T18:35:22.708 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_removes": 0 2023-07-14T18:35:22.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.709 INFO:teuthology.orchestra.run.smithi171.stdout: "osd-slow-ops": { 2023-07-14T18:35:22.709 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_ops_count": 0 2023-07-14T18:35:22.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.709 INFO:teuthology.orchestra.run.smithi171.stdout: "purge_queue": { 2023-07-14T18:35:22.709 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops": 0, 2023-07-14T18:35:22.709 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops_high_water": 16, 2023-07-14T18:35:22.709 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing": 0, 2023-07-14T18:35:22.709 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_high_water": 8, 2023-07-14T18:35:22.710 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed": 236, 2023-07-14T18:35:22.710 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_item_in_journal": 0 2023-07-14T18:35:22.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.710 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T18:35:22.710 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2023-07-14T18:35:22.710 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2023-07-14T18:35:22.710 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2023-07-14T18:35:22.710 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 6248862, 2023-07-14T18:35:22.711 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 452583331, 2023-07-14T18:35:22.711 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:22.711 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 6248862, 2023-07-14T18:35:22.711 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2023-07-14T18:35:22.711 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2023-07-14T18:35:22.711 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 6248862, 2023-07-14T18:35:22.711 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 452583331, 2023-07-14T18:35:22.711 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2023-07-14T18:35:22.712 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.712 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.712 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.712 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:22.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.712 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_bytes": { 2023-07-14T18:35:22.712 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2023-07-14T18:35:22.712 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2023-07-14T18:35:22.713 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2023-07-14T18:35:22.713 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2023-07-14T18:35:22.713 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2023-07-14T18:35:22.713 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:22.713 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2023-07-14T18:35:22.713 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 1873, 2023-07-14T18:35:22.713 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 4790322, 2023-07-14T18:35:22.713 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 1873, 2023-07-14T18:35:22.714 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 4790322, 2023-07-14T18:35:22.714 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2023-07-14T18:35:22.714 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.714 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.714 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.714 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:22.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.715 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_ops": { 2023-07-14T18:35:22.715 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2023-07-14T18:35:22.715 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 1024, 2023-07-14T18:35:22.715 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2023-07-14T18:35:22.715 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2023-07-14T18:35:22.715 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2023-07-14T18:35:22.715 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:22.715 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2023-07-14T18:35:22.716 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 1873, 2023-07-14T18:35:22.716 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 1873, 2023-07-14T18:35:22.716 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 1873, 2023-07-14T18:35:22.716 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 1873, 2023-07-14T18:35:22.716 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2023-07-14T18:35:22.716 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.716 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.716 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.717 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:22.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.717 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle": { 2023-07-14T18:35:22.717 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2023-07-14T18:35:22.717 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2023-07-14T18:35:22.717 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2023-07-14T18:35:22.717 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 236, 2023-07-14T18:35:22.717 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 25724, 2023-07-14T18:35:22.718 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:22.718 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 236, 2023-07-14T18:35:22.718 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2023-07-14T18:35:22.718 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2023-07-14T18:35:22.718 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 102, 2023-07-14T18:35:22.718 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 25724, 2023-07-14T18:35:22.718 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2023-07-14T18:35:22.718 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.719 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.719 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.719 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:22.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:22.719 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle-0x5640cdee60a0": { 2023-07-14T18:35:22.719 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2023-07-14T18:35:22.719 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2023-07-14T18:35:22.719 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2023-07-14T18:35:22.720 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 2975, 2023-07-14T18:35:22.720 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 4642005, 2023-07-14T18:35:22.720 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:22.720 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 2975, 2023-07-14T18:35:22.720 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2023-07-14T18:35:22.720 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2023-07-14T18:35:22.720 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 638, 2023-07-14T18:35:22.720 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 4642005, 2023-07-14T18:35:22.721 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2023-07-14T18:35:22.721 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:22.721 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:22.721 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:22.721 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:22.721 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:22.721 INFO:teuthology.orchestra.run.smithi171.stdout:} 2023-07-14T18:35:22.880 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:22 smithi133 ceph-mon[126384]: pgmap v908: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2023-07-14T18:35:22.889 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:22 smithi171 ceph-mon[129241]: pgmap v908: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2023-07-14T18:35:22.939 INFO:tasks.check_counter:Daemon mds.f mds.exported=4 2023-07-14T18:35:22.940 INFO:tasks.check_counter:Daemon mds.f mds.imported=23 2023-07-14T18:35:22.940 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=98 2023-07-14T18:35:22.940 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=304 2023-07-14T18:35:22.941 DEBUG:tasks.check_counter:Getting stats from i 2023-07-14T18:35:22.941 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph daemon mds.i perf dump 2023-07-14T18:35:23.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:22 smithi161 ceph-mon[128993]: pgmap v908: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2023-07-14T18:35:23.538 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T18:35:24.813 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2023-07-14T18:35:24.813 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T18:35:24.813 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 4081, 2023-07-14T18:35:24.813 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 4081, 2023-07-14T18:35:24.813 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 1007816, 2023-07-14T18:35:24.814 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 11788789, 2023-07-14T18:35:24.814 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 14, 2023-07-14T18:35:24.814 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 6, 2023-07-14T18:35:24.814 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.348585384, 2023-07-14T18:35:24.814 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.116313285, 2023-07-14T18:35:24.814 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.068767921, 2023-07-14T18:35:24.814 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.140702628, 2023-07-14T18:35:24.814 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:35:24.815 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 4078, 2023-07-14T18:35:24.815 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.435063788, 2023-07-14T18:35:24.815 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000106685 2023-07-14T18:35:24.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.815 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:35:24.815 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 4, 2023-07-14T18:35:24.815 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000304, 2023-07-14T18:35:24.815 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000076 2023-07-14T18:35:24.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.816 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-07-14T18:35:24.816 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 448 2023-07-14T18:35:24.816 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.816 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T18:35:24.816 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 12861, 2023-07-14T18:35:24.816 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 14467, 2023-07-14T18:35:24.817 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 3556616, 2023-07-14T18:35:24.817 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 8368405, 2023-07-14T18:35:24.817 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 13, 2023-07-14T18:35:24.817 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 6, 2023-07-14T18:35:24.817 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.619254733, 2023-07-14T18:35:24.817 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.280544528, 2023-07-14T18:35:24.817 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.473034035, 2023-07-14T18:35:24.817 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.001319760, 2023-07-14T18:35:24.818 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:35:24.818 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 14465, 2023-07-14T18:35:24.818 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 1.561825393, 2023-07-14T18:35:24.818 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000107972 2023-07-14T18:35:24.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.818 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:35:24.818 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 12587, 2023-07-14T18:35:24.818 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.004046673, 2023-07-14T18:35:24.819 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000321 2023-07-14T18:35:24.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.819 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 2192, 2023-07-14T18:35:24.819 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 610720 2023-07-14T18:35:24.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.819 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T18:35:24.819 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 11035, 2023-07-14T18:35:24.819 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 14585, 2023-07-14T18:35:24.820 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 3153543, 2023-07-14T18:35:24.820 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 4365322, 2023-07-14T18:35:24.820 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 7, 2023-07-14T18:35:24.820 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 5, 2023-07-14T18:35:24.820 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.685238506, 2023-07-14T18:35:24.820 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.267805437, 2023-07-14T18:35:24.820 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.471607524, 2023-07-14T18:35:24.821 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.026562792, 2023-07-14T18:35:24.821 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:35:24.821 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 14586, 2023-07-14T18:35:24.821 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 1.927938903, 2023-07-14T18:35:24.821 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000132177 2023-07-14T18:35:24.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.821 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:35:24.821 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 23474, 2023-07-14T18:35:24.822 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.004306013, 2023-07-14T18:35:24.822 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000183 2023-07-14T18:35:24.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.822 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 635264, 2023-07-14T18:35:24.822 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 203616 2023-07-14T18:35:24.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.822 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2023-07-14T18:35:24.822 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 1, 2023-07-14T18:35:24.823 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2023-07-14T18:35:24.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.823 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-MDSRank": { 2023-07-14T18:35:24.823 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2023-07-14T18:35:24.823 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2023-07-14T18:35:24.823 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 7768, 2023-07-14T18:35:24.823 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 5.666478628, 2023-07-14T18:35:24.823 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000729464 2023-07-14T18:35:24.824 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:24.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.824 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-PurgeQueue": { 2023-07-14T18:35:24.824 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2023-07-14T18:35:24.824 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2023-07-14T18:35:24.824 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 269, 2023-07-14T18:35:24.824 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.008132213, 2023-07-14T18:35:24.824 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000030231 2023-07-14T18:35:24.825 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:24.825 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.825 INFO:teuthology.orchestra.run.smithi171.stdout: "mds": { 2023-07-14T18:35:24.825 INFO:teuthology.orchestra.run.smithi171.stdout: "request": 9765, 2023-07-14T18:35:24.825 INFO:teuthology.orchestra.run.smithi171.stdout: "reply": 9721, 2023-07-14T18:35:24.825 INFO:teuthology.orchestra.run.smithi171.stdout: "reply_latency": { 2023-07-14T18:35:24.825 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 9721, 2023-07-14T18:35:24.825 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 68.706322396, 2023-07-14T18:35:24.826 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.007067824 2023-07-14T18:35:24.826 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.826 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_reply": 0, 2023-07-14T18:35:24.826 INFO:teuthology.orchestra.run.smithi171.stdout: "forward": 44, 2023-07-14T18:35:24.826 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_complete": 0, 2023-07-14T18:35:24.826 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_keys": 0, 2023-07-14T18:35:24.826 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_commit": 11, 2023-07-14T18:35:24.827 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_split": 0, 2023-07-14T18:35:24.827 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_merge": 0, 2023-07-14T18:35:24.827 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes": 1537, 2023-07-14T18:35:24.827 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_top": 11, 2023-07-14T18:35:24.827 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_bottom": 1347, 2023-07-14T18:35:24.827 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pin_tail": 179, 2023-07-14T18:35:24.827 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pinned": 1520, 2023-07-14T18:35:24.827 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_expired": 282, 2023-07-14T18:35:24.828 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_with_caps": 70, 2023-07-14T18:35:24.828 INFO:teuthology.orchestra.run.smithi171.stdout: "caps": 70, 2023-07-14T18:35:24.828 INFO:teuthology.orchestra.run.smithi171.stdout: "subtrees": 15, 2023-07-14T18:35:24.828 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse": 51619, 2023-07-14T18:35:24.828 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_hit": 37263, 2023-07-14T18:35:24.828 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_forward": 38, 2023-07-14T18:35:24.828 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_discover": 71, 2023-07-14T18:35:24.828 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_dir_fetch": 0, 2023-07-14T18:35:24.829 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_remote_ino": 0, 2023-07-14T18:35:24.829 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_lock": 11840, 2023-07-14T18:35:24.829 INFO:teuthology.orchestra.run.smithi171.stdout: "load_cent": 591, 2023-07-14T18:35:24.829 INFO:teuthology.orchestra.run.smithi171.stdout: "q": 0, 2023-07-14T18:35:24.829 INFO:teuthology.orchestra.run.smithi171.stdout: "exported": 8, 2023-07-14T18:35:24.829 INFO:teuthology.orchestra.run.smithi171.stdout: "exported_inodes": 201, 2023-07-14T18:35:24.829 INFO:teuthology.orchestra.run.smithi171.stdout: "imported": 9, 2023-07-14T18:35:24.829 INFO:teuthology.orchestra.run.smithi171.stdout: "imported_inodes": 210, 2023-07-14T18:35:24.830 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_dir_fetch": 0, 2023-07-14T18:35:24.830 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_backtrace_fetch": 180, 2023-07-14T18:35:24.830 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_peer_discover": 0, 2023-07-14T18:35:24.830 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rfiles": 1047, 2023-07-14T18:35:24.830 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rbytes": 9049017265, 2023-07-14T18:35:24.830 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rsnaps": 0, 2023-07-14T18:35:24.830 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_fetch": 180, 2023-07-14T18:35:24.831 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_set_tag": 0, 2023-07-14T18:35:24.831 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_repaired": 0, 2023-07-14T18:35:24.831 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_inotable_repaired": 0, 2023-07-14T18:35:24.831 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_inodes": 5, 2023-07-14T18:35:24.831 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_base_inodes": 0, 2023-07-14T18:35:24.831 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dirfrag_rstats": 5, 2023-07-14T18:35:24.832 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_file_inodes": 175, 2023-07-14T18:35:24.832 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_inode_file_caps": 26, 2023-07-14T18:35:24.832 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_revoke": 626, 2023-07-14T18:35:24.832 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_grant": 1581, 2023-07-14T18:35:24.832 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_trunc": 0, 2023-07-14T18:35:24.832 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-14T18:35:24.832 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-14T18:35:24.832 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps": 2579, 2023-07-14T18:35:24.833 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps_dirty": 1575, 2023-07-14T18:35:24.833 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_cap_release": 142, 2023-07-14T18:35:24.833 INFO:teuthology.orchestra.run.smithi171.stdout: "process_request_cap_release": 4304 2023-07-14T18:35:24.833 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.833 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache": { 2023-07-14T18:35:24.833 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update": 95, 2023-07-14T18:35:24.833 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update_receipt": 339, 2023-07-14T18:35:24.833 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_try_discover": 183, 2023-07-14T18:35:24.834 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_send_discover": 332, 2023-07-14T18:35:24.834 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_handle_discover": 138, 2023-07-14T18:35:24.834 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays": 0, 2023-07-14T18:35:24.834 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_delayed": 0, 2023-07-14T18:35:24.834 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_enqueuing": 0, 2023-07-14T18:35:24.834 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_created": 1059, 2023-07-14T18:35:24.834 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_enqueued": 26, 2023-07-14T18:35:24.834 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_reintegrated": 4170, 2023-07-14T18:35:24.835 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_migrated": 0, 2023-07-14T18:35:24.835 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_processing": 0, 2023-07-14T18:35:24.835 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_enqueued": 0, 2023-07-14T18:35:24.835 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_prioritized": 0, 2023-07-14T18:35:24.835 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_started": 0, 2023-07-14T18:35:24.835 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_completed": 0, 2023-07-14T18:35:24.835 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_enqueue_scrub": 0, 2023-07-14T18:35:24.835 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_exportdir": 8, 2023-07-14T18:35:24.836 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_flush": 0, 2023-07-14T18:35:24.836 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragmentdir": 0, 2023-07-14T18:35:24.836 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragstats": 0, 2023-07-14T18:35:24.836 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_inodestats": 0 2023-07-14T18:35:24.836 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.836 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log": { 2023-07-14T18:35:24.836 INFO:teuthology.orchestra.run.smithi171.stdout: "evadd": 5734, 2023-07-14T18:35:24.837 INFO:teuthology.orchestra.run.smithi171.stdout: "evex": 0, 2023-07-14T18:35:24.837 INFO:teuthology.orchestra.run.smithi171.stdout: "evtrm": 0, 2023-07-14T18:35:24.837 INFO:teuthology.orchestra.run.smithi171.stdout: "ev": 5734, 2023-07-14T18:35:24.837 INFO:teuthology.orchestra.run.smithi171.stdout: "evexg": 0, 2023-07-14T18:35:24.837 INFO:teuthology.orchestra.run.smithi171.stdout: "evexd": 0, 2023-07-14T18:35:24.837 INFO:teuthology.orchestra.run.smithi171.stdout: "segadd": 8, 2023-07-14T18:35:24.837 INFO:teuthology.orchestra.run.smithi171.stdout: "segex": 0, 2023-07-14T18:35:24.837 INFO:teuthology.orchestra.run.smithi171.stdout: "segtrm": 0, 2023-07-14T18:35:24.838 INFO:teuthology.orchestra.run.smithi171.stdout: "seg": 8, 2023-07-14T18:35:24.838 INFO:teuthology.orchestra.run.smithi171.stdout: "segexg": 0, 2023-07-14T18:35:24.838 INFO:teuthology.orchestra.run.smithi171.stdout: "segexd": 0, 2023-07-14T18:35:24.838 INFO:teuthology.orchestra.run.smithi171.stdout: "expos": 4194304, 2023-07-14T18:35:24.838 INFO:teuthology.orchestra.run.smithi171.stdout: "wrpos": 14667491, 2023-07-14T18:35:24.838 INFO:teuthology.orchestra.run.smithi171.stdout: "rdpos": 0, 2023-07-14T18:35:24.838 INFO:teuthology.orchestra.run.smithi171.stdout: "jlat": { 2023-07-14T18:35:24.838 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 3836, 2023-07-14T18:35:24.839 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 39.866964359, 2023-07-14T18:35:24.839 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.010392847 2023-07-14T18:35:24.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.839 INFO:teuthology.orchestra.run.smithi171.stdout: "replayed": 0 2023-07-14T18:35:24.839 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.839 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_mem": { 2023-07-14T18:35:24.839 INFO:teuthology.orchestra.run.smithi171.stdout: "ino": 109, 2023-07-14T18:35:24.839 INFO:teuthology.orchestra.run.smithi171.stdout: "ino+": 343, 2023-07-14T18:35:24.840 INFO:teuthology.orchestra.run.smithi171.stdout: "ino-": 234, 2023-07-14T18:35:24.840 INFO:teuthology.orchestra.run.smithi171.stdout: "dir": 54, 2023-07-14T18:35:24.840 INFO:teuthology.orchestra.run.smithi171.stdout: "dir+": 193, 2023-07-14T18:35:24.840 INFO:teuthology.orchestra.run.smithi171.stdout: "dir-": 139, 2023-07-14T18:35:24.840 INFO:teuthology.orchestra.run.smithi171.stdout: "dn": 1537, 2023-07-14T18:35:24.840 INFO:teuthology.orchestra.run.smithi171.stdout: "dn+": 1845, 2023-07-14T18:35:24.840 INFO:teuthology.orchestra.run.smithi171.stdout: "dn-": 308, 2023-07-14T18:35:24.840 INFO:teuthology.orchestra.run.smithi171.stdout: "cap": 70, 2023-07-14T18:35:24.841 INFO:teuthology.orchestra.run.smithi171.stdout: "cap+": 412, 2023-07-14T18:35:24.841 INFO:teuthology.orchestra.run.smithi171.stdout: "cap-": 342, 2023-07-14T18:35:24.841 INFO:teuthology.orchestra.run.smithi171.stdout: "rss": 56984, 2023-07-14T18:35:24.841 INFO:teuthology.orchestra.run.smithi171.stdout: "heap": 198960 2023-07-14T18:35:24.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.841 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_server": { 2023-07-14T18:35:24.841 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_client_request": 31694, 2023-07-14T18:35:24.841 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_server_request": 109, 2023-07-14T18:35:24.842 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_request": 9765, 2023-07-14T18:35:24.842 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_session": 1918, 2023-07-14T18:35:24.842 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_peer_request": 276, 2023-07-14T18:35:24.842 INFO:teuthology.orchestra.run.smithi171.stdout: "req_create_latency": { 2023-07-14T18:35:24.842 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 83, 2023-07-14T18:35:24.842 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.104715173, 2023-07-14T18:35:24.842 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.001261628 2023-07-14T18:35:24.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.843 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getattr_latency": { 2023-07-14T18:35:24.843 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 12, 2023-07-14T18:35:24.843 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.040382441, 2023-07-14T18:35:24.843 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.003365203 2023-07-14T18:35:24.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.843 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getfilelock_latency": { 2023-07-14T18:35:24.843 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:24.844 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:24.844 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:24.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.844 INFO:teuthology.orchestra.run.smithi171.stdout: "req_link_latency": { 2023-07-14T18:35:24.844 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1033, 2023-07-14T18:35:24.844 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 2.811456632, 2023-07-14T18:35:24.844 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.002721642 2023-07-14T18:35:24.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.845 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookup_latency": { 2023-07-14T18:35:24.845 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 3128, 2023-07-14T18:35:24.845 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 6.881716396, 2023-07-14T18:35:24.845 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.002200037 2023-07-14T18:35:24.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.845 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookuphash_latency": { 2023-07-14T18:35:24.845 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:24.845 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:24.846 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:24.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.846 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupino_latency": { 2023-07-14T18:35:24.846 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:24.846 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:24.846 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:24.846 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.846 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupname_latency": { 2023-07-14T18:35:24.847 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:24.847 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:24.847 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:24.847 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.847 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupparent_latency": { 2023-07-14T18:35:24.847 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:24.847 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:24.847 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:24.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.848 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupsnap_latency": { 2023-07-14T18:35:24.848 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:24.848 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:24.848 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:24.848 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.848 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lssnap_latency": { 2023-07-14T18:35:24.848 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:24.849 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:24.849 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:24.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.849 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mkdir_latency": { 2023-07-14T18:35:24.849 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:24.849 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:24.849 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:24.849 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.850 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mknod_latency": { 2023-07-14T18:35:24.850 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:24.850 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:24.850 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:24.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.850 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mksnap_latency": { 2023-07-14T18:35:24.850 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:24.850 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:24.851 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:24.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.851 INFO:teuthology.orchestra.run.smithi171.stdout: "req_open_latency": { 2023-07-14T18:35:24.851 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 194, 2023-07-14T18:35:24.851 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.391200307, 2023-07-14T18:35:24.851 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.002016496 2023-07-14T18:35:24.851 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.852 INFO:teuthology.orchestra.run.smithi171.stdout: "req_readdir_latency": { 2023-07-14T18:35:24.852 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 42, 2023-07-14T18:35:24.852 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.089343085, 2023-07-14T18:35:24.852 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.002127216 2023-07-14T18:35:24.852 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.852 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rename_latency": { 2023-07-14T18:35:24.852 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 4173, 2023-07-14T18:35:24.852 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 55.614166383, 2023-07-14T18:35:24.853 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.013327142 2023-07-14T18:35:24.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.853 INFO:teuthology.orchestra.run.smithi171.stdout: "req_renamesnap_latency": { 2023-07-14T18:35:24.853 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:24.853 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:24.853 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:24.853 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.853 INFO:teuthology.orchestra.run.smithi171.stdout: "req_snapdiff_latency": { 2023-07-14T18:35:24.854 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:24.854 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:24.854 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:24.854 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.854 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmdir_latency": { 2023-07-14T18:35:24.854 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:24.854 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:24.854 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:24.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.855 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmsnap_latency": { 2023-07-14T18:35:24.855 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:24.855 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:24.855 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:24.855 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.855 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmxattr_latency": { 2023-07-14T18:35:24.855 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:24.856 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:24.856 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:24.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.856 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setattr_latency": { 2023-07-14T18:35:24.856 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:24.856 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:24.856 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:24.856 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.857 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setdirlayout_latency": { 2023-07-14T18:35:24.857 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:24.857 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:24.857 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:24.857 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.857 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setfilelock_latency": { 2023-07-14T18:35:24.857 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:24.858 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:24.858 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:24.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.858 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setlayout_latency": { 2023-07-14T18:35:24.858 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:24.858 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:24.858 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:24.858 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.859 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setxattr_latency": { 2023-07-14T18:35:24.859 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:24.859 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:24.859 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:24.859 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.859 INFO:teuthology.orchestra.run.smithi171.stdout: "req_symlink_latency": { 2023-07-14T18:35:24.859 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:24.859 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:24.860 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:24.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.860 INFO:teuthology.orchestra.run.smithi171.stdout: "req_unlink_latency": { 2023-07-14T18:35:24.860 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1056, 2023-07-14T18:35:24.860 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 2.773341979, 2023-07-14T18:35:24.860 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.002626270 2023-07-14T18:35:24.860 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.860 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_revoke_eviction": 0, 2023-07-14T18:35:24.861 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_acquisition_throttle": 0, 2023-07-14T18:35:24.861 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getvxattr_latency": { 2023-07-14T18:35:24.861 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:24.861 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:24.861 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:24.861 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:24.861 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.861 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_sessions": { 2023-07-14T18:35:24.862 INFO:teuthology.orchestra.run.smithi171.stdout: "session_count": 1, 2023-07-14T18:35:24.862 INFO:teuthology.orchestra.run.smithi171.stdout: "session_add": 2, 2023-07-14T18:35:24.862 INFO:teuthology.orchestra.run.smithi171.stdout: "session_remove": 1, 2023-07-14T18:35:24.862 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_open": 1, 2023-07-14T18:35:24.862 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_stale": 0, 2023-07-14T18:35:24.862 INFO:teuthology.orchestra.run.smithi171.stdout: "total_load": 464, 2023-07-14T18:35:24.862 INFO:teuthology.orchestra.run.smithi171.stdout: "average_load": 464, 2023-07-14T18:35:24.862 INFO:teuthology.orchestra.run.smithi171.stdout: "avg_session_uptime": 1138 2023-07-14T18:35:24.863 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.863 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2023-07-14T18:35:24.863 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2023-07-14T18:35:24.863 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2023-07-14T18:35:24.863 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T18:35:24.863 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2023-07-14T18:35:24.863 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T18:35:24.863 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2023-07-14T18:35:24.864 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T18:35:24.864 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T18:35:24.864 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T18:35:24.864 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T18:35:24.864 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T18:35:24.864 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2023-07-14T18:35:24.864 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T18:35:24.864 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T18:35:24.865 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2023-07-14T18:35:24.865 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2023-07-14T18:35:24.865 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2023-07-14T18:35:24.865 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2023-07-14T18:35:24.865 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T18:35:24.865 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T18:35:24.865 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T18:35:24.865 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T18:35:24.866 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T18:35:24.866 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2023-07-14T18:35:24.866 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2023-07-14T18:35:24.866 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2023-07-14T18:35:24.866 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T18:35:24.866 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T18:35:24.866 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2023-07-14T18:35:24.866 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2023-07-14T18:35:24.867 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2023-07-14T18:35:24.867 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2023-07-14T18:35:24.867 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T18:35:24.867 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2023-07-14T18:35:24.867 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T18:35:24.867 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2023-07-14T18:35:24.867 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 285178, 2023-07-14T18:35:24.867 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 186, 2023-07-14T18:35:24.868 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2023-07-14T18:35:24.868 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2023-07-14T18:35:24.868 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2023-07-14T18:35:24.868 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2023-07-14T18:35:24.868 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2023-07-14T18:35:24.868 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2023-07-14T18:35:24.868 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2023-07-14T18:35:24.869 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2023-07-14T18:35:24.869 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 8792, 2023-07-14T18:35:24.869 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 168, 2023-07-14T18:35:24.869 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T18:35:24.869 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2023-07-14T18:35:24.869 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2023-07-14T18:35:24.869 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2023-07-14T18:35:24.869 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 1542908, 2023-07-14T18:35:24.870 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 51618, 2023-07-14T18:35:24.870 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2023-07-14T18:35:24.870 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2023-07-14T18:35:24.870 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2023-07-14T18:35:24.870 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2023-07-14T18:35:24.870 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.870 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter": { 2023-07-14T18:35:24.870 INFO:teuthology.orchestra.run.smithi171.stdout: "op_active": 0, 2023-07-14T18:35:24.871 INFO:teuthology.orchestra.run.smithi171.stdout: "op_laggy": 0, 2023-07-14T18:35:24.871 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send": 4474, 2023-07-14T18:35:24.871 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send_bytes": 10600177, 2023-07-14T18:35:24.871 INFO:teuthology.orchestra.run.smithi171.stdout: "op_resend": 27, 2023-07-14T18:35:24.871 INFO:teuthology.orchestra.run.smithi171.stdout: "op_reply": 4474, 2023-07-14T18:35:24.871 INFO:teuthology.orchestra.run.smithi171.stdout: "op_latency": { 2023-07-14T18:35:24.871 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 4474, 2023-07-14T18:35:24.871 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 66.555817626, 2023-07-14T18:35:24.872 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.014876132 2023-07-14T18:35:24.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.872 INFO:teuthology.orchestra.run.smithi171.stdout: "op_inflight": 0, 2023-07-14T18:35:24.872 INFO:teuthology.orchestra.run.smithi171.stdout: "oplen_avg": { 2023-07-14T18:35:24.872 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 4474, 2023-07-14T18:35:24.872 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 4699 2023-07-14T18:35:24.872 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.872 INFO:teuthology.orchestra.run.smithi171.stdout: "op": 4474, 2023-07-14T18:35:24.873 INFO:teuthology.orchestra.run.smithi171.stdout: "op_r": 205, 2023-07-14T18:35:24.873 INFO:teuthology.orchestra.run.smithi171.stdout: "op_w": 4269, 2023-07-14T18:35:24.873 INFO:teuthology.orchestra.run.smithi171.stdout: "op_rmw": 0, 2023-07-14T18:35:24.873 INFO:teuthology.orchestra.run.smithi171.stdout: "op_pg": 0, 2023-07-14T18:35:24.873 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_stat": 0, 2023-07-14T18:35:24.873 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_create": 10, 2023-07-14T18:35:24.873 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_read": 25, 2023-07-14T18:35:24.873 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_write": 3861, 2023-07-14T18:35:24.874 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writefull": 97, 2023-07-14T18:35:24.874 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writesame": 0, 2023-07-14T18:35:24.874 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_append": 0, 2023-07-14T18:35:24.874 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_zero": 0, 2023-07-14T18:35:24.874 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_truncate": 0, 2023-07-14T18:35:24.874 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_delete": 140, 2023-07-14T18:35:24.874 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_mapext": 0, 2023-07-14T18:35:24.874 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_sparse_read": 0, 2023-07-14T18:35:24.875 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_clonerange": 0, 2023-07-14T18:35:24.875 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_getxattr": 180, 2023-07-14T18:35:24.875 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_setxattr": 20, 2023-07-14T18:35:24.875 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_cmpxattr": 0, 2023-07-14T18:35:24.875 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_rmxattr": 0, 2023-07-14T18:35:24.875 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_resetxattrs": 0, 2023-07-14T18:35:24.875 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_call": 0, 2023-07-14T18:35:24.875 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_watch": 0, 2023-07-14T18:35:24.876 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_notify": 0, 2023-07-14T18:35:24.876 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_src_cmpxattr": 0, 2023-07-14T18:35:24.876 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls": 0, 2023-07-14T18:35:24.876 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls_filter": 0, 2023-07-14T18:35:24.876 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_other": 0, 2023-07-14T18:35:24.876 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_active": 0, 2023-07-14T18:35:24.876 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_send": 0, 2023-07-14T18:35:24.876 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_resend": 0, 2023-07-14T18:35:24.877 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_ping": 0, 2023-07-14T18:35:24.877 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_active": 0, 2023-07-14T18:35:24.877 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_send": 0, 2023-07-14T18:35:24.877 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_resend": 0, 2023-07-14T18:35:24.877 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_active": 0, 2023-07-14T18:35:24.877 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_send": 0, 2023-07-14T18:35:24.877 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_resend": 0, 2023-07-14T18:35:24.877 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_active": 0, 2023-07-14T18:35:24.878 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_send": 0, 2023-07-14T18:35:24.878 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_resend": 0, 2023-07-14T18:35:24.878 INFO:teuthology.orchestra.run.smithi171.stdout: "command_active": 0, 2023-07-14T18:35:24.878 INFO:teuthology.orchestra.run.smithi171.stdout: "command_send": 0, 2023-07-14T18:35:24.878 INFO:teuthology.orchestra.run.smithi171.stdout: "command_resend": 0, 2023-07-14T18:35:24.878 INFO:teuthology.orchestra.run.smithi171.stdout: "map_epoch": 92, 2023-07-14T18:35:24.878 INFO:teuthology.orchestra.run.smithi171.stdout: "map_full": 0, 2023-07-14T18:35:24.878 INFO:teuthology.orchestra.run.smithi171.stdout: "map_inc": 3, 2023-07-14T18:35:24.879 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_sessions": 12, 2023-07-14T18:35:24.879 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_open": 12, 2023-07-14T18:35:24.879 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_close": 0, 2023-07-14T18:35:24.879 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_laggy": 0, 2023-07-14T18:35:24.879 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_wr": 243, 2023-07-14T18:35:24.879 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_rd": 0, 2023-07-14T18:35:24.879 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_del": 123 2023-07-14T18:35:24.879 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.880 INFO:teuthology.orchestra.run.smithi171.stdout: "oft": { 2023-07-14T18:35:24.880 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_objs": 1, 2023-07-14T18:35:24.880 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_kv_pairs": 0, 2023-07-14T18:35:24.880 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_updates": 0, 2023-07-14T18:35:24.880 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_removes": 0 2023-07-14T18:35:24.880 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.880 INFO:teuthology.orchestra.run.smithi171.stdout: "osd-slow-ops": { 2023-07-14T18:35:24.881 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_ops_count": 0 2023-07-14T18:35:24.881 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.881 INFO:teuthology.orchestra.run.smithi171.stdout: "purge_queue": { 2023-07-14T18:35:24.881 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops": 0, 2023-07-14T18:35:24.881 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops_high_water": 15, 2023-07-14T18:35:24.881 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing": 0, 2023-07-14T18:35:24.881 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_high_water": 3, 2023-07-14T18:35:24.881 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed": 26, 2023-07-14T18:35:24.882 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_item_in_journal": 0 2023-07-14T18:35:24.882 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.882 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T18:35:24.882 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2023-07-14T18:35:24.882 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2023-07-14T18:35:24.882 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2023-07-14T18:35:24.882 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 27973, 2023-07-14T18:35:24.882 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 5368590, 2023-07-14T18:35:24.883 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:24.883 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 27973, 2023-07-14T18:35:24.883 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2023-07-14T18:35:24.883 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2023-07-14T18:35:24.883 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 27973, 2023-07-14T18:35:24.883 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 5368590, 2023-07-14T18:35:24.883 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2023-07-14T18:35:24.883 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:24.884 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:24.884 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:24.884 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:24.884 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.884 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_bytes": { 2023-07-14T18:35:24.884 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2023-07-14T18:35:24.884 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2023-07-14T18:35:24.884 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2023-07-14T18:35:24.885 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2023-07-14T18:35:24.885 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2023-07-14T18:35:24.885 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:24.885 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2023-07-14T18:35:24.885 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 4474, 2023-07-14T18:35:24.885 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 10603011, 2023-07-14T18:35:24.885 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 4474, 2023-07-14T18:35:24.885 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 10603011, 2023-07-14T18:35:24.886 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2023-07-14T18:35:24.886 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:24.886 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:24.886 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:24.886 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:24.886 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.886 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_ops": { 2023-07-14T18:35:24.886 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2023-07-14T18:35:24.887 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 1024, 2023-07-14T18:35:24.887 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2023-07-14T18:35:24.887 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2023-07-14T18:35:24.887 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2023-07-14T18:35:24.887 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:24.887 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2023-07-14T18:35:24.887 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 4474, 2023-07-14T18:35:24.887 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 4474, 2023-07-14T18:35:24.888 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 4474, 2023-07-14T18:35:24.888 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 4474, 2023-07-14T18:35:24.888 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2023-07-14T18:35:24.888 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:24.888 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:24.888 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:24.888 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:24.889 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.889 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle": { 2023-07-14T18:35:24.889 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2023-07-14T18:35:24.889 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2023-07-14T18:35:24.889 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2023-07-14T18:35:24.889 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 26, 2023-07-14T18:35:24.889 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 2834, 2023-07-14T18:35:24.889 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:24.890 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 26, 2023-07-14T18:35:24.890 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2023-07-14T18:35:24.890 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2023-07-14T18:35:24.890 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 25, 2023-07-14T18:35:24.890 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 2834, 2023-07-14T18:35:24.890 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2023-07-14T18:35:24.890 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:24.890 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:24.891 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:24.891 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:24.891 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:24.891 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle-0x5634962d80a0": { 2023-07-14T18:35:24.891 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2023-07-14T18:35:24.891 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2023-07-14T18:35:24.891 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2023-07-14T18:35:24.891 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 5734, 2023-07-14T18:35:24.892 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 10473187, 2023-07-14T18:35:24.892 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:24.892 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 5734, 2023-07-14T18:35:24.892 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2023-07-14T18:35:24.892 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2023-07-14T18:35:24.892 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 3836, 2023-07-14T18:35:24.892 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 10473187, 2023-07-14T18:35:24.892 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2023-07-14T18:35:24.893 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:24.893 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:24.893 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:24.893 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:24.893 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:24.893 INFO:teuthology.orchestra.run.smithi171.stdout:} 2023-07-14T18:35:24.894 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:24 smithi133 ceph-mon[126384]: pgmap v909: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.4 KiB/s wr, 6 op/s 2023-07-14T18:35:24.950 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:24 smithi171 ceph-mon[129241]: pgmap v909: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.4 KiB/s wr, 6 op/s 2023-07-14T18:35:25.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:24 smithi161 ceph-mon[128993]: pgmap v909: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.4 KiB/s wr, 6 op/s 2023-07-14T18:35:25.206 INFO:tasks.check_counter:Daemon mds.i mds.exported=8 2023-07-14T18:35:25.206 INFO:tasks.check_counter:Daemon mds.i mds.imported=9 2023-07-14T18:35:25.206 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=95 2023-07-14T18:35:25.207 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=339 2023-07-14T18:35:25.207 DEBUG:tasks.check_counter:Getting stats from l 2023-07-14T18:35:25.207 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c -- ceph daemon mds.l perf dump 2023-07-14T18:35:25.776 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T18:35:26.880 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:26 smithi133 ceph-mon[126384]: pgmap v910: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.2 KiB/s wr, 6 op/s 2023-07-14T18:35:27.071 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:26 smithi171 ceph-mon[129241]: pgmap v910: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.2 KiB/s wr, 6 op/s 2023-07-14T18:35:27.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:26 smithi161 ceph-mon[128993]: pgmap v910: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.2 KiB/s wr, 6 op/s 2023-07-14T18:35:27.696 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2023-07-14T18:35:27.696 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T18:35:27.697 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 0, 2023-07-14T18:35:27.697 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 2, 2023-07-14T18:35:27.697 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 0, 2023-07-14T18:35:27.697 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 0, 2023-07-14T18:35:27.697 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 4, 2023-07-14T18:35:27.698 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 0, 2023-07-14T18:35:27.698 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.001069942, 2023-07-14T18:35:27.698 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.000000000, 2023-07-14T18:35:27.698 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.000203193, 2023-07-14T18:35:27.698 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T18:35:27.698 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:35:27.698 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:27.699 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:27.699 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:27.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:27.699 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:35:27.699 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:27.699 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:27.699 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:27.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:27.700 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-14T18:35:27.700 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 0 2023-07-14T18:35:27.700 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:27.700 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T18:35:27.700 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 6, 2023-07-14T18:35:27.700 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 267, 2023-07-14T18:35:27.700 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 3312, 2023-07-14T18:35:27.701 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 198832, 2023-07-14T18:35:27.701 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 4, 2023-07-14T18:35:27.701 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 2, 2023-07-14T18:35:27.701 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.012191169, 2023-07-14T18:35:27.701 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.008738515, 2023-07-14T18:35:27.701 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.001405834, 2023-07-14T18:35:27.701 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T18:35:27.701 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:35:27.702 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 266, 2023-07-14T18:35:27.702 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.121677622, 2023-07-14T18:35:27.702 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000457434 2023-07-14T18:35:27.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:27.702 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:35:27.702 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:27.702 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:27.702 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:27.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:27.703 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-07-14T18:35:27.703 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 198832 2023-07-14T18:35:27.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:27.703 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T18:35:27.703 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 351, 2023-07-14T18:35:27.703 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 348, 2023-07-14T18:35:27.703 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 509824, 2023-07-14T18:35:27.704 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 199088, 2023-07-14T18:35:27.704 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 2, 2023-07-14T18:35:27.704 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 1, 2023-07-14T18:35:27.704 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.047310049, 2023-07-14T18:35:27.704 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.017205385, 2023-07-14T18:35:27.704 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.018075028, 2023-07-14T18:35:27.704 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.010173500, 2023-07-14T18:35:27.704 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T18:35:27.705 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 347, 2023-07-14T18:35:27.705 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.109566895, 2023-07-14T18:35:27.705 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000315754 2023-07-14T18:35:27.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:27.705 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2023-07-14T18:35:27.705 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:27.705 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:27.705 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:27.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:27.706 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 509824, 2023-07-14T18:35:27.706 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 199088 2023-07-14T18:35:27.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:27.706 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2023-07-14T18:35:27.706 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 0, 2023-07-14T18:35:27.706 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2023-07-14T18:35:27.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:27.707 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2023-07-14T18:35:27.707 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2023-07-14T18:35:27.707 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2023-07-14T18:35:27.707 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T18:35:27.707 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2023-07-14T18:35:27.707 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T18:35:27.707 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2023-07-14T18:35:27.707 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T18:35:27.708 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T18:35:27.708 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T18:35:27.708 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T18:35:27.708 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T18:35:27.708 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2023-07-14T18:35:27.708 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T18:35:27.708 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T18:35:27.709 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2023-07-14T18:35:27.709 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2023-07-14T18:35:27.709 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2023-07-14T18:35:27.709 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2023-07-14T18:35:27.709 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T18:35:27.709 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T18:35:27.709 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T18:35:27.709 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T18:35:27.710 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T18:35:27.710 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2023-07-14T18:35:27.710 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2023-07-14T18:35:27.710 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2023-07-14T18:35:27.710 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T18:35:27.710 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T18:35:27.710 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2023-07-14T18:35:27.710 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2023-07-14T18:35:27.710 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2023-07-14T18:35:27.711 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2023-07-14T18:35:27.711 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T18:35:27.711 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2023-07-14T18:35:27.711 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T18:35:27.711 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2023-07-14T18:35:27.711 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 148744, 2023-07-14T18:35:27.711 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 30, 2023-07-14T18:35:27.711 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2023-07-14T18:35:27.712 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2023-07-14T18:35:27.712 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2023-07-14T18:35:27.712 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2023-07-14T18:35:27.712 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2023-07-14T18:35:27.712 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2023-07-14T18:35:27.713 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2023-07-14T18:35:27.713 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2023-07-14T18:35:27.713 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 0, 2023-07-14T18:35:27.713 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 0, 2023-07-14T18:35:27.713 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T18:35:27.713 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2023-07-14T18:35:27.713 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2023-07-14T18:35:27.713 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2023-07-14T18:35:27.714 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 928, 2023-07-14T18:35:27.714 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 2, 2023-07-14T18:35:27.714 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2023-07-14T18:35:27.714 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2023-07-14T18:35:27.714 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2023-07-14T18:35:27.714 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2023-07-14T18:35:27.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2023-07-14T18:35:27.714 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T18:35:27.715 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2023-07-14T18:35:27.715 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2023-07-14T18:35:27.715 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2023-07-14T18:35:27.715 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 353, 2023-07-14T18:35:27.715 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 460190, 2023-07-14T18:35:27.715 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2023-07-14T18:35:27.715 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 353, 2023-07-14T18:35:27.715 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2023-07-14T18:35:27.716 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2023-07-14T18:35:27.716 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 353, 2023-07-14T18:35:27.716 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 460190, 2023-07-14T18:35:27.716 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2023-07-14T18:35:27.716 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2023-07-14T18:35:27.716 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2023-07-14T18:35:27.716 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2023-07-14T18:35:27.716 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:27.717 INFO:teuthology.orchestra.run.smithi171.stdout: } 2023-07-14T18:35:27.717 INFO:teuthology.orchestra.run.smithi171.stdout:} 2023-07-14T18:35:28.042 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2023-07-14T18:35:28.043 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2023-07-14T18:35:28.043 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2023-07-14T18:35:28.043 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2023-07-14T18:35:28.043 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-07-14T18:35:28.053 INFO:tasks.kclient:Unmounting kernel clients... 2023-07-14T18:35:28.053 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T18:35:28.054 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T18:35:28.142 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T18:35:28.143 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T18:35:28.189 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T18:35:28.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.416+0000 7ff0ee3ff700 1 -- 172.21.15.133:0/3080445464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0e81017b0 msgr2=0x7ff0e8108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:28.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.416+0000 7ff0ee3ff700 1 --2- 172.21.15.133:0/3080445464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0e81017b0 0x7ff0e8108070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009f90 tx=0x7ff0e000b330 comp rx=0 tx=0).stop 2023-07-14T18:35:28.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.416+0000 7ff0ee3ff700 1 -- 172.21.15.133:0/3080445464 shutdown_connections 2023-07-14T18:35:28.436 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.416+0000 7ff0ee3ff700 1 --2- 172.21.15.133:0/3080445464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0e81017b0 0x7ff0e8108070 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:28.436 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.416+0000 7ff0ee3ff700 1 --2- 172.21.15.133:0/3080445464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0e80ffc30 0x7ff0e8100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:28.436 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.416+0000 7ff0ee3ff700 1 --2- 172.21.15.133:0/3080445464 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0e8100e20 0x7ff0e8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:28.436 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.416+0000 7ff0ee3ff700 1 -- 172.21.15.133:0/3080445464 >> 172.21.15.133:0/3080445464 conn(0x7ff0e80fb210 msgr2=0x7ff0e80fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:28.436 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.416+0000 7ff0ee3ff700 1 -- 172.21.15.133:0/3080445464 shutdown_connections 2023-07-14T18:35:28.436 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.416+0000 7ff0ee3ff700 1 -- 172.21.15.133:0/3080445464 wait complete. 2023-07-14T18:35:28.439 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.417+0000 7ff0ee3ff700 1 Processor -- start 2023-07-14T18:35:28.440 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.417+0000 7ff0ee3ff700 1 -- start start 2023-07-14T18:35:28.440 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.418+0000 7ff0ee3ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0e80ffc30 0x7ff0e80743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:28.440 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.418+0000 7ff0ee3ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0e8100e20 0x7ff0e8072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:28.441 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.418+0000 7ff0ee3ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0e81017b0 0x7ff0e8072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:28.441 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.418+0000 7ff0ee3ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e8073710 con 0x7ff0e80ffc30 2023-07-14T18:35:28.441 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.418+0000 7ff0ee3ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e8073880 con 0x7ff0e8100e20 2023-07-14T18:35:28.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.418+0000 7ff0ee3ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e819fa20 con 0x7ff0e81017b0 2023-07-14T18:35:28.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.418+0000 7ff0e7fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0e80ffc30 0x7ff0e80743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:28.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.418+0000 7ff0ec99c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0e81017b0 0x7ff0e8072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:28.443 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.418+0000 7ff0e77fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0e8100e20 0x7ff0e8072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:28.443 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.418+0000 7ff0ec99c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0e81017b0 0x7ff0e8072f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:56240/0 (socket says 172.21.15.133:56240) 2023-07-14T18:35:28.443 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.418+0000 7ff0ec99c700 1 -- 172.21.15.133:0/1710077089 learned_addr learned my addr 172.21.15.133:0/1710077089 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:35:28.443 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.418+0000 7ff0ec99c700 1 -- 172.21.15.133:0/1710077089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0e8100e20 msgr2=0x7ff0e8072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:28.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.418+0000 7ff0ec99c700 1 --2- 172.21.15.133:0/1710077089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0e8100e20 0x7ff0e8072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:28.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.418+0000 7ff0ec99c700 1 -- 172.21.15.133:0/1710077089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0e80ffc30 msgr2=0x7ff0e80743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:28.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.418+0000 7ff0ec99c700 1 --2- 172.21.15.133:0/1710077089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0e80ffc30 0x7ff0e80743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:28.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.418+0000 7ff0ec99c700 1 -- 172.21.15.133:0/1710077089 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d8009520 con 0x7ff0e81017b0 2023-07-14T18:35:28.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.419+0000 7ff0ec99c700 1 --2- 172.21.15.133:0/1710077089 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0e81017b0 0x7ff0e8072f70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0007930 tx=0x7ff0e0004ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:28.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.419+0000 7ff0e57fa700 1 -- 172.21.15.133:0/1710077089 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e000e040 con 0x7ff0e81017b0 2023-07-14T18:35:28.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.419+0000 7ff0e57fa700 1 -- 172.21.15.133:0/1710077089 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0e000bda0 con 0x7ff0e81017b0 2023-07-14T18:35:28.446 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.419+0000 7ff0ee3ff700 1 -- 172.21.15.133:0/1710077089 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0dc0095f0 con 0x7ff0e81017b0 2023-07-14T18:35:28.446 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.419+0000 7ff0e57fa700 1 -- 172.21.15.133:0/1710077089 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e00052a0 con 0x7ff0e81017b0 2023-07-14T18:35:28.446 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.419+0000 7ff0ee3ff700 1 -- 172.21.15.133:0/1710077089 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e0009c40 con 0x7ff0e81017b0 2023-07-14T18:35:28.446 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.420+0000 7ff0ee3ff700 1 -- 172.21.15.133:0/1710077089 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e8106a20 con 0x7ff0e81017b0 2023-07-14T18:35:28.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.422+0000 7ff0e57fa700 1 -- 172.21.15.133:0/1710077089 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff0e001d470 con 0x7ff0e81017b0 2023-07-14T18:35:28.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.423+0000 7ff0e57fa700 1 --2- 172.21.15.133:0/1710077089 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff0d00a11a0 0x7ff0d00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:28.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.423+0000 7ff0e57fa700 1 -- 172.21.15.133:0/1710077089 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff0e00ce4b0 con 0x7ff0e81017b0 2023-07-14T18:35:28.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.424+0000 7ff0e7fff700 1 --2- 172.21.15.133:0/1710077089 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff0d00a11a0 0x7ff0d00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:28.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.424+0000 7ff0e7fff700 1 --2- 172.21.15.133:0/1710077089 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff0d00a11a0 0x7ff0d00a3650 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8100c80 tx=0x7ff0d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:28.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.425+0000 7ff0e57fa700 1 -- 172.21.15.133:0/1710077089 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7ff0e0097a20 con 0x7ff0e81017b0 2023-07-14T18:35:28.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.625+0000 7ff0ee3ff700 1 -- 172.21.15.133:0/1710077089 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0e802cb10 con 0x7ff0e81017b0 2023-07-14T18:35:28.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.627+0000 7ff0e57fa700 1 -- 172.21.15.133:0/1710077089 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24520 (secure 0 0 0) 0x7ff0e009b040 con 0x7ff0e81017b0 2023-07-14T18:35:28.646 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:35:28.646 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":92,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","created":"2023-07-14T18:06:49.561582+0000","modified":"2023-07-14T18:15:49.951050+0000","last_up_change":"2023-07-14T18:13:29.836099+0000","last_in_change":"2023-07-14T18:13:11.606710+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T18:10:03.945039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T18:14:15.451340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T18:14:16.892215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T18:14:20.493175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"85a29de7-d712-422a-b647-d9e90091aaf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6803","nonce":3076510801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6805","nonce":3076510801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6809","nonce":3076510801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6807","nonce":3076510801}]},"public_addr":"172.21.15.133:6803/3076510801","cluster_addr":"172.21.15.133:6805/3076510801","heartbeat_back_addr":"172.21.15.133:6809/3076510801","heartbeat_front_addr":"172.21.15.133:6807/3076510801","state":["exists","up"]},{"osd":1,"uuid":"80ad4e78-7917-495e-b119-99f7c7c36243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6801","nonce":1841901852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6803","nonce":1841901852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6807","nonce":1841901852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6805","nonce":1841901852}]},"public_addr":"172.21.15.161:6801/1841901852","cluster_addr":"172.21.15.161:6803/1841901852","heartbeat_back_addr":"172.21.15.161:6807/1841901852","heartbeat_front_addr":"172.21.15.161:6805/1841901852","state":["exists","up"]},{"osd":2,"uuid":"fc4bfd37-1fdd-47c6-a48e-6d910daf9e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6801","nonce":346103099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6803","nonce":346103099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6807","nonce":346103099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6805","nonce":346103099}]},"public_addr":"172.21.15.171:6801/346103099","cluster_addr":"172.21.15.171:6803/346103099","heartbeat_back_addr":"172.21.15.171:6807/346103099","heartbeat_front_addr":"172.21.15.171:6805/346103099","state":["exists","up"]},{"osd":3,"uuid":"4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6811","nonce":2152994143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6813","nonce":2152994143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6817","nonce":2152994143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6815","nonce":2152994143}]},"public_addr":"172.21.15.133:6811/2152994143","cluster_addr":"172.21.15.133:6813/2152994143","heartbeat_back_addr":"172.21.15.133:6817/2152994143","heartbeat_front_addr":"172.21.15.133:6815/2152994143","state":["exists","up"]},{"osd":4,"uuid":"4601a1c2-254b-4c4d-9b15-8fc952b3d2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6809","nonce":47213291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6811","nonce":47213291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6815","nonce":47213291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6813","nonce":47213291}]},"public_addr":"172.21.15.161:6809/47213291","cluster_addr":"172.21.15.161:6811/47213291","heartbeat_back_addr":"172.21.15.161:6815/47213291","heartbeat_front_addr":"172.21.15.161:6813/47213291","state":["exists","up"]},{"osd":5,"uuid":"6cca180a-9140-4e49-835b-86842dae0574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6809","nonce":777556751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6811","nonce":777556751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6815","nonce":777556751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6813","nonce":777556751}]},"public_addr":"172.21.15.171:6809/777556751","cluster_addr":"172.21.15.171:6811/777556751","heartbeat_back_addr":"172.21.15.171:6815/777556751","heartbeat_front_addr":"172.21.15.171:6813/777556751","state":["exists","up"]},{"osd":6,"uuid":"ea0299ff-7740-4494-a3a9-1503b3fc0ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6819","nonce":2884152358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6821","nonce":2884152358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6825","nonce":2884152358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6823","nonce":2884152358}]},"public_addr":"172.21.15.133:6819/2884152358","cluster_addr":"172.21.15.133:6821/2884152358","heartbeat_back_addr":"172.21.15.133:6825/2884152358","heartbeat_front_addr":"172.21.15.133:6823/2884152358","state":["exists","up"]},{"osd":7,"uuid":"b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6817","nonce":4183705749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6819","nonce":4183705749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6823","nonce":4183705749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6821","nonce":4183705749}]},"public_addr":"172.21.15.161:6817/4183705749","cluster_addr":"172.21.15.161:6819/4183705749","heartbeat_back_addr":"172.21.15.161:6823/4183705749","heartbeat_front_addr":"172.21.15.161:6821/4183705749","state":["exists","up"]},{"osd":8,"uuid":"87dffc07-bf48-48c8-b418-5c4b509686d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6817","nonce":2771494483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6819","nonce":2771494483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6823","nonce":2771494483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6821","nonce":2771494483}]},"public_addr":"172.21.15.171:6817/2771494483","cluster_addr":"172.21.15.171:6819/2771494483","heartbeat_back_addr":"172.21.15.171:6823/2771494483","heartbeat_front_addr":"172.21.15.171:6821/2771494483","state":["exists","up"]},{"osd":9,"uuid":"e0832870-79ce-4bce-8045-58b7a7f6dc69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6827","nonce":2066164149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6829","nonce":2066164149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6833","nonce":2066164149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6831","nonce":2066164149}]},"public_addr":"172.21.15.133:6827/2066164149","cluster_addr":"172.21.15.133:6829/2066164149","heartbeat_back_addr":"172.21.15.133:6833/2066164149","heartbeat_front_addr":"172.21.15.133:6831/2066164149","state":["exists","up"]},{"osd":10,"uuid":"081f0484-d134-4b2e-9c48-ef8e6a646422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6825","nonce":2162333626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6827","nonce":2162333626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6831","nonce":2162333626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6829","nonce":2162333626}]},"public_addr":"172.21.15.161:6825/2162333626","cluster_addr":"172.21.15.161:6827/2162333626","heartbeat_back_addr":"172.21.15.161:6831/2162333626","heartbeat_front_addr":"172.21.15.161:6829/2162333626","state":["exists","up"]},{"osd":11,"uuid":"4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6825","nonce":3403894313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6827","nonce":3403894313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6831","nonce":3403894313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6829","nonce":3403894313}]},"public_addr":"172.21.15.171:6825/3403894313","cluster_addr":"172.21.15.171:6827/3403894313","heartbeat_back_addr":"172.21.15.171:6831/3403894313","heartbeat_front_addr":"172.21.15.171:6829/3403894313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:17.816229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:39.638543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:00.543317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:26.988921+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:48.644694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:10.049860+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:35.986508+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:57.753846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:19.018245+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:44.460450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:06.279960+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:27.955006+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/1832569241":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6801/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:0/4219815304":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:6801/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/432253615":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1840700480":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1807775439":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1089030734":"2023-07-15T18:07:47.346704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T18:35:28.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.630+0000 7ff0ee3ff700 1 -- 172.21.15.133:0/1710077089 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff0d00a11a0 msgr2=0x7ff0d00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:28.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.630+0000 7ff0ee3ff700 1 --2- 172.21.15.133:0/1710077089 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff0d00a11a0 0x7ff0d00a3650 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8100c80 tx=0x7ff0d8009260 comp rx=0 tx=0).stop 2023-07-14T18:35:28.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.630+0000 7ff0ee3ff700 1 -- 172.21.15.133:0/1710077089 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0e81017b0 msgr2=0x7ff0e8072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:28.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.630+0000 7ff0ee3ff700 1 --2- 172.21.15.133:0/1710077089 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0e81017b0 0x7ff0e8072f70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0007930 tx=0x7ff0e0004ce0 comp rx=0 tx=0).stop 2023-07-14T18:35:28.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.630+0000 7ff0ee3ff700 1 -- 172.21.15.133:0/1710077089 shutdown_connections 2023-07-14T18:35:28.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.630+0000 7ff0ee3ff700 1 --2- 172.21.15.133:0/1710077089 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7ff0d00a11a0 0x7ff0d00a3650 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:28.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.630+0000 7ff0ee3ff700 1 --2- 172.21.15.133:0/1710077089 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0e81017b0 0x7ff0e8072f70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:28.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.630+0000 7ff0ee3ff700 1 --2- 172.21.15.133:0/1710077089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0e80ffc30 0x7ff0e80743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:28.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.630+0000 7ff0ee3ff700 1 --2- 172.21.15.133:0/1710077089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0e8100e20 0x7ff0e8072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:28.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.630+0000 7ff0ee3ff700 1 -- 172.21.15.133:0/1710077089 >> 172.21.15.133:0/1710077089 conn(0x7ff0e80fb210 msgr2=0x7ff0e80fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:28.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.630+0000 7ff0ee3ff700 1 -- 172.21.15.133:0/1710077089 shutdown_connections 2023-07-14T18:35:28.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.630+0000 7ff0ee3ff700 1 -- 172.21.15.133:0/1710077089 wait complete. 2023-07-14T18:35:28.668 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T18:10:03.945039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T18:14:15.451340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T18:14:16.892215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T18:14:20.493175+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T18:35:28.669 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T18:35:28.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.855+0000 7f5fd97ff700 1 -- 172.21.15.133:0/1314229069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fd40ffc20 msgr2=0x7f5fd4100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:28.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.855+0000 7f5fd97ff700 1 --2- 172.21.15.133:0/1314229069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fd40ffc20 0x7f5fd4100030 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4009990 tx=0x7f5fc4009ca0 comp rx=0 tx=0).stop 2023-07-14T18:35:28.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.855+0000 7f5fd97ff700 1 -- 172.21.15.133:0/1314229069 shutdown_connections 2023-07-14T18:35:28.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.855+0000 7f5fd97ff700 1 --2- 172.21.15.133:0/1314229069 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fd41017a0 0x7f5fd4108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:28.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.855+0000 7f5fd97ff700 1 --2- 172.21.15.133:0/1314229069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fd40ffc20 0x7f5fd4100030 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:28.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.855+0000 7f5fd97ff700 1 --2- 172.21.15.133:0/1314229069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fd4100e10 0x7f5fd4101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:28.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.855+0000 7f5fd97ff700 1 -- 172.21.15.133:0/1314229069 >> 172.21.15.133:0/1314229069 conn(0x7f5fd40fb260 msgr2=0x7f5fd40fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:28.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.855+0000 7f5fd97ff700 1 -- 172.21.15.133:0/1314229069 shutdown_connections 2023-07-14T18:35:28.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.855+0000 7f5fd97ff700 1 -- 172.21.15.133:0/1314229069 wait complete. 2023-07-14T18:35:28.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.856+0000 7f5fd97ff700 1 Processor -- start 2023-07-14T18:35:28.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.856+0000 7f5fd97ff700 1 -- start start 2023-07-14T18:35:28.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.856+0000 7f5fd97ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fd40ffc20 0x7f5fd41959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:28.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.857+0000 7f5fd97ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fd4100e10 0x7f5fd4195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:28.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.857+0000 7f5fd97ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fd41017a0 0x7f5fd419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:28.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.857+0000 7f5fd97ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd419d530 con 0x7f5fd4100e10 2023-07-14T18:35:28.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.857+0000 7f5fd97ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd419d670 con 0x7f5fd40ffc20 2023-07-14T18:35:28.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.857+0000 7f5fd97ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd419d800 con 0x7f5fd41017a0 2023-07-14T18:35:28.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.857+0000 7f5fd2ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fd40ffc20 0x7f5fd41959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:28.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.857+0000 7f5fd2ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fd40ffc20 0x7f5fd41959b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:57506/0 (socket says 172.21.15.133:57506) 2023-07-14T18:35:28.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.857+0000 7f5fd2ffd700 1 -- 172.21.15.133:0/475224615 learned_addr learned my addr 172.21.15.133:0/475224615 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:35:28.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.857+0000 7f5fd27fc700 1 --2- 172.21.15.133:0/475224615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fd4100e10 0x7f5fd4195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:28.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.857+0000 7f5fd37fe700 1 --2- 172.21.15.133:0/475224615 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fd41017a0 0x7f5fd419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:28.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.857+0000 7f5fd2ffd700 1 -- 172.21.15.133:0/475224615 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fd41017a0 msgr2=0x7f5fd419cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:28.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.857+0000 7f5fd2ffd700 1 --2- 172.21.15.133:0/475224615 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fd41017a0 0x7f5fd419cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:28.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.857+0000 7f5fd2ffd700 1 -- 172.21.15.133:0/475224615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fd4100e10 msgr2=0x7f5fd4195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:28.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.857+0000 7f5fd2ffd700 1 --2- 172.21.15.133:0/475224615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fd4100e10 0x7f5fd4195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:28.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.857+0000 7f5fd2ffd700 1 -- 172.21.15.133:0/475224615 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc40095f0 con 0x7f5fd40ffc20 2023-07-14T18:35:28.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.857+0000 7f5fd37fe700 1 --2- 172.21.15.133:0/475224615 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fd41017a0 0x7f5fd419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:35:28.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.857+0000 7f5fd2ffd700 1 --2- 172.21.15.133:0/475224615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fd40ffc20 0x7f5fd41959b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4005740 tx=0x7f5fc4005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:28.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.858+0000 7f5fbbfff700 1 -- 172.21.15.133:0/475224615 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc401d070 con 0x7f5fd40ffc20 2023-07-14T18:35:28.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.858+0000 7f5fbbfff700 1 -- 172.21.15.133:0/475224615 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fc4021410 con 0x7f5fd40ffc20 2023-07-14T18:35:28.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.858+0000 7f5fd97ff700 1 -- 172.21.15.133:0/475224615 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd419da80 con 0x7f5fd40ffc20 2023-07-14T18:35:28.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.858+0000 7f5fbbfff700 1 -- 172.21.15.133:0/475224615 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4017650 con 0x7f5fd40ffc20 2023-07-14T18:35:28.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.858+0000 7f5fd97ff700 1 -- 172.21.15.133:0/475224615 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd419dea0 con 0x7f5fd40ffc20 2023-07-14T18:35:28.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.859+0000 7f5fd97ff700 1 -- 172.21.15.133:0/475224615 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd410b440 con 0x7f5fd40ffc20 2023-07-14T18:35:28.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.861+0000 7f5fbbfff700 1 -- 172.21.15.133:0/475224615 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f5fc4021580 con 0x7f5fd40ffc20 2023-07-14T18:35:28.883 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:28 smithi133 ceph-mon[126384]: pgmap v911: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.5 KiB/s wr, 9 op/s 2023-07-14T18:35:28.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.862+0000 7f5fbbfff700 1 --2- 172.21.15.133:0/475224615 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5fbc0a1150 0x7f5fbc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:28.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.863+0000 7f5fbbfff700 1 -- 172.21.15.133:0/475224615 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5fc40ce2b0 con 0x7f5fd40ffc20 2023-07-14T18:35:28.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.863+0000 7f5fd27fc700 1 --2- 172.21.15.133:0/475224615 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5fbc0a1150 0x7f5fbc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:28.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.863+0000 7f5fd27fc700 1 --2- 172.21.15.133:0/475224615 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5fbc0a1150 0x7f5fbc0a3600 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009c60 tx=0x7f5fc8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:28.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:28.866+0000 7f5fbbfff700 1 -- 172.21.15.133:0/475224615 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f5fc4097700 con 0x7f5fd40ffc20 2023-07-14T18:35:29.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:28 smithi161 ceph-mon[128993]: pgmap v911: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.5 KiB/s wr, 9 op/s 2023-07-14T18:35:29.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.066+0000 7f5fd97ff700 1 -- 172.21.15.133:0/475224615 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5fd4106b00 con 0x7f5fd40ffc20 2023-07-14T18:35:29.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.066+0000 7f5fbbfff700 1 -- 172.21.15.133:0/475224615 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5fc409ad20 con 0x7f5fd40ffc20 2023-07-14T18:35:29.085 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-07-14T18:35:29.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.069+0000 7f5fd97ff700 1 -- 172.21.15.133:0/475224615 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5fbc0a1150 msgr2=0x7f5fbc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:29.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.069+0000 7f5fd97ff700 1 --2- 172.21.15.133:0/475224615 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5fbc0a1150 0x7f5fbc0a3600 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009c60 tx=0x7f5fc8009260 comp rx=0 tx=0).stop 2023-07-14T18:35:29.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.069+0000 7f5fd97ff700 1 -- 172.21.15.133:0/475224615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fd40ffc20 msgr2=0x7f5fd41959b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:29.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.069+0000 7f5fd97ff700 1 --2- 172.21.15.133:0/475224615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fd40ffc20 0x7f5fd41959b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4005740 tx=0x7f5fc4005770 comp rx=0 tx=0).stop 2023-07-14T18:35:29.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.069+0000 7f5fd97ff700 1 -- 172.21.15.133:0/475224615 shutdown_connections 2023-07-14T18:35:29.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.069+0000 7f5fd97ff700 1 --2- 172.21.15.133:0/475224615 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f5fbc0a1150 0x7f5fbc0a3600 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:29.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.069+0000 7f5fd97ff700 1 --2- 172.21.15.133:0/475224615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fd40ffc20 0x7f5fd41959b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:29.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.069+0000 7f5fd97ff700 1 --2- 172.21.15.133:0/475224615 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fd41017a0 0x7f5fd419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:29.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.069+0000 7f5fd97ff700 1 --2- 172.21.15.133:0/475224615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fd4100e10 0x7f5fd4195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:29.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.069+0000 7f5fd97ff700 1 -- 172.21.15.133:0/475224615 >> 172.21.15.133:0/475224615 conn(0x7f5fd40fb260 msgr2=0x7f5fd40fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:29.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.069+0000 7f5fd97ff700 1 -- 172.21.15.133:0/475224615 shutdown_connections 2023-07-14T18:35:29.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.069+0000 7f5fd97ff700 1 -- 172.21.15.133:0/475224615 wait complete. 2023-07-14T18:35:29.089 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:28 smithi171 ceph-mon[129241]: pgmap v911: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.5 KiB/s wr, 9 op/s 2023-07-14T18:35:29.107 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T18:35:29.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.293+0000 7f49a15ff700 1 -- 172.21.15.133:0/2151513257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f499c101790 msgr2=0x7f499c108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:29.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.293+0000 7f49a15ff700 1 --2- 172.21.15.133:0/2151513257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f499c101790 0x7f499c108050 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f499400b0a0 tx=0x7f499400b3b0 comp rx=0 tx=0).stop 2023-07-14T18:35:29.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.293+0000 7f49a15ff700 1 -- 172.21.15.133:0/2151513257 shutdown_connections 2023-07-14T18:35:29.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.293+0000 7f49a15ff700 1 --2- 172.21.15.133:0/2151513257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f499c101790 0x7f499c108050 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:29.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.293+0000 7f49a15ff700 1 --2- 172.21.15.133:0/2151513257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f499c0ffc10 0x7f499c100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:29.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.293+0000 7f49a15ff700 1 --2- 172.21.15.133:0/2151513257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f499c100e00 0x7f499c101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:29.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.293+0000 7f49a15ff700 1 -- 172.21.15.133:0/2151513257 >> 172.21.15.133:0/2151513257 conn(0x7f499c0fb210 msgr2=0x7f499c0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:29.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.293+0000 7f49a15ff700 1 -- 172.21.15.133:0/2151513257 shutdown_connections 2023-07-14T18:35:29.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.293+0000 7f49a15ff700 1 -- 172.21.15.133:0/2151513257 wait complete. 2023-07-14T18:35:29.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.294+0000 7f49a15ff700 1 Processor -- start 2023-07-14T18:35:29.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.294+0000 7f49a15ff700 1 -- start start 2023-07-14T18:35:29.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.295+0000 7f49a15ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f499c0ffc10 0x7f499c197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:29.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.295+0000 7f49a15ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f499c100e00 0x7f499c1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:29.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.295+0000 7f49a15ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f499c101790 0x7f499c19f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:29.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.295+0000 7f49a15ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c19f800 con 0x7f499c101790 2023-07-14T18:35:29.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.295+0000 7f49a15ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c19f970 con 0x7f499c100e00 2023-07-14T18:35:29.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.295+0000 7f49a15ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c19fb30 con 0x7f499c0ffc10 2023-07-14T18:35:29.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.295+0000 7f499affd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f499c0ffc10 0x7f499c197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:29.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.295+0000 7f499affd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f499c0ffc10 0x7f499c197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:56264/0 (socket says 172.21.15.133:56264) 2023-07-14T18:35:29.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.295+0000 7f499affd700 1 -- 172.21.15.133:0/919746845 learned_addr learned my addr 172.21.15.133:0/919746845 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:35:29.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.295+0000 7f499a7fc700 1 --2- 172.21.15.133:0/919746845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f499c100e00 0x7f499c1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:29.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.295+0000 7f499b7fe700 1 --2- 172.21.15.133:0/919746845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f499c101790 0x7f499c19f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:29.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.295+0000 7f499affd700 1 -- 172.21.15.133:0/919746845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f499c100e00 msgr2=0x7f499c1981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:29.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.295+0000 7f499affd700 1 --2- 172.21.15.133:0/919746845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f499c100e00 0x7f499c1981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:29.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.295+0000 7f499affd700 1 -- 172.21.15.133:0/919746845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f499c101790 msgr2=0x7f499c19f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:29.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.295+0000 7f499affd700 1 --2- 172.21.15.133:0/919746845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f499c101790 0x7f499c19f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:29.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.295+0000 7f499affd700 1 -- 172.21.15.133:0/919746845 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4994009d00 con 0x7f499c0ffc10 2023-07-14T18:35:29.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.295+0000 7f499affd700 1 --2- 172.21.15.133:0/919746845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f499c0ffc10 0x7f499c197c90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f498c00e970 tx=0x7f498c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:29.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.296+0000 7f4983fff700 1 -- 172.21.15.133:0/919746845 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c009790 con 0x7f499c0ffc10 2023-07-14T18:35:29.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.296+0000 7f49a15ff700 1 -- 172.21.15.133:0/919746845 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499c19fe10 con 0x7f499c0ffc10 2023-07-14T18:35:29.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.296+0000 7f4983fff700 1 -- 172.21.15.133:0/919746845 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f498c00eec0 con 0x7f499c0ffc10 2023-07-14T18:35:29.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.296+0000 7f4983fff700 1 -- 172.21.15.133:0/919746845 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c003cf0 con 0x7f499c0ffc10 2023-07-14T18:35:29.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.296+0000 7f49a15ff700 1 -- 172.21.15.133:0/919746845 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499c1a0230 con 0x7f499c0ffc10 2023-07-14T18:35:29.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.297+0000 7f49a15ff700 1 -- 172.21.15.133:0/919746845 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499c10b440 con 0x7f499c0ffc10 2023-07-14T18:35:29.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.299+0000 7f4983fff700 1 -- 172.21.15.133:0/919746845 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f498c00ccd0 con 0x7f499c0ffc10 2023-07-14T18:35:29.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.300+0000 7f4983fff700 1 --2- 172.21.15.133:0/919746845 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f49840a1150 0x7f49840a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:29.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.300+0000 7f4983fff700 1 -- 172.21.15.133:0/919746845 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f498c014070 con 0x7f499c0ffc10 2023-07-14T18:35:29.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.300+0000 7f499a7fc700 1 --2- 172.21.15.133:0/919746845 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f49840a1150 0x7f49840a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:29.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.301+0000 7f499a7fc700 1 --2- 172.21.15.133:0/919746845 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f49840a1150 0x7f49840a3600 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f499c199190 tx=0x7f4990009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:29.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.304+0000 7f4983fff700 1 -- 172.21.15.133:0/919746845 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f498c0962b0 con 0x7f499c0ffc10 2023-07-14T18:35:29.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.506+0000 7f49a15ff700 1 -- 172.21.15.133:0/919746845 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f499c1a08d0 con 0x7f499c0ffc10 2023-07-14T18:35:29.525 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.506+0000 7f4983fff700 1 -- 172.21.15.133:0/919746845 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f498c0998d0 con 0x7f499c0ffc10 2023-07-14T18:35:29.525 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-07-14T18:35:29.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.509+0000 7f49a15ff700 1 -- 172.21.15.133:0/919746845 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f49840a1150 msgr2=0x7f49840a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:29.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.509+0000 7f49a15ff700 1 --2- 172.21.15.133:0/919746845 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f49840a1150 0x7f49840a3600 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f499c199190 tx=0x7f4990009260 comp rx=0 tx=0).stop 2023-07-14T18:35:29.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.509+0000 7f49a15ff700 1 -- 172.21.15.133:0/919746845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f499c0ffc10 msgr2=0x7f499c197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:29.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.509+0000 7f49a15ff700 1 --2- 172.21.15.133:0/919746845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f499c0ffc10 0x7f499c197c90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f498c00e970 tx=0x7f498c00ec80 comp rx=0 tx=0).stop 2023-07-14T18:35:29.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.509+0000 7f49a15ff700 1 -- 172.21.15.133:0/919746845 shutdown_connections 2023-07-14T18:35:29.530 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.509+0000 7f49a15ff700 1 --2- 172.21.15.133:0/919746845 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f49840a1150 0x7f49840a3600 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:29.530 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.509+0000 7f49a15ff700 1 --2- 172.21.15.133:0/919746845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f499c101790 0x7f499c19f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:29.530 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.509+0000 7f49a15ff700 1 --2- 172.21.15.133:0/919746845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f499c0ffc10 0x7f499c197c90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:29.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.509+0000 7f49a15ff700 1 --2- 172.21.15.133:0/919746845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f499c100e00 0x7f499c1981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:29.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.509+0000 7f49a15ff700 1 -- 172.21.15.133:0/919746845 >> 172.21.15.133:0/919746845 conn(0x7f499c0fb210 msgr2=0x7f499c0fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:29.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.509+0000 7f49a15ff700 1 -- 172.21.15.133:0/919746845 shutdown_connections 2023-07-14T18:35:29.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.509+0000 7f49a15ff700 1 -- 172.21.15.133:0/919746845 wait complete. 2023-07-14T18:35:29.546 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T18:35:29.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.729+0000 7efec1bff700 1 -- 172.21.15.133:0/3900339983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efebc100e10 msgr2=0x7efebc101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:29.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.729+0000 7efec1bff700 1 --2- 172.21.15.133:0/3900339983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efebc100e10 0x7efebc101260 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009990 tx=0x7efeb0009ca0 comp rx=0 tx=0).stop 2023-07-14T18:35:29.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.729+0000 7efec1bff700 1 -- 172.21.15.133:0/3900339983 shutdown_connections 2023-07-14T18:35:29.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.729+0000 7efec1bff700 1 --2- 172.21.15.133:0/3900339983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efebc1017a0 0x7efebc108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:29.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.729+0000 7efec1bff700 1 --2- 172.21.15.133:0/3900339983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efebc0ffc20 0x7efebc100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:29.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.729+0000 7efec1bff700 1 --2- 172.21.15.133:0/3900339983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efebc100e10 0x7efebc101260 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:29.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.729+0000 7efec1bff700 1 -- 172.21.15.133:0/3900339983 >> 172.21.15.133:0/3900339983 conn(0x7efebc0fb260 msgr2=0x7efebc0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:29.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.729+0000 7efec1bff700 1 -- 172.21.15.133:0/3900339983 shutdown_connections 2023-07-14T18:35:29.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.729+0000 7efec1bff700 1 -- 172.21.15.133:0/3900339983 wait complete. 2023-07-14T18:35:29.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.730+0000 7efec1bff700 1 Processor -- start 2023-07-14T18:35:29.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.730+0000 7efec1bff700 1 -- start start 2023-07-14T18:35:29.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.731+0000 7efec1bff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efebc0ffc20 0x7efebc197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:29.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.731+0000 7efec1bff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efebc100e10 0x7efebc198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:29.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.731+0000 7efec1bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efebc1017a0 0x7efebc19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:29.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.731+0000 7efec1bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efebc19f690 con 0x7efebc1017a0 2023-07-14T18:35:29.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.731+0000 7efec1bff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efebc19f7d0 con 0x7efebc0ffc20 2023-07-14T18:35:29.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.731+0000 7efec1bff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efebc19f960 con 0x7efebc100e10 2023-07-14T18:35:29.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.731+0000 7efebaffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efebc100e10 0x7efebc198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:29.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.731+0000 7efebaffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efebc100e10 0x7efebc198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:56278/0 (socket says 172.21.15.133:56278) 2023-07-14T18:35:29.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.731+0000 7efebb7fe700 1 --2- 172.21.15.133:0/457901243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efebc0ffc20 0x7efebc197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:29.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.731+0000 7efebaffd700 1 -- 172.21.15.133:0/457901243 learned_addr learned my addr 172.21.15.133:0/457901243 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:35:29.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.731+0000 7efebbfff700 1 --2- 172.21.15.133:0/457901243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efebc1017a0 0x7efebc19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:29.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.731+0000 7efebaffd700 1 -- 172.21.15.133:0/457901243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efebc0ffc20 msgr2=0x7efebc197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:29.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.731+0000 7efebaffd700 1 --2- 172.21.15.133:0/457901243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efebc0ffc20 0x7efebc197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:29.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.731+0000 7efebaffd700 1 -- 172.21.15.133:0/457901243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efebc1017a0 msgr2=0x7efebc19f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:29.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.731+0000 7efebaffd700 1 --2- 172.21.15.133:0/457901243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efebc1017a0 0x7efebc19f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:29.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.731+0000 7efebaffd700 1 -- 172.21.15.133:0/457901243 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb00095f0 con 0x7efebc100e10 2023-07-14T18:35:29.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.731+0000 7efebb7fe700 1 --2- 172.21.15.133:0/457901243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efebc0ffc20 0x7efebc197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:35:29.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.732+0000 7efebaffd700 1 --2- 172.21.15.133:0/457901243 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efebc100e10 0x7efebc198050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efeb000b680 tx=0x7efeb0005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:29.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.732+0000 7efeb8ff9700 1 -- 172.21.15.133:0/457901243 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb001d070 con 0x7efebc100e10 2023-07-14T18:35:29.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.732+0000 7efeb8ff9700 1 -- 172.21.15.133:0/457901243 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efeb0022470 con 0x7efebc100e10 2023-07-14T18:35:29.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.732+0000 7efeb8ff9700 1 -- 172.21.15.133:0/457901243 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb0017650 con 0x7efebc100e10 2023-07-14T18:35:29.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.732+0000 7efec1bff700 1 -- 172.21.15.133:0/457901243 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efebc19fbe0 con 0x7efebc100e10 2023-07-14T18:35:29.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.732+0000 7efec1bff700 1 -- 172.21.15.133:0/457901243 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efebc1a0010 con 0x7efebc100e10 2023-07-14T18:35:29.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.733+0000 7efec1bff700 1 -- 172.21.15.133:0/457901243 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efebc1922a0 con 0x7efebc100e10 2023-07-14T18:35:29.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.735+0000 7efeb8ff9700 1 -- 172.21.15.133:0/457901243 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7efeb00225e0 con 0x7efebc100e10 2023-07-14T18:35:29.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.736+0000 7efeb8ff9700 1 --2- 172.21.15.133:0/457901243 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efea40a1150 0x7efea40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:29.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.737+0000 7efeb8ff9700 1 -- 172.21.15.133:0/457901243 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7efeb00ce1f0 con 0x7efebc100e10 2023-07-14T18:35:29.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.737+0000 7efebb7fe700 1 --2- 172.21.15.133:0/457901243 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efea40a1150 0x7efea40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:29.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.737+0000 7efebb7fe700 1 --2- 172.21.15.133:0/457901243 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efea40a1150 0x7efea40a3600 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efebc100c70 tx=0x7efeac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:29.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.738+0000 7efeb8ff9700 1 -- 172.21.15.133:0/457901243 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7efeb0097640 con 0x7efebc100e10 2023-07-14T18:35:29.880 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:29 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1710077089' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:35:29.880 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:29 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/475224615' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:35:29.880 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:29 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/919746845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:35:29.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.935+0000 7efec1bff700 1 -- 172.21.15.133:0/457901243 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efebc02ca70 con 0x7efebc100e10 2023-07-14T18:35:29.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.936+0000 7efeb8ff9700 1 -- 172.21.15.133:0/457901243 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7efeb009ac60 con 0x7efebc100e10 2023-07-14T18:35:29.954 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2023-07-14T18:35:29.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.938+0000 7efec1bff700 1 -- 172.21.15.133:0/457901243 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efea40a1150 msgr2=0x7efea40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:29.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.938+0000 7efec1bff700 1 --2- 172.21.15.133:0/457901243 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efea40a1150 0x7efea40a3600 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efebc100c70 tx=0x7efeac009260 comp rx=0 tx=0).stop 2023-07-14T18:35:29.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.938+0000 7efec1bff700 1 -- 172.21.15.133:0/457901243 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efebc100e10 msgr2=0x7efebc198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:29.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.938+0000 7efec1bff700 1 --2- 172.21.15.133:0/457901243 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efebc100e10 0x7efebc198050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efeb000b680 tx=0x7efeb0005740 comp rx=0 tx=0).stop 2023-07-14T18:35:29.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.938+0000 7efec1bff700 1 -- 172.21.15.133:0/457901243 shutdown_connections 2023-07-14T18:35:29.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.938+0000 7efec1bff700 1 --2- 172.21.15.133:0/457901243 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7efea40a1150 0x7efea40a3600 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:29.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.938+0000 7efec1bff700 1 --2- 172.21.15.133:0/457901243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efebc0ffc20 0x7efebc197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:29.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.938+0000 7efec1bff700 1 --2- 172.21.15.133:0/457901243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efebc1017a0 0x7efebc19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:29.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.938+0000 7efec1bff700 1 --2- 172.21.15.133:0/457901243 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efebc100e10 0x7efebc198050 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:29.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.938+0000 7efec1bff700 1 -- 172.21.15.133:0/457901243 >> 172.21.15.133:0/457901243 conn(0x7efebc0fb260 msgr2=0x7efebc106700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:29.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.938+0000 7efec1bff700 1 -- 172.21.15.133:0/457901243 shutdown_connections 2023-07-14T18:35:29.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:29.938+0000 7efec1bff700 1 -- 172.21.15.133:0/457901243 wait complete. 2023-07-14T18:35:29.977 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T18:35:30.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:29 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1710077089' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:35:30.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:29 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/475224615' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:35:30.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:29 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/919746845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:35:30.089 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:29 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1710077089' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:35:30.089 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:29 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/475224615' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T18:35:30.089 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:29 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/919746845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T18:35:30.177 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.159+0000 7f6ac47ff700 1 -- 172.21.15.133:0/3511637820 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6abc100e10 msgr2=0x7f6abc101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:30.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.159+0000 7f6ac47ff700 1 --2- 172.21.15.133:0/3511637820 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6abc100e10 0x7f6abc101260 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6aac009990 tx=0x7f6aac009ca0 comp rx=0 tx=0).stop 2023-07-14T18:35:30.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.159+0000 7f6ac47ff700 1 -- 172.21.15.133:0/3511637820 shutdown_connections 2023-07-14T18:35:30.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.159+0000 7f6ac47ff700 1 --2- 172.21.15.133:0/3511637820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6abc1017a0 0x7f6abc108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:30.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.159+0000 7f6ac47ff700 1 --2- 172.21.15.133:0/3511637820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6abc0ffc20 0x7f6abc100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:30.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.159+0000 7f6ac47ff700 1 --2- 172.21.15.133:0/3511637820 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6abc100e10 0x7f6abc101260 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:30.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.159+0000 7f6ac47ff700 1 -- 172.21.15.133:0/3511637820 >> 172.21.15.133:0/3511637820 conn(0x7f6abc0fb260 msgr2=0x7f6abc0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:30.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.159+0000 7f6ac47ff700 1 -- 172.21.15.133:0/3511637820 shutdown_connections 2023-07-14T18:35:30.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.159+0000 7f6ac47ff700 1 -- 172.21.15.133:0/3511637820 wait complete. 2023-07-14T18:35:30.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.160+0000 7f6ac47ff700 1 Processor -- start 2023-07-14T18:35:30.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.160+0000 7f6ac47ff700 1 -- start start 2023-07-14T18:35:30.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.161+0000 7f6ac47ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6abc0ffc20 0x7f6abc1958d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:30.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.161+0000 7f6ac47ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6abc100e10 0x7f6abc195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:30.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.161+0000 7f6ac47ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6abc1017a0 0x7f6abc19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:30.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.161+0000 7f6ac47ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc19d450 con 0x7f6abc0ffc20 2023-07-14T18:35:30.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.161+0000 7f6ac47ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc19d590 con 0x7f6abc1017a0 2023-07-14T18:35:30.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.161+0000 7f6ac47ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc19d720 con 0x7f6abc100e10 2023-07-14T18:35:30.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.161+0000 7f6ac259b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6abc0ffc20 0x7f6abc1958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:30.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.161+0000 7f6ac1d9a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6abc100e10 0x7f6abc195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:30.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.161+0000 7f6ac2d9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6abc1017a0 0x7f6abc19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:30.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.161+0000 7f6ac1d9a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6abc100e10 0x7f6abc195e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:56288/0 (socket says 172.21.15.133:56288) 2023-07-14T18:35:30.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.161+0000 7f6ac1d9a700 1 -- 172.21.15.133:0/3940395395 learned_addr learned my addr 172.21.15.133:0/3940395395 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:35:30.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.161+0000 7f6ac259b700 1 -- 172.21.15.133:0/3940395395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6abc100e10 msgr2=0x7f6abc195e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:30.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.161+0000 7f6ac259b700 1 --2- 172.21.15.133:0/3940395395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6abc100e10 0x7f6abc195e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:30.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.161+0000 7f6ac259b700 1 -- 172.21.15.133:0/3940395395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6abc1017a0 msgr2=0x7f6abc19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:30.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.161+0000 7f6ac259b700 1 --2- 172.21.15.133:0/3940395395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6abc1017a0 0x7f6abc19ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:30.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.161+0000 7f6ac259b700 1 -- 172.21.15.133:0/3940395395 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab8009520 con 0x7f6abc0ffc20 2023-07-14T18:35:30.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.161+0000 7f6ac2d9c700 1 --2- 172.21.15.133:0/3940395395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6abc1017a0 0x7f6abc19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:35:30.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.162+0000 7f6ac259b700 1 --2- 172.21.15.133:0/3940395395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6abc0ffc20 0x7f6abc1958d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6ab800e9c0 tx=0x7f6ab800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:30.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.162+0000 7f6ab37fe700 1 -- 172.21.15.133:0/3940395395 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab800cb70 con 0x7f6abc0ffc20 2023-07-14T18:35:30.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.162+0000 7f6ac47ff700 1 -- 172.21.15.133:0/3940395395 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ab4009c40 con 0x7f6abc0ffc20 2023-07-14T18:35:30.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.162+0000 7f6ab37fe700 1 -- 172.21.15.133:0/3940395395 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ab800ee50 con 0x7f6abc0ffc20 2023-07-14T18:35:30.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.162+0000 7f6ab37fe700 1 -- 172.21.15.133:0/3940395395 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab8018680 con 0x7f6abc0ffc20 2023-07-14T18:35:30.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.162+0000 7f6ac47ff700 1 -- 172.21.15.133:0/3940395395 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aac0095f0 con 0x7f6abc0ffc20 2023-07-14T18:35:30.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.163+0000 7f6ac47ff700 1 -- 172.21.15.133:0/3940395395 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6abc1071c0 con 0x7f6abc0ffc20 2023-07-14T18:35:30.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.165+0000 7f6ab37fe700 1 -- 172.21.15.133:0/3940395395 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6ab8010430 con 0x7f6abc0ffc20 2023-07-14T18:35:30.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.166+0000 7f6ab37fe700 1 --2- 172.21.15.133:0/3940395395 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6aa80a1150 0x7f6aa80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:30.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.166+0000 7f6ab37fe700 1 -- 172.21.15.133:0/3940395395 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6ab8014070 con 0x7f6abc0ffc20 2023-07-14T18:35:30.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.167+0000 7f6ac1d9a700 1 --2- 172.21.15.133:0/3940395395 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6aa80a1150 0x7f6aa80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:30.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.167+0000 7f6ac1d9a700 1 --2- 172.21.15.133:0/3940395395 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6aa80a1150 0x7f6aa80a3600 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6aac00b2e0 tx=0x7f6aac00b850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:30.188 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.170+0000 7f6ab37fe700 1 -- 172.21.15.133:0/3940395395 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f6ab8096450 con 0x7f6abc0ffc20 2023-07-14T18:35:30.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.371+0000 7f6ac47ff700 1 -- 172.21.15.133:0/3940395395 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6abc061250 con 0x7f6abc0ffc20 2023-07-14T18:35:30.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.372+0000 7f6ab37fe700 1 -- 172.21.15.133:0/3940395395 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6ab8099a70 con 0x7f6abc0ffc20 2023-07-14T18:35:30.391 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-07-14T18:35:30.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.374+0000 7f6ac47ff700 1 -- 172.21.15.133:0/3940395395 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6aa80a1150 msgr2=0x7f6aa80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:30.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.374+0000 7f6ac47ff700 1 --2- 172.21.15.133:0/3940395395 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6aa80a1150 0x7f6aa80a3600 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6aac00b2e0 tx=0x7f6aac00b850 comp rx=0 tx=0).stop 2023-07-14T18:35:30.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.374+0000 7f6ac47ff700 1 -- 172.21.15.133:0/3940395395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6abc0ffc20 msgr2=0x7f6abc1958d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:30.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.374+0000 7f6ac47ff700 1 --2- 172.21.15.133:0/3940395395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6abc0ffc20 0x7f6abc1958d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6ab800e9c0 tx=0x7f6ab800ecd0 comp rx=0 tx=0).stop 2023-07-14T18:35:30.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.374+0000 7f6ac47ff700 1 -- 172.21.15.133:0/3940395395 shutdown_connections 2023-07-14T18:35:30.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.374+0000 7f6ac47ff700 1 --2- 172.21.15.133:0/3940395395 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f6aa80a1150 0x7f6aa80a3600 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:30.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.374+0000 7f6ac47ff700 1 --2- 172.21.15.133:0/3940395395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6abc1017a0 0x7f6abc19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:30.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.374+0000 7f6ac47ff700 1 --2- 172.21.15.133:0/3940395395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6abc0ffc20 0x7f6abc1958d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:30.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.374+0000 7f6ac47ff700 1 --2- 172.21.15.133:0/3940395395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6abc100e10 0x7f6abc195e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:30.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.374+0000 7f6ac47ff700 1 -- 172.21.15.133:0/3940395395 >> 172.21.15.133:0/3940395395 conn(0x7f6abc0fb260 msgr2=0x7f6abc104300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:30.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.374+0000 7f6ac47ff700 1 -- 172.21.15.133:0/3940395395 shutdown_connections 2023-07-14T18:35:30.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.374+0000 7f6ac47ff700 1 -- 172.21.15.133:0/3940395395 wait complete. 2023-07-14T18:35:30.413 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T18:35:30.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.594+0000 7fcc217ff700 1 -- 172.21.15.133:0/2827767662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc1c0ffc20 msgr2=0x7fcc1c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:30.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.594+0000 7fcc217ff700 1 --2- 172.21.15.133:0/2827767662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc1c0ffc20 0x7fcc1c100030 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c009990 tx=0x7fcc0c009ca0 comp rx=0 tx=0).stop 2023-07-14T18:35:30.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.594+0000 7fcc217ff700 1 -- 172.21.15.133:0/2827767662 shutdown_connections 2023-07-14T18:35:30.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.594+0000 7fcc217ff700 1 --2- 172.21.15.133:0/2827767662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc1c1017a0 0x7fcc1c108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:30.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.594+0000 7fcc217ff700 1 --2- 172.21.15.133:0/2827767662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc1c0ffc20 0x7fcc1c100030 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:30.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.594+0000 7fcc217ff700 1 --2- 172.21.15.133:0/2827767662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc1c100e10 0x7fcc1c101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:30.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.594+0000 7fcc217ff700 1 -- 172.21.15.133:0/2827767662 >> 172.21.15.133:0/2827767662 conn(0x7fcc1c0fb260 msgr2=0x7fcc1c0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:30.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.594+0000 7fcc217ff700 1 -- 172.21.15.133:0/2827767662 shutdown_connections 2023-07-14T18:35:30.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.594+0000 7fcc217ff700 1 -- 172.21.15.133:0/2827767662 wait complete. 2023-07-14T18:35:30.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.595+0000 7fcc217ff700 1 Processor -- start 2023-07-14T18:35:30.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.595+0000 7fcc217ff700 1 -- start start 2023-07-14T18:35:30.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.595+0000 7fcc217ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc1c0ffc20 0x7fcc1c0743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:30.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.595+0000 7fcc217ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc1c100e10 0x7fcc1c072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:30.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.595+0000 7fcc217ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc1c1017a0 0x7fcc1c072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:30.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.595+0000 7fcc217ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c073710 con 0x7fcc1c1017a0 2023-07-14T18:35:30.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.595+0000 7fcc217ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c073880 con 0x7fcc1c100e10 2023-07-14T18:35:30.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.595+0000 7fcc217ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c19f9a0 con 0x7fcc1c0ffc20 2023-07-14T18:35:30.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.595+0000 7fcc1affd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc1c0ffc20 0x7fcc1c0743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:30.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.596+0000 7fcc1affd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc1c0ffc20 0x7fcc1c0743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:56312/0 (socket says 172.21.15.133:56312) 2023-07-14T18:35:30.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.596+0000 7fcc1affd700 1 -- 172.21.15.133:0/258250010 learned_addr learned my addr 172.21.15.133:0/258250010 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:35:30.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.596+0000 7fcc1a7fc700 1 --2- 172.21.15.133:0/258250010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc1c100e10 0x7fcc1c072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:30.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.596+0000 7fcc1b7fe700 1 --2- 172.21.15.133:0/258250010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc1c1017a0 0x7fcc1c072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:30.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.596+0000 7fcc1affd700 1 -- 172.21.15.133:0/258250010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc1c100e10 msgr2=0x7fcc1c072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:30.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.596+0000 7fcc1affd700 1 --2- 172.21.15.133:0/258250010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc1c100e10 0x7fcc1c072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:30.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.596+0000 7fcc1affd700 1 -- 172.21.15.133:0/258250010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc1c1017a0 msgr2=0x7fcc1c072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:30.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.596+0000 7fcc1affd700 1 --2- 172.21.15.133:0/258250010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc1c1017a0 0x7fcc1c072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:30.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.596+0000 7fcc1affd700 1 -- 172.21.15.133:0/258250010 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0c0095f0 con 0x7fcc1c0ffc20 2023-07-14T18:35:30.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.596+0000 7fcc1affd700 1 --2- 172.21.15.133:0/258250010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc1c0ffc20 0x7fcc1c0743f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c005720 tx=0x7fcc0c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:30.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.596+0000 7fcc03fff700 1 -- 172.21.15.133:0/258250010 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c01d070 con 0x7fcc1c0ffc20 2023-07-14T18:35:30.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.596+0000 7fcc217ff700 1 -- 172.21.15.133:0/258250010 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1c19fc20 con 0x7fcc1c0ffc20 2023-07-14T18:35:30.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.596+0000 7fcc03fff700 1 -- 172.21.15.133:0/258250010 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc0c021410 con 0x7fcc1c0ffc20 2023-07-14T18:35:30.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.596+0000 7fcc03fff700 1 -- 172.21.15.133:0/258250010 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c017670 con 0x7fcc1c0ffc20 2023-07-14T18:35:30.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.597+0000 7fcc217ff700 1 -- 172.21.15.133:0/258250010 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1c1a00a0 con 0x7fcc1c0ffc20 2023-07-14T18:35:30.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.598+0000 7fcc217ff700 1 -- 172.21.15.133:0/258250010 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc1c10b3e0 con 0x7fcc1c0ffc20 2023-07-14T18:35:30.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.600+0000 7fcc03fff700 1 -- 172.21.15.133:0/258250010 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fcc0c021580 con 0x7fcc1c0ffc20 2023-07-14T18:35:30.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.601+0000 7fcc03fff700 1 --2- 172.21.15.133:0/258250010 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fcc040a1150 0x7fcc040a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:30.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.601+0000 7fcc03fff700 1 -- 172.21.15.133:0/258250010 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fcc0c0cdda0 con 0x7fcc1c0ffc20 2023-07-14T18:35:30.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.601+0000 7fcc1a7fc700 1 --2- 172.21.15.133:0/258250010 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fcc040a1150 0x7fcc040a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:30.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.602+0000 7fcc1a7fc700 1 --2- 172.21.15.133:0/258250010 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fcc040a1150 0x7fcc040a3600 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009c60 tx=0x7fcc10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:30.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.603+0000 7fcc03fff700 1 -- 172.21.15.133:0/258250010 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fcc0c0971f0 con 0x7fcc1c0ffc20 2023-07-14T18:35:30.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.817+0000 7fcc217ff700 1 -- 172.21.15.133:0/258250010 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcc1c106b40 con 0x7fcc1c0ffc20 2023-07-14T18:35:30.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.817+0000 7fcc03fff700 1 -- 172.21.15.133:0/258250010 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v92) v1 ==== 81+0+597 (secure 0 0 0) 0x7fcc0c026020 con 0x7fcc1c0ffc20 2023-07-14T18:35:30.837 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/1832569241 2023-07-15T18:07:47.346704+0000 2023-07-14T18:35:30.837 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6801/2986586981 2023-07-15T18:07:47.346704+0000 2023-07-14T18:35:30.838 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6800/2986586981 2023-07-15T18:07:47.346704+0000 2023-07-14T18:35:30.838 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/4219815304 2023-07-15T18:07:47.346704+0000 2023-07-14T18:35:30.838 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6800/58641721 2023-07-15T18:07:19.109711+0000 2023-07-14T18:35:30.838 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6801/58641721 2023-07-15T18:07:19.109711+0000 2023-07-14T18:35:30.838 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/432253615 2023-07-15T18:07:19.109711+0000 2023-07-14T18:35:30.838 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/1840700480 2023-07-15T18:07:19.109711+0000 2023-07-14T18:35:30.838 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/1807775439 2023-07-15T18:07:19.109711+0000 2023-07-14T18:35:30.839 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/1089030734 2023-07-15T18:07:47.346704+0000 2023-07-14T18:35:30.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.820+0000 7fcc217ff700 1 -- 172.21.15.133:0/258250010 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fcc040a1150 msgr2=0x7fcc040a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:30.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.820+0000 7fcc217ff700 1 --2- 172.21.15.133:0/258250010 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fcc040a1150 0x7fcc040a3600 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009c60 tx=0x7fcc10009260 comp rx=0 tx=0).stop 2023-07-14T18:35:30.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.820+0000 7fcc217ff700 1 -- 172.21.15.133:0/258250010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc1c0ffc20 msgr2=0x7fcc1c0743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:30.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.820+0000 7fcc217ff700 1 --2- 172.21.15.133:0/258250010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc1c0ffc20 0x7fcc1c0743f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c005720 tx=0x7fcc0c009580 comp rx=0 tx=0).stop 2023-07-14T18:35:30.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.820+0000 7fcc217ff700 1 -- 172.21.15.133:0/258250010 shutdown_connections 2023-07-14T18:35:30.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.820+0000 7fcc217ff700 1 --2- 172.21.15.133:0/258250010 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7fcc040a1150 0x7fcc040a3600 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:30.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.820+0000 7fcc217ff700 1 --2- 172.21.15.133:0/258250010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc1c1017a0 0x7fcc1c072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:30.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.820+0000 7fcc217ff700 1 --2- 172.21.15.133:0/258250010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc1c0ffc20 0x7fcc1c0743f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:30.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.820+0000 7fcc217ff700 1 --2- 172.21.15.133:0/258250010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc1c100e10 0x7fcc1c072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:30.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.820+0000 7fcc217ff700 1 -- 172.21.15.133:0/258250010 >> 172.21.15.133:0/258250010 conn(0x7fcc1c0fb260 msgr2=0x7fcc1c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:30.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.820+0000 7fcc217ff700 1 -- 172.21.15.133:0/258250010 shutdown_connections 2023-07-14T18:35:30.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:30.820+0000 7fcc217ff700 1 -- 172.21.15.133:0/258250010 wait complete. 2023-07-14T18:35:30.841 INFO:teuthology.orchestra.run.smithi133.stderr:listed 10 entries 2023-07-14T18:35:30.856 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:30 smithi133 ceph-mon[126384]: pgmap v912: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.6 KiB/s wr, 6 op/s 2023-07-14T18:35:30.857 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:30 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/457901243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:35:30.857 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:30 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3940395395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:35:30.859 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-07-14T18:35:30.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:35:30.859 DEBUG:teuthology.orchestra.run.smithi133:> sudo umount /tmp/cephfs 2023-07-14T18:35:31.027 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2023-07-14T18:35:31.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:35:31.028 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2023-07-14T18:35:31.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:30 smithi161 ceph-mon[128993]: pgmap v912: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.6 KiB/s wr, 6 op/s 2023-07-14T18:35:31.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:30 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/457901243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:35:31.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:30 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3940395395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:35:31.089 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:30 smithi171 ceph-mon[129241]: pgmap v912: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.6 KiB/s wr, 6 op/s 2023-07-14T18:35:31.089 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:30 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/457901243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T18:35:31.089 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:30 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3940395395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T18:35:31.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:35:31.124 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2023-07-14T18:35:31.182 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--tmp-cephfs (id: 0) 2023-07-14T18:35:31.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:35:31.183 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns delete ceph-ns--tmp-cephfs 2023-07-14T18:35:31.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:35:31.256 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link delete ceph-brx 2023-07-14T18:35:31.376 INFO:tasks.kclient:Unmounting kernel clients... 2023-07-14T18:35:31.376 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T18:35:31.376 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T18:35:31.422 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275000k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/d741b323bc8f2b121064bd43a31641b5432c22383390593167b8c7f97dac5378/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/VWZAH4AMQLKHBCQLJ5BCNPPQVI:/var/lib/containers/storage/overlay/l/UOQIS7HIV5TAM5I2WTWWP77B7Y,upperdir=/var/lib/containers/storage/overlay/d741b323bc8f2b121064bd43a31641b5432c22383390593167b8c7f97dac5378/diff,workdir=/var/lib/containers/storage/overlay/d741b323bc8f2b121064bd43a31641b5432c22383390593167b8c7f97dac5378/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/22d46f007270791eec85ff0401a0e73403376febc2f988166716fcf2bec08111/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/VWZAH4AMQLKHBCQLJ5BCNPPQVI:/var/lib/containers/storage/overlay/l/UOQIS7HIV5TAM5I2WTWWP77B7Y,upperdir=/var/lib/containers/storage/overlay/22d46f007270791eec85ff0401a0e73403376febc2f988166716fcf2bec08111/diff,workdir=/var/lib/containers/storage/overlay/22d46f007270791eec85ff0401a0e73403376febc2f988166716fcf2bec08111/work,redirect_dir=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275000k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/41855158573f453d3699db279b271f892928382adf8190bb75b9bba545ac38dd/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/VWZAH4AMQLKHBCQLJ5BCNPPQVI:/var/lib/containers/storage/overlay/l/UOQIS7HIV5TAM5I2WTWWP77B7Y,upperdir=/var/lib/containers/storage/overlay/41855158573f453d3699db279b271f892928382adf8190bb75b9bba545ac38dd/diff,workdir=/var/lib/containers/storage/overlay/41855158573f453d3699db279b271f892928382adf8190bb75b9bba545ac38dd/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6dbd84c5c112c550708a6fdf7f8181eaf801f00ed1afc5b1b49de22b11a52cf9/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/VWZAH4AMQLKHBCQLJ5BCNPPQVI:/var/lib/containers/storage/overlay/l/UOQIS7HIV5TAM5I2WTWWP77B7Y,upperdir=/var/lib/containers/storage/overlay/6dbd84c5c112c550708a6fdf7f8181eaf801f00ed1afc5b1b49de22b11a52cf9/diff,workdir=/var/lib/containers/storage/overlay/6dbd84c5c112c550708a6fdf7f8181eaf801f00ed1afc5b1b49de22b11a52cf9/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/abf9eb9046a0fd811b3c958643f40ea4d44c36066626740ae44f4a617495a2e6/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/VWZAH4AMQLKHBCQLJ5BCNPPQVI:/var/lib/containers/storage/overlay/l/UOQIS7HIV5TAM5I2WTWWP77B7Y,upperdir=/var/lib/containers/storage/overlay/abf9eb9046a0fd811b3c958643f40ea4d44c36066626740ae44f4a617495a2e6/diff,workdir=/var/lib/containers/storage/overlay/abf9eb9046a0fd811b3c958643f40ea4d44c36066626740ae44f4a617495a2e6/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b4ff2fd5202ffe223bbde0aba98558678ed1083755bf4d5acf3ef08aae5c9338/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/VWZAH4AMQLKHBCQLJ5BCNPPQVI:/var/lib/containers/storage/overlay/l/UOQIS7HIV5TAM5I2WTWWP77B7Y,upperdir=/var/lib/containers/storage/overlay/b4ff2fd5202ffe223bbde0aba98558678ed1083755bf4d5acf3ef08aae5c9338/diff,workdir=/var/lib/containers/storage/overlay/b4ff2fd5202ffe223bbde0aba98558678ed1083755bf4d5acf3ef08aae5c9338/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/00cddaa32dd71f4c918575e04e6f152a660c478b77436a14097c8d353cc94e0d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/VWZAH4AMQLKHBCQLJ5BCNPPQVI:/var/lib/containers/storage/overlay/l/UOQIS7HIV5TAM5I2WTWWP77B7Y,upperdir=/var/lib/containers/storage/overlay/00cddaa32dd71f4c918575e04e6f152a660c478b77436a14097c8d353cc94e0d/diff,workdir=/var/lib/containers/storage/overlay/00cddaa32dd71f4c918575e04e6f152a660c478b77436a14097c8d353cc94e0d/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/8b3097d70c36bb85e52a00ef92597eb71440fc4b4773da3b8dc2738a6e7279fd/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/VWZAH4AMQLKHBCQLJ5BCNPPQVI:/var/lib/containers/storage/overlay/l/UOQIS7HIV5TAM5I2WTWWP77B7Y,upperdir=/var/lib/containers/storage/overlay/8b3097d70c36bb85e52a00ef92597eb71440fc4b4773da3b8dc2738a6e7279fd/diff,workdir=/var/lib/containers/storage/overlay/8b3097d70c36bb85e52a00ef92597eb71440fc4b4773da3b8dc2738a6e7279fd/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/7d89e585537ce28a70af2e0bcfae74cdc2233eae462481033b894f3916850816/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/VWZAH4AMQLKHBCQLJ5BCNPPQVI:/var/lib/containers/storage/overlay/l/UOQIS7HIV5TAM5I2WTWWP77B7Y,upperdir=/var/lib/containers/storage/overlay/7d89e585537ce28a70af2e0bcfae74cdc2233eae462481033b894f3916850816/diff,workdir=/var/lib/containers/storage/overlay/7d89e585537ce28a70af2e0bcfae74cdc2233eae462481033b894f3916850816/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9e74eb4c57b3145980f6fdc9d5bdfdd04b79880023f1208f17e5fd602053dc74/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/VWZAH4AMQLKHBCQLJ5BCNPPQVI:/var/lib/containers/storage/overlay/l/UOQIS7HIV5TAM5I2WTWWP77B7Y,upperdir=/var/lib/containers/storage/overlay/9e74eb4c57b3145980f6fdc9d5bdfdd04b79880023f1208f17e5fd602053dc74/diff,workdir=/var/lib/containers/storage/overlay/9e74eb4c57b3145980f6fdc9d5bdfdd04b79880023f1208f17e5fd602053dc74/work,redirect_dir=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-07-14T18:35:31.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:35:31.422 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2023-07-14T18:35:31.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T18:35:31.507 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link delete ceph-brx 2023-07-14T18:35:31.703 INFO:teuthology.orchestra.run.smithi133.stderr:Cannot find device "ceph-brx" 2023-07-14T18:35:31.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T18:35:31.707 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-07-14T18:35:31.719 INFO:tasks.cephadm:Teardown begin 2023-07-14T18:35:31.720 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-14T18:35:31.721 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T18:35:31.880 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:31 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/258250010' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T18:35:31.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.901+0000 7f3bfedff700 1 -- 172.21.15.133:0/59133531 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3bf80ffc30 msgr2=0x7f3bf8100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:31.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.901+0000 7f3bfedff700 1 --2- 172.21.15.133:0/59133531 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3bf80ffc30 0x7f3bf8100040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3bec0098f0 tx=0x7f3bec009c00 comp rx=0 tx=0).stop 2023-07-14T18:35:31.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.901+0000 7f3bfedff700 1 -- 172.21.15.133:0/59133531 shutdown_connections 2023-07-14T18:35:31.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.901+0000 7f3bfedff700 1 --2- 172.21.15.133:0/59133531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bf81017b0 0x7f3bf8108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:31.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.901+0000 7f3bfedff700 1 --2- 172.21.15.133:0/59133531 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3bf80ffc30 0x7f3bf8100040 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:31.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.901+0000 7f3bfedff700 1 --2- 172.21.15.133:0/59133531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bf8100e20 0x7f3bf8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:31.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.901+0000 7f3bfedff700 1 -- 172.21.15.133:0/59133531 >> 172.21.15.133:0/59133531 conn(0x7f3bf80fb210 msgr2=0x7f3bf80fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:31.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.901+0000 7f3bfedff700 1 -- 172.21.15.133:0/59133531 shutdown_connections 2023-07-14T18:35:31.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.901+0000 7f3bfedff700 1 -- 172.21.15.133:0/59133531 wait complete. 2023-07-14T18:35:31.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.902+0000 7f3bfedff700 1 Processor -- start 2023-07-14T18:35:31.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.902+0000 7f3bfedff700 1 -- start start 2023-07-14T18:35:31.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.902+0000 7f3bfedff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bf80ffc30 0x7f3bf80743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:31.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.902+0000 7f3bfedff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bf8100e20 0x7f3bf8072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:31.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.902+0000 7f3bfedff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3bf81017b0 0x7f3bf8072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:31.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.902+0000 7f3bfedff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf8073710 con 0x7f3bf8100e20 2023-07-14T18:35:31.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.902+0000 7f3bfedff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf8073880 con 0x7f3bf80ffc30 2023-07-14T18:35:31.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.902+0000 7f3bfedff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf819f9f0 con 0x7f3bf81017b0 2023-07-14T18:35:31.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.903+0000 7f3bfcb9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bf80ffc30 0x7f3bf80743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:31.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.903+0000 7f3bfcb9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bf80ffc30 0x7f3bf80743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:57610/0 (socket says 172.21.15.133:57610) 2023-07-14T18:35:31.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.903+0000 7f3bfcb9b700 1 -- 172.21.15.133:0/4269583329 learned_addr learned my addr 172.21.15.133:0/4269583329 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:35:31.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.903+0000 7f3bfd39c700 1 --2- 172.21.15.133:0/4269583329 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3bf81017b0 0x7f3bf8072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:31.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.903+0000 7f3bf7fff700 1 --2- 172.21.15.133:0/4269583329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bf8100e20 0x7f3bf8072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:31.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.903+0000 7f3bfcb9b700 1 -- 172.21.15.133:0/4269583329 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3bf81017b0 msgr2=0x7f3bf8072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:31.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.903+0000 7f3bfcb9b700 1 --2- 172.21.15.133:0/4269583329 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3bf81017b0 0x7f3bf8072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:31.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.903+0000 7f3bfcb9b700 1 -- 172.21.15.133:0/4269583329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bf8100e20 msgr2=0x7f3bf8072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:31.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.903+0000 7f3bfcb9b700 1 --2- 172.21.15.133:0/4269583329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bf8100e20 0x7f3bf8072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:31.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.903+0000 7f3bfcb9b700 1 -- 172.21.15.133:0/4269583329 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bec009520 con 0x7f3bf80ffc30 2023-07-14T18:35:31.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.903+0000 7f3bfcb9b700 1 --2- 172.21.15.133:0/4269583329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bf80ffc30 0x7f3bf80743f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3bec009fd0 tx=0x7f3bec00b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:31.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.903+0000 7f3bf5ffb700 1 -- 172.21.15.133:0/4269583329 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec01c070 con 0x7f3bf80ffc30 2023-07-14T18:35:31.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.903+0000 7f3bf5ffb700 1 -- 172.21.15.133:0/4269583329 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bec00f7d0 con 0x7f3bf80ffc30 2023-07-14T18:35:31.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.903+0000 7f3bfedff700 1 -- 172.21.15.133:0/4269583329 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf819fc70 con 0x7f3bf80ffc30 2023-07-14T18:35:31.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.903+0000 7f3bf5ffb700 1 -- 172.21.15.133:0/4269583329 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec003a20 con 0x7f3bf80ffc30 2023-07-14T18:35:31.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.904+0000 7f3bfedff700 1 -- 172.21.15.133:0/4269583329 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf81a00f0 con 0x7f3bf80ffc30 2023-07-14T18:35:31.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.905+0000 7f3bfedff700 1 -- 172.21.15.133:0/4269583329 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf8107220 con 0x7f3bf80ffc30 2023-07-14T18:35:31.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.907+0000 7f3bf5ffb700 1 -- 172.21.15.133:0/4269583329 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f3bec020030 con 0x7f3bf80ffc30 2023-07-14T18:35:31.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.908+0000 7f3bf5ffb700 1 --2- 172.21.15.133:0/4269583329 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3be80a11a0 0x7f3be80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:31.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.908+0000 7f3bf5ffb700 1 -- 172.21.15.133:0/4269583329 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3bec0ce500 con 0x7f3bf80ffc30 2023-07-14T18:35:31.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.908+0000 7f3bf7fff700 1 --2- 172.21.15.133:0/4269583329 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3be80a11a0 0x7f3be80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:31.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.909+0000 7f3bf7fff700 1 --2- 172.21.15.133:0/4269583329 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3be80a11a0 0x7f3be80a3650 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3be4005fd0 tx=0x7f3be4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:31.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:31.910+0000 7f3bf5ffb700 1 -- 172.21.15.133:0/4269583329 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f3bec097950 con 0x7f3bf80ffc30 2023-07-14T18:35:32.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:31 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/258250010' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T18:35:32.089 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:31 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/258250010' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T18:35:32.172 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.153+0000 7f3bfedff700 1 -- 172.21.15.133:0/4269583329 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3bf802c8e0 con 0x7f3bf80ffc30 2023-07-14T18:35:32.173 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.154+0000 7f3bf5ffb700 1 -- 172.21.15.133:0/4269583329 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+10857 (secure 0 0 0) 0x7f3bec084e60 con 0x7f3bf80ffc30 2023-07-14T18:35:32.174 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:35:32.174 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3168151271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3168151271},{"type":"v1","addr":"172.21.15.133:6837","nonce":3168151271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14544,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/3093144261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3093144261},{"type":"v1","addr":"172.21.15.133:6839","nonce":3093144261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14556,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/3282902068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":3282902068},{"type":"v1","addr":"172.21.15.133:6841","nonce":3282902068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/4023127259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4023127259},{"type":"v1","addr":"172.21.15.161:6833","nonce":4023127259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2722373740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2722373740},{"type":"v1","addr":"172.21.15.171:6833","nonce":2722373740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2399005074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2399005074},{"type":"v1","addr":"172.21.15.133:6835","nonce":2399005074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":49,"flags":178,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"balance_automate":true},"ever_allowed_features":160,"explicitly_allowed_features":160,"created":"2023-07-14T18:14:17.440873+0000","modified":"2023-07-14T18:34:56.545210+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24557,"mds_1":24541,"mds_2":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3725502564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3725502564},{"type":"v1","addr":"172.21.15.161:6835","nonce":3725502564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6835/1877362116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1877362116},{"type":"v1","addr":"172.21.15.171:6835","nonce":1877362116}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/3458064798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3458064798},{"type":"v1","addr":"172.21.15.161:6837","nonce":3458064798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/106205332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":106205332},{"type":"v1","addr":"172.21.15.171:6837","nonce":106205332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.171:6839/816201198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":816201198},{"type":"v1","addr":"172.21.15.171:6839","nonce":816201198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/2503865871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2503865871},{"type":"v1","addr":"172.21.15.161:6839","nonce":2503865871}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T18:35:32.176 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.157+0000 7f3bfedff700 1 -- 172.21.15.133:0/4269583329 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3be80a11a0 msgr2=0x7f3be80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:32.176 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.157+0000 7f3bfedff700 1 --2- 172.21.15.133:0/4269583329 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3be80a11a0 0x7f3be80a3650 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3be4005fd0 tx=0x7f3be4009310 comp rx=0 tx=0).stop 2023-07-14T18:35:32.176 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.157+0000 7f3bfedff700 1 -- 172.21.15.133:0/4269583329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bf80ffc30 msgr2=0x7f3bf80743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:32.176 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.157+0000 7f3bfedff700 1 --2- 172.21.15.133:0/4269583329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bf80ffc30 0x7f3bf80743f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3bec009fd0 tx=0x7f3bec00b790 comp rx=0 tx=0).stop 2023-07-14T18:35:32.177 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.157+0000 7f3bfedff700 1 -- 172.21.15.133:0/4269583329 shutdown_connections 2023-07-14T18:35:32.177 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.157+0000 7f3bfedff700 1 --2- 172.21.15.133:0/4269583329 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f3be80a11a0 0x7f3be80a3650 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:32.177 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.157+0000 7f3bfedff700 1 --2- 172.21.15.133:0/4269583329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bf80ffc30 0x7f3bf80743f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:32.177 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.157+0000 7f3bfedff700 1 --2- 172.21.15.133:0/4269583329 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3bf81017b0 0x7f3bf8072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:32.177 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.157+0000 7f3bfedff700 1 --2- 172.21.15.133:0/4269583329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bf8100e20 0x7f3bf8072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:32.177 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.157+0000 7f3bfedff700 1 -- 172.21.15.133:0/4269583329 >> 172.21.15.133:0/4269583329 conn(0x7f3bf80fb210 msgr2=0x7f3bf8104360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:32.177 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.157+0000 7f3bfedff700 1 -- 172.21.15.133:0/4269583329 shutdown_connections 2023-07-14T18:35:32.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.157+0000 7f3bfedff700 1 -- 172.21.15.133:0/4269583329 wait complete. 2023-07-14T18:35:32.178 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 49 2023-07-14T18:35:32.195 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T18:35:32.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.373+0000 7f64debff700 1 -- 172.21.15.133:0/110722711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d8100e20 msgr2=0x7f64d8101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:32.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.373+0000 7f64debff700 1 --2- 172.21.15.133:0/110722711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d8100e20 0x7f64d8101270 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f64c4009990 tx=0x7f64c4009ca0 comp rx=0 tx=0).stop 2023-07-14T18:35:32.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.373+0000 7f64debff700 1 -- 172.21.15.133:0/110722711 shutdown_connections 2023-07-14T18:35:32.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.373+0000 7f64debff700 1 --2- 172.21.15.133:0/110722711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64d81017b0 0x7f64d8108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:32.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.373+0000 7f64debff700 1 --2- 172.21.15.133:0/110722711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64d80ffc30 0x7f64d8100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:32.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.373+0000 7f64debff700 1 --2- 172.21.15.133:0/110722711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d8100e20 0x7f64d8101270 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:32.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.373+0000 7f64debff700 1 -- 172.21.15.133:0/110722711 >> 172.21.15.133:0/110722711 conn(0x7f64d80fb210 msgr2=0x7f64d80fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:32.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.373+0000 7f64debff700 1 -- 172.21.15.133:0/110722711 shutdown_connections 2023-07-14T18:35:32.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.373+0000 7f64debff700 1 -- 172.21.15.133:0/110722711 wait complete. 2023-07-14T18:35:32.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.374+0000 7f64debff700 1 Processor -- start 2023-07-14T18:35:32.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.374+0000 7f64debff700 1 -- start start 2023-07-14T18:35:32.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.374+0000 7f64debff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64d80ffc30 0x7f64d8197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:32.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.374+0000 7f64debff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64d8100e20 0x7f64d81980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:32.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.374+0000 7f64debff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d81017b0 0x7f64d819f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:32.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.374+0000 7f64debff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d819f700 con 0x7f64d80ffc30 2023-07-14T18:35:32.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.374+0000 7f64debff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d819f840 con 0x7f64d81017b0 2023-07-14T18:35:32.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.374+0000 7f64debff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d819f9d0 con 0x7f64d8100e20 2023-07-14T18:35:32.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.375+0000 7f64d7fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64d8100e20 0x7f64d81980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:32.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.375+0000 7f64dd19c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d81017b0 0x7f64d819f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:32.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.375+0000 7f64d7fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64d8100e20 0x7f64d81980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:56350/0 (socket says 172.21.15.133:56350) 2023-07-14T18:35:32.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.375+0000 7f64d7fff700 1 -- 172.21.15.133:0/2450253158 learned_addr learned my addr 172.21.15.133:0/2450253158 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:35:32.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.375+0000 7f64dc99b700 1 --2- 172.21.15.133:0/2450253158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64d80ffc30 0x7f64d8197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:32.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.375+0000 7f64d7fff700 1 -- 172.21.15.133:0/2450253158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d81017b0 msgr2=0x7f64d819f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:32.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.375+0000 7f64d7fff700 1 --2- 172.21.15.133:0/2450253158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d81017b0 0x7f64d819f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:32.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.375+0000 7f64d7fff700 1 -- 172.21.15.133:0/2450253158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64d80ffc30 msgr2=0x7f64d8197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:32.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.375+0000 7f64d7fff700 1 --2- 172.21.15.133:0/2450253158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64d80ffc30 0x7f64d8197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:32.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.375+0000 7f64d7fff700 1 -- 172.21.15.133:0/2450253158 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c40095f0 con 0x7f64d8100e20 2023-07-14T18:35:32.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.375+0000 7f64d7fff700 1 --2- 172.21.15.133:0/2450253158 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64d8100e20 0x7f64d81980c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f64c4004760 tx=0x7f64c4004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:32.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.375+0000 7f64d5ffb700 1 -- 172.21.15.133:0/2450253158 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c401d070 con 0x7f64d8100e20 2023-07-14T18:35:32.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.375+0000 7f64debff700 1 -- 172.21.15.133:0/2450253158 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d819fc50 con 0x7f64d8100e20 2023-07-14T18:35:32.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.375+0000 7f64d5ffb700 1 -- 172.21.15.133:0/2450253158 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64c4004830 con 0x7f64d8100e20 2023-07-14T18:35:32.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.375+0000 7f64d5ffb700 1 -- 172.21.15.133:0/2450253158 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4005350 con 0x7f64d8100e20 2023-07-14T18:35:32.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.376+0000 7f64debff700 1 -- 172.21.15.133:0/2450253158 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d81a0020 con 0x7f64d8100e20 2023-07-14T18:35:32.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.376+0000 7f64debff700 1 -- 172.21.15.133:0/2450253158 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d810b4c0 con 0x7f64d8100e20 2023-07-14T18:35:32.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.378+0000 7f64d5ffb700 1 -- 172.21.15.133:0/2450253158 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f64c4005510 con 0x7f64d8100e20 2023-07-14T18:35:32.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.380+0000 7f64d5ffb700 1 --2- 172.21.15.133:0/2450253158 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f64c80a11a0 0x7f64c80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:32.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.380+0000 7f64d5ffb700 1 -- 172.21.15.133:0/2450253158 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f64c40cddb0 con 0x7f64d8100e20 2023-07-14T18:35:32.399 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.380+0000 7f64dc99b700 1 --2- 172.21.15.133:0/2450253158 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f64c80a11a0 0x7f64c80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:32.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.381+0000 7f64dc99b700 1 --2- 172.21.15.133:0/2450253158 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f64c80a11a0 0x7f64c80a3650 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f64d8100c80 tx=0x7f64cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:32.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.381+0000 7f64d5ffb700 1 -- 172.21.15.133:0/2450253158 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f64c4097200 con 0x7f64d8100e20 2023-07-14T18:35:32.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.627+0000 7f64debff700 1 -- 172.21.15.133:0/2450253158 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64d81a0630 con 0x7f64d8100e20 2023-07-14T18:35:32.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.628+0000 7f64d5ffb700 1 -- 172.21.15.133:0/2450253158 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+10857 (secure 0 0 0) 0x7f64c409a820 con 0x7f64d8100e20 2023-07-14T18:35:32.647 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:35:32.647 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3168151271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3168151271},{"type":"v1","addr":"172.21.15.133:6837","nonce":3168151271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14544,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/3093144261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3093144261},{"type":"v1","addr":"172.21.15.133:6839","nonce":3093144261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14556,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/3282902068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":3282902068},{"type":"v1","addr":"172.21.15.133:6841","nonce":3282902068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/4023127259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4023127259},{"type":"v1","addr":"172.21.15.161:6833","nonce":4023127259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2722373740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2722373740},{"type":"v1","addr":"172.21.15.171:6833","nonce":2722373740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2399005074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2399005074},{"type":"v1","addr":"172.21.15.133:6835","nonce":2399005074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":49,"flags":178,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"balance_automate":true},"ever_allowed_features":160,"explicitly_allowed_features":160,"created":"2023-07-14T18:14:17.440873+0000","modified":"2023-07-14T18:34:56.545210+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24557,"mds_1":24541,"mds_2":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3725502564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3725502564},{"type":"v1","addr":"172.21.15.161:6835","nonce":3725502564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6835/1877362116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1877362116},{"type":"v1","addr":"172.21.15.171:6835","nonce":1877362116}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/3458064798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3458064798},{"type":"v1","addr":"172.21.15.161:6837","nonce":3458064798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/106205332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":106205332},{"type":"v1","addr":"172.21.15.171:6837","nonce":106205332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.171:6839/816201198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":816201198},{"type":"v1","addr":"172.21.15.171:6839","nonce":816201198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/2503865871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2503865871},{"type":"v1","addr":"172.21.15.161:6839","nonce":2503865871}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T18:35:32.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.630+0000 7f64debff700 1 -- 172.21.15.133:0/2450253158 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f64c80a11a0 msgr2=0x7f64c80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:32.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.630+0000 7f64debff700 1 --2- 172.21.15.133:0/2450253158 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f64c80a11a0 0x7f64c80a3650 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f64d8100c80 tx=0x7f64cc009260 comp rx=0 tx=0).stop 2023-07-14T18:35:32.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.630+0000 7f64debff700 1 -- 172.21.15.133:0/2450253158 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64d8100e20 msgr2=0x7f64d81980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:32.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.630+0000 7f64debff700 1 --2- 172.21.15.133:0/2450253158 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64d8100e20 0x7f64d81980c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f64c4004760 tx=0x7f64c4004790 comp rx=0 tx=0).stop 2023-07-14T18:35:32.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.630+0000 7f64debff700 1 -- 172.21.15.133:0/2450253158 shutdown_connections 2023-07-14T18:35:32.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.630+0000 7f64debff700 1 --2- 172.21.15.133:0/2450253158 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f64c80a11a0 0x7f64c80a3650 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:32.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.630+0000 7f64debff700 1 --2- 172.21.15.133:0/2450253158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d81017b0 0x7f64d819f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:32.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.630+0000 7f64debff700 1 --2- 172.21.15.133:0/2450253158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64d80ffc30 0x7f64d8197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:32.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.630+0000 7f64debff700 1 --2- 172.21.15.133:0/2450253158 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64d8100e20 0x7f64d81980c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:32.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.630+0000 7f64debff700 1 -- 172.21.15.133:0/2450253158 >> 172.21.15.133:0/2450253158 conn(0x7f64d80fb210 msgr2=0x7f64d80fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:32.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.631+0000 7f64debff700 1 -- 172.21.15.133:0/2450253158 shutdown_connections 2023-07-14T18:35:32.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.631+0000 7f64debff700 1 -- 172.21.15.133:0/2450253158 wait complete. 2023-07-14T18:35:32.652 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 49 2023-07-14T18:35:32.669 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T18:35:32.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.848+0000 7f21425ff700 1 -- 172.21.15.133:0/2793957656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f213c100e20 msgr2=0x7f213c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:32.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.848+0000 7f21425ff700 1 --2- 172.21.15.133:0/2793957656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f213c100e20 0x7f213c101270 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2130009990 tx=0x7f2130009ca0 comp rx=0 tx=0).stop 2023-07-14T18:35:32.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.848+0000 7f21425ff700 1 -- 172.21.15.133:0/2793957656 shutdown_connections 2023-07-14T18:35:32.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.848+0000 7f21425ff700 1 --2- 172.21.15.133:0/2793957656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f213c1017b0 0x7f213c108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:32.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.848+0000 7f21425ff700 1 --2- 172.21.15.133:0/2793957656 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f213c0ffc30 0x7f213c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:32.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.848+0000 7f21425ff700 1 --2- 172.21.15.133:0/2793957656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f213c100e20 0x7f213c101270 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:32.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.848+0000 7f21425ff700 1 -- 172.21.15.133:0/2793957656 >> 172.21.15.133:0/2793957656 conn(0x7f213c0fb210 msgr2=0x7f213c0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:32.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.848+0000 7f21425ff700 1 -- 172.21.15.133:0/2793957656 shutdown_connections 2023-07-14T18:35:32.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.848+0000 7f21425ff700 1 -- 172.21.15.133:0/2793957656 wait complete. 2023-07-14T18:35:32.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.849+0000 7f21425ff700 1 Processor -- start 2023-07-14T18:35:32.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.849+0000 7f21425ff700 1 -- start start 2023-07-14T18:35:32.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.850+0000 7f21425ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f213c0ffc30 0x7f213c195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:32.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.850+0000 7f21425ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f213c100e20 0x7f213c195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:32.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.850+0000 7f21425ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f213c1017b0 0x7f213c19cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:32.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.850+0000 7f21425ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c19d4c0 con 0x7f213c0ffc30 2023-07-14T18:35:32.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.850+0000 7f21425ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c19d600 con 0x7f213c100e20 2023-07-14T18:35:32.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.850+0000 7f21425ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c19d790 con 0x7f213c1017b0 2023-07-14T18:35:32.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.850+0000 7f213bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f213c0ffc30 0x7f213c195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:32.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.850+0000 7f213b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f213c100e20 0x7f213c195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:32.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.850+0000 7f2140b9c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f213c1017b0 0x7f213c19cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:32.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.850+0000 7f213b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f213c100e20 0x7f213c195e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:57648/0 (socket says 172.21.15.133:57648) 2023-07-14T18:35:32.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.850+0000 7f213b7fe700 1 -- 172.21.15.133:0/3423827339 learned_addr learned my addr 172.21.15.133:0/3423827339 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:35:32.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.850+0000 7f2140b9c700 1 -- 172.21.15.133:0/3423827339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f213c100e20 msgr2=0x7f213c195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:32.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.850+0000 7f2140b9c700 1 --2- 172.21.15.133:0/3423827339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f213c100e20 0x7f213c195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:32.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.850+0000 7f2140b9c700 1 -- 172.21.15.133:0/3423827339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f213c0ffc30 msgr2=0x7f213c195940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:32.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.850+0000 7f2140b9c700 1 --2- 172.21.15.133:0/3423827339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f213c0ffc30 0x7f213c195940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:32.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.850+0000 7f2140b9c700 1 -- 172.21.15.133:0/3423827339 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21300095f0 con 0x7f213c1017b0 2023-07-14T18:35:32.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.850+0000 7f213b7fe700 1 --2- 172.21.15.133:0/3423827339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f213c100e20 0x7f213c195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:35:32.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.850+0000 7f213bfff700 1 --2- 172.21.15.133:0/3423827339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f213c0ffc30 0x7f213c195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:35:32.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.851+0000 7f2140b9c700 1 --2- 172.21.15.133:0/3423827339 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f213c1017b0 0x7f213c19cef0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f213400efc0 tx=0x7f213400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:32.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.851+0000 7f21397fa700 1 -- 172.21.15.133:0/3423827339 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2134018070 con 0x7f213c1017b0 2023-07-14T18:35:32.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.851+0000 7f21425ff700 1 -- 172.21.15.133:0/3423827339 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f213c19da10 con 0x7f213c1017b0 2023-07-14T18:35:32.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.851+0000 7f21397fa700 1 -- 172.21.15.133:0/3423827339 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f213400f040 con 0x7f213c1017b0 2023-07-14T18:35:32.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.851+0000 7f21397fa700 1 -- 172.21.15.133:0/3423827339 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213400cd40 con 0x7f213c1017b0 2023-07-14T18:35:32.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.851+0000 7f21425ff700 1 -- 172.21.15.133:0/3423827339 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f213c19df20 con 0x7f213c1017b0 2023-07-14T18:35:32.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.852+0000 7f21425ff700 1 -- 172.21.15.133:0/3423827339 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f213c107180 con 0x7f213c1017b0 2023-07-14T18:35:32.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.854+0000 7f21397fa700 1 -- 172.21.15.133:0/3423827339 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f2134003a00 con 0x7f213c1017b0 2023-07-14T18:35:32.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.855+0000 7f21397fa700 1 --2- 172.21.15.133:0/3423827339 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f21240a11a0 0x7f21240a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:32.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.855+0000 7f21397fa700 1 -- 172.21.15.133:0/3423827339 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f21340cc9d0 con 0x7f213c1017b0 2023-07-14T18:35:32.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.855+0000 7f213bfff700 1 --2- 172.21.15.133:0/3423827339 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f21240a11a0 0x7f21240a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:32.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.856+0000 7f213bfff700 1 --2- 172.21.15.133:0/3423827339 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f21240a11a0 0x7f21240a3650 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f213c100c80 tx=0x7f212c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:32.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:32.857+0000 7f21397fa700 1 -- 172.21.15.133:0/3423827339 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f2134095e20 con 0x7f213c1017b0 2023-07-14T18:35:33.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:32 smithi161 ceph-mon[128993]: pgmap v913: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.6 KiB/s wr, 8 op/s 2023-07-14T18:35:33.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:32 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/4269583329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:35:33.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.058+0000 7f21425ff700 1 -- 172.21.15.133:0/3423827339 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f213c02caf0 con 0x7f213c1017b0 2023-07-14T18:35:33.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.061+0000 7f21397fa700 1 -- 172.21.15.133:0/3423827339 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24520 (secure 0 0 0) 0x7f213404dfe0 con 0x7f213c1017b0 2023-07-14T18:35:33.079 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:35:33.080 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":92,"fsid":"06d0f5fa-2271-11ee-9b33-001a4aab830c","created":"2023-07-14T18:06:49.561582+0000","modified":"2023-07-14T18:15:49.951050+0000","last_up_change":"2023-07-14T18:13:29.836099+0000","last_in_change":"2023-07-14T18:13:11.606710+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T18:10:03.945039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T18:14:15.451340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T18:14:16.892215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T18:14:20.493175+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"85a29de7-d712-422a-b647-d9e90091aaf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6803","nonce":3076510801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6805","nonce":3076510801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6809","nonce":3076510801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3076510801},{"type":"v1","addr":"172.21.15.133:6807","nonce":3076510801}]},"public_addr":"172.21.15.133:6803/3076510801","cluster_addr":"172.21.15.133:6805/3076510801","heartbeat_back_addr":"172.21.15.133:6809/3076510801","heartbeat_front_addr":"172.21.15.133:6807/3076510801","state":["exists","up"]},{"osd":1,"uuid":"80ad4e78-7917-495e-b119-99f7c7c36243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6801","nonce":1841901852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6803","nonce":1841901852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6807","nonce":1841901852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1841901852},{"type":"v1","addr":"172.21.15.161:6805","nonce":1841901852}]},"public_addr":"172.21.15.161:6801/1841901852","cluster_addr":"172.21.15.161:6803/1841901852","heartbeat_back_addr":"172.21.15.161:6807/1841901852","heartbeat_front_addr":"172.21.15.161:6805/1841901852","state":["exists","up"]},{"osd":2,"uuid":"fc4bfd37-1fdd-47c6-a48e-6d910daf9e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6801","nonce":346103099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6803","nonce":346103099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6807","nonce":346103099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":346103099},{"type":"v1","addr":"172.21.15.171:6805","nonce":346103099}]},"public_addr":"172.21.15.171:6801/346103099","cluster_addr":"172.21.15.171:6803/346103099","heartbeat_back_addr":"172.21.15.171:6807/346103099","heartbeat_front_addr":"172.21.15.171:6805/346103099","state":["exists","up"]},{"osd":3,"uuid":"4e1cfbc2-5c62-4d5c-bfd0-9df4ad5981cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6811","nonce":2152994143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6813","nonce":2152994143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6817","nonce":2152994143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2152994143},{"type":"v1","addr":"172.21.15.133:6815","nonce":2152994143}]},"public_addr":"172.21.15.133:6811/2152994143","cluster_addr":"172.21.15.133:6813/2152994143","heartbeat_back_addr":"172.21.15.133:6817/2152994143","heartbeat_front_addr":"172.21.15.133:6815/2152994143","state":["exists","up"]},{"osd":4,"uuid":"4601a1c2-254b-4c4d-9b15-8fc952b3d2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6809","nonce":47213291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6811","nonce":47213291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6815","nonce":47213291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":47213291},{"type":"v1","addr":"172.21.15.161:6813","nonce":47213291}]},"public_addr":"172.21.15.161:6809/47213291","cluster_addr":"172.21.15.161:6811/47213291","heartbeat_back_addr":"172.21.15.161:6815/47213291","heartbeat_front_addr":"172.21.15.161:6813/47213291","state":["exists","up"]},{"osd":5,"uuid":"6cca180a-9140-4e49-835b-86842dae0574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6809","nonce":777556751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6811","nonce":777556751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6815","nonce":777556751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":777556751},{"type":"v1","addr":"172.21.15.171:6813","nonce":777556751}]},"public_addr":"172.21.15.171:6809/777556751","cluster_addr":"172.21.15.171:6811/777556751","heartbeat_back_addr":"172.21.15.171:6815/777556751","heartbeat_front_addr":"172.21.15.171:6813/777556751","state":["exists","up"]},{"osd":6,"uuid":"ea0299ff-7740-4494-a3a9-1503b3fc0ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6819","nonce":2884152358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6821","nonce":2884152358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6825","nonce":2884152358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":2884152358},{"type":"v1","addr":"172.21.15.133:6823","nonce":2884152358}]},"public_addr":"172.21.15.133:6819/2884152358","cluster_addr":"172.21.15.133:6821/2884152358","heartbeat_back_addr":"172.21.15.133:6825/2884152358","heartbeat_front_addr":"172.21.15.133:6823/2884152358","state":["exists","up"]},{"osd":7,"uuid":"b8f07ece-7d4b-4b62-b393-9e2b9ff6bccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6817","nonce":4183705749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6819","nonce":4183705749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6823","nonce":4183705749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":4183705749},{"type":"v1","addr":"172.21.15.161:6821","nonce":4183705749}]},"public_addr":"172.21.15.161:6817/4183705749","cluster_addr":"172.21.15.161:6819/4183705749","heartbeat_back_addr":"172.21.15.161:6823/4183705749","heartbeat_front_addr":"172.21.15.161:6821/4183705749","state":["exists","up"]},{"osd":8,"uuid":"87dffc07-bf48-48c8-b418-5c4b509686d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6817","nonce":2771494483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6819","nonce":2771494483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6823","nonce":2771494483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2771494483},{"type":"v1","addr":"172.21.15.171:6821","nonce":2771494483}]},"public_addr":"172.21.15.171:6817/2771494483","cluster_addr":"172.21.15.171:6819/2771494483","heartbeat_back_addr":"172.21.15.171:6823/2771494483","heartbeat_front_addr":"172.21.15.171:6821/2771494483","state":["exists","up"]},{"osd":9,"uuid":"e0832870-79ce-4bce-8045-58b7a7f6dc69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6827","nonce":2066164149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6829","nonce":2066164149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6833","nonce":2066164149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2066164149},{"type":"v1","addr":"172.21.15.133:6831","nonce":2066164149}]},"public_addr":"172.21.15.133:6827/2066164149","cluster_addr":"172.21.15.133:6829/2066164149","heartbeat_back_addr":"172.21.15.133:6833/2066164149","heartbeat_front_addr":"172.21.15.133:6831/2066164149","state":["exists","up"]},{"osd":10,"uuid":"081f0484-d134-4b2e-9c48-ef8e6a646422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6825","nonce":2162333626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6827","nonce":2162333626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6831","nonce":2162333626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2162333626},{"type":"v1","addr":"172.21.15.161:6829","nonce":2162333626}]},"public_addr":"172.21.15.161:6825/2162333626","cluster_addr":"172.21.15.161:6827/2162333626","heartbeat_back_addr":"172.21.15.161:6831/2162333626","heartbeat_front_addr":"172.21.15.161:6829/2162333626","state":["exists","up"]},{"osd":11,"uuid":"4fdb5a1f-2ad8-45fc-8c10-5e90cacb14aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6825","nonce":3403894313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6827","nonce":3403894313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6831","nonce":3403894313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":3403894313},{"type":"v1","addr":"172.21.15.171:6829","nonce":3403894313}]},"public_addr":"172.21.15.171:6825/3403894313","cluster_addr":"172.21.15.171:6827/3403894313","heartbeat_back_addr":"172.21.15.171:6831/3403894313","heartbeat_front_addr":"172.21.15.171:6829/3403894313","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:17.816229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:09:39.638543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:00.543317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:26.988921+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:10:48.644694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:10.049860+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:35.986508+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:11:57.753846+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:19.018245+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:12:44.460450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:06.279960+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T18:13:27.955006+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/1832569241":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6801/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/2986586981":"2023-07-15T18:07:47.346704+0000","172.21.15.133:0/4219815304":"2023-07-15T18:07:47.346704+0000","172.21.15.133:6800/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:6801/58641721":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/432253615":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1840700480":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1807775439":"2023-07-15T18:07:19.109711+0000","172.21.15.133:0/1089030734":"2023-07-15T18:07:47.346704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T18:35:33.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.062+0000 7f21425ff700 1 -- 172.21.15.133:0/3423827339 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f21240a11a0 msgr2=0x7f21240a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:33.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.062+0000 7f21425ff700 1 --2- 172.21.15.133:0/3423827339 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f21240a11a0 0x7f21240a3650 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f213c100c80 tx=0x7f212c009260 comp rx=0 tx=0).stop 2023-07-14T18:35:33.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.062+0000 7f21425ff700 1 -- 172.21.15.133:0/3423827339 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f213c1017b0 msgr2=0x7f213c19cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:33.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.062+0000 7f21425ff700 1 --2- 172.21.15.133:0/3423827339 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f213c1017b0 0x7f213c19cef0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f213400efc0 tx=0x7f213400c3a0 comp rx=0 tx=0).stop 2023-07-14T18:35:33.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.062+0000 7f21425ff700 1 -- 172.21.15.133:0/3423827339 shutdown_connections 2023-07-14T18:35:33.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.062+0000 7f21425ff700 1 --2- 172.21.15.133:0/3423827339 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f21240a11a0 0x7f21240a3650 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:33.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.062+0000 7f21425ff700 1 --2- 172.21.15.133:0/3423827339 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f213c1017b0 0x7f213c19cef0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:33.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.062+0000 7f21425ff700 1 --2- 172.21.15.133:0/3423827339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f213c0ffc30 0x7f213c195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:33.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.062+0000 7f21425ff700 1 --2- 172.21.15.133:0/3423827339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f213c100e20 0x7f213c195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:33.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.062+0000 7f21425ff700 1 -- 172.21.15.133:0/3423827339 >> 172.21.15.133:0/3423827339 conn(0x7f213c0fb210 msgr2=0x7f213c1042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:33.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.063+0000 7f21425ff700 1 -- 172.21.15.133:0/3423827339 shutdown_connections 2023-07-14T18:35:33.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.063+0000 7f21425ff700 1 -- 172.21.15.133:0/3423827339 wait complete. 2023-07-14T18:35:33.089 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:32 smithi171 ceph-mon[129241]: pgmap v913: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.6 KiB/s wr, 8 op/s 2023-07-14T18:35:33.089 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:32 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/4269583329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:35:33.098 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:32 smithi133 ceph-mon[126384]: pgmap v913: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.6 KiB/s wr, 8 op/s 2023-07-14T18:35:33.099 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:32 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/4269583329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:35:33.102 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-14T18:35:33.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.296+0000 7f92bd1ff700 1 -- 172.21.15.133:0/9432039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92b80730a0 msgr2=0x7f92b8073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:33.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.296+0000 7f92bd1ff700 1 --2- 172.21.15.133:0/9432039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92b80730a0 0x7f92b8073510 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f92ac009990 tx=0x7f92ac009ca0 comp rx=0 tx=0).stop 2023-07-14T18:35:33.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.296+0000 7f92bd1ff700 1 -- 172.21.15.133:0/9432039 shutdown_connections 2023-07-14T18:35:33.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.296+0000 7f92bd1ff700 1 --2- 172.21.15.133:0/9432039 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92b8107ad0 0x7f92b8109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:33.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.296+0000 7f92bd1ff700 1 --2- 172.21.15.133:0/9432039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b8074680 0x7f92b8072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:33.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.296+0000 7f92bd1ff700 1 --2- 172.21.15.133:0/9432039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92b80730a0 0x7f92b8073510 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:33.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.296+0000 7f92bd1ff700 1 -- 172.21.15.133:0/9432039 >> 172.21.15.133:0/9432039 conn(0x7f92b80fb210 msgr2=0x7f92b80fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:33.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.296+0000 7f92bd1ff700 1 -- 172.21.15.133:0/9432039 shutdown_connections 2023-07-14T18:35:33.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.296+0000 7f92bd1ff700 1 -- 172.21.15.133:0/9432039 wait complete. 2023-07-14T18:35:33.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.297+0000 7f92bd1ff700 1 Processor -- start 2023-07-14T18:35:33.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.297+0000 7f92bd1ff700 1 -- start start 2023-07-14T18:35:33.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.297+0000 7f92bd1ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b80730a0 0x7f92b8197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:33.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.297+0000 7f92bd1ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92b8074680 0x7f92b8198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:33.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.297+0000 7f92bd1ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92b8107ad0 0x7f92b819f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:33.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.297+0000 7f92bd1ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b819f990 con 0x7f92b80730a0 2023-07-14T18:35:33.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.298+0000 7f92bd1ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b819fad0 con 0x7f92b8074680 2023-07-14T18:35:33.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.298+0000 7f92bd1ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b819fc60 con 0x7f92b8107ad0 2023-07-14T18:35:33.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.298+0000 7f92b6d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b80730a0 0x7f92b8197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:33.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.298+0000 7f92b659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92b8074680 0x7f92b8198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:33.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.298+0000 7f92b6d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b80730a0 0x7f92b8197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51416/0 (socket says 172.21.15.133:51416) 2023-07-14T18:35:33.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.298+0000 7f92b6d9d700 1 -- 172.21.15.133:0/1216095734 learned_addr learned my addr 172.21.15.133:0/1216095734 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:35:33.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.298+0000 7f92b759e700 1 --2- 172.21.15.133:0/1216095734 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92b8107ad0 0x7f92b819f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:33.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.298+0000 7f92b6d9d700 1 -- 172.21.15.133:0/1216095734 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92b8107ad0 msgr2=0x7f92b819f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:33.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.298+0000 7f92b6d9d700 1 --2- 172.21.15.133:0/1216095734 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92b8107ad0 0x7f92b819f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:33.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.298+0000 7f92b6d9d700 1 -- 172.21.15.133:0/1216095734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92b8074680 msgr2=0x7f92b8198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:33.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.298+0000 7f92b6d9d700 1 --2- 172.21.15.133:0/1216095734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92b8074680 0x7f92b8198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:33.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.298+0000 7f92b6d9d700 1 -- 172.21.15.133:0/1216095734 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac0095f0 con 0x7f92b80730a0 2023-07-14T18:35:33.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.298+0000 7f92b6d9d700 1 --2- 172.21.15.133:0/1216095734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b80730a0 0x7f92b8197e10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f92a800efc0 tx=0x7f92a800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:33.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.299+0000 7f929ffff700 1 -- 172.21.15.133:0/1216095734 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a800c9c0 con 0x7f92b80730a0 2023-07-14T18:35:33.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.299+0000 7f929ffff700 1 -- 172.21.15.133:0/1216095734 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92a800f040 con 0x7f92b80730a0 2023-07-14T18:35:33.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.299+0000 7f92bd1ff700 1 -- 172.21.15.133:0/1216095734 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b819fee0 con 0x7f92b80730a0 2023-07-14T18:35:33.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.299+0000 7f929ffff700 1 -- 172.21.15.133:0/1216095734 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a8014480 con 0x7f92b80730a0 2023-07-14T18:35:33.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.299+0000 7f92bd1ff700 1 -- 172.21.15.133:0/1216095734 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b81a0310 con 0x7f92b80730a0 2023-07-14T18:35:33.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.300+0000 7f92bd1ff700 1 -- 172.21.15.133:0/1216095734 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92b810d660 con 0x7f92b80730a0 2023-07-14T18:35:33.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.302+0000 7f929ffff700 1 -- 172.21.15.133:0/1216095734 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f92a800cb20 con 0x7f92b80730a0 2023-07-14T18:35:33.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.303+0000 7f929ffff700 1 --2- 172.21.15.133:0/1216095734 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f92a00a11d0 0x7f92a00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:33.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.303+0000 7f929ffff700 1 -- 172.21.15.133:0/1216095734 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f92a80cce60 con 0x7f92b80730a0 2023-07-14T18:35:33.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.303+0000 7f92b659c700 1 --2- 172.21.15.133:0/1216095734 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f92a00a11d0 0x7f92a00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:33.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.304+0000 7f92b659c700 1 --2- 172.21.15.133:0/1216095734 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f92a00a11d0 0x7f92a00a3680 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00b2e0 tx=0x7f92ac005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:33.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.307+0000 7f929ffff700 1 -- 172.21.15.133:0/1216095734 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f92a8096220 con 0x7f92b80730a0 2023-07-14T18:35:33.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.582+0000 7f92bd1ff700 1 -- 172.21.15.133:0/1216095734 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f92b81a0b90 con 0x7f92b80730a0 2023-07-14T18:35:33.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.590+0000 7f929ffff700 1 -- 172.21.15.133:0/1216095734 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) v1 ==== 161+0+0 (secure 0 0 0) 0x7f92a8099840 con 0x7f92b80730a0 2023-07-14T18:35:33.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.593+0000 7f92bd1ff700 1 -- 172.21.15.133:0/1216095734 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f92a00a11d0 msgr2=0x7f92a00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:33.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.593+0000 7f92bd1ff700 1 --2- 172.21.15.133:0/1216095734 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f92a00a11d0 0x7f92a00a3680 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00b2e0 tx=0x7f92ac005fb0 comp rx=0 tx=0).stop 2023-07-14T18:35:33.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.593+0000 7f92bd1ff700 1 -- 172.21.15.133:0/1216095734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b80730a0 msgr2=0x7f92b8197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:33.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.593+0000 7f92bd1ff700 1 --2- 172.21.15.133:0/1216095734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b80730a0 0x7f92b8197e10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f92a800efc0 tx=0x7f92a800c3a0 comp rx=0 tx=0).stop 2023-07-14T18:35:33.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.593+0000 7f92bd1ff700 1 -- 172.21.15.133:0/1216095734 shutdown_connections 2023-07-14T18:35:33.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.593+0000 7f92bd1ff700 1 --2- 172.21.15.133:0/1216095734 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f92a00a11d0 0x7f92a00a3680 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:33.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.593+0000 7f92bd1ff700 1 --2- 172.21.15.133:0/1216095734 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92b8107ad0 0x7f92b819f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:33.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.593+0000 7f92bd1ff700 1 --2- 172.21.15.133:0/1216095734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b80730a0 0x7f92b8197e10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:33.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.593+0000 7f92bd1ff700 1 --2- 172.21.15.133:0/1216095734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92b8074680 0x7f92b8198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:33.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.593+0000 7f92bd1ff700 1 -- 172.21.15.133:0/1216095734 >> 172.21.15.133:0/1216095734 conn(0x7f92b80fb210 msgr2=0x7f92b8109e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:33.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.593+0000 7f92bd1ff700 1 -- 172.21.15.133:0/1216095734 shutdown_connections 2023-07-14T18:35:33.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.593+0000 7f92bd1ff700 1 -- 172.21.15.133:0/1216095734 wait complete. 2023-07-14T18:35:33.614 INFO:teuthology.orchestra.run.smithi133.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-14T18:35:33.631 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-14T18:35:33.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.824+0000 7f911fbff700 1 -- 172.21.15.133:0/3998670404 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9118107ad0 msgr2=0x7f9118109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:33.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.824+0000 7f911fbff700 1 --2- 172.21.15.133:0/3998670404 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9118107ad0 0x7f9118109f50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9110009f90 tx=0x7f911000b330 comp rx=0 tx=0).stop 2023-07-14T18:35:33.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.824+0000 7f911fbff700 1 -- 172.21.15.133:0/3998670404 shutdown_connections 2023-07-14T18:35:33.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.824+0000 7f911fbff700 1 --2- 172.21.15.133:0/3998670404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9118074680 0x7f9118072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:33.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.824+0000 7f911fbff700 1 --2- 172.21.15.133:0/3998670404 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9118107ad0 0x7f9118109f50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:33.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.824+0000 7f911fbff700 1 --2- 172.21.15.133:0/3998670404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91180730a0 0x7f9118073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:33.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.824+0000 7f911fbff700 1 -- 172.21.15.133:0/3998670404 >> 172.21.15.133:0/3998670404 conn(0x7f91180fb210 msgr2=0x7f91180fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:33.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.824+0000 7f911fbff700 1 -- 172.21.15.133:0/3998670404 shutdown_connections 2023-07-14T18:35:33.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.824+0000 7f911fbff700 1 -- 172.21.15.133:0/3998670404 wait complete. 2023-07-14T18:35:33.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.825+0000 7f911fbff700 1 Processor -- start 2023-07-14T18:35:33.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.826+0000 7f911fbff700 1 -- start start 2023-07-14T18:35:33.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.826+0000 7f911fbff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91180730a0 0x7f9118197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:33.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.826+0000 7f911fbff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9118074680 0x7f91181983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:33.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.826+0000 7f911fbff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9118107ad0 0x7f911819f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:33.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.826+0000 7f911fbff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911819fa30 con 0x7f91180730a0 2023-07-14T18:35:33.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.826+0000 7f911fbff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911819fb70 con 0x7f9118074680 2023-07-14T18:35:33.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.826+0000 7f911fbff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911819fd00 con 0x7f9118107ad0 2023-07-14T18:35:33.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.826+0000 7f911d99b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91180730a0 0x7f9118197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:33.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.826+0000 7f911e19c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9118107ad0 0x7f911819f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:33.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.826+0000 7f911d99b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91180730a0 0x7f9118197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51434/0 (socket says 172.21.15.133:51434) 2023-07-14T18:35:33.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.826+0000 7f911d99b700 1 -- 172.21.15.133:0/4158831465 learned_addr learned my addr 172.21.15.133:0/4158831465 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:35:33.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.826+0000 7f911d19a700 1 --2- 172.21.15.133:0/4158831465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9118074680 0x7f91181983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:33.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.827+0000 7f911e19c700 1 -- 172.21.15.133:0/4158831465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9118074680 msgr2=0x7f91181983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:33.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.827+0000 7f911e19c700 1 --2- 172.21.15.133:0/4158831465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9118074680 0x7f91181983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:33.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.827+0000 7f911e19c700 1 -- 172.21.15.133:0/4158831465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91180730a0 msgr2=0x7f9118197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:33.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.827+0000 7f911e19c700 1 --2- 172.21.15.133:0/4158831465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91180730a0 0x7f9118197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:33.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.827+0000 7f911e19c700 1 -- 172.21.15.133:0/4158831465 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9110009c40 con 0x7f9118107ad0 2023-07-14T18:35:33.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.827+0000 7f911e19c700 1 --2- 172.21.15.133:0/4158831465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9118107ad0 0x7f911819f460 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f911000b6c0 tx=0x7f9110004c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:33.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.827+0000 7f910effd700 1 -- 172.21.15.133:0/4158831465 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911000e040 con 0x7f9118107ad0 2023-07-14T18:35:33.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.827+0000 7f910effd700 1 -- 172.21.15.133:0/4158831465 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f911000bbf0 con 0x7f9118107ad0 2023-07-14T18:35:33.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.827+0000 7f911fbff700 1 -- 172.21.15.133:0/4158831465 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911819ff80 con 0x7f9118107ad0 2023-07-14T18:35:33.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.827+0000 7f910effd700 1 -- 172.21.15.133:0/4158831465 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9110005040 con 0x7f9118107ad0 2023-07-14T18:35:33.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.827+0000 7f911fbff700 1 -- 172.21.15.133:0/4158831465 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91181a03a0 con 0x7f9118107ad0 2023-07-14T18:35:33.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.829+0000 7f911fbff700 1 -- 172.21.15.133:0/4158831465 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f911810d660 con 0x7f9118107ad0 2023-07-14T18:35:33.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.830+0000 7f910effd700 1 -- 172.21.15.133:0/4158831465 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f911001d470 con 0x7f9118107ad0 2023-07-14T18:35:33.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.832+0000 7f910effd700 1 --2- 172.21.15.133:0/4158831465 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f91040a1150 0x7f91040a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:33.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.832+0000 7f910effd700 1 -- 172.21.15.133:0/4158831465 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f91100ce150 con 0x7f9118107ad0 2023-07-14T18:35:33.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.832+0000 7f911d99b700 1 --2- 172.21.15.133:0/4158831465 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f91040a1150 0x7f91040a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:33.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.832+0000 7f911d99b700 1 --2- 172.21.15.133:0/4158831465 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f91040a1150 0x7f91040a3600 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9114009d00 tx=0x7f9114009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:33.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:33.835+0000 7f910effd700 1 -- 172.21.15.133:0/4158831465 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f91100974a0 con 0x7f9118107ad0 2023-07-14T18:35:33.880 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:33 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mon-a[126360]: 2023-07-14T18:35:33.583+0000 7fc96d976700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-14T18:35:33.880 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:33 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2450253158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:35:33.880 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:33 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3423827339' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:35:33.880 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:33 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1216095734' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-14T18:35:33.881 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:33 smithi133 ceph-mon[126384]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-14T18:35:33.881 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:33 smithi133 ceph-mon[126384]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-14T18:35:33.881 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:33 smithi133 ceph-mon[126384]: osdmap e93: 12 total, 12 up, 12 in 2023-07-14T18:35:33.881 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:33 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1216095734' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-14T18:35:33.881 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:33 smithi133 ceph-mon[126384]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-07-14T18:35:34.074 INFO:journalctl@ceph.mds.b.smithi161.stdout:Jul 14 18:35:33 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-b[154371]: ignoring --setuser ceph since I am not root 2023-07-14T18:35:34.074 INFO:journalctl@ceph.mds.b.smithi161.stdout:Jul 14 18:35:33 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-b[154371]: ignoring --setgroup ceph since I am not root 2023-07-14T18:35:34.075 INFO:journalctl@ceph.mds.b.smithi161.stdout:Jul 14 18:35:33 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-b[154371]: starting mds.b at 2023-07-14T18:35:34.075 INFO:journalctl@ceph.mds.e.smithi161.stdout:Jul 14 18:35:33 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-e[153691]: ignoring --setuser ceph since I am not root 2023-07-14T18:35:34.075 INFO:journalctl@ceph.mds.e.smithi161.stdout:Jul 14 18:35:33 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-e[153691]: ignoring --setgroup ceph since I am not root 2023-07-14T18:35:34.075 INFO:journalctl@ceph.mds.e.smithi161.stdout:Jul 14 18:35:33 smithi161 ceph-mds[153715]: -- 172.21.15.161:0/245986071 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56375abff1e0 con 0x56375abd4c00 2023-07-14T18:35:34.075 INFO:journalctl@ceph.mds.e.smithi161.stdout:Jul 14 18:35:33 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-e[153691]: starting mds.e at 2023-07-14T18:35:34.076 INFO:journalctl@ceph.mds.h.smithi161.stdout:Jul 14 18:35:33 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-h[154030]: ignoring --setuser ceph since I am not root 2023-07-14T18:35:34.076 INFO:journalctl@ceph.mds.h.smithi161.stdout:Jul 14 18:35:33 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-h[154030]: ignoring --setgroup ceph since I am not root 2023-07-14T18:35:34.076 INFO:journalctl@ceph.mds.h.smithi161.stdout:Jul 14 18:35:33 smithi161 ceph-mds[154054]: -- 172.21.15.161:0/1246759153 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f2168371e0 con 0x55f21680c800 2023-07-14T18:35:34.076 INFO:journalctl@ceph.mds.h.smithi161.stdout:Jul 14 18:35:33 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-h[154030]: starting mds.h at 2023-07-14T18:35:34.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:33 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2450253158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:35:34.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:33 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3423827339' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:35:34.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:33 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1216095734' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-14T18:35:34.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:33 smithi161 ceph-mon[128993]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-14T18:35:34.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:33 smithi161 ceph-mon[128993]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-14T18:35:34.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:33 smithi161 ceph-mon[128993]: osdmap e93: 12 total, 12 up, 12 in 2023-07-14T18:35:34.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:33 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1216095734' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-14T18:35:34.078 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:33 smithi161 ceph-mon[128993]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-07-14T18:35:34.089 INFO:journalctl@ceph.mds.c.smithi171.stdout:Jul 14 18:35:33 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-c[155279]: ignoring --setuser ceph since I am not root 2023-07-14T18:35:34.090 INFO:journalctl@ceph.mds.c.smithi171.stdout:Jul 14 18:35:33 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-c[155279]: ignoring --setgroup ceph since I am not root 2023-07-14T18:35:34.090 INFO:journalctl@ceph.mds.c.smithi171.stdout:Jul 14 18:35:33 smithi171 ceph-mds[155303]: -- 172.21.15.171:0/3622184517 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558f0c3c71e0 con 0x558f0c39cc00 2023-07-14T18:35:34.090 INFO:journalctl@ceph.mds.c.smithi171.stdout:Jul 14 18:35:33 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-c[155279]: starting mds.c at 2023-07-14T18:35:34.091 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:33 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2450253158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T18:35:34.091 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:33 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3423827339' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T18:35:34.091 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:33 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1216095734' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-14T18:35:34.092 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:33 smithi171 ceph-mon[129241]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-14T18:35:34.092 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:33 smithi171 ceph-mon[129241]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-14T18:35:34.092 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:33 smithi171 ceph-mon[129241]: osdmap e93: 12 total, 12 up, 12 in 2023-07-14T18:35:34.092 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:33 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1216095734' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-14T18:35:34.093 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:33 smithi171 ceph-mon[129241]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-07-14T18:35:34.093 INFO:journalctl@ceph.mds.f.smithi171.stdout:Jul 14 18:35:33 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-f[154932]: ignoring --setuser ceph since I am not root 2023-07-14T18:35:34.093 INFO:journalctl@ceph.mds.f.smithi171.stdout:Jul 14 18:35:33 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-f[154932]: ignoring --setgroup ceph since I am not root 2023-07-14T18:35:34.094 INFO:journalctl@ceph.mds.f.smithi171.stdout:Jul 14 18:35:33 smithi171 ceph-mds[154956]: -- 172.21.15.171:0/1625582462 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f2cafef1e0 con 0x55f2cafc4c00 2023-07-14T18:35:34.094 INFO:journalctl@ceph.mds.f.smithi171.stdout:Jul 14 18:35:33 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-f[154932]: starting mds.f at 2023-07-14T18:35:34.094 INFO:journalctl@ceph.mds.i.smithi171.stdout:Jul 14 18:35:33 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-i[155617]: ignoring --setuser ceph since I am not root 2023-07-14T18:35:34.095 INFO:journalctl@ceph.mds.i.smithi171.stdout:Jul 14 18:35:33 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-i[155617]: ignoring --setgroup ceph since I am not root 2023-07-14T18:35:34.095 INFO:journalctl@ceph.mds.i.smithi171.stdout:Jul 14 18:35:33 smithi171 ceph-mds[155641]: -- 172.21.15.171:0/732304245 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b5bc7ed1e0 con 0x55b5bc7c2c00 2023-07-14T18:35:34.095 INFO:journalctl@ceph.mds.i.smithi171.stdout:Jul 14 18:35:33 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-i[155617]: starting mds.i at 2023-07-14T18:35:34.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.092+0000 7f911fbff700 1 -- 172.21.15.133:0/4158831465 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f911810d7f0 con 0x7f9118107ad0 2023-07-14T18:35:34.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.630+0000 7f910effd700 1 -- 172.21.15.133:0/4158831465 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) v1 ==== 104+0+0 (secure 0 0 0) 0x7f911009aac0 con 0x7f9118107ad0 2023-07-14T18:35:34.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.633+0000 7f911fbff700 1 -- 172.21.15.133:0/4158831465 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f91040a1150 msgr2=0x7f91040a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:34.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.633+0000 7f911fbff700 1 --2- 172.21.15.133:0/4158831465 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f91040a1150 0x7f91040a3600 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9114009d00 tx=0x7f9114009310 comp rx=0 tx=0).stop 2023-07-14T18:35:34.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.633+0000 7f911fbff700 1 -- 172.21.15.133:0/4158831465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9118107ad0 msgr2=0x7f911819f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:34.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.633+0000 7f911fbff700 1 --2- 172.21.15.133:0/4158831465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9118107ad0 0x7f911819f460 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f911000b6c0 tx=0x7f9110004c60 comp rx=0 tx=0).stop 2023-07-14T18:35:34.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.633+0000 7f911fbff700 1 -- 172.21.15.133:0/4158831465 shutdown_connections 2023-07-14T18:35:34.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.633+0000 7f911fbff700 1 --2- 172.21.15.133:0/4158831465 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f91040a1150 0x7f91040a3600 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:34.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.633+0000 7f911fbff700 1 --2- 172.21.15.133:0/4158831465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9118107ad0 0x7f911819f460 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:34.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.633+0000 7f911fbff700 1 --2- 172.21.15.133:0/4158831465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91180730a0 0x7f9118197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:34.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.633+0000 7f911fbff700 1 --2- 172.21.15.133:0/4158831465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9118074680 0x7f91181983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:34.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.633+0000 7f911fbff700 1 -- 172.21.15.133:0/4158831465 >> 172.21.15.133:0/4158831465 conn(0x7f91180fb210 msgr2=0x7f9118109cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:34.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.633+0000 7f911fbff700 1 -- 172.21.15.133:0/4158831465 shutdown_connections 2023-07-14T18:35:34.653 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.633+0000 7f911fbff700 1 -- 172.21.15.133:0/4158831465 wait complete. 2023-07-14T18:35:34.672 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-14T18:35:34.880 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:34 smithi133 ceph-mon[126384]: pgmap v914: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 7 op/s 2023-07-14T18:35:34.881 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:34 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/4158831465' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T18:35:34.881 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:34 smithi133 ceph-mon[126384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T18:35:34.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.864+0000 7f67cedff700 1 -- 172.21.15.133:0/4070403930 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67c8100e10 msgr2=0x7f67c8101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:34.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.864+0000 7f67cedff700 1 --2- 172.21.15.133:0/4070403930 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67c8100e10 0x7f67c8101260 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f67b4009990 tx=0x7f67b4009ca0 comp rx=0 tx=0).stop 2023-07-14T18:35:34.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.864+0000 7f67cedff700 1 -- 172.21.15.133:0/4070403930 shutdown_connections 2023-07-14T18:35:34.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.864+0000 7f67cedff700 1 --2- 172.21.15.133:0/4070403930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67c80ffc20 0x7f67c8100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:34.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.864+0000 7f67cedff700 1 --2- 172.21.15.133:0/4070403930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c81017a0 0x7f67c8108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:34.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.864+0000 7f67cedff700 1 --2- 172.21.15.133:0/4070403930 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67c8100e10 0x7f67c8101260 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:34.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.864+0000 7f67cedff700 1 -- 172.21.15.133:0/4070403930 >> 172.21.15.133:0/4070403930 conn(0x7f67c80fb260 msgr2=0x7f67c80fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:34.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.864+0000 7f67cedff700 1 -- 172.21.15.133:0/4070403930 shutdown_connections 2023-07-14T18:35:34.885 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.864+0000 7f67cedff700 1 -- 172.21.15.133:0/4070403930 wait complete. 2023-07-14T18:35:34.886 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.865+0000 7f67cedff700 1 Processor -- start 2023-07-14T18:35:34.886 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.865+0000 7f67cedff700 1 -- start start 2023-07-14T18:35:34.886 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.866+0000 7f67cedff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67c80ffc20 0x7f67c80743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:34.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.866+0000 7f67cedff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c8100e10 0x7f67c8072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:34.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.866+0000 7f67cedff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67c81017a0 0x7f67c8072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:34.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.866+0000 7f67cedff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c8073710 con 0x7f67c8100e10 2023-07-14T18:35:34.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.866+0000 7f67cedff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c8073880 con 0x7f67c80ffc20 2023-07-14T18:35:34.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.866+0000 7f67cedff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c819f9b0 con 0x7f67c81017a0 2023-07-14T18:35:34.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.866+0000 7f67c7fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c8100e10 0x7f67c8072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:34.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.866+0000 7f67cd39c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67c81017a0 0x7f67c8072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:34.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.866+0000 7f67c7fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c8100e10 0x7f67c8072a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51456/0 (socket says 172.21.15.133:51456) 2023-07-14T18:35:34.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.866+0000 7f67ccb9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67c80ffc20 0x7f67c80743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:34.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.866+0000 7f67c7fff700 1 -- 172.21.15.133:0/2313477728 learned_addr learned my addr 172.21.15.133:0/2313477728 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:35:34.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.866+0000 7f67ccb9b700 1 -- 172.21.15.133:0/2313477728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67c81017a0 msgr2=0x7f67c8072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:34.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.866+0000 7f67ccb9b700 1 --2- 172.21.15.133:0/2313477728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67c81017a0 0x7f67c8072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:34.889 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.866+0000 7f67ccb9b700 1 -- 172.21.15.133:0/2313477728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c8100e10 msgr2=0x7f67c8072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:34.889 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.866+0000 7f67ccb9b700 1 --2- 172.21.15.133:0/2313477728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c8100e10 0x7f67c8072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:34.889 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.866+0000 7f67ccb9b700 1 -- 172.21.15.133:0/2313477728 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b40095f0 con 0x7f67c80ffc20 2023-07-14T18:35:34.889 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.867+0000 7f67ccb9b700 1 --2- 172.21.15.133:0/2313477728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67c80ffc20 0x7f67c80743f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f67bc00e9c0 tx=0x7f67bc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:34.889 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.867+0000 7f67c5ffb700 1 -- 172.21.15.133:0/2313477728 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc004220 con 0x7f67c80ffc20 2023-07-14T18:35:34.889 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.867+0000 7f67c5ffb700 1 -- 172.21.15.133:0/2313477728 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67bc004380 con 0x7f67c80ffc20 2023-07-14T18:35:34.889 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.867+0000 7f67cedff700 1 -- 172.21.15.133:0/2313477728 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c819fc90 con 0x7f67c80ffc20 2023-07-14T18:35:34.889 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.867+0000 7f67c5ffb700 1 -- 172.21.15.133:0/2313477728 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc003be0 con 0x7f67c80ffc20 2023-07-14T18:35:34.890 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.867+0000 7f67cedff700 1 -- 172.21.15.133:0/2313477728 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c81a01a0 con 0x7f67c80ffc20 2023-07-14T18:35:34.890 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.868+0000 7f67cedff700 1 -- 172.21.15.133:0/2313477728 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c810b3e0 con 0x7f67c80ffc20 2023-07-14T18:35:34.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.870+0000 7f67c5ffb700 1 -- 172.21.15.133:0/2313477728 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f67bc003d40 con 0x7f67c80ffc20 2023-07-14T18:35:34.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.871+0000 7f67c5ffb700 1 --2- 172.21.15.133:0/2313477728 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f67b80a1150 0x7f67b80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:34.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.871+0000 7f67c5ffb700 1 -- 172.21.15.133:0/2313477728 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f67bc014070 con 0x7f67c80ffc20 2023-07-14T18:35:34.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.871+0000 7f67c7fff700 1 --2- 172.21.15.133:0/2313477728 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f67b80a1150 0x7f67b80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:34.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.872+0000 7f67c7fff700 1 --2- 172.21.15.133:0/2313477728 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f67b80a1150 0x7f67b80a3600 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f67b400b2e0 tx=0x7f67b4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:34.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:34.873+0000 7f67c5ffb700 1 -- 172.21.15.133:0/2313477728 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f67bc0968d0 con 0x7f67c80ffc20 2023-07-14T18:35:35.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:34 smithi161 ceph-mon[128993]: pgmap v914: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 7 op/s 2023-07-14T18:35:35.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:34 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/4158831465' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T18:35:35.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:34 smithi161 ceph-mon[128993]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T18:35:35.089 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:34 smithi171 ceph-mon[129241]: pgmap v914: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 7 op/s 2023-07-14T18:35:35.089 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:34 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/4158831465' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T18:35:35.090 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:34 smithi171 ceph-mon[129241]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T18:35:35.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.076+0000 7f67cedff700 1 -- 172.21.15.133:0/2313477728 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f67c8106b00 con 0x7f67c80ffc20 2023-07-14T18:35:35.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.638+0000 7f67c5ffb700 1 -- 172.21.15.133:0/2313477728 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v94) v1 ==== 181+0+0 (secure 0 0 0) 0x7f67bc01f340 con 0x7f67c80ffc20 2023-07-14T18:35:35.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.642+0000 7f67cedff700 1 -- 172.21.15.133:0/2313477728 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f67b80a1150 msgr2=0x7f67b80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:35.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.642+0000 7f67cedff700 1 --2- 172.21.15.133:0/2313477728 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f67b80a1150 0x7f67b80a3600 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f67b400b2e0 tx=0x7f67b4005fb0 comp rx=0 tx=0).stop 2023-07-14T18:35:35.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.642+0000 7f67cedff700 1 -- 172.21.15.133:0/2313477728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67c80ffc20 msgr2=0x7f67c80743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:35.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.642+0000 7f67cedff700 1 --2- 172.21.15.133:0/2313477728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67c80ffc20 0x7f67c80743f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f67bc00e9c0 tx=0x7f67bc00ecd0 comp rx=0 tx=0).stop 2023-07-14T18:35:35.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.643+0000 7f67cedff700 1 -- 172.21.15.133:0/2313477728 shutdown_connections 2023-07-14T18:35:35.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.643+0000 7f67cedff700 1 --2- 172.21.15.133:0/2313477728 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f67b80a1150 0x7f67b80a3600 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:35.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.643+0000 7f67cedff700 1 --2- 172.21.15.133:0/2313477728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67c80ffc20 0x7f67c80743f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:35.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.643+0000 7f67cedff700 1 --2- 172.21.15.133:0/2313477728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67c81017a0 0x7f67c8072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:35.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.643+0000 7f67cedff700 1 --2- 172.21.15.133:0/2313477728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c8100e10 0x7f67c8072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:35.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.643+0000 7f67cedff700 1 -- 172.21.15.133:0/2313477728 >> 172.21.15.133:0/2313477728 conn(0x7f67c80fb260 msgr2=0x7f67c80fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:35.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.643+0000 7f67cedff700 1 -- 172.21.15.133:0/2313477728 shutdown_connections 2023-07-14T18:35:35.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.643+0000 7f67cedff700 1 -- 172.21.15.133:0/2313477728 wait complete. 2023-07-14T18:35:35.666 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' removed 2023-07-14T18:35:35.691 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-14T18:35:35.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.894+0000 7f84a99ff700 1 -- 172.21.15.133:0/3311368711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84a40ffc20 msgr2=0x7f84a4100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:35.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.894+0000 7f84a99ff700 1 --2- 172.21.15.133:0/3311368711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84a40ffc20 0x7f84a4100030 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8494009990 tx=0x7f8494009ca0 comp rx=0 tx=0).stop 2023-07-14T18:35:35.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.894+0000 7f84a99ff700 1 -- 172.21.15.133:0/3311368711 shutdown_connections 2023-07-14T18:35:35.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.894+0000 7f84a99ff700 1 --2- 172.21.15.133:0/3311368711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84a41017a0 0x7f84a4108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:35.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.894+0000 7f84a99ff700 1 --2- 172.21.15.133:0/3311368711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84a40ffc20 0x7f84a4100030 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:35.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.894+0000 7f84a99ff700 1 --2- 172.21.15.133:0/3311368711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84a4100e10 0x7f84a4101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:35.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.894+0000 7f84a99ff700 1 -- 172.21.15.133:0/3311368711 >> 172.21.15.133:0/3311368711 conn(0x7f84a40fb260 msgr2=0x7f84a40fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:35.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.894+0000 7f84a99ff700 1 -- 172.21.15.133:0/3311368711 shutdown_connections 2023-07-14T18:35:35.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.894+0000 7f84a99ff700 1 -- 172.21.15.133:0/3311368711 wait complete. 2023-07-14T18:35:35.915 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.895+0000 7f84a99ff700 1 Processor -- start 2023-07-14T18:35:35.915 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.895+0000 7f84a99ff700 1 -- start start 2023-07-14T18:35:35.915 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.895+0000 7f84a99ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84a40ffc20 0x7f84a4197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:35.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.895+0000 7f84a99ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84a4100e10 0x7f84a4198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:35.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.895+0000 7f84a99ff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84a41017a0 0x7f84a419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:35.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.895+0000 7f84a99ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a419f6d0 con 0x7f84a4100e10 2023-07-14T18:35:35.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.895+0000 7f84a99ff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a419f810 con 0x7f84a41017a0 2023-07-14T18:35:35.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.895+0000 7f84a99ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a419f9a0 con 0x7f84a40ffc20 2023-07-14T18:35:35.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.896+0000 7f84a2ffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84a40ffc20 0x7f84a4197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:35.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.896+0000 7f84a2ffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84a40ffc20 0x7f84a4197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.133:56448/0 (socket says 172.21.15.133:56448) 2023-07-14T18:35:35.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.896+0000 7f84a2ffd700 1 -- 172.21.15.133:0/1009421091 learned_addr learned my addr 172.21.15.133:0/1009421091 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:35:35.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.896+0000 7f84a27fc700 1 --2- 172.21.15.133:0/1009421091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84a4100e10 0x7f84a4198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:35.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.896+0000 7f84a37fe700 1 --2- 172.21.15.133:0/1009421091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84a41017a0 0x7f84a419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:35.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.896+0000 7f84a2ffd700 1 -- 172.21.15.133:0/1009421091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84a41017a0 msgr2=0x7f84a419f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:35.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.896+0000 7f84a2ffd700 1 --2- 172.21.15.133:0/1009421091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84a41017a0 0x7f84a419f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:35.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.896+0000 7f84a2ffd700 1 -- 172.21.15.133:0/1009421091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84a4100e10 msgr2=0x7f84a4198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:35.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.896+0000 7f84a2ffd700 1 --2- 172.21.15.133:0/1009421091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84a4100e10 0x7f84a4198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:35.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.896+0000 7f84a2ffd700 1 -- 172.21.15.133:0/1009421091 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84940095f0 con 0x7f84a40ffc20 2023-07-14T18:35:35.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.896+0000 7f84a27fc700 1 --2- 172.21.15.133:0/1009421091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84a4100e10 0x7f84a4198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:35:35.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.896+0000 7f84a37fe700 1 --2- 172.21.15.133:0/1009421091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84a41017a0 0x7f84a419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T18:35:35.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.896+0000 7f84a2ffd700 1 --2- 172.21.15.133:0/1009421091 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84a40ffc20 0x7f84a4197b50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8494004710 tx=0x7f8494009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:35.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.897+0000 7f84a89fd700 1 -- 172.21.15.133:0/1009421091 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849401d070 con 0x7f84a40ffc20 2023-07-14T18:35:35.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.897+0000 7f84a89fd700 1 -- 172.21.15.133:0/1009421091 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8494021410 con 0x7f84a40ffc20 2023-07-14T18:35:35.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.897+0000 7f84a99ff700 1 -- 172.21.15.133:0/1009421091 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a419fc20 con 0x7f84a40ffc20 2023-07-14T18:35:35.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.897+0000 7f84a89fd700 1 -- 172.21.15.133:0/1009421091 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8494017650 con 0x7f84a40ffc20 2023-07-14T18:35:35.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.897+0000 7f84a99ff700 1 -- 172.21.15.133:0/1009421091 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a41a00a0 con 0x7f84a40ffc20 2023-07-14T18:35:35.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.898+0000 7f84a99ff700 1 -- 172.21.15.133:0/1009421091 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a41057e0 con 0x7f84a40ffc20 2023-07-14T18:35:35.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.900+0000 7f84a89fd700 1 -- 172.21.15.133:0/1009421091 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f8494021580 con 0x7f84a40ffc20 2023-07-14T18:35:35.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.901+0000 7f84a89fd700 1 --2- 172.21.15.133:0/1009421091 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f848c0a1150 0x7f848c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:35.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.901+0000 7f84a89fd700 1 -- 172.21.15.133:0/1009421091 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8753+0+0 (secure 0 0 0) 0x7f84940cdf10 con 0x7f84a40ffc20 2023-07-14T18:35:35.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.901+0000 7f84a27fc700 1 --2- 172.21.15.133:0/1009421091 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f848c0a1150 0x7f848c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:35.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.902+0000 7f84a27fc700 1 --2- 172.21.15.133:0/1009421091 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f848c0a1150 0x7f848c0a3600 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8498011490 tx=0x7f8498009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:35.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:35.905+0000 7f84a89fd700 1 -- 172.21.15.133:0/1009421091 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f8494097480 con 0x7f84a40ffc20 2023-07-14T18:35:36.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:35 smithi161 ceph-mon[128993]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-14T18:35:36.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:35 smithi161 ceph-mon[128993]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-14T18:35:36.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:35 smithi161 ceph-mon[128993]: Cluster is now healthy 2023-07-14T18:35:36.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:35 smithi161 ceph-mon[128993]: mds.? [v2:172.21.15.171:6837/1548764327,v1:172.21.15.171:6839/1548764327] up:boot 2023-07-14T18:35:36.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:35 smithi161 ceph-mon[128993]: mds.? [v2:172.21.15.171:6834/3478978329,v1:172.21.15.171:6835/3478978329] up:boot 2023-07-14T18:35:36.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:35 smithi161 ceph-mon[128993]: mds.? [v2:172.21.15.171:6836/1612394820,v1:172.21.15.171:6838/1612394820] up:boot 2023-07-14T18:35:36.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:35 smithi161 ceph-mon[128993]: mds.? [v2:172.21.15.161:6838/2079923375,v1:172.21.15.161:6839/2079923375] up:boot 2023-07-14T18:35:36.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:35 smithi161 ceph-mon[128993]: mds.? [v2:172.21.15.161:6834/1407500965,v1:172.21.15.161:6835/1407500965] up:boot 2023-07-14T18:35:36.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:35 smithi161 ceph-mon[128993]: mds.? [v2:172.21.15.161:6836/1102468649,v1:172.21.15.161:6837/1102468649] up:boot 2023-07-14T18:35:36.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:35 smithi161 ceph-mon[128993]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-14T18:35:36.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:35 smithi161 ceph-mon[128993]: fsmap 12 up:standby 2023-07-14T18:35:36.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:35 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T18:35:36.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:35 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T18:35:36.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:35 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T18:35:36.076 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:35 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T18:35:36.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:35 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T18:35:36.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:35 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T18:35:36.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:35 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/2313477728' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T18:35:36.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:35 smithi161 ceph-mon[128993]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T18:35:36.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:35 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:35:36.089 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:35 smithi171 ceph-mon[129241]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-14T18:35:36.089 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:35 smithi171 ceph-mon[129241]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-14T18:35:36.090 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:35 smithi171 ceph-mon[129241]: Cluster is now healthy 2023-07-14T18:35:36.090 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:35 smithi171 ceph-mon[129241]: mds.? [v2:172.21.15.171:6837/1548764327,v1:172.21.15.171:6839/1548764327] up:boot 2023-07-14T18:35:36.090 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:35 smithi171 ceph-mon[129241]: mds.? [v2:172.21.15.171:6834/3478978329,v1:172.21.15.171:6835/3478978329] up:boot 2023-07-14T18:35:36.090 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:35 smithi171 ceph-mon[129241]: mds.? [v2:172.21.15.171:6836/1612394820,v1:172.21.15.171:6838/1612394820] up:boot 2023-07-14T18:35:36.090 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:35 smithi171 ceph-mon[129241]: mds.? [v2:172.21.15.161:6838/2079923375,v1:172.21.15.161:6839/2079923375] up:boot 2023-07-14T18:35:36.090 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:35 smithi171 ceph-mon[129241]: mds.? [v2:172.21.15.161:6834/1407500965,v1:172.21.15.161:6835/1407500965] up:boot 2023-07-14T18:35:36.090 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:35 smithi171 ceph-mon[129241]: mds.? [v2:172.21.15.161:6836/1102468649,v1:172.21.15.161:6837/1102468649] up:boot 2023-07-14T18:35:36.091 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:35 smithi171 ceph-mon[129241]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-14T18:35:36.091 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:35 smithi171 ceph-mon[129241]: fsmap 12 up:standby 2023-07-14T18:35:36.091 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:35 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T18:35:36.091 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:35 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T18:35:36.091 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:35 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T18:35:36.091 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:35 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T18:35:36.091 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:35 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T18:35:36.091 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:35 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T18:35:36.092 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:35 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/2313477728' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T18:35:36.092 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:35 smithi171 ceph-mon[129241]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T18:35:36.092 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:35 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:35:36.130 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:35 smithi133 ceph-mon[126384]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-14T18:35:36.130 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:35 smithi133 ceph-mon[126384]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-14T18:35:36.131 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:35 smithi133 ceph-mon[126384]: Cluster is now healthy 2023-07-14T18:35:36.131 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:35 smithi133 ceph-mon[126384]: mds.? [v2:172.21.15.171:6837/1548764327,v1:172.21.15.171:6839/1548764327] up:boot 2023-07-14T18:35:36.131 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:35 smithi133 ceph-mon[126384]: mds.? [v2:172.21.15.171:6834/3478978329,v1:172.21.15.171:6835/3478978329] up:boot 2023-07-14T18:35:36.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:35 smithi133 ceph-mon[126384]: mds.? [v2:172.21.15.171:6836/1612394820,v1:172.21.15.171:6838/1612394820] up:boot 2023-07-14T18:35:36.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:35 smithi133 ceph-mon[126384]: mds.? [v2:172.21.15.161:6838/2079923375,v1:172.21.15.161:6839/2079923375] up:boot 2023-07-14T18:35:36.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:35 smithi133 ceph-mon[126384]: mds.? [v2:172.21.15.161:6834/1407500965,v1:172.21.15.161:6835/1407500965] up:boot 2023-07-14T18:35:36.133 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:35 smithi133 ceph-mon[126384]: mds.? [v2:172.21.15.161:6836/1102468649,v1:172.21.15.161:6837/1102468649] up:boot 2023-07-14T18:35:36.133 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:35 smithi133 ceph-mon[126384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-14T18:35:36.133 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:35 smithi133 ceph-mon[126384]: fsmap 12 up:standby 2023-07-14T18:35:36.135 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:35 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T18:35:36.135 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:35 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T18:35:36.135 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:35 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T18:35:36.135 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:35 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T18:35:36.135 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:35 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T18:35:36.136 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:35 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T18:35:36.136 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:35 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/2313477728' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T18:35:36.136 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:35 smithi133 ceph-mon[126384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T18:35:36.136 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:35 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T18:35:36.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.145+0000 7f84a99ff700 1 -- 172.21.15.133:0/1009421091 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f84a402c8e0 con 0x7f84a40ffc20 2023-07-14T18:35:36.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.652+0000 7f84a89fd700 1 -- 172.21.15.133:0/1009421091 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v95) v1 ==== 169+0+0 (secure 0 0 0) 0x7f849409aaa0 con 0x7f84a40ffc20 2023-07-14T18:35:36.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.657+0000 7f84a99ff700 1 -- 172.21.15.133:0/1009421091 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f848c0a1150 msgr2=0x7f848c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:36.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.657+0000 7f84a99ff700 1 --2- 172.21.15.133:0/1009421091 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f848c0a1150 0x7f848c0a3600 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8498011490 tx=0x7f8498009260 comp rx=0 tx=0).stop 2023-07-14T18:35:36.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.657+0000 7f84a99ff700 1 -- 172.21.15.133:0/1009421091 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84a40ffc20 msgr2=0x7f84a4197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:36.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.657+0000 7f84a99ff700 1 --2- 172.21.15.133:0/1009421091 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84a40ffc20 0x7f84a4197b50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8494004710 tx=0x7f8494009580 comp rx=0 tx=0).stop 2023-07-14T18:35:36.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.658+0000 7f84a99ff700 1 -- 172.21.15.133:0/1009421091 shutdown_connections 2023-07-14T18:35:36.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.658+0000 7f84a99ff700 1 --2- 172.21.15.133:0/1009421091 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f848c0a1150 0x7f848c0a3600 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:36.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.658+0000 7f84a99ff700 1 --2- 172.21.15.133:0/1009421091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84a41017a0 0x7f84a419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:36.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.658+0000 7f84a99ff700 1 --2- 172.21.15.133:0/1009421091 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84a40ffc20 0x7f84a4197b50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:36.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.658+0000 7f84a99ff700 1 --2- 172.21.15.133:0/1009421091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84a4100e10 0x7f84a4198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:36.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.658+0000 7f84a99ff700 1 -- 172.21.15.133:0/1009421091 >> 172.21.15.133:0/1009421091 conn(0x7f84a40fb260 msgr2=0x7f84a40fc070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:36.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.658+0000 7f84a99ff700 1 -- 172.21.15.133:0/1009421091 shutdown_connections 2023-07-14T18:35:36.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.659+0000 7f84a99ff700 1 -- 172.21.15.133:0/1009421091 wait complete. 2023-07-14T18:35:36.679 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' removed 2023-07-14T18:35:36.700 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-07-14T18:35:36.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.886+0000 7f0522fff700 1 -- 172.21.15.133:0/3713230885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f051c1017a0 msgr2=0x7f051c108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:36.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.886+0000 7f0522fff700 1 --2- 172.21.15.133:0/3713230885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f051c1017a0 0x7f051c108060 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f051400b0a0 tx=0x7f051400b3b0 comp rx=0 tx=0).stop 2023-07-14T18:35:36.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.886+0000 7f0522fff700 1 -- 172.21.15.133:0/3713230885 shutdown_connections 2023-07-14T18:35:36.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.886+0000 7f0522fff700 1 --2- 172.21.15.133:0/3713230885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f051c0ffc20 0x7f051c100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:36.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.886+0000 7f0522fff700 1 --2- 172.21.15.133:0/3713230885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f051c1017a0 0x7f051c108060 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:36.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.886+0000 7f0522fff700 1 --2- 172.21.15.133:0/3713230885 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f051c100e10 0x7f051c101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:36.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.886+0000 7f0522fff700 1 -- 172.21.15.133:0/3713230885 >> 172.21.15.133:0/3713230885 conn(0x7f051c0fb260 msgr2=0x7f051c0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:36.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.886+0000 7f0522fff700 1 -- 172.21.15.133:0/3713230885 shutdown_connections 2023-07-14T18:35:36.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.886+0000 7f0522fff700 1 -- 172.21.15.133:0/3713230885 wait complete. 2023-07-14T18:35:36.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.887+0000 7f0522fff700 1 Processor -- start 2023-07-14T18:35:36.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.887+0000 7f0522fff700 1 -- start start 2023-07-14T18:35:36.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.887+0000 7f0522fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f051c0ffc20 0x7f051c197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:36.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.887+0000 7f0522fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f051c100e10 0x7f051c198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:36.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.887+0000 7f0522fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f051c1017a0 0x7f051c19f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:36.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.887+0000 7f0522fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051c19f830 con 0x7f051c0ffc20 2023-07-14T18:35:36.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.887+0000 7f0522fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051c19f9a0 con 0x7f051c100e10 2023-07-14T18:35:36.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.887+0000 7f0522fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051c19fb60 con 0x7f051c1017a0 2023-07-14T18:35:36.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.888+0000 7f0520d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f051c0ffc20 0x7f051c197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:36.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.888+0000 7f0520d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f051c0ffc20 0x7f051c197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46172/0 (socket says 172.21.15.133:46172) 2023-07-14T18:35:36.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.888+0000 7f052159c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f051c1017a0 0x7f051c19f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:36.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.888+0000 7f0520d9b700 1 -- 172.21.15.133:0/3185761218 learned_addr learned my addr 172.21.15.133:0/3185761218 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T18:35:36.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.888+0000 7f051bfff700 1 --2- 172.21.15.133:0/3185761218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f051c100e10 0x7f051c198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:36.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.888+0000 7f052159c700 1 -- 172.21.15.133:0/3185761218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f051c100e10 msgr2=0x7f051c198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:36.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.888+0000 7f052159c700 1 --2- 172.21.15.133:0/3185761218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f051c100e10 0x7f051c198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:36.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.888+0000 7f052159c700 1 -- 172.21.15.133:0/3185761218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f051c0ffc20 msgr2=0x7f051c197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:36.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.888+0000 7f052159c700 1 --2- 172.21.15.133:0/3185761218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f051c0ffc20 0x7f051c197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:36.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.888+0000 7f052159c700 1 -- 172.21.15.133:0/3185761218 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0514009d00 con 0x7f051c1017a0 2023-07-14T18:35:36.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.888+0000 7f052159c700 1 --2- 172.21.15.133:0/3185761218 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f051c1017a0 0x7f051c19f260 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f051400b070 tx=0x7f051400bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:36.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.888+0000 7f0519ffb700 1 -- 172.21.15.133:0/3185761218 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051400e040 con 0x7f051c1017a0 2023-07-14T18:35:36.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.888+0000 7f0519ffb700 1 -- 172.21.15.133:0/3185761218 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0514009480 con 0x7f051c1017a0 2023-07-14T18:35:36.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.889+0000 7f0522fff700 1 -- 172.21.15.133:0/3185761218 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f051c19fde0 con 0x7f051c1017a0 2023-07-14T18:35:36.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.889+0000 7f0519ffb700 1 -- 172.21.15.133:0/3185761218 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05140052a0 con 0x7f051c1017a0 2023-07-14T18:35:36.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.889+0000 7f0522fff700 1 -- 172.21.15.133:0/3185761218 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f051c1a01b0 con 0x7f051c1017a0 2023-07-14T18:35:36.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.890+0000 7f0522fff700 1 -- 172.21.15.133:0/3185761218 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f051c10b450 con 0x7f051c1017a0 2023-07-14T18:35:36.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.891+0000 7f0519ffb700 1 -- 172.21.15.133:0/3185761218 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f051401d480 con 0x7f051c1017a0 2023-07-14T18:35:36.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.893+0000 7f0519ffb700 1 --2- 172.21.15.133:0/3185761218 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f050c0a11d0 0x7f050c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T18:35:36.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.893+0000 7f0519ffb700 1 -- 172.21.15.133:0/3185761218 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8252+0+0 (secure 0 0 0) 0x7f05140cd8e0 con 0x7f051c1017a0 2023-07-14T18:35:36.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.893+0000 7f0520d9b700 1 --2- 172.21.15.133:0/3185761218 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f050c0a11d0 0x7f050c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T18:35:36.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.893+0000 7f0520d9b700 1 --2- 172.21.15.133:0/3185761218 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f050c0a11d0 0x7f050c0a3680 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f051c1991c0 tx=0x7f0510009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T18:35:36.914 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:36.896+0000 7f0519ffb700 1 -- 172.21.15.133:0/3185761218 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f0514097050 con 0x7f051c1017a0 2023-07-14T18:35:37.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:36 smithi161 ceph-mon[128993]: pgmap v916: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 8 op/s 2023-07-14T18:35:37.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:36 smithi161 ceph-mon[128993]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-14T18:35:37.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:36 smithi161 ceph-mon[128993]: osdmap e94: 12 total, 12 up, 12 in 2023-07-14T18:35:37.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:36 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/1009421091' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T18:35:37.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:36 smithi161 ceph-mon[128993]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T18:35:37.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:35:37.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:35:37.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:36 smithi161 ceph-mon[128993]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:35:37.089 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:36 smithi171 ceph-mon[129241]: pgmap v916: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 8 op/s 2023-07-14T18:35:37.089 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:36 smithi171 ceph-mon[129241]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-14T18:35:37.090 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:36 smithi171 ceph-mon[129241]: osdmap e94: 12 total, 12 up, 12 in 2023-07-14T18:35:37.090 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:36 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/1009421091' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T18:35:37.090 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:36 smithi171 ceph-mon[129241]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T18:35:37.090 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:36 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:35:37.090 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:36 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:35:37.090 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:36 smithi171 ceph-mon[129241]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:35:37.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:37.096+0000 7f0522fff700 1 -- 172.21.15.133:0/3185761218 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f051c10b610 con 0x7f051c1017a0 2023-07-14T18:35:37.130 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:36 smithi133 ceph-mon[126384]: pgmap v916: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 8 op/s 2023-07-14T18:35:37.130 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:36 smithi133 ceph-mon[126384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-14T18:35:37.131 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:36 smithi133 ceph-mon[126384]: osdmap e94: 12 total, 12 up, 12 in 2023-07-14T18:35:37.131 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:36 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/1009421091' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T18:35:37.131 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:36 smithi133 ceph-mon[126384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T18:35:37.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:36 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T18:35:37.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:36 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T18:35:37.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:36 smithi133 ceph-mon[126384]: from='mgr.14152 172.21.15.133:0/1472610612' entity='mgr.x' 2023-07-14T18:35:37.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:37.662+0000 7f0519ffb700 1 -- 172.21.15.133:0/3185761218 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v96) v1 ==== 178+0+0 (secure 0 0 0) 0x7f051409a670 con 0x7f051c1017a0 2023-07-14T18:35:37.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:37.666+0000 7f0522fff700 1 -- 172.21.15.133:0/3185761218 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f050c0a11d0 msgr2=0x7f050c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:37.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:37.666+0000 7f0522fff700 1 --2- 172.21.15.133:0/3185761218 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f050c0a11d0 0x7f050c0a3680 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f051c1991c0 tx=0x7f0510009260 comp rx=0 tx=0).stop 2023-07-14T18:35:37.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:37.666+0000 7f0522fff700 1 -- 172.21.15.133:0/3185761218 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f051c1017a0 msgr2=0x7f051c19f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T18:35:37.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:37.666+0000 7f0522fff700 1 --2- 172.21.15.133:0/3185761218 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f051c1017a0 0x7f051c19f260 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f051400b070 tx=0x7f051400bc80 comp rx=0 tx=0).stop 2023-07-14T18:35:37.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:37.667+0000 7f0522fff700 1 -- 172.21.15.133:0/3185761218 shutdown_connections 2023-07-14T18:35:37.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:37.667+0000 7f0522fff700 1 --2- 172.21.15.133:0/3185761218 >> [v2:172.21.15.133:6800/196197529,v1:172.21.15.133:6801/196197529] conn(0x7f050c0a11d0 0x7f050c0a3680 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:37.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:37.667+0000 7f0522fff700 1 --2- 172.21.15.133:0/3185761218 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f051c1017a0 0x7f051c19f260 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:37.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:37.667+0000 7f0522fff700 1 --2- 172.21.15.133:0/3185761218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f051c0ffc20 0x7f051c197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:37.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:37.667+0000 7f0522fff700 1 --2- 172.21.15.133:0/3185761218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f051c100e10 0x7f051c198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T18:35:37.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:37.667+0000 7f0522fff700 1 -- 172.21.15.133:0/3185761218 >> 172.21.15.133:0/3185761218 conn(0x7f051c0fb260 msgr2=0x7f051c106660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T18:35:37.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:37.668+0000 7f0522fff700 1 -- 172.21.15.133:0/3185761218 shutdown_connections 2023-07-14T18:35:37.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T18:35:37.668+0000 7f0522fff700 1 -- 172.21.15.133:0/3185761218 wait complete. 2023-07-14T18:35:37.687 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data_ec' removed 2023-07-14T18:35:37.710 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T18:35:37.807 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T18:35:37.855 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T18:35:37.900 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-07-14T18:35:37.901 DEBUG:teuthology.orchestra.run.smithi133:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-14T18:35:37.967 DEBUG:teuthology.orchestra.run.smithi161:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-14T18:35:37.986 DEBUG:teuthology.orchestra.run.smithi171:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-14T18:35:38.005 INFO:tasks.cephadm:Stopping all daemons... 2023-07-14T18:35:38.006 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-07-14T18:35:38.006 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mon.a 2023-07-14T18:35:38.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:37 smithi161 ceph-mon[128993]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-07-14T18:35:38.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:37 smithi161 ceph-mon[128993]: osdmap e95: 12 total, 12 up, 12 in 2023-07-14T18:35:38.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:37 smithi161 ceph-mon[128993]: from='client.? 172.21.15.133:0/3185761218' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T18:35:38.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:37 smithi161 ceph-mon[128993]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T18:35:38.080 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:37 smithi133 ceph-mon[126384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-07-14T18:35:38.080 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:37 smithi133 ceph-mon[126384]: osdmap e95: 12 total, 12 up, 12 in 2023-07-14T18:35:38.080 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:37 smithi133 ceph-mon[126384]: from='client.? 172.21.15.133:0/3185761218' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T18:35:38.080 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:37 smithi133 ceph-mon[126384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T18:35:38.089 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:37 smithi171 ceph-mon[129241]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-07-14T18:35:38.090 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:37 smithi171 ceph-mon[129241]: osdmap e95: 12 total, 12 up, 12 in 2023-07-14T18:35:38.090 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:37 smithi171 ceph-mon[129241]: from='client.? 172.21.15.133:0/3185761218' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T18:35:38.090 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:37 smithi171 ceph-mon[129241]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T18:35:38.380 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:38 smithi133 systemd[1]: Stopping Ceph mon.a for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:35:38.877 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:38 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mon-a[126360]: 2023-07-14T18:35:38.597+0000 7fc973181700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:35:38.877 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:38 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mon-a[126360]: 2023-07-14T18:35:38.597+0000 7fc973181700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-07-14T18:35:39.130 INFO:journalctl@ceph.mon.a.smithi133.stdout:Jul 14 18:35:38 smithi133 bash[186108]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mon-a 2023-07-14T18:35:39.458 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mon.a.service' 2023-07-14T18:35:39.647 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:35:39.648 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-07-14T18:35:39.648 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-07-14T18:35:39.648 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mon.b 2023-07-14T18:35:40.075 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:39 smithi161 systemd[1]: Stopping Ceph mon.b for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:35:40.824 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:40 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mon-b[128969]: 2023-07-14T18:35:40.493+0000 7f26216d3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:35:40.824 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:40 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mon-b[128969]: 2023-07-14T18:35:40.493+0000 7f26216d3700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-07-14T18:35:44.795 INFO:journalctl@ceph.mon.b.smithi161.stdout:Jul 14 18:35:44 smithi161 bash[160750]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mon-b 2023-07-14T18:35:44.803 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mon.b.service' 2023-07-14T18:35:44.999 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:35:45.000 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-07-14T18:35:45.000 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-07-14T18:35:45.000 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mon.c 2023-07-14T18:35:45.339 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:45 smithi171 systemd[1]: Stopping Ceph mon.c for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:35:47.716 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:47 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mon-c[129217]: 2023-07-14T18:35:47.320+0000 7fee60f31700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:35:47.716 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:47 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mon-c[129217]: 2023-07-14T18:35:47.321+0000 7fee60f31700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-07-14T18:35:50.584 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mon.c.service' 2023-07-14T18:35:50.839 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:50 smithi171 bash[161938]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mon-c 2023-07-14T18:35:50.839 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:50 smithi171 systemd[1]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mon.c.service: Succeeded. 2023-07-14T18:35:50.840 INFO:journalctl@ceph.mon.c.smithi171.stdout:Jul 14 18:35:50 smithi171 systemd[1]: Stopped Ceph mon.c for 06d0f5fa-2271-11ee-9b33-001a4aab830c. 2023-07-14T18:35:50.981 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:35:50.982 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-07-14T18:35:50.982 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-07-14T18:35:50.982 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mgr.x 2023-07-14T18:35:51.380 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:35:51 smithi133 systemd[1]: Stopping Ceph mgr.x for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:35:54.879 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:35:54 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x[126634]: 2023-07-14T18:35:54.412+0000 7fa6bb8be700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:35:59.463 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Jul 14 18:35:59 smithi133 bash[186253]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-x 2023-07-14T18:35:59.763 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mgr.x.service' 2023-07-14T18:35:59.852 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:35:59.852 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-07-14T18:35:59.852 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-07-14T18:35:59.853 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mgr.y 2023-07-14T18:36:00.298 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:35:59 smithi161 systemd[1]: Stopping Ceph mgr.y for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:36:00.566 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:36:00 smithi161 bash[160915]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-y 2023-07-14T18:36:00.567 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Jul 14 18:36:00 smithi161 systemd[1]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-07-14T18:36:00.755 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mgr.y.service' 2023-07-14T18:36:00.793 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:36:00.794 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-07-14T18:36:00.794 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-07-14T18:36:00.794 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mgr.z 2023-07-14T18:36:01.088 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:36:00 smithi171 systemd[1]: Stopping Ceph mgr.z for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:36:01.506 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:36:01 smithi171 bash[162100]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mgr-z 2023-07-14T18:36:01.687 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Jul 14 18:36:01 smithi171 systemd[1]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-07-14T18:36:01.691 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mgr.z.service' 2023-07-14T18:36:01.730 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:36:01.730 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-07-14T18:36:01.731 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-07-14T18:36:01.731 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.0 2023-07-14T18:36:02.129 INFO:journalctl@ceph.osd.0.smithi133.stdout:Jul 14 18:36:01 smithi133 systemd[1]: Stopping Ceph osd.0 for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:36:02.629 INFO:journalctl@ceph.osd.0.smithi133.stdout:Jul 14 18:36:02 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-0[139322]: 2023-07-14T18:36:02.233+0000 7f6530e14700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:36:02.629 INFO:journalctl@ceph.osd.0.smithi133.stdout:Jul 14 18:36:02 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-0[139322]: 2023-07-14T18:36:02.233+0000 7f6530e14700 -1 osd.0 96 *** Got signal Terminated *** 2023-07-14T18:36:02.629 INFO:journalctl@ceph.osd.0.smithi133.stdout:Jul 14 18:36:02 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-0[139322]: 2023-07-14T18:36:02.233+0000 7f6530e14700 -1 osd.0 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T18:36:08.429 INFO:journalctl@ceph.osd.0.smithi133.stdout:Jul 14 18:36:08 smithi133 bash[186400]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-0 2023-07-14T18:36:10.468 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.0.service' 2023-07-14T18:36:10.559 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:36:10.559 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-07-14T18:36:10.559 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-07-14T18:36:10.560 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.1 2023-07-14T18:36:10.901 INFO:journalctl@ceph.osd.1.smithi161.stdout:Jul 14 18:36:10 smithi161 systemd[1]: Stopping Ceph osd.1 for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:36:11.323 INFO:journalctl@ceph.osd.1.smithi161.stdout:Jul 14 18:36:10 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-1[134087]: 2023-07-14T18:36:10.883+0000 7fcd348d6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:36:11.323 INFO:journalctl@ceph.osd.1.smithi161.stdout:Jul 14 18:36:10 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-1[134087]: 2023-07-14T18:36:10.883+0000 7fcd348d6700 -1 osd.1 96 *** Got signal Terminated *** 2023-07-14T18:36:11.324 INFO:journalctl@ceph.osd.1.smithi161.stdout:Jul 14 18:36:10 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-1[134087]: 2023-07-14T18:36:10.883+0000 7fcd348d6700 -1 osd.1 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T18:36:16.823 INFO:journalctl@ceph.osd.1.smithi161.stdout:Jul 14 18:36:16 smithi161 bash[161061]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-1 2023-07-14T18:36:18.260 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.1.service' 2023-07-14T18:36:18.300 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:36:18.301 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-07-14T18:36:18.301 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-07-14T18:36:18.301 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.2 2023-07-14T18:36:18.588 INFO:journalctl@ceph.osd.2.smithi171.stdout:Jul 14 18:36:18 smithi171 systemd[1]: Stopping Ceph osd.2 for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:36:19.088 INFO:journalctl@ceph.osd.2.smithi171.stdout:Jul 14 18:36:18 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-2[135063]: 2023-07-14T18:36:18.647+0000 7f6207d18700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:36:19.088 INFO:journalctl@ceph.osd.2.smithi171.stdout:Jul 14 18:36:18 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-2[135063]: 2023-07-14T18:36:18.647+0000 7f6207d18700 -1 osd.2 96 *** Got signal Terminated *** 2023-07-14T18:36:19.088 INFO:journalctl@ceph.osd.2.smithi171.stdout:Jul 14 18:36:18 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-2[135063]: 2023-07-14T18:36:18.647+0000 7f6207d18700 -1 osd.2 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T18:36:24.540 INFO:journalctl@ceph.osd.2.smithi171.stdout:Jul 14 18:36:24 smithi171 bash[162247]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-2 2023-07-14T18:36:26.206 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.2.service' 2023-07-14T18:36:26.245 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:36:26.246 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-07-14T18:36:26.246 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-07-14T18:36:26.247 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.3 2023-07-14T18:36:26.629 INFO:journalctl@ceph.osd.3.smithi133.stdout:Jul 14 18:36:26 smithi133 systemd[1]: Stopping Ceph osd.3 for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:36:27.129 INFO:journalctl@ceph.osd.3.smithi133.stdout:Jul 14 18:36:26 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-3[144787]: 2023-07-14T18:36:26.762+0000 7f415e28d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:36:27.129 INFO:journalctl@ceph.osd.3.smithi133.stdout:Jul 14 18:36:26 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-3[144787]: 2023-07-14T18:36:26.762+0000 7f415e28d700 -1 osd.3 96 *** Got signal Terminated *** 2023-07-14T18:36:27.129 INFO:journalctl@ceph.osd.3.smithi133.stdout:Jul 14 18:36:26 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-3[144787]: 2023-07-14T18:36:26.762+0000 7f415e28d700 -1 osd.3 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T18:36:30.378 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:29 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:29.906+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:05.538087+0000 front 2023-07-14T18:36:05.537995+0000 (oldest deadline 2023-07-14T18:36:29.038095+0000) 2023-07-14T18:36:30.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:30 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:30.116+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:30.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:30 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:30.433+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:30.879 INFO:journalctl@ceph.osd.3.smithi133.stdout:Jul 14 18:36:30 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-3[144787]: 2023-07-14T18:36:30.362+0000 7f4158880700 -1 osd.3 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:04.965092+0000 front 2023-07-14T18:36:04.965426+0000 (oldest deadline 2023-07-14T18:36:30.265142+0000) 2023-07-14T18:36:30.879 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:30 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:30.465+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:31.363 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:30 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:30.932+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:05.538087+0000 front 2023-07-14T18:36:05.537995+0000 (oldest deadline 2023-07-14T18:36:29.038095+0000) 2023-07-14T18:36:31.565 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:31 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:31.162+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:31.629 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:31 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:31.444+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:31.629 INFO:journalctl@ceph.osd.3.smithi133.stdout:Jul 14 18:36:31 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-3[144787]: 2023-07-14T18:36:31.346+0000 7f4158880700 -1 osd.3 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:04.965092+0000 front 2023-07-14T18:36:04.965426+0000 (oldest deadline 2023-07-14T18:36:30.265142+0000) 2023-07-14T18:36:31.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:31 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:31.412+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:31.838 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:31 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:31.548+0000 7f76b0b4b700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.659240+0000 front 2023-07-14T18:36:06.659325+0000 (oldest deadline 2023-07-14T18:36:31.359386+0000) 2023-07-14T18:36:32.378 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:31 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:31.933+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:05.538087+0000 front 2023-07-14T18:36:05.537995+0000 (oldest deadline 2023-07-14T18:36:29.038095+0000) 2023-07-14T18:36:32.537 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:32 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:32.209+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:32.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:32 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:32.376+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:32.837 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:32 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:32.521+0000 7f76b0b4b700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.659240+0000 front 2023-07-14T18:36:06.659325+0000 (oldest deadline 2023-07-14T18:36:31.359386+0000) 2023-07-14T18:36:32.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:32 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:32.428+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:33.378 INFO:journalctl@ceph.osd.3.smithi133.stdout:Jul 14 18:36:32 smithi133 bash[186674]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-3 2023-07-14T18:36:33.379 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:32 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:32.891+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:05.538087+0000 front 2023-07-14T18:36:05.537995+0000 (oldest deadline 2023-07-14T18:36:29.038095+0000) 2023-07-14T18:36:33.571 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:33 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:33.181+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:33.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:33 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:33.389+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:33.837 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:33 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:33.554+0000 7f76b0b4b700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.659240+0000 front 2023-07-14T18:36:06.659325+0000 (oldest deadline 2023-07-14T18:36:31.359386+0000) 2023-07-14T18:36:33.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:33 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:33.405+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:34.345 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:33 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:33.929+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:05.538087+0000 front 2023-07-14T18:36:05.537995+0000 (oldest deadline 2023-07-14T18:36:29.038095+0000) 2023-07-14T18:36:34.560 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:34 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:34.156+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:34.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:34 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:34.427+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:34.784 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.3.service' 2023-07-14T18:36:34.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:34 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:34.385+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:34.837 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:34 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:34.544+0000 7f76b0b4b700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.659240+0000 front 2023-07-14T18:36:06.659325+0000 (oldest deadline 2023-07-14T18:36:31.359386+0000) 2023-07-14T18:36:34.873 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:36:34.873 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-07-14T18:36:34.873 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-07-14T18:36:34.874 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.4 2023-07-14T18:36:35.128 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:34 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:34.976+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:05.538087+0000 front 2023-07-14T18:36:05.537995+0000 (oldest deadline 2023-07-14T18:36:29.038095+0000) 2023-07-14T18:36:35.220 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jul 14 18:36:34 smithi161 systemd[1]: Stopping Ceph osd.4 for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:36:35.542 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:35 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:35.151+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:35.572 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:35 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:35.423+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:35.573 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jul 14 18:36:35 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-4[139499]: 2023-07-14T18:36:35.203+0000 7f4f14cfb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:36:35.573 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jul 14 18:36:35 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-4[139499]: 2023-07-14T18:36:35.203+0000 7f4f14cfb700 -1 osd.4 96 *** Got signal Terminated *** 2023-07-14T18:36:35.573 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jul 14 18:36:35 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-4[139499]: 2023-07-14T18:36:35.203+0000 7f4f14cfb700 -1 osd.4 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T18:36:35.837 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:35 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:35.526+0000 7f76b0b4b700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.659240+0000 front 2023-07-14T18:36:06.659325+0000 (oldest deadline 2023-07-14T18:36:31.359386+0000) 2023-07-14T18:36:35.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:35 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:35.413+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:36.378 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:36 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:36.024+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:05.538087+0000 front 2023-07-14T18:36:05.537995+0000 (oldest deadline 2023-07-14T18:36:29.038095+0000) 2023-07-14T18:36:36.556 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:36 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:36.127+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:36.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:36 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:36.463+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:36.837 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:36 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:36.540+0000 7f76b0b4b700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.659240+0000 front 2023-07-14T18:36:06.659325+0000 (oldest deadline 2023-07-14T18:36:31.359386+0000) 2023-07-14T18:36:36.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:36 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:36.407+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:37.378 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:37 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:37.029+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:05.538087+0000 front 2023-07-14T18:36:05.537995+0000 (oldest deadline 2023-07-14T18:36:29.038095+0000) 2023-07-14T18:36:37.378 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:37 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:37.029+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:13.040020+0000 front 2023-07-14T18:36:13.039984+0000 (oldest deadline 2023-07-14T18:36:36.539921+0000) 2023-07-14T18:36:37.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:37 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:37.136+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:37.822 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jul 14 18:36:37 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-4[139499]: 2023-07-14T18:36:37.358+0000 7f4f112f2700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:13.303868+0000 front 2023-07-14T18:36:13.303897+0000 (oldest deadline 2023-07-14T18:36:36.804061+0000) 2023-07-14T18:36:37.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:37 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:37.457+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:37.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:37 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:37.457+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:36:37.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:37 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:37.455+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:37.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:37 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:37.455+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:36:38.088 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:37 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:37.587+0000 7f76b0b4b700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.659240+0000 front 2023-07-14T18:36:06.659325+0000 (oldest deadline 2023-07-14T18:36:31.359386+0000) 2023-07-14T18:36:38.378 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:38 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:38.018+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:05.538087+0000 front 2023-07-14T18:36:05.537995+0000 (oldest deadline 2023-07-14T18:36:29.038095+0000) 2023-07-14T18:36:38.378 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:38 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:38.018+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:13.040020+0000 front 2023-07-14T18:36:13.039984+0000 (oldest deadline 2023-07-14T18:36:36.539921+0000) 2023-07-14T18:36:38.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:38 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:38.169+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:38.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:38 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:38.169+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:36:38.822 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jul 14 18:36:38 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-4[139499]: 2023-07-14T18:36:38.352+0000 7f4f112f2700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:13.303868+0000 front 2023-07-14T18:36:13.303897+0000 (oldest deadline 2023-07-14T18:36:36.804061+0000) 2023-07-14T18:36:38.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:38 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:38.438+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:38.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:38 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:38.438+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:36:38.837 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:38 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:38.571+0000 7f76b0b4b700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.659240+0000 front 2023-07-14T18:36:06.659325+0000 (oldest deadline 2023-07-14T18:36:31.359386+0000) 2023-07-14T18:36:38.838 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:38 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:38.571+0000 7f76b0b4b700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.460464+0000 front 2023-07-14T18:36:12.460482+0000 (oldest deadline 2023-07-14T18:36:37.760508+0000) 2023-07-14T18:36:38.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:38 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:38.420+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:38.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:38 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:38.420+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:36:39.378 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:39 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:39.049+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:05.538087+0000 front 2023-07-14T18:36:05.537995+0000 (oldest deadline 2023-07-14T18:36:29.038095+0000) 2023-07-14T18:36:39.378 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:39 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:39.049+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:13.040020+0000 front 2023-07-14T18:36:13.039984+0000 (oldest deadline 2023-07-14T18:36:36.539921+0000) 2023-07-14T18:36:39.544 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:39 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:39.172+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:39.544 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:39 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:39.172+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:36:39.822 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jul 14 18:36:39 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-4[139499]: 2023-07-14T18:36:39.311+0000 7f4f112f2700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:13.303868+0000 front 2023-07-14T18:36:13.303897+0000 (oldest deadline 2023-07-14T18:36:36.804061+0000) 2023-07-14T18:36:39.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:39 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:39.428+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:39.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:39 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:39.428+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:36:39.837 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:39 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:39.527+0000 7f76b0b4b700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.659240+0000 front 2023-07-14T18:36:06.659325+0000 (oldest deadline 2023-07-14T18:36:31.359386+0000) 2023-07-14T18:36:39.838 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:39 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:39.527+0000 7f76b0b4b700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.460464+0000 front 2023-07-14T18:36:12.460482+0000 (oldest deadline 2023-07-14T18:36:37.760508+0000) 2023-07-14T18:36:39.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:39 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:39.462+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:39.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:39 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:39.462+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:36:40.378 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:40 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:40.079+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:05.538087+0000 front 2023-07-14T18:36:05.537995+0000 (oldest deadline 2023-07-14T18:36:29.038095+0000) 2023-07-14T18:36:40.378 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:40 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:40.079+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:13.040020+0000 front 2023-07-14T18:36:13.039984+0000 (oldest deadline 2023-07-14T18:36:36.539921+0000) 2023-07-14T18:36:40.525 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:40 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:40.158+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:40.526 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:40 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:40.158+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:36:40.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:40 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:40.382+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:40.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:40 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:40.382+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:36:40.838 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:40 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:40.509+0000 7f76b0b4b700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.659240+0000 front 2023-07-14T18:36:06.659325+0000 (oldest deadline 2023-07-14T18:36:31.359386+0000) 2023-07-14T18:36:40.838 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:40 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:40.509+0000 7f76b0b4b700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.460464+0000 front 2023-07-14T18:36:12.460482+0000 (oldest deadline 2023-07-14T18:36:37.760508+0000) 2023-07-14T18:36:40.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:40 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:40.446+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:40.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:40 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:40.446+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:36:41.202 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:40 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:40.823+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:36:41.323 INFO:journalctl@ceph.osd.4.smithi161.stdout:Jul 14 18:36:41 smithi161 bash[161325]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-4 2023-07-14T18:36:41.323 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:41 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:41.226+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.766329+0000 front 2023-07-14T18:36:15.766365+0000 (oldest deadline 2023-07-14T18:36:41.066475+0000) 2023-07-14T18:36:41.378 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:41 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:41.104+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:05.538087+0000 front 2023-07-14T18:36:05.537995+0000 (oldest deadline 2023-07-14T18:36:29.038095+0000) 2023-07-14T18:36:41.378 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:41 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:41.104+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:13.040020+0000 front 2023-07-14T18:36:13.039984+0000 (oldest deadline 2023-07-14T18:36:36.539921+0000) 2023-07-14T18:36:41.479 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:41 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:41.186+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:41.479 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:41 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:41.186+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:36:41.782 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:41 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:41.373+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:41.782 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:41 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:41.373+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:36:41.837 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:41 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:41.463+0000 7f76b0b4b700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.659240+0000 front 2023-07-14T18:36:06.659325+0000 (oldest deadline 2023-07-14T18:36:31.359386+0000) 2023-07-14T18:36:41.838 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:41 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:41.463+0000 7f76b0b4b700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.460464+0000 front 2023-07-14T18:36:12.460482+0000 (oldest deadline 2023-07-14T18:36:37.760508+0000) 2023-07-14T18:36:41.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:41 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:41.406+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:41.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:41 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:41.406+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:36:42.193 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:41 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:41.836+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:36:42.388 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:42 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:42.249+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.766329+0000 front 2023-07-14T18:36:15.766365+0000 (oldest deadline 2023-07-14T18:36:41.066475+0000) 2023-07-14T18:36:42.452 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:42 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:42.124+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:05.538087+0000 front 2023-07-14T18:36:05.537995+0000 (oldest deadline 2023-07-14T18:36:29.038095+0000) 2023-07-14T18:36:42.452 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:42 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:42.124+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:13.040020+0000 front 2023-07-14T18:36:13.039984+0000 (oldest deadline 2023-07-14T18:36:36.539921+0000) 2023-07-14T18:36:42.481 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:42 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:42.177+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:42.481 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:42 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:42.177+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:36:42.525 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.4.service' 2023-07-14T18:36:42.564 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:36:42.564 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-07-14T18:36:42.565 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-07-14T18:36:42.565 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.5 2023-07-14T18:36:42.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:42 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:42.372+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:42.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:42 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:42.372+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:36:42.837 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:42 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:42.465+0000 7f76b0b4b700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.659240+0000 front 2023-07-14T18:36:06.659325+0000 (oldest deadline 2023-07-14T18:36:31.359386+0000) 2023-07-14T18:36:42.838 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:42 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:42.465+0000 7f76b0b4b700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.460464+0000 front 2023-07-14T18:36:12.460482+0000 (oldest deadline 2023-07-14T18:36:37.760508+0000) 2023-07-14T18:36:42.838 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:42 smithi171 systemd[1]: Stopping Ceph osd.5 for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:36:42.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:42 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:42.435+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:42.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:42 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:42.435+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:36:43.220 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:42 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:42.902+0000 7f76b4554700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:36:43.221 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:42 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:42.902+0000 7f76b4554700 -1 osd.5 96 *** Got signal Terminated *** 2023-07-14T18:36:43.221 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:42 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:42.902+0000 7f76b4554700 -1 osd.5 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T18:36:43.221 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:42 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:42.822+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:36:43.423 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:43 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:43.150+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:05.538087+0000 front 2023-07-14T18:36:05.537995+0000 (oldest deadline 2023-07-14T18:36:29.038095+0000) 2023-07-14T18:36:43.423 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:43 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:43.150+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:13.040020+0000 front 2023-07-14T18:36:13.039984+0000 (oldest deadline 2023-07-14T18:36:36.539921+0000) 2023-07-14T18:36:43.572 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:43 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:43.290+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.766329+0000 front 2023-07-14T18:36:15.766365+0000 (oldest deadline 2023-07-14T18:36:41.066475+0000) 2023-07-14T18:36:43.573 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:43 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:43.403+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:43.573 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:43 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:43.403+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:36:43.587 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:43 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:43.415+0000 7f76b0b4b700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.659240+0000 front 2023-07-14T18:36:06.659325+0000 (oldest deadline 2023-07-14T18:36:31.359386+0000) 2023-07-14T18:36:43.587 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:43 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:43.415+0000 7f76b0b4b700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.460464+0000 front 2023-07-14T18:36:12.460482+0000 (oldest deadline 2023-07-14T18:36:37.760508+0000) 2023-07-14T18:36:43.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:43 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:43.204+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:43.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:43 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:43.204+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:36:43.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:43 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:43.407+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:43.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:43 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:43.407+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:36:44.087 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:43 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:43.777+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:36:44.435 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:44 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:44.122+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:05.538087+0000 front 2023-07-14T18:36:05.537995+0000 (oldest deadline 2023-07-14T18:36:29.038095+0000) 2023-07-14T18:36:44.435 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:44 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:44.122+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:13.040020+0000 front 2023-07-14T18:36:13.039984+0000 (oldest deadline 2023-07-14T18:36:36.539921+0000) 2023-07-14T18:36:44.572 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:44 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:44.265+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.766329+0000 front 2023-07-14T18:36:15.766365+0000 (oldest deadline 2023-07-14T18:36:41.066475+0000) 2023-07-14T18:36:44.573 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:44 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:44.372+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:44.573 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:44 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:44.372+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:36:44.587 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:44 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:44.442+0000 7f76b0b4b700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.659240+0000 front 2023-07-14T18:36:06.659325+0000 (oldest deadline 2023-07-14T18:36:31.359386+0000) 2023-07-14T18:36:44.588 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:44 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:44.442+0000 7f76b0b4b700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.460464+0000 front 2023-07-14T18:36:12.460482+0000 (oldest deadline 2023-07-14T18:36:37.760508+0000) 2023-07-14T18:36:44.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:44 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:44.193+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:44.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:44 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:44.193+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:36:44.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:44 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:44.418+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:44.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:44 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:44.418+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:36:45.087 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:44 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:44.729+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:36:45.378 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:45 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:45.079+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:05.538087+0000 front 2023-07-14T18:36:05.537995+0000 (oldest deadline 2023-07-14T18:36:29.038095+0000) 2023-07-14T18:36:45.378 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:45 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:45.079+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:13.040020+0000 front 2023-07-14T18:36:13.039984+0000 (oldest deadline 2023-07-14T18:36:36.539921+0000) 2023-07-14T18:36:45.572 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:45 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:45.286+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.766329+0000 front 2023-07-14T18:36:15.766365+0000 (oldest deadline 2023-07-14T18:36:41.066475+0000) 2023-07-14T18:36:45.573 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:45 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:45.286+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:21.067072+0000 front 2023-07-14T18:36:21.067194+0000 (oldest deadline 2023-07-14T18:36:44.567143+0000) 2023-07-14T18:36:45.573 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:45 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:45.368+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:45.573 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:45 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:45.368+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:36:45.587 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:45 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:45.447+0000 7f76b0b4b700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.659240+0000 front 2023-07-14T18:36:06.659325+0000 (oldest deadline 2023-07-14T18:36:31.359386+0000) 2023-07-14T18:36:45.588 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:45 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:45.447+0000 7f76b0b4b700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.460464+0000 front 2023-07-14T18:36:12.460482+0000 (oldest deadline 2023-07-14T18:36:37.760508+0000) 2023-07-14T18:36:45.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:45 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:45.230+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:45.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:45 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:45.230+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:36:45.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:45 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:45.374+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:45.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:45 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:45.374+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:36:46.087 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:45 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:45.707+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:36:46.359 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:46 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:46.078+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:05.538087+0000 front 2023-07-14T18:36:05.537995+0000 (oldest deadline 2023-07-14T18:36:29.038095+0000) 2023-07-14T18:36:46.359 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:46 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:46.078+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:13.040020+0000 front 2023-07-14T18:36:13.039984+0000 (oldest deadline 2023-07-14T18:36:36.539921+0000) 2023-07-14T18:36:46.360 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:46 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:46.078+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:22.941932+0000 front 2023-07-14T18:36:22.941914+0000 (oldest deadline 2023-07-14T18:36:45.241828+0000) 2023-07-14T18:36:46.511 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:46 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:46.184+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:46.511 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:46 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:46.184+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:36:46.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:46 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:46.343+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:46.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:46 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:46.343+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:36:46.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:46 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:46.343+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:36:46.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:46 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:46.333+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.766329+0000 front 2023-07-14T18:36:15.766365+0000 (oldest deadline 2023-07-14T18:36:41.066475+0000) 2023-07-14T18:36:46.823 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:46 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:46.333+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:21.067072+0000 front 2023-07-14T18:36:21.067194+0000 (oldest deadline 2023-07-14T18:36:44.567143+0000) 2023-07-14T18:36:46.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:46 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:46.384+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:46.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:46 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:46.384+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:36:46.824 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:46 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:46.384+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:36:46.837 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:46 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:46.495+0000 7f76b0b4b700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.659240+0000 front 2023-07-14T18:36:06.659325+0000 (oldest deadline 2023-07-14T18:36:31.359386+0000) 2023-07-14T18:36:46.837 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:46 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:46.495+0000 7f76b0b4b700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.460464+0000 front 2023-07-14T18:36:12.460482+0000 (oldest deadline 2023-07-14T18:36:37.760508+0000) 2023-07-14T18:36:46.838 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:46 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:46.745+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:36:46.838 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:46 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:46.745+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.526794+0000 front 2023-07-14T18:36:20.526737+0000 (oldest deadline 2023-07-14T18:36:45.826883+0000) 2023-07-14T18:36:47.378 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:47 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:47.047+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:05.538087+0000 front 2023-07-14T18:36:05.537995+0000 (oldest deadline 2023-07-14T18:36:29.038095+0000) 2023-07-14T18:36:47.378 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:47 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:47.047+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:13.040020+0000 front 2023-07-14T18:36:13.039984+0000 (oldest deadline 2023-07-14T18:36:36.539921+0000) 2023-07-14T18:36:47.378 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:47 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:47.047+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:22.941932+0000 front 2023-07-14T18:36:22.941914+0000 (oldest deadline 2023-07-14T18:36:45.241828+0000) 2023-07-14T18:36:47.500 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:47 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:47.227+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:47.501 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:47 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:47.227+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:36:47.501 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:47 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:47.227+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:36:47.572 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:47 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:47.289+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.766329+0000 front 2023-07-14T18:36:15.766365+0000 (oldest deadline 2023-07-14T18:36:41.066475+0000) 2023-07-14T18:36:47.573 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:47 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:47.289+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:21.067072+0000 front 2023-07-14T18:36:21.067194+0000 (oldest deadline 2023-07-14T18:36:44.567143+0000) 2023-07-14T18:36:47.573 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:47 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:47.359+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:47.574 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:47 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:47.359+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:36:47.574 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:47 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:47.359+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:36:47.783 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:47 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:47.484+0000 7f76b0b4b700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.659240+0000 front 2023-07-14T18:36:06.659325+0000 (oldest deadline 2023-07-14T18:36:31.359386+0000) 2023-07-14T18:36:47.783 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:47 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5[140576]: 2023-07-14T18:36:47.484+0000 7f76b0b4b700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.460464+0000 front 2023-07-14T18:36:12.460482+0000 (oldest deadline 2023-07-14T18:36:37.760508+0000) 2023-07-14T18:36:47.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:47 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:47.379+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:47.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:47 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:47.379+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:36:47.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:47 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:47.379+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:36:48.087 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:47 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:47.767+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:36:48.088 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:47 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:47.767+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.526794+0000 front 2023-07-14T18:36:20.526737+0000 (oldest deadline 2023-07-14T18:36:45.826883+0000) 2023-07-14T18:36:48.349 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:48 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:48.039+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:05.538087+0000 front 2023-07-14T18:36:05.537995+0000 (oldest deadline 2023-07-14T18:36:29.038095+0000) 2023-07-14T18:36:48.350 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:48 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:48.039+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:13.040020+0000 front 2023-07-14T18:36:13.039984+0000 (oldest deadline 2023-07-14T18:36:36.539921+0000) 2023-07-14T18:36:48.350 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:48 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:48.039+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:22.941932+0000 front 2023-07-14T18:36:22.941914+0000 (oldest deadline 2023-07-14T18:36:45.241828+0000) 2023-07-14T18:36:48.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:48 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:48.196+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:48.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:48 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:48.196+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:36:48.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:48 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:48.196+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:36:48.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:48 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:48.333+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:48.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:48 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:48.333+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:36:48.629 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:48 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:48.333+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:36:48.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:48 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:48.318+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.766329+0000 front 2023-07-14T18:36:15.766365+0000 (oldest deadline 2023-07-14T18:36:41.066475+0000) 2023-07-14T18:36:48.823 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:48 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:48.318+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:21.067072+0000 front 2023-07-14T18:36:21.067194+0000 (oldest deadline 2023-07-14T18:36:44.567143+0000) 2023-07-14T18:36:48.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:48 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:48.312+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:48.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:48 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:48.312+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:36:48.824 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:48 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:48.312+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:36:49.088 INFO:journalctl@ceph.osd.5.smithi171.stdout:Jul 14 18:36:48 smithi171 bash[162510]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-5 2023-07-14T18:36:49.088 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:48 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:48.773+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:36:49.088 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:48 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:48.773+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.526794+0000 front 2023-07-14T18:36:20.526737+0000 (oldest deadline 2023-07-14T18:36:45.826883+0000) 2023-07-14T18:36:49.364 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:49 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:49.009+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:05.538087+0000 front 2023-07-14T18:36:05.537995+0000 (oldest deadline 2023-07-14T18:36:29.038095+0000) 2023-07-14T18:36:49.365 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:49 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:49.009+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:13.040020+0000 front 2023-07-14T18:36:13.039984+0000 (oldest deadline 2023-07-14T18:36:36.539921+0000) 2023-07-14T18:36:49.365 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:49 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:49.009+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:22.941932+0000 front 2023-07-14T18:36:22.941914+0000 (oldest deadline 2023-07-14T18:36:45.241828+0000) 2023-07-14T18:36:49.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:49 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:49.223+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:49.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:49 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:49.223+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:36:49.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:49 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:49.223+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:36:49.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:49 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:49.348+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:49.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:49 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:49.348+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:36:49.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:49 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:49.348+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:36:49.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:49 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:49.333+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.766329+0000 front 2023-07-14T18:36:15.766365+0000 (oldest deadline 2023-07-14T18:36:41.066475+0000) 2023-07-14T18:36:49.823 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:49 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:49.333+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:21.067072+0000 front 2023-07-14T18:36:21.067194+0000 (oldest deadline 2023-07-14T18:36:44.567143+0000) 2023-07-14T18:36:49.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:49 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:49.338+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:49.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:49 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:49.338+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:36:49.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:49 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:49.338+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:36:50.015 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:49 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:49.795+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:36:50.015 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:49 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:49.795+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.526794+0000 front 2023-07-14T18:36:20.526737+0000 (oldest deadline 2023-07-14T18:36:45.826883+0000) 2023-07-14T18:36:50.337 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:50 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:50.232+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:50.338 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:50 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:50.232+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:36:50.338 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:50 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:50.232+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:36:50.362 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:50 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:50.015+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:05.538087+0000 front 2023-07-14T18:36:05.537995+0000 (oldest deadline 2023-07-14T18:36:29.038095+0000) 2023-07-14T18:36:50.363 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:50 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:50.015+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:13.040020+0000 front 2023-07-14T18:36:13.039984+0000 (oldest deadline 2023-07-14T18:36:36.539921+0000) 2023-07-14T18:36:50.363 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:50 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:50.015+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:22.941932+0000 front 2023-07-14T18:36:22.941914+0000 (oldest deadline 2023-07-14T18:36:45.241828+0000) 2023-07-14T18:36:50.417 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.5.service' 2023-07-14T18:36:50.458 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:36:50.458 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-07-14T18:36:50.459 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-07-14T18:36:50.459 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.6 2023-07-14T18:36:50.628 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:50 smithi133 systemd[1]: Stopping Ceph osd.6 for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:36:50.629 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:50 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:50.346+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:50.629 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:50 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:50.346+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:36:50.629 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:50 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:50.346+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:36:50.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:50 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:50.311+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.766329+0000 front 2023-07-14T18:36:15.766365+0000 (oldest deadline 2023-07-14T18:36:41.066475+0000) 2023-07-14T18:36:50.823 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:50 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:50.311+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:21.067072+0000 front 2023-07-14T18:36:21.067194+0000 (oldest deadline 2023-07-14T18:36:44.567143+0000) 2023-07-14T18:36:50.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:50 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:50.385+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:50.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:50 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:50.385+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:36:50.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:50 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:50.385+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:36:51.087 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:50 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:50.760+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:36:51.088 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:50 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:50.760+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.526794+0000 front 2023-07-14T18:36:20.526737+0000 (oldest deadline 2023-07-14T18:36:45.826883+0000) 2023-07-14T18:36:51.378 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:50 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:50.972+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:05.538087+0000 front 2023-07-14T18:36:05.537995+0000 (oldest deadline 2023-07-14T18:36:29.038095+0000) 2023-07-14T18:36:51.378 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:50 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:50.972+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:13.040020+0000 front 2023-07-14T18:36:13.039984+0000 (oldest deadline 2023-07-14T18:36:36.539921+0000) 2023-07-14T18:36:51.378 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:50 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:50.972+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:22.941932+0000 front 2023-07-14T18:36:22.941914+0000 (oldest deadline 2023-07-14T18:36:45.241828+0000) 2023-07-14T18:36:51.379 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:50 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:50.974+0000 7f405c6bc700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:36:51.379 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:50 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:50.974+0000 7f405c6bc700 -1 osd.6 96 *** Got signal Terminated *** 2023-07-14T18:36:51.379 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:50 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:50.974+0000 7f405c6bc700 -1 osd.6 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T18:36:51.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:51 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:51.187+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:51.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:51 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:51.187+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:36:51.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:51 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:51.187+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:36:51.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:51 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:51.357+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.766329+0000 front 2023-07-14T18:36:15.766365+0000 (oldest deadline 2023-07-14T18:36:41.066475+0000) 2023-07-14T18:36:51.823 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:51 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:51.357+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:21.067072+0000 front 2023-07-14T18:36:21.067194+0000 (oldest deadline 2023-07-14T18:36:44.567143+0000) 2023-07-14T18:36:51.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:51 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:51.347+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:51.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:51 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:51.347+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:36:51.824 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:51 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:51.347+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:36:51.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:51 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:51.378+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:51.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:51 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:51.378+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:36:51.879 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:51 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:51.378+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:36:52.089 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:51 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:51.807+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:36:52.089 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:51 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:51.807+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.526794+0000 front 2023-07-14T18:36:20.526737+0000 (oldest deadline 2023-07-14T18:36:45.826883+0000) 2023-07-14T18:36:52.358 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:52 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:52.012+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:05.538087+0000 front 2023-07-14T18:36:05.537995+0000 (oldest deadline 2023-07-14T18:36:29.038095+0000) 2023-07-14T18:36:52.359 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:52 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:52.012+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:13.040020+0000 front 2023-07-14T18:36:13.039984+0000 (oldest deadline 2023-07-14T18:36:36.539921+0000) 2023-07-14T18:36:52.359 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:52 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:52.012+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:22.941932+0000 front 2023-07-14T18:36:22.941914+0000 (oldest deadline 2023-07-14T18:36:45.241828+0000) 2023-07-14T18:36:52.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:52 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:52.223+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:52.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:52 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:52.223+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:36:52.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:52 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:52.223+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:36:52.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:52 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:52.342+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:52.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:52 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:52.342+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:36:52.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:52 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:52.342+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:36:52.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:52 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:52.405+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.766329+0000 front 2023-07-14T18:36:15.766365+0000 (oldest deadline 2023-07-14T18:36:41.066475+0000) 2023-07-14T18:36:52.823 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:52 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:52.405+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:21.067072+0000 front 2023-07-14T18:36:21.067194+0000 (oldest deadline 2023-07-14T18:36:44.567143+0000) 2023-07-14T18:36:52.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:52 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:52.368+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:52.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:52 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:52.368+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:36:52.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:52 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:52.368+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:36:53.230 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:52 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:52.844+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:36:53.231 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:52 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:52.844+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.526794+0000 front 2023-07-14T18:36:20.526737+0000 (oldest deadline 2023-07-14T18:36:45.826883+0000) 2023-07-14T18:36:53.348 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:53 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:53.021+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:05.538087+0000 front 2023-07-14T18:36:05.537995+0000 (oldest deadline 2023-07-14T18:36:29.038095+0000) 2023-07-14T18:36:53.349 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:53 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:53.021+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:13.040020+0000 front 2023-07-14T18:36:13.039984+0000 (oldest deadline 2023-07-14T18:36:36.539921+0000) 2023-07-14T18:36:53.349 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:53 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:53.021+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:22.941932+0000 front 2023-07-14T18:36:22.941914+0000 (oldest deadline 2023-07-14T18:36:45.241828+0000) 2023-07-14T18:36:53.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:53 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:53.215+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:53.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:53 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:53.215+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:36:53.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:53 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:53.215+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:36:53.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:53 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:53.332+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:53.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:53 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:53.332+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:36:53.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:53 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:53.332+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:36:53.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:53 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:53.437+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.766329+0000 front 2023-07-14T18:36:15.766365+0000 (oldest deadline 2023-07-14T18:36:41.066475+0000) 2023-07-14T18:36:53.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:53 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:53.437+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:21.067072+0000 front 2023-07-14T18:36:21.067194+0000 (oldest deadline 2023-07-14T18:36:44.567143+0000) 2023-07-14T18:36:53.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:53 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:53.348+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:53.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:53 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:53.348+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:36:53.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:53 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:53.348+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:36:54.225 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:53 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:53.866+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:36:54.225 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:53 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:53.866+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.526794+0000 front 2023-07-14T18:36:20.526737+0000 (oldest deadline 2023-07-14T18:36:45.826883+0000) 2023-07-14T18:36:54.226 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:53 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:53.866+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.128157+0000 front 2023-07-14T18:36:31.127978+0000 (oldest deadline 2023-07-14T18:36:53.427979+0000) 2023-07-14T18:36:54.378 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:54 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:54.065+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:05.538087+0000 front 2023-07-14T18:36:05.537995+0000 (oldest deadline 2023-07-14T18:36:29.038095+0000) 2023-07-14T18:36:54.378 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:54 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:54.065+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:13.040020+0000 front 2023-07-14T18:36:13.039984+0000 (oldest deadline 2023-07-14T18:36:36.539921+0000) 2023-07-14T18:36:54.378 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:54 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:54.065+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:22.941932+0000 front 2023-07-14T18:36:22.941914+0000 (oldest deadline 2023-07-14T18:36:45.241828+0000) 2023-07-14T18:36:54.379 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:54 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:54.312+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:54.379 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:54 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:54.312+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:36:54.379 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:54 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:54.312+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:36:54.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:54 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:54.209+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:54.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:54 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:54.209+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:36:54.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:54 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:54.209+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:36:54.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:54 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:54.412+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.766329+0000 front 2023-07-14T18:36:15.766365+0000 (oldest deadline 2023-07-14T18:36:41.066475+0000) 2023-07-14T18:36:54.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:54 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:54.412+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:21.067072+0000 front 2023-07-14T18:36:21.067194+0000 (oldest deadline 2023-07-14T18:36:44.567143+0000) 2023-07-14T18:36:54.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:54 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:54.347+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:54.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:54 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:54.347+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:36:54.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:54 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:54.347+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:36:55.215 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:54 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:54.868+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:36:55.216 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:54 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:54.868+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.526794+0000 front 2023-07-14T18:36:20.526737+0000 (oldest deadline 2023-07-14T18:36:45.826883+0000) 2023-07-14T18:36:55.216 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:54 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:54.868+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.128157+0000 front 2023-07-14T18:36:31.127978+0000 (oldest deadline 2023-07-14T18:36:53.427979+0000) 2023-07-14T18:36:55.378 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:55 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:55.110+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:05.538087+0000 front 2023-07-14T18:36:05.537995+0000 (oldest deadline 2023-07-14T18:36:29.038095+0000) 2023-07-14T18:36:55.378 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:55 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:55.110+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:13.040020+0000 front 2023-07-14T18:36:13.039984+0000 (oldest deadline 2023-07-14T18:36:36.539921+0000) 2023-07-14T18:36:55.378 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:55 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6[150293]: 2023-07-14T18:36:55.110+0000 7f4056caf700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:22.941932+0000 front 2023-07-14T18:36:22.941914+0000 (oldest deadline 2023-07-14T18:36:45.241828+0000) 2023-07-14T18:36:55.379 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:55 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:55.273+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:55.379 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:55 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:55.273+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:36:55.379 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:55 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:55.273+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:36:55.379 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:55 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:55.273+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.252367+0000 front 2023-07-14T18:36:31.252344+0000 (oldest deadline 2023-07-14T18:36:54.752574+0000) 2023-07-14T18:36:55.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:55 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:55.200+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:55.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:55 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:55.200+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:36:55.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:55 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:55.200+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:36:55.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:55 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:55.419+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.766329+0000 front 2023-07-14T18:36:15.766365+0000 (oldest deadline 2023-07-14T18:36:41.066475+0000) 2023-07-14T18:36:55.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:55 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:55.419+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:21.067072+0000 front 2023-07-14T18:36:21.067194+0000 (oldest deadline 2023-07-14T18:36:44.567143+0000) 2023-07-14T18:36:55.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:55 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:55.358+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:55.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:55 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:55.358+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:36:55.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:55 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:55.358+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:36:56.204 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:55 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:55.904+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:36:56.204 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:55 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:55.904+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.526794+0000 front 2023-07-14T18:36:20.526737+0000 (oldest deadline 2023-07-14T18:36:45.826883+0000) 2023-07-14T18:36:56.204 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:55 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:55.904+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.128157+0000 front 2023-07-14T18:36:31.127978+0000 (oldest deadline 2023-07-14T18:36:53.427979+0000) 2023-07-14T18:36:56.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:56 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:56.188+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:56.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:56 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:56.188+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:36:56.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:56 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:56.188+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:36:56.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:56 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:56.252+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:56.629 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:56 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:56.252+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:36:56.629 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:56 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:56.252+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:36:56.629 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:56 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:56.252+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.252367+0000 front 2023-07-14T18:36:31.252344+0000 (oldest deadline 2023-07-14T18:36:54.752574+0000) 2023-07-14T18:36:56.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:56 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:56.453+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.766329+0000 front 2023-07-14T18:36:15.766365+0000 (oldest deadline 2023-07-14T18:36:41.066475+0000) 2023-07-14T18:36:56.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:56 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:56.453+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:21.067072+0000 front 2023-07-14T18:36:21.067194+0000 (oldest deadline 2023-07-14T18:36:44.567143+0000) 2023-07-14T18:36:56.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:56 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:56.319+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:56.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:56 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:56.319+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:36:56.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:56 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:56.319+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:36:57.217 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:56 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:56.857+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:36:57.218 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:56 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:56.857+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.526794+0000 front 2023-07-14T18:36:20.526737+0000 (oldest deadline 2023-07-14T18:36:45.826883+0000) 2023-07-14T18:36:57.218 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:56 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:56.857+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.128157+0000 front 2023-07-14T18:36:31.127978+0000 (oldest deadline 2023-07-14T18:36:53.427979+0000) 2023-07-14T18:36:57.572 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:57 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:57.487+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.766329+0000 front 2023-07-14T18:36:15.766365+0000 (oldest deadline 2023-07-14T18:36:41.066475+0000) 2023-07-14T18:36:57.572 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:57 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:57.487+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:21.067072+0000 front 2023-07-14T18:36:21.067194+0000 (oldest deadline 2023-07-14T18:36:44.567143+0000) 2023-07-14T18:36:57.573 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:57 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:57.290+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:57.573 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:57 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:57.290+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:36:57.573 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:57 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:57.290+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:36:57.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:57 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:57.202+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:57.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:57 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:57.202+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:36:57.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:57 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:57.202+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:36:57.627 INFO:journalctl@ceph.osd.6.smithi133.stdout:Jul 14 18:36:57 smithi133 bash[186954]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-6 2023-07-14T18:36:57.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:57 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:57.238+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:57.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:57 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:57.238+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:36:57.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:57 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:57.238+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:36:57.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:57 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:57.238+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.252367+0000 front 2023-07-14T18:36:31.252344+0000 (oldest deadline 2023-07-14T18:36:54.752574+0000) 2023-07-14T18:36:58.087 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:57 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:57.818+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:36:58.088 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:57 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:57.818+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.526794+0000 front 2023-07-14T18:36:20.526737+0000 (oldest deadline 2023-07-14T18:36:45.826883+0000) 2023-07-14T18:36:58.088 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:57 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:57.818+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.128157+0000 front 2023-07-14T18:36:31.127978+0000 (oldest deadline 2023-07-14T18:36:53.427979+0000) 2023-07-14T18:36:58.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:58 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:58.192+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:58.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:58 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:58.192+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:36:58.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:58 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:58.192+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:36:58.604 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:58 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:58.245+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:58.604 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:58 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:58.245+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:36:58.605 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:58 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:58.245+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:36:58.605 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:58 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:58.245+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.252367+0000 front 2023-07-14T18:36:31.252344+0000 (oldest deadline 2023-07-14T18:36:54.752574+0000) 2023-07-14T18:36:58.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:58 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:58.458+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.766329+0000 front 2023-07-14T18:36:15.766365+0000 (oldest deadline 2023-07-14T18:36:41.066475+0000) 2023-07-14T18:36:58.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:58 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:58.458+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:21.067072+0000 front 2023-07-14T18:36:21.067194+0000 (oldest deadline 2023-07-14T18:36:44.567143+0000) 2023-07-14T18:36:58.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:58 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:58.458+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:29.268424+0000 front 2023-07-14T18:36:29.268539+0000 (oldest deadline 2023-07-14T18:36:58.068831+0000) 2023-07-14T18:36:58.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:58 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:58.333+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:58.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:58 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:58.333+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:36:58.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:58 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:58.333+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:36:58.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:58 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:58.333+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:36:59.079 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.6.service' 2023-07-14T18:36:59.087 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:58 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:58.794+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:36:59.087 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:58 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:58.794+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.526794+0000 front 2023-07-14T18:36:20.526737+0000 (oldest deadline 2023-07-14T18:36:45.826883+0000) 2023-07-14T18:36:59.087 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:58 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:58.794+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.128157+0000 front 2023-07-14T18:36:31.127978+0000 (oldest deadline 2023-07-14T18:36:53.427979+0000) 2023-07-14T18:36:59.168 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:36:59.168 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-07-14T18:36:59.168 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-07-14T18:36:59.169 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.7 2023-07-14T18:36:59.378 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:59 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:59.213+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:36:59.378 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:59 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:59.213+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:36:59.378 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:59 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:59.213+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:36:59.378 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:36:59 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:36:59.213+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.252367+0000 front 2023-07-14T18:36:31.252344+0000 (oldest deadline 2023-07-14T18:36:54.752574+0000) 2023-07-14T18:36:59.458 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:59 smithi161 systemd[1]: Stopping Ceph osd.7 for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:36:59.458 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:59 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:59.334+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:36:59.458 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:59 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:59.334+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:36:59.459 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:59 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:59.334+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:36:59.459 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:36:59 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:36:59.334+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:36:59.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:59 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:59.212+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:36:59.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:59 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:59.212+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:36:59.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:36:59 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:36:59.212+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:36:59.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:59 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:59.441+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.766329+0000 front 2023-07-14T18:36:15.766365+0000 (oldest deadline 2023-07-14T18:36:41.066475+0000) 2023-07-14T18:36:59.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:59 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:59.441+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:21.067072+0000 front 2023-07-14T18:36:21.067194+0000 (oldest deadline 2023-07-14T18:36:44.567143+0000) 2023-07-14T18:36:59.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:59 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:59.441+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:29.268424+0000 front 2023-07-14T18:36:29.268539+0000 (oldest deadline 2023-07-14T18:36:58.068831+0000) 2023-07-14T18:36:59.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:59 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:59.491+0000 7faacd8c0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:36:59.823 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:59 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:59.491+0000 7faacd8c0700 -1 osd.7 96 *** Got signal Terminated *** 2023-07-14T18:36:59.823 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:36:59 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:36:59.491+0000 7faacd8c0700 -1 osd.7 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T18:37:00.087 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:59 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:59.806+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:37:00.087 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:59 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:59.806+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.526794+0000 front 2023-07-14T18:36:20.526737+0000 (oldest deadline 2023-07-14T18:36:45.826883+0000) 2023-07-14T18:37:00.088 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:36:59 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:36:59.806+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.128157+0000 front 2023-07-14T18:36:31.127978+0000 (oldest deadline 2023-07-14T18:36:53.427979+0000) 2023-07-14T18:37:00.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:00 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:00.245+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:00.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:00 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:00.245+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:00.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:00 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:00.245+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:00.627 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:00 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:00.244+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:37:00.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:00 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:00.244+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:37:00.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:00 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:00.244+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:37:00.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:00 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:00.244+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.252367+0000 front 2023-07-14T18:36:31.252344+0000 (oldest deadline 2023-07-14T18:36:54.752574+0000) 2023-07-14T18:37:00.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:37:00 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:37:00.398+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.766329+0000 front 2023-07-14T18:36:15.766365+0000 (oldest deadline 2023-07-14T18:36:41.066475+0000) 2023-07-14T18:37:00.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:37:00 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:37:00.398+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:21.067072+0000 front 2023-07-14T18:36:21.067194+0000 (oldest deadline 2023-07-14T18:36:44.567143+0000) 2023-07-14T18:37:00.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:37:00 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:37:00.398+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:29.268424+0000 front 2023-07-14T18:36:29.268539+0000 (oldest deadline 2023-07-14T18:36:58.068831+0000) 2023-07-14T18:37:00.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:00 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:00.375+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:00.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:00 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:00.375+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:00.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:00 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:00.375+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:00.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:00 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:00.375+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:01.280 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:00 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:00.840+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:37:01.280 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:00 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:00.840+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.526794+0000 front 2023-07-14T18:36:20.526737+0000 (oldest deadline 2023-07-14T18:36:45.826883+0000) 2023-07-14T18:37:01.280 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:00 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:00.840+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.128157+0000 front 2023-07-14T18:36:31.127978+0000 (oldest deadline 2023-07-14T18:36:53.427979+0000) 2023-07-14T18:37:01.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:01 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:01.264+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:01.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:01 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:01.264+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:01.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:01 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:01.264+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:01.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:01 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:01.260+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:37:01.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:01 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:01.260+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:37:01.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:01 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:01.260+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:37:01.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:01 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:01.260+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.252367+0000 front 2023-07-14T18:36:31.252344+0000 (oldest deadline 2023-07-14T18:36:54.752574+0000) 2023-07-14T18:37:01.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:37:01 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:37:01.436+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.766329+0000 front 2023-07-14T18:36:15.766365+0000 (oldest deadline 2023-07-14T18:36:41.066475+0000) 2023-07-14T18:37:01.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:37:01 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:37:01.436+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:21.067072+0000 front 2023-07-14T18:36:21.067194+0000 (oldest deadline 2023-07-14T18:36:44.567143+0000) 2023-07-14T18:37:01.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:37:01 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:37:01.436+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:29.268424+0000 front 2023-07-14T18:36:29.268539+0000 (oldest deadline 2023-07-14T18:36:58.068831+0000) 2023-07-14T18:37:01.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:01 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:01.420+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:01.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:01 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:01.420+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:01.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:01 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:01.420+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:01.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:01 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:01.420+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:02.295 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:01 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:01.849+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:37:02.295 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:01 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:01.849+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.526794+0000 front 2023-07-14T18:36:20.526737+0000 (oldest deadline 2023-07-14T18:36:45.826883+0000) 2023-07-14T18:37:02.295 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:01 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:01.849+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.128157+0000 front 2023-07-14T18:36:31.127978+0000 (oldest deadline 2023-07-14T18:36:53.427979+0000) 2023-07-14T18:37:02.296 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:01 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:01.849+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.829680+0000 front 2023-07-14T18:36:39.829782+0000 (oldest deadline 2023-07-14T18:37:01.529641+0000) 2023-07-14T18:37:02.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:02 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:02.279+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:02.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:02 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:02.279+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:02.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:02 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:02.279+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:02.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:02 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:02.279+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:02.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:02 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:02.275+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:37:02.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:02 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:02.275+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:37:02.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:02 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:02.275+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:37:02.629 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:02 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:02.275+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.252367+0000 front 2023-07-14T18:36:31.252344+0000 (oldest deadline 2023-07-14T18:36:54.752574+0000) 2023-07-14T18:37:02.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:37:02 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:37:02.440+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.766329+0000 front 2023-07-14T18:36:15.766365+0000 (oldest deadline 2023-07-14T18:36:41.066475+0000) 2023-07-14T18:37:02.823 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:37:02 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:37:02.440+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:21.067072+0000 front 2023-07-14T18:36:21.067194+0000 (oldest deadline 2023-07-14T18:36:44.567143+0000) 2023-07-14T18:37:02.823 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:37:02 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:37:02.440+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:29.268424+0000 front 2023-07-14T18:36:29.268539+0000 (oldest deadline 2023-07-14T18:36:58.068831+0000) 2023-07-14T18:37:02.823 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:37:02 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:37:02.440+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:38.069204+0000 front 2023-07-14T18:36:38.069237+0000 (oldest deadline 2023-07-14T18:37:02.169373+0000) 2023-07-14T18:37:02.824 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:02 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:02.371+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:02.824 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:02 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:02.371+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:02.824 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:02 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:02.371+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:02.825 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:02 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:02.371+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:03.252 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:02 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:02.894+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:37:03.253 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:02 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:02.894+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.526794+0000 front 2023-07-14T18:36:20.526737+0000 (oldest deadline 2023-07-14T18:36:45.826883+0000) 2023-07-14T18:37:03.253 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:02 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:02.894+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.128157+0000 front 2023-07-14T18:36:31.127978+0000 (oldest deadline 2023-07-14T18:36:53.427979+0000) 2023-07-14T18:37:03.253 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:02 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:02.894+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.829680+0000 front 2023-07-14T18:36:39.829782+0000 (oldest deadline 2023-07-14T18:37:01.529641+0000) 2023-07-14T18:37:03.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:03 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:03.237+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:03.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:03 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:03.237+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:03.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:03 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:03.237+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:03.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:03 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:03.237+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:03.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:03 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:03.313+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:37:03.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:03 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:03.313+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:37:03.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:03 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:03.313+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:37:03.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:03 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:03.313+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.252367+0000 front 2023-07-14T18:36:31.252344+0000 (oldest deadline 2023-07-14T18:36:54.752574+0000) 2023-07-14T18:37:03.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:37:03 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:37:03.391+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.766329+0000 front 2023-07-14T18:36:15.766365+0000 (oldest deadline 2023-07-14T18:36:41.066475+0000) 2023-07-14T18:37:03.823 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:37:03 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:37:03.391+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:21.067072+0000 front 2023-07-14T18:36:21.067194+0000 (oldest deadline 2023-07-14T18:36:44.567143+0000) 2023-07-14T18:37:03.823 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:37:03 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:37:03.391+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:29.268424+0000 front 2023-07-14T18:36:29.268539+0000 (oldest deadline 2023-07-14T18:36:58.068831+0000) 2023-07-14T18:37:03.823 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:37:03 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:37:03.391+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:38.069204+0000 front 2023-07-14T18:36:38.069237+0000 (oldest deadline 2023-07-14T18:37:02.169373+0000) 2023-07-14T18:37:03.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:03 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:03.393+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:03.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:03 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:03.393+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:03.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:03 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:03.393+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:03.824 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:03 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:03.393+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:04.275 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:03 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:03.856+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:37:04.275 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:03 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:03.856+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.526794+0000 front 2023-07-14T18:36:20.526737+0000 (oldest deadline 2023-07-14T18:36:45.826883+0000) 2023-07-14T18:37:04.275 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:03 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:03.856+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.128157+0000 front 2023-07-14T18:36:31.127978+0000 (oldest deadline 2023-07-14T18:36:53.427979+0000) 2023-07-14T18:37:04.275 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:03 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:03.856+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.829680+0000 front 2023-07-14T18:36:39.829782+0000 (oldest deadline 2023-07-14T18:37:01.529641+0000) 2023-07-14T18:37:04.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:04 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:04.259+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:04.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:04 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:04.259+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:04.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:04 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:04.259+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:04.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:04 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:04.259+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:04.627 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:04 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:04.342+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:37:04.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:04 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:04.342+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:37:04.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:04 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:04.342+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:37:04.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:04 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:04.342+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.252367+0000 front 2023-07-14T18:36:31.252344+0000 (oldest deadline 2023-07-14T18:36:54.752574+0000) 2023-07-14T18:37:04.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:37:04 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:37:04.393+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.766329+0000 front 2023-07-14T18:36:15.766365+0000 (oldest deadline 2023-07-14T18:36:41.066475+0000) 2023-07-14T18:37:04.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:37:04 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:37:04.393+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:21.067072+0000 front 2023-07-14T18:36:21.067194+0000 (oldest deadline 2023-07-14T18:36:44.567143+0000) 2023-07-14T18:37:04.822 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:37:04 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:37:04.393+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:29.268424+0000 front 2023-07-14T18:36:29.268539+0000 (oldest deadline 2023-07-14T18:36:58.068831+0000) 2023-07-14T18:37:04.823 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:37:04 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7[145027]: 2023-07-14T18:37:04.393+0000 7faac76b2700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:38.069204+0000 front 2023-07-14T18:36:38.069237+0000 (oldest deadline 2023-07-14T18:37:02.169373+0000) 2023-07-14T18:37:04.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:04 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:04.438+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:04.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:04 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:04.438+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:04.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:04 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:04.438+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:04.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:04 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:04.438+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:05.302 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:04 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:04.894+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:37:05.303 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:04 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:04.894+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.526794+0000 front 2023-07-14T18:36:20.526737+0000 (oldest deadline 2023-07-14T18:36:45.826883+0000) 2023-07-14T18:37:05.303 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:04 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:04.894+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.128157+0000 front 2023-07-14T18:36:31.127978+0000 (oldest deadline 2023-07-14T18:36:53.427979+0000) 2023-07-14T18:37:05.303 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:04 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:04.894+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.829680+0000 front 2023-07-14T18:36:39.829782+0000 (oldest deadline 2023-07-14T18:37:01.529641+0000) 2023-07-14T18:37:05.468 INFO:journalctl@ceph.osd.7.smithi161.stdout:Jul 14 18:37:05 smithi161 bash[161601]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-7 2023-07-14T18:37:05.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:05 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:05.287+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:05.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:05 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:05.287+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:05.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:05 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:05.287+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:05.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:05 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:05.287+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:05.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:05 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:05.339+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:37:05.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:05 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:05.339+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:37:05.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:05 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:05.339+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:37:05.629 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:05 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:05.339+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.252367+0000 front 2023-07-14T18:36:31.252344+0000 (oldest deadline 2023-07-14T18:36:54.752574+0000) 2023-07-14T18:37:05.818 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:05 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:05.452+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:05.818 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:05 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:05.452+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:05.818 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:05 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:05.452+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:05.818 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:05 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:05.452+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:05.818 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:05 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:05.452+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.013712+0000 front 2023-07-14T18:36:39.013773+0000 (oldest deadline 2023-07-14T18:37:04.814193+0000) 2023-07-14T18:37:06.264 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:05 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:05.924+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:37:06.265 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:05 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:05.924+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.526794+0000 front 2023-07-14T18:36:20.526737+0000 (oldest deadline 2023-07-14T18:36:45.826883+0000) 2023-07-14T18:37:06.265 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:05 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:05.924+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.128157+0000 front 2023-07-14T18:36:31.127978+0000 (oldest deadline 2023-07-14T18:36:53.427979+0000) 2023-07-14T18:37:06.265 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:05 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:05.924+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.829680+0000 front 2023-07-14T18:36:39.829782+0000 (oldest deadline 2023-07-14T18:37:01.529641+0000) 2023-07-14T18:37:06.453 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.7.service' 2023-07-14T18:37:06.493 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:37:06.494 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-07-14T18:37:06.494 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-07-14T18:37:06.494 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.8 2023-07-14T18:37:06.526 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:06 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:06.249+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:06.526 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:06 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:06.249+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:06.527 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:06 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:06.249+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:06.527 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:06 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:06.249+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:06.627 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:06 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:06.303+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:37:06.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:06 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:06.303+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:37:06.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:06 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:06.303+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:37:06.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:06 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:06.303+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.252367+0000 front 2023-07-14T18:36:31.252344+0000 (oldest deadline 2023-07-14T18:36:54.752574+0000) 2023-07-14T18:37:06.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:06 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:06.303+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:34.753382+0000 front 2023-07-14T18:36:34.753342+0000 (oldest deadline 2023-07-14T18:37:05.354033+0000) 2023-07-14T18:37:06.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:06 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:06.458+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:06.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:06 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:06.458+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:06.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:06 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:06.458+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:06.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:06 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:06.458+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:06.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:06 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:06.458+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.013712+0000 front 2023-07-14T18:36:39.013773+0000 (oldest deadline 2023-07-14T18:37:04.814193+0000) 2023-07-14T18:37:06.837 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:06 smithi171 systemd[1]: Stopping Ceph osd.8 for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:37:07.255 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:06 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:06.831+0000 7f6d571d6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:37:07.255 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:06 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:06.831+0000 7f6d571d6700 -1 osd.8 96 *** Got signal Terminated *** 2023-07-14T18:37:07.255 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:06 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:06.831+0000 7f6d571d6700 -1 osd.8 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T18:37:07.256 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:06 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:06.934+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:37:07.256 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:06 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:06.934+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.526794+0000 front 2023-07-14T18:36:20.526737+0000 (oldest deadline 2023-07-14T18:36:45.826883+0000) 2023-07-14T18:37:07.256 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:06 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:06.934+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.128157+0000 front 2023-07-14T18:36:31.127978+0000 (oldest deadline 2023-07-14T18:36:53.427979+0000) 2023-07-14T18:37:07.256 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:06 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:06.934+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.829680+0000 front 2023-07-14T18:36:39.829782+0000 (oldest deadline 2023-07-14T18:37:01.529641+0000) 2023-07-14T18:37:07.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:07 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:07.239+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:07.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:07 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:07.239+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:07.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:07 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:07.239+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:07.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:07 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:07.239+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:07.627 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:07 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:07.330+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:37:07.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:07 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:07.330+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:37:07.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:07 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:07.330+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:37:07.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:07 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:07.330+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.252367+0000 front 2023-07-14T18:36:31.252344+0000 (oldest deadline 2023-07-14T18:36:54.752574+0000) 2023-07-14T18:37:07.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:07 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:07.330+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:34.753382+0000 front 2023-07-14T18:36:34.753342+0000 (oldest deadline 2023-07-14T18:37:05.354033+0000) 2023-07-14T18:37:07.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:07 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:07.503+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:07.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:07 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:07.504+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:07.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:07 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:07.504+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:07.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:07 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:07.504+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:07.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:07 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:07.504+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.013712+0000 front 2023-07-14T18:36:39.013773+0000 (oldest deadline 2023-07-14T18:37:04.814193+0000) 2023-07-14T18:37:08.282 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:07 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:07.949+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:37:08.282 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:07 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:07.949+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.526794+0000 front 2023-07-14T18:36:20.526737+0000 (oldest deadline 2023-07-14T18:36:45.826883+0000) 2023-07-14T18:37:08.282 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:07 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:07.949+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.128157+0000 front 2023-07-14T18:36:31.127978+0000 (oldest deadline 2023-07-14T18:36:53.427979+0000) 2023-07-14T18:37:08.283 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:07 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:07.949+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.829680+0000 front 2023-07-14T18:36:39.829782+0000 (oldest deadline 2023-07-14T18:37:01.529641+0000) 2023-07-14T18:37:08.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:08 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:08.267+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:08.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:08 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:08.267+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:08.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:08 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:08.267+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:08.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:08 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:08.267+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:08.627 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:08 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:08.357+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:37:08.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:08 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:08.357+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:37:08.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:08 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:08.357+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:37:08.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:08 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:08.357+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.252367+0000 front 2023-07-14T18:36:31.252344+0000 (oldest deadline 2023-07-14T18:36:54.752574+0000) 2023-07-14T18:37:08.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:08 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:08.357+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:34.753382+0000 front 2023-07-14T18:36:34.753342+0000 (oldest deadline 2023-07-14T18:37:05.354033+0000) 2023-07-14T18:37:08.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:08 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:08.499+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:08.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:08 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:08.499+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:08.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:08 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:08.499+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:08.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:08 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:08.499+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:08.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:08 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:08.499+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.013712+0000 front 2023-07-14T18:36:39.013773+0000 (oldest deadline 2023-07-14T18:37:04.814193+0000) 2023-07-14T18:37:09.321 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:08 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:08.980+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:37:09.322 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:08 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:08.980+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.526794+0000 front 2023-07-14T18:36:20.526737+0000 (oldest deadline 2023-07-14T18:36:45.826883+0000) 2023-07-14T18:37:09.322 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:08 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:08.980+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.128157+0000 front 2023-07-14T18:36:31.127978+0000 (oldest deadline 2023-07-14T18:36:53.427979+0000) 2023-07-14T18:37:09.322 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:08 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:08.980+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.829680+0000 front 2023-07-14T18:36:39.829782+0000 (oldest deadline 2023-07-14T18:37:01.529641+0000) 2023-07-14T18:37:09.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:09 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:09.305+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:09.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:09 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:09.305+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:09.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:09 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:09.305+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:09.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:09 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:09.305+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:09.627 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:09 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:09.358+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:37:09.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:09 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:09.358+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:37:09.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:09 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:09.358+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:37:09.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:09 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:09.358+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.252367+0000 front 2023-07-14T18:36:31.252344+0000 (oldest deadline 2023-07-14T18:36:54.752574+0000) 2023-07-14T18:37:09.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:09 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:09.358+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:34.753382+0000 front 2023-07-14T18:36:34.753342+0000 (oldest deadline 2023-07-14T18:37:05.354033+0000) 2023-07-14T18:37:09.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:09 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:09.511+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:09.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:09 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:09.511+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:09.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:09 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:09.511+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:09.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:09 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:09.511+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:09.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:09 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:09.511+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.013712+0000 front 2023-07-14T18:36:39.013773+0000 (oldest deadline 2023-07-14T18:37:04.814193+0000) 2023-07-14T18:37:09.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:09 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:09.511+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:47.715375+0000 front 2023-07-14T18:36:47.715425+0000 (oldest deadline 2023-07-14T18:37:08.815165+0000) 2023-07-14T18:37:10.337 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:09 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:09.966+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:37:10.337 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:09 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:09.966+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.526794+0000 front 2023-07-14T18:36:20.526737+0000 (oldest deadline 2023-07-14T18:36:45.826883+0000) 2023-07-14T18:37:10.337 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:09 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:09.966+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.128157+0000 front 2023-07-14T18:36:31.127978+0000 (oldest deadline 2023-07-14T18:36:53.427979+0000) 2023-07-14T18:37:10.337 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:09 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:09.966+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.829680+0000 front 2023-07-14T18:36:39.829782+0000 (oldest deadline 2023-07-14T18:37:01.529641+0000) 2023-07-14T18:37:10.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:10 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:10.465+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:10.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:10 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:10.465+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:10.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:10 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:10.465+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:10.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:10 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:10.465+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:10.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:10 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:10.465+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.013712+0000 front 2023-07-14T18:36:39.013773+0000 (oldest deadline 2023-07-14T18:37:04.814193+0000) 2023-07-14T18:37:10.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:10 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:10.465+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:47.715375+0000 front 2023-07-14T18:36:47.715425+0000 (oldest deadline 2023-07-14T18:37:08.815165+0000) 2023-07-14T18:37:10.837 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:10 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:10.326+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:10.837 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:10 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:10.326+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:10.837 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:10 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:10.326+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:10.837 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:10 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:10.326+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:10.877 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:10 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:10.363+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:37:10.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:10 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:10.363+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:37:10.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:10 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:10.363+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:37:10.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:10 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:10.363+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.252367+0000 front 2023-07-14T18:36:31.252344+0000 (oldest deadline 2023-07-14T18:36:54.752574+0000) 2023-07-14T18:37:10.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:10 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:10.363+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:34.753382+0000 front 2023-07-14T18:36:34.753342+0000 (oldest deadline 2023-07-14T18:37:05.354033+0000) 2023-07-14T18:37:10.879 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:10 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:10.363+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.354776+0000 front 2023-07-14T18:36:45.355045+0000 (oldest deadline 2023-07-14T18:37:10.054540+0000) 2023-07-14T18:37:11.337 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:10 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:10.922+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:10.526122+0000 front 2023-07-14T18:36:10.526139+0000 (oldest deadline 2023-07-14T18:36:40.526380+0000) 2023-07-14T18:37:11.337 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:10 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:10.922+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.526794+0000 front 2023-07-14T18:36:20.526737+0000 (oldest deadline 2023-07-14T18:36:45.826883+0000) 2023-07-14T18:37:11.337 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:10 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:10.922+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.128157+0000 front 2023-07-14T18:36:31.127978+0000 (oldest deadline 2023-07-14T18:36:53.427979+0000) 2023-07-14T18:37:11.338 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:10 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:10.922+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.829680+0000 front 2023-07-14T18:36:39.829782+0000 (oldest deadline 2023-07-14T18:37:01.529641+0000) 2023-07-14T18:37:11.338 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:10 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8[146141]: 2023-07-14T18:37:10.922+0000 7f6d4f7c5700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:46.131053+0000 front 2023-07-14T18:36:46.131075+0000 (oldest deadline 2023-07-14T18:37:10.231113+0000) 2023-07-14T18:37:11.627 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:11 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:11.351+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:37:11.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:11 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:11.351+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:37:11.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:11 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:11.351+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:37:11.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:11 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:11.351+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.252367+0000 front 2023-07-14T18:36:31.252344+0000 (oldest deadline 2023-07-14T18:36:54.752574+0000) 2023-07-14T18:37:11.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:11 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:11.351+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:34.753382+0000 front 2023-07-14T18:36:34.753342+0000 (oldest deadline 2023-07-14T18:37:05.354033+0000) 2023-07-14T18:37:11.629 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:11 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:11.351+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.354776+0000 front 2023-07-14T18:36:45.355045+0000 (oldest deadline 2023-07-14T18:37:10.054540+0000) 2023-07-14T18:37:11.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:11 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:11.466+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:11.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:11 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:11.466+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:11.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:11 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:11.466+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:11.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:11 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:11.466+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:11.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:11 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:11.466+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.013712+0000 front 2023-07-14T18:36:39.013773+0000 (oldest deadline 2023-07-14T18:37:04.814193+0000) 2023-07-14T18:37:11.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:11 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:11.466+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:47.715375+0000 front 2023-07-14T18:36:47.715425+0000 (oldest deadline 2023-07-14T18:37:08.815165+0000) 2023-07-14T18:37:11.840 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:11 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:11.343+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:11.840 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:11 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:11.343+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:11.841 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:11 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:11.343+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:11.841 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:11 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:11.343+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:11.841 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:11 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:11.343+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.600897+0000 front 2023-07-14T18:36:45.600916+0000 (oldest deadline 2023-07-14T18:37:10.901000+0000) 2023-07-14T18:37:12.627 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:12 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:12.311+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:37:12.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:12 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:12.311+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:37:12.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:12 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:12.311+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:37:12.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:12 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:12.311+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.252367+0000 front 2023-07-14T18:36:31.252344+0000 (oldest deadline 2023-07-14T18:36:54.752574+0000) 2023-07-14T18:37:12.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:12 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:12.311+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:34.753382+0000 front 2023-07-14T18:36:34.753342+0000 (oldest deadline 2023-07-14T18:37:05.354033+0000) 2023-07-14T18:37:12.629 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:12 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:12.311+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.354776+0000 front 2023-07-14T18:36:45.355045+0000 (oldest deadline 2023-07-14T18:37:10.054540+0000) 2023-07-14T18:37:12.695 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:12 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:12.381+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:12.695 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:12 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:12.381+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:12.695 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:12 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:12.381+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:12.695 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:12 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:12.381+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:12.695 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:12 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:12.381+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.600897+0000 front 2023-07-14T18:36:45.600916+0000 (oldest deadline 2023-07-14T18:37:10.901000+0000) 2023-07-14T18:37:12.821 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:12 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:12.430+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:12.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:12 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:12.430+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:12.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:12 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:12.430+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:12.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:12 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:12.430+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:12.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:12 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:12.430+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.013712+0000 front 2023-07-14T18:36:39.013773+0000 (oldest deadline 2023-07-14T18:37:04.814193+0000) 2023-07-14T18:37:12.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:12 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:12.430+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:47.715375+0000 front 2023-07-14T18:36:47.715425+0000 (oldest deadline 2023-07-14T18:37:08.815165+0000) 2023-07-14T18:37:13.087 INFO:journalctl@ceph.osd.8.smithi171.stdout:Jul 14 18:37:12 smithi171 bash[162791]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-8 2023-07-14T18:37:13.627 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:13 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:13.298+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:37:13.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:13 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:13.298+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:37:13.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:13 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:13.298+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:37:13.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:13 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:13.298+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.252367+0000 front 2023-07-14T18:36:31.252344+0000 (oldest deadline 2023-07-14T18:36:54.752574+0000) 2023-07-14T18:37:13.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:13 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:13.298+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:34.753382+0000 front 2023-07-14T18:36:34.753342+0000 (oldest deadline 2023-07-14T18:37:05.354033+0000) 2023-07-14T18:37:13.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:13 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:13.298+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.354776+0000 front 2023-07-14T18:36:45.355045+0000 (oldest deadline 2023-07-14T18:37:10.054540+0000) 2023-07-14T18:37:13.812 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:13 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:13.341+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:13.812 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:13 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:13.341+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:13.812 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:13 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:13.341+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:13.812 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:13 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:13.341+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:13.812 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:13 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:13.341+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.600897+0000 front 2023-07-14T18:36:45.600916+0000 (oldest deadline 2023-07-14T18:37:10.901000+0000) 2023-07-14T18:37:13.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:13 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:13.402+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:13.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:13 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:13.402+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:13.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:13 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:13.402+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:13.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:13 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:13.402+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:13.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:13 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:13.402+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.013712+0000 front 2023-07-14T18:36:39.013773+0000 (oldest deadline 2023-07-14T18:37:04.814193+0000) 2023-07-14T18:37:13.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:13 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:13.402+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:47.715375+0000 front 2023-07-14T18:36:47.715425+0000 (oldest deadline 2023-07-14T18:37:08.815165+0000) 2023-07-14T18:37:14.138 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.8.service' 2023-07-14T18:37:14.218 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:37:14.219 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-07-14T18:37:14.219 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-07-14T18:37:14.219 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.9 2023-07-14T18:37:14.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:14 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:14.294+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:14.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:14 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:14.294+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:14.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:14 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:14.294+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:14.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:14 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:14.294+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:14.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:14 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:14.294+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.600897+0000 front 2023-07-14T18:36:45.600916+0000 (oldest deadline 2023-07-14T18:37:10.901000+0000) 2023-07-14T18:37:14.627 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:14 smithi133 systemd[1]: Stopping Ceph osd.9 for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:37:14.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:14 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:14.321+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:37:14.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:14 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:14.321+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:37:14.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:14 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:14.321+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:37:14.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:14 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:14.321+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.252367+0000 front 2023-07-14T18:36:31.252344+0000 (oldest deadline 2023-07-14T18:36:54.752574+0000) 2023-07-14T18:37:14.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:14 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:14.321+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:34.753382+0000 front 2023-07-14T18:36:34.753342+0000 (oldest deadline 2023-07-14T18:37:05.354033+0000) 2023-07-14T18:37:14.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:14 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:14.321+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.354776+0000 front 2023-07-14T18:36:45.355045+0000 (oldest deadline 2023-07-14T18:37:10.054540+0000) 2023-07-14T18:37:14.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:14 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:14.442+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:14.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:14 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:14.442+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:14.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:14 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:14.442+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:14.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:14 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:14.442+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:14.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:14 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:14.442+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.013712+0000 front 2023-07-14T18:36:39.013773+0000 (oldest deadline 2023-07-14T18:37:04.814193+0000) 2023-07-14T18:37:14.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:14 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:14.442+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:47.715375+0000 front 2023-07-14T18:36:47.715425+0000 (oldest deadline 2023-07-14T18:37:08.815165+0000) 2023-07-14T18:37:15.127 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:14 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:14.712+0000 7f7ef8fbc700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:37:15.128 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:14 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:14.712+0000 7f7ef8fbc700 -1 osd.9 96 *** Got signal Terminated *** 2023-07-14T18:37:15.128 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:14 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:14.712+0000 7f7ef8fbc700 -1 osd.9 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T18:37:15.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:15 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:15.293+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:15.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:15 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:15.293+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:15.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:15 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:15.293+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:15.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:15 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:15.293+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:15.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:15 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:15.293+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.600897+0000 front 2023-07-14T18:36:45.600916+0000 (oldest deadline 2023-07-14T18:37:10.901000+0000) 2023-07-14T18:37:15.627 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:15 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:15.335+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:37:15.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:15 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:15.335+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:37:15.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:15 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:15.335+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:37:15.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:15 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:15.335+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.252367+0000 front 2023-07-14T18:36:31.252344+0000 (oldest deadline 2023-07-14T18:36:54.752574+0000) 2023-07-14T18:37:15.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:15 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:15.335+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:34.753382+0000 front 2023-07-14T18:36:34.753342+0000 (oldest deadline 2023-07-14T18:37:05.354033+0000) 2023-07-14T18:37:15.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:15 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:15.335+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.354776+0000 front 2023-07-14T18:36:45.355045+0000 (oldest deadline 2023-07-14T18:37:10.054540+0000) 2023-07-14T18:37:15.821 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:15 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:15.417+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:15.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:15 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:15.417+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:15.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:15 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:15.417+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:15.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:15 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:15.417+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:15.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:15 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:15.417+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.013712+0000 front 2023-07-14T18:36:39.013773+0000 (oldest deadline 2023-07-14T18:37:04.814193+0000) 2023-07-14T18:37:15.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:15 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:15.417+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:47.715375+0000 front 2023-07-14T18:36:47.715425+0000 (oldest deadline 2023-07-14T18:37:08.815165+0000) 2023-07-14T18:37:16.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:16 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:16.251+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:16.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:16 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:16.251+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:16.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:16 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:16.251+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:16.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:16 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:16.251+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:16.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:16 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:16.251+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.600897+0000 front 2023-07-14T18:36:45.600916+0000 (oldest deadline 2023-07-14T18:37:10.901000+0000) 2023-07-14T18:37:16.627 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:16 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:16.322+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:37:16.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:16 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:16.322+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:37:16.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:16 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:16.322+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:37:16.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:16 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:16.322+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.252367+0000 front 2023-07-14T18:36:31.252344+0000 (oldest deadline 2023-07-14T18:36:54.752574+0000) 2023-07-14T18:37:16.629 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:16 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:16.322+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:34.753382+0000 front 2023-07-14T18:36:34.753342+0000 (oldest deadline 2023-07-14T18:37:05.354033+0000) 2023-07-14T18:37:16.629 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:16 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:16.322+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.354776+0000 front 2023-07-14T18:36:45.355045+0000 (oldest deadline 2023-07-14T18:37:10.054540+0000) 2023-07-14T18:37:16.821 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:16 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:16.409+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:16.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:16 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:16.409+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:16.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:16 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:16.409+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:16.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:16 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:16.409+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:16.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:16 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:16.409+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.013712+0000 front 2023-07-14T18:36:39.013773+0000 (oldest deadline 2023-07-14T18:37:04.814193+0000) 2023-07-14T18:37:16.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:16 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:16.409+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:47.715375+0000 front 2023-07-14T18:36:47.715425+0000 (oldest deadline 2023-07-14T18:37:08.815165+0000) 2023-07-14T18:37:17.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:17 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:17.207+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:17.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:17 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:17.207+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:17.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:17 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:17.207+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:17.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:17 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:17.207+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:17.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:17 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:17.207+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.600897+0000 front 2023-07-14T18:36:45.600916+0000 (oldest deadline 2023-07-14T18:37:10.901000+0000) 2023-07-14T18:37:17.627 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:17 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:17.336+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:37:17.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:17 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:17.336+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:37:17.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:17 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:17.336+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:37:17.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:17 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:17.336+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.252367+0000 front 2023-07-14T18:36:31.252344+0000 (oldest deadline 2023-07-14T18:36:54.752574+0000) 2023-07-14T18:37:17.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:17 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:17.336+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:34.753382+0000 front 2023-07-14T18:36:34.753342+0000 (oldest deadline 2023-07-14T18:37:05.354033+0000) 2023-07-14T18:37:17.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:17 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:17.336+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.354776+0000 front 2023-07-14T18:36:45.355045+0000 (oldest deadline 2023-07-14T18:37:10.054540+0000) 2023-07-14T18:37:17.821 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:17 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:17.381+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:17.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:17 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:17.381+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:17.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:17 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:17.381+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:17.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:17 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:17.381+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:17.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:17 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:17.381+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.013712+0000 front 2023-07-14T18:36:39.013773+0000 (oldest deadline 2023-07-14T18:37:04.814193+0000) 2023-07-14T18:37:17.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:17 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:17.381+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:47.715375+0000 front 2023-07-14T18:36:47.715425+0000 (oldest deadline 2023-07-14T18:37:08.815165+0000) 2023-07-14T18:37:18.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:18 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:18.225+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:18.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:18 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:18.225+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:18.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:18 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:18.225+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:18.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:18 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:18.225+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:18.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:18 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:18.225+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.600897+0000 front 2023-07-14T18:36:45.600916+0000 (oldest deadline 2023-07-14T18:37:10.901000+0000) 2023-07-14T18:37:18.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:18 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:18.225+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:55.002136+0000 front 2023-07-14T18:36:55.002074+0000 (oldest deadline 2023-07-14T18:37:17.901991+0000) 2023-07-14T18:37:18.627 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:18 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:18.333+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:37:18.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:18 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:18.333+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:37:18.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:18 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:18.333+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:37:18.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:18 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:18.333+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.252367+0000 front 2023-07-14T18:36:31.252344+0000 (oldest deadline 2023-07-14T18:36:54.752574+0000) 2023-07-14T18:37:18.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:18 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:18.333+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:34.753382+0000 front 2023-07-14T18:36:34.753342+0000 (oldest deadline 2023-07-14T18:37:05.354033+0000) 2023-07-14T18:37:18.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:18 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:18.333+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.354776+0000 front 2023-07-14T18:36:45.355045+0000 (oldest deadline 2023-07-14T18:37:10.054540+0000) 2023-07-14T18:37:18.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:18 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:18.332+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:18.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:18 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:18.332+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:18.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:18 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:18.332+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:18.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:18 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:18.332+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:18.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:18 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:18.332+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.013712+0000 front 2023-07-14T18:36:39.013773+0000 (oldest deadline 2023-07-14T18:37:04.814193+0000) 2023-07-14T18:37:18.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:18 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:18.332+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:47.715375+0000 front 2023-07-14T18:36:47.715425+0000 (oldest deadline 2023-07-14T18:37:08.815165+0000) 2023-07-14T18:37:18.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:18 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:18.332+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:52.817001+0000 front 2023-07-14T18:36:52.817016+0000 (oldest deadline 2023-07-14T18:37:17.417491+0000) 2023-07-14T18:37:19.572 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:19 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:19.290+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:19.572 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:19 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:19.290+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:19.572 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:19 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:19.290+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:19.573 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:19 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:19.290+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:19.573 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:19 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:19.290+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.013712+0000 front 2023-07-14T18:36:39.013773+0000 (oldest deadline 2023-07-14T18:37:04.814193+0000) 2023-07-14T18:37:19.573 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:19 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:19.290+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:47.715375+0000 front 2023-07-14T18:36:47.715425+0000 (oldest deadline 2023-07-14T18:37:08.815165+0000) 2023-07-14T18:37:19.573 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:19 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:19.290+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:52.817001+0000 front 2023-07-14T18:36:52.817016+0000 (oldest deadline 2023-07-14T18:37:17.417491+0000) 2023-07-14T18:37:19.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:19 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:19.258+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:19.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:19 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:19.258+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:19.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:19 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:19.258+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:19.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:19 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:19.258+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:19.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:19 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:19.258+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.600897+0000 front 2023-07-14T18:36:45.600916+0000 (oldest deadline 2023-07-14T18:37:10.901000+0000) 2023-07-14T18:37:19.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:19 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:19.258+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:55.002136+0000 front 2023-07-14T18:36:55.002074+0000 (oldest deadline 2023-07-14T18:37:17.901991+0000) 2023-07-14T18:37:19.877 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:19 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:19.365+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.147785+0000 front 2023-07-14T18:36:06.147742+0000 (oldest deadline 2023-07-14T18:36:30.247872+0000) 2023-07-14T18:37:19.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:19 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:19.365+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:12.549125+0000 front 2023-07-14T18:36:12.548910+0000 (oldest deadline 2023-07-14T18:36:36.649137+0000) 2023-07-14T18:37:19.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:19 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:19.365+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.750627+0000 front 2023-07-14T18:36:20.750424+0000 (oldest deadline 2023-07-14T18:36:46.050574+0000) 2023-07-14T18:37:19.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:19 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:19.365+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:31.252367+0000 front 2023-07-14T18:36:31.252344+0000 (oldest deadline 2023-07-14T18:36:54.752574+0000) 2023-07-14T18:37:19.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:19 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:19.365+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:34.753382+0000 front 2023-07-14T18:36:34.753342+0000 (oldest deadline 2023-07-14T18:37:05.354033+0000) 2023-07-14T18:37:19.878 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:19 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9[155773]: 2023-07-14T18:37:19.365+0000 7f7ef3db0700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.354776+0000 front 2023-07-14T18:36:45.355045+0000 (oldest deadline 2023-07-14T18:37:10.054540+0000) 2023-07-14T18:37:20.572 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:20 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:20.286+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:20.572 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:20 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:20.286+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:20.572 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:20 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:20.286+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:20.572 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:20 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:20.286+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:20.572 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:20 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:20.286+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.013712+0000 front 2023-07-14T18:36:39.013773+0000 (oldest deadline 2023-07-14T18:37:04.814193+0000) 2023-07-14T18:37:20.572 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:20 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:20.286+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:47.715375+0000 front 2023-07-14T18:36:47.715425+0000 (oldest deadline 2023-07-14T18:37:08.815165+0000) 2023-07-14T18:37:20.573 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:20 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:20.286+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:52.817001+0000 front 2023-07-14T18:36:52.817016+0000 (oldest deadline 2023-07-14T18:37:17.417491+0000) 2023-07-14T18:37:20.586 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:20 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:20.248+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:20.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:20 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:20.248+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:20.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:20 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:20.248+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:20.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:20 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:20.248+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:20.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:20 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:20.248+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.600897+0000 front 2023-07-14T18:36:45.600916+0000 (oldest deadline 2023-07-14T18:37:10.901000+0000) 2023-07-14T18:37:20.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:20 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:20.248+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:55.002136+0000 front 2023-07-14T18:36:55.002074+0000 (oldest deadline 2023-07-14T18:37:17.901991+0000) 2023-07-14T18:37:21.283 INFO:journalctl@ceph.osd.9.smithi133.stdout:Jul 14 18:37:21 smithi133 bash[187228]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-9 2023-07-14T18:37:21.586 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:21 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:21.297+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:21.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:21 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:21.297+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:21.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:21 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:21.297+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:21.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:21 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:21.297+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:21.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:21 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:21.297+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.600897+0000 front 2023-07-14T18:36:45.600916+0000 (oldest deadline 2023-07-14T18:37:10.901000+0000) 2023-07-14T18:37:21.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:21 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:21.297+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:55.002136+0000 front 2023-07-14T18:36:55.002074+0000 (oldest deadline 2023-07-14T18:37:17.901991+0000) 2023-07-14T18:37:21.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:21 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:21.335+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:21.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:21 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:21.335+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:21.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:21 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:21.335+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:21.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:21 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:21.335+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:21.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:21 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:21.335+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.013712+0000 front 2023-07-14T18:36:39.013773+0000 (oldest deadline 2023-07-14T18:37:04.814193+0000) 2023-07-14T18:37:21.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:21 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:21.335+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:47.715375+0000 front 2023-07-14T18:36:47.715425+0000 (oldest deadline 2023-07-14T18:37:08.815165+0000) 2023-07-14T18:37:21.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:21 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:21.335+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:52.817001+0000 front 2023-07-14T18:36:52.817016+0000 (oldest deadline 2023-07-14T18:37:17.417491+0000) 2023-07-14T18:37:22.586 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:22 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:22.307+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:22.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:22 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:22.307+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:22.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:22 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:22.307+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:22.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:22 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:22.307+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:22.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:22 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:22.307+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.600897+0000 front 2023-07-14T18:36:45.600916+0000 (oldest deadline 2023-07-14T18:37:10.901000+0000) 2023-07-14T18:37:22.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:22 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:22.307+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:55.002136+0000 front 2023-07-14T18:36:55.002074+0000 (oldest deadline 2023-07-14T18:37:17.901991+0000) 2023-07-14T18:37:22.821 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:22 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:22.358+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:22.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:22 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:22.358+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:22.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:22 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:22.358+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:22.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:22 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:22.358+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:22.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:22 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:22.358+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.013712+0000 front 2023-07-14T18:36:39.013773+0000 (oldest deadline 2023-07-14T18:37:04.814193+0000) 2023-07-14T18:37:22.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:22 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:22.358+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:47.715375+0000 front 2023-07-14T18:36:47.715425+0000 (oldest deadline 2023-07-14T18:37:08.815165+0000) 2023-07-14T18:37:22.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:22 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:22.358+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:52.817001+0000 front 2023-07-14T18:36:52.817016+0000 (oldest deadline 2023-07-14T18:37:17.417491+0000) 2023-07-14T18:37:23.012 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.9.service' 2023-07-14T18:37:23.101 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:37:23.102 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-07-14T18:37:23.102 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-07-14T18:37:23.102 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.10 2023-07-14T18:37:23.453 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:23 smithi161 systemd[1]: Stopping Ceph osd.10 for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:37:23.454 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:23 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:23.329+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:23.454 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:23 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:23.329+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:23.454 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:23 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:23.329+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:23.454 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:23 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:23.329+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:23.454 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:23 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:23.329+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.013712+0000 front 2023-07-14T18:36:39.013773+0000 (oldest deadline 2023-07-14T18:37:04.814193+0000) 2023-07-14T18:37:23.454 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:23 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:23.329+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:47.715375+0000 front 2023-07-14T18:36:47.715425+0000 (oldest deadline 2023-07-14T18:37:08.815165+0000) 2023-07-14T18:37:23.455 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:23 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:23.329+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:52.817001+0000 front 2023-07-14T18:36:52.817016+0000 (oldest deadline 2023-07-14T18:37:17.417491+0000) 2023-07-14T18:37:23.586 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:23 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:23.282+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:23.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:23 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:23.282+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:23.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:23 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:23.282+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:23.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:23 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:23.282+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:23.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:23 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:23.282+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.600897+0000 front 2023-07-14T18:36:45.600916+0000 (oldest deadline 2023-07-14T18:37:10.901000+0000) 2023-07-14T18:37:23.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:23 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:23.282+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:55.002136+0000 front 2023-07-14T18:36:55.002074+0000 (oldest deadline 2023-07-14T18:37:17.901991+0000) 2023-07-14T18:37:23.821 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:23 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:23.438+0000 7ff62a720700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:37:23.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:23 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:23.438+0000 7ff62a720700 -1 osd.10 96 *** Got signal Terminated *** 2023-07-14T18:37:23.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:23 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:23.438+0000 7ff62a720700 -1 osd.10 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T18:37:24.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:24 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:24.304+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:24.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:24 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:24.304+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:24.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:24 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:24.304+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:24.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:24 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:24.304+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:24.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:24 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:24.304+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.600897+0000 front 2023-07-14T18:36:45.600916+0000 (oldest deadline 2023-07-14T18:37:10.901000+0000) 2023-07-14T18:37:24.589 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:24 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:24.304+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:55.002136+0000 front 2023-07-14T18:36:55.002074+0000 (oldest deadline 2023-07-14T18:37:17.901991+0000) 2023-07-14T18:37:24.821 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:24 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:24.377+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:24.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:24 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:24.377+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:24.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:24 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:24.377+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:24.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:24 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:24.377+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:24.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:24 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:24.377+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.013712+0000 front 2023-07-14T18:36:39.013773+0000 (oldest deadline 2023-07-14T18:37:04.814193+0000) 2023-07-14T18:37:24.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:24 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:24.377+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:47.715375+0000 front 2023-07-14T18:36:47.715425+0000 (oldest deadline 2023-07-14T18:37:08.815165+0000) 2023-07-14T18:37:24.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:24 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:24.377+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:52.817001+0000 front 2023-07-14T18:36:52.817016+0000 (oldest deadline 2023-07-14T18:37:17.417491+0000) 2023-07-14T18:37:25.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:25 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:25.268+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:25.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:25 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:25.268+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:25.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:25 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:25.268+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:25.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:25 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:25.268+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:25.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:25 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:25.268+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.600897+0000 front 2023-07-14T18:36:45.600916+0000 (oldest deadline 2023-07-14T18:37:10.901000+0000) 2023-07-14T18:37:25.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:25 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:25.268+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:55.002136+0000 front 2023-07-14T18:36:55.002074+0000 (oldest deadline 2023-07-14T18:37:17.901991+0000) 2023-07-14T18:37:25.821 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:25 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:25.391+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:25.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:25 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:25.391+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:25.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:25 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:25.391+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:25.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:25 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:25.391+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:25.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:25 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:25.391+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.013712+0000 front 2023-07-14T18:36:39.013773+0000 (oldest deadline 2023-07-14T18:37:04.814193+0000) 2023-07-14T18:37:25.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:25 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:25.391+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:47.715375+0000 front 2023-07-14T18:36:47.715425+0000 (oldest deadline 2023-07-14T18:37:08.815165+0000) 2023-07-14T18:37:25.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:25 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:25.391+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:52.817001+0000 front 2023-07-14T18:36:52.817016+0000 (oldest deadline 2023-07-14T18:37:17.417491+0000) 2023-07-14T18:37:26.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:26 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:26.239+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:26.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:26 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:26.239+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:26.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:26 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:26.239+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:26.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:26 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:26.239+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:26.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:26 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:26.239+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.600897+0000 front 2023-07-14T18:36:45.600916+0000 (oldest deadline 2023-07-14T18:37:10.901000+0000) 2023-07-14T18:37:26.589 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:26 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:26.239+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:55.002136+0000 front 2023-07-14T18:36:55.002074+0000 (oldest deadline 2023-07-14T18:37:17.901991+0000) 2023-07-14T18:37:26.821 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:26 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:26.433+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:26.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:26 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:26.433+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:26.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:26 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:26.433+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:26.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:26 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:26.433+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:26.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:26 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:26.433+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.013712+0000 front 2023-07-14T18:36:39.013773+0000 (oldest deadline 2023-07-14T18:37:04.814193+0000) 2023-07-14T18:37:26.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:26 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:26.433+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:47.715375+0000 front 2023-07-14T18:36:47.715425+0000 (oldest deadline 2023-07-14T18:37:08.815165+0000) 2023-07-14T18:37:26.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:26 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:26.433+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:52.817001+0000 front 2023-07-14T18:36:52.817016+0000 (oldest deadline 2023-07-14T18:37:17.417491+0000) 2023-07-14T18:37:27.586 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:27 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:27.248+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:27.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:27 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:27.248+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:27.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:27 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:27.248+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:27.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:27 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:27.248+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:27.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:27 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:27.248+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.600897+0000 front 2023-07-14T18:36:45.600916+0000 (oldest deadline 2023-07-14T18:37:10.901000+0000) 2023-07-14T18:37:27.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:27 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:27.248+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:55.002136+0000 front 2023-07-14T18:36:55.002074+0000 (oldest deadline 2023-07-14T18:37:17.901991+0000) 2023-07-14T18:37:27.821 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:27 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:27.426+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:27.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:27 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:27.426+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:27.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:27 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:27.426+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:27.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:27 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:27.426+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:27.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:27 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:27.426+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.013712+0000 front 2023-07-14T18:36:39.013773+0000 (oldest deadline 2023-07-14T18:37:04.814193+0000) 2023-07-14T18:37:27.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:27 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:27.426+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:47.715375+0000 front 2023-07-14T18:36:47.715425+0000 (oldest deadline 2023-07-14T18:37:08.815165+0000) 2023-07-14T18:37:27.823 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:27 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:27.426+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:52.817001+0000 front 2023-07-14T18:36:52.817016+0000 (oldest deadline 2023-07-14T18:37:17.417491+0000) 2023-07-14T18:37:28.586 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:28 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:28.298+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:28.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:28 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:28.298+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:28.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:28 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:28.298+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:28.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:28 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:28.298+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:28.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:28 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:28.298+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.600897+0000 front 2023-07-14T18:36:45.600916+0000 (oldest deadline 2023-07-14T18:37:10.901000+0000) 2023-07-14T18:37:28.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:28 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:28.298+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:55.002136+0000 front 2023-07-14T18:36:55.002074+0000 (oldest deadline 2023-07-14T18:37:17.901991+0000) 2023-07-14T18:37:28.821 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:28 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:28.423+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:07.107141+0000 front 2023-07-14T18:36:07.107031+0000 (oldest deadline 2023-07-14T18:36:29.407068+0000) 2023-07-14T18:37:28.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:28 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:28.423+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:15.208767+0000 front 2023-07-14T18:36:15.208863+0000 (oldest deadline 2023-07-14T18:36:37.409607+0000) 2023-07-14T18:37:28.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:28 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:28.423+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:20.310589+0000 front 2023-07-14T18:36:20.310735+0000 (oldest deadline 2023-07-14T18:36:45.610748+0000) 2023-07-14T18:37:28.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:28 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:28.423+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-07-14T18:36:30.312093+0000 front 2023-07-14T18:36:30.311981+0000 (oldest deadline 2023-07-14T18:36:57.912570+0000) 2023-07-14T18:37:28.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:28 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:28.423+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:39.013712+0000 front 2023-07-14T18:36:39.013773+0000 (oldest deadline 2023-07-14T18:37:04.814193+0000) 2023-07-14T18:37:28.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:28 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:28.423+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:47.715375+0000 front 2023-07-14T18:36:47.715425+0000 (oldest deadline 2023-07-14T18:37:08.815165+0000) 2023-07-14T18:37:28.822 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:28 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10[150490]: 2023-07-14T18:37:28.423+0000 7ff624512700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:52.817001+0000 front 2023-07-14T18:36:52.817016+0000 (oldest deadline 2023-07-14T18:37:17.417491+0000) 2023-07-14T18:37:29.571 INFO:journalctl@ceph.osd.10.smithi161.stdout:Jul 14 18:37:29 smithi161 bash[161870]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-10 2023-07-14T18:37:29.586 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:29 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:29.280+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:29.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:29 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:29.280+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:29.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:29 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:29.280+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:29.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:29 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:29.280+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:29.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:29 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:29.280+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.600897+0000 front 2023-07-14T18:36:45.600916+0000 (oldest deadline 2023-07-14T18:37:10.901000+0000) 2023-07-14T18:37:29.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:29 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:29.280+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:55.002136+0000 front 2023-07-14T18:36:55.002074+0000 (oldest deadline 2023-07-14T18:37:17.901991+0000) 2023-07-14T18:37:29.589 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:29 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:29.280+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2023-07-14T18:36:59.003117+0000 front 2023-07-14T18:36:59.003184+0000 (oldest deadline 2023-07-14T18:37:29.003723+0000) 2023-07-14T18:37:30.586 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:30 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:30.236+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:30.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:30 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:30.236+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:30.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:30 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:30.236+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:30.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:30 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:30.236+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:30.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:30 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:30.236+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.600897+0000 front 2023-07-14T18:36:45.600916+0000 (oldest deadline 2023-07-14T18:37:10.901000+0000) 2023-07-14T18:37:30.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:30 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:30.236+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:55.002136+0000 front 2023-07-14T18:36:55.002074+0000 (oldest deadline 2023-07-14T18:37:17.901991+0000) 2023-07-14T18:37:30.588 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:30 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:30.236+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2023-07-14T18:36:59.003117+0000 front 2023-07-14T18:36:59.003184+0000 (oldest deadline 2023-07-14T18:37:29.003723+0000) 2023-07-14T18:37:30.636 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.10.service' 2023-07-14T18:37:30.675 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:37:30.675 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-07-14T18:37:30.676 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-07-14T18:37:30.676 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.11 2023-07-14T18:37:31.035 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:30 smithi171 systemd[1]: Stopping Ceph osd.11 for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:37:31.036 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:31 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:31.019+0000 7fc92ba37700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:37:31.285 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:31 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:31.019+0000 7fc92ba37700 -1 osd.11 96 *** Got signal Terminated *** 2023-07-14T18:37:31.285 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:31 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:31.019+0000 7fc92ba37700 -1 osd.11 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T18:37:31.285 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:31 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:31.269+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:31.285 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:31 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:31.269+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:31.286 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:31 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:31.269+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:31.286 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:31 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:31.269+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:31.586 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:31 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:31.269+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.600897+0000 front 2023-07-14T18:36:45.600916+0000 (oldest deadline 2023-07-14T18:37:10.901000+0000) 2023-07-14T18:37:31.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:31 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:31.269+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:55.002136+0000 front 2023-07-14T18:36:55.002074+0000 (oldest deadline 2023-07-14T18:37:17.901991+0000) 2023-07-14T18:37:31.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:31 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:31.269+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2023-07-14T18:36:59.003117+0000 front 2023-07-14T18:36:59.003184+0000 (oldest deadline 2023-07-14T18:37:29.003723+0000) 2023-07-14T18:37:32.586 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:32 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:32.229+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:32.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:32 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:32.229+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:32.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:32 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:32.229+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:32.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:32 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:32.229+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:32.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:32 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:32.229+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.600897+0000 front 2023-07-14T18:36:45.600916+0000 (oldest deadline 2023-07-14T18:37:10.901000+0000) 2023-07-14T18:37:32.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:32 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:32.229+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:55.002136+0000 front 2023-07-14T18:36:55.002074+0000 (oldest deadline 2023-07-14T18:37:17.901991+0000) 2023-07-14T18:37:32.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:32 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:32.229+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2023-07-14T18:36:59.003117+0000 front 2023-07-14T18:36:59.003184+0000 (oldest deadline 2023-07-14T18:37:29.003723+0000) 2023-07-14T18:37:33.586 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:33 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:33.218+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:33.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:33 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:33.218+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:33.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:33 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:33.218+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:33.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:33 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:33.218+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:33.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:33 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:33.218+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.600897+0000 front 2023-07-14T18:36:45.600916+0000 (oldest deadline 2023-07-14T18:37:10.901000+0000) 2023-07-14T18:37:33.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:33 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:33.218+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:55.002136+0000 front 2023-07-14T18:36:55.002074+0000 (oldest deadline 2023-07-14T18:37:17.901991+0000) 2023-07-14T18:37:33.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:33 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:33.218+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2023-07-14T18:36:59.003117+0000 front 2023-07-14T18:36:59.003184+0000 (oldest deadline 2023-07-14T18:37:29.003723+0000) 2023-07-14T18:37:34.586 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:34 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:34.189+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:34.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:34 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:34.189+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:34.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:34 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:34.189+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:34.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:34 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:34.189+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:34.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:34 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:34.189+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.600897+0000 front 2023-07-14T18:36:45.600916+0000 (oldest deadline 2023-07-14T18:37:10.901000+0000) 2023-07-14T18:37:34.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:34 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:34.189+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:55.002136+0000 front 2023-07-14T18:36:55.002074+0000 (oldest deadline 2023-07-14T18:37:17.901991+0000) 2023-07-14T18:37:34.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:34 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:34.189+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2023-07-14T18:36:59.003117+0000 front 2023-07-14T18:36:59.003184+0000 (oldest deadline 2023-07-14T18:37:29.003723+0000) 2023-07-14T18:37:35.586 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:35 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:35.219+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-07-14T18:36:06.792302+0000 front 2023-07-14T18:36:06.792258+0000 (oldest deadline 2023-07-14T18:36:29.092380+0000) 2023-07-14T18:37:35.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:35 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:35.219+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2023-07-14T18:36:14.894492+0000 front 2023-07-14T18:36:14.894309+0000 (oldest deadline 2023-07-14T18:36:37.794408+0000) 2023-07-14T18:37:35.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:35 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:35.219+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2023-07-14T18:36:23.496504+0000 front 2023-07-14T18:36:23.496411+0000 (oldest deadline 2023-07-14T18:36:46.396685+0000) 2023-07-14T18:37:35.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:35 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:35.219+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2023-07-14T18:36:36.899161+0000 front 2023-07-14T18:36:36.899201+0000 (oldest deadline 2023-07-14T18:37:02.199111+0000) 2023-07-14T18:37:35.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:35 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:35.219+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2023-07-14T18:36:45.600897+0000 front 2023-07-14T18:36:45.600916+0000 (oldest deadline 2023-07-14T18:37:10.901000+0000) 2023-07-14T18:37:35.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:35 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:35.219+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-07-14T18:36:55.002136+0000 front 2023-07-14T18:36:55.002074+0000 (oldest deadline 2023-07-14T18:37:17.901991+0000) 2023-07-14T18:37:35.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:35 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:35.219+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2023-07-14T18:36:59.003117+0000 front 2023-07-14T18:36:59.003184+0000 (oldest deadline 2023-07-14T18:37:29.003723+0000) 2023-07-14T18:37:35.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:35 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11[151674]: 2023-07-14T18:37:35.219+0000 7fc92602a700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.171:6820 osd.8 since back 2023-07-14T18:37:09.004156+0000 front 2023-07-14T18:37:09.004186+0000 (oldest deadline 2023-07-14T18:37:34.804542+0000) 2023-07-14T18:37:36.912 INFO:journalctl@ceph.osd.11.smithi171.stdout:Jul 14 18:37:36 smithi171 bash[163060]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-osd-11 2023-07-14T18:37:38.275 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@osd.11.service' 2023-07-14T18:37:38.314 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:37:38.315 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-07-14T18:37:38.315 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-07-14T18:37:38.316 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.a 2023-07-14T18:37:38.865 INFO:journalctl@ceph.mds.a.smithi133.stdout:Jul 14 18:37:38 smithi133 systemd[1]: Stopping Ceph mds.a for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:37:39.132 INFO:journalctl@ceph.mds.a.smithi133.stdout:Jul 14 18:37:38 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-a[161131]: 2023-07-14T18:37:38.849+0000 7fcf5eb62700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:37:39.133 INFO:journalctl@ceph.mds.a.smithi133.stdout:Jul 14 18:37:38 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-a[161131]: 2023-07-14T18:37:38.849+0000 7fcf5eb62700 -1 mds.a *** got signal Terminated *** 2023-07-14T18:37:41.877 INFO:journalctl@ceph.mds.a.smithi133.stdout:Jul 14 18:37:41 smithi133 bash[187508]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-a 2023-07-14T18:37:42.094 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.a.service' 2023-07-14T18:37:42.181 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:37:42.181 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-07-14T18:37:42.181 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-07-14T18:37:42.182 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.d 2023-07-14T18:37:42.376 INFO:journalctl@ceph.mds.d.smithi133.stdout:Jul 14 18:37:42 smithi133 systemd[1]: Stopping Ceph mds.d for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:37:43.127 INFO:journalctl@ceph.mds.d.smithi133.stdout:Jul 14 18:37:42 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-d[160134]: 2023-07-14T18:37:42.689+0000 7f685b3a7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:37:43.127 INFO:journalctl@ceph.mds.d.smithi133.stdout:Jul 14 18:37:42 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-d[160134]: 2023-07-14T18:37:42.689+0000 7f685b3a7700 -1 mds.d *** got signal Terminated *** 2023-07-14T18:37:44.627 INFO:journalctl@ceph.mds.d.smithi133.stdout:Jul 14 18:37:44 smithi133 bash[187654]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-d 2023-07-14T18:37:44.822 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.d.service' 2023-07-14T18:37:44.905 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:37:44.906 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-07-14T18:37:44.906 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-07-14T18:37:44.907 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.g 2023-07-14T18:37:45.126 INFO:journalctl@ceph.mds.g.smithi133.stdout:Jul 14 18:37:44 smithi133 systemd[1]: Stopping Ceph mds.g for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:37:45.876 INFO:journalctl@ceph.mds.g.smithi133.stdout:Jul 14 18:37:45 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-g[160746]: 2023-07-14T18:37:45.390+0000 7fc8ecaf5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:37:45.877 INFO:journalctl@ceph.mds.g.smithi133.stdout:Jul 14 18:37:45 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-g[160746]: 2023-07-14T18:37:45.390+0000 7fc8ecaf5700 -1 mds.g *** got signal Terminated *** 2023-07-14T18:37:50.108 INFO:journalctl@ceph.mds.g.smithi133.stdout:Jul 14 18:37:49 smithi133 bash[187801]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-g 2023-07-14T18:37:50.128 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.g.service' 2023-07-14T18:37:50.211 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:37:50.211 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-07-14T18:37:50.212 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-07-14T18:37:50.212 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.j 2023-07-14T18:37:50.376 INFO:journalctl@ceph.mds.j.smithi133.stdout:Jul 14 18:37:50 smithi133 systemd[1]: Stopping Ceph mds.j for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:37:51.127 INFO:journalctl@ceph.mds.j.smithi133.stdout:Jul 14 18:37:50 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-j[159749]: 2023-07-14T18:37:50.696+0000 7f4671f1a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:37:51.128 INFO:journalctl@ceph.mds.j.smithi133.stdout:Jul 14 18:37:50 smithi133 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-j[159749]: 2023-07-14T18:37:50.696+0000 7f4671f1a700 -1 mds.j *** got signal Terminated *** 2023-07-14T18:37:53.626 INFO:journalctl@ceph.mds.j.smithi133.stdout:Jul 14 18:37:53 smithi133 bash[187946]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-j 2023-07-14T18:37:53.978 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.j.service' 2023-07-14T18:37:54.063 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:37:54.064 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-07-14T18:37:54.066 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-07-14T18:37:54.067 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.b 2023-07-14T18:37:54.418 INFO:journalctl@ceph.mds.b.smithi161.stdout:Jul 14 18:37:54 smithi161 systemd[1]: Stopping Ceph mds.b for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:37:54.821 INFO:journalctl@ceph.mds.b.smithi161.stdout:Jul 14 18:37:54 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-b[154371]: 2023-07-14T18:37:54.399+0000 7fdfa0824700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:37:54.821 INFO:journalctl@ceph.mds.b.smithi161.stdout:Jul 14 18:37:54 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-b[154371]: 2023-07-14T18:37:54.399+0000 7fdfa0824700 -1 mds.b *** got signal Terminated *** 2023-07-14T18:37:58.195 INFO:journalctl@ceph.mds.b.smithi161.stdout:Jul 14 18:37:58 smithi161 bash[162149]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-b 2023-07-14T18:37:58.385 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.b.service' 2023-07-14T18:37:58.423 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:37:58.423 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-07-14T18:37:58.424 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-07-14T18:37:58.424 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.e 2023-07-14T18:37:58.826 INFO:journalctl@ceph.mds.e.smithi161.stdout:Jul 14 18:37:58 smithi161 systemd[1]: Stopping Ceph mds.e for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:37:59.320 INFO:journalctl@ceph.mds.e.smithi161.stdout:Jul 14 18:37:58 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-e[153691]: 2023-07-14T18:37:58.815+0000 7f8292a4d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:37:59.321 INFO:journalctl@ceph.mds.e.smithi161.stdout:Jul 14 18:37:58 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-e[153691]: 2023-07-14T18:37:58.815+0000 7f8292a4d700 -1 mds.e *** got signal Terminated *** 2023-07-14T18:38:02.172 INFO:journalctl@ceph.mds.e.smithi161.stdout:Jul 14 18:38:02 smithi161 bash[162288]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-e 2023-07-14T18:38:02.360 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.e.service' 2023-07-14T18:38:02.395 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:38:02.396 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-07-14T18:38:02.396 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-07-14T18:38:02.396 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.h 2023-07-14T18:38:02.770 INFO:journalctl@ceph.mds.h.smithi161.stdout:Jul 14 18:38:02 smithi161 systemd[1]: Stopping Ceph mds.h for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:38:03.070 INFO:journalctl@ceph.mds.h.smithi161.stdout:Jul 14 18:38:02 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-h[154030]: 2023-07-14T18:38:02.754+0000 7f5a5dbc1700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:38:03.071 INFO:journalctl@ceph.mds.h.smithi161.stdout:Jul 14 18:38:02 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-h[154030]: 2023-07-14T18:38:02.754+0000 7f5a5dbc1700 -1 mds.h *** got signal Terminated *** 2023-07-14T18:38:06.320 INFO:journalctl@ceph.mds.h.smithi161.stdout:Jul 14 18:38:06 smithi161 bash[162428]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-h 2023-07-14T18:38:06.394 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.h.service' 2023-07-14T18:38:06.428 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:38:06.428 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-07-14T18:38:06.429 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-07-14T18:38:06.429 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.k 2023-07-14T18:38:06.820 INFO:journalctl@ceph.mds.k.smithi161.stdout:Jul 14 18:38:06 smithi161 systemd[1]: Stopping Ceph mds.k for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:38:07.070 INFO:journalctl@ceph.mds.k.smithi161.stdout:Jul 14 18:38:06 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-k[153343]: 2023-07-14T18:38:06.804+0000 7f84c0b17700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:38:07.071 INFO:journalctl@ceph.mds.k.smithi161.stdout:Jul 14 18:38:06 smithi161 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-k[153343]: 2023-07-14T18:38:06.804+0000 7f84c0b17700 -1 mds.k *** got signal Terminated *** 2023-07-14T18:38:10.231 INFO:journalctl@ceph.mds.k.smithi161.stdout:Jul 14 18:38:09 smithi161 bash[162570]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-k 2023-07-14T18:38:10.237 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.k.service' 2023-07-14T18:38:10.270 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:38:10.271 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-07-14T18:38:10.271 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-07-14T18:38:10.272 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.c 2023-07-14T18:38:10.586 INFO:journalctl@ceph.mds.c.smithi171.stdout:Jul 14 18:38:10 smithi171 systemd[1]: Stopping Ceph mds.c for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:38:11.085 INFO:journalctl@ceph.mds.c.smithi171.stdout:Jul 14 18:38:10 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-c[155279]: 2023-07-14T18:38:10.629+0000 7fce7ba4e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:38:11.086 INFO:journalctl@ceph.mds.c.smithi171.stdout:Jul 14 18:38:10 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-c[155279]: 2023-07-14T18:38:10.629+0000 7fce7ba4e700 -1 mds.c *** got signal Terminated *** 2023-07-14T18:38:14.336 INFO:journalctl@ceph.mds.c.smithi171.stdout:Jul 14 18:38:14 smithi171 bash[163370]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-c 2023-07-14T18:38:14.425 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.c.service' 2023-07-14T18:38:14.463 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:38:14.464 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-07-14T18:38:14.464 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-07-14T18:38:14.465 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.f 2023-07-14T18:38:14.831 INFO:journalctl@ceph.mds.f.smithi171.stdout:Jul 14 18:38:14 smithi171 systemd[1]: Stopping Ceph mds.f for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:38:15.085 INFO:journalctl@ceph.mds.f.smithi171.stdout:Jul 14 18:38:14 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-f[154932]: 2023-07-14T18:38:14.816+0000 7f5f8cb45700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:38:15.086 INFO:journalctl@ceph.mds.f.smithi171.stdout:Jul 14 18:38:14 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-f[154932]: 2023-07-14T18:38:14.816+0000 7f5f8cb45700 -1 mds.f *** got signal Terminated *** 2023-07-14T18:38:18.594 INFO:journalctl@ceph.mds.f.smithi171.stdout:Jul 14 18:38:18 smithi171 bash[163517]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-f 2023-07-14T18:38:18.699 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.f.service' 2023-07-14T18:38:18.736 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:38:18.737 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-07-14T18:38:18.737 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-07-14T18:38:18.738 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.i 2023-07-14T18:38:19.085 INFO:journalctl@ceph.mds.i.smithi171.stdout:Jul 14 18:38:18 smithi171 systemd[1]: Stopping Ceph mds.i for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:38:19.585 INFO:journalctl@ceph.mds.i.smithi171.stdout:Jul 14 18:38:19 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-i[155617]: 2023-07-14T18:38:19.111+0000 7f3d421a0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:38:19.586 INFO:journalctl@ceph.mds.i.smithi171.stdout:Jul 14 18:38:19 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-i[155617]: 2023-07-14T18:38:19.111+0000 7f3d421a0700 -1 mds.i *** got signal Terminated *** 2023-07-14T18:38:22.184 INFO:journalctl@ceph.mds.i.smithi171.stdout:Jul 14 18:38:22 smithi171 bash[163661]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-i 2023-07-14T18:38:22.370 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.i.service' 2023-07-14T18:38:22.405 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:38:22.406 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-07-14T18:38:22.406 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-07-14T18:38:22.406 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.l 2023-07-14T18:38:22.782 INFO:journalctl@ceph.mds.l.smithi171.stdout:Jul 14 18:38:22 smithi171 systemd[1]: Stopping Ceph mds.l for 06d0f5fa-2271-11ee-9b33-001a4aab830c... 2023-07-14T18:38:23.085 INFO:journalctl@ceph.mds.l.smithi171.stdout:Jul 14 18:38:22 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-l[154591]: 2023-07-14T18:38:22.767+0000 7f68e19c0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T18:38:23.086 INFO:journalctl@ceph.mds.l.smithi171.stdout:Jul 14 18:38:22 smithi171 ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-l[154591]: 2023-07-14T18:38:22.767+0000 7f68e19c0700 -1 mds.l *** got signal Terminated *** 2023-07-14T18:38:27.393 INFO:journalctl@ceph.mds.l.smithi171.stdout:Jul 14 18:38:27 smithi171 bash[163800]: ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c-mds-l 2023-07-14T18:38:27.490 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-06d0f5fa-2271-11ee-9b33-001a4aab830c@mds.l.service' 2023-07-14T18:38:27.524 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T18:38:27.525 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-07-14T18:38:27.525 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c --force --keep-logs 2023-07-14T18:38:27.912 INFO:teuthology.orchestra.run.smithi133.stdout:Deleting cluster with fsid: 06d0f5fa-2271-11ee-9b33-001a4aab830c 2023-07-14T18:38:31.301 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c --force --keep-logs 2023-07-14T18:38:31.681 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: 06d0f5fa-2271-11ee-9b33-001a4aab830c 2023-07-14T18:38:33.299 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c --force --keep-logs 2023-07-14T18:38:33.640 INFO:teuthology.orchestra.run.smithi171.stdout:Deleting cluster with fsid: 06d0f5fa-2271-11ee-9b33-001a4aab830c 2023-07-14T18:38:35.240 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T18:38:35.319 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T18:38:35.355 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T18:38:35.385 INFO:tasks.cephadm:Archiving crash dumps... 2023-07-14T18:38:35.387 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/crash to /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7337992/remote/smithi133/crash 2023-07-14T18:38:35.388 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/crash -- . 2023-07-14T18:38:35.515 INFO:teuthology.orchestra.run.smithi133.stderr:tar: /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-14T18:38:35.515 INFO:teuthology.orchestra.run.smithi133.stderr:tar: Error is not recoverable: exiting now 2023-07-14T18:38:35.519 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/crash to /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7337992/remote/smithi161/crash 2023-07-14T18:38:35.524 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/crash -- . 2023-07-14T18:38:35.590 INFO:teuthology.orchestra.run.smithi161.stderr:tar: /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-14T18:38:35.591 INFO:teuthology.orchestra.run.smithi161.stderr:tar: Error is not recoverable: exiting now 2023-07-14T18:38:35.592 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/crash to /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7337992/remote/smithi171/crash 2023-07-14T18:38:35.593 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/crash -- . 2023-07-14T18:38:35.637 INFO:teuthology.orchestra.run.smithi171.stderr:tar: /var/lib/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-14T18:38:35.638 INFO:teuthology.orchestra.run.smithi171.stderr:tar: Error is not recoverable: exiting now 2023-07-14T18:38:35.638 INFO:tasks.cephadm:Checking cluster log for badness... 2023-07-14T18:38:35.639 DEBUG:teuthology.orchestra.run.smithi133:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-07-14T18:38:35.722 INFO:teuthology.orchestra.run.smithi133.stderr:grep: /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph.log: No such file or directory 2023-07-14T18:38:35.727 INFO:tasks.cephadm:Compressing logs... 2023-07-14T18:38:35.727 DEBUG:teuthology.orchestra.run.smithi133:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-14T18:38:35.732 DEBUG:teuthology.orchestra.run.smithi161:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-14T18:38:35.735 DEBUG:teuthology.orchestra.run.smithi171:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-14T18:38:35.773 INFO:teuthology.orchestra.run.smithi161.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-07-14T18:38:35.773 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.10.log 2023-07-14T18:38:35.784 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.h.log 2023-07-14T18:38:35.785 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.e.log 2023-07-14T18:38:35.785 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.4.log 2023-07-14T18:38:35.785 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.1.log 2023-07-14T18:38:35.786 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.k.log 2023-07-14T18:38:35.786 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-volume.log 2023-07-14T18:38:35.786 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.7.log 2023-07-14T18:38:35.787 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mon.b.log 2023-07-14T18:38:35.787 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.b.log 2023-07-14T18:38:35.787 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mgr.y.log 2023-07-14T18:38:35.787 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-14T18:38:35.788 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mgr.y.log: /var/log/ceph/cephadm.log: 94.3% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.k.log.gz 2023-07-14T18:38:35.789 INFO:teuthology.orchestra.run.smithi161.stderr: 93.2% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mgr.y.log.gz 2023-07-14T18:38:35.794 INFO:teuthology.orchestra.run.smithi171.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.5.log 2023-07-14T18:38:35.794 INFO:teuthology.orchestra.run.smithi171.stderr:: No such file or directory 2023-07-14T18:38:35.796 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.2.log 2023-07-14T18:38:35.796 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.11.log 2023-07-14T18:38:35.797 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.5.log: /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mgr.z.log 2023-07-14T18:38:35.797 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.8.log 2023-07-14T18:38:35.797 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.f.log 2023-07-14T18:38:35.797 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.c.log 2023-07-14T18:38:35.797 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.i.log 2023-07-14T18:38:35.798 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mon.c.log 2023-07-14T18:38:35.798 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.c.log: /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-volume.log 2023-07-14T18:38:35.798 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.l.log 2023-07-14T18:38:35.799 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-14T18:38:35.800 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-volume.log: /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.l.log: /var/log/ceph/cephadm.log: 93.3% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mgr.z.log.gz 2023-07-14T18:38:35.801 INFO:teuthology.orchestra.run.smithi171.stderr: 94.3% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.l.log.gz 2023-07-14T18:38:35.816 INFO:teuthology.orchestra.run.smithi133.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-07-14T18:38:35.817 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.0.log 2023-07-14T18:38:35.818 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.g.log 2023-07-14T18:38:35.818 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.a.log 2023-07-14T18:38:35.819 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.6.log 2023-07-14T18:38:35.820 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.3.log 2023-07-14T18:38:35.821 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.d.log 2023-07-14T18:38:35.822 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mgr.x.log 2023-07-14T18:38:35.823 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.9.log 2023-07-14T18:38:35.823 INFO:teuthology.orchestra.run.smithi133.stderr: 94.3% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.g.log.gz 2023-07-14T18:38:35.824 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mgr.x.log: 94.2% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.a.log.gz 2023-07-14T18:38:35.825 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-volume.log 2023-07-14T18:38:35.826 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.j.log 2023-07-14T18:38:35.827 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mon.a.log 2023-07-14T18:38:35.828 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.j.log: 94.3% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.d.log.gz 2023-07-14T18:38:35.828 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-14T18:38:35.829 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2023-07-14T18:38:35.832 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: 94.1% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.j.log.gz 2023-07-14T18:38:35.854 INFO:teuthology.orchestra.run.smithi133.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2023-07-14T18:38:35.974 INFO:teuthology.orchestra.run.smithi161.stderr: 91.5% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-volume.log.gz 2023-07-14T18:38:36.049 INFO:teuthology.orchestra.run.smithi161.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-14T18:38:36.153 INFO:teuthology.orchestra.run.smithi133.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-14T18:38:36.168 INFO:teuthology.orchestra.run.smithi133.stderr: 91.5% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-volume.log.gz 2023-07-14T18:38:36.206 INFO:teuthology.orchestra.run.smithi171.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-14T18:38:36.542 INFO:teuthology.orchestra.run.smithi133.stderr: 90.1% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mgr.x.log.gz 2023-07-14T18:38:36.679 INFO:teuthology.orchestra.run.smithi171.stderr: 91.5% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-volume.log.gz 2023-07-14T18:38:36.891 INFO:teuthology.orchestra.run.smithi161.stderr: 93.7% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.e.log.gz 2023-07-14T18:38:37.405 INFO:teuthology.orchestra.run.smithi161.stderr: 95.4% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.h.log.gz 2023-07-14T18:38:37.455 INFO:teuthology.orchestra.run.smithi171.stderr: 91.9% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mon.c.log.gz 2023-07-14T18:38:37.934 INFO:teuthology.orchestra.run.smithi161.stderr: 91.8% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mon.b.log.gz 2023-07-14T18:38:39.216 INFO:teuthology.orchestra.run.smithi171.stderr: 94.2% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.c.log.gz 2023-07-14T18:38:39.562 INFO:teuthology.orchestra.run.smithi133.stderr: 90.4% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mon.a.log.gz 2023-07-14T18:39:05.258 INFO:teuthology.orchestra.run.smithi171.stderr: 94.3% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.i.log.gz 2023-07-14T18:42:02.954 INFO:teuthology.orchestra.run.smithi161.stderr: 91.0% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.10.log.gz 2023-07-14T18:42:09.487 INFO:teuthology.orchestra.run.smithi171.stderr: 97.7% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.f.log.gz 2023-07-14T18:43:04.580 INFO:teuthology.orchestra.run.smithi161.stderr: 90.9% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.1.log.gz 2023-07-14T18:43:09.779 INFO:teuthology.orchestra.run.smithi171.stderr: 91.0% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.8.log.gz 2023-07-14T18:43:22.806 INFO:teuthology.orchestra.run.smithi133.stderr: 91.0% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.0.log.gz 2023-07-14T18:43:28.720 INFO:teuthology.orchestra.run.smithi171.stderr: 91.1% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.11.log.gz 2023-07-14T18:43:36.703 INFO:teuthology.orchestra.run.smithi171.stderr: 91.1% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.2.log.gz 2023-07-14T18:43:46.932 INFO:teuthology.orchestra.run.smithi133.stderr: 91.1% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.6.log.gz 2023-07-14T18:43:50.741 INFO:teuthology.orchestra.run.smithi133.stderr: 90.9% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.3.log.gz 2023-07-14T18:44:22.710 INFO:teuthology.orchestra.run.smithi161.stderr: 98.1% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-mds.b.log.gz 2023-07-14T18:44:28.282 INFO:teuthology.orchestra.run.smithi161.stderr: 90.9% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.7.log.gz 2023-07-14T18:44:42.513 INFO:teuthology.orchestra.run.smithi133.stderr: 91.0% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.9.log.gz 2023-07-14T18:44:42.516 INFO:teuthology.orchestra.run.smithi133.stderr: 2023-07-14T18:44:42.517 INFO:teuthology.orchestra.run.smithi133.stderr:real 6m6.754s 2023-07-14T18:44:42.517 INFO:teuthology.orchestra.run.smithi133.stderr:user 8m27.560s 2023-07-14T18:44:42.517 INFO:teuthology.orchestra.run.smithi133.stderr:sys 1m6.178s 2023-07-14T18:45:05.777 INFO:teuthology.orchestra.run.smithi171.stderr: 90.8% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.5.log.gz 2023-07-14T18:45:05.810 INFO:teuthology.orchestra.run.smithi171.stderr: 2023-07-14T18:45:05.811 INFO:teuthology.orchestra.run.smithi171.stderr:real 6m30.064s 2023-07-14T18:45:05.811 INFO:teuthology.orchestra.run.smithi171.stderr:user 8m7.112s 2023-07-14T18:45:05.811 INFO:teuthology.orchestra.run.smithi171.stderr:sys 0m19.496s 2023-07-14T18:45:19.631 INFO:teuthology.orchestra.run.smithi161.stderr: 91.3% -- replaced with /var/log/ceph/06d0f5fa-2271-11ee-9b33-001a4aab830c/ceph-osd.4.log.gz 2023-07-14T18:45:19.643 INFO:teuthology.orchestra.run.smithi161.stderr: 2023-07-14T18:45:19.643 INFO:teuthology.orchestra.run.smithi161.stderr:real 6m43.897s 2023-07-14T18:45:19.643 INFO:teuthology.orchestra.run.smithi161.stderr:user 8m32.339s 2023-07-14T18:45:19.643 INFO:teuthology.orchestra.run.smithi161.stderr:sys 0m21.469s 2023-07-14T18:45:19.644 INFO:tasks.cephadm:Archiving logs... 2023-07-14T18:45:19.644 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7337992/remote/smithi133/log 2023-07-14T18:45:19.645 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-14T18:46:33.565 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7337992/remote/smithi161/log 2023-07-14T18:46:33.567 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-14T18:47:41.272 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7337992/remote/smithi171/log 2023-07-14T18:47:41.273 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-14T18:48:49.842 INFO:tasks.cephadm:Removing cluster... 2023-07-14T18:48:49.843 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c --force 2023-07-14T18:48:50.660 INFO:teuthology.orchestra.run.smithi133.stdout:Deleting cluster with fsid: 06d0f5fa-2271-11ee-9b33-001a4aab830c 2023-07-14T18:48:53.026 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c --force 2023-07-14T18:48:53.599 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: 06d0f5fa-2271-11ee-9b33-001a4aab830c 2023-07-14T18:48:54.629 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 06d0f5fa-2271-11ee-9b33-001a4aab830c --force 2023-07-14T18:48:55.317 INFO:teuthology.orchestra.run.smithi171.stdout:Deleting cluster with fsid: 06d0f5fa-2271-11ee-9b33-001a4aab830c 2023-07-14T18:48:56.407 INFO:tasks.cephadm:Removing cephadm ... 2023-07-14T18:48:56.408 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-14T18:48:56.457 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-14T18:48:56.493 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-14T18:48:56.519 INFO:tasks.cephadm:Teardown complete 2023-07-14T18:48:56.519 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-07-14T18:48:56.530 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-07-14T18:48:56.531 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-14T18:48:56.535 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-14T18:48:56.538 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-14T18:48:58.818 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-07-14T18:48:58.875 INFO:teuthology.task.clock:Checking final clock skew... 2023-07-14T18:48:58.876 DEBUG:teuthology.orchestra.run.smithi133:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T18:48:58.897 DEBUG:teuthology.orchestra.run.smithi161:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T18:48:58.901 DEBUG:teuthology.orchestra.run.smithi171:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T18:48:58.918 INFO:teuthology.orchestra.run.smithi161.stderr:bash: ntpq: command not found 2023-07-14T18:48:58.922 INFO:teuthology.orchestra.run.smithi171.stderr:bash: ntpq: command not found 2023-07-14T18:48:58.937 INFO:teuthology.orchestra.run.smithi133.stderr:bash: ntpq: command not found 2023-07-14T18:48:58.941 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T18:48:58.942 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2023-07-14T18:48:58.942 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 63 +905us[ +928us] +/- 90ms 2023-07-14T18:48:58.943 INFO:teuthology.orchestra.run.smithi161.stdout:^* hv02.front.sepia.ceph.com 3 6 377 62 -4966us[-4944us] +/- 81ms 2023-07-14T18:48:58.943 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 63 +4271us[+4293us] +/- 80ms 2023-07-14T18:48:58.943 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T18:48:58.957 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T18:48:58.957 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2023-07-14T18:48:58.957 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 0 +945us[ +945us] +/- 91ms 2023-07-14T18:48:58.958 INFO:teuthology.orchestra.run.smithi171.stdout:^* hv02.front.sepia.ceph.com 3 6 377 3 -5052us[-5062us] +/- 82ms 2023-07-14T18:48:58.958 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 +4283us[+4283us] +/- 81ms 2023-07-14T18:48:58.958 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T18:48:59.056 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T18:48:59.057 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2023-07-14T18:48:59.057 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 2 +988us[ +988us] +/- 91ms 2023-07-14T18:48:59.057 INFO:teuthology.orchestra.run.smithi133.stdout:^* hv02.front.sepia.ceph.com 3 6 377 5 -4988us[-5013us] +/- 82ms 2023-07-14T18:48:59.057 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 1 +4341us[+4341us] +/- 81ms 2023-07-14T18:48:59.057 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T18:48:59.058 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-07-14T18:48:59.069 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-07-14T18:48:59.071 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-07-14T18:48:59.094 DEBUG:teuthology.orchestra.run.smithi133:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-14T18:48:59.353 DEBUG:teuthology.orchestra.run.smithi161:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-14T18:48:59.567 DEBUG:teuthology.orchestra.run.smithi171:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-14T18:48:59.770 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T18:48:59.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T18:48:59.849 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T18:48:59.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T18:48:59.887 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T18:48:59.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T18:48:59.925 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-07-14T18:48:59.938 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-07-14T18:48:59.955 INFO:teuthology.task.internal:Duration was 3689.821942 seconds 2023-07-14T18:48:59.955 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-07-14T18:48:59.995 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-07-14T18:48:59.996 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-14T18:49:00.002 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-14T18:49:00.006 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-14T18:49:00.085 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T18:49:00.102 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T18:49:00.190 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T18:49:00.417 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-07-14T18:49:00.417 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi133.front.sepia.ceph.com 2023-07-14T18:49:00.417 DEBUG:teuthology.orchestra.run.smithi133:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-14T18:49:00.494 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi161.front.sepia.ceph.com 2023-07-14T18:49:00.494 DEBUG:teuthology.orchestra.run.smithi161:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-14T18:49:00.525 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi171.front.sepia.ceph.com 2023-07-14T18:49:00.525 DEBUG:teuthology.orchestra.run.smithi171:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-14T18:49:00.554 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-07-14T18:49:00.554 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-14T18:49:00.559 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-14T18:49:00.566 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-14T18:49:00.706 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-07-14T18:49:00.706 DEBUG:teuthology.orchestra.run.smithi133:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-14T18:49:00.711 DEBUG:teuthology.orchestra.run.smithi161:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-14T18:49:00.713 DEBUG:teuthology.orchestra.run.smithi171:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-14T18:49:00.878 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-07-14T18:49:00.889 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-07-14T18:49:00.889 DEBUG:teuthology.orchestra.run.smithi133:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-14T18:49:00.973 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-14T18:49:01.022 DEBUG:teuthology.orchestra.run.smithi171:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-14T18:49:01.052 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-07-14T18:49:01.063 DEBUG:teuthology.orchestra.run.smithi133:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-14T18:49:01.068 DEBUG:teuthology.orchestra.run.smithi161:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-14T18:49:01.070 DEBUG:teuthology.orchestra.run.smithi171:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-14T18:49:01.115 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = core 2023-07-14T18:49:01.128 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = core 2023-07-14T18:49:01.132 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = core 2023-07-14T18:49:01.167 DEBUG:teuthology.orchestra.run.smithi133:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-14T18:49:01.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T18:49:01.209 DEBUG:teuthology.orchestra.run.smithi161:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-14T18:49:01.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T18:49:01.226 DEBUG:teuthology.orchestra.run.smithi171:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-14T18:49:01.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T18:49:01.243 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-07-14T18:49:01.254 INFO:teuthology.task.internal:Transferring archived files... 2023-07-14T18:49:01.255 DEBUG:teuthology.misc:Transferring archived files from smithi133:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7337992/remote/smithi133 2023-07-14T18:49:01.256 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-14T18:49:01.335 DEBUG:teuthology.misc:Transferring archived files from smithi161:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7337992/remote/smithi161 2023-07-14T18:49:01.335 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-14T18:49:01.372 DEBUG:teuthology.misc:Transferring archived files from smithi171:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7337992/remote/smithi171 2023-07-14T18:49:01.373 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-14T18:49:01.409 INFO:teuthology.task.internal:Removing archive directory... 2023-07-14T18:49:01.409 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-14T18:49:01.414 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-14T18:49:01.416 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-14T18:49:01.461 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-07-14T18:49:01.474 INFO:teuthology.task.internal:Not uploading archives. 2023-07-14T18:49:01.474 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-07-14T18:49:01.485 INFO:teuthology.task.internal:Tidying up after the test... 2023-07-14T18:49:01.486 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-14T18:49:01.497 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-14T18:49:01.499 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-14T18:49:01.513 INFO:teuthology.orchestra.run.smithi161.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 14 18:49 /home/ubuntu/cephtest 2023-07-14T18:49:01.518 INFO:teuthology.orchestra.run.smithi171.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 14 18:49 /home/ubuntu/cephtest 2023-07-14T18:49:01.535 INFO:teuthology.orchestra.run.smithi133.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 14 18:49 /home/ubuntu/cephtest 2023-07-14T18:49:01.538 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-07-14T18:49:01.549 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-07-14T18:49:01.572 INFO:teuthology.nuke:Checking targets against current locks 2023-07-14T18:49:05.774 DEBUG:teuthology.nuke:shortname: smithi133 2023-07-14T18:49:05.774 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T18:49:05.798 DEBUG:teuthology.nuke:shortname: smithi161 2023-07-14T18:49:05.799 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T18:49:05.802 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7337992', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 17:37:21.090790', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-14T18:49:05.822 DEBUG:teuthology.nuke:shortname: smithi171 2023-07-14T18:49:05.822 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T18:49:05.825 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7337992', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 17:37:21.099246', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-14T18:49:05.841 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi171.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7337992', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 17:37:21.101371', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-14T18:49:05.857 INFO:teuthology.orchestra.console:Power off smithi133 2023-07-14T18:49:05.858 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T18:49:05.882 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:49:05.901 INFO:teuthology.orchestra.console:Power off smithi171 2023-07-14T18:49:05.901 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T18:49:05.917 INFO:teuthology.orchestra.console:Power off smithi161 2023-07-14T18:49:05.918 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T18:49:05.928 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:49:05.941 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:49:09.942 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:49:09.955 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:49:09.964 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:49:14.056 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:49:14.066 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:49:14.076 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:49:19.170 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:49:19.178 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:49:19.186 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:49:24.281 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:49:24.290 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:49:24.298 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:49:29.394 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:49:29.405 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:49:29.413 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:49:34.509 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:49:34.519 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:49:34.528 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:49:39.622 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:49:39.631 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:49:39.639 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:49:39.834 INFO:teuthology.orchestra.console:Power off for smithi133 completed 2023-07-14T18:49:39.850 INFO:teuthology.orchestra.console:Power off for smithi161 completed 2023-07-14T18:49:44.745 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:49:49.857 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:49:50.067 INFO:teuthology.orchestra.console:Power off for smithi171 completed 2023-07-14T18:49:50.173 INFO:teuthology.run:Summary data: client.0-kernel-sha1: a440c79eee0258ae07e4988d8690fe6187b8b7a3 description: fs:workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/postgres}} duration: 3689.8219423294067 failure_reason: 'error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds' flavor: default owner: scheduled_pdonnell@teuthology status: fail success: false 2023-07-14T18:49:50.173 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T18:49:55.263 INFO:teuthology.run:FAIL